Abstract
Abstract
Batch learning approaches cannot cope with the concept drift inherent in the stock market stream data. On the other hand, incremental learning has not been fully explored as a solution to the problem of concept drift in the stock market stream data. We propose ADIN-Forecast that detects changes in stock features and quickly adapts the model structure to mitigate the drawbacks of data shifts through incremental learning. ADIN-Forecast uses a Gated Recurrent Unit (GRU), enabling the self-growth of layers and hidden units that can dynamically adjust to the changes in the data distribution. Self-growing model architectures are known to experience catastrophic forgetting. To counter this, we have developed a control mechanism capable of activating or deactivating layers and applying a penalty coefficient to the layers’ weights depending on the occurrence of concept drifts. Forecasting model GRU in ADIN-Forecast can also be substituted with other neural networks, such as MLP, RNN, and LSTM. ADIN-Forecast uses the difference between the PCA eigenvectors for the two consecutive data windows to detect changes and offers a model that evolves dynamically according to these changes while ensuring memory and time efficiency through its incremental nature. We evaluated our methodology on the CSI 300 dataset in the open-source quantitative investment platform Qlib and compared it with other studies in the field.