Department of Computer Engineering

Batch learning approaches cannot cope with the concept drift inherent in the stock market stream data. On the other hand, incremental learning has not been fully explored as a solution to the problem of concept drift in the stock market stream data. We propose ADIN-Forecast that detects changes in stock features and quickly adapts the model structure to mitigate the drawbacks of data shifts through incremental learning. ADIN-Forecast uses a Gated Recurrent Unit (GRU), enabling the self-growth of layers and hidden units that can dynamically adjust to the changes in the data distribution. Self-growing model architectures are known to experience catastrophic forgetting. To counter this, we have developed a control mechanism capable of activating or deactivating layers and applying a penalty coefficient to the layers’ weights depending on the occurrence of concept drifts. Forecasting model GRU in ADIN-Forecast can also be substituted with other neural networks, such as MLP, RNN, and LSTM. ADIN-Forecast uses the difference between the PCA eigenvectors for the two consecutive data windows to detect changes and offers a model that evolves dynamically according to these changes while ensuring memory and time efficiency through its incremental nature. We evaluated our methodology on the CSI 300 dataset in the open-source quantitative investment platform Qlib and compared it with other studies in the field.