簡易檢索 / 詳目顯示
| 研究生： |
賴皓千 Lai, Hao-Chien |
|---|---|
| 論文名稱： |
基於擴散式資料增強與SimSiam架構之時間序列自監督表示學習研究 Diffusion-Augmented Contrastive Representation Learning for Time-Series Forecasting |
| 指導教授： |
蔡炎龍
Tsai, Yen-Lung |
| 口試委員: |
陳天進
Chen, Ten-Ging 張宜武 Chang, Yi-Wu |
| 學位類別： |
碩士
Master |
| 系所名稱： |
理學院 - 應用數學系 Department of Mathematical Sciences |
| 論文出版年： | 2025 |
| 畢業學年度： | 113 |
| 語文別： | 中文 |
| 論文頁數： | 48 |
| 中文關鍵詞： | 擴散模型 、對比學習 、時間序列 、資料增強 、股票市場 、正樣本生成 、模式一致性 、結構保留 、無監督學習 、回報預測 、異常檢測 |
| 外文關鍵詞： | Diffusion Models, Contrastive Learning, Time-Series Data, Data Augmentation, Stock Market, Positive Sample Generation, Pattern Consistency, Structural Preservation, Unsupervised Learning, Return Prediction, Anomaly Detection |
| 相關次數： | 點閱：193 下載：20 |
| 分享至: |
| 查詢本校圖書館目錄 查詢臺灣博碩士論文知識加值系統 勘誤回報 |
在對比學習（Contrastive Learning）中，資料增強是生成正樣本的關鍵手段，對模型效果有著重要影響。在圖像數據中，常見的增強方法如裁剪、翻轉等可以生成有效的正樣本，但在時間序列數據中，這些方法可能破壞數據的時序結構及內部關係，導致模型學習效果下降。儘管擴散模型（Diffusion Models）已成為時間序列數據分析與預測的有效工具，但其在對比學習資料增強中的應用尚未被廣泛討論，部分原因在於傳統的擴散模型生成過程多依賴隨機採樣，難以生成與特定數據對應的正樣本。
為解決這一挑戰，本研究設計了一種針對時間序列數據的擴散模型應用手法，摒棄傳統隨機採樣策略，通過重新編輯數據生成具有模式一致性和結構保留的正樣本，並將其應用於對比學習框架。實驗結果表明，該方法在台灣股票市場數據上的應用顯著提升了模型的特徵表徵能力，在回報預測和異常檢測等下游任務中展現出優越性能，尤其是在資料稀缺或不平衡的情境下效果尤為顯著。本研究不僅填補了擴散模型在對比學習中的研究空白，還為時間序列數據的資料增強提供了一種新穎的解決方案。
Data augmentation is a critical component in contrastive learning (CL) for generating positive samples, significantly impacting the model’s performance. While common augmentation methods such as cropping and flipping are effective for image data, these approaches often disrupt the temporal structure and relationships in time-series data, leading to suboptimal learning outcomes. Although diffusion models have become powerful tools for analyzing and forecasting time-series data, their application in data augmentation for contrastive learning remains underexplored. One reason is that conventional diffusion model approaches rely on random sampling, which generates points from the data distribution rather than specific positive samples corresponding to existing data.
To address this limitation, this study proposes a novel approach to applying diffusion models for time-series data. By discarding the traditional random sampling strategy, we utilize a tailored editing process to generate positive samples that preserve pattern consistency and structural integrity. These samples are then integrated into a contrastive learning framework. Experimental results demonstrate that the proposed method significantly enhances feature representation on Taiwan stock market data, achieving superior performance in downstream tasks such as return prediction and anomaly detection, particularly in data-scarce or imbalanced scenarios. This study not only bridges the gap in utilizing diffusion models for contrastive learning but also provides an innovative solution for time-series data augmentation.
致謝 ii
中文摘要 iii
Abstract iv
Contents vi
List of Tables viii
List of Figures ix
1 Introduction 1
1.1 Research Background 1
1.2 Research Problem 2
1.3 Research Status 3
1.4 Existing Problems and Limitations 4
1.5 Research Objectives and Scope 6
1.5.1 Research Objectives 6
1.5.2 Scope of the Study 7
2 Methodology 9
2.1 Overview of the Proposed Method 9
2.2 Diffusion-Based Augmentation for Time Series 10
2.2.1 Fundamentals and Mathematical Formulation of Diffusion Models 11
2.2.2 Application to Data Augmentation 12
2.2.3 Architecture of the Denoising Network 13
2.3 SimSiam Architecture 15
2.3.1 Data and Augmentation 15
2.3.2 Siamese Network Structure 16
2.3.3 Loss Function 17
2.3.4 Training Flow 18
2.3.5 Advantages of Diffusion-Based Augmentation 18
2.4 Advantages of the Proposed Method 19
3 Experimental Design and Training Process 20
3.1 Dataset Description 20
3.2 Data Processing and Preprocessing 21
3.3 Diffusion Model Training 22
3.4 SimSiam Training Procedure 23
3.5 Downstream Task and Evaluation Metrics 25
4 Experimental Results and Analysis 27
4.1 Impact of Data Augmentation Methods and Encoder Architectures on Performance 27
4.2 Effect of Diffusion Steps on Model Performance 28
4.3 Comparison and Summary of Model Variations 29
4.4 Evaluation Metrics and Their Relevance in Financial Classification 32
5 Conclusion 35
Appendix A Implementation Details 38
A.1 Diffusion-Based Augmentation Function 38
A.2 Transformer Encoder for Time-Series 40
A.3 SimSiam Model and Loss for Time-Series 41
A.4 Training Loop for SimSiam with Diffusion-Based Augmentation 42
Bibliography 45
[1] Chen, T., Kornblith, S., Norouzi, M., & Hinton, G. (2020). A simple framework for contrastive learning of visual representations. In *International conference on machine learning* (pp. 1597-1607). PMLR.
[2] Chen, X., & He, K. (2021). Exploring simple siamese representation learning. In *Proceedings of the IEEE/CVF conference on computer vision and pattern recognition* (pp. 15750-15758).
[3] Demirel, B. U., & Holz, C. (2023). Finding order in chaos: A novel data augmentation method for time series in contrastive learning. *Advances in Neural Information Processing Systems*, 36, 30750-30783.
[4] Dhariwal, P., & Nichol, A. (2021). Diffusion models beat gans on image synthesis. *Advances in neural information processing systems*, 34, 8780-8794.
[5] Esteban, C., Hyland, S. L., & Rätsch, G. (2017). Real-valued (medical) time series generation with recurrent conditional gans.
[6] Guo, Z., Wang, H., Yang, J., & Miller, D. J. (2015). A stock market forecasting model combining two-directional two-dimensional principal component analysis and radial basis function neural network. *PloS one*, 10(4), e0122385.
[7] He, K., Fan, H., Wu, Y., Xie, S., & Girshick, R. (2020). Momentum contrast for unsupervised visual representation learning. In *Proceedings of the IEEE/CVF conference on computer vision and pattern recognition* (pp. 9729-9738).
[8] Ho, J., Jain, A., & Abbeel, P. (2020). Denoising diffusion probabilistic models. *Advances in neural information processing systems*, 33, 6840-6851.
[9] Iwana, B. K., & Uchida, S. (2021). An empirical survey of data augmentation for time series classification with neural networks. *PLOS ONE*, 16(7).
[10] Jing, B., Wang, Y., Sui, G., Hong, J., He, J., Yang, Y., Li, D., & Ren, K. (2024, October). Automated contrastive learning strategy search for time series. In *Proceedings of the 33rd ACM International Conference on Information and Knowledge Management, CIKM '24* (pp. 4612-4620). ACM.
[11] Kalbande, D., Prabhu, P., Gharat, A., & Rajabally, T. (2021). A fraud detection system using machine learning. In *2021 12th International Conference on Computing Communication and Networking Technologies (ICCCNT)* (pp. 1-7). IEEE.
[12] Kong, Z., Ping, W., Huang, J., Zhao, K., & Catanzaro, B. (2021). Diffwave: A versatile diffusion model for audio synthesis.
[13] Lee, S., Lee, G., Kim, H., Kim, J., & Uh, Y. (2023). Sequential data generation with groupwise diffusion process.
[14] Lin, L., Li, Z., Li, R., Li, X., & Gao, J. (2024). Diffusion models for time-series applications: a survey. *Frontiers of Information Technology & Electronic Engineering*, 25(1), 19-41.
[15] Luo, D., Cheng, W., Wang, Y., Xu, D., Ni, J., Yu, W., Zhang, X., Liu, Y., Chen, Y., Chen, H., et al. (2023). Time series contrastive learning with information-aware augmentations. In *Proceedings of the AAAI Conference on Artificial Intelligence* (Vol. 37, pp. 4534-4542).
[16] Ma, C., & Yan, S. (2022). Deep learning in the chinese stock market: the role of technical indicators. *Finance Research Letters*, 49, 103025.
[17] Meng, C., He, Y., Song, Y., Song, J., Wu, J., Zhu, J. Y., & Ermon, S. (2022). Sdedit: Guided image synthesis and editing with stochastic differential equations.
[18] Rasul, K., Seward, C., Schuster, I., & Vollgraf, R. (2021). Autoregressive denoising diffusion models for multivariate probabilistic time series forecasting. In *International conference on machine learning* (pp. 8857-8868). PMLR.
[19] Ronneberger, O., Fischer, P., & Brox, T. (2015). U-net: Convolutional networks for biomedical image segmentation. In *Medical image computing and computer-assisted intervention–MICCAI 2015: 18th international conference, Munich, Germany, October 5-9, 2015, proceedings, part III 18* (pp. 234-241). Springer.
[20] Shobayo, O., Adeyemi-Longe, S., Popoola, O., & Ogunleye, B. (2024, October). Innovative sentiment analysis and prediction of stock price using finbert, gpt-4 and logistic regression: A data-driven approach. *Big Data and Cognitive Computing*, 8(11), 143.
[21] Solis-Martin, D., Galan-Paez, J., & Borrego-Diaz, J. (2023). D3a-ts: Denoising-driven data augmentation in time series.
[22] Song, J., Meng, C., & Ermon, S. (2022). Denoising diffusion implicit models.
[23] Wang, T., & Isola, P. (2020). Understanding contrastive representation learning through alignment and uniformity on the hypersphere. In *International conference on machine learning* (pp. 9929-9939). PMLR.
[24] Wang, W., Song, H., Si, S., Lu, W., & Cai, Z. (2024). Data augmentation based on diffusion probabilistic model for remaining useful life estimation of aero-engines. *Reliability Engineering & System Safety*, 252, 110394.
[25] Wen, Q., Sun, L., Yang, F., Song, X., Gao, J., Wang, X., & Xu, H. (2021, August). Time series data augmentation for deep learning: A survey. In *Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence, IJCAI-2021* (pp. 4653-4660). International Joint Conferences on Artificial Intelligence Organization.
[26] Yoon, J., Jarrett, D., & van der Schaar, M. (2019). Time-series generative adversarial networks. In H. Wallach, H. Larochelle, A. Beygelzimer, F. d'Alché-Buc, E. Fox, & R. Garnett (Eds.), *Advances in Neural Information Processing Systems* (Vol. 32). Curran Associates, Inc.
[27] Zerveas, G., Jayaraman, S., Patel, D., Bhamidipaty, A., & Eickhoff, C. (2021). A transformer-based framework for multivariate time series representation learning. In *Proceedings of the 27th ACM SIGKDD conference on knowledge discovery & data mining* (pp. 2114-2124).
