本研究引入深度 Q 學習方法，建構一個自動化股票交易系統，研究範圍包含台灣股票市場 14 家科技業公司。研究期間為 2016 年 1 月 4 日至 2020年 12 月 31 日。本研究數據資料有兩種型態 (1) 股票資訊，(2) 股票資訊加上匯率參數。我們將深度 Q 學習的模型，與不同模型和其他策略相比較，以檢測深度 Q 學習是否更適用於股票交易。實證結果發現支持向量機與神經網路在實務面上難以進行股票交易操作，而深度 Q 學習的模型則具有相對好的成效。尤其，加入匯率參數的深度 Q 學習，獲得的報酬皆優於買入持有策略和台灣加權股價指數。

This research introduces the Deep Q learning model to construct an automated stock trading system. Our samples are 14 Taiwanese technology companies. Specifically, we include two types of data, (1) stock information and (2) stock information and exchange rate parameters, which are collected from the Taiwan stock market. The sampling period is from Jan 4, 2016 to Dec 31, 2020. We compare our main model, Deep Q learning, with different models and strategies to examine whether Deep Q learning is more applicable to stock trading. The empirical results show that it is difficult for Support vector machines and Neural networks to operate stock trading; however, Deep Q learning demonstrates better performance. In particular, the return rate of the Deep Q learning model is higher than the Buy-and-hold strategy and Taiwan Weighted Stock Index if considering exchange rate parameters.

王泓仁. 台幣匯率對我國經濟金融活動之影響. 中央銀行季刊》,(),–, Wang, Hung-Jen (),“e Impacts of NT Dollar Exchange Rates on Taiwan＇s Economy＂, Quarterly Reviews, Central Bank of the Republic of China (Taiwan),(), 2005.

徐千婷. 匯率與總體經濟變數之關係: 台灣實證分析, 2006.

Wei Bao, Jun Yue, and Yulei Rao. A deep learning framework for financial time series using stacked autoencoders and long-short term memory. PloS one, 12(7):e0180944, 2017.

Alessio Brini and Daniele Tantari. Deep reinforcement trading with predictable returns. arXiv preprint arXiv:2104.14683, 2021.

Arthur Charpentier, Romuald Elie, and Carl Remlinger. Reinforcement learning in economics and finance. Computational Economics, pages 1–38, 2021.

Corinna Cortes and Vladimir Vapnik. Support-vector networks. Machine learning, 20(3): 273–297, 1995.

Li Deng and Dong Yu. Deep learning: methods and applications. Foundations and trends in signal processing, 7(3–4):197–387, 2014.

Eugene F Fama. The behavior of stock-market prices. The journal of Business, 38(1): 34–105, 1965.

Eugene F Fama. Random walks in stock market prices. Financial analysts journal, 51(1): 75–80, 1995.

Ian Goodfellow, Yoshua Bengio, Aaron Courville, and Yoshua Bengio. Deep learning, volume 1. MIT press Cambridge, 2016.

Chien Yi Huang. Financial trading as a game: A deep reinforcement learning approach. arXiv preprint arXiv:1807.02787, 2018.

Leslie Pack Kaelbling, Michael L Littman, and Andrew W Moore. Reinforcement learning: A survey. Journal of artificial intelligence research, 4:237–285, 1996.

Konstantina Kourou, Themis P Exarchos, Konstantinos P Exarchos, Michalis V Karamouzis, and Dimitrios I Fotiadis. Machine learning applications in cancer prognosis and prediction. Computational and structural biotechnology journal, 13:8–17, 2015.

Yann LeCun, Yoshua Bengio, and Geoffrey Hinton. Deep learning. nature, 521(7553): 436–444, 2015.

Jae Won Lee. Stock price prediction using reinforcement learning. In ISIE 2001. 2001 IEEE International Symposium on Industrial Electronics Proceedings (Cat. No. 01TH8570), volume 1, pages 690–695. IEEE, 2001.

Jae Won Lee, Jonghun Park, O Jangmin, Jongwoo Lee, and Euyseok Hong. A multiagent approach to q-learning for daily stock trading. IEEE Transactions on Systems, Man, and Cybernetics-Part A: Systems and Humans, 37(6):864–877, 2007.

Pierre Menard, Omar Darwiche Domingues, Xuedong Shang, and Michal Valko. Ucb momentum q-learning: Correcting the bias without forgetting. arXiv preprint arXiv: 2103.01312, 2021.

Volodymyr Mnih, Koray Kavukcuoglu, David Silver, Alex Graves, Ioannis Antonoglou, Daan Wierstra, and Martin Riedmiller. Playing atari with deep reinforcement learning. arXiv preprint arXiv:1312.5602, 2013.

Arun Nair, Praveen Srinivasan, Sam Blackwell, Cagdas Alcicek, Rory Fearon, Alessandro De Maria, Vedavyas Panneershelvam, Mustafa Suleyman, Charles Beattie, Stig Petersen, et al. Massively parallel methods for deep reinforcement learning. arXiv preprint arXiv: 1507.04296, 2015.

Jiquan Ngiam, Aditya Khosla, Mingyu Kim, Juhan Nam, Honglak Lee, and Andrew Y Ng. Multimodal deep learning. In ICML, 2011.

Kate Phylaktis and Fabiola Ravazzolo. Stock prices and exchange rate dynamics. Journal of international Money and Finance, 24(7):1031–1053, 2005.

G William Schwert. Business cycles, financial crises, and stock volatility. In CarnegieRochester Conference series on public policy, volume 31, pages 83–125. E

Richard S Sutton and Andrew G Barto. Reinforcement learning: An introduction. MIT press, 2018.

Zhuoran Xiong, Xiao-Yang Liu, Shan Zhong, Hongyang Yang, and Anwar Walid. Practical deep reinforcement learning approach for stock trading. arXiv preprint arXiv:1811.07522, 2018.

Paul D Yoo, Maria H Kim, and Tony Jan. Machine learning techniques and use of event information for stock market prediction: A survey and evaluation. In International Conference on Computational Intelligence for Modelling, Control and Automation and International Conference on Intelligent Agents, Web Technologies and Internet Commerce (CIMCA-IAWTIC’06), volume 2, pages 835–841. IEEE, 2005.