資產配置一直都是投資市場中重要的課題，要如何做好投資組合管理，各家基金公司都有各自的方法，而隨著金融科技(Financial Technology, Fintech)的迅速地發展下，越來越多資產管理公司運用機器學習或深度學習進行資產配置，像是國外的資產管理公司就有推出AI智能選股的基金，而台灣也在這個浪潮下，快速地往這方面發展。隨著金融商品發展，境內組合型基金已經從兩年多前的兩百多檔，成長到現在的三百多檔，因此，組合型基金成為投資人新的投資工具，投資人如果是購買組合型基金，就相當於把資金分散投資在多檔基金，自動就達成資產配置的效果。
因此，本研究欲導入強化學習來建構組合型基金的資產配置，透過強化學習來決定投資組合中每一檔基金的權重，進而達到最佳策略，而強化學習的獎勵機制正好與資產的報酬不謀而合，所以非常合適。本研究運用強化學習建構了三個子模型，並與指標進行比較，研究結果發現，年化夏普值平均提升了十五個百分點，不僅顯示了資產配置對投資組合的重要性，也顯示強化學習運用在組合型基金資產配置的可行性。

Asset allocation has always been an important issue in the investment market. Each investment company has its own key to do a good job in portfolio management . with the rapid development of financial technology (Financial Technology, Fintech), more and more asset management companies use machine learning or deep learning for asset allocation. For example, foreign asset management companies launch funds based on AI selection and Taiwan is also rapidly developing in this direction . With the development of financial commodities, domestic Fund of funds have grown from two hundred to three hundred. Therefore, Fund of funds have become a new investment tool for investors.
Therefore, this study wants to introduce reinforcement learning to construct the asset allocation of Fund of funds. By using reinforcement learning, we can determine the weight of each fund in the portfolio to achieve the optimal allocation and the reward in reinforcement learning coincides with the return of assets. This study uses reinforcement learning to construct three sub-models and compare them with benchmark. The results show that the annualized Sharpe ratio has increased by an average of fifteen percentage, which not only shows the importance of asset allocation to the portfolio, but also shows that feasibility in Fund of funds allocation by using reinforcement learning.

第一章 緒論 1
第一節 研究動機 1
第二節 研究目的 2
第三節 研究架構 3
第二章 文獻探討 4
第一節 資產配置 4
第二節 共同基金及組合型基金 7
第三節 強化學習應用於資產配置相關文獻 11
第三章 研究方法 13
第一節 強化學習 13
第二節 強化學習演算法 20
第三節 績效指標 24
第四章 實證分析 26
第一節 建構投組 26
第二節 資料處理 29
第三節 強化學習設計 30
第四節 模型設計 32
第五節 結果分析 35
第五章 結論與展望 44
第一節 結論 44
第二節 未來展望 45
參考文獻 46
附件 49

中文部分
[1] 呂美瑩 (2003)，台灣發展組合型基金之可行性研究，台灣大學財務金融學研究所碩士論文。

[2] 李佩靜 (2005)， 應用 DEA 投資組合效率指數於台灣組合型基金之研究 (Doctoral dissertation, 長庚大學).

[3] 邱麗珍（2010），國內開放型共同基金規模與績效之關聯性探討，國立中山大學國際經營管理碩士班。

[4] 沈蔓君 (2012)， 台灣組合型基金績效研究 (Doctoral dissertation, 輔仁大學).

[5] 陳瑞璽, 洪碧霞, & 劉喻欣 (2014)， 組合型基金是否優於一般共同基金? 風險與報酬及基金特性之探討. 管理與系統, 21(2), 363-392.

[6] 劉上瑋 (2017)，深度增強學習在動態資產上之應用-以美國ETF為例，國立政治大學金融學系研究所碩士學位論文。

[7] 葉致緯 (2015)， 國內跨國股票組合型基金效率之研究—以運作時間達六年為例. 中正大學經濟系國際經濟研究所學位論文, 1-69.

英文部分
[1] Bellman, R.E. (1957). Dynamic Programming. Princeton University Press, Princeton, NJ. Republished 2003.

[2] Black, F. and Litterman, R. (1992). “Global Portfolio Optimization.” Financial Analysts Journal, September/October, 28-43.

[3] Brinson, G. P., Singer, B. D., & Beebower, G. L. (1991). Determinants of portfolio performance II: An update. Financial Analysts Journal, 47(3), 40-48.

[4] Filos, A. (2019). Reinforcement Learning for Portfolio Management. arXiv preprint arXiv:1909.09571.

[5] Fothergill, M., & Coke, C. (2001). Funds of hedge funds: an introduction to multi-manager funds. The Journal of Alternative Investments, 4(2), 7-16.

[6] Golec, J. H. (1996). The effects of mutual fund managers' characteristics on their portfolio performance, risk and fees. Financial Services Review, 5(2), 133-147.

[7] He, G. and Litterman, R. (1999). “The Intuition Behind Black-Litterman Model Portfolios.” Investment Management Research, Goldman, Sachs & Company, December

[8] Hensel, C. R., Ezra, D. D., & Ilkiw, J. H. (1991). The importance of the asset allocation decision. Financial Analysts Journal, 47(4), 65-72.

[9] Jiang, Z., Xu, D., & Liang, J. (2017). A deep reinforcement learning framework for the financial portfolio management problem. arXiv preprint arXiv:1706.10059.

[10] Markowitz, H. (1952) Portfolio Selection. The Journal of Finance, Vol. 7, No. 1, pp. 77-91. March. 1952.

[11] Meng, T. L., & Khushi, M. (2019). Reinforcement Learning in Financial Markets. Data, 4(3), 110.

[12] Michaud, R. O. (1989). The Markowitz optimization enigma: Is‘optimized’optimal?. Financial Analysts Journal, 45(1), 31-42.

[13] Pendharkar, P. C., & Cusatis, P. (2018). Trading financial indices with reinforcement learning agents. Expert Systems with Applications, 103, 1-13.

[14] Perold, A. F., and W. F. Sharpe, (1988), Dynamic strategies for asset allocation,
Financial Analysts Journal, 44,p.16.p.

[15] Sharpe, W. F. (1966). Mutual fund performance. The Journal of business, 39(1), 119-138.

[16] Sharpe, W. F. (1994). The sharpe ratio. Journal of portfolio management, 21(1), 49-58.

[17] Sutton, R. S., & Barto, A. G. (1998). Introduction to reinforcement learning (Vol. 135). Cambridge: MIT press

[18] Watkins, C. J. C. H., Dayan, P. (1992). Q-learning. Machine Learning, 8:279–292