Machine learning is revolutionizing business operations across industry sectors. Among different learning techniques, deep reinforcement learning (DRL) has received broad attention in recent years due to the salient performance of AlphaGo, an artificial intelligence (AI) system empowered by DRL. DRL is a model-free and data-driven approach to develop near-optimal policies for sequential decision-making problems. Intrigued by the success of DRL in various fields, we, in this study, assess the applicability of DRL to multi-period inventory control under stochastic demand, which is a classical Markov Decision Process problem. Working with the largest distributor of electronics manufacturing services (EMS) in the world, we propose deep Q-networks (DQN) for intelligent inventory control (IIC). Facing erratic and non-stationary demand for electronic components with limited market life cycle, the distributor could not infer the exact demand distribution and solve the inventory optimization problem analytically in a finite-horizon with lost sales setting. Hence, we develop DQN by specifying relevant state and decision inputs, and then designing a data-driven simulation environment, in which the agent is trained over thousands of episodes. For trained items, DQN outperforms the benchmark in a few ways. First, DQN can reduce the total inventory by at least 40% while achieving better service level. Second, when penalty parameter increases, DQN can effectively reduce the amount of out-of-stock. While we transfer trained DQN into testing sets, within the same item, the out-of-sample performance is excellent. For other unseen items, we use the Maximum Entropy Bootstrap to train ensemble DDQN and make our DRL agent more robust. Given the promising results in our experiments, we discuss implications, limitations, and further directions for applying DRL/DQN to business decision-making problems.

Arulkumaran, K., Deisenroth, P. M., Brundage, M., & Bharath, A. A. (2017) A brief survey of deep reinforcement learning. ArXiv: 1708.05866v2.
Chaharsooghi, S., Heydari, J., & Zegordi, S. (2008) A reinforcement learning model for supply chain ordering management: An application to the beer game. Decision Support Systems, 45(4): 949-959.
Chollet, F. (2017) Deep Learning with Python. Manning Publications.
Giannoccaro, I., & Pontrandolfo, P. (2002) Inventory management in supply chains: A reinforcement learning approach. International Journal of Production Economics, 78(2): 153-161.
Gijsbrechts, J., Boute, R., Zhang, D., & Van Mieghem, J. (2019) Can Deep Reinforcement Learning Improve Inventory Management? Performance on Dual Sourcing, Lost Sales and Multi-Echelon Problems. Available at SSRN https://papers.ssrn.com/sol3/papers.cfm?abstract_id=3302881.
Gosavi, A. (2009) Reinforcement learning: A tutorial survey and recent advances. INFORMS Journal on Computing, 21(3): 177-345.
Kingma, D. P., & Ba, J. (2017). Adam: A Method for Stochastic Optimization. ArXiv:1412.6980v9.
Lin, L. J. (1992) Self-improving reactive agents based on reinforcement learning, planning and teaching. Machine Learning, 8(3-4): 293-321.
Mnih V., Kavukcuoglu, K., Silver, D., Graves, A., Antonoglou, I., Wierstra, D., & Riedmiller, M. (2013) Playing Atari with deep reinforcement learning. ArXiv:1312.5602v1.
Mnih, V., Kavukcuoglu, K., Silver, D., Rusu, A. A., Veness, J., Bellemare, M. G., Graves, A., Riedmiller, M., Fidjeland, A. K., & Ostrovski, G., Petersen, S., Beattie, C., Sadik, A., Antonoglou, I., King, H., Kumaran, D., Wierstra, D., Legg, S., & Hassabis, D. (2015) Human-level control through deep reinforcement learning. Nature, 518: 529-533.
Oroojlooyjadid, A., Snyder, L., & Takáč, M. (2019) Applying deep learning to the newsvendor problem. IISE Transactions, in press.
Oroojlooyjadid, A., Nazari, M., Snyder, L., & Takáč, M. (2019b) A deep Q-Network for the beer game: A deep reinforcement learning algorithm to solve inventory optimization problems. ArXiv:1708.05924v3.
Porteus, E. L. (2002) Foundations of Stochastic Inventory Theory. Stanford University Press, California.
Qi, X., Wu, G., Boriboonsomsin, K., Barth, M. J., & Gonder, J. (2016) Data-driven reinforcement learning–based real-time energy management system for plug-in hybrid electric vehicles. Transportation Research Record, 2572(1), 1-8.
Schaul, T., Quan, J., Antonoglou, I., & Silver, D. (2015). Prioritized Experience Replay. ArXiv: 1511.05952v4.
Silver, D., Schrittwieser, J., Simonyan, K., Antonoglou, I., Huang, A., Guez, A., Hubert, T., Baker, L., Lai, M., Bolton, A., Chen, Y., Lillicrap, T., Hui, F., Sifre, L., van den Driessche, G., Graepel, T., Hassabis, D. (2017) Mastering the game of Go without human knowledge. Nature 550: 354-359.
Sutton, R. S. and Barto, A. G. (2018) Reinforcement Learning: An Introduction. Second edition, MIT Press, Cambridge.
Van Hasselt, H., Guez, A., and Silver, D. (2015). Deep Reinforcement Learning with Double Q-Learning. ArXiv:1509.06461.
Vinod, H.D. and Lòpez-de-Lacalle, J. (2009). Maximum entropy bootstrap for time series. The meboot R Package. J. Stat. Softw., 29 (2009), pp. 1-19