隨著影像編解碼技術的演進由H.264進入H.265、VP9，壓縮後的高傳真影像資料量可以降低近50%，加上有線/無線網路傳輸技術與設備規格大幅提升，網路傳輸成本降低，目前網路上傳輸佔比高達6.5成的流量是影像串流，影像解析度也由主流全高傳真1080 FHD (Full High Definition) 的圖像解析度提升至4K超高傳真UHD (Ultra High Definition)。目前現行人工智慧物件偵測與辨識擁有廣泛的應用，由特徵辨識到視頻分析等，物件檢測在安全、醫療、體育、交通、工廠等領域有關鍵應用。目前的視頻影像物件辨識與偵測研究大多聚焦於全高傳真FHD (1080p) 解析度，有鑒於視頻影像解析度日益提升，此份研究是以4K高傳真視頻影像串流 (4K Ultra High Definition Video Streaming) 應用爲出發，以NPU搭載不同的硬體架構 (CPU, GPGPU, SoC)及軟體流程，評測出較優的Video-Based的AI推理加速器架構。

With the evolution of video encoding and decoding technology from H.264 to H.265 and VP9, the compressed data size of high-resolution video can be reduced by nearly 50%. In addition, the significant improvements in wired/wireless network transmission technology and network device specifications on network bandwidth, the cost of network transmission is decreased gradually. At present, streaming video accounts for as much as 65% of the data traffic in the internet, and the video image resolution has been upgraded from the mainstream Full Hight Definition (FHD) 1080p to 4K Ultra High Definition (UHD). Currently, artificial intelligence object detection and recognition have a wide range of applications from character recognition to video analysis. Object detection plays a critical role in various industries such as security, healthcare, sports, transportation, and smart factory. Most of the current studies on artificial intelligence object recognition focus on FHD 1080p image resolution. Considering the increasing image resolution, this thesis focuses on 4K UHD video streaming applications for object detection. This study evaluates various hardware architectures (CPU, GPGPU, SoC) and correspondent software processes with NPU to determine the optimal Video-Based AI inference accelerator architecture.

[1] Ali Borji, Ming–Ming Cheng, Huaizu Jiang and Jia Li. “Salient Object Detection: A Benchmark”, in IEEE Transactions on Image Processing, Vol. 24, No. 12, pp. 5706-5723, December 2015.
[2] Licheng Jiao, (Fellow, IEEE), Fan Zhang, Fang Liu, (Senior Member, IEEE), Shuyuan Yang, (Senior Member, IEEE), Lingling Li, (Member, IEEE), Zhixi Feng, (Member, IEEE), AND Rong Qu, (Senior Member, IEEE). “A Survey of Deep Learning-Based Object Detection”, in IEEE Access, In version September 20, 2019.
[3] Micaela Verucchi, Gianluca Brilli, Davide Sapienza, Mattia Verasaniy, Marco Arenay, Francesco Gatti, Alessandro Capotondi, Roberto Cavicchioli, Marko Bertogna and Marco Solieri. “A Systematic Assessment of Embedded Neural Networks for Object Detection”, in IEEE pp. 937-944, 2020.
[4] Rafael Padilla, Sergio L. Netto, Eduardo A. B. da Silva. “A Survey on Performance Metrics for Object-Detection Algorithms”, Proceedings of the IWSSIP, pp. 237-242, 2020.
[5] Zhengxia Zou, Keyan Chen, Zhenwei Shi, Member Ieee, Yuhong Guo, Jieping Ye, Fellow IEEE. “Object Detection in 20 Years: A Survey”, Proceedings of the IEEE, Vol. 111, No.3, pp. 257-276, March 2023.
[6] Petros Amanatidis, George Iosifidis, Dimitris Karampatzakis. “Comparative Evaluation of Machine Learning Inference Machines on Edge-class Devices”, in PCI '21: Proceedings of the 25th Pan-Hellenic Conference on Informatics, pp 102–106, November 2021.
[7] Wootae Jeon, Kyungtae Kang, Jong-Chan Kim. “End-to-End Delay Analysis and Optimization of Object Detection Module for Autonomous Driving”, Kookmin University, Korea.
[8] Wikipedia, https://zh.wikipedia.org/zh-tw/GStreamer
[9] YUV. In Wikipedia, the free encyclopedia. Retrieved May 19, 2022, from https://zh.wikipedia.org/zh-tw/YUV
[10] RGB Color Model. In Wikipedia, the free encyclopedia. Retrieved 2023, from https://en.wikipedia.org/wiki/RGB_color_model
[11] Wim Taymans, Steve Baker, Any Wingo, Ronald S.Bultje, Stefan Kost. “GStreamer Application Development Manual (1.0.8)”, Ch. 3.3.
[12] OGY Stoilov. “4K Ultra HD TV Broadcast Standards Move Forward With 10bit HEVC Up To 60fps.”, 13 July 2014.
[13] Aakash K. Shetty, Ishani Saha, Rutvik M. Sanghvi, Siddhesh A. Save, Yashkumar J. Patel. “A Review: Object Detection Models”, 2021 6th International Conference for Convergence in Technology, Pune, India. Apr 02-04, 2021.
[14] Yifan Liu, BingHang Lu, Jingyu Peng, Zihao Zhang. “Research on the Use of YOLOv5 Object Detection Algorithm in Mask Wearing Recognition”, World Scientific Research Journal, Vol.6 Issue 11, 2020.
[15] Kaiyue Liu, Haitong Tang, Shuang He, Qin Yu, Yulong Xiong, Nizhuan Wang. “Performance Validation of Yolo Variants for Object Detection”, BIC 2021, January 22–24, 2021, Harbin, China
[16] 邱志聖，策略行銷分析：架構與實務應用
[17] Mengxi Hanyao, Yibo Jin, Zhuzhong Qianx, Sheng Zhangx, Sanglu Lu. “Edge-assisted Online On-device Object Detection for Real-time Video Analytics”, IEEE Infocom 2021.
[18] Fpsdisplaysink. In Gstreamer Freedesktop Org. Retrieved July 30, 2023, from https://gstreamer.freedesktop.org/documentation/debugutilsbad/fpsdisplaysink.html?gi-language=c#fpsdisplaysink-page
[19] Vijay Janapa Reddi, Christine Cheng, David Kanter, Peter Mattson, Guenther Schmuelling, Carole-Jean Wu, Brian Anderson, Maximilien Breughe, Mark Charlebois, William Chou, Ramesh Chukka, Cody Coleman, Sam Davis, Pan Deng, Greg Diamos, Jared Duke, Dave Fick, J. Scott Gardner, Itay Hubara, Sachin Idgunji, Thomas B. Jablin, Jeff Jiao, Tom St. John, Pankaj Kanwar, David Lee, Jeffery Liao, Anton Lokhmotov, Francisco Massa, Peng Meng, Paulius Micikevicius, Colin Osborne, Gennady Pekhimenko, Arun Tejusve Raghunath Rajan, Dilip Sequeira, Ashish Sirasao, Fei Sun,xxiii Hanlin Tang, Michael Thomson, Frank Wei, Ephrem Wu, Lingjie Xu, Koichi Yamada, Bing Yu, George Yuan, Aaron Zhong, Peizhao Zhang, Yuchen Zhou. “MLPerf Inference Benchmark”, 2020 ACM/IEEE 47th Annual International Symposium on Computer Architecture (ISCA).
[20] Ayoub Benali Amjoud, Mustapha Amrouch. “Object Detection Using Deep Learning, CNNs and Vision Transformers: A Review”, IEEE Access Vol. 11, 10 April 2023.
[21] 關有智. “4K電視技術概述”, https://rnd.pts.org.tw/
[22] H.264/MPEG-4 AVC. In Wikipedia, the free encyclopedia. Retrieved April 28, 2023, from https://zh.wikipedia.org/zh-tw/H.264/MPEG-4_AVC
[23] “CHROMA SUBSAMPLING TECHNIQUES” [Online image]. (2023). https://www.red.com/red-101/video-chroma-subsampling