本論文提出深度學習方法應用於智慧家庭情境，包含水果品質辨識及水果種類辨識。採用 Swin Transformer V2 模型辨識4種水果品質：蘋果、香蕉、番石榴和橘子。在弱標籤條件下準確率50.81%顯示模型無法由不具語義標籤有效學習水果品質視覺特徵，且於好或壞水果品質分類呈現平均預測結果。在強標籤條件下，準確率 98.98%顯示模型於學習水果品質視覺特徵表現極佳，且於好或壞水果品質分類誤判率極低，顯示高品質標籤對於模型學習的重要性。採用YOLOv9 模型進行十四種水果辨識：蘋果、奇異果、香蕉、柳橙、椰子、桃子、櫻桃、梨子、石榴、鳳梨、西瓜、甜瓜、葡萄和草莓，該模型訓練效能於 mAP@0.5 指標達到 98.07%準確度。本研究的貢獻在於提供社交輔助機器人應用即時訓練模型，且為未來智慧家庭應用提供指南。

This thesis proposes deep learning methods in smart home scenario including fruit quality recognition and fruit type recognition. Swin Transformer V2 model was adopted to recognize the fruit quality of four types of fruits: apple, banana, guava and orange. In weak label condition, the 46.6 percent accuracy show that the model cannot learn visual characteristics of the fruit quality efficiently using without semantic labels, and has average prediction results in the good or bad fruit quality classification. In strong label condition, the 94.2 percent accuracy reveals the high performance of the model in learning visual characteristics of the fruit quality, and has very low error rate in the good or bad fruit quality classification that high quality label is important to the model. YOLOv9 model was adopted in social robot to recognize fourteen fruit classes: apple, kiwi, banana, orange, coconut, peach, cherry, pear, pomegranate, pineapple, watermelon, melon, grape and strawberry, and the training performance of the model achieved a mean Average Precision at 0.5 of 98.07 percent. The contribution of the work that provides the training models for SAR applications in real time, and also provides guidelines for future smart home applications.

[1] S. Y. Chen, “Kalman filter for robot vision: a survey,” IEEE Transactions on Industrial Electronics, vol. 59, no. 11, pp. 4409–4420, 2012, doi: 10.1109/TIE.2011.2162714.
[2] A. Ayub, C. L. Nehaniv and K. Dautenhahn, “Don’t forget to buy milk: contextually aware grocery reminder household robot,” in proceedings of the 2022 IEEE International Conference on Development and Learning (ICDL), pp. 299–306, 2022, doi: 10.1109/ICDL53763.2022.9962208.
[3] K. A. Perumal, M. A. M. Ali and Z. H. Yahya, “Fire fighter robot with night vision camera,” in proceedings of the 2019 IEEE 15th International Colloquium on Signal Processing & Its Applications (CSPA), pp. 270–274, 2019, doi: 10.1109/CSPA.2019.8696077.
[4] C.-J. Liu, Y.-H. Li, J. Y.-H. Chiang, C.-Y. Wu, Y.-C. Sung, J.-Y. Lee, A. T.-Y. Chou and L. C.-L. Chen, “Social robot in home reality game,” in proceedings of the Taiwan Academic Network Conference 2024 (TANET 2024), pp. 332–337, 2024.
[5] C.-Y. Wang, I. H. Yeh and H.-Y. M. Liao, “YOLOv9: learning what you want to learn using programmable gradient information,” A. Leonardis, E. Ricci, S. Roth, O. Russakovsky, T. Sattler and G. Varol (eds), in proceedings of the ECCV 2024. ECCV 2024, Lecture Notes in Computer Science, vol. 15089, Springer, Cham, 2024, doi: 10.1007/978-3-031-72751-1_1.
[6] A. Dosovitskiy, L. Beyer, A. Kolesnikov, D. Weissenborn, X. Zhai, T. Unterthiner, M. Dehghani, M. Minderer, G. Heigold, S. Gelly, J. Uszkoreit and N. Houlsby, “An image is worth 16x16 words: transformers for image recognition at scale,” arXiv preprint arXiv: 2010.11929, 2020.
[7] J. Schmidhuber, “Deep learning in neural networks: an overview,” Neural Networks, vol. 61, pp. 85–117, 2015, doi: 10.1016/j.neunet.2014.09.003.
[8] A. B. Amjoud and M. Amrouch, “Object detection using deep learning, CNNs and vision transformers: a review,” IEEE Access, vol. 11, pp. 35479–35516, 2023, doi: 10.1109/ACCESS.2023.3266093.
[9] A. Shahzad, X. Gao, A. Yasin, K. Javed and S. M. Anwar, “A vision-based path planning and object tracking framework for 6-DOF robotic manipulator,” IEEE Access, vol. 8, pp. 203158–203167, 2020, doi: 10.1109/ACCESS.2020.3037540.
[10] A. Shahzad, X. Gao, A. Yasin, K. Javed and S. M. Anwar, “A vision-based path planning and object tracking framework for 6-DOF robotic manipulator,” IEEE Access, vol. 8, pp. 203158–203167, 2020, doi: 10.1109/ACCESS.2020.3037540.
[11] M. Kraus, N. Wagner, W. Minker, A. Agrawal, A. Schmidt, P. Krishna and W. Ertel, “KURT: a household assistance robot capable of proactive dialogue,” in proceedings of the 2022 17th ACM/IEEE International Conference on Human-Robot Interaction (HRI), pp. 855–859, 2022, doi: 10.1109/HRI53351.2022.9889357.
[12] M. T. Habib, D. M. Raza, M. M. Islam, D. B. Victor and M. A. I. Arif, “Applications of computer vision and machine learning in agriculture: a state-of-the-art glimpse,” in proceedings of the 2022 International Conference on Innovative Trends in Information Technology (ICITIIT), pp. 1–5, 2022, doi: 10.1109/ICITIIT54346.2022.9744150.
[13] A. Bochkovskiy, C.-Y. Wang and H.-Y. M. Liao, “YOLOv4: optimal speed and accuracy of object detection,” arXiv preprint arXiv:2004.10934, 2020.
[14] D. Karakaya, O. Ulucan and M. Turkan, “A comparative analysis on fruit freshness classification,” in proceedings of the 2019 Innovations in Intelligent Systems and Applications Conference (ASYU), pp. 1–4, 2019, doi: 10.1109/ASYU48272.2019.8946385.
[15] K. Sangeetha, P. V. Raja, S. S, S. J and R. S, “Classification of fruits and its quality prediction using deep learning,” in proceedings of the 2024 5th International Conference on Intelligent Communication Technologies and Virtual Mobile Networks (ICICV), pp. 342c346, 2024, doi: 10.1109/ICICV62344.2024.00059.
[16] B. Xiao, M. Nguyen and W. Q. Yan, “Fruit ripeness identification using transformers,” Appl Intell, vol. 53, pp. 22488–22499, 2023.
[17] R. Park, “Fruit Quality Classification,” 2022.
https://www.kaggle.com/datasets/ryandpark/fruit-quality-classification
[18] Ultralytics Inc., “Ultralytics YOLO,” Sep. 2025.
https://github.com/ultralytics/ultralytics
[19] Food Detection Dataset, “common-fruits-detection Computer Vision Model,” https://universe.roboflow.com/food-detection-dataset/common-fruits-detection, 2022.