隨著高等教育國際化的發展與跨國學習需求的持續成長，學生在進行留學規劃時面臨資訊過載、資訊零散，以及決策負荷過高等挑戰。傳統留學諮詢高度仰賴真人顧問，雖能提供具脈絡性的個人化建議，卻受限於服務時間、人力與人力成本，難以即時回應大量且多樣化的需求。近年來，生成式人工智慧與大型語言模型（Large Language Model, LLM）的崛起雖為自動化諮詢提供新契機，但其「幻覺」現象在涉及教育決策時仍具極高風險。而「檢索增強生成」（Retrieval-Augmented Generation, RAG）技術能整合外部專業知識庫，提供大型語言模型具可解釋性與來源依據的回應，有助於提升留學諮詢內容的準確性。本研究旨在開發一套整合 LLM 與 RAG 技術之「AI 留學諮詢代理人」，協助使用者透過自然語言對話獲得精準資訊，並探討其在任務導向諮詢情境中是否有助於提升任務完成度與使用者體驗，並分析不同人工智慧態度受試者的成效差異。本研究採用實驗研究法中的對抗平衡設計（Counterbalanced Design），招募 30 名高中生作為研究對象，隨機分派後輪流體驗「AI 留學諮詢代理人」輔以完成英國留學諮詢任之實驗組，以及「Google AI 網路搜尋引擎」輔以完成英國留學諮詢任之控制組。研究針對任務完成度（含教育制度、學校類型、個人條件等六面向）、對話氛圍（溝通有效性、社會存在感、知覺溫暖），以及聊天機器人優使性進行量化評估。此外，透過滯後序列分析（Lag Sequential Analysis, LSA）探究高、低任務完成度者在諮詢過程中的行為轉移模式，並結合半結構式訪談深入了解受試者對於系統之感受與建議。
研究結果顯示，採用「AI 留學諮詢代理人」輔以完成英國留學諮詢任之實驗組在整體任務完成度上顯著優於採用「Google AI 網路搜尋引擎」之控制組，尤其在「個人條件討論」面向成效最為顯著，顯示「AI 留學諮詢代理人」能有效引導使用者釐清個人背景需求。此外，實驗組在「互動功能與對話品質」之優使性感受上亦顯著優於控制組，顯示其在脈絡維持與需求理解上較符合使用者期待。然而，在對話氛圍感上，受試者對兩系統之社會存在感與知覺溫暖感受相當，未達顯著差異。在受試者特性方面，系統對「中人工智慧整體態度」使用者的任務完成度提升效果最為顯著。行為歷程分析進一步指出，高任務完成者具有較強的任務監控意識，傾向在產出規劃後再透過「資訊詢問」來深化諮詢內容；低任務完成者則較常陷入連續任務要求或反覆澄清的循環中。訪談結果亦指出，系統的引導機制能降低資訊整合負荷，提升決策效率。
綜合而言，本研究證實專門化設計之「AI 留學諮詢代理人」在輔助複雜教育決策中具備高度潛力，能有效將分散留學資訊轉化為具脈絡性的個人化建議，為未來智慧型留學諮詢平台導入生成式 AI 提供實證基礎與設計參考。未來可進一步強化資料來源的透明標註與更新機制，並將此模式擴展應用至其他專業諮詢領域。此外，亦建議深化系統的適性化引導，針對不同 AI 態度與數位素養的使用者提供差異化支持，建構具備邏輯連貫性與高度可信賴度的決策支援系統。

As international higher education and global learning demands continue to grow, students face significant challenges during the study abroad planning process, such as information overload, fragmented data, and high cognitive burdens. Traditional study abroad consulting relies heavily on human advisors; while providing contextualized and personalized advice, it is often limited by service hours and labor costs, making it difficult to meet diverse needs in real-time.
Although the rise of Large Language Models (LLMs) offers new opportunities for automated consultation, the phenomenon of "hallucinations" remains a risk in high-stakes educational decision-making. Retrieval-Augmented Generation (RAG) technology can integrate external professional knowledge bases to provide responses with explainability and source evidence, helping to improve the accuracy of consulting content. This study aims to develop an "AI Study Abroad Consulting Agent" integrating LLM and RAG technologies to help users obtain precise information through natural language interaction and to explore its application benefits in task-oriented consulting scenarios.
Accordingly, this study explores whether the assistive consulting model of the "AI Study Abroad Consulting Agent" can improve task completion and user experience, while analyzing performance differences among users with varying attitudes toward artificial intelligence. A counterbalanced experimental design was adopted, recruiting 30 high school students as participants who experienced both the "AI Study Abroad Consulting Agent" and the "Google AI Search Engine" in alternate orders to complete UK study abroad consulting tasks. Quantitative evaluations focused on task completion (covering six dimensions including educational systems and personal conditions), conversational atmosphere (communicative effectiveness, social presence, perceived warmth), and chatbot usability. Furthermore, Lag Sequential Analysis (LSA) was used to examine the behavioral transition patterns between high and low task-performance users, combined with semi-structured interviews to gain deeper insights into participants' perceptions and suggestions.
The results revealed that the experimental group significantly outperformed the control group in overall task completion, particularly in the "discussing personal conditions" dimension, demonstrating that the system effectively guides users in clarifying their background needs. Additionally, the experimental group received significantly higher ratings for "interaction functionality and dialogue quality" in usability perception, indicating that its context maintenance and need understanding better met user expectations. However, regarding the conversational atmosphere, no significant difference was found between the two systems in terms of social presence or perceived warmth. Regarding user characteristics, the system was most effective for users with a "moderate general attitude toward AI". Behavioral analysis further indicated that high-performance users exhibited stronger task monitoring, tending to deepen the consultation through "information inquiry" after receiving an output; low-performance users more often fell into cycles of continuous task requests or repetitive clarifications. Interview data also suggested that the system's guidance mechanism reduced the burden of information integration and improved decision-making efficiency.
In conclusion, this study demonstrates that a specialized "AI Study Abroad Consulting Agent" holds great potential in supporting complex educational decision-making by transforming fragmented information into contextualized personal suggestions. This provides an empirical foundation and design reference for future intelligent consulting platforms incorporating generative AI. Future research could further strengthen transparent source labeling and update mechanisms, expanding this model to other professional consulting domains. Additionally, it is recommended to deepen the adaptive guidance of the system to provide differentiated support for users with varying AI attitudes and digital literacy, building a decision support system with logical consistency and high reliability.

Al-Oraini, B. S. (2025). Chatbot dynamics: Trust, social presence and customer satisfaction in AI-driven services. Journal of Innovative Digital Transformation, 2(2), 109–130. https://doi.org/10.1108/JIDT-08-2024-0022
Alavi, M., Leidner, D., & Mousavi, R. (2024). Knowledge management perspective of generative artificial intelligence (GenAI). Alavi, Maryam, 1-12.
Arnold, M., Goldschmitt, M., & Rigotti, T. (2023). Dealing with information overload: A comprehensive review. Frontiers in Psychology, 14, Article 1122200. https://doi.org/10.3389/fpsyg.2023.1122200
Bodycott, P. (2009). Choosing a higher education study abroad destination: What mainland Chinese parents and students rate as important. Journal of Research in International Education, 8(3), 349–373. https://doi.org/10.1177/1475240909345818
Bollen, D., Knijnenburg, B. P., Willemsen, M. C., & Graus, M. (2025). The choice overload effect in online recommender systems. Manufacturing & Service Operations Management. https://doi.org/10.1287/msom.2022.0659
Borsci, S., Malizia, A., Schmettow, M., Van Der Velde, F., Tariverdiyeva, G., Balaji, D., & Chamberlain, A. (2022). The Chatbot Usability Scale: The design and pilot of a usability scale for interaction with AI-based conversational agents. Personal and Ubiquitous Computing, 26(1), 95–119. https://doi.org/10.1007/s00779-021-01582-9
Borsci, S., Schmettow, M. Re-examining the Chatbot Usability Scale (BUS-11) to assess user experience with customer relationship management chatbots. Personal and Ubiquitous Computing, 28(6), 1033-1044.
Braggaar, A., Liebrecht, C., Van Miltenburg, E., & Krahmer, E. (2026). Evaluating task-oriented dialogue systems: A systematic review of measures, constructs and their operationalisations. Northern European Journal of Language Technology, 12(1). https://doi.org/10.3384/nejlt.2000-1533.2026.5940
Caffaro, F., & Rizzo, G. (2024). Knowledge-enhanced conversational agents. Journal of Computer Science and Technology, 39(3), 585–609. https://doi.org/10.1007/s11390-024-2883-4
Caldarini, G., Jaf, S., & McGarry, K. (2022). A literature survey of recent advances in chatbots. Information, 13(1), 41.
Chaves, A. P., Egbert, J., Hocking, T., Doerry, E., & Gerosa, M. A. (2022). Chatbots language design: The influence of language variation on user experience with tourist assistant chatbots. ACM Transactions on Computer-Human Interaction, 29(2), 1-38.
Chen, Q., Gong, Y., Lu, Y., & Tang, J. (2022). Classifying and measuring the service quality of AI chatbot in frontline service. Journal of Business Research, 145, 552-568.
Cubillo, J. M., Sánchez, J., & Cerviño, J. (2006). International students’ decision-making process. International Journal of Educational Management, 20(2), 101–115. https://doi.org/10.1108/09513540610646091
Cureton, E. E. (1957). The upper and lower twenty-seven per cent rule. Psychometrika, 22(3), 293-296.
Dawood, M. (2024). Assessing the effectiveness of chatbots in providing personalized academic advising and support to higher education students: A narrative literature review. Studies in Technology Enhanced Learning, 4(1).
Deng, Z., & Yan, J. (2025). The Effect of Perceived Warmth, Competence, and Social Presence of AI-Driven Chabots on Consumers’ Engagement and Satisfaction. SAGE Open, 15(3), 21582440251365438.
Ferraro, C., Demsar, V., Sands, S., Restrepo, M., & Campbell, C. (2024). The paradoxes of generative AI-enabled customer service: A guide for managers. Business Horizons, 67(5), 549-559.
Feuerriegel, S., Hartmann, J., Janiesch, C., & Zschech, P. (2024). Generative AI. Business & Information Systems Engineering, 66, 111–126. https://doi.org/10.1007/s12599-023-00834-7
Gao, Y., Xiong, Y., Gao, X., Jia, K., Pan, J., Bi, Y., Dai, Y., Sun, J., Wang, M., & Wang, H. (2024). Retrieval-augmented generation for large language models: A survey. arXiv. https://doi.org/10.48550/arXiv.2312.10997
Grassini, S. (2025). Distinct predictors of positive attitudes toward artificial intelligence and general technology: Big five traits, gender, and age. Behaviour & Information Technology. https://doi.org/10.1080/0144929X.2025.2598623
Grossman, R., Liu, S., Chen, M. K., Smith, M., Borcea, C., & Chen, Y. (2026). How Generative AI Disrupts Search: An Empirical Study of Google Search, Gemini, and AI Overviews. arXiv preprint arXiv:2604.27790.
He, R., Cao, J., & Tan, T. (2025). Generative artificial intelligence: a historical perspective. National Science Review, 12(5), nwaf050.
Hou, G., Li, X., & Wang, H. (2024). How to improve older adults’ trust and intentions to use virtual health agents: An extended technology acceptance model. Humanities and Social Sciences Communications, 11, 1677. https://doi.org/10.1057/s41599-024-04232-6
Huang, T. J. (2024). Translating authentic selves into authentic applications: Private college consulting and selective college admissions. Sociology of Education, 97(2), 174-192. https://doi.org/10.1177/00380407231202975
Jenneboer, L., Herrando, C., & Constantinides, E. (2022). The impact of chatbots on customer loyalty: A systematic literature review. Journal of theoretical and applied electronic commerce research, 17(1), 212-229.
Jin, S. V., & Youn, S. (2023). Social presence and imagery processing as predictors of chatbot continuance intention in human-AI-interaction. International Journal of Human–Computer Interaction, 39(9), 1874-1886.
Juquelier, A., Poncin, I., & Hazée, S. (2025). Empathic chatbots: A double-edged sword in customer experiences. Journal of Business Research, 188, Article 115074. https://doi.org/10.1016/j.jbusres.2024.115074
Kallio, H., Pietilä, A.-M., Johnson, M., & Kangasniemi, M. (2016). Systematic methodological review: developing a framework for a qualitative semi-structured interview guide. Journal of Advanced Nursing, 72(12), 2954–2965. doi:10.1111/jan.13031
Kasneci, E., Seßler, K., Küchemann, S., Bannert, M., Dementieva, D., Fischer, F., ... & Kasneci, G. (2023). ChatGPT for good? On opportunities and challenges of large language models for education. Learning and individual differences, 103, 102274.
Kaya, F., Aydın, F., Schepman, A., Rodway, P., Yetişensoy, O., & Demir Kaya, M. (2024). The roles of personality traits, AI anxiety, and demographic factors in attitudes toward artificial intelligence. International Journal of Human–Computer Interaction. https://doi.org/10.1080/10447318.2022.2151730
Kelley, Truman L. "The selection of upper and lower groups for the validation of test items." Journal of educational psychology 30.1 (1939): 17.
Labadze, L., Grigolia, M., & Machaidze, L. (2023). Role of AI chatbots in education: Systematic literature review. International Journal of Educational Technology in Higher Education, 20, Article 56. https://doi.org/10.1186/s41239-023-00426-1
Lee, J., Lee, D., & Lee, J. G. (2024). Influence of rapport and social presence with an AI psychotherapy chatbot on users’ self-disclosure. International Journal of Human–Computer Interaction, 40(7), 1620-1631.
Lewis, P., Perez, E., Piktus, A., Petroni, F., Karpukhin, V., Goyal, N., ... & Kiela, D. (2020). Retrieval-augmented generation for knowledge-intensive nlp tasks. Advances in neural information processing systems, 33, 9459-9474.
López-Solís, O., Luzuriaga-Jaramillo, A., Bedoya-Jara, M., Naranjo-Santamaría, J., Bonilla-Jurado, D., & Acosta-Vargas, P. (2025). Effect of generative artificial intelligence on strategic decision-making in entrepreneurial business initiatives: A systematic literature review. Administrative Sciences, 15(2), 66.
Maltz, E., Murphy, K. E., & Hand, M. L. (2007). Decision support for university enrollment management: Implementation and experience. Decision Support Systems, 44(1), 106–123. https://doi.org/10.1016/j.dss.2007.03.008
Manigandan, L., & Alur, S. (2024). An in-depth investigation into the influence of chatbot usability and age on continuous intention to use: A comprehensive study. Asia Pacific Journal of Information Systems, 34(1), 351–371. https://doi.org/10.14329/apjis.2024.34.1.351
Mazzarol, T., & Soutar, G. N. (2002). “Push-pull” factors influencing international student destination choice. International Journal of Educational Management, 16(2), 82–90. https://doi.org/10.1108/09513540210418403
Meng, H., Lu, X., & Xu, J. (2025). The impact of chatbot response strategies and emojis usage on customers’ purchase intention: The mediating roles of psychological distance and performance expectancy. Behavioral Sciences, 15(2), 117. https://doi.org/10.3390/bs15020117
Rahim, N. I. M., Iahad, N. A., Yusof, A. F., & Al-Sharafi, M. A. (2022). AI-based chatbots adoption model for higher-education institutions: A hybrid PLS-SEM-neural network modelling approach. Sustainability, 14(19), 12726.
Oh, C. S., Bailenson, J. N., & Welch, G. F. (2018). A systematic review of social presence: Definition, antecedents, and implications. Frontiers in Robotics and AI, 5, 114.
Papneja, H., & Yadav, N. (2025). Self-disclosure to conversational AI: a literature review, emergent framework, and directions for future research. Personal and ubiquitous computing, 29(2), 119-151.
Park, S. (2025). Generative artificial intelligence and misinformation: A review. AI & Society. https://doi.org/10.1007/s00146-025-02620-3
Peng, M., Xu, Z., & Huang, H. (2021). How does information overload affect consumers’ online decision process? An event-related potentials study. Frontiers in Neuroscience, 15, Article 695852. https://doi.org/10.3389/fnins.2021.695852
Pollatsek, A., & Well, A. D. (1995). On the use of counterbalanced designs in cognitive research: a suggestion for a better and more powerful analysis. Journal of Experimental psychology: Learning, memory, and Cognition, 21(3), 785.
Roy, D., & Dutta, M. (2022). A systematic review and research perspective on recommender systems. Journal of Big Data, 9, Article 59. https://doi.org/10.1186/s40537-022-00592-5
Schepman, A., & Rodway, P. (2023). The General Attitudes towards Artificial Intelligence Scale (GAAIS): Confirmatory Validation and Associations with Personality, Corporate Distrust, and General Trust. International Journal of Human–Computer Interaction, 39(13), 2724–2741. https://doi.org/10.1080/10447318.2022.2085400
Schepman, A., & Rodway, P. (2026). Validation of the Short General Attitudes Towards Artificial Intelligence Scale: The Short GAAIS-10. International Journal of Human–Computer Interaction, 1-17.
Siro, C., Aliannejadi, M., & de Rijke, M. (2022). Understanding user satisfaction with task-oriented dialogue systems. In Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval. Association for Computing Machinery. https://doi.org/10.1145/3477495.3531798
Su-Fang Yeh, Meng-Hsin Wu, Tze-Yu Chen, Yen-Chun Lin, XiJing Chang, You-Hsuan Chiang, and Yung-Ju Chang. 2022. How to Guide Task-oriented Chatbot Users, and When: A Mixed-methods Study of Combinations of Chatbot Guidance Types and Timings. In Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems (CHI '22). Association for Computing Machinery, New York, NY, USA, Article 488, 1–16. https://doi.org/10.1145/3491102.3501941
Thieme, S. (2017). Educational consultants in Nepal: professionalization of services for students who want to study abroad. Mobilities, 12(2), 243–258. https://doi.org/10.1080/17450101.2017.1292780
Yan, L., Greiff, S., Teuber, Z., & Gašević, D. (2023). Practical and ethical challenges of large language models in education: A systematic scoping review. British Journal of Educational Technology, 55(1), 90–112. https://doi.org/10.1111/bjet.13370
Yang M, Peng X, Wang Q, Zhao YC, Wang X (2025;), "Is being human-like beneficial? The effect of anthropomorphism on chatbot persuasion in e-commerce". Internet Research, Vol. ahead-of-print No. ahead-of-print. https://doi.org/10.1108/INTR-10-2023-0866
Yuksekgonul, M., Bianchi, F., Boen, J., Liu, S., Huang, Z., Guestrin, C., & Zou, J. (2024). Textgrad: Automatic" differentiation" via text. arXiv preprint arXiv:2406.07496.
Zhang, Z., Takanobu, R., Zhu, Q., Huang, M., & Zhu, X. (2020). Recent advances and challenges in task-oriented dialog systems. Science China Technological Sciences, 63(10), 2011-2027.