A Virtual Support Agent for University Students Powered by a Large Language Model: Conversational User Experience Design Considerations

Joseph Benjamin Ilagan, Wolverix Skyler Yu, Samantha Mae See, Stephanie Rayco

In Artificial Intelligence in HCI (HCII 2025), LNCS, pp. 253–262. Springer, 2025.

Download PDF (accepted manuscript) Published version (Springer)

Abstract

This paper presents a pilot study of a university virtual support agent system powered by a Large Language Model (LLM) engineered to address student inquiries regarding processes and policies involving a student exchange program. Traditional approaches to implementing chatbots in university student support services have had their shortcomings. This study investigates the enablement of conversational support using generative artificial intelligence and natural conversation properties inherent in LLMs and how they may overcome such shortcomings from earlier attempts at conversational support. The LLMs’ susceptibility to ‘hallucination’ is mitigated through a combined approach of few-shot learning, retrieval augmented generation (RAG), and chain of thought (CoT) libraries in the training phase. The system utilizes OpenAI’s GPT-4o model behind a Python-based web framework, enabling the conversational interface used by the students. The information system is grounded in earlier email correspondences from program heads to students, PDF files containing primers, and other sources. The pilot implementation of the system involves an outbound student exchange program in a private university in the Philippines. Sophomore students were invited to test and evaluate the system and interviewed after the chatbot sessions.

Keywords: Conversational UX; Large language models; LLM; RAG; Chatbots; University chatbots; Generative AI; GenAI

About this copy

Accepted manuscript. This version of the contribution has been accepted for publication, after peer review (when applicable) but is not the Version of Record and does not reflect post-acceptance improvements, or any corrections. The Version of Record is available online at: http://dx.doi.org/10.1007/978-3-031-93415-5_15. Use of this Accepted Version is subject to the publisher’s Accepted Manuscript terms of use https://www.springernature.com/gp/open-research/policies/accepted-manuscript-terms

Cite as

Joseph Benjamin Ilagan, Wolverix Skyler Yu, Samantha Mae See, Stephanie Rayco (2025). A Virtual Support Agent for University Students Powered by a Large Language Model: Conversational User Experience Design Considerations. In Artificial Intelligence in HCI (HCII 2025), LNCS, pp. 253–262. Springer. https://doi.org/10.1007/978-3-031-93415-5_15