Pengembangan Chatbot Customer Service dengan Retrieval-Augmented Generation pada Usaha Mikro Kecil Menengah KampusMadu
DOI:
https://doi.org/10.47065/bulletincsr.v6i4.1185Keywords:
Chatbot; Customer Service; Large Language Model; Retrieval-Augmented Generation; MSMEAbstract
Customer service in Micro, Small, and Medium Enterprises (MSMEs) is generally still conducted manually, resulting in delayed responses, inconsistent information, and high operational workload. This study aims to develop and evaluate a customer service chatbot based on Large Language Model using the Retrieval-Augmented Generation (RAG) method as an automation solution for customer service at KampusMadu MSME. The system was developed using an adaptive Waterfall approach encompassing requirements analysis, design, implementation, and testing stages. Testing was conducted through three approaches: functional testing using Black Box Testing method, retrieval accuracy testing using Precision@3, Recall@3, and Hit Rate@3 metrics, and usability testing using the System Usability Scale (SUS) involving 10 respondents. The test results show that all system functions operate as required, with average Precision@3 of 63.3%, Recall@3 of 72.5%, and Hit Rate@3 of 100%. The SUS score obtained was 69.75 (category "OK"), indicating that the system is reasonably acceptable to users. The chatbot system was successfully integrated into the KampusMadu website as an interactive widget directly accessible to customers. This study demonstrates that the application of RAG in MSME chatbots can effectively support the automation of customer service in a more responsive, informative, and contextual manner.
Downloads
References
P. Peykani, F. Ramezanlou, C. Tanasescu, and S. Ghanidel, “Large Language Models: A Structured Taxonomy and Review of Challenges, Limitations, Solutions, and Future Directions,” Appl. Sci., vol. 15, no. 14, p. 8103, Jul. 2025, doi: 10.3390/app15148103.
P. Lewis et al., “Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks,” Adv. Neural Inf. Process. Syst., vol. 2020-Decem, Apr. 2021, [Online]. Available: http://arxiv.org/abs/2005.11401
Gufranaka Samudra, Ahmad Turmudi Zy, and Ermanto, “Implementasi Retrieval Augmented Generation (RAG) Dalam Perancangan Chatbot Kesehatan Pencernaan,” JSAI (Journal Sci. Appl. Informatics), vol. 8, no. 1, pp. 181–188, Jan. 2025, doi: 10.36085/jsai.v8i1.7678.
L. P. Dynia and J. S. Wibowo, “Pengembangan Chatbot Dialogflow Untuk Rekomendasi UMKM Kuliner Kota Semarang Berbasis Natural Language Processing,” Techno.Com, vol. 24, no. 2, pp. 405–418, May 2025, doi: 10.62411/tc.v24i2.12372.
R. H. Ardiansyah and A. G. Sulaksono, “Layanan pelanggan berbasis Natural Language Processing melalui chatbot pada aplikasi pesan,” J. Inf. Syst. Appl. Dev., vol. 1, no. 1, pp. 29–37, Mar. 2023, doi: 10.26905/jisad.v1i1.9858.
G. D. Albert and A. Voutama, “Pengembangan Chatbot Berbasis Pdf Menggunakan Local Retrieval-Augmented Generation (Rag) Dan Ollama,” J. Inform. dan Tek. Elektro Terap., vol. 13, no. 2, Apr. 2025, doi: 10.23960/jitet.v13i2.6361.
N. Agustina, R. Mardaweni, D. Komara, R. M. Widiastuti, and H. Gusdevi, “Aplikasi Chatbot Desa Wisata Cimindi Menggunakan Natural Language Processing,” Naratif J. Nas. Riset, Apl. dan Tek. Inform., vol. 6, no. 1, pp. 42–49, Jun. 2024, doi: 10.53580/naratif.v6i1.282.
D. R. Mukti and A. Salam, “The Development of a Deployment System Architecture for a Flask-Based Chatbot Using an LSTM NLP Model for Customer Service Question & Answer,” J. Appl. Informatics Comput., vol. 9, no. 3, pp. 732–740, 2025, doi: 10.30871/jaic.v9i3.9305.
A. Saravanos and M. X. Curinga, “Simulating the Software Development Lifecycle: The Waterfall Model,” Appl. Syst. Innov., vol. 6, no. 6, p. 108, Nov. 2023, doi: 10.3390/asi6060108.
Aisya Tyanafisya et al., “Black Box Testing Using Equivalence Partitioning Technique On Bakkar Website,” J. INSTEK (Informatika Sains dan Teknol., vol. 10, no. 1, pp. 230–238, May 2025, doi: 10.24252/instek.v10i1.52951.
D. Vuji?, A. Njeguš, and N. Ba?anin Džakula, “Enhancing Retrieval - Augmented Generation with Graph-Based Retrieval and Generative Modeling,” in Proceedings of the International Scientific Conference - Sinteza 2025, Beograd, Serbia: Singidunum University, 2025, pp. 3–9. doi: 10.15308/Sinteza-2025-3-9.
A. Brown, M. Roman, and B. Devereux, “A Systematic Literature Review of Retrieval-Augmented Generation: Techniques, Metrics, and Challenges,” Big Data Cogn. Comput., vol. 9, no. 12, 2025, doi: 10.3390/bdcc9120320.
R. Rosmiati, “Analisis Dan Pengujian Sistem Menggunakan Black Box Testing Equivalence Partitioning,” J. Sains Komput. dan Teknol. Inf., vol. 3, no. 2, pp. 56–63, May 2021, doi: 10.33084/jsakti.v3i2.1932.
M. Singh, E. Ray, M. Ferradou, and A. Barraza-Urbina, “Metric@CustomerN: Evaluating Metrics at a Customer Level in E-Commerce,” CEUR Workshop Proc., vol. 3450, 2023.
F. V. Krasnov, “Learning-to-Rank in B2B e-commerce catalogs: A digital exper-iment and conversion analysis,” Bus. Informatics, vol. 20, no. 1, pp. 54–66, Mar. 2026, doi: 10.17323/2587-814X.2026.1.54.66.
F. I. Apridal and F. Nova, “Development of an Interactive Chatbot Using Sahabat-AI Model with Retrieval-Augmented Generation Method,” vol. 19, no. 1, pp. 1380–1396, 2026, doi: 10.24036/jtip.v19i1.1105.
Y. Mao et al., “A Survey on LoRA of Large Language Models,” Front. Comput. Sci., vol. 19, no. 7, p. 197605, Oct. 2024, doi: 10.1007/s11704-024-40663-9.
T. T. Procko and O. Ochoa, “Graph Retrieval-Augmented Generation for Large Language Models: A Survey,” in 2024 Conference on AI, Science, Engineering, and Technology (AIxSET), IEEE, Sep. 2024, pp. 166–169. doi: 10.1109/AIxSET62544.2024.00030.
C. Duarte Brito, A. Luís Lopes, and F. Batista, “Chatbot based on LLMs for Support and Development Teams,” Atas da Conf. da Assoc. Port. Sist. Inf., 2025, [Online]. Available: https://aisel.aisnet.org/capsi2025/20/
Dhia Tawakalna and Junta Zeniarja, “Sistem Chatbot Kesehatan Mental Berbasis Llm Dengan Deteksi Emosi Dan Retrieval Augmented Generation,” Rabit J. Teknol. dan Sist. Inf. Univrab, vol. 11, no. 1, pp. 1398–1412, Jan. 2026, doi: 10.36341/rabit.v11i1.7347.
F. I. Falaq and N. Saputra, “Pengembangan Retrieval-Augmented Generation (Rag) Chatbot Menggunakan Metode Design Thinking Untuk Mendukung Personalized Learning,” PINTER J. Pendidik. Tek. Inform. dan Komput., vol. 9, no. 2, pp. 1–10, Dec. 2025, doi: 10.21009/pinter.9.2.1.
Bila bermanfaat silahkan share artikel ini
Berikan Komentar Anda terhadap artikel Pengembangan Chatbot Customer Service dengan Retrieval-Augmented Generation pada Usaha Mikro Kecil Menengah KampusMadu
ARTICLE HISTORY
How to Cite
Issue
Section
Copyright (c) 2026 Ahmad Zakky Zamani, Novi Tristanti
This work is licensed under a Creative Commons Attribution 4.0 International License.
Authors who publish with this journal agree to the following terms:
- Authors retain copyright and grant the journal right of first publication with the work simultaneously licensed under Creative Commons Attribution 4.0 International License that allows others to share the work with an acknowledgment of the work's authorship and initial publication in this journal.
- Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgment of its initial publication in this journal.
- Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work (Refer to The Effect of Open Access).
