Comparative Analysis of SVM and IndoBERT for Intent Classification in Indonesian Overtime Chatbots
Abstract
Digital transformation in higher education requires the development of intelligent and adaptive information systems, including services such as overtime submission for university staff. Chatbots offer a promising solution to enhance user interaction with the E-LEMBUR system. However, developing chatbots in academic settings poses challenges, including limited training data, complex overtime policies, and diverse institutional terminology. This study compares two intent classification approaches: Support Vector Machine (SVM), a traditional machine learning method, and IndoBERT, a transformer-based model designed for the Indonesian language. The dataset comprises 250 real user queries from the overtime system at Institut Teknologi Sepuluh Nopember (ITS). Experimental results show IndoBERT achieves 87% accuracy, slightly outperforming SVM at 85%. While IndoBERT offers better accuracy, it demands higher computational resources, presenting a trade-off between performance and efficiency. This study contributes by validating IndoBERT’s effectiveness on a limited dataset, establishing an initial benchmark for intent classification in overtime chatbots, and offering implementation recommendations aligned with university IT infrastructure. These findings lay the groundwork for developing context-aware information systems for staff services in Indonesian higher education.
References
Santosa, R., Fariza, A., & Arifin, F. (2024). Classification of flood disaster level news articles using Machine Learning. *Indonesian Journal of Computer Science*, 13(1), 264–275.
Adamopoulou, E., & Moussiades, L. (2020). An overview of chatbot technology. In Artificial Intelligence Applications and Innovations: 16th IFIP WG 12.5 International Conference, AIAI 2020, Neos Marmaras, Greece, June 5–7, 2020, Proceedings, Part II 16 (pp. 373-383). Springer International Publishing.
Wang, P., Fan, E., & Wang, P. (2020). Comparative Analysis of Image Classification Algorithms Based on Traditional Machine Learning and Deep Learning. Pattern Recognition Letters, 136, 1-9. https://doi.org/10.1016/j.patrec.2020.07.042
Kamath, C. N., Bukhari, S. S., & Dengel, A. (2018). Comparative Study between Traditional Machine Learning and Deep Learning Approaches for Text Classification. DocEng '18: ACM Symposium on Document Engineering, 1-11. https://doi.org/10.1145/3209280.3209526
Han, S., & Lee, M. K. (2022). FAQ chatbot and inclusive learning in massive open online courses. Computers & Education, 179, 104395.
Hikmawati, E., Maulidevi, N. U., & Surendro, K. (2021). Minimum threshold determination method based on dataset characteristics in association rule mining. Journal of Big Data, 8, 1-17.
Nabiilah, G. Z., Alam, I. N., Purwanto, E. S., & Hidayat, M. F. (2024). Indonesian multilabel classification using IndoBERT embedding and MBERT classification. International Journal of Electrical & Computer Engineering (2088-8708), 14(1).
Muftie, F., & Haris, M. (2023, August). Indobert based data augmentation for Indonesian text classification. In 2023 International Conference on Information Technology Research and Innovation (ICITRI) (pp. 128-132). IEEE.
Rochim, A. F., Widyaningrum, K., & Eridani, D. (2021, December). Performance Comparison of Support Vector Machine Kernel Functions in Classifying COVID-19 Sentiment. In 2021 4th International Seminar on Research of Information Technology and Intelligent Systems (ISRITI) (pp. 224-228). IEEE.
Salleh, S. A., Khalid, N., Danny, N., Zaki, N. A. M., Ustuner, M., Latif, Z. A., & Foronda, V. (2024). Support Vector Machine (SVM) and Object Based Classification in Earth Linear Features Extraction: A Comparison. Revue Internationale de Géomatique, 33.
Abdullah, D. M., & Abdulazeez, A. M. (2021). Machine learning applications based on SVM Classification a Review. Qubahan Academic Journal, 1(2), 81-90.
Perdana, R. S., & Adikara, P. P. (2025). Multi-task Learning for Named Entity Recognition and Intent Classification in Natural Language Understanding Applications. Journal of Information Systems Engineering and Business Intelligence, 11(1), 1-16.
Putri, S. A., Fadhila, S., & Umam, K. (2024). Peran Chatbot dalam Meningkatkan Responsivitas dan Efisiensi Pelayanan Publik pada Era Digitalisasi. Prosiding Seri Praktikum Ilmu-Ilmu Sosial-Politik, 1(1), 153-159.
Kraus, S., Jones, P., Kailer, N., Weinmann, A., Chaparro-Banegas, N., & Roig-Tierno, N. (2021). Digital transformation: An overview of the current state of the art of research. Sage Open, 11(3), 21582440211047576.
Miao, J., & Zhu, W. (2022). Precision–recall curve (PRC) classification trees. Evolutionary intelligence, 15(3), 1545-1569.
Heydarian, M., Doyle, T. E., & Samavi, R. (2022). MLCM: Multi-label confusion matrix. Ieee Access, 10, 19083-19095.
Pan, L., Hang, C. W., Sil, A., & Potdar, S. (2022, June). Improved text classification via contrastive adversarial training. In Proceedings of the AAAI Conference on Artificial Intelligence (Vol. 36, No. 10, pp. 11130-11138).
Assayed, S., Shaalan, K., & Alkhatib, M. (2023). A chatbot intent classifier for supporting high school students. EAI Endorsed Transactions on Scalable Information Systems, 1.
Souha, A., Ouaddi, C., Benaddi, L., & Jakimi, A. (2023, December). Pre-trained models for intent classification in chatbot: Comparative study and critical analysis. In 2023 6th international conference on advanced communication technologies and networking (CommNet) (pp. 1-6). IEEE.
Balisa, D., Leffia, A., & Shino, Y. (2024). Memanfaatkan fungsi sistem informasi manajemen: Prospek dan tantangan di dunia bisnis. Jurnal MENTARI: Manajemen, Pendidikan dan Teknologi Informasi, 2(2), 123-133.
