Comparing Four Classifiers for Imbalanced Computer Engineering Student Feedback
Abstract
Open-ended course evaluation comments provide detailed information about students' learning experiences, but large volumes of text are difficult to review consistently by hand. This study compares Decision Tree, Support Vector Machine (SVM), Random Forest, and Gradient Boosting for sentiment classification of Computer Engineering student comments. The initial dataset contained 1,075 comments. Compound-score labeling produced 1,065 neutral comments (99.07%), five negative comments (0.47%), and five positive comments (0.47%). After incomplete records were removed, 415 observations remained for modeling, and the test set contained 207 observations. Before SMOTE, Random Forest and SVM achieved 98.06% accuracy, while Decision Tree and Gradient Boosting achieved 97.58%. After SMOTE, SVM provided the best balance, with 98.55% accuracy, 66.18% macro precision, 66.67% macro recall, and 66.42% macro F1-score. Although overall accuracy was high, the positive class was not detected. Therefore, SVM was the most balanced model in this experiment, while macro-averaged metrics and class-wise errors remain essential when evaluating highly imbalanced sentiment data
References
Adyanata, I. K., Gunadi, I. G. A., & Sunarya, I. M. G. (2026). Comparison of SMOTE, class weighting, and classical machine learning models on the ID-SMSA Indonesian stock market dataset. Journal of System and Computer Engineering, 7(3), 229–240. https://doi.org/10.61628/jsce.v7i3.2688
Andriyani, D., Faqih, A., & Permana, S. E. (2025). The effect of SMOTE application on Support Vector Machine performance in sentiment classification on imbalanced datasets. Journal of Artificial Intelligence and Engineering Applications, 4(2), 752–757. https://doi.org/10.59934/jaiea.v4i2.742
Boudegzdame, N., Sedki, K., Tspora, R., & Lamy, J.-B. (2024). SMOTE: Are we learning to classify or to detect synthetic data? In Proceedings of the 16th International Conference on Agents and Artificial Intelligence (ICAART) (Vol. 3, pp. 283–290). SciTePress. https://doi.org/10.5220/0012325300003636
Dake, D. K., & Gyimah, E. (2023). Using sentiment analysis to evaluate qualitative students' responses. Education and Information Technologies, 28, 4629–4647. https://doi.org/10.1007/s10639-022-11349-1
Dervenis, C., Kanakis, G., & Fitsilis, P. (2024). Sentiment analysis of student feedback: A comparative study employing lexicon and machine learning techniques. Studies in Educational Evaluation, 83, 101406. https://doi.org/10.1016/j.stueduc.2024.101406
Deshpande, S. B., Tangod, K. K., Srinivasaiah, S. H., Alahmadi, A. A., Alwetaishi, M., Michael, G. K. O., & Rajendran, S. (2025). Elevating educational insights: Sentiment analysis of faculty feedback using advanced machine learning models. Advances in Continuous and Discrete Models, 2025, 89. https://doi.org/10.1186/s13662-025-03933-9
Elreedy, D., Atiya, A. F., & Kamalov, F. (2024). A theoretical distribution analysis of synthetic minority oversampling technique (SMOTE) for imbalanced learning. Machine Learning, 113, 4903–4923. https://doi.org/10.1007/s10994-022-06296-4
Grimalt-Álvaro, C., & Usart, M. (2024). Sentiment analysis for formative assessment in higher education: A systematic literature review. Journal of Computing in Higher Education, 36, 647–682. https://doi.org/10.1007/s12528-023-09370-5
Hanif, K. H., Fadllullah, A., Muntiari, N. R., & Fahrezi, I. A. (2025). A comparative sentiment analysis of Computer Engineering student feedback using Decision Trees and SVM. Jurnal Inotera, 10(1), 71–82. https://doi.org/10.31572/inotera.Vol10.Iss1.2025.ID436
Hanif, K. H., Muntiari, N. R., Harto, D., & Wiranata, D. S. (2026). Perbandingan analisis sentimen komentar mahasiswa Prodi Teknik Komputer menggunakan algoritma Decision Tree, Support Vector Machine (SVM) dan Random Forest. INSECT (Informatics and Security): Jurnal Teknik Informatika, 12(1), 24–33. https://doi.org/10.33506/insect.v12i01.5144
Hutami, W. P., Wijayanto, H., & Sulvianti, I. D. (2022). Penerapan Support Vector Machine dengan SMOTE untuk klasifikasi sentimen pemberitaan Omnibus Law pada situs CNNIndonesia.com. Xplore: Journal of Statistics, 11(1), 26–35. https://doi.org/10.29244/xplore.v11i1.852
Karfindo, Turaina, R., & Saputra, R. (2023). Optimalisasi klasifikasi umpan balik mahasiswa terhadap layanan kampus dengan sinergi Random Forest dan SMOTE. Jurnal Nasional Komputasi dan Teknologi Informasi, 6(6), 820–827. https://doi.org/10.32672/jnkti.v6i6.7269
Kasumba, R., & Neumann, M. (2024). Practical sentiment analysis for education: The power of student crowdsourcing. Proceedings of the AAAI Conference on Artificial Intelligence, 38(21), 23110–23118. https://doi.org/10.1609/aaai.v38i21.30356
Kosolwattana, T., Liu, C., Hu, R., Han, S., Chen, H., & Lin, Y. (2023). A self-inspected adaptive SMOTE algorithm (SASMOTE) for highly imbalanced data classification in healthcare. BioData Mining, 16, 15. https://doi.org/10.1186/s13040-023-00330-4
Mihai, G. (2024). Sentiment analysis of students' feedback using the VADER model for teaching process assessment. Economics and Applied Informatics, 30(3), 196–203. https://doi.org/10.35219/eai15840409445
Muntiari, N. R., Hanif, K. H., & Nisa, I. C. (2023). Perbandingan algoritma regresi logistik, Support Vector Machine, dan Gradient Boosting pada analisis sentimen data komentar siswa. Jurnal Ilmu Komputer dan Teknologi, 4(2), 27–32. https://doi.org/10.35960/ikomti.v4i2.1286
Putra, A. N. C., Haviana, S. F. C., & Badie'ah. (2023). Sentimen analisis komentar mahasiswa EDOM dengan metode Support Vector Machine (SVM). Seminar Riset Mahasiswa - Computer & Electrical (SERIMA-CE), 1(1), 260–264.
Sediatmoko, N. S., Nataliani, Y., & Suryady, I. (2024). Sentiment analysis of customer review using classification algorithms and SMOTE for handling imbalanced class. Indonesian Journal of Information Systems, 7(1), 38–52. https://doi.org/10.24002/ijis.v7i1.8879
Sidik, P., Sunarya, I. M. G., & Gunadi, I. G. A. (2025). Comparison of Random Forest and Support Vector Machine methods in sentiment analysis of student satisfaction questionnaire comments at ITB STIKOM Bali. Journal of Applied Informatics and Computing, 9(3), 794–802. https://doi.org/10.30871/jaic.v9i3.9617
Singgalen, Y. A. (2024a). Performance evaluation of SVM with synthetic minority over-sampling technique in sentiment classification. Jurnal Mantik, 8(1), 326–336. https://doi.org/10.35335/mantik.v8i1.5077
Singgalen, Y. A. (2024b). Comparative analysis of DT and SVM model performance with SMOTE in sentiment classification. KLIK: Kajian Ilmiah Informatika dan Komputer, 4(5), 2485–2494. https://doi.org/10.30865/klik.v4i5.1828
Tuan, T. A., Loan, D. T. T., & Kittiphattanabawon, N. (2025). Understanding students' sentiment from feedback with a new feature selection and semantics networks. International Journal of Data and Network Science, 9(1), 253–266. https://doi.org/10.5267/j.ijdns.2024.7.010
Widayanti, R., & Kasih, F. C. (2026). Comparative analysis of baseline IndoBERT, class-weighted IndoBERT, and SMOTE with Support Vector Machine for handling imbalanced sentiment classification in Indonesian. Jurnal Teknik Informatika (JUTIF), 7(3), 2857–2875. https://doi.org/10.52436/1.jutif.2026.7.3.5692
Yang, W., Pan, C., & Zhang, Y. (2022). An oversampling method for imbalanced data based on spatial distribution of minority samples SD-KMSMOTE. Scientific Reports, 12, 16820. https://doi.org/10.1038/s41598-022-21046-1
