DNN versus CNN for Abusive Audio Classification in YouTube Shorts
Abstract
Audio-based abusive language in short-form videos is difficult to moderate when the spoken content does not appear in captions. This study compares a deep neural network (DNN) trained on a 183-dimensional engineered feature vector with a convolutional neural network (CNN) trained on 64 by 87 Mel-spectrograms for classifying eight Indonesian abusive keywords. From 923 YouTube Shorts, 272 segments were manually curated, and 151 source clips were augmented into 906 samples standardized to 2 seconds at 22,050 Hz. Evaluation used stratified 5-fold cross-validation, a 70:30 stratified holdout, class-wise metrics, and confusion matrices. The DNN achieved a mean cross-validation accuracy of 87.08 percent with a standard deviation of 2.12 percentage points, and a holdout accuracy of 83.82 percent with a macro F1-score of 83.24 percent. The CNN reached a mean cross-validation accuracy of 81.33 percent with a standard deviation of 30.48 percentage points, but only 13.97 percent holdout accuracy and a 4.00 percent macro F1-score, indicating severe instability and class collapse. The engineered-feature DNN was therefore more stable in this small-data setting, although the results do not establish an intrinsic architectural superiority. Limitations include synthetic augmentation, class imbalance, single-rater curation, an isolated eight-word vocabulary, and no transfer-learning or full-video evaluation.
References
Chang, C.-C., & Lin, C.-J. (2011). LIBSVM: A library for support vector machines. ACM Transactions on Intelligent Systems and Technology, 2(3), 27. https://doi.org/10.1145/1961189.1961199
Chollet, F., et al. (2015). Keras [Computer software]. https://keras.io
Cortes, C., & Vapnik, V. (1995). Support-vector networks. Machine Learning, 20(3), 273–297. https://doi.org/10.1007/BF00994018
Davis, S. B., & Mermelstein, P. (1980). Comparison of parametric representations for monosyllabic word recognition in continuously spoken sentences. IEEE Transactions on Acoustics, Speech, and Signal Processing, 28(4), 357–366. https://doi.org/10.1109/TASSP.1980.1163420
Giannakopoulos, T., & Pikrakis, A. (2014). Introduction to audio analysis: A MATLAB approach. Academic Press. https://doi.org/10.1016/C2012-0-03524-7
Glorot, X., Bordes, A., & Bengio, Y. (2011). Deep sparse rectifier neural networks. In Proceedings of the 14th International Conference on Artificial Intelligence and Statistics (AISTATS) (Vol. 15, pp. 315–323). PMLR.
Ibrohim, M. O., & Budi, I. (2023). Hate speech and abusive language detection in Indonesian social media: Progress and challenges. Heliyon, 9(8), e18647. https://doi.org/10.1016/j.heliyon.2023.e18647
Ioffe, S., & Szegedy, C. (2015). Batch normalization: Accelerating deep network training by reducing internal covariate shift. In Proceedings of the 32nd International Conference on Machine Learning (ICML) (Vol. 37, pp. 448–456). PMLR.
Jenkins, W. K., Jones, D. L., & Hunsinger, B. J. (2002). Discrete-time signal processing. In Reference data for engineers (9th ed.). Newnes. https://doi.org/10.1016/B978-075067291-7/50030-3
Kingma, D. P., & Ba, J. (2015). Adam: A method for stochastic optimization. In Proceedings of the 3rd International Conference on Learning Representations (ICLR). https://doi.org/10.48550/arXiv.1412.6980
LeCun, Y., Bengio, Y., & Hinton, G. (2015). Deep learning. Nature, 521(7553), 436–444. https://doi.org/10.1038/nature14539
McFee, B., Raffel, C., Liang, D., Ellis, D. P. W., McVicar, M., Battenberg, E., & Nieto, O. (2015). librosa: Audio and music signal analysis in Python. In Proceedings of the 14th Python in Science Conference (pp. 18–25). https://doi.org/10.25080/Majora-7b98e3ed-003
Pedregosa, F., Varoquaux, G., Gramfort, A., Michel, V., Thirion, B., Grisel, O., Blondel, M., Prettenhofer, P., Weiss, R., Dubourg, V., Vanderplas, J., Passos, A., Cournapeau, D., Brucher, M., Perrot, M., & Duchesnay, É. (2011). Scikit-learn: Machine learning in Python. Journal of Machine Learning Research, 12, 2825–2830.
Piczak, K. J. (2015). Environmental sound classification with convolutional neural networks. In 2015 IEEE 25th International Workshop on Machine Learning for Signal Processing (MLSP). https://doi.org/10.1109/MLSP.2015.7324337
Pratama, E. F., Setyawan, R. A., & Bororing, J. E. (2026). DNN vs CNN for YouTube Shorts abusive word classification. Journal of System and Computer Engineering, 7(3). https://doi.org/10.61628/jsce.v7i3.2759
Purwins, H., Li, B., Virtanen, T., Schlüter, J., Chang, S.-Y., & Sainath, T. (2019). Deep learning for audio signal processing. IEEE Journal of Selected Topics in Signal Processing, 13(2), 206–219. https://doi.org/10.1109/JSTSP.2019.2908700
Salamon, J., & Bello, J. P. (2017). Deep convolutional neural networks and data augmentation for environmental sound classification. IEEE Signal Processing Letters, 24(3), 279–283. https://doi.org/10.1109/LSP.2017.2657381
Salim, C. E. R., & Suhartono, D. (2021). Long short-term memory for hate speech and abusive language detection on Indonesian YouTube comment section. In Proceedings of the 11th International Workshop on Computer Science and Engineering (WCSE). https://doi.org/10.18178/wcse.2021.06.029
Sercu, T., Puhrsch, C., Kingsbury, B., & LeCun, Y. (2016). Very deep multilingual convolutional neural networks for LVCSR. In 2016 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) (pp. 4955–4959). https://doi.org/10.1109/ICASSP.2016.7472620
Shi, T., et al. (2023). Recent advances in plant disease severity assessment using convolutional neural networks. Scientific Reports, 13(1). https://doi.org/10.1038/s41598-023-29230-7
Sokolova, M., & Lapalme, G. (2009). A systematic analysis of performance measures for classification tasks. Information Processing & Management, 45(4), 427–437. https://doi.org/10.1016/j.ipm.2009.03.002
Srivastava, N., Hinton, G., Krizhevsky, A., Sutskever, I., & Salakhutdinov, R. (2014). Dropout: A simple way to prevent neural networks from overfitting. Journal of Machine Learning Research, 15(56), 1929–1958.
Supriyanto, H. A., Setyawan, R. A., & Bororing, J. E. (2026). Automatic detection of toxic content in short videos using deep learning-based text and audio feature integration. Journal of Deep Learning, Computer Vision and Digital Image Processing, 4(2), 221–235. https://doi.org/10.61255/decoding.v4i2.1540
Waseem, Z. (2016). Are you a racist or am I seeing things? Annotator influence on hate speech detection on Twitter. In Proceedings of the First Workshop on NLP and Computational Social Science (pp. 138–142). https://doi.org/10.18653/v1/W16-5618
Wen, J., et al. (2019). How serious is data leakage in deep learning studies on Alzheimer’s disease classification? Annual Meeting of the Organization for Human Brain Mapping (OHBM). https://hal.science/hal-02105133
Zhang, C., Bengio, S., Hardt, M., Recht, B., & Vinyals, O. (2021). Understanding deep learning (still) requires rethinking generalization. Communications of the ACM, 64(3), 107–115. https://doi.org/10.1145/3446776
