Topic-Aware Multi-Class Classification for Financial Complaints: Comparing BERTopic With Classical Machine Learning Algorithms

Abstract

In today's digital world, customers can utilize a variety of communication channels, such as business emails, consumer forms, feedback platforms, and dedicated complaint websites, to communicate their complaints. This study compares the performance of the supervised Bidirectional Encoder Representations from Transformers for Topic Modeling (BERTopic) with traditional machine learning algorithms, including Random Forest (RF), Support Vector Machines (SVM), Logistic Regression (LR), Naive Bayes (NB), K-nearest Neighbors (KNN), and eXtreme Gradient Boosting (XGBoost), for multi-class classification of financial customer complaints. The dataset consists of 16,715 balanced training data and 3,808 test data across five different categories, with the financial complaint data. Experimental results demonstrate that traditional machine learning models, particularly XGBoost, SVM, and LR, achieved the highest classification performance with accuracy rates close to 88%. BERTopic showed a competitive performance with an accuracy of 82.48%. The results suggest that while BERTopic offers interpretability advantages through topic modeling techniques, traditional algorithms provide higher accuracy. This study highlights the promising potential for future financial text analysis and customer complaint classification using hybrid methods, which could lead to more detailed, topic-aware classification approaches. © 2025 IEEE.In today's digital world, customers can utilize a variety of communication channels, such as business emails, consumer forms, feedback platforms, and dedicated complaint websites, to communicate their complaints. This study compares the performance of the supervised Bidirectional Encoder Representations from Transformers for Topic Modeling (BERTopic) with traditional machine learning algorithms, including Random Forest (RF), Support Vector Machines (SVM), Logistic Regression (LR), Naive Bayes (NB), K-nearest Neighbors (KNN), and eXtreme Gradient Boosting (XGBoost), for multi-class classification of financial customer complaints. The dataset consists of 16,715 balanced training data and 3,808 test data across five different categories, with the financial complaint data. Experimental results demonstrate that traditional machine learning models, particularly XGBoost, SVM, and LR, achieved the highest classification performance with accuracy rates close to 88%. BERTopic showed a competitive performance with an accuracy of 82.48%. The results suggest that while BERTopic offers interpretability advantages through topic modeling techniques, traditional algorithms provide higher accuracy. This study highlights the promising potential for future financial text analysis and customer complaint classification using hybrid methods, which could lead to more detailed, topic-aware classification approaches.