Binisa Giri , Hashmath Fathima , Kelechi Nwachukwu , Bikesh Regmi and Kofi Nyarko , Morgan State University, USA

Cyber Shield is an automated graph augmented abusive language and interaction detection system designed to identify harmful content including toxic interaction, hate speech, and general negative sentiment that is prevalent on social media platforms. As part of integrating a robust sentiment component into the system, we evaluated four widely used sentiment analysis models: BERT, RoBERTa, VADER, and Text Blob based on their complementary strengths and methodological diversity. BERT and RoBERTa represent string transformers architectures capable of capturing contextual meaning in noisy social media texts. VADER provides a lexicon based model optimized for informal online communication, offering a lightweight alternative to transformers. TextBlob is a traditional NLP baseline to benchmark improvements offered by more contemporary models. Together, this combination allows for a comprehensive comparison across model families, ensuring evidence-based model selection for the Cyber Shield project. These models were evaluated on a Kaggle dataset containing social media comments labeled with three sentiment classes (i.e., negative, positive, and neutral) serving as the ground truth. Each model’s performance was measured using confusion matrices, accuracy, macro F1, weighted F1, and per class F1 scores. Our findings show that with an initial sample of 3000 texts, classical lexicon based models (i.e. VADER) and the traditional NLP baseline model (i.e., Text Blob), significantly outperformed transformer based models. TextBlob achieved the strongest performance results in this phase, underscoring the challenges of applying general pre-trained transformers to real world sentiment classification without domain specific fine tuning. However, after expanding the dataset to 18000+ samples per sentiment class and rerunning the evaluation with the updated RoBERTa sentiment model, the performance trend shifted. The updated RoBERTa model demonstrated substantial improvement and outperformed the earlier transformer results.

Abusive Language Detection, Sentiment Analysis, Transformer Models, Lexicon-based models, Social Media Moderation, Performance Metrics.