HM-HSD: HIERARCHICAL MULTI-TASK DUAL-TRANSFORMER FRAMEWORK FOR HATE SPEECH DETECTION

Abstract

Online social media has facilitated the spread of communication across continents and seas, and yet it has also allowed destructive speech, such as hate speech that results in psychological damage, polarization, hatred, and violence offline. The automation of hate speech detection is crucial because manual moderation does not scale with the volume and variety of user-generated content. However, existing methods remain limited in terms of accuracy and generalization, especially concerning sarcasm, coded Hate, or implicit harmful intent. We present a hierarchical multimodal hate speech detection (HM-HSD) framework featuring dual transformer encoders and auxiliary learning and adaptive optimization mechanisms for enhanced performance. The model uses ELECTRA-small and DistilRoBERTa as complementary encoders to capture different linguistic features, and then uses a fusion module to aggregate semantic signals hierarchically. The approach is multitask, where MSMH hate-speech classification is trained jointly with other self-supervised auxiliary tasks based on the same biLSTM, such as sarcasm prediction, toxicity detection, and coded-hate recognition. A stable adaptive softmax weight sharing scheme that dynamically adjusts the contribution of tasks among training to make better generalisation over different hate expressions and is less prone to over-fitting. We trained and tested our approach on the HateSpeechDatasetBalanced, a dataset of 726,119 annotated text samples with a near-equal class distribution. Experimental results indicate that HM-HSD achieves 94.25% accuracy in general, with strong class-wise performance and stable generalization across different training epochs. When benchmarked against strong transformer-based baselines, such as ELECTRA (89.46%) and DistilBERT (89.05%), our HM-HSD shows 4-5% accuracy gains and provides significant performance improvements against evolving hateful patterns. Our findings confirm the effectiveness and scalability of this multitask learning with adaptive optimization for real-world hate speech moderation systems.