Gated Transformer with Dual Attention for Rumour Category Detection on Social Platforms
DOI:
https://doi.org/10.32628/IJSRST2513104Keywords:
Transformer, Rumour Detection, Dual Attention, Gated Recurrent Unit (GRU), Social Media, Text Classification, Interpretability, Fake News,, Discourse AnalysisAbstract
The spread of misinformation and unverified content on social media has become a major concern in the digital age. Accurate categorization of rumours—into support, denial, query, or comment—is essential to assess the credibility and impact of online discourse. In this work, we propose a simplified yet effective deep learning framework that combines a Transformer-based encoder with a dual attention mechanism. The model captures fine-grained word-level semantics and post-level contextual relevance within conversation threads. By integrating gated recurrent units (GRUs) with word and post-level attention, the framework enhances the ability to distinguish rumour types based on linguistic and discourse cues. Experimental evaluations on two benchmark datasets, PHEME and RumourEval-19, demonstrate strong performance in terms of accuracy and F1-score. Furthermore, attention visualizations provide interpretability, making the model’s predictions more transparent and trustworthy.
📊 Article Downloads
References
Zubiaga, A., Liakata, M., & Procter, R. (2018). Detection and resolution of rumours in social media: A survey. ACM Computing Surveys, 51(2), 1–36. DOI: https://doi.org/10.1145/3161603
Shu, K., Sliva, A., Wang, S., Tang, J., & Liu, H. (2017). Fake news detection on social media: A data mining perspective. ACM SIGKDD Explorations, 19(1), 22–36. DOI: https://doi.org/10.1145/3137597.3137600
Zubiaga, A., Aker, A., Bontcheva, K., Liakata, M., & Procter, R. (2018). Detection and resolution of rumours in social media: A survey. Information Processing & Management, 54(2), 146–160. DOI: https://doi.org/10.1016/j.ipm.2017.11.009
Qazvinian, V., Rosengren, E., Radev, D., & Mei, Q. (2011). Rumor has it: Identifying misinformation in microblogs. Proceedings of EMNLP, 1589–1599.
Ma, J., Gao, W., & Wong, K. (2017). Detect rumors in microblog posts using propagation structure via kernel learning. ACL, 708–717. DOI: https://doi.org/10.18653/v1/P17-1066
Ma, J., Gao, W., & Wong, K. (2016). Detecting rumor threads in microblogging networks. AAAI, 101–110.
Liu, X., Nourbakhsh, A., Li, Q., Fang, R., & Shah, S. (2015). Real-time rumor debunking on Twitter. CIKM, 1867–1870. DOI: https://doi.org/10.1145/2806416.2806651
Yang, Z., Yang, D., Dyer, C., He, X., Smola, A., & Hovy, E. (2016). Hierarchical attention networks for document classification. NAACL-HLT, 1480–1489. DOI: https://doi.org/10.18653/v1/N16-1174
Zhou, X., & Zafarani, R. (2020). A survey of fake news: Fundamental theories, detection methods, and opportunities. ACM Computing Surveys, 53(5), 1–40. DOI: https://doi.org/10.1145/3395046
Mikolov, T., Chen, K., Corrado, G., & Dean, J. (2013). Efficient estimation of word representations in vector space. arXiv:1301.3781.
Devlin, J., Chang, M. W., Lee, K., & Toutanova, K. (2019). BERT: Pre-training of deep bidirectional transformers for language understanding. NAACL-HLT, 4171–4186.
Liu, Y., Ott, M., Goyal, N., Du, J., Joshi, M., Chen, D., … & Stoyanov, V. (2019). RoBERTa: A robustly optimized BERT pretraining approach. arXiv:1907.11692.
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., … & Polosukhin, I. (2017). Attention is all you need. NeurIPS, 5998–6008.
Lin, Z., Feng, M., Santos, C. N. D., Yu, M., Xiang, B., Zhou, B., & Bengio, Y. (2017). A structured self-attentive sentence embedding. ICLR.
Kumar, S., Asthana, R., Upadhyay, S., Akhtar, M. S., & Ekbal, A. (2020). Synergistic multiple attention network for rumour verification on social media. Information Processing & Management, 57(2), 102–128.
Wu, L., Rao, Y., Yang, Y., & Yu, H. (2015). False rumor detection on Sina Weibo by propagation structures. ICDE, 651–662. DOI: https://doi.org/10.1109/ICDE.2015.7113322
Khattar, D., Goud, J. S., Gupta, M., & Varma, V. (2019). Multimodal deep learning for fake news detection. WWW, 2915–2921. DOI: https://doi.org/10.1145/3308558.3313552
Zhang, Q., Zhang, J., Dong, Y., & Philip, S. Y. (2019). Fake news detection with deep diffusive network model. IJCAI, 3812–3818.
Hochreiter, S., & Schmidhuber, J. (1997). Long short-term memory. Neural Computation, 9(8), 1735–1780. DOI: https://doi.org/10.1162/neco.1997.9.8.1735
Cho, K., Van Merriënboer, B., Gulcehre, C., Bahdanau, D., Bougares, F., Schwenk, H., & Bengio, Y. (2014). Learning phrase representations using RNN encoder–decoder for statistical machine translation. EMNLP, 1724–1734. DOI: https://doi.org/10.3115/v1/D14-1179
Ma, J., Gao, W., Mitra, P., Kwon, S., Jansen, B. J., Wong, K. F., & Cha, M. (2016). Detecting rumors from microblogs with recurrent neural networks. IJCAI, 3818–3824.
Bian, T., Xiao, X., Xu, T., Zhao, P., Huang, W., Rong, Y., … & Huang, J. (2020). Rumor detection on social media with bi-directional graph convolutional networks. AAAI, 549–556. DOI: https://doi.org/10.1609/aaai.v34i01.5393
Yuan, C., Ma, Q., Zhou, W., Han, J., Hu, S., & Hu, X. (2019). Jointly embedding the local and global relations of heterogeneous graph for rumor detection. ICDM, 796–805. DOI: https://doi.org/10.1109/ICDM.2019.00090
Gorrell, G., Kochkina, E., Liakata, M., Aker, A., Zubiaga, A., & Bontcheva, K. (2019). SemEval-2019 task 7: RumourEval. SemEval, 845–854.
Zubiaga, A., Liakata, M., Procter, R., Hoi, G. W. S., & Tolmie, P. (2016). Analysing how people orient to and spread rumours in social media by looking at conversational threads. PLoS ONE, 11(3), e0150989. DOI: https://doi.org/10.1371/journal.pone.0150989
Downloads
Published
Issue
Section
License
Copyright (c) 2025 International Journal of Scientific Research in Science and Technology
This work is licensed under a Creative Commons Attribution 4.0 International License.
https://creativecommons.org/licenses/by/4.0
