Cómo entrenar a un algoritmo para detectar el lenguaje de odio en el conflicto Israel-Palestina

Antonio Rico Sulayes

doi:10.31637/epsir-2025-1199

Authors

Antonio Rico Sulayes Universidad de las Américas Puebla

DOI:

https://doi.org/10.31637/epsir-2025-1199

Keywords:

ideological discourse, hate speech, corpora, digital communication, interrater agreement, sentiment analysis, computational linguistics, Israel-Palestine conflict

Abstract

Introduction: Automatic detection of toxic behavior in digital communication, such as hate speech targeting social groups, is explored in sentiment analysis, a task of natural language processing. Detecting hate speech in the Israel-Palestine conflict is particularly complex because there are both, a hate speech towards and a support speech for each of the two groups involved. Methodology: Since an algorithm needs to learn to recognize this behavior using examples, a corpus was created using comments related to the conflict in social media. A set of rules were also designed for a four-way tagging process. Ideological discourse is characterized by a group that argues they have the truth in opposition to another group whose beliefs they see as an ideology, while the opposite view is held by the other group. Results: This article shows the validity level achieved in the interrater agreement with various polarity levels, including support, hate and neutral. Conclusions: A high level of validity is reached with two opposite polarity levels, but including a level of neutrality increases the complexity and reduces the validity coefficient. At the end, potential applications of the created corpus in the context of security and intelligence are discussed.

Downloads

Download data is not yet available.

Author Biography

Antonio Rico Sulayes, Universidad de las Américas Puebla

D. in Computational Linguistics from Georgetown University, Washington, DC. He has taught linguistics and computation in Mexico, Colombia and the United States. He has also worked as a computational linguist for institutions such as the World Health Organization and for several Pentagon contractors. He is the author of more than 40 articles and 3 research books. He has lectured in a dozen countries and is a founding member of the Mexican Association of Natural Language Processing. He is currently a full-time research professor at the Universidad de las Américas Puebla in Mexico. His research focuses on language technologies, forensic linguistics and lexical studies.

References

Chilton, P. y Schäffner, C. (2011). Discourse, Ethnicity and Racism. En T. A. van Dijk. (ed.) Discourse Studies: A Multidisciplinary Introduction (2a ed.) (pp. 303-330). SAGE. DOI: https://doi.org/10.4135/9781446289068.n15

DATAtab (2024). Cohen's Kappa. https://datatab.net/tutorial/cohens-kappa

Davidson, T., Warmsley, D., Macy, M. y Weber, I. (2017). Automated Hate Speech Detection and the Problem of Offensive Language. Proceedings of the International AAAI Conference on Web and Social Media, 11(1), 512-515. https://doi.org/10.1609/icwsm.v11i1.14955 DOI: https://doi.org/10.1609/icwsm.v11i1.14955

de Paula, A. F. M. y Schlicht, I. B. (2021). AI-UPV at IberLEF-2021 DETOXIS task: Toxicity Detection in Immigration-Related Web News Comments Using Transformers and Statistical Models. Proceedings of the Iberian Languages Evaluation Forum (IberLEF 2021) CEUR-WS, 2943, 547-566. https://doi.org/10.48550/arXiv.2111.04530

exportcomments.com (2024). Export Social Media Comments. https://exportcomments.com/

Fortuna, P. y Nunes, S. (2019). A Survey on Automatic Detection of Hate Speech in Text. ACM Computing Surveys, 51(4), 1-30. https://doi.org/10.1145/3232676 DOI: https://doi.org/10.1145/3232676

Gagliardone, I., Gal, D., Alves, T. y Martinez, G. (2015). Countering online hate speech. United Nations Educational, Scientific and Cultural Organization.

Google API Client (2024). Google API Python client library (version 1.0.290.0). [Software]. https://github.com/googleapis/google-api-python-client

GraphPad (2024). Quantify agreement with kappa. https://www.graphpad.com/quickcalcs/kappa1.cfm

Jiwani, Y. y Richardson, J. E. (2011). Discourse, Ethnicity and Racism. En T. A. van Dijk. (ed.) Discourse Studies: A Multidisciplinary Introduction (2a ed.) (pp. 241-262). SAGE. DOI: https://doi.org/10.4135/9781446289068.n12

Jurafsky, D. y Martin, J. H. (2023). Speech and Language Processing: An Introduction to Language Natural Processing, Computational Linguistics, and Speech Recognition (3a ed.). Available on line: https://web.stanford.edu/~jurafsky/slp3/

Lingiardi, V., Carone, N., Semeraro, G., Musto, C., D’Amico, M. y Brena, S. (2020). Mapping Twitter hate speech towards social and sexual minorities: A lexicon-based approach to semantic content analysis. Behaviour & Information Technology, 39(7), 711-721. https://doi.org/10.1080/0144929X.2019.1607903 DOI: https://doi.org/10.1080/0144929X.2019.1607903

Manning, C. D., Raghavan, P. y Schütze, H. (2008). Introduction to Information Retrieval. Cambridge University Press. DOI: https://doi.org/10.1017/CBO9780511809071

Meta-analysis (2024). Cohen’s kappa free calculator.

https://meta-analysis.actilab.onl/cohen-kappa-free-calculator/#risultati

Nobata, C., Tetreault, J., Thomas, A., Mehdad, Y. y Chang, Y. (2016). Abusive Language Detection in Online User Content. Proceedings of the 25th International Conference on World Wide Web, 145-153. https://doi.org/10.1145/2872427.2883062 DOI: https://doi.org/10.1145/2872427.2883062

Poletto, F., Basile, V., Sanguinetti, M., Bosco, C. y Patti, V. (2021). Resources and benchmark corpora for hate speech detection: A systematic review. Language Resources and Evaluation, 55(2), 477-523. https://doi.org/10.1007/s10579-020-09502-8 DOI: https://doi.org/10.1007/s10579-020-09502-8

Rico-Sulayes, A. (2018). Authorship Attribution on Crime-Related Social Media: Research on the darknet in forensic linguistics. Aracne Editrice.

Rico-Sulayes, A. (2020). General Lexicon-Based Complex Word Identification Extended with Stem N-grams and Morphological Engines. Proceedings of the Iberian Languages Evaluation Forum (IberLEF 2020), Spain, CEUR-WS, 2664, 15-23.

Rico-Sulayes, A. y Monsalve-Pulido, J. (2022). A Proposal and Comparison of Supervised and Unsupervised Classification Techniques for Sentiment Analysis in Tourism Data. Proceedings of the Iberian Languages Evaluation Forum (IberLEF 2022), Spain, CEUR-WS, 3202, 1-8.

Rieder, B. (2015). YouTube Data Tools (Version 1.42) [Software]. https://ytdt.digitalmethods.net/

Spasić, L. W. y Buerki, A. (2020). Idiom-Based Features in Sentiment Analysis: Cutting the Gordian Knot. IEEE Transactions on Affective Computing, 11(2), 189-199. http://dx.doi.org/10.1109/TAFFC.2017.2777842 DOI: https://doi.org/10.1109/TAFFC.2017.2777842

Villasís-Keever, M. Á., Márquez-González, H., Zurita-Cruz, J. N., Miranda-Novales, G. y Escamilla-Núñez, A. (2018). El protocolo de investigación VII. Validez y confiabilidad de las mediciones. Revista Alergia México, 65(4), 414-421. https://doi.org/10.29262/ram.v65i4.560 DOI: https://doi.org/10.29262/ram.v65i4.560

Waseem, Z. (2016). Are You a Racist or Am I Seeing Things? Annotator Influence on Hate Speech Detection on Twitter. Proceedings of the First Workshop on NLP and Computational Social Science, 138-142. https://doi.org/10.18653/v1/W16-5618 DOI: https://doi.org/10.18653/v1/W16-5618

van Dijk, T. A. (2011). Discourse and Ideology. En T. A. van Dijk (ed.), Discourse Studies: A Multidisciplinary Introduction (2a ed.) (pp. 379-407). SAGE. DOI: https://doi.org/10.4135/9781446289068.n18