Appraisal-Based Audit of Ideological Bias in Large Language Models Across Politics, Gender, Culture, and Technology
DOI: https://doi.org/10.65840/jllcd.v3i1.37
generative artificial intelligence, Appraisal Theory, ideological bias, discourse analysis, critical AI literacy
Abstract
This study examines how large language models construct evaluative meanings in response to political, gender, cultural, and technological issues. Using a qualitative-descriptive approach based on content analysis and Appraisal Theory frameworks, this study analyzed the outputs of ChatGPT, Gemini, and Claude to identify the patterns of Affect, Judgment, and Graduation in the framing of discourse. The results show that the AI response is not neutral, but contains systematic ideological biases. The emerging pattern shows a tendency to legitimize the liberal-Western paradigm through the strengthening of individual autonomy, criticism of state authority, and moral judgments that favor certain perspectives, while the Global South view is often given a more problematic framing of the Global South. In the gender realm, AI still reproduces a double bind to female leadership; in the cultural realm, AI tends to highlight anxiety over homogenization; and in the technological realm, AI displays strong epistemic insecurities. These findings confirm that AI output needs to be understood as a discursive practice that helps shape the legitimacy of meaning, so biased audits, algorithmic transparency, and critical AI literacy are needed in the use of generative systems.
References
Alghazo, S., Rabab'ah, G., Salam El-Dakhs, D. A., & Mustafa, A. (2025). Engagement strategies in human-written and AI-generated academic essays: A corpus-based study. Ampersand, 15, 100237. https://doi.org/10.1016/j.amper.2025.100237
Amoore, L., Campolo, A., Jacobsen, B., & Rella, L. (2024). A world model: On the political logics of generative AI. Political Geography, 113, 103134. https://doi.org/10.1016/j.polgeo.2024.103134
Anders, A. D., & Dux Speltz, E. (2025). Developing generative AI literacies through self-regulated learning: A human-centered approach. Computers and Education: Artificial Intelligence, 9, 100482. https://doi.org/10.1016/j.caeai.2025.100482
Archambault, S. G., Ramachandran, S., Acosta, E., & Fu, S. (2024). Ethical dimensions of algorithmic literacy for college students: Case studies and cross-disciplinary connections. The Journal of Academic Librarianship, 50(3), 102865. https://doi.org/10.1016/j.acalib.2024.102865
Bazazi, S., Karpus, J., & Yasseri, T. (2025). AI's assigned gender affects human-AI cooperation. IScience, 28(12), 113905. https://doi.org/10.1016/j.isci.2025.113905
Bengtsson, M. (2016). How to plan and perform a qualitative study using content analysis. NursingPlus Open, 2, 8–14. https://doi.org/10.1016/j.npls.2016.01.001
Berber Sardinha, T. (2024). AI-generated vs human-authored texts: A multidimensional comparison. Applied Corpus Linguistics, 4(1), 100083. https://doi.org/10.1016/j.acorp.2023.100083
Chen, B., Zhang, Z., Langrené, N., & Zhu, S. (2025). Unleashing the potential of prompt engineering for large language models. Patterns, 6(6), 101260. https://doi.org/10.1016/j.patter.2025.101260
Chiu, T. K. F., Ahmad, Z., Ismailov, M., & Sanusi, I. T. (2024). What are artificial intelligence literacy and competency? A comprehensive framework to support them. Computers and Education Open, 6, 100171. https://doi.org/10.1016/j.caeo.2024.100171
Chuan, C.-H., Sun, R., Tian, S., & Tsai, W.-H. S. (2024). EXplainable Artificial Intelligence (XAI) for facilitating recognition of algorithmic bias: An experiment from imposed users' perspectives. Telematics and Informatics, 91, 102135. https://doi.org/10.1016/j.tele.2024.102135
Currie, G., Hewis, J., & Ebbs, P. (2025). Gender bias in text-to-image generative artificial intelligence depiction of Australian paramedics and first responders. Australasian Emergency Care, 28(2), 103–109. https://doi.org/10.1016/j.auec.2024.11.003
Erlingsson, C., & Brysiewicz, P. (2017). A hands-on guide to doing content analysis. African Journal of Emergency Medicine, 7(3), 93–99. https://doi.org/10.1016/j.afjem.2017.08.001
Francisco, M., & Linnér, B.-O. (2023). AI and the governance of sustainable development. An idea analysis of the European Union, the United Nations, and the World Economic Forum. Environmental Science & Policy, 150, 103590. https://doi.org/10.1016/j.envsci.2023.103590
Hashemi, A., & Mahdavirad, F. (2023). A cross-cultural, cross-disciplinary, and cross-gender study on Appraisal resources in PhD dissertation abstracts: Martin & White's (2005) Appraisal Theory in focus. Heliyon, 9(11), e22074. https://doi.org/10.1016/j.heliyon.2023.e22074
Ho, J. Q. H., Hartanto, A., Koh, A., & Majeed, N. M. (2025). Gender biases within Artificial Intelligence and ChatGPT: Evidence, Sources of Biases and Solutions. Computers in Human Behavior: Artificial Humans, 4, 100145. https://doi.org/10.1016/j.chbah.2025.100145
Jiang, F. (Kevin), & Hyland, K. (2025). Rhetorical distinctions: Comparing metadiscourse in essays by ChatGPT and students. English for Specific Purposes, 79, 17–29. https://doi.org/10.1016/j.esp.2025.03.001
Kasapoglu, T., Masso, A., & Calzati, S. (2021). Unpacking algorithms as technologies of power: Syrian refugees and data experts on algorithmic governance. Digital Geography and Society, 2, 100016. https://doi.org/10.1016/j.diggeo.2021.100016
Kashiha, H. (2025). From algorithms to annotations: Rethinking feedback practices in academic writing through AI-human comparison. Journal of Second Language Writing, 70, 101254. https://doi.org/10.1016/j.jslw.2025.101254
Khalifa, M., & Albadawy, M. (2024). Using artificial intelligence in academic writing and research: An essential productivity tool. Computer Methods and Programs in Biomedicine Update, 5, 100145. https://doi.org/10.1016/j.cmpbup.2024.100145
Knoth, N., Tolzin, A., Janson, A., & Leimeister, J. M. (2024). AI literacy and its implications for prompt engineering strategies. Computers and Education: Artificial Intelligence, 6, 100225. https://doi.org/10.1016/j.caeai.2024.100225
Kong, X., & Liu, C. (2024). A comparative genre analysis of AI-generated and scholar-written abstracts for English review articles in international journals. Journal of English for Academic Purposes, 71, 101432. https://doi.org/10.1016/j.jeap.2024.101432
Lim, S., & Schmälzle, R. (2024). The effect of source disclosure on evaluation of AI-generated messages. Computers in Human Behavior: Artificial Humans, 2(1), 100058. https://doi.org/10.1016/j.chbah.2024.100058
Liu, N. T. Y., Kirshner, S. N., & Lim, E. T. K. (2023). Is algorithm aversion WEIRD? A cross-country comparison of individual-differences and algorithm aversion. Journal of Retailing and Consumer Services, 72, 103259. https://doi.org/10.1016/j.jretconser.2023.103259
Longo, L., Brcic, M., Cabitza, F., Choi, J., Confalonieri, R., Ser, J. Del, Guidotti, R., Hayashi, Y., Herrera, F., Holzinger, A., Jiang, R., Khosravi, H., Lecue, F., Malgieri, G., Páez, A., Samek, W., Schneider, J., Speith, T., & Stumpf, S. (2024). Explainable Artificial Intelligence (XAI) 2.0: A manifesto of open challenges and interdisciplinary research directions. Information Fusion, 106, 102301. https://doi.org/10.1016/j.inffus.2024.102301
Ma, M., Ng, D. T. K., Liu, Z., & Wong, G. K. W. (2025). Fostering responsible AI literacy: A systematic review of K-12 AI ethics education. Computers and Education: Artificial Intelligence, 8, 100422. https://doi.org/10.1016/j.caeai.2025.100422
Mahmud, H., Islam, A. K. M. N., Ahmed, S. I., & Smolander, K. (2022). What influences algorithmic decision-making? A systematic literature review on algorithm aversion. Technological Forecasting and Social Change, 175, 121390. https://doi.org/10.1016/j.techfore.2021.121390
Messer, U. (2025). How do people react to political bias in generative artificial intelligence (AI)? Computers in Human Behavior: Artificial Humans, 3, 100108. https://doi.org/10.1016/j.chbah.2024.100108
Monteiro, N., & Singh, V. (2025). The wheel of artificial intelligence governance. Sustainable Futures, 10, 101279. https://doi.org/10.1016/j.sftr.2025.101279
Moorhouse, B. L., Wan, Y., Wu, C., Wu, M., & Ho, T. Y. (2025). Generative AI tools and empowerment in L2 academic writing. System, 133, 103779. https://doi.org/10.1016/j.system.2025.103779
Motoki, F. Y. S., Pinho Neto, V., & Rangel, V. (2025). Assessing political bias and value misalignment in generative artificial intelligence. Journal of Economic Behavior & Organization, 234, 106904. https://doi.org/10.1016/j.jebo.2025.106904
Newstead, T., Eager, B., & Wilson, S. (2023). How AI can perpetuate – Or help mitigate – Gender bias in leadership. Organizational Dynamics, 52(4), 100998. https://doi.org/10.1016/j.orgdyn.2023.100998
Ng, D. T. K., Leung, J. K. L., Chu, S. K. W., & Qiao, M. S. (2021). Conceptualizing AI literacy: An exploratory review. Computers and Education: Artificial Intelligence, 2, 100041. https://doi.org/10.1016/j.caeai.2021.100041
Park, J. (2025). A systematic literature review of generative artificial intelligence (GenAI) literacy in schools. Computers and Education: Artificial Intelligence, 9, 100487. https://doi.org/10.1016/j.caeai.2025.100487
Pinski, M., & Benlian, A. (2024). AI literacy for users – A comprehensive review and future research directions of learning methods, components, and effects. Computers in Human Behavior: Artificial Humans, 2(1), 100062. https://doi.org/10.1016/j.chbah.2024.100062
Schilke, O., & Reimann, M. (2025). The transparency dilemma: How AI disclosure erodes trust. Organizational Behavior and Human Decision Processes, 188, 104405. https://doi.org/10.1016/j.obhdp.2025.104405
Stolpe, K., & Hallström, J. (2024). Artificial intelligence literacy for technology education. Computers and Education Open, 6, 100159. https://doi.org/10.1016/j.caeo.2024.100159
Su, J., Ng, D. T. K., & Chu, S. K. W. (2023). Artificial Intelligence (AI) Literacy in Early Childhood Education: The Challenges and Opportunities. Computers and Education: Artificial Intelligence, 4, 100124. https://doi.org/10.1016/j.caeai.2023.100124
Veldhuis, A., Lo, P. Y., Kenny, S., & Antle, A. N. (2025). Critical Artificial Intelligence literacy: A scoping review and framework synthesis. International Journal of Child-Computer Interaction, 43, 100708. https://doi.org/10.1016/j.ijcci.2024.100708
Wei, X., Kumar, N., & Zhang, H. (2025). Addressing bias in generative AI: Challenges and research opportunities in information management. Information & Management, 62(2), 104103. https://doi.org/10.1016/j.im.2025.104103
White, P. R. (2025). Appraisal theory and the analysis of point of view in news and views journalism – unpacking journalistic “persuasiveness.” Language & Communication, 100, 95–107. https://doi.org/10.1016/j.langcom.2024.11.005
Whitehead, M., & Collier, W. G. A. (2023). Incidental governmentality: Big tech and the hidden rationalities of government. Digital Geography and Society, 5, 100071. https://doi.org/10.1016/j.diggeo.2023.100071
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2026 Ebenezer Agbaglo, Emmanuel Mensah Bonsu
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
Journal of Language, Literature, and Cultural Dynamics is licensed under a 