Algorithmic Justice in Education Through De-Biasing: Towards Politically Actionable Evidence That is Rooted in Identity Theory and De-Colonial Thought
Main Article Content
Abstract
As Artificial Intelligence (AI) algorithms are increasingly used in education, research shows that the use of these algorithms is not without cost. Instead, AI algorithms are prone to biases which are discussed a lot in various domains. The core strength of this contribution is to anchor the discussion of biases in the specific domain of physics education and to discuss the biases in front of a description of the domain-specific inequalities along physics identity development of students. The database consists of the written answers of 527 students to around 30 items from a five-week-period of physics classes in a digital learning environment. Two concrete biases of AI algorithms in physics education and possible approaches to identify and reduce these biases are investigated quantitatively. In a critical discussion from a feminist and de-colonial perspective, it is highlighted that the chosen approaches seem to have promising potentials to mitigate negative effects on under-served students´ physics identity development. Besides, relevant limitations lead to conclusions that additional counter-measures are needed in order to break out of the vicious cycle of reproduction of historically grown inequalities in physics education. The domain-specific analysis can serve as orientation for other domains as well in order to tackle the challenges of AI algorithmic bias effectively and efficiently.
Article Details
FLR adopts the Attribution-NonCommercial-NoDerivs Creative Common License (BY-NC-ND). That is, Copyright for articles published in this journal is retained by the authors with, however, first publication rights granted to the journal. By virtue of their appearance in this open access journal, articles are free to use, with proper attribution, in educational and other non-commercial settings.
References
Aikins, M. A., Bremberger, T., Aikins, J. K., Gyamerah, D., & Yıldırım-Caliman, D. (2021). Afrozensus 2020: Perspektiven, Anti-Schwarze Rassismuserfahrungen und Engagement Schwarzer, afrikanischer und afrodiasporischer Menschen in Deutschland. https://afrozensus.de
Archer, L., Dawson, E., DeWitt, J., Seakins, A., & Wong, B. (2015). ‘“Science Capital”’: A Conceptual, Methodological, and Empirical Argument for Extending Bourdieusian Notions of Capital Beyond the Arts. Journal of Research in Science Teaching, 52(7), 992–948. https://doi.org/10.1002/tea.21227
Avraamidou, L. (2019). “I am a young immigrant woman doing physics and on top of that I am Muslim”: Identities, intersections, and negotiations. Journal of Research in Science Teaching, 57, 311–341. https:/doi.org/10.1002/tea.21593
Bachsleitner, A., Lämmchen, R., & Maaz, K. (Eds). (2022). Soziale Ungleichheit des Bildungserwerbs von der Vorschule bis zur Hochschule: Eine Forschungssynthese zwei Jahrzehnte nach PISA. Waxmann. https://doi.org/10.31244/9783830996248
Bagga-Gupta, S. (2025). Imaginings and Re-imaginings in Doing Multiversal Science. Editor’s Introduction. In S. Bagga-Gupta (Ed.), The Palgrave Handbook of Decolonising the Educational and Language Sciences (pp. 1–22). Springer Nature Switzerland. https://doi.org/10.1007/978-3-031-80322-2_1
Baker, R., & Hawn, A. (2021). Algorithmic Bias in Education. https://doi.org/10.1007/s40593-021-00285-9
Bergmann, U., Bonefeld-Dahl, C., Dignum, V., Gagné, J.-F., Metzinger, T., Petit, N., Steinacker, S., Van Wynsberghe, A., & Yeung, K. (Eds). (2019). Ethics Guidelines for Trustworthy AI. European Commission - High-Level Expert Group on Artificial Intelligence. https://digital-strategy.ec.europa.eu/en/library/ethics-guidelines-trustworthy-ai
Bodnar, K., Hofkens, T., Wang, M.-T., & Schunn, C. (2020). Science Identity Predicts Science Career Aspiration Across Gender and Race, but Especially for Boys. International Journal of Gender, Science and Technology, 12(1), 32–45.
Bolukbasi, T., Chang, K.-W., Zou, J., Saligrama, V., & Kalai, A. (2016). Man is to computer programmer as woman is to homemaker? Debiasing word embeddings. Proceedings of the 30th International Conference on Neural Information Processing Systems, 4356–4364. https://dl.acm.org/doi/10.5555/3157382.3157584
Calabrese Barton, A., Kang, H., Tan, E., O’Neill, T. B., Bautista-Guerra, J., & Brecklin, C. (2013). Crafting a Future in Science: Tracing Middle School Girls’ Identity Work Over Time and Space. American Educational Research Journal, 50(1), 37–75. https://doi.org/10.3102/0002831212458142
Carlone, H. B., & Johnson, A. (2007). Understanding the Science Experiences of Successful Women of Color: Science Identity as an Analytic Lens. Journal of Research in Science Teaching, 44(8), 1187–1218. https://doi.org/10.1002/tea.20237
Cass, C. A. P., Hazari, Z., Cribbs, J., Sadler, P. M., & Sonnert, G. (2011). Examining the impact of mathematics identity on the choice of engineering careers for male and female students. 2011 Frontiers in Education Conference (FIE), F2H-1-F2H-5. https://doi.org/10.1109/FIE.2011.6142881
Cerratto Pargman, T., & McGrath, C. (2021). Mapping the Ethics of Learning Analytics in Higher Education: A Systematic Literature Review of Empirical Research. Journal of Learning Analytics, 8(2), 123–139. https://doi.org/10.18608/jla.2021.1
Cerratto Pargman, T., McGrath, C., Viberg, O., Kitto, K., Knight, S., & Ferguson, R. (2021). Responsible Learning Analytics: Creating just, ethical, and caring LA systems. Companion Proceedings. LAK21. https://www.solaresearch.org/wp-content/uploads/2021/04/LAK21_CompanionProceedings.pdf
Chan, B., Schweter, S., & Möller, T. (2020). German’s Next Language Model. Proceedings of the 28th International Conference on Computational Linguistics, 6788–6796. https://doi.org/10.18653/v1/2020.coling-main.598
Chan, S. (2023). My decoloniality is not your decoloniality: The new multiverse – an opinion piece. Social Dynamics, 49(2), 369–375. https://doi.org/10.1080/02533952.2023.2240151
Cheuk, T. (2021). Can AI be racist? Color‐evasiveness in the application of machine learning to science assessments. Science Education, 1–12. https:/doi.org/10.1002/sce.21671
Çolakoğlu, J., Steegh, A., & Parchmann, I. (2023). Reimagining informal STEM learning opportunities to foster STEM identity development in underserved learners. Frontiers in Education, 8, 1–16. https://doi.org/10.3389/feduc.2023.1082747
Collins, P. H. (1990). Black Feminist Thought: Knowledge, Consciousness and the Politics of Empowerment. https:/doi.org/10.4324/9780203900055
Costanza-Chock, S. (2020). Design justice: Community-led practices to build the worlds we need. The MIT Press. https://doi.org/10.7551/mitpress/12255.001.0001
Crenshaw, K. (1989). Demarginalizing the Intersection of Race and Sex: A Black Feminist Critique of Antidiscrimination Doctrine, Feminist Theory and Antiracist Politics. University of Chicago Legal Forum, 1989(8), 139–167.
Dennis, M., Masthoff, J., & Mellish, C. (2016). Adapting Progress Feedback and Emotional Support to Learner Personality. International Journal of Artificial Intelligence in Education, 26(3), 877–931. https://doi.org/10.1007/s40593-015-0059-7
Diakopoulus, N., Friedler, S., Arenas, M., Barocas, S., Hay, M., Howe, B., Jagadish, H. V., Unsworth, K., Sahuguet, A., Venkatasubramanian, S., Wilson, C., Yu, C., & Zevenbergen, B. (2021). Principles for Accountable Algorithms and a Social Impact Statement for Algorithms. FAT/ML. https://www.fatml.org/resources/principles-for-accountable-algorithms
Doroudi, S., & Brunskill, E. (2019). Fairer but Not Fair Enough On the Equitability of Knowledge Tracing. Proceedings of the 9th International Conference on Learning Analytics & Knowledge, 335–339. https://doi.org/10.1145/3303772.3303838
Dou, R., Hazari, Z., Dabney, K., Sonnert, G., & Sadler, P. (2019). Early informal STEM experiences and STEM identity: The importance of talking science. Science Education, 103, 623–637. https://doi.org/10.1002/sce.21499
Dressel, J., & Farid, H. (2018). The accuracy, fairness, and limits of predicting recidivism. Science Advances, 4(1), eaao5580. https://doi.org/10.1126/sciadv.aao5580
Düchs, G., & Ingold, G.-L. (2018). Frauenanteil bleibt stabil. Physik Journal, 17(8/9), 32–37.
El-Mafaalani, A. (2021). Mythos Bildung (2nd edn). Kiepenheuer & Witsch (KiWi).
Erden, D. (2020). KI und Beschäftigung: Das Ende menschlicher Vorurteile oder der Beginn von Diskriminierung 2.0? In Wenn KI, dann feministisch (pp. 77–90). netzforma* eV. https://netzforma.org/publikation-wenn-ki-dann-feministisch-impulse-aus-wissenschaft-und-aktivismus
Escobar, A. (2017). Designs for the Pluriverse: Radical Interdependence, Autonomy, and the Making of Worlds. Duke University Press. http://www.jstor.org/stable/j.ctv11smgs6
Fletcher, R. R., Nakeshimana, A., & Olubeko, O. (2021). Addressing Fairness, Bias, and Appropriate Use of Artificial Intelligence and Machine Learning in Global Health. Frontiers in Artificial Intelligence, 3, 561802. https://doi.org/10.3389/frai.2020.561802
Floridi, L., Cowls, J., Beltrametti, M., Chatila, R., Chazerand, P., Dignum, V., Luetge, C., Madelin, R., Pagallo, U., Rossi, F., Schafer, B., Valcke, P., & Vayena, E. (2018). AI4People—An Ethical Framework for a Good AI Society: Opportunities, Risks, Principles, and Recommendations. Minds & Machines, 28(4), 689–707. https://doi.org/10.1007/s11023-018-9482-5
Folayan, D. (2025). From Colonial Legacies to Decolonial Futures: Explorations of Coloniality Through Oxbridge and Lagdan. In S. Bagga-Gupta (Ed.), The Palgrave Handbook of Decolonising the Educational and Language Sciences (pp. 665–699). Springer Nature Switzerland. https://doi.org/10.1007/978-3-031-80322-2_24
Freire, P. (1970). Pedagogy of the Oppressed. Penguin Random House UK.
Gago, V. (2019). La potencia feminista. O el deseo de cambiarlo todo. Traficantes de Sueños. https://traficantes.net/sites/default/files/pdfs/TDS_map55_La%20potencia%20feminista_web.pdf
García Segura, A. (2021). Se’ dör stè̈—Somos arte: Las enseñanzas del awá—We are art: The teachings of awá. International Tree Fund.
Gardner, J., Brooks, C., & Baker, R. (2019). Evaluating the Fairness of Predictive Student Models Through Slicing Analysis. LAK19: Proceedings of the 9th International Conference on Learning Analytics & Knowledge, 225–234. https://doi.org/https://doi.org/10.1145/3303772.3303791
Gee, J. P. (2000). Identity as an Analytic Lens for Research in Education. Review of Research in Education, 25, 99. https://doi.org/10.2307/1167322
Gombert, S., Di Mitri, D., Karademir, O., Kubsch, M., Kolbe, H., Tautz, S., Grimm, A., Bohm, I., Neumann, K., & Drachsler, H. (2022). Coding energy knowledge in constructed responses with explainable NLP models. Journal of Computer Assisted Learning, jcal.12767. https:/doi.org/10.1111/jcal.12767
Gómez Torres, J. (2023). La educación indígena y las trampas discursivas de la modernidad: Los casos de Guatemala y Costa Rica (Primera edición). EDUPUC, Editoriales Universitarias Públicas Costarricenses.
Grimm, A., Bohm, I., Karademir, O., Gombert, S., Kubsch, M., Di Mitri, D., Borgards, L., Strauß, S., Drachsler, H., Neumann, K., & Rummel, N. (2025). Learning Progression Analytics—Analysing and Promoting Learning Progressions to Develop Skills (LPA-AFLEK) (Version 1.0.0) [Data set]. GESIS. https://doi.org/10.4232/1.14414
Grimm, A., Steegh, A., Çolakoğlu, J., Kubsch, M., & Neumann, K. (2023). Positioning responsible learning analytics in the context of STEM identities of under-served students. Frontiers in Education, 7. https://doi.org/https://doi.org/10.3389/feduc.2022.1082748
Grimm, A., Steegh, A., Kubsch, M., & Neumann, K. (2023). Learning Analytics in Physics Education: Equity- Focused Decision-Making Lacks Guidance! Journal of Learning Analytics, 10(1), 71–84. https://doi.org/10.18608/jla.2023.7793
Guo, W., & Caliskan, A. (2021). Detecting Emergent Intersectional Biases: Contextualized Word Embeddings Contain a Distribution of Human-like Biases. Proceedings of the 2021 AAAI/ACM Conference on AI, Ethics, and Society, 122–133. https://doi.org/10.1145/3461702.3462536
Hazari, Z., Chari, D., Potvin, G., & Brewe, E. (2020). The context dependence of physics identity: Examining the role of performance/competence, recognition, interest, and sense of belonging for lower and upper female physics undergraduates. Journal of Research in Science Teaching, 57(10), 1583–1607. https://doi.org/10.1002/tea.21644
Hazari, Z., Sonnert, G., Sadler, P., & Shanahan, M. (2010). Connecting high school physics experiences, outcome expectations, physics identity, and physics career choice: A gender study. Journal of Research in Science Teaching, 47(8), 978–1003. https://doi.org/10.1002/tea.20363
hooks, bell. (2003). Teaching Community—A Pedagogy of Hope. Routledge. https://doi.org/10.4324/9780203957769
Hooks, B., Truth, S., Davis, A. Y., The Combahee River Collective, Smith, B., Lorde, A., Hill Collins, P., & Crenshaw, K. (2022). Schwarzer Feminismus: Grundlagentexte (N. A. Kelly, Ed.; 2. Aufl). Unrast Verlag.
Hutchinson, B., Prabhakaran, V., Denton, E., Webster, K., Zhong, Y., & Denuyl, S. (2020). Social Biases in NLP Models as Barriers for Persons with Disabilities. Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, 5491–5501. https://doi.org/10.18653/v1/2020.acl-main.487
Jara Murillo, C. V., & García Segura, A. (2022). Së́bliwak Francisco García ttö̀ o Las palabras de Francisco García. Universidad de Costa Rica. https://dibo.site/2024/03/21/sebliwak-francisco-garcia-tto/
Karademir, O., Di Mitri, D., Schneider, J., Jivet, I., Allmang, J., Gombert, S., Kubsch, M., Neumann, K., & Drachsler, H. (2024). I don’t have time! But keep me in the loop: Co‐designing requirements for a learning analytics cockpit with teachers. Journal of Computer Assisted Learning, 40(6), 2681–2699. https://doi.org/10.1111/jcal.12997
Kayumova, S., & Dou, R. (2022). Equity and justice in science education: Toward a pluriverse of multiple identities and onto‐epistemologies. Science Education, 106, 1097–1117. https://doi.org/10.1002/sce.21750
Kitto, K., & Knight, S. (2019). Practical ethics for building learning analytics. British Journal of Educational Technology, 50(6), 2855–2870. https://doi.org/10.1111/bjet.12868
Kordzadeh, N., & Ghasemaghaei, M. (2022). Algorithmic bias: Review, synthesis, and future research directions. European Journal of Information Systems, 31(3), 388–409. https://doi.org/10.1080/0960085X.2021.1927212
Ladewig, A., Keller, M., & Klusmann, U. (2020). Sense of Belonging as an Important Factor in the Pursuit of Physics: Does It Also Matter for Female Participants of the German Physics Olympiad? Frontiers in Psychology, 11, 2685. https://doi.org/10.3389/fpsyg.2020.548781
Latif, E., Zhai, X., & Liu, L. (2023). AI Gender Bias, Disparities, and Fairness: Does Training Data Matter? arXiv. https://doi.org/10.48550/arXiv.2312.10833
Li, L., Sha, L., Li, Y., Raković, M., Rong, J., Joksimovic, S., Neil, S., Gašević, D., & Chen, G. (2023). Moral Machines or Tyranny of the Majority? A Systematic Review on Predictive Bias in Education. LAK23: 13th International Learning Analytics and Knowledge Conference, 499–508. https://doi.org/10.1145/3576050.3576119
Li, W., Brooks, C., & Schaub, F. (2019). The Impact of Student Opt-Out on Educational Predictive Models. Proceedings of the 9th International Conference on Learning Analytics & Knowledge, 411–420. https://doi.org/10.1145/3303772.3303809
Lohaus, M., Perrot, M., & von Luxburg, U. (2020). Too Relaxed to Be Fair. Proceedings of Machine Learning Research, 119, 6360--6369. https://proceedings.mlr.press/v119/lohaus20a.html
Maaz, K., Artelt, C., Brugger, P., Buchholz, S., Kühne, S., Leerhoff, H., Rauschenbach, T., Schrader, J., & Seeber, S. (2022). Bildung in Deutschland 2022. Autor:innengruppe Bildungs berichterstattung. https://doi.org/10.3278/6001820hw
Mignolo, W. D. (2007). Delinking. The rhetoric of modernity, the logic of coloniality and the grammar of de-coloniality. Taylor & Francis Online, 21(2–3), 449–514. https://doi.org/10.1080/09502380601162647
Mitchell, S., Potash, E., D´Amour, A., & Lum, K. (2021). Algorithmic Fairness: Choices, Assumptions, and Definitions. Annual Review of Statistics and Its Application, 8, 141–163. https://doi.org/10.1146/annurev-statistics-042720-125902
Ngũgĩ wa Thiong’o. (2011). Decolonising the mind: The politics of language in African literature (Reprinted). Currey [u.a.].
OECD. (2016). Excellence and equity in education (Volume I; PISA 2015 Results). OECD.
OECD. (2024). Education at a Glance 2024: OECD Indicators. OECD Publishing. https://doi.org/10.1787/c00cad36-en
Ogette, T. (2019). Exit racism (5th edn). unrast-Verlag.
Pardo, A., Jovanovic, J., Dawson, S., Gašević, D., & Mirriahi, N. (2019). Using learning analytics to scale the provision of personalised feedback. British Journal of Educational Technology, 50(1), 128–138. https://doi.org/10.1111/bjet.12592
Perry, M., & Bradley, L. (2025). Pluriversal Literacies: Perspectives and Practices for Sustainable and Anticolonial Futures. In S. Bagga-Gupta (Ed.), The Palgrave Handbook of Decolonising the Educational and Language Sciences (pp. 585–612). Springer Nature Switzerland. https://doi.org/10.1007/978-3-031-80322-2_21
Phillips, P. J., Hahn, C. A., Fontana, P. C., Broniatowski, D. A., & Przybocki, M. A. (2020). Four Principles of Explainable Artificial Intelligence. National Institute of Standards and Technology. https://doi.org/10.6028/NIST.IR.8312-draft
Pinson, H., Feniger, Y., & Barak, Y. (2020). Explaining a reverse gender gap in advanced physics and computer science course‐taking: An exploratory case study comparing Hebrew‐speaking and Arabic‐speaking high schools in Israel. Journal of Research in Science Teaching, 57(8), 1177–1198. https://doi.org/10.1002/tea.21622
Prinsloo, P., & Kaliisa, R. (2022). Learning Analytics on the African Continent: An Emerging Research Focus and Practice. Journal of Learning Analytics, 1–18. https://doi.org/10.18608/jla.2022.7539
Prinsloo, P., & Slade, S. (2017). Chapter 4: Ethics and Learning Analytics: Charting the (Un)Charted. In Handbook of Learning Analytics (1st edn, pp. 49–57). SoLAR. https://doi.org/10.18608/hla17.004
Prinsloo, P., & Slade, S. (2018). Mapping responsible learning analytics: A critical proposal. In Responsible Analytics & Data Mining in Education: Global Perspectives on Quality, Support, and Decision-Making. Routledge.
Quijano, A. (2007). Coloniality and Modernity/Rationality. Cultural Studies, 21(2–3), 168–178. https://doi.org/10.1080/09502380601164353
Rosa, K., & Moore Mensah, F. (2016). Educational pathways of Black women physicists: Stories of experiencing and overcoming obstacles in life. Physical Review Physics Education Research, 12(2), Article 2. https://doi.org/10.1103/PhysRevPhysEducRes.12.020113
Sha, L., Rakovic, M., Das, A., Gasevic, D., & Chen, G. (2022). Leveraging Class Balancing Techniques to Alleviate Algorithmic Bias for Predictive Tasks in Education. IEEE Transactions on Learning Technologies, 15(4), 481–492. https://doi.org/10.1109/TLT.2022.3196278
Shahbazi, N., Lin, Y., Asudeh, A., & Jagadish, H. V. (2023). Representation Bias in Data: A Survey on Identification and Resolution Techniques. ACM Computing Surveys, 55(13s), 1–39. https://doi.org/10.1145/3588433
Shanahan, M.-C. (2009). Identity in science learning: Exploring the attention given to agency and structure in studies of identity. Studies in Science Education, 45(1), 43–64. https://doi.org/10.1080/03057260802681847
Slade, S. (2016). The Open University Ethical use of Student Data for Learning Analytics Policy. The Open University. https://doi.org/10.13140/RG.2.1.1317.4164
Suresh, H., & Guttag, J. (2021). A Framework for Understanding Sources of Harm throughout the Machine Learning Life Cycle. EAAMO ’21: Equity and Access in Algorithms, Mechanisms, and Optimization, 1–9. https://doi.org/10.1145/3465416.3483305
Tan, Y. C., & Celis, L. E. (2019). Assessing Social and Intersectional Biases in Contextualized Word Representations. Conference on Neural Information Processing Systems. https://proceedings.neurips.cc/paper_files/paper/2019/file/201d546992726352471cfea6b0df0a48-Paper.pdf
Traag, V. A., & Waltman, L. (2022). Causal foundations of bias, disparity and fairness. arXiv. https://doi.org/10.48550/arXiv.2207.13665
Traxler, A. L., Cid, X. C., Blue, J., & Barthelemy, R. (2016). Enriching gender in physics education research: A binary past and a complex future. Physical Review Physics Education Research, 12(020114), 1–15. https://doi.org/10.1103/PhysRevPhysEducRes.12.020114
Uttamchandani, S., & Quick, J. (2022). An Introduction to Fairness, Absence of Bias, and Equity in Learning Analytics. In D. Gašević & A. Merceron, The Handbook of Learning Analytics (2nd edn, pp. 205–212). SOLAR. https://doi.org/10.18608/hla22.020
Wolf, T., Debut, L., Sanh, V., Chaumond, J., Delangue, C., Moi, A., Cistac, P., Rault, T., Louf, R., Funtowicz, M., Davison, J., Shleifer, S., Von Platen, P., Ma, C., Jernite, Y., Plu, J., Xu, C., Le Scao, T., Gugger, S., … Rush, A. (2020). Transformers: State-of-the-Art Natural Language Processing. Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, 38–45. https://doi.org/10.18653/v1/2020.emnlp-demos.6
Yeung, K. (2019). Responsibility and AI (DGI(2019)05; Issue DGI(2019)05). Council of Europe. https://rm.coe.int/responsability-and-ai-en/168097d9c5