From Tool to Tutor: Socratic AI Tutoring, Metacognitive Engagement, and Prior Knowledge as Determinants of Learning Gains in Gateway STEM Courses
DOI:
https://doi.org/10.55737/rl.2025.41184Keywords:
Socratic AI Tutoring, Intelligent Tutoring Systems, Gateway STEM, Metacognitive Engagement, Prior Knowledge, Proximal DevelopmentAbstract
Gateway STEM courses introductory courses such as Algebra carry disproportionately high failure and withdrawal rates, creating a critical bottleneck in undergraduate STEM pipelines. Intelligent tutoring systems (ITS) have demonstrated consistently positive learning effects over conventional instruction, yet the mechanisms underlying these gains, particularly for students entering with limited prior preparation, remain incompletely theorized. This study presents a quasi-experimental pretest-posttest framework comparing a Socratic AI Tutoring System (SATS) with traditional instructor-led instruction across two intact sections of a gateway STEM course (target N = 120-200). Drawing on Social Constructivism, Cognitive Load Theory, and Socratic Pedagogy, the study tests four hypotheses: that SATS produces higher learning gains (H1), that metacognitive engagement mediates this effect (H2), that prior knowledge moderates the treatment effect (H3), and that the mediation pathway is itself moderated by prior knowledge (H4). Data was analyzed by using ANCOVA, Hayes’ PROCESS Models 4, 1, and 7 respectively. Findings demonstrate that Socratic AI tutoring enhances academic performance especially among low-prior-knowledge learners, with metacognitive engagement serving as the primary mechanism of effect.
References
Abdullah, J., Lenando, H., & Narayanan, P. (2025). A TRIZ and socratic AI-based problem-solving framework. IFIP Advances in Information and Communication Technology, 273-287. https://doi.org/10.1007/978-3-032-08847-5_20
Alzen, J. L., Langdon, L. S., & Otero, V. K. (2018). A logistic regression investigation of the relationship between the learning assistant model and failure rates in introductory STEM courses. International Journal of STEM Education, 5(1). https://doi.org/10.1186/s40594-018-0152-1
Boghossian, P. (2003). How socratic pedagogy works. Informal Logic, 23(2). https://doi.org/10.22329/il.v23i2.2170
Borchers, C., Houk, A., Aleven, V., & Koedinger, K. R. (2025). Engagement and learning benefits of goal setting with rewards in Human-AI tutoring. Lecture Notes in Computer Science, 46-59. https://doi.org/10.1007/978-3-031-98459-4_4
Bouamor, H., Pino, J., & Bali, K. (2023). Proceedings of the 2023 conference on empirical methods in natural language processing. Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing.
Changkui LI. (2025). AI as a socratic dialogue partner: An intervention study on enhancing students’ critical thinking skills. Artificial Intelligence Education Studies, 1(4), 1-11. https://doi.org/10.6914/aiese.010401
Chen, X. (2013). STEM Attrition: College Students' Paths into and out of STEM Fields. Statistical Analysis Report. NCES 2014-001. National Center for Education Statistics.
DaCosta, J. (2025). Marginalization of Black Males in Science, Technology, Engineering, and Mathematics: Barriers and Pathways to Accessing Physics. In Diverse Leadership Perspectives in Education: From K-12 to Higher Education (pp. 145–184). IGI Global Scientific Publishing.
Davis, P. C., & Steinglass, E. E. (1997). A dialogue about Socratic teaching. NYU Rev. L. & Soc. Change, 23, 249.
Fakour, H., & Imani, M. (2025a). Socratic wisdom in the age of AI: a comparative study of ChatGPT and human tutors in enhancing critical thinking skills. Frontiers in Education, 10(1528603). https://doi.org/10.3389/feduc.2025.1528603
Fakour, H., & Imani, M. (2025b). Socratic wisdom in the age of AI: A comparative study of ChatGPT and human tutors in enhancing critical thinking skills. Frontiers in Education, 10, 1528603. https://doi.org/10.3389/feduc.2025.1528603
Feng, L. (2025). Investigating the effects of artificial intelligence-assisted language learning strategies on cognitive load and learning outcomes: A comparative study. Journal of Educational Computing Research, 62(8), 1961–1994. https://doi.org/10.1177/07356331241268349
Gunsaldi, M. S., Guner, E. G., Uckan, M., & Bati, K. (2025). The impact of generative AI applications on student learning outcomes in science education: A systematic review. Journal of Education in Science, Environment and Health, 11(3), 196–208. https://doi.org/10.55549/jeseh.840
Gurung, A., Lin, J., Gutterman, J., Thomas, D. R., Houk, A., Gupta, S., Brunskill, E., Branstetter, L., Aleven, V., & Koedinger, K. (2025). Human tutoring improves the impact of AI tutor use on learning outcomes. In Lecture Notes in Computer Science (pp. 393–407). Springer Nature Switzerland.
Hagos, T. (2026). Socratic method of questioning: the effect on improving students’ understanding and application of chemical kinetics concepts. Chemistry Education Research and Practice. https://doi.org/10.1039/d5rp00216h
Hasan, M. R., & Khan, B. (2023). An AI-based intervention for improving undergraduate STEM learning. PLOS ONE, 18(7), e0288844. https://doi.org/10.1371/journal.pone.0288844
Hashemi Tonekaboni, N., & Soleymani, S. (2026). Socratic Method Revisited: Human-AI Dialogue for Knowledge Creation and Internalization.
Hayes, A. F. (2017). Introduction to mediation, moderation, and conditional process analysis: A regression-based approach. Guilford publications.
Jeong, S. J. (2026). Quality of employment of high school graduates: Focusing on the effect of student vocational education and training experience. International Journal of Training and Development, ijtd.70024. https://doi.org/10.1111/ijtd.70024
Kjallstrom, B. (2025, October 22). EON Reality Unveils Brainy Socratic Tutor 2.0 The World’s First AI-Powered Personal Mentor Delivering Bloom’s 2 Sigma Effect at Global Scale. EON Reality - AI Assisted XR-Based Knowledge Transfer for Education and Industry. https://eonreality.com/eon-reality-unveils-brainy-socratic-tutor-2-0-the-worlds-first-ai-powered-personal-mentor-delivering-blooms-2-sigma-effect-at-global-scale/
Kulik, J. A., & Fletcher, J. D. (2016). Effectiveness of intelligent tutoring systems. Review of Educational Research, 86(1), 42-78. https://doi.org/10.3102/0034654315581420
Kwak, M. (2025). The Effectiveness of AI-Driven Tools in Improving Student Learning Outcomes Compared to Traditional Methods. Issues in Information Systems, 26(4), 233–247. https://doi.org/10.48009/4_iis_2025_120
Lineman, J., Sweet, M., & Sutton, F. (2025). Beyond Content: Leveraging AI and Metacognitive Strategies for Transformative Learning in Higher Education. The Transnational Journal of Business, 10(1). https://doi.org/10.64010/IWMI3821
Ma, W., Adesope, O. O., Nesbit, J. C., & Liu, Q. (2014). Intelligent tutoring systems and learning outcomes: A meta-analysis. Journal of Educational Psychology, 106(4), 901–918. https://doi.org/10.1037/a0037123
Mazari, N. (2025). Building metacognitive skills using AI tools to help higher education students reflect on their learning process (Vol. 13). RHS-Revista Humanismo y Sociedad.
Mühendise, P. T. A., & Karaarslan, Ö. Ü. E. (n.d.). MÜHENDİSLİK-MİMARLIK EĞİTİMİNDE YAPAY ZEKÂ DESTEKLİ SOKRATİK ÖĞRENME DENEYİMİ.
Rana, A., Vaidya, P., & Hu, Y.-C. (2025). Transformative role of large language models in education and comparative analysis with traditional technologies. Education and Information Technologies, 1–26. https://doi.org/10.1007/s10639-025-13854-5
Schraw, G., & Dennison, R. S. (1994). Assessing Metacognitive Awareness. Contemporary Educational Psychology, 19, 460-475. https://doi.org/10.1006/ceps.1994.1033
Sofologi, M., Katsarou, D. V., Tsirides, A., & Efthymiou, E. (2025). The synergy of artificial intelligence and education: New perspectives of an innovative artificial tutoring in school settings. In Prompt Engineering and Generative AI Applications for Teaching and Learning (pp. 189–202). IGI Global Scientific Publishing.
Steenbergen-Hu, S., & Cooper, H. (2014). A meta-analysis of the effectiveness of intelligent tutoring systems on college students’ academic learning. Journal of Educational Psychology, 106(2), 331–347. https://doi.org/10.1037/a0034752
Sunil, K., & Thakkar, A. (2025). SocraticAI: Transforming LLMs into guided CS tutors through scaffolded interaction. In arXiv [cs.CY]. https://doi.org/10.48550/arXiv.2512.03501
Sweller, J. (1988). load during problem solving: 职 On/earn/ng. Cognitive Science, 12(2).
Tezer, M. (2025). Metacognitive Engagement in AI-Supported Learning: Frameworks, Challenges, and Transformations. In M. Tezer (Ed.), Education and Human Development (Vol. 43). IntechOpen. https://doi.org/10.5772/intechopen.1012658
Tse Hung, J., Lee, J., & Cui, C. (2024, June 5). Socratic Mind—AI-Powered Oral Assessment with Socratic Questioning. https://solve.mit.edu/solutions/90692
Tzanoulinou, D., Triantafyllopoulos, L., Paxinou, E., Feretzakis, G., Kalles, D., & Verykios, V. (2025, July). The Socratic Turn in AI: Reviewing the Transformation of LLMs into Dialogical Educators. In 2025 16th International Conference on Information, Intelligence, Systems & Applications (IISA) (pp. 1-8). IEEE. https://doi.org/10.1109/IISA66859.2025.11311184
VanLEHN, K. (2011). The relative effectiveness of human tutoring, intelligent tutoring systems, and other tutoring systems. Educational Psychologist, 46(4), 197–221. https://doi.org/10.1080/00461520.2011.611369
Vygotsky, L. S. (1978). Mind in society: The development of higher psychological processes (Vol. 86). Harvard university press.
Wagner, L. (2024, March 13). AI Support Can Prevent College Students from Failing STEM Classes, Study Shows [News]. Yahoo News. https://www.yahoo.com/news/ai-support-prevent-college-students-170100404.html
Xu, W., & Ouyang, F. (2022). A systematic review of AI role in the educational system based on a proposed conceptual framework. Education and Information Technologies, 27(3), 4195–4223. https://doi.org/10.1007/s10639-021-10774-y
Xu, Y. J. (2013). Career outcomes of STEM and non-STEM college graduates: Persistence in majored-field and influential factors in career choices. Research in Higher Education, 54(3), 349–382. https://doi.org/10.1007/s11162-012-9275-2
Yingling, S. A. (2018). A causal-comparative quasi-experimental study: Self-efficacy and underrepresented minorities (URMs) success in high school STEM advanced academic placement (AAP) courses (Doctoral dissertation, Northcentral University).
Zhai, Y., & Nezakatgoo, B. (2025a). Evaluating AI-Powered Applications for Enhancing Undergraduate Students’ Metacognitive Strategies, Self-Determined Motivation, and Social Learning in English Language Education. Scientific Reports, 15(1), 35199. https://doi.org/10.1038/s41598-025-19118-z
Zhao, X., Chowdhury, S., & Chowdhury, T. (2020). Integrating evidence-based learning in engineering and computer science gateway courses. 2020 ASEE Virtual Annual Conference Content Access Proceedings.
Downloads
Published
Issue
Section
License
Copyright (c) -1 Dr. Syed Rizwan Ali, Muhammad Omar Khan, Dr. Muhammad Faraz, Syed Ali Imran
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.
