A COMPARATIVE ANALYSIS OF AI-ASSISTED VS NON-AI-ASSISTEDLEARNING OUTCOMES AMONG PGDM/MBA STUDENTS
Abstract
Even though Generative AI is being more and more widely used across higher education, studies on the impacts of AI on the learning outcomes remain sparse and the evidence conflicting. To study the differences in learning between AI supported learning and non-AI supported learning among graduate level management students, 389 participants of a PGDM/MBA program responded to an online survey. Survey questions were framed around learning outcomes (e.g. self evaluation and problem solving). AI use was classified into non-AI supported learning (22 participants) and AI supported learning (367 participants). Primary results included no significant difference in five learning outcomes across the groups even after adjusting for potential confounding variables,, except for self-evaluation. There was an adjusted difference for the comparison in favor of the small group (non-AI assisted). Secondary analyses categorized the participants into groups based on the frequency of AI supported learning (partially vs. mostly AI assisted). Unsurprisingly, both comparison groups did not differ from the small non-AI supported learning group across the five primary learning outcomes. Findings of this study do not support claims of either beneficial or harmful impacts of AI assisted learning on self-reported outcomes of PGDM/MBA students,, instead highlighting the need for better integration and stronger future studies on AI supported learning.
How to Cite
References
- 1. Abbas, M., Jam, F. A., & Khan, T. I. (2024). Is it harmful or helpful? Examining the causes and consequences of generative AI usage among university students. International Journal of Educational Technology in Higher Education, 21, Article 10. https://doi.org/10.1186/s41239-024-00444-7
- 2. Acosta-Enriquez, B. G., Arbulú Ballesteros, M. A., Huamaní Jordan, O., López Roca, C., & Saavedra Tirado, K. (2024). Analysis of college students’ attitudes toward the use of ChatGPT in their academic activities: Effect of intent to use, verification of information and responsible use. BMC Psychology, 12, Article 255. https://doi.org/10.1186/s40359-024-01764-z
- 3. Bond, M., Khosravi, H., De Laat, M., Bergdahl, N., Negrea, V., Oxley, E., Pham, P., Chong, S. W., & Siemens, G. (2024). A meta systematic review of artificial intelligence in higher education: A call for increased ethics, collaboration, and rigour. International Journal of Educational Technology in Higher Education, 21, Article 4.
- https://doi.org/10.1186/s41239-023-00436-z
- 4. Chan, C. K. Y. (2023). A comprehensive AI policy education framework for university teaching and learning. International Journal of Educational Technology in Higher Education, 20, Article 38.
- https://doi.org/10.1186/s41239-023-00408-3
- 5. Chan, C. K. Y., & Hu, W. (2023). Students’ voices on generative AI: Perceptions, benefits, and challenges in higher education. International Journal of Educational Technology in Higher Education, 20, Article 43.
- https://doi.org/10.1186/s41239-023-00411-8
- 6. Chen, L., Chen, P., & Lin, Z. (2020). Artificial intelligence in education: A review. IEEE Access, 8, 75264–75278. https://doi.org/10.1109/ACCESS.2020.2988510
- 7. Cotton, D. R. E., Cotton, P. A., & Shipway, J. R. (2024). Chatting and cheating: Ensuring academic integrity in the era of ChatGPT. Innovations in Education and Teaching International, 61(2), 228–239.
- https://doi.org/10.1080/14703297.2023.2190148
- 8. Crompton, H., & Burke, D. (2023). Artificial intelligence in higher education: The state of the field. International Journal of Educational Technology in Higher Education, 20, Article 22. https://doi.org/10.1186/s41239-023-00392-8
- 9. Crompton, H., & Burke, D. (2024). The educational affordances and challenges of ChatGPT: State of the field. TechTrends, 68, 380–392. https://doi.org/10.1007/s11528-024-00939-0
- 10. Farrokhnia, M., Banihashem, S. K., Noroozi, O., & Wals, A. (2024). A SWOT analysis of ChatGPT: Implications for educational practice and research. Innovations in Education and Teaching International, 61(3), 460–474. https://doi.org/10.1080/14703297.2023.2195846
- 11. Grassini, S. (2023). Shaping the future of education: Exploring the potential and consequences of AI and ChatGPT in educational settings. Education Sciences, 13(7), Article 692.
- https://doi.org/10.3390/educsci13070692
- 12. Kasneci, E., Seßler, K., Küchemann, S., Bannert, M., Dementieva, D., Fischer, F., Gasser, U., Groh, G., Günnemann, S., Hüllermeier, E., Krusche, S., Kutyniok, G., Michaeli, T., Nerdel, C., Pfeffer, J., Poquet, O., Sailer, M., Schmidt, A., Seidel, T., … Kasneci, G. (2023). ChatGPT for good? On opportunities and challenges of large language models for education. Learning and Individual Differences, 103, Article 102274.
- https://doi.org/10.1016/j.lindif.2023.102274
- 13. Knowles, M. S. (1975). Self-directed learning: A guide for learners and teachers. Association Press.
- 14. Lai, J. W. (2024). Adapting self-regulated learning in an age of generative artificial intelligence chatbots. Future Internet, 16(6), Article 218. https://doi.org/10.3390/fi16060218
- 15. Lim, W. M., Gunasekara, A., Pallant, J. L., Pallant, J. I., & Pechenkina, E. (2023). Generative AI and the future of education: Ragnarök or reformation? A paradoxical perspective from management educators. The International Journal of Management Education, 21(2), Article 100790. https://doi.org/10.1016/j.ijme.2023.100790
- 16. Lo, C. K. (2023). What is the impact of ChatGPT on education? A rapid review of the literature. Education Sciences, 13(4), Article 410. https://doi.org/10.3390/educsci13040410
- 17. Ouyang, F., Zheng, L., & Jiao, P. (2022). Artificial intelligence in online higher education: A systematic review of empirical research from 2011 to 2020. Education and Information Technologies, 27, 7893–7925. https://doi.org/10.1007/s10639-022-10925-9
- 18. Parasuraman, R., & Riley, V. (1997). Humans and automation: Use, misuse, disuse, abuse. Human Factors, 39(2), 230–253. https://doi.org/10.1518/001872097778543886
- 19. Perkins, M. (2023). Academic integrity considerations of AI large language models in the post-pandemic era: ChatGPT and beyond. Journal of University Teaching & Learning Practice, 20(2), Article 07. https://doi.org/10.53761/1.20.02.07
- 20. Risko, E. F., & Gilbert, S. J. (2016). Cognitive offloading. Trends in Cognitive Sciences, 20(9), 676–688. https://doi.org/10.1016/j.tics.2016.07.002
- 21. Stojanov, A., Liu, Q., & Koh, J. H. L. (2024). University students’ self-reported reliance on ChatGPT for learning: A latent profile analysis. Computers and Education: Artificial Intelligence, 6, Article 100243.
- https://doi.org/10.1016/j.caeai.2024.100243
- 22. Strzelecki, A. (2024). To use or not to use ChatGPT in higher education? A study of students’ acceptance and use of technology. Interactive Learning Environments, 32(9), 5142–5155.
- https://doi.org/10.1080/10494820.2023.2209881
- 23. Sudan, T., Hans, A., & Taggar, R. (2024). Transformative learning with ChatGPT: Analyzing adoption trends and implications for business management students in India. Interactive Technology and Smart Education, 21(4), 735–772. https://doi.org/10.1108/ITSE-10-2023-0202
- 24. Sullivan, M., Kelly, A., & McLaughlan, P. (2023). ChatGPT in higher education: Considerations for academic integrity and student learning. Journal of Applied Learning & Teaching, 6(1), 31–40.
- https://doi.org/10.37074/jalt.2023.6.1.17
- 25. Sweller, J. (1988). Cognitive load during problem solving: Effects on learning. Cognitive Science, 12(2), 257–285. https://doi.org/10.1207/s15516709cog1202_4
- 26. Sweller, J., van Merriënboer, J. J. G., & Paas, F. (2019). Cognitive architecture and instructional design: 20 years later. Educational Psychology Review, 31, 261–292. https://doi.org/10.1007/s10648-019-09465-5
- 27. Tlili, A., Shehata, B., Adarkwah, M. A., Bozkurt, A., Hickey, D. T., Huang, R., & Agyemang, B. (2023). What if the devil is my guardian angel: ChatGPT as a case study of using chatbots in education. Smart Learning Environments, 10, Article 15. https://doi.org/10.1186/s40561-023-00237-x
- 28. UNESCO. (2023). Guidance for generative AI in education and research. United Nations Educational, Scientific and Cultural Organization. https://unesdoc.unesco.org/ark:/48223/pf0000386693
- 29. Uppal, K., & Hajian, S. (2025). Students’ perceptions of ChatGPT in higher education: A study of academic enhancement, procrastination, and ethical concerns. European Journal of Educational Research, 14(1), 199–211. https://doi.org/10.12973/eu-jer.14.1.199
- 30. Valcea, S., Hamdani, M. R., & Wang, S. (2024). Exploring the impact of ChatGPT on business school education: Prospects, boundaries, and paradoxes. Journal of Management Education, 48(5), 915–947. https://doi.org/10.1177/10525629241261313
- 31. Xia, Q., Weng, X., Ouyang, F., Lin, T. J., & Chiu, T. K. F. (2024). A scoping review on how generative artificial intelligence transforms assessment in higher education. International Journal of Educational Technology in Higher Education, 21, Article 40. https://doi.org/10.1186/s41239-024-00468-z
- 32. Yu, C., Yan, J., & Cai, N. (2024). ChatGPT in higher education: Factors influencing ChatGPT user satisfaction and continued use intention. Frontiers in Education, 9, Article 1354929. https://doi.org/10.3389/feduc.2024.1354929
- 33. Zhang, K., & Aslan, A. B. (2021). AI technologies for education: Recent research and future directions. Computers and Education: Artificial Intelligence, 2, Article 100025. https://doi.org/10.1016/j.caeai.2021.100025
- 34. Zhou, X., Teng, D., & Al-Samarraie, H. (2024). The mediating role of generative AI self-regulation on students’ critical thinking and problem-solving. Education Sciences, 14(12), Article 1302. https://doi.org/10.3390/educsci14121302
- 35. Zimmerman, B. J. (2002). Becoming a self-regulated learner: An overview. Theory Into Practice, 41(2), 64–70. https://doi.org/10.1207/s15430421tip4102_2