Validation of the Teachers AI-TPACK Scale for the Indian Educational Setting

Sourav Choudhury; Joy Prakash Deb; Pratima Pradhan; Aishwarya Mishra

doi:10.52756/ijerr.2024.v43spl.009

Authors

Sourav Choudhury School of Education, Manipal GlobalNxt University, Malaysia https://orcid.org/0000-0001-9678-8688
Joy Prakash Deb Department of Education, Fakir Mohan University, Balasore, Odisha-756019, India https://orcid.org/0009-0008-7856-9164
Pratima Pradhan Department of Education, Fakir Mohan University, Balasore, Odisha-756019, India https://orcid.org/0000-0002-7707-5372
Aishwarya Mishra Department of Education, Fakir Mohan University, Balasore, Odisha-756019, India https://orcid.org/0009-0001-3915-7334

DOI:

https://doi.org/10.52756/ijerr.2024.v43spl.009

Keywords:

Artificial Intelligence, confirmatory factor analysis, exploratory factor analysis, India, scale validation, teachers TPACK framework

Abstract

Educators work in extremely dynamic and complex classroom environments where they must continuously alter and update their understanding. Specifically, possession of rich, organised, and integrated knowledge from various domains including knowledge on subject-matter, knowledge of students' thinking and learning, and, increasingly, knowledge of technology is essential. By integrating these three aspects, Mishra and Koehler developed the Technological Pedagogical and Content Knowledge (TPACK) framework in 2006 which offers a thorough and comprehensive method for incorporating technology into the education setting. On the parallel, use of information technology has rapidly increased in the field of education, especially with the introduction of Artificial Intelligence (AI). Thus, the Technological Pedagogical Content Knowledge (TPACK) framework needed to be updated to reflect the growing incorporation of AI into educational standards. Hence, investigator Ning and colleagues in the year 2024 built a framework for incorporating AI into TPACK and developed a robust scale titled Teachers AI-TPACK Scale that measures the teachers competencies in incorporating AI into their teaching environment. The objective of this work was to test the validity of the scale in the Indian educational setting. With a sample size of 660 teaching faculties in universities and colleges across India, this study followed the routine stages such as construct validity analysis in the form of Exploratory Factor Analysis using SPSS V27, followed by Confirmatory Factor Analysis in AMOS software. The original scale with 39 items across seven dimensions were retained throughout the validation process and resulted in a high reliability score of 0.907. This provides compelling evidence for the validity and reliability of the teachers AI-TPACK scale in measuring Indian educators' knowledge and skills at the juncture of AI with pedagogy, technology and content. This is currently the only scale available to measure this construct in India.

References

Aithal, P. S., & Aithal, S. (2019). Analysis of higher education in Indian National education policy proposal 2019 and its implementation challenges. International Journal of Applied Engineering and Management Letters (IJAEML), 3(2), 1–35. Retrieved from https://supublication.com/index.php/ijaeml/article/view/497 DOI: https://doi.org/10.47992/IJAEML.2581.7000.0039

Behling, O., & Law, K. S. (2000). Translating questionnaires and other research instruments: Problems and solutions (Vol. 133). sage. https://books.google.com/books?hl=en&lr=&id=qu9C-OgWI-C&oi=fnd&pg=PP7&dq=Behling,+O.,+%26+Law,+K.+S.+(2000).+Translating+Questionnaires+and+Other+Research+Instruments:+Problems+and+Solutions.+Sage+Publications.&ots=HaVJHtmJ4r&sig=CFQyxuDucExeuy3NXLMstINrMBc

Bhattacharya, I., & Sharma, K. (2007). India in the knowledge economy–an electronic paradigm. International Journal of Educational Management, 21(6), 543–568. https://doi.org/10.1108/09513540710780055 DOI: https://doi.org/10.1108/09513540710780055

Byrne, B. M. (2013). Structural equation modeling with Mplus: Basic concepts, applications, and programming. routledge. DOI: https://doi.org/10.4324/9780203807644

https://www.taylorfrancis.com/books/mono/10.4324/9780203807644/structural-equation-modeling-mplus-barbara-byrne

Celik, I. (2023). Towards Intelligent-TPACK: An empirical study on teachers’ professional knowledge to ethically integrate artificial intelligence (AI)-based tools into education. Computers in Human Behavior, 138, 107468. https://doi.org/10.1016/j.chb.2022.107468 DOI: https://doi.org/10.1016/j.chb.2022.107468

Chai, C. S., Koh, J. H. L., & Tsai, C.C. (2010). Facilitating preservice teachers’ development of technological, pedagogical, and content knowledge (TPACK). Journal of Educational Technology & Society, 13(4), 63–73. https://www.jstor.org/stable/jeductechsoci.13.4.63

Chai, C. S., Koh, J. H. L., & Tsai, C.C. (2013). A review of technological pedagogical content knowledge. Journal of Educational Technology & Society, 16(2), 31–51. https://www.jstor.org/stable/jeductechsoci.16.2.31

Chai, C. S., Tan, L., Deng, F., & Koh, J. H. L. (2017). Examining pre-service teachers’ design capacities for web-based 21st century new culture of learning. Australasian Journal of Educational Technology, 33(2). https://ajet.org.au/index.php/AJET/article/view/3013 DOI: https://doi.org/10.14742/ajet.3013

Chen, L., Chen, P., & Lin, Z. (2020). Artificial intelligence in education: A review. Ieee Access, 8, 75264–75278. 10.1109/ACCESS.2020.2988510 DOI: https://doi.org/10.1109/ACCESS.2020.2988510

Cheng, K.H., & Tsai, C.C. (2019). A case study of immersive virtual field trips in an elementary classroom: Students’ learning experience and teacher-student interaction behaviors. Computers & Education, 140, 103600. https://doi.org/10.1016/j.compedu.2019.103600 DOI: https://doi.org/10.1016/j.compedu.2019.103600

Cheung, G. W., & Rensvold, R. B. (2000). Assessing Extreme and Acquiescence Response Sets in Cross-Cultural Research Using Structural Equations Modeling. Journal of Cross-Cultural Psychology, 31(2), 187–212. https://doi.org/10.1177/0022022100031002003 DOI: https://doi.org/10.1177/0022022100031002003

Choudhury, S., Deb, J. P., Biswas, S., & Pramanik, A. (2024). Social Media Disorder Scale: Structure, Reliability and Validity in Indian Context. International Journal of Experimental Research and Review, 41(Spl Vol), 290–304. https://doi.org/10.52756/ijerr.2024.v41spl.024 DOI: https://doi.org/10.52756/ijerr.2024.v41spl.024

Conway, J. M., & Huffcutt, A. I. (2003). A Review and Evaluation of Exploratory Factor Analysis Practices in Organizational Research. Organizational Research Methods, 6(2), 147–168. https://doi.org/10.1177/1094428103251541 DOI: https://doi.org/10.1177/1094428103251541

Costello, A. B., & Osborne, J. (2019). Best practices in exploratory factor analysis: Four recommendations for getting the most from your analysis. Practical Assessment, Research, and Evaluation, 10(1), 7.

Dede, C. (2011). Reconceptualizing technology integration to meet the necessity of transformation. Journal of Curriculum and Instruction, 5(1), 4–16. h ttps://doi.org/10.3776/joci.2011.v5n1p4-16 DOI: https://doi.org/10.3776/joci.2011.v5n1p4-16

DeVellis, R. F. (2003). Scale development: Theory and applications. SAGE publications. Thousand Okas, London, New Delhi.

Doering, A., Koseoglu, S., Scharber, C., Henrickson, J., & Lanegran, D. (2014). Technology Integration in K–12 Geography Education Using TPACK as a Conceptual Model. Journal of Geography, 113(6), 223–237. https://doi.org/10.1080/00221341.2014.896393 DOI: https://doi.org/10.1080/00221341.2014.896393

Dubey, G., Hasan, M., & Alam, A. (2022). Artificial intelligence (AI) and Indian education system: Promising applications, potential effectiveness and challenges. Towards Excellence, 14(2), 259–269. DOI: https://doi.org/10.37867/TE140223

Ferguson, E., & Cox, T. (1993). Exploratory Factor Analysis: A Users’Guide. International Journal of Selection and Assessment, 1(2), 84–94. https://doi.org/10.1111/j.1468-2389.1993.tb00092.x DOI: https://doi.org/10.1111/j.1468-2389.1993.tb00092.x

Foulger, T. S., Graziano, K. J., Schmidt-Crawford, D., & Slykhuis, D. A. (2017). Teacher educator technology competencies. Journal of Technology and Teacher Education, 25(4), 413–448. https://www.learntechlib.org/primary/p/181966/.

Gerbing, D. W., & Hamilton, J. G. (1996). Viability of exploratory factor analysis as a precursor to confirmatory factor analysis. Structural Equation Modeling: A Multidisciplinary Journal, 3(1), 62–72. https://doi.org/10.1080/10705519609540030 DOI: https://doi.org/10.1080/10705519609540030

Glaser, R. (1984). Education and thinking: The role of knowledge. American Psychologist, 39(2), 93. https://doi.org/10.1037/0003-066X.39.2.93 DOI: https://doi.org/10.1037//0003-066X.39.2.93

Graham, C. R. (2011). Theoretical considerations for understanding technological pedagogical content knowledge (TPACK). Computers & Education, 57(3), 1953–1960. https://doi.org/10.1016/j.compedu.2011.04.010 DOI: https://doi.org/10.1016/j.compedu.2011.04.010

Gupta, P., Akhtar, S., & Mittal, P. (2024). Monopolisation via Technology Adoption in Institutions of Higher Education – Evidence from India. International Journal of Experimental Research and Review, 41(Spl Vol), 168-179. https://doi.org/10.52756/ijerr.2024.v41spl.014 DOI: https://doi.org/10.52756/ijerr.2024.v41spl.014

Gupta, S., Dubey, C., Weersma, L., Vats, R., Rajesh, D., Oleksand, K., & Ratan, R. (2023). Competencies for the academy and market perspective: an approach to the un-sustainable development goals. Int. J. Exp. Res. Rev., 32, 70-88. https://doi.org/10.52756/ijerr.2023.v32.005 DOI: https://doi.org/10.52756/ijerr.2023.v32.005

Hair, J. F. (2009). Multivariate data analysis. https://digitalcommons.kennesaw.edu/facpubs/2925/

Hambleton, R. K., Merenda, P. F., & Spielberger, C. D. (2004). Issues, designs, and technical guidelines for adapting tests into multiple languages and cultures. In Adapting educational and psychological tests for cross-cultural assessment, pp. 15–50. Psychology Press. https://www.taylorfrancis.com/chapters/edit/10.4324/9781410611758-6/issues-designs-technical-guidelines-adapting-tests-multiple-languages-cultures DOI: https://doi.org/10.4324/9781410611758-6

Holmes, W., Bialik, M., & Fadel, C. (2019). Artificial intelligence in education promises and implications for teaching and learning. Center for Curriculum Redesign. https://discovery.ucl.ac.uk/id/eprint/10139722/

Hooper, D., Coughlan, J., & Mullen, M. R. (2008). Structural equation modelling: Guidelines for determining model fit. Electron J. Bus Res Methods, 6(1), 53–60.

Hrastinski, S., Olofsson, A. D., Arkenback, C., Ekström, S., Ericsson, E., Fransson, G., Jaldemark, J., Ryberg, T., Öberg, L.M., Fuentes, A., Gustafsson, U., Humble, N., Mozelius, P., Sundgren, M., & Utterberg, M. (2019). Critical Imaginaries and Reflections on Artificial Intelligence and Robots in Postdigital K-12 Education. Postdigital Science and Education, 1(2), 427–445. https://doi.org/10.1007/s42438-019-00046-x DOI: https://doi.org/10.1007/s42438-019-00046-x

Hutcheson, G. D., & Sofroniou, N. (1999). The multivariate social scientist: Introductory statistics using generalized linear models. Sage. https://www.torrossa.com/en/resources/an/4912571

Irrinki, M. K. (2021). Learning through ICT–the role of Indian higher education platforms during a pandemic. Library Philosophy and Practice, pp. 1–19.

Jimoyiannis, A. (2010). Designing and implementing an integrated technological pedagogical science knowledge framework for science teachers' professional development. Computers & Education, 55(3), 1259–1269. https://doi.org/10.1016/j.compedu.2010.05.022 DOI: https://doi.org/10.1016/j.compedu.2010.05.022

Kaiser, H. F. (1960). The Application of Electronic Computers to Factor Analysis. Educational and Psychological Measurement, 20(1), 141–151. https://doi.org/10.1177/001316446002000116 DOI: https://doi.org/10.1177/001316446002000116

Kaiser, H. F. (1974). An index of factorial simplicity. Psychometrika, 39(1), 31–36. https://doi.org/10.1007/BF02291575 DOI: https://doi.org/10.1007/BF02291575

Kanbul, S., Adamu, I., Usman, A. G., & Abba, S. I. (2022). Coupling TPACK instructional model with computing artificial intelligence techniques to determine technical and vocational education teacher’s computer and ICT tools competence. https://www.preprints.org/manuscript/202203.0048 DOI: https://doi.org/10.20944/preprints202203.0048.v1

Kasinathan, G., & Yogesh, K. S. (2019). Exploring AI in Indian school education. National Seminar on “Innovative Practices and Research in the Era of Digi Education, pp. 21–22. https://itforchange.net/sites/default/files/2019-08/Paper-AI-and-education-Seminar-by-UOH-March-20-2019-with-abstract.pdf

Kline, R. B. (2023). Principles and practice of structural equation modeling. Guilford publications.

Koehler, M., & Mishra, P. (2009). What is technological pedagogical content knowledge (TPACK)? Contemporary Issues in Technology and Teacher Education, 9(1), 60–70. https://www.learntechlib.org/primary/p/29544/

Koh, J. H. L., Chai, C. S., & Tsai, C.C. (2013). Examining practicing teachers’ perceptions of technological pedagogical content knowledge (TPACK) pathways: A structural equation modeling approach. Instructional Science, 41(4), 793–809. https://doi.org/10.1007/s11251-012-9249-y DOI: https://doi.org/10.1007/s11251-012-9249-y

Laurillard, D. (2008). Technology enhanced learning as a tool for pedagogical innovation. Journal of Philosophy of Education, 42(3–4), 521–533. https://doi.org/10.1111/j.1467-9752.2008.00658.x DOI: https://doi.org/10.1111/j.1467-9752.2008.00658.x

Lavi, R., Tal, M., & Dori, Y. J. (2021). Perceptions of STEM alumni and students on developing 21st century skills through methods of teaching and learning. Studies in Educational Evaluation, 70, 101002. https://doi.org/10.1016/j.stueduc.2021.101002 DOI: https://doi.org/10.1016/j.stueduc.2021.101002

Leinhardt, G., & Greeno, J. G. (1986). The cognitive skill of teaching. Journal of Educational Psychology, 78(2), 75. https://doi.org/10.1037/0022-0663.78.2.75 DOI: https://doi.org/10.1037//0022-0663.78.2.75

Mahmud, M. M., & Wong, S. F. (2022). Digital age: The importance of 21st century skills among the undergraduates. Frontiers in Education, 7. https://doi.org/10.3389/feduc.2022.950553 DOI: https://doi.org/10.3389/feduc.2022.950553

Mishra, P. (2019). Considering Contextual Knowledge: The TPACK Diagram Gets an Upgrade. Journal of Digital Learning in Teacher Education, 35(2), 76–78. https://doi.org/10.1080/21532974.2019.1588611 DOI: https://doi.org/10.1080/21532974.2019.1588611

Mishra, P., & Koehler, M. J. (2006). Technological Pedagogical Content Knowledge: A Framework for Teacher Knowledge. Teachers College Record, 108(6), 1017–1054. https://doi.org/10.1111/j.1467-9620.2006.00684.x DOI: https://doi.org/10.1111/j.1467-9620.2006.00684.x

Mishra, P., Spiro, R. J., & Feltovich, P. J. (1996). Technology, representation, and cognition: The prefiguring of knowledge in cognitive flexibility hypertexts. Advances in Discourse Processes, 58, 287–305.

Mulaik, S. A., James, L. R., Van Alstine, J., Bennett, N., Lind, S., & Stilwell, C. D. (1989). Evaluation of goodness-of-fit indices for structural equation models. Psychological Bulletin, 105(3), 430. https://doi.org/10.1037/0033-2909.105.3.430 DOI: https://doi.org/10.1037/0033-2909.105.3.430

NEP_Final_English_0.pdf. (n.d.). Retrieved August 5, 2024, from

https://www.education.gov.in/sites/upload_files/mhrd/files/NEP_Final_English_0.pdf

Ng, D. T. K., Leung, J. K. L., Chu, S. K. W., & Qiao, M. S. (2021). Conceptualizing AI literacy: An exploratory review. Computers and Education: Artificial Intelligence, 2, 100041. DOI: https://doi.org/10.1016/j.caeai.2021.100041

Niess, M. L. (2013). Central Component Descriptors for Levels of Technological Pedagogical Content Knowledge. Journal of Educational Computing Research, 48(2), 173–198. https://doi.org/10.2190/EC.48.2.d DOI: https://doi.org/10.2190/EC.48.2.d

Ning, Y., Zhang, C., Xu, B., Zhou, Y., & Wijaya, T. T. (2024). Teachers’ AI-TPACK: Exploring the relationship between knowledge elements. Sustainability, 16(3), 978. https://doi.org/10.3390/su16030978 DOI: https://doi.org/10.3390/su16030978

Nunnally, J., & Bernstein, I. (1994). Psychometric Theory 3rd edition (MacGraw-Hill, New York).

Perry, C. M., Nicholls, J. G., Clough, P., & Craven, R. (2004). Perceived Control in School: A Multinational Comparison of Elementary and Secondary Students. Educational Psychology, 24(3), 223-244.

Roll, I., & Wylie, R. (2016). Evolution and Revolution in Artificial Intelligence in Education. International Journal of Artificial Intelligence in Education, 26(2), 582–599. https://doi.org/10.1007/s40593-016-0110-3 DOI: https://doi.org/10.1007/s40593-016-0110-3

Roussinos, D., & Jimoyiannis, A. (2019). Examining Primary Education Teachers’ Perceptions of TPACK and the Related Educational Context Factors. Journal of Research on Technology in Education, 51(4), 377–397. https://doi.org/10.1080/15391523.2019.1666323 DOI: https://doi.org/10.1080/15391523.2019.1666323

Schmidt, D. A., Baran, E., Thompson, A. D., Mishra, P., Koehler, M. J., & Shin, T. S. (2009). Technological Pedagogical Content Knowledge (TPACK): The Development and Validation of an Assessment Instrument for Preservice Teachers. Journal of Research on Technology in Education, 42(2), 123–149. https://doi.org/10.1080/15391523.2009.10782544 DOI: https://doi.org/10.1080/15391523.2009.10782544

Seufert, S., Guggemos, J., & Sailer, M. (2021). Technology-related knowledge, skills, and attitudes of pre-and in-service teachers: The current situation and emerging trends. Computers in Human Behavior, 115, 106552. https://doi.org/10.1016/j.chb.2020.106552 DOI: https://doi.org/10.1016/j.chb.2020.106552

Sharma, D., & Singh, V. (2010). ICT in Universities of the Western Himalayan Region of India II: A Comparative SWOT Analysis (arXiv:1002.1193). arXiv. http://arxiv.org/abs/1002.1193

Shulman, L. S. (1986). Those Who Understand: Knowledge Growth in Teaching. Educational Researcher, 15(2), 4–14. https://doi.org/10.3102/0013189X015002004 DOI: https://doi.org/10.3102/0013189X015002004

Spiro, R. J., & Jehng, J.-C. (2012). Cognitive flexibility and hypertext: Theory and technology for the nonlinear and multidimensional traversal of complex subject matter. In Cognition, education, and multimedia. pp. 163–205. Routledge. https://api.taylorfrancis.com/content/chapters/edit/download?identifierName=doi&identifierValue=10.4324/9780203052174-7&type=chapterpdf DOI: https://doi.org/10.4324/9780203052174-11

Van de Vijver, F. J., & Leung, K. (2021). Methods and data analysis for cross-cultural research (Vol. 116). Cambridge University Press. DOI: https://doi.org/10.1017/9781107415188

Vinay, S. B. (2023). Application of Artificial Intelligence (AI) In School Teaching and Learning Process-Review and Analysis. Information Technology and Management, 14(1), 1–5. https://doi.org/10.17605/OSF.IO/AERNV

Voogt, J., Fisser, P., Pareja Roblin, N., Tondeur, J., & Van Braak, J. (2013). Technological pedagogical content knowledge – a review of the literature. Journal of Computer Assisted Learning, 29(2), 109–121. https://doi.org/10.1111/j.1365-2729.2012.00487.x DOI: https://doi.org/10.1111/j.1365-2729.2012.00487.x

Williams, B., Onsman, A., & Brown, T. (2010). Exploratory Factor Analysis: A Five-Step Guide for Novices. Australasian Journal of Paramedicine, 8, 1–13. https://doi.org/10.33151/ajp.8.3.93 DOI: https://doi.org/10.33151/ajp.8.3.93

Xu, L. (2020). The Dilemma and Countermeasures of AI in Educational Application. 2020, 4th International Conference on Computer Science and Artificial Intelligence, pp. 289–294. https://doi.org/10.1145/3445815.3445863 DOI: https://doi.org/10.1145/3445815.3445863

Zawacki-Richter, O., Marín, V. I., Bond, M., & Gouverneur, F. (2019). Systematic review of research on artificial intelligence applications in higher education – where are the educators? International Journal of Educational Technology in Higher Education, 16(1), 39. https://doi.org/10.1186/s41239-019-0171-0 DOI: https://doi.org/10.1186/s41239-019-0171-0