The Use of ChatGPT Applications in Learning: Impact on Understanding and Student Engagement in TVET Institutions

Authors

  • Elisnorazmaliza Ab Hamid Ab Hamid Department of Information Technology and Communication Politeknik Besut Terengganu Author
  • Hartati Maskur Department of Information Technology and Communication Politeknik Besut Terengganu Author
  • Roshila Abdul Mutalib Department of Information Technology and Communication Politeknik Besut Terengganu Author

DOI:

https://doi.org/10.53840/myjict8-2-98

Keywords:

technical education, TVET, ChatGPT, AI, artificial intelligence, online learning

Abstract

In the field of Technical and Vocational Education and Training (TVET), the use of artificial intelligence (AI) technology has the potential to enrich the learning experience. However, the challenge lies in determining the level of skills required by students to optimize the learning experience and enhance understanding of technical concepts and student engagement. The main objective of this study is to examine the relationship between students' understanding and engagement based on their skill levels in using ChatGPT application in the TVET education sector. A pilot study was conducted with 30 respondents for all 24 items to measure the reliability of the research instrument. The findings indicated a Cronbach's alpha value of 0.982, suggesting that all constructed items are suitable for use as a measurement tool in assessing the needs of the developed survey instrument. Independent sample t-test analyses were conducted to determine if there were significant differences in understanding and student engagement based on students' skills in using the ChatGPT application. A total of 173 respondents participated in this study, comprising 125 students from the Information and Communication Technology Department (ICTD) and 48 students from the Design and Visual Communication Department (DVCD) at Polytechnic Besut Terengganu (PBT). Data obtained from the survey were analyzed using Statistical Packages for Social Sciences version 26.0. Descriptive analysis of minimum comparisons showed a positive impact, indicating that students in TVET institutions agree that the use of the ChatGPT application in learning influences their level of understanding and engagement based on high minimum values. However, independent sample t-test analysis showed differing results. While there was no significant difference in understanding based on the skills of using the application, there was a significant difference in student engagement. This suggests that although students consider the ChatGPT application to have an overall positive impact, the benefits are consistent with students' understanding without making a difference to a specific group. Skills in using this application have a more significant impact on student engagement than on their understanding. Despite the overall positive impact of the application, a focus on student engagement may be a key factor in enhancing the effectiveness of this application in the TVET learning context. To improve the effectiveness of the ChatGPT application, further research on factors contributing to differences in students' engagement levels needs to be conducted. Further studies could also involve comparative research between TVET institutions to identify differences in perceptions and the use of the ChatGPT application. Long-term studies to measure the impact of the ChatGPT application on academic achievement could provide a deeper understanding of its long-term effectiveness in facilitating the learning process in TVET institutions. Opportunities and challenges related to AI in education should also be investigated to support and expand these findings in various technical disciplines and the TVET education context.

Downloads

Download data is not yet available.

References

K. Roose. (2022). The brilliance and weirdness of ChatGPT. The New York Times. Retrieved December 18, 2022.

S. Lock. (2022). What is AI chatbot phenomenon ChatGPT and could it replace humans? The Guardian. Retrieved December 5, 2022.

Deng, J., & Lin, Y. (2023). The Benefits and Challenges of ChatGPT: An Overview. Frontiers in Computing and Intelligent Systems. https://doi.org/10.54097/fcis.v2i2.4465.

Abdel-Messih, M., & Boulos, M. (2023). ChatGPT in Clinical Toxicology.. JMIR medical education. https://doi.org/10.2196/46876.

Wang, J., Hwang, G., & Chang, C. (2021). Directions of the 100 most cited chatbot-related human behavior research: A review of academic publications. Comput. Educ. Artif. Intell., 2, 100023. https://doi.org/10.1016/J.CAEAI.2021.100023.

Berendt, B., Littlejohn, A., & Blakemore, M. (2020). AI in education: learner choice and fundamental rights. Learning, Media and Technology, 45, 312 - 324. https://doi.org/10.1080/17439884.2020.1786399.

Cope, B., Kalantzis, M., & Searsmith, D. (2020). Artificial intelligence for education: Knowledge and its assessment in AI-enabled learning ecologies. Educational Philosophy and Theory, 53, 1229 - 1245. https://doi.org/10.1080/00131857.2020.1728732.

Biswas, S. (2022). Role of ChatGPT in Computer Programming. Mesopotamian Journal of Computer Science. https://doi.org/10.58496/mjcsc/2022/004.

Zhai, X., Chu, X., Chai, C., Jong, M., Istenič, A., Spector, M., Liu, J., Yuan, J., & Li, Y. (2021). A Review of Artificial Intelligence (AI) in Education from 2010 to 2020. Complex., 2021, 8812542:1-8812542:18. https://doi.org/10.1155/2021/8812542.

How, M., & Hung, W. (2019). Educing AI-Thinking in Science, Technology, Engineering, Arts, and Mathematics (STEAM) Education. Education Sciences. https://doi.org/10.3390/EDUCSCI9030184.

Gilson, A., Safranek, C., Huang, T., Socrates, V., Chi, L., Taylor, R., & Chartash, D. (2023). How Does ChatGPT Perform on the United States Medical Licensing Examination? The Implications of Large Language Models for Medical Education and Knowledge Assessment. JMIR Medical Education, 9. https://doi.org/10.2196/45312.

Yue, M., Jong, M., & Dai, Y. (2022). Pedagogical Design of K-12 Artificial Intelligence Education: A Systematic Review. Sustainability. https://doi.org/10.3390/su142315620.

García-Martínez, I., Fernández-Batanero, J., Fernández-Cerero, J., & León, S. (2023). Analysing the Impact of Artificial Intelligence and Computational Sciences on Student Performance: Systematic Review and Meta-analysis. Journal of New Approaches in Educational Research. https://doi.org/10.7821/naer.2023.1.1240.

Ali, J., Shamsan, M., Hezam, T., & Mohammed, A. (2023). Impact of ChatGPT on Learning Motivation: Journal of English Studies in Arabia Felix. https://doi.org/10.56540/jesaf.v2i1.51.

Wekerle, C., Daumiller, M., & Kollar, I. (2020). Using digital technology to promote higher education learning: The importance of different learning activities and their relations to learning outcomes. Journal of Research on Technology in Education, 54, 1 - 17. https://doi.org/10.1080/15391523.2020.1799455.

Tiwari, R. (2023). The integration of AI and machine learning in education and its potential to personalize and improve student learning experiences. Interantional Journal Of Scientific Research In Engineering And Management. https://doi.org/10.55041/ijsrem17645.

Schools Ban ChatGPT Amid Fears of Artificial Intelligence-Assisted Cheating. Available online: https://www.voanews.com/a/schools-ban-chatgpt-amid-fears-of-artificial-intelligence-assisted-cheating/6949800.html (accessed on 10 March 2023).

Qin, C., Zhang, A., Zhang, Z., Chen, J., Yasunaga, M., & Yang, D. (2023). Is ChatGPT a General-Purpose Natural Language Processing Task Solver?. ArXiv, abs/2302.06476. https://doi.org/10.48550/arXiv.2302.06476.

Mhlanga, D. Open AI in Education, the Responsible and Ethical Use of ChatGPT Towards Lifelong Learning. SSRN 2023, 4354422. Available online: https://papers.ssrn.com/sol3/papers.cfm?abstract_id=4354422 (accessed on 10 March 2023).

Megahed, F.M.; Chen, Y.J.; Ferris, J.A.; Knoth, S.; Jones-Farmer, L.A. How Generative AI Models such as ChatGPT Can be (Mis)Used in SPC Practice, Education, and Research? An Exploratory Study. arXiv 2023, arXiv:2302.10916.

Chua, Y.P. (2020). Asas Statistik Penyelidikan: Analisis Data Skala Likert (2nd ed.).

Kuala Lumpur: McGraw-Hill Education

Published

09-07-2024

Issue

Section

Articles

How to Cite

The Use of ChatGPT Applications in Learning: Impact on Understanding and Student Engagement in TVET Institutions. (2024). Malaysian Journal of Information and Communication Technology (MyJICT), 8(2), 78-87. https://doi.org/10.53840/myjict8-2-98