Using generative AI in teaching “Zar zaman” poetry in the 8th grade: a pilot study

Abstract

The article examines the possibilities and risks of using generative artificial intelligence (ChatGPT GPT-4o) in teaching one of the most complex sections of the Kazakh literature school curriculum – "Zar Zaman" poetry (Shortanbai Kanaiuly, Dulat Babataiuly, Murat Monkeuly) to 8th-grade students. The aim of the study is to empirically evaluate the didactic effectiveness and potential threats of using AI in national literature lessons. A pilot study with a quasi-experimental design was conducted in a secondary school in Almaty over 6 weeks (12 lessons). In the experimental group (n=27), traditional teaching was supplemented with ChatGPT elements, while the control group (n=25) was taught using only the traditional methodology. The students' learning quality was assessed using an author-designed 10-point rubric covering 6 criteria. Statistical analysis was performed using the Mann-Whitney U test. Results revealed a statistically significant improvement in the experimental group for the skills of "understanding the historical context" (+39%) and "comparative analysis" (+48%) (p < 0.01). However, no significant difference was observed for the "independent opinion" criterion (p > 0.05), which is consistent with the issue of "academic dependency" discussed in international science. Furthermore, the study identified cases of AI hallucinations (confabulation) due to a lack of understanding of national-historical and cultural codes. In conclusion, generative AI can be an effective supplementary tool, but it requires strict teacher moderation and mandatory fact-checking by students. To reduce AI dependency, the article proposes specific prompt templates and the "own opinion first – then AI" strategy.