A double blind peer reviewed online publication with in-print supplement since 2010    ISSN: 1949-260X

JTCLT Abstract

Volume 15 Number 1, 2024
Full issue PDF

Poole, F. J., & & Coss, M. (2024). Can ChatGPT Reliably and Accurately Apply a Rubric to L2 Writing Assessments? The Devil is in the Prompt(s). Journal of Technology and Chinese Language Teaching, 15(1), 1-24.

Full paper


This paper investigates the effectiveness of ChatGPT, a generative AI tool, in assessing second language (L2) writing. The study explores the practicality of employing ChatGPT as an assessment tool, focusing on the accuracy and reliability of the AI-generated scores compared to human raters. Various prompting strategies were tested to understand their impact on the effectiveness of ChatGPT in this context. The paper also examines the reliability of ChatGPT scores across different writing topics. The findings demonstrate that ChatGPT can serve as a valuable tool in L2 writing assessment, provided that it is used strategically with well-crafted prompts. The study contributes to the growing body of research on automated writing assessment tools, particularly in the realm of L2 learning, and offers insights into the practical application of such tools in educational settings.


This website is supported by
Department of World Languages, Literatrues, and Cultures, Middle Tennessee State University
Page last updated: 2020-12-31