Wang, T. (2024). The Design and Application of a Chinese Audio-Visual Corpus Based on AI Technology. Journal of Technology and Chinese Language Teaching, 15(1), 70-81.
[王涛. (2024). 基于AI技术的CVC中文视听语料库设计与应用. 科技与中文教学 (Journal of Technology and Chinese Language Teaching), 15(1), 70-81.]

Full paper

Abstract/摘要：

Cultivating an AI-powered language teaching ecosystem has introduced a new model of human-computer collaboration. Many learners are acquiring information and knowledge in a manner that is more contextualized and intelligent. Many educators are adaptive to explore a variety of media resources and AI technologies to guide learners in developing capabilities in problems resolving and knowledge integration. This article describes the design and application of a Chinese Audio-Visual Corpus (CVC) that collects visual language materials from native Chinese speakers in their daily lives to associate textbook content, ontological knowledge, and video materials data in order to provide learners with context-aware teaching resource applications. This AI-based audio-visual corpus offers resourceful and intelligent language teaching methods with the priority of video content retrieval. In addition to serving language teaching, the annotated results from this audio-visual corpus will aid in the training of language models in the interdisciplinary field of Natural Language Processing (NLP) and Computer Vision (CV). It meets the demand for multimodal big data in artificial intelligence for applications such as multimodal discourse analysis, speech act recognition, and sentiment analysis, thereby contributing to the future development of the field of artificial intelligence.

AI赋能下的语言教学生态环境带来了新型人机协作关系，学习者获取信息与知识的方式变得更加场景化、智能化，教师需要利用多种资源媒介和AI技术手段引导学生形成主动探究问题、整合知识的能力。CVC中文视听语料库通过采集来自汉语母语者生活中使用的视频语言材料，将教材内容、本体知识和视频语料数据相关联，为用户提供场景化的教学资源应用服务。基于AI技术的视听语料库应用可提供以视频内容检索为核心的资源型、智慧化语言教学手段。在服务语言教学同时，视频语料标注结果将有助于自然语言处理（NLP）和计算机视觉（CV）交叉领域的语言模型训练，在言语行为识别、多模态分析、情感分析等方面满足人工智能对多模态大数据的需求，反哺人工智能领域的未来发展进程。