Workshop A (2-day, In Thai)

Title: Language Test Design and Alignment to the CEFR
(2-day workshop in Thai)

Viphavee Vongpumivitch, National Tsing Hua University
Napat Jitpaisarnwattana, Silapakorn University

Workshop Description 

     Ever since its publication in 2001, the Common European Framework of Reference for Languages (CEFR) has expanded its influence far beyond Europe, and Thailand is no exception. In January 2014, the Thai Ministry of Education announced that the CEFR would be adopted as a framework for English language education reform, impacting curriculum design, classroom teaching, teacher training, learning objectives, and testing (Office of the Basic Education Commission, Thailand Ministry of Education, 2014). Over the past decade, stakeholders in language education have become increasingly familiar with the CEFR’s six reference levels – A1, A2, B1, B2, C1, and C2. Numerous domestic English tests now report results linked to these levels.

     Nevertheless, some language educators and assessment practitioners continue to have questions regarding the CEFR: Is the CEFR itself a test? How can new tests be developed based on the CEFR? How do we translate the CEFR’s can-do statements into concrete test items? How should we interpret CEFR-based test scores? To what extent can we trust the claims made by test developers that their test results are aligned with the CEFR?

     This workshop, presented in Thai at the very first LTRC to be held in Bangkok, seeks to address these questions by drawing on official training resources provided by the Council of Europe, the Association of Languages Testers in Europe (ALTE), the European Association for Language Testing and Assessment (EALTA), the British Council, and the UK Association for Language Testing and Assessment (UKALTA). The two-day workshop will consist of four parts:

     Part 1 will introduce the core components of the CEFR based on the CEFR Manual (Council of Europe, 2001) and the Companion Volume (Council of Europe, 2020). Key aspects of the CEFR, including its action-oriented approach, illustrative descriptive scales, and concepts regarding language assessment, will be covered. Through group activities, participants will explore the CEFR’s four modes of communication: reception (listening and reading), production (speaking and writing), interaction (spoken and written), and mediation (facilitating communication between individuals or groups who cannot communicate directly).

     Part 2 will cover the fundamental considerations in language testing – validity, reliability and the test development process (Council of Europe, 2011). Participants will then learn about the required steps involved in CEFR test alignment, as detailed in the CEFR Alignment Handbook (Figueras, Little, and O’Sullivan, 2022). These include test specification, standardization, standard setting, and validation. Due to time constraints, this part of the workshop will describe each stage of the CEFR alignment rather than engaging the participants in actual test development or alignment.

     Part 3 will showcase examples of English tests developed based on the CEFR, primarily the Cambridge Main Suites of English Examinations (https://www.cambridgeenglish.org/exams-and-tests/qualifications/). Additionally, we will also discuss a local English proficiency test at a Thai university to provide insights into the issues involving score interpretation and use within a Thai university context.

     Part 4 will address performance tasks in CEFR-based tests that require human ratings. Topics will include rater training, rater behaviors, and the integration of automated scoring systems, such as e-raters with human raters in the rating process. The workshop will conclude by exploring the potential of AI-powered technology to support local raters in consistent scoring and reduce measurement errors caused by raters’ behaviors.

     By the end of this two-day workshop, participants are expected to gain a comprehensive understanding of the CEFR and the necessary steps for CEFR test alignment. Equipped with this knowledge, they will be able to critically evaluate claims made by test developers about the alignment of their tests with the CEFR, and take the right steps towards developing a CEFR-based test.

References: 

     Council of Europe. (2001). Common European Framework of Reference for Languages: Learning, Teaching, Assessment. Cambridge University Press.

     Council of Europe. (2011). Manual for Language Test Development and Examining: For Use with the CEFR. Association of Languages Testers in Europe (ALTE).

     Council of Europe. (2020). Common European Framework of Reference for Languages: Learning, Teaching, Assessment – Companion Volume. Council of Europe Publishing. Figueras, N., Little, D., & O’Sullivan, B. (Eds.) (2022). Aligning Language Education with the CEFR: A Handbook. British Council, UKALTA, EALTA & ALTE.

     Office of the Basic Education Commission, Thailand Ministry of Education. (2014). Guidelines Following the Ministry of Education's Announcement on the Policy for English Language Teaching Reform. [Source in Thai] สำนักงานคณะกรรมการการศึกษาขั้นพื้นฐาน กระทรวงศึกษาธิการ. แนวปฏิบัติ ตามประกาศกระทรวงศึกษาธิการ เรื่อง นโยบายการปฏิรูปการเรียนการสอนภาษาอังกฤษ. พฤษภาคม ๒๕๕๗.

Presenter Biodata: 

Viphavee Vongpumivitch received her PhD in language assessment from University of California, Los Angeles. She has been teaching at the Department of Foreign Languages and Literature, National Tsing Hua University in Taiwan for the past twenty years. In 2015 she gave her first workshop on the CEFR to Thai language teaching and testing professionals while she was working as a visiting scholar at the Graduate Institute of Language and Communication, National Institute of Development Administration (NIDA). Since then, she had conducted several workshops on CEFR for English teachers around Thailand. She was one of the plenary speakers at the 5th International Conference on Foreign Language Learning and Teaching (FLLT 2018) organized by the Language Institute, Thammasat University.

Napat Jitpaisarnwattana is a lecturer of English and Computer-assisted Language Learning at Silpakorn University, Thailand. He recently finished his Master of Studies focusing machine learning and automated assessment at Homerton College, University of Cambridge. He received his PhD in Applied Linguistics from KMUTT and an MSc in Teaching English Language in University Settings from Oxford University. He is editor of Malaysian Journal of ELT Research and associate editor of rEFLections. His research interests include Computer-assisted Language Learning (CALL), Language MOOCs, the Internet of things, digital wellbeing, digital literacies, learning analytics, machine learning, AI in language education, technology-mediated language assessment and learning-oriented assessment (LOA).