A socio-cognitive approach to evaluating general language proficiency speaking test assessment criteria

This study explores general language proficiency testing, with a specific emphasis on speaking assessment criteria. Despite the real-world focus of these tests, the criteria used have been largely developed based on the judgments and perceptions of language professionals, potentially compromising the meaningfulness of test scores. The primary objective of this research is to determine whether current assessment criteria represent the non-language specialist perspective and the intended post-test context of language use in the real-world (i.e., beyond the classroom). To achieve this, the study first identifies the salient features of non-native test performance speech production from the non-language specialist and language specialist perspective, avoiding pre-determined criteria. This captures the listening behavior profiles of target groups. Second, the study determines the structural relationship of these features when used as criteria for rating speaking performances. The results are compared with the criteria and componential structure of current general language proficiency assessments to ascertain the degree of representation. The study employs a mixed-methods multiphase design, incorporating a qualitative phase with a quantitative phase. Cambridge speaking tests from the Main Suite collection of exams were selected as a representative general language proficiency testing system for data collection. The findings of this research may contribute to enhanced context-of-use-based validity of general language proficiency speaking tests.

2024

dissertation

true