This Article Full Text (PDF) References Alert me when this article is cited Alert me if a correction is posted Citation Map Services Email this article to a friend Similar articles in this journal Alert me to new issues of the journal Add to Saved Citations Download to citation manager Request Permissions Request Reprints Add to My Marked Citations Citing Articles Citing Articles via Google Scholar Citing Articles via Scopus Google Scholar Articles by Schaefer, E. Search for Related Content Social Bookmarking                         What's this?

Rater bias patterns in an EFL writing assessment

Ochanomizu University, Japan, schaefer.edward{at}ocha.ac.jp

The present study employed multi-faceted Rasch measurement (MFRM) to explore the rater bias patterns of native English-speaker (NES) raters when they rate EFL essays. Forty NES raters rated 40 essays written by female Japanese university students on a single topic adapted from the TOEFL Test of Written English (TWE). The essays were assessed using a six-category rating scale (Content, Organization, Style and Quality of Expression, Language Use, Mechanics, and Fluency). MFRM revealed several recurring bias patterns among rater subgroups. In rater—category bias interactions, if Content and/or Organization were rated severely, then Language Use and/or Mechanics were rated leniently, and vice versa. In rater—writer bias interactions, there tended to be more severe or lenient bias towards higher ability writers than lower ability writers. Some raters also rated higher ability writers more severely and lower ability writers more leniently than expected. This study has implications for issues of rater training in L2 writing assessment.

Key Words: multi-faceted Rasch analysis • rater bias studies • rating scale analysis • second language writing assessment

Language Testing, Vol. 25, No. 4, 465-493 (2008)
DOI: 10.1177/0265532208094273


CiteULike    Complore    Connotea    Del.icio.us    Digg    Reddit    Technorati    Twitter    What's this?