This Article Full Text (PDF) References Alert me when this article is cited Alert me if a correction is posted Citation Map Services Email this article to a friend Similar articles in this journal Alert me to new issues of the journal Add to Saved Citations Download to citation manager Request Permissions Request Reprints Add to My Marked Citations Citing Articles Citing Articles via HighWire Citing Articles via Google Scholar Citing Articles via Scopus Google Scholar Articles by Schoonen, R. Search for Related Content Social Bookmarking                         What's this?

Generalizability of writing scores: an application of structural equation modeling

University of Amsterdam, rob.schoonen{at}uva.nl

The assessment of writing ability is notoriously difficult. Different facets of the assessment seem to influence its outcome. Besides the writer’s writing proficiency, the topic of the assignment, the features or traits scored (e.g., content or language use) and even the way in which these traits are scored (e.g., holistically or analytically) may contribute to the writer’s score. In this article, the effect of these facets is estimated in a generalizability study using variance analytic techniques. A structural equation modeling (SEM) approach is used to estimate the variance components in the writing scores. Eighty-nine grade 6 students (aged 11-12 years) wrote four essays, each of which was scored by five raters using two scoring methods (i.e., holistically and analytically) for two traits (i.e., Content and Organization, and Language Use). Analyses of these ratings showed that the generalizability of writing scores and the effects of raters and topics are very much dependent on the way the essays are scored and the trait that is scored. The overall picture is that writing tasks contribute more to the score variance than raters do.

Language Testing, Vol. 22, No. 1, 1-30 (2005)
DOI: 10.1191/0265532205lt295oa


CiteULike    Complore    Connotea    Del.icio.us    Digg    Reddit    Technorati    Twitter    What's this?

This article has been cited by other articles:

				A. Gebril Score generalizability of academic writing tasks: Does one test method fit it all? Language Testing, October 1, 2009; 26(4): 507 - 531. [Abstract] [PDF]

				T. Eckes Rater types in writing performance assessments: A classification approach to rater variability Language Testing, April 1, 2008; 25(2): 155 - 185. [Abstract] [PDF]

				Y. Sawaki Construct validation of analytic rating scales in a speaking assessment: Reporting a score profile and a composite Language Testing, July 1, 2007; 24(3): 355 - 390. [Abstract] [PDF]