This Article Full Text (PDF) References Alert me when this article is cited Alert me if a correction is posted Citation Map Services Email this article to a friend Similar articles in this journal Alert me to new issues of the journal Add to Saved Citations Download to citation manager Request Permissions Request Reprints Add to My Marked Citations Citing Articles Citing Articles via HighWire Citing Articles via Google Scholar Citing Articles via Scopus Google Scholar Articles by Freedle, R. Articles by Kostin, I. Search for Related Content Social Bookmarking                         What's this?

Does the text matter in a multiple-choice test of comprehension? the case for the construct validity of TOEFL's minitalks

Educational Testing Service, Princeton, NJ, rfreedle{at}ets.org

Irene Kostin

Educational Testing Service, Princeton, NJ

The current study addresses a specific construct validity issue regarding multiple-choice language-comprehension tests by focusing on TOEFL’s minitalk passages: Is there evidence that examinees attend to the text passages in answering the test items? To address this problem, we analysed a large sample (n = 337) of minitalk items. The content and structure of the items and their associated text passages were represented by a set of predictor variables that included a wide variety of text and item characteristics identified from the experimental language-comprehension literature. Stepwise and hierarchical regression techniques showed that at least 33% of the item difficulty variance could be accounted for primarily by variables that reflected the content and structure of the whole passage and/or selected portions of the passage; item characteristics, however, accounted for very little of the variance. The pattern of these results was interpreted, with qualifications, as favouring the construct validity of TOEFL’s minitalks. Our methodology also allowed a detailed comparison between TOEFL reading and listening (minitalk) items. Several criticisms concerning multiple-choice language-comprehension tests were addressed. Future work is suggested.

Language Testing, Vol. 16, No. 1, 2-32 (1999)
DOI: 10.1177/026553229901600102


CiteULike    Complore    Connotea    Del.icio.us    Digg    Reddit    Technorati    Twitter    What's this?

This article has been cited by other articles:

				M.-Y. Song Do divisible subskills exist in second language (L2) comprehension? A structural equation modeling approach Language Testing, October 1, 2008; 25(4): 435 - 464. [Abstract] [PDF]

				N. T. Carr The factor structure of test task characteristics and examinee performance Language Testing, July 1, 2006; 23(3): 269 - 289. [Abstract] [PDF]

				L. Trites and M. McGroarty Reading to learn and reading to integrate: new tasks for reading comprehension tests? Language Testing, April 1, 2005; 22(2): 174 - 210. [Abstract] [PDF]

				A. Jafarpur Is the test constructor a facet? Language Testing, January 1, 2003; 20(1): 57 - 87. [Abstract] [PDF]

				L. F. Bachman Some reflections on task-based language performance assessment Language Testing, October 1, 2002; 19(4): 453 - 476. [Abstract] [PDF]

				C. Elder Assessing the language proficiency of teachers: are there any border controls? Language Testing, April 1, 2001; 18(2): 149 - 170. [Abstract] [PDF]

				L. F. Bachman Modern language testing at the turn of the century: assuring that what we count counts Language Testing, January 1, 2000; 17(1): 1 - 42. [Abstract] [PDF]