This Article Full Text (OnlineFirst PDF) All Versions of this Article: 0013164409332217v1 69/6/978    most recent Alert me when this article is cited Alert me if a correction is posted Services Email this article to a friend Similar articles in this journal Alert me to new issues of the journal Add to Saved Citations Download to citation manager Request Permissions Request Reprints Add to My Marked Citations Citing Articles Citing Articles via Google Scholar Citing Articles via Scopus Google Scholar Articles by Attali, Y. Articles by Powers, D. Search for Related Content Social Bookmarking                         What's this?

Validity of Scores for a Developmental Writing Scale Based on Automated Scoring

Educational Testing Service

^* To whom correspondence should be addressed. E-mail: yattali{at}ets.org.

	Abstract

A developmental writing scale for timed essay-writing performance was created on the basis of automatically computed indicators of writing fluency, word choice, and conventions of standard written English. In a large-scale data collection effort that involved a national sample of more than 12,000 students from 4th, 6th, 8th, 10th, and 12th grade, students wrote (in 30-min sessions) up to four essays in two modes of writing on topics selected from a pool of 20 topics. Scale scores were created by combining essay indicators in a standard way to compute essay scores that shared the same scoring standards across essay prompts and student grade levels. A series of ancillary analyses and studies were conducted to examine the validity of scale scores. Cross-classified random effects modeling of scores confirmed that the particular prompts on which essays are written have little effect on scores. The reliability of scores was found to be higher compared to previous reliability estimates of human essay scores. A human scoring experiment confirmed that the developmental sensitivity of scale scores and human scores was similar. A longitudinal study confirmed the expected gains in scores over a 1-year period.

First published on March 11, 2009, doi:10.1177/0013164409332217

Educational and Psychological Measurement 2009;69:978.

A more recent version of this article appeared on December 1, 2009


CiteULike    Complore    Connotea    Del.icio.us    Digg    Reddit    Technorati    Twitter    What's this?