Advanced Search

Journal Navigation

Journal Home

Subscriptions

Archive

Contact Us

Table of Contents

Click here for more information on Research and Evaluation in Education and Psychology, 3e

CiteULike is a free service for managing and discovering scholarly references - click here to get started.

Sign In to gain access to subscriptions and/or personal tools.
Educational and Psychological Measurement
This Article
Right arrow Full Text (PDF)
Right arrow References
Right arrow Alert me when this article is cited
Right arrow Alert me if a correction is posted
Services
Right arrow Email this article to a friend
Right arrow Similar articles in this journal
Right arrow Alert me to new issues of the journal
Right arrow Add to Saved Citations
Right arrow Download to citation manager
Right arrowRequest Permissions
Right arrow Request Reprints
Right arrow Add to My Marked Citations
Citing Articles
Right arrow Citing Articles via Google Scholar
Right arrow Citing Articles via Scopus
Google Scholar
Right arrow Articles by Shermis, M. D.
Right arrow Articles by Harrington, S.
Right arrow Search for Related Content
Social Bookmarking
Add to CiteULike   Add to Complore   Add to Connotea   Add to Del.icio.us   Add to Digg   Add to Reddit   Add to Technorati   Add to Twitter  
What's this?

Trait Ratings for Automated Essay Grading

Mark D. Shermis

Florida International Universitymshermis{at}fiu.edu

Chantal Mees Koch

Indiana University Purdue University Indianapolis

Ellis B. Page

Duke University

Timothy Z. Keith

University of Texas at Austin

Susanmarie Harrington

Indiana University Purdue University Indianapolis

This study employed an automated grader to evaluate essays, both holistically and with the rating of traits (content, organization, style, mechanics, and creativity) for Webbased student essays serving as placement tests at a large Midwestern university. The authors report the results of two combined experiments, based on random selection from 1,193 essays. In the first experiment, the essays of 807 students were used to create statistical predictions for the essay-grading software. In the second experiment, the ratings from a separate, random sample of 386 essays were used to compare the ratings of six human judges against those generated by the computer. The interjudge correlation of the human raters alone was r = .71. But the interrater reliability of all six judges in combination with computer scoring reached .83. The essay-grading software was an efficient means for evaluating the essays, with a capacity for grading approximately six documents every second. Other potential feedback measures for use in writing courses are also discussed.

Educational and Psychological Measurement, Vol. 62, No. 1, 5-18 (2002)
DOI: 10.1177/0013164402062001001
Add to CiteULike CiteULike   Add to Complore Complore   Add to Connotea Connotea   Add to Del.icio.us Del.icio.us   Add to Digg Digg   Add to Reddit Reddit   Add to Technorati Technorati   Add to Twitter Twitter    What's this?