This Article Full Text (OnlineFirst PDF) All Versions of this Article: 0013164408323237v1 69/3/404    most recent Alert me when this article is cited Alert me if a correction is posted Services Email this article to a friend Similar articles in this journal Alert me to new issues of the journal Add to Saved Citations Download to citation manager Request Permissions Request Reprints Add to My Marked Citations Citing Articles Citing Articles via Google Scholar Citing Articles via Scopus Google Scholar Articles by Romano, J. L. Articles by Kromrey, J. D. Search for Related Content Social Bookmarking                         What's this?

What Are the Consequences If the Assumption of Independent Observations Is Violated in Reliability Generalization Meta-Analysis Studies?

^* To whom correspondence should be addressed. E-mail: jromano{at}aus.edu.

	Abstract

This study was conducted to evaluate alternative analysis strategies for the meta-analysis method of reliability generalization when the reliability estimates are not statistically independent. Five approaches to dealing with the violation of independence were implemented: ignoring the violation and treating each observation as independent, calculating one mean or median from each study, selecting only one observation per study, and using a mixed-effects model. Monte Carlo methods were used to simulate samples under known and controlled population conditions. The results suggest that the type of approach does not have a noticeable impact on the accuracy of the reliability results but that researchers should be cautious when the intraclass correlation is relatively large. The simulations in this study also resulted in very poor confidence band coverage.

First published on September 3, 2008, doi:10.1177/0013164408323237

Educational and Psychological Measurement 2009;69:404.

A more recent version of this article appeared on June 1, 2009


CiteULike    Complore    Connotea    Del.icio.us    Digg    Reddit    Technorati    Twitter    What's this?