This Article Full Text (PDF) All Versions of this Article: 0013164408318762v1 69/1/62    most recent References Alert me when this article is cited Alert me if a correction is posted Services Email this article to a friend Similar articles in this journal Alert me to new issues of the journal Add to Saved Citations Download to citation manager Request Permissions Request Reprints Add to My Marked Citations Citing Articles Citing Articles via Google Scholar Citing Articles via Scopus Google Scholar Articles by Nugent, W. R. Search for Related Content Social Bookmarking                         What's this?

Construct Validity Invariance and Discrepancies in Meta-Analytic Effect Sizes Based on Different Measures

A Simulation Study

University of Tennessee Knoxville, wnugent{at}utk.edu

Critical to meta-analysis is the presumption that effect sizes based on different measures are directly comparable. Recent theoretical work has shown that an invariance condition—universe score, or construct, validity invariance—must hold for either observed score or reliability-corrected effect sizes based on different measures to be directly comparable. To date, however, no research has been done investigating how crucial violations of this invariance condition are for either effect size discrepancy across different measures or the outcomes of a meta-analysis. This article reports the results of a simulation study of the possible effects that violations of construct validity invariance have on (a) the differences in population construct level standardized mean difference effect sizes based on different measurement procedures; (b) the variability in population construct level correlation effect sizes based on different measurement procedures; and (c) the results of a meta-analysis. Results suggest that considerable variability in effect sizes can exist across measurement procedures that fail to meet universe score validity invariance and that this variability has the potential to negatively affect meta-analytic results.

Key Words: effect sizes • meta-analysis • effect size comparability • effect size invariance • construct validity and meta-analysis • construct validity and effect sizes

This version was published on February 1, 2009

Educational and Psychological Measurement, Vol. 69, No. 1, 62-78 (2009)
DOI: 10.1177/0013164408318762


CiteULike    Complore    Connotea    Del.icio.us    Digg    Reddit    Technorati    Twitter    What's this?