| Title: | Variability in Pretest-Posttest Correlation Coefficients by Student Achievement Level |
| Description: | State assessments are increasingly used as outcome measures for education evaluations. The scaling of state assessments produces variability in measurement error, with the conditional standard error of measurement increasing as average student ability moves toward the tails of the achievement distribution. This report examines the variability in pretest-posttest correlation coefficients of state assessment data for samples of low-performing, average-performing, and proficient students to illustrate how sample characteristics (including the measurement error of observed scores) affect pretest-posttest correlation coefficients. As an application, this report highlights how statistical power can be attenuated when correlation coefficients vary according to sample characteristics. Achievement data from four states and two large districts in both English/Language Arts and Mathematics for three recent years are examined. The results confirm that pretest-posttest correlation coefficients are smaller for samples of low performers, reducing statistical power for impact studies. Substantial variation across state assessments was also found. These findings suggest that it may be useful to assess the pretest-posttest correlation coefficients of state assessments for an intervention’s target population during the planning phase of a study. |
| Online Availability: |
 Need Help Viewing PDF files? |
| Cover Date: | September 2011 |
| Web Release: | September 7, 2011 |
| Publication #: | NCEE 20114033 |
| Center/Program: | NCEE |
| Authors: | Russell Cole, Joshua Haimson, and Irma Perez-Johnson: Mathematica Policy Research. Henry May, University of Pennsylvania Consortium for Policy Research in Education |
| Type of Product: | Evaluation Report |
| Keywords: | |
| Questions: |
For questions about the content of this Evaluation Report, please contact: Erin Pollard. |