The What Works Clearinghouse (WWC) is expanding its research design standards to cover single case studies and regression discontinuity designs. Both sets of new standards reflect IES's desire to expand coverage in the clearinghouse of credible evidence on the effectiveness of programs, policies, and practices as much as possible.
Single case studies are used heavily in special education research to evaluate the effectiveness of interventions for low-incidence population groups, such as blind students. In this type of study, an intervention participant acts as his/her own control, with the treatment being applied and withdrawn, repeatedly. Throughout the study, researchers make continuous assessments of an individual and use that data to gauge whether a specific intervention was effective.
The addition of the regression discontinuity design standards allows WWC to assess the evidence from studies that capitalize on situations in which decisions about who gets an intervention are based on their score on a continuous criterion variable relative to a threshhold. For example, students scoring below the 30th percentile on the state reading test might be assigned to mandatory summer school. In a regression discontinuity study, impacts are estimated by comparing outcomes for cases just above and just below the threshold.
Early experience of the WWC applying these standards suggests that a majority of existing studies using each type of design may not meet standards. However, as the WWC gains experience applying these new standards and the researchers using the standards become familiar with the standards and their application, we expect that the quality of and reporting on studies using these designs to improve.
Visit the WWC website to learn more about these new study design standards.
A recent Government Accountability Office report recommending some improvements to the What Works Clearinghouse provided some perspective on state and school district awareness of WWC activities.
The report, Improved Dissemination and Timely Product Release Would Enhance the Usefulness of the What Works Clearinghouse, contained findings from a survey of state education agencies and school districts. They were reported in a review GAO recently presented to the Department of Education.
Established by the Institute of Education Sciences in 2002, the What Works Clearinghouse acts as a source for scientific evidence in education. The WWC produces user-friendly practice guides for educators, assesses the rigor of research evidence on the effectiveness of interventions such as curricula and other programs, and develops and implements standards for education research.
The findings around awareness and use include:
The key recommendations in the study have already been implemented or are in the process of being implemented. IES hopes to have all aspects of these recommendations fully implemented early in fiscal year 2011.