Technical Working Papers

In the course of developing review standards, conducting reviews, and producing reports, the What Works Clearinghouse (WWC) has produced working papers providing guidance for addressing difficult technical issues. By making these papers public, the WWC aims to (a) advance the state-of-the-art in the design, implementation, analysis, and reporting of causal research and (b) increase the transparency with which the work of the WWC is carried out.

What Works Clearinghouse Extent of Evidence Categorization
The Extent of Evidence Categorization was developed to tell readers how much evidence was used to determine the intervention rating, focusing on the number and sizes of studies... (more) (30 KB)

Effect Size Substantive Interpretation Guidelines: Issues in the Interpretation of Effect Sizes
When authors communicate the findings of their studies, there is often a focus on whether or not some intervention had the intended effect, and less attention to how much of an effect the intervention had. That is, it may be possible to state that some reading intervention increased reading scores more than the usual reading instructional techniques, but it is often more difficult for readers to determine how much of a difference the intervention made. Readers need to know if the intervention's effects are large or small, meaningful or trivial. These guidelines describe... (more) (151 KB)

Technical Details of WWC-Conducted Computations
To assist in the interpretation of study findings and facilitate comparisons of findings across studies, the WWC computes the effect sizes (ES) and the improvement indices associated with study findings on outcome measures relevant to the WWC's review. In addition, the WWC computes the levels of statistical significance... (more) (114 KB)

Teacher-Intervention Confound
In some studies reviewed by the WWC, only one teacher is assigned to each condition. In particular, three different kinds of studies involve only one teacher per condition. The technical guidance discusses... (more) (47 KB)

What Works Clearinghouse Improvement Index
In order to help readers judge the practical importance of an intervention's effect, the WWC translates the effect size of the intervention's effect into an "improvement index." The improvement index represents... (more) (16 KB)

What Works Clearinghouse Study Design Classification Revised
To be eligible for WWC review, a study must be a randomized trial, a regression discontinuity design, or a quasi-experiment with equating of pretest differences. The questions and examples below are meant to help WWC staff to classify properly the design of each potentially relevant... (more) (43 KB)

What Works Clearinghouse Tutorial on Mismatch between Unit of Assignment and Unit of Analysis
This brief "tutorial" describes the problem and implications of "mismatch" between units of assignment and units of analysis in a study. When units of assignment (for example, classrooms) do not match the units of analysis (for example, students) in a study of an intervention and this feature of the study's design is ignored in the study's data analysis, this can affect statements of significance and other measures of the precision of the findings. This tutorial describes... (more) (16 KB)