Submissions

• Does a study need to be published to be submitted to the WWC for review? Does the WWC only accept studies on interventions that are being reviewed?
• How can I submit a study for review, and what is the time frame for the review of the study?
• When an individual study is submitted for review, does the WWC search for other studies about the same intervention?
• Can a study be submitted to more than one review at the same time? Can studies be submitted to the WWC and concurrently to other registries?
• Is there a minimum number of studies required in order to nominate an intervention for review?
• Are all studies submitted to the Registry of Randomized Controlled Trials (RCTs) reviewed by the WWC? How can I know if my collaborators already submitted a study?

Does a study need to be published to be submitted to the WWC for review? Does the WWC only accept studies on interventions that are being reviewed?

The WWC accepts individual studies that are publicly available, even if they have not been published in a peer-reviewed journal or other similar format. The WWC Help Desk welcomes the submission of studies for review through the “Contact Us” form on the “About Us” page.

If the WWC has begun a review that could include the study, it is sent to the review team. When a topic area is ready to prioritize the next set of interventions for review, the review team incorporates all submitted studies and nominated interventions into the prioritization process (see Question A2 for more information on the prioritization process), supplementing the submitted information with a comprehensive literature search. If a study is submitted for an intervention that has recently been prioritized for review for an intervention report, then the study is likely to be reviewed much sooner than a study for an intervention that has not been prioritized for review. Studies that do not fit within an existing review effort are cataloged internally to ensure they can be easily indentified for future reviews. Currently, the broad topics under review are literacy, early childhood education, special education, postsecondary education, and teacher quality. Individual studies that have received substantial media attention or are sponsored by IES may also be reviewed more quickly as single study reviews.

How can I submit a study for review, and what is the time frame for the review of the study?

Studies can be submitted for review through the WWC Help Desk using the “Contact Us” form on the “About Us” page. The WWC prioritizes the education topics and interventions being reviewed on an annual basis; therefore, it is difficult to estimate when a review of a submitted individual study might be completed.

When an individual study is submitted for review, does the WWC search for other studies about the same intervention?

The WWC may search for other studies about the intervention depending upon the product for which the individual study may be reviewed. For example, if the individual study is of an intervention that is the current focus of a review, it will be added to the existing literature search for that intervention. If the individual study is of an intervention that has not been prioritized for review for an intervention report, then a literature search will be conducted when the review is started. If the individual study is of an intervention that does not have an active topic area, it will be catalogued and included in literature searches when the topic area is active. An individual study may also be considered for a single study review, in which case no additional literature searches are conducted.

Can a study be submitted to more than one review at the same time? Can studies be submitted to the WWC and concurrently to other registries?

Yes, studies can be submitted to more than one review effort, including the WWC, at the same time. Within the WWC, an individual study can be included in more than one review effort. For example, if a study examines a comprehensive intervention for young children, it could be reviewed by both the elementary math area and the beginning reading area. When a topic area is ready to prioritize the next set of interventions for review, teams incorporate all submitted studies into the prioritization process, supplementing the submitted information with a comprehensive literature search. Studies can be submitted to the WWC Help Desk using the “Contact Us” form on the “About Us” page. The WWC will review studies independent of whether the studies have been submitted to other registries.

Is there a minimum number of studies required in order to nominate an intervention for review?

No. The WWC welcomes all intervention nominations and study submissions, which will be considered when the WWC sets priorities for the review efforts annually and when interventions are prioritized within a review effort. Intervention nominations and studies can be submitted to the WWC Help Desk using the “Contact Us” form on the “About Us” page.

Are all studies submitted to the Registry of Randomized Controlled Trials (RCTs)reviewed by the WWC? How can I know if my collaborators already submitted a study?

The Registry of Randomized Controlled Trials (RCTs) is the WWC’s online database of completed and in-progress RCTs in education. Researchers and developers submit their information to be included in the searchable database. The information they submit includes the name of the principal investigator, the start date and anticipated end date, details about the sample and the study design, and the primary outcome. To find out whether an RCT has been submitted to the registry, you can search based on keywords or authors’ names.

The registry of RCTs is included in all WWC literature searches, and relevant studies in the registry are reviewed by the WWC. However, the WWC does not automatically review all studies submitted to the registry. To find reviewed studies, you can search the studies database, which includes citations for nearly 10,000 studies that have been reviewed. The WWC has no central searchable repository of studies that have not been reviewed.

Study Designs

• Are there study designs other than randomized controlled trials that demonstrate causal evidence?
• How are the findings from different types of study designs (that is, RCTs, QEDs, RDDs, and SCDs) combined? Are they weighted?
• Will the WWC review any additional types of study designs in the near future?
• What is the best possible rating for a study that does not use random assignment?

Are there study designs other than randomized controlled trials that demonstrate causal evidence?

Yes. A well-executed RCT is frequently described as the gold standard, but it is not the only design that provides causal evidence of effectiveness. The WWC reviews four types of study designs than can provide causal evidence according to WWC standards: randomized controlled trials, quasi-experimental designs (QEDs), regression discontinuity designs (RDDs), and single-case designs (SCDs). Consult the WWC Procedures and Standards Handbook for additional details on how these types of designs are rated.

How are the findings from different types of study designs (that is, RCTs, QEDs, RDDs, and SCDs) combined? Are they weighted?

As long as the study meets standards, with or without reservations, RCTs and QEDs are treated similarly in the reporting of study details and findings. Effect sizes from these designs are combined using an unweighted average. The design may play a role in determining an intervention rating, as the highest rating of “positive effects” can only be obtained if at least one study meets WWC standards without reservations. To date, the WWC pilot standards for regression discontinuity and single-case designs have only been applied to judge evidence from individual studies. The WWC has not determined whether or how findings from studies with these designs will be incorporated into reports that combine findings across studies.

Will the WWC review any additional types of study designs in the near future?

Initially the WWC only considered RCTs and quasi-experimental comparison group designs. A pilot set of standards for SCD studies is being used to review studies with reversal/withdrawal designs, multiple baseline or multiple probe designs, and alternating treatment designs. A second set of pilot standards is being used for reviewing RDD studies. Consult the WWC Procedures and Standards Handbook for additional details on how these types of designs are rated. At this time, the WWC does not intend to add additional study designs to those eligible for review under WWC standards.

What is the best possible rating for a study that does not use random assignment?

Nonexperimental comparison group studies (such as QEDs) can receive, at best, a rating of meets WWC group design standards with reservations. The primary reason is that all nonexperimental comparison group studies include a risk of bias in the effect due to how the sample was selected. Some regression discontinuity and single-case design studies can meet WWC pilot regression discontinuity or single-case design standards without reservations, provided the study was designed and implemented in a manner that addresses potential causes of bias. Consult the WWC Procedures and Standards Handbook for additional details on how these types of designs are rated.

Reviews

• How many studies does the WWC review in a typical year?
• Can developers hire the WWC to conduct independent reviews?
• How do you become a certified reviewer? Is there a need only for reviewers with education backgrounds? What about reviewers from other fields?
• How many people are involved in a typical study review, and how long does a typical study review take to complete?
• What is the process for the external review of intervention reports? How are external peer reviewers selected?

How many studies does the WWC review in a typical year?

Although literature searches can identify thousands of studies, the WWC only reviews studies with eligible designs (RCTs, QEDs, RDDs, and SCDs). The number of studies reviewed varies based on the availability of IES resources for the WWC and review efforts being conducted in a given year. The WWC reviews several hundred studies each year.

Can developers hire the WWC to conduct independent reviews?

The WWC is a federally funded initiative overseen by the U. S. Department of Education’s Institute of Education Sciences (IES). Although developers are encouraged to submit studies to the WWC for review, the WWC does not conduct any reviews on behalf of developers. As part of the WWC’s comprehensive search of the literature, when conducting a systematic review, the WWC will contact developers when research on their intervention is being reviewed to ensure that we have identified all relevant research on their intervention. As described in the WWC Procedures and Standards Handbook, interventions with more rigorous research receive priority for WWC reviews.

How do you become a certified reviewer? Is there a need only for reviewers with education backgrounds? What about reviewers from other fields?

The WWC sponsors reviewer certification trainings for people to become certified reviewers. When an upcoming training is available to the public, it is announced on the WWC website, along with information on the application process. Applicants provide information about their education and qualifications, which helps the WWC select candidates who are likely to complete the certification process successfully. The WWC attempts to meet the demand through annual training offerings. The WWC does not require candidates to have a degree in education, although they should have training in research design and methods. Trainees have to pass a knowledge test and conduct a practice study review in order to become certified reviewers.

To receive the latest announcements from the WWC website, please sign up for the News Flash email updates from IES. Visit this link, choose the National Center for Education Evaluation and Regional Assistance (NCEE) option (the second one), and select “What Works Clearinghouse.”

How many people are involved in a typical study review, and how long does a typical study review take to complete?

Two to three individuals are involved in each study review. If the first reviewer determines the rating is meets WWC group design standards with/without reservations (or could meet with additional information), then a second review by a different reviewer is conducted. A third reviewer (with substantial WWC experience) confirms the rating of the first two reviews and reconciles any differences. If the first reviewer determines the study rating is does not meet WWC group design standards, a senior reviewer checks to confirm the rating. If the rating is confirmed, the review ends after these two reviewers. If the senior reviewer does not confirm the rating, it is forwarded for subsequent review. All regression discontinuity and single-case design studies are reviewed by at least three reviewers (two initial reviews and a verification review by a third, senior reviewer). Additionally, IES staff and external peer reviewers review WWC products, including whether WWC standards and procedures were followed in the review and reporting of individual studies.

The length of time to complete a study review will vary. It might be completed quickly, or it could require an author query to obtain additional information and/or consultation with WWC management. Individual reviews may take 30 minutes or a number of hours. It could take less than a day or a number of weeks to finalize the review of an individual study.

What is the process for the external review of intervention reports? How are external peer reviewers selected?

All WWC reports are subject to several layers of quality assurance. For each report, a final round of review is conducted by peer reviewers external to the WWC (for some products, these reviewers are WWC-certified but do not work for the WWC). Peer reviewers are certified WWC reviewers and are given assignments through a “blind” process. The WWC staff conducting reviews does not know who the peer reviewers are. WWC staff must address or otherwise respond to all comments received through the external peer review process. For additional details about the layers of quality assurance on the WWC, consult the WWC Procedures and Standards Handbook.

Handbook/Standards

• What is the status of the WWC Procedures and Standards Handbook Version 3.0?
• Who do I contact if I have comments on the WWC standards?
• Does the WWC have statistics on the percentage of studies reviewed that met standards, met standards with reservations, or didn't meet standards?
• Are there WWC evidence standards regarding the fidelity of implementation of interventions in studies, and is this information included in WWC reports?
• Research shows that randomization with a small sample can create groups that are not equivalent. It is clear from the WWC Handbook that a small RCT with low attrition wouldn’t be required to demonstrate equivalence. Does the sample size affect the study rating? Does a study need to have a minimum sample size to meet WWC standards?

What is the status of the WWC Procedures and Standards Handbook Version 3.0?

The most recent update of the WWC Procedures and Standards Handbook—Version 3.0—is still in progress. The final release date for this updated version of the Handbook will be in early 2014. An IES newsflash will announce when the updated handbook is released. To sign up for the IES newsflash, please visit this link, choose the National Center for Education Evaluation and Regional Assistance (NCEE) option (second one), and select “What Works Clearinghouse.” Readers who want to provide feedback on the Handbook, or the WWC in general, may contact the WWC Help Desk using the “Contact Us” form found under “About Us” on the WWC website.

Who do I contact if I have comments on the WWC standards?

You may submit comments on the WWC standards using the “Contact Us” form found under “About Us” on the WWC website. If you have specific questions about how the WWC standards were applied in a particular product, please use the “Contact Us” form to describe your questions or concerns; be sure to identify the product.

Does the WWC have statistics on the percentage of studies reviewed that met standards, met standards with reservations, or didn't meet standards?

To date, the WWC has identified nearly 10,000 studies for use in its products. Just under half of those were determined to be eligible for review, and approximately 7% met standards. In recent years, as WWC standards have become publicly available, researchers have begun to design studies to meet WWC standards.

Are there WWC evidence standards regarding the fidelity of implementation of interventions in studies, and is this information included in WWC reports?

There are no evidence standards regarding fidelity of implementation. However, because understanding implementation fidelity is important to understanding the impact of an intervention, the WWC summarizes the implementation details included in studies when reporting on an intervention. Currently, the WWC does not have any plans to develop standards for implementation fidelity, for two main reasons.

First, most of the effectiveness research reviewed by the WWC does not take place in a laboratory environment; rather, the research examines the impact of interventions implemented in a classroom environment with competing demands for teachers. This environment provides the best test for whether the intervention is effective when applied in a real-world setting. These competing demands may lead to variation in implementation, but understanding this variation is critical information for educators. For example, if a curriculum was not implemented with fidelity because it was too difficult or too resource intensive, that should be part of the description of the research. A principal should not be misled into thinking he or she could expect an implementation of the intervention that was not observed in practice.

Second, many authors report little, if any, information on implementation. Requiring those studies to report on fidelity would eliminate useful information on the effectiveness of an intervention. Ideally, the WWC would like to report better information on implementation and systematically indicate whether implementation was strong or weak using objective measures. The WWC recommends all study authors read the study author guide for suggestions on information about implementation and fidelity that could be included in the study report.

Research shows that randomization with a small sample can create groups that are not equivalent. It is clear from the WWC Handbook that a small RCT with low attrition wouldn’t be required to demonstrate equivalence. Does the sample size affect the study rating? Does a study need to have a minimum sample size to meet WWC standards?

As long as there is more than one unit in each condition, sample size does not currently affect the study rating. The WWC standards require more than one unit in each condition to limit the possibility that a factor other than the intervention causes the reported effect. For example, if there is a single school in each condition, the effects may be due to the intervention, another initiative in the intervention school, or an initiative in the comparison school. The small sample size will affect the statistical significance of the findings, and significance is reported and incorporated into the WWC evidence rating. Although a low-attrition RCT is not required to adjust for baseline differences, the WWC prefers to report impacts adjusted for baseline differences, as they are more precise estimates of impacts. If the study reports an analysis that statistically controls for the baseline differences, the WWC will use those results in the review. If the study does not report an analysis that statistically controls for baseline differences, but does report baseline means and standard deviations, the WWC will make a difference-in-differences adjustment to account for baseline differences.

Eligibility

• Does a study have to be published in a peer-reviewed journal to be considered for WWC review?
• Are studies that are conducted by intervention developers eligible for review?
• Why does the WWC typically not review studies that are more than 20 years old, and should older studies be discussed?
• Will the WWC agree to not review a study based on an author request?
• Does the intervention being examined by an intervention report have to be used in the treatment condition in a study to be eligible for review?
• How does the WWC review research that spans different topic areas?

Does a study have to be published in a peer-reviewed journal to be considered for WWC review?

Studies do not have to be peer-reviewed to be eligible for review. The WWC is comprehensive in its systematic review process and includes any type of publicly available research, including dissertations and working papers.

Are studies that are conducted by intervention developers eligible for review?

Yes. The WWC does not have any inclusion criteria based on who conducted or funded a study.

Why does the WWC typically not review studies that are more than 20 years old, and should older studies be discussed?

At the beginning of each WWC review effort, the time frame for the literature search is set to go back 20 years in order to focus on the most relevant research, as educational environments and individual interventions change over time. For example, study of a supplemental math curriculum conducted 30 years ago would likely compare students receiving supplemental instruction to students not receiving only the core curriculum. The impact of the supplemental instruction is calculated by comparing these two groups. Because schools’ core curricula have changed considerably over the past 30 years, the impact of the supplemental curriculum may have been larger 30 years ago than it is today. By focusing on more recent research, we improve the chances that the findings of the review are relevant to today’s classroom teachers. This limit has the additional benefit of ensuring efficient use of the federal resources supporting WWC reviews; the resources needed to extend all WWC reviews into older research would reduce the total number of interventions that the WWC could review each year, and few older studies used methods that would meet current standards.

The WWC updates review protocols but does not change the time frame for the literature search. For example, the WWC Evidence Review Protocol for Beginning Reading (version 2.1) was released in August 2012 and has a literature search time frame starting in 1983, which reflects the start of this review in 2003.

Will the WWC agree to not review a study based on an author request?

The WWC has received requests from authors to not review an individual study. The reasons for the request vary, including failure of the intervention group to implement the entire intervention, or the comparison group receiving a similar version of the intervention. The WWC will not agree to not review an individual study for any of these reasons.

The WWC aims to be the central and trusted source of scientific evidence for what works in education. To achieve this goal, the WWC reviews all relevant studies in the public domain. To ensure that WWC products incorporate all the relevant literature, we cannot honor requests from study authors to exclude or suppress individual studies. If the study reports or the WWC learns about methodological or implementation issues, the review will take those factors into consideration when determining whether a study meets standards and is an appropriate evaluation of an intervention.

Does the intervention being examined by an intervention report have to be used in the treatment condition in a study to be eligible for review?

No. The WWC will also review studies in which the intervention being examined was labeled the comparison condition. An individual study that compares two interventions (for example, Math Curriculum A and Math Curriculum B) may be included in multiple products—one intervention report may focus on Math Curriculum A while another intervention report may focus on Math Curriculum B. The WWC accepts a broad range of conditions as comparisons, including whatever instruction students normally receive (business as usual), an alternate intervention, or no instruction. In each study that meets standards, the WWC reports the intervention and comparison conditions as implemented, as well as any implementation information that the authors report. This documentation helps WWC users interpret the study findings.

How does the WWC review research that spans different topic areas?

Each WWC topic area is guided by a specific protocol. Among other things, the protocol specifies the characteristics of studies that are eligible for review. For example, protocols will include specific guidance on the types of students that must be included in the study (that is, age/grade range and general education and/or special education). Some studies can be eligible for review under multiple topic area protocols. For example, studies examining the effects of the Building Blocks curriculum are eligible for review under the Early Childhood Education review protocol and the Teaching Math to Young Children practice guide review. In these cases, subsequent reviews will use the most recent review as a starting point, making any changes necessary due to differences in the protocols guiding the reviews. The landing page for each report identifies related reports reviewed under different protocols and provides links to the reports.

Quality Assurance

• What happens when an issue is referred to the WWC Quality Review Team?
• If an error is discovered in an intervention report, will the WWC mention the change in the revised intervention report?

What happens when an issue is referred to the WWC Quality Review Team?

An issue is referred to the WWC Quality Review Team when a question is raised regarding the application of WWC standards and procedures in a particular product about individual studies. The issue is assigned to a quality assurance reviewer who undertakes an independent assessment of the studies in question. The WWC-certified reviewer, who was not involved in the previous review, examines the initial study review materials and determines whether procedures were applied correctly or an error was made. The quality review team’s report is then reviewed by WWC leadership and used to respond to the inquirer. If report revisions are required, the inquirer is also notified when the revised report is available. The timing for these steps will vary depending on the complexity of the issue.

If an error is discovered in an intervention report, will the WWC mention the change in the revised intervention report?

Yes. If an error is identified in any report, the report is updated, with indications of when and why the modification was made.

Presentation of Findings

• How does the WWC handle reviews of interventions implemented over multiple years (that is, multiyear interventions)? When a study reports multiple outcomes at different points in time, how does the WWC prioritize which outcomes to report?
• What information does the WWC provide to help generalize the findings of individual studies to a broader population of schools and districts?
• Why are the findings of the WWC reviews sometimes different from findings in published meta-analyses?
• If the WWC’s interpretation of findings differs from authors’ interpretation of findings, why are the authors not given an opportunity to respond to the WWC’s interpretation?
• In some areas, it is accepted practice to extrapolate results from a short study. Will the WWC extrapolate findings from short studies?
• Is there a minimum number of studies or students that need to be reviewed to determine the effectiveness rating of an intervention?
• How can a meta-analysis, or other systematic reviews, inform a reader about the effectiveness of an educational intervention? Why not just look at individual studies?

How does the WWC handle reviews of interventions implemented over multiple years (that is, multiyear interventions)? When a study reports multiple outcomes at different points in time, how does the WWC prioritize which outcomes to report?

When interventions are implemented over several years, outcomes can be measured multiple times (for example, at the end of each school year). If an intervention has multiple outcomes, only one outcome is used as the primary finding in the report, and only that outcome will be used when determining the intervention’s rating. The choice of the primary outcome varies across areas, and the review protocols establish the priority given to reporting outcomes. Some areas, such as those focused on behavior, give priority to outcomes reported immediately, whereas others, such as dropout prevention, may give priority to outcomes after an extended period of time. Specific guidance on these priorities can be found in the “Types of Outcomes to Be Included” section of the review protocols. Note that even though the WWC review focuses on the primary outcome identified in the protocol, information on the other outcomes is presented in supplemental tables of the intervention report.

What information does the WWC provide to help generalize the findings of individual studies to a broader population of schools and districts?

In addition to assessing the internal (causal) validity of individual studies that is reflected in a study rating, the WWC reports contextual details that help readers understand the relevance of the findings. For each study that meets standards, the WWC reports the study setting, sample, implementation, and the instruction, if any, received by students in the comparison group. The intervention reports also include an extent of evidence categorization that cautions readers when findings are based on small samples or an individual study.

Why are the findings of the WWC reviews sometimes different from findings in published meta-analyses?

Two of the most common reasons the findings from a WWC systematic review might differ from those of other reviews are the scope of the review and standards for research. Each review effort by the WWC specifies the particular age of the students, types of interventions, and types of outcomes of interest. Other review efforts may define the scope of the review differently. Second, the WWC reviews research against a set of standards for high quality research and reports findings only from studies that meet those standards. Other reviews might have different standards, and thus, include studies that the WWC would not include in its review.

If the WWC’s interpretation of findings differs from authors’ interpretation of findings, why are the authors not given an opportunity to respond to the WWC’s interpretation?

The WWC might report study findings differently than an author did for a few reasons. As described in the WWC Procedures and Standards Handbook, the WWC will make three adjustments to findings. Two of the adjustments can result in the WWC reporting a different statistical significance than the study author. The first adjustment makes a post hoc adjustment to the p-value to account for the clustering of data if the treatment was assigned at the cluster level and the data were analyzed at the student level without accounting for the clustered nature of the data. The second adjustment corrects for multiple comparisons (outcomes or groups) in a single domain. The final adjustment might change the magnitude of the effect. The WWC might make an adjustment for baseline differences if the study authors did not. In addition to these adjustments, the WWC reports Hedges’ g using the pooled standard deviation as the denominator, whereas study authors might use different effect sizes or other ways of presenting findings. Any adjustments the WWC makes, and any differences between the WWC-calculated finding and the author-reported finding, are documented in the publication. If study authors believe there are errors in the application of standards, they may submit questions to the WWC Help Desk using the “Contact Us” form found under “About Us” on the WWC website.

In some areas, it is accepted practice to extrapolate results from a short study. Will the WWC extrapolate findings from short studies?

The WWC reports only on the actual findings as presented in the publication. The WWC does not extrapolate findings, even from short studies in areas in which it is accepted practice.

Is there a minimum number of studies or students that need to be reviewed to determine the effectiveness rating of an intervention?

The WWC has criteria for each of the possible effectiveness ratings (positive, potentially positive, no discernible, mixed, potentially negative, or negative). To be rated positive or negative, there must be at least two studies that meet standards, with at least one receiving the top rating of meets WWC group design standards without reservations. The other effectiveness ratings can be achieved based on one or more studies. The number of students does not affect the effectiveness rating. The numbers of both students and studies contribute to the extent of evidence rating. For additional details on effectiveness ratings, consult the WWC Procedures and Standards Handbook.

How can a meta-analysis, or other systematic reviews, inform a reader about the effectiveness of an educational intervention? Why not just look at individual studies?

With very few exceptions, the findings of an individual study apply narrowly to the sample used and the setting in which the study was conducted. Additionally, the estimated effect of an intervention may be higher or lower than the “true” effect in any one study. Therefore, looking at the results of just one study may present an incomplete picture of the intervention’s broader expected effect. A meta-analysis or systematic review based on multiple studies that vary with respect to environment and population may give more accurate information to inform educational decisions.

A meta-analysis, or other systematic review, summarizes the findings of all studies that meet the review protocol requirements. The findings of each meta- analysis reflect the scope of the review, and two reviews on the same topic but with different scopes may yield different results. Like a systematic review, a meta-analysis can reflect a comprehensive literature search. Meta-analyses and systematic reviews help readers understand the body of evidence for an intervention without having to identify and read each individual study.

Using the WWC

• Does the WWC maintain a database or archive of study reviews?
• How can users find information on the effectiveness of interventions for students?
• Does the Registry of Randomized Controlled Trials (RCTs) include trials that have begun as well as completed trials? Does the WWC require RCTs to register before outcomes are observed?

Does the WWC maintain a database or archive of study reviews?

The WWC maintains a study database of all WWC-reviewed studies. The database is searchable by keyword, topic, publication, and rating.

How can users find information on the effectiveness of interventions for students?

Individuals searching for information on specific outcomes or the effectiveness of particular interventions can use the “Find What Works” feature on the WWC website. A summary table will be generated that provides the improvement index, effectiveness rating, and extent of evidence for each intervention. It will also provide the research context for the findings and a brief description of the intervention. Users can click on “Research Details” to export a summary as an Excel file.

Searches and results can be further narrowed by grade, special population, delivery method, or program type, or limited to the interventions that show evidence of effectiveness. Users can click on a particular intervention name to find the intervention report, and they can access the full reference list from the report.

Does the Registry of Randomized Controlled Trials (RCTs) include trials that have begun as well as completed trials? Does the WWC require RCTs to register before outcomes are observed?

An RCT can be added to the WWC Registry of RCTs whether it is just starting or completed. We ask that researchers update their entries when the RCT is completed. The WWC does not require preregistration of RCTs, but appreciates authors who register their trials in the early stages to enhance transparency.

Organization

• What is the relationship between the Regional Educational Laboratories (RELs) and the WWC?

What is the relationship between the Regional Educational Laboratories (RELs) and the WWC?

The REL Program and the WWC are both federally funded initiatives housed within the National Center for Education Evaluation and Regional Assistance. Both efforts provide educators with research-based assistance, but they achieve these goals through different means. The WWC conducts comprehensive reviews of scientific evidence and creates products that summarize the research. The RELs conduct and support a variety of high quality studies on key regional education priorities and work directly with educators to provide technical assistance and training on the latest research. Each REL has at least two WWC-certified reviewers.