REL Peer Review Process

Quality Assurance for Applied Research and Development Products

Peer review is used to evaluate applied research and development products in virtually all fields of science-from medical studies that examine the effectiveness of drugs to articles about the discovery of new planets or ancient species. Scientists use the peer review process to ensure that the information they report is accurate, relevant, original, and supported by appropriate evidence. Simply put, peer review is a method by which scientists who are experts in a particular field examine another scientist's work to verify that it makes a valid contribution to the evidence base. With that assurance, a scientist can report his or her work to the public, and the public can trust the work.

All Regional Educational Laboratory (REL) applied research and development products are required to undergo rigorous external peer review. This ensures that all applied research and development products meet the Institute of Education Sciences (IES) standards for scientifically valid research before being published as online applied research and development products on the REL website at http://ies.ed.gov/ncee/rel. In this way, policymakers and practitioners, the primary users of REL applied research and development products, can be assured that these applied research and development products have met high standards for scientific quality, and that the information in the applied research and development products is valid and reliable, and therefore can be trusted.

Research that is published without undergoing rigorous peer review may not be vetted for the appropriateness of the data, sampling procedures, process used for data collection, analytic methods, and/or how the interpretations are drawn. Such applied research and development products may reach conclusions that are not supported by reliable evidence, oftentimes suggesting policy prescriptions that go far beyond the actual evidence. As a result, policymakers and practitioners in their role as decision-makers can be faced with confusing claims in publications that are not peer reviewed. Without peer review, it is often difficult for the non-researcher to sort valid factual claims supported by evidence from unwarranted conclusions based on opinion or an unscientific approach. The bottom line is this: peer review matters—it takes the guesswork out of interpreting research findings.

Peer review is a critical component of the quality assurance process at IES. All completed major REL products is subjected to peer review prior to public release. This involves review by content and methodology experts who have no conflicts of interest related to the product being reviewed. The peer review focuses on relevance and significance of the product, the data and analysis methods used, and the presentation of the goals, data and methods, and findings. Published products are expected to meet the following criteria:

Relevance and significance:

The study addresses a well-defined question that can be investigated empirically
The findings are relevant, accessible, and useful to the target audience
The study uses and builds on the existing research base and offer new knowledge

Data and methods:

The data sources are clearly and completely described and appropriate for addressing the research questions
The methods used in the analyses are clearly identified, appropriate for the research questions, and implemented correctly
The strengths and limitations of the data and methods are clearly stated

Presentation of findings:

The results of the study provide credible answer to the research questions
The findings are presented accurately and objectively
Major limitations of the study findings are well documented