National Board for Education Sciences 5-Year Report, 2003 Through 2008

In a 1999 National Research Council report, the committee wrote:

One striking fact is that the complex world of education—unlike defense, health care, or industrial production—does not rest on a strong research base. In no other field are personal experience and ideology so frequently relied on to make policy choices, and in no other field is the research base so inadequate and little used.*

Others, including members of Congress, shared the view that education research had not provided education policymakers and practitioners with the information and tools they needed to improve education in our country. When the Institute of Education Sciences (Institute) was established in November 2002, many in the education research and policy community cried déjà vu. As some observed, why would anyone expect the Institute to accomplish what its predecessors—the National Institute of Education and the Office of Educational Research and Improvement—had not? On the basis of its evaluation, the members of the National Board for Education Sciences (Board) conclude that in a relatively brief period of time, the Institute has made exceptional progress in improving the rigor and relevance of education research in our nation. Under the leadership of its first director, Grover J. Whitehurst, the Institute has accomplished what many believed could not be done.

The framework for the Institute’s nonideological, high-quality work was wisely established by Congress in the Education Sciences Reform Act of 2002 (ESRA). Because that Act has generated such strong results, the Board is recommending its rapid reauthorization, with a set of modest amendments meant to improve its clarity and make it even stronger. The Board recognizes that transformation of education into an evidence-based field is an enormous task. It will need to involve everyone from federal and state policymakers to local education leaders, administrators, teachers, and parents. Over the past 6 years, a new direction has been set for education research. We now need to stay on course to arrive at this destination.

The Institute was established to provide rigorous evidence on which to ground education practice and policy. In its first 6 years, the Institute has made exceptional progress in improving the rigor of education research and evaluation. By design, the goal of improved rigor has driven the majority of staff activity to date, under the assumption that the threshold condition for making education an evidence-based field is producing findings that can be trusted. The Board concurs with the strategic decision to emphasize rigor. Our judgment of exceptional progress is based on the following evidence: (1) the high standards reflected in the peer review system; (2) the strong external ratings of the quality of the funded research grants; and (3) the high quality of the research designs of the evaluations contracted through the Institute’s National Center for Education Evaluation and Regional Assistance (NCEE). To build the nation’s capacity to conduct rigorous education research, the Institute has launched predoctoral and postdoctoral research training programs. Based on the GRE scores of the predoctoral fellows and the research productivity of the predoctoral and postdoctoral fellows, as evidenced by the numbers of research publications to date, the Institute has made substantial progress in developing a new generation of education scientists who are well-equipped to conduct high-quality research. Finally, in light of concerns that the Institute has narrowly focused its research funding on projects that use experimental methods to evaluate the effectiveness of interventions, the Board examined the Institute’s funding announcements and diversity of the research grant portfolio with respect to research questions and research methods employed. The Board found that the Institute clearly requests research projects that are diverse in purpose (e.g., exploring malleable factors, developing and validating assessments) and in methodological requirements (e.g., correlational, descriptive, observational, quasi-experimental, and experimental methodologies) and has developed a diverse research portfolio in which roughly one-fourth of the projects are experimental or quasi-experimental evaluations of the impact of interventions on education outcomes.

No matter how technically sound research activities might be, if they do not address the issues and questions that are of concern to education policymakers and practitioners, the research will not be used to inform education policy and practice. The Institute has made substantial progress in improving the relevance and usefulness of education research, evaluation, and statistics. This is an area that has drawn increased attention from staff as the rigor of the work improved, and the Board agrees with this strategic shift. It should be continued in the future. Our judgment of substantial progress is based on the following: (1) satisfactory ratings of the relevance of funded research projects by education leaders and administrators; (2) establishment of long-term focused programs of research that address fundamental education issues in our nation (e.g., improving reading, writing, mathematics, and science achievement); (3) the work of the National Research and Development Centers in key policy areas; (4) dramatic improvements in the timeliness of the release of data from the Institute’s National Center for Education Statistics (NCES); (5) high ratings of the relevance of NCES reports; and (6) increased efforts to improve the timeliness of the release of NCEE evaluation reports.

Generating rigorous and relevant research is a necessary step to the transformation of education into an evidence-based field, but the Institute cannot stop there. The Institute must disseminate the knowledge it produces in ways that enable education policymakers and practitioners to use that information. In this arena, the Institute has also made substantial progress, particularly in the last 2 years, and this area should continue to receive sustained staff attention in the future. Our finding of substantial progress is based upon the quality and use of the systems and programs the Institute has created (e.g., What Works Clearinghouse [WWC], College Navigator) or revamped (e.g., Education Resources Information Center [ERIC]) to disseminate practical information to education leaders, practitioners, parents, and students. The WWC practice guide Organizing Instruction and Study to Improve Student Learning has been downloaded nearly 30,000 times since its release last September; the Turning Around Chronically Low-Performing Schools practice guide has been downloaded nearly 20,000 times since its release in May 2008. The College Navigator was rated by Money Magazine as one of the best places to start a college search. Our finding was also supported by examination of the programs the Institute has established to help education leaders and decisionmakers obtain better data and become better consumers of education research, such as the Statewide Longitudinal Data Systems, the National Forum on Education Statistics, and the NCES Summer Data Conference.

Despite the Board’s view that the Institute has been instrumental in improving the rigor, relevance, and accessibility of federally funded educational research, much remains to be done to institutionalize the gains made and build on them. For this reason, the full report includes recommendations from the Board regarding the details of the Institute’s work and the reauthorization of the ESRA. However, the main message from our evaluation is that going forward, the Institute should maintain the direction that Congress articulated and that the Institute’s leadership and staff have executed so well.