National Board for Education Sciences 2008 Annual Report August 2007 to July 2008

On November 5, 2002, Congress passed the Education Sciences Reform Act of 2002 (ESRA), establishing the Institute of Education Sciences (IES, or the Institute) and its board of directors, the National Board for Education Sciences (NBES, or the Board). The Institute reports to Congress yearly on the condition of education in the United States. The Institute provides thorough and objective evaluations of federal programs, sponsors research relevant and useful to educators and others (such as policymakers), and serves as a trusted source of unbiased information on what works in education.

The NBES consists of voting and ex-officio members (see Appendix A). The 15 voting members of the Board are nominated by the President and confirmed by the Senate. Members Richard Milgram and Herbert Walberg completed their respective terms in November 2007. Board chair Craig Ramey finished his term in March 2008. In 2007 and 2008, the President nominated and the Senate confirmed the following individuals to serve a second term on the Board: Jonathan Baron, Carol D'Amico, F. Philip Handy, Eric Hanushek, and Sally Shaywitz. In November 2008, the Board will have nine member vacancies without taking into account nominations and pending confirmations beyond the release of this report. During this reporting period (August 2007 through July 2008), the Board held meetings in September 2007 and January and May 2008.

Under the leadership of the elected chair, Robert Granger, and vice-chair, Beth Ann Bryan, the Board currently operates with three committees: communication, legislation, and IES evaluation. Each committee deliberates on appropriate Board and Institute issues and subsequently presents its findings to the full Board for consideration.

Recognitions

In 2007, the U.S. Office of Management and Budget (OMB) awarded the research, development, and dissemination programs at IES an effective rating, the highest score on its Program Assessment Rating Tool (PART). Two other IES programs, statistics and the National Assessment of Educational Progress, had received effective scores in previous rounds of PART review. OMB found that "IES has transformed the quality and rigor of education research within the Department of Education and increased the demand for scientifically based evidence of effectiveness in the education field as a whole."

OMB's PART was developed to assess and improve the performance and results of federally funded programs. The PART looks at all factors having a bearing on program performance, including program purpose and design; performance measurement, evaluations, and strategic planning; program management; and program results. The entire U.S. Department of Education has five programs rated effective, three of which are within IES (research, development, and dissemination; statistics; and the National Assessment of Educational Progress). Overall, OMB has assessed more than 1,000 programs across the federal government, only 18 percent of which have been scored effective.

On October 15, 2007, Dr. Grover Whitehurst, the first and current Director of the Institute, received the Peter H. Rossi Award from the Association for Public Policy Analysis and Management for contributions to the development of evaluation science.

Communications and Outreach

Web-Based Dissemination

The What Works Clearinghouse continues to attract an increasing number of visits to the website in fiscal year 2008. The total number of monthly visits between October 2007 and June 2008 grew from 52,060 to 372,389, with practice guides continuing to be the most popular download. Visitors downloaded nearly 12,000 copies of the Turning Around Chronically Low-Performing Schools Practice Guide in the first month of posting in May 2008.

Education Resources Information Center (ERIC)

ERIC is the world's largest and most frequently used digital library of education, comprising more than 1.2 million bibliographic records and more than 214,000 full-text materials indexed from 1966 to the present. Funded through the Institute of Education Sciences, ERIC was modernized under a contract awarded in 2004, transitioning from a paper and microfiche collection built over the prior 35 years by 16 ERIC clearinghouses to a centralized electronic database. Prior to the modernization, approximately 7 million searches were reported in 2001. After modernization, utilization rose rapidly to 93 million searches in 2007. The entire microfiche collection of 350,000 full-text documents has been converted to digital image files, preserving the historic ERIC collection along with the digital library. ERIC is also re-distributed under terms of a license agreement, with major subscription services and Internet search engines providing access to ERIC.

Urban Education Research Task Force

IES convened the Urban Education Research Task Force to obtain counsel on a broad range of issues pertaining to research to improve urban education. Task force members include superintendents of large urban districts, leading researchers who have focused on urban education, and other education decisionmakers. The task force has met three times since January 2007 and is preparing a report to IES, Congress, and the next administration on research needs in urban education.

Society for Research on Educational Effectiveness

The Society for Research on Educational Effectiveness was organized in 2005 with IES support to provide a professional home for researchers who study the causal effects of education interventions, practices, programs, and policies. SREE held its second national conference in March 2008 in Crystal City, Virginia. The widely attended conference focused on the needs of those who must make decisions about how best to educate students and the technical features of educational research. The conference brought together a community of individuals interested in the application of scientific research methods to pressing education issues, problems, and questions.

SREE will provide an organizational capacity to disseminate and consolidate the highest research findings by sponsoring three publications.

• Journal of Research on Educational Effectiveness (JREE). JREE is a peer-reviewed journal that publishes full-length articles on research of cause-and-effect relations important for education.
• The Handbook of Research on Educational Effectiveness. Published in three volumes, The Handbook of Research on Educational Effectiveness will contain both seminal works and reports of recent developments related to the study of cause-and-effect relations important for education.
• Research Notes on Educational Effectiveness (RNEE). RNEE will serve as a rapid turnaround, peer-reviewed electronic outlet to disseminate recent research findings in the form of accessible research briefs.

National Conference of State Legislatures (NCSL)

The National Conference of State Legislatures received a 4-year grant from IES in September 2006 to support state education policy and legislative decisionmaking that is informed by scientific research. The NCSL project is organized through 10 seminars on particular topics pertinent to state legislators, starting with teacher quality and dropout prevention. Three seminars in 2007–2008 each attracted approximately 30 chairs of state education committees, and 2 seminars for state legislative staff also had full attendance.

Annual Research Conference

The Institute of Education Sciences held its third annual research conference June 10–12, 2008, in Washington, DC. Over 830 grant and contract awardees and IES staff attended. The conference was organized around the theme "Education Research: What's Now? What's Next?" and represented the broadest assemblage of IES-funded education researchers to date. The conference featured 18 panel sessions, 8 topical open forum discussions, and 130 poster presentations by predoctoral and postdoctoral fellows from training programs at 17 universities, as well as 219 posters showcasing IES-supported research. The inaugural Outstanding Predoctoral Fellow Award was also presented during this conference.

National Center for Education Statistics (NCES)

As reauthorized by Congress under the Education Sciences Reform Act of 2002 (P.L.107-279), NCES has as its responsibility to "collect, report, analyze, and disseminate statistical data related to education in the United States and in other nations."

Program Highlights

Teacher compensation study. In 2007, NCES launched a new teacher compensation study. In the fi rst year, seven states supplied NCES with compensation data for each teacher in the state. Next year, 18 states will provide data, representing more than one third of the teachers in the country. These data will enable more accurate comparisons of teacher compensation across states and at different levels of seniority and credentials.

Statewide Longitudinal Data System (SLDS) Grant Program. The SLDS was authorized by the Educational Technical Assistance Act of 2002 and is designed to aid K–12 state education agencies in developing and implementing longitudinal data systems. All 50 states, 5 territories, and the District of Columbia are eligible to apply. These are competitive, cooperative agreements that extend for 3 years and range between $1 million to $6 million per state. These data systems provide for longitudinal tracking of students through school, which facilitates the reporting of accurate graduation rates and helps identify conditions that are associated with success in school and in the workforce.

In November 2005, the first year of the grant program, IES awarded grants to 14 states and in June 2007, grants were awarded to an additional 13 states. In total, $115 million in grants have been awarded under SLDS. In this year's request for applications, the fi rst goal is to support states so they can report true cohort high school graduation rates by 2012. The second goal is to assist states in expanding existing K–12 statewide data systems to link to postsecondary data and/or workforce data.

National Assessment of Educational Progress (NAEP). Since August 2007, NAEP's accomplishments include: the release of NAEP Report Cards at 4^th and 8^th grades for reading, mathematics, and writing, including results for states and 11 districts participating in the Trial Urban District Assessment (TUDA); and a ground-breaking report on state No Child Left Behind performance standards at 4^th and 8^th grades, permitting the first-ever comparisons between and across states using NAEP as a psychometrically appropriate common scale. 

NAEP continues to prepare for major expansions. In 2007, NAEP received an additional appropriation of $10 million annually for the inclusion of seven additional large urban districts that volunteered for TUDA and to initiate the first pilot 12^th-grade state assessment in mathematics and reading. Eleven states have volunteered to participate.

College Navigator. College Navigator is a user-friendly web resource that makes it easy for students, parents, and others to learn about colleges and universities throughout the nation. It has proven to be extremely successful, winning accolades from news organizations and college counselors; most importantly, it receives heavy use by the public. The site averages close to 30,000 visits a day. A recently added Spanish-language version of College Navigator and the existing English-language version are revisited regularly to determine how they can be improved.

National Center for Education Research (NCER)

NCER supports rigorous research that contributes to the solution of significant education problems in the United States.

Overall, for fiscal year 2008, IES reviewed 851 research, research training program (predoctoral and postdoctoral), and research and development center grant applications, and funded 104 of those applications (12%). Since its inception, IES has increased significantly the number of applications and the number of grant awards.

Program Highlights

Research training programs. During 2004, 2005, and 2008, NCER awarded 13 5-year grants to launch interdisciplinary predoctoral research training programs in the education sciences to doctoral-degree-granting institutions. To date, these programs have supported 233 graduate fellows. In addition, since 2005, NCER has supported a number of postdoctoral research training programs through which 30 postdoctoral fellows have been or are being trained.

As of the summer of 2008, 39 predoctoral fellows have obtained employment following the completion of the Ph.D. Of these, 16 (41%) are employed as university or college faculty members, all but one as a full-time faculty member. Thirteen (33.3%) are employed as postdoctoral or research associates at universities, and seven (18%) have other education research positions (e.g., at nonprofit education or education research consulting firms). Only three (7.7%) fellows who have completed their Ph.D. are employed in positions that are outside the university and are not focused on education research.

In April 2008, NCER sponsored a 1-day workshop for 135 participants on evaluating state and local education programs and policies. The purpose was to explain to staff from state education agencies or districts how a rigorous impact evaluation could be embedded in new or existing programs that are not yet widely implemented.

Practice guides. NCER produced two practice guides in 2008. Developed by a panel of national experts, the practice guides bring together the best available evidence to provide educators with specific and coherent evidence-based recommendations to address a particular challenge. NCER's Encouraging Girls in Math and Science gives teachers specific recommendations on fostering girls' interest in and skill development in math and science. These recommendations can be carried out in the classroom without requiring systemic change. A second practice guide, Organizing Instruction and Study to Improve Student Learning, reflects an expert panel's consensus on some of the most important principles to emerge from research on learning and memory.

National research and development centers. The centers contribute significantly to the solution of education problems in the United States by developing, testing, and disseminating new approaches to improve teaching and learning and, ultimately, student achievement. To date, IES has funded 10 centers. In 2006, NCER funded two research and development centers that focused on education policy: the National Center for Analysis of Longitudinal Data in Education Research (CALDER) and the National Center on Performance Incentives (NCPI). Using longitudinal state and district administrative databases from Florida, Missouri, New York, North Carolina, Texas, and Washington, CALDER researchers are examining how state and local teacher policies on hiring, compensation, and certification, and state and local governance policies on accountability and choice, relate to student outcomes. NCPI is focusing on the use and effectiveness of various pay-for-performance strategies. In addition to this large randomized field trial, NCPI is conducting reviews of incentive-focused studies and smaller studies of other pay-for-performance programs.

National Center for Special Education Research (NCSER)

In December 2004, Congress reauthorized the Individuals with Disabilities Education Act (IDEA) and in doing so authorized the NCSER as part of IES. As specified in P.L.108-446, the mission of NCSER is to

• sponsor research to expand knowledge and understanding of the needs of infants, toddlers, and children with disabilities in order to improve the developmental, educational, and transitional results of such individuals;
• sponsor research to improve services provided under, and support the implementation of, the Individuals with Disabilities Education Act (20 U.S.C. 1400 et seq.); and
• evaluate the implementation of the Individuals with Disabilities Education Act in coordination with the National Center for Education Evaluation and Regional Assistance.

Program Highlights

Research workshop. In April 2008, NCSER sponsored a 2-day workshop designed to give researchers specialized training to improve the rigor of special education research using single case methodologies that incorporate quantitative analyses. Thirty-nine people attended the NCSER Single Case Design Workshop.

Research programs. NCSER is accumulating a strong portfolio of projects that are developing Response to Intervention (RTI) models or are validating measurement systems to identify students at risk for disabilities. NCSER has funded five research projects to study the development of large-scale assessments that are more accessible and appropriate for an expanded range of students, including students with disabilities. The intent is to design tests that minimize the need for accommodations and modifications, and that produce scores that can be used validly in accountability systems regardless of student disabilities.

National Center for Education Evaluation and Regional Assistance (NCEE)

NCEE is responsible for conducting rigorous evaluations of federal programs, synthesizing and disseminating information from evaluation and research, and providing technical assistance to improve student achievement.

Program Highlights

Reading First Impact Study: Interim Report. The Reading First program provides assistance to states and districts in using research-based reading programs and instructional materials for students in kindergarten through third grade and in introducing related professional development and assessments. The interim report presents the impacts of Reading First on classroom reading instruction and student reading comprehension during the 2004–05 and 2005–06 school years.

The evaluation found that Reading First had positive, statistically significant impacts on the total class time spent on the five essential components of reading instruction promoted by the program. The study also found that, on average across the 18 study sites, Reading First did not have statistically significant impacts on student reading comprehension test scores in grades 1–3. A final report on the impacts from 2004–2007 (3 school years with Reading First funding) and on the relationships between changes in instructional practice and student reading comprehension is expected in late 2008.

Enhanced Academic Instruction in After-School Programs. Enhanced Academic Instruction in After-School Programs is the first of two reports presenting implementation and impact findings after 1 year of program operation. This study tests whether interventions of structured approaches to academic instruction in after-school programs (one for reading and one for math) produce better academic outcomes than regular after-school services that consist primarily of help with homework or locally assembled materials that do not follow a structured curriculum.

The evaluation found a statistically significant difference in student achievement between students in the math after-school programs and those in the regular afterschool activities. In study sites implementing the reading program, there was no statistically significant difference in reading achievement between students in the reading after-school program and those in the regular after-school activities.

Evaluation of IES

NBES is required to prepare a report to the Director of IES, the Secretary of Education, and the appropriate congressional committees that will provide recommendations on actions that might be taken to enhance the ability of IES to carry out its priorities and mission. The Board has undertaken an evaluation of the Institute of Education Sciences with the assistance of its contractor, Synergy Enterprises, Inc.

The primary goal of the evaluation is to determine the extent to which IES has been effective in carrying out its priorities and mission. The key objectives of the evaluation are to

• provide valid and reliable evidence related to the Institute's effectiveness, progress, and overall impact, within the available timeframe and scope of work;
• provide the groundwork for future evaluations and the collection of ongoing performance data that can be used to measure the Institute's progress over time; and
• provide policy and program recommendations based on the findings to enhance the ability of the Institute to carry out its priorities and mission.

The evaluation will achieve these objectives by focusing on three central IES goals: increasing rigor, increasing relevance, and increasing utilization of Institute research. The evaluation report is designed to provide input that the Board can use in advising Congress regarding reauthorization of the Education Sciences Reform Act (ESRA).

NBES Resolutions

NBES passed the following resolutions to add value to the work of IES. (See Appendix B for a list of NBES-approved resolutions since its inception.)

Oversight Function in the Awards Process

At its January 2008 meeting, the Board resolved that: "the Board will review and advise the IES Director on grant awards where the proposed grantee is selected out of rank order of applicant scores that result from peer review for scientific merit." The intention of the motion was to create transparency regarding situations in which an exception is made. No such exception has been made to date.

Family Educational Rights and Privace Act (FERPA)

The need for research-based information has grown exponentially since the passage of the No Child Left Behind Act. In addition, Statewide Longitudinal Research Grants are producing a vast accumulation of data that could yield invaluable information about policies that would improve student achievement, including approaches to helping chronically underperforming schools and enhancing teacher effectiveness. To mine those data, researchers need access to them. A potential obstacle to this access is the Family Educational Rights and Privacy Act of 1974, also known as the Buckley Amendment. The intent of the Act is to protect the rights of students and to ensure the privacy and accuracy of education records. The Act applies to all institutions that receive federal aid administered by the Secretary of Education.

State education agencies and local school districts have been uncertain about when and how education records can be provided for research because of confusion over how FERPA applies to protecting the data.

NBES recognizes the challenges in simultaneously protecting individual privacy and accessing data for research activities. The Board supports changes that represent an appropriate balance between preserving students' privacy, promoting their safety, and facilitating research and accountability that will help ensure a quality education system. Board member Eric Hanushek summarized the issue as follows:

In order to meet the goals of NCLB and to improve the performance of our schools, we must be able to provide states and districts with information about effective policies and programs, and that requires learning from the experiences of districts that are doing a good job. The administrative data on programs and student achievement provide just such an opportunity—and one that was in fact anticipated in FERPA, which included exceptions to general restrictions on data access for research that would promote school improvements. The challenge has been implementing these provisions in ways that simultaneously provide protection for confidentiality of the individual student data.

In March 2008, the Department of Education released draft regulations designed to clarify access and use of student-level administrative data that are protected by FERPA. These regulations have gone through a formal comment period but have yet to be finalized. At the May 2008 meeting of the Board, there was an extensive discussion of FERPA, the proposed regulations, and the accompanying guidance that had been released by the Department of Education. The following resolutions related to FERPA and support of greater access to data were passed by NBES:

Action # 1

"The Board commends the Secretary and the Department of Education for moving forward in developing new regulations and guidance about how to maintain confidentiality of educational data under FERPA while also providing for research uses of student and school data. We urge the Department to finalize these regulations quickly, incorporating the major clarifications that have been submitted in comments."

Action #2

"We urge the Congress to expand on the program of supporting statewide longitudinal data systems by requiring that states accepting funding under this program agree to make data in these systems available to qualified researchers (subject to FERPA) for the purpose of research that is intended to help improve student achievement."

Next Steps

Next Board Meeting

The NBES will hold its next meeting September 9 and 10, 2008, in Washington, DC.

5-Year Report

The Education Sciences Reform Act (ESRA) is set to expire in October 2008. At the May 2008 NBES meeting, members approved recommendations to enhance ESRA upon reauthorization. In fall 2008, the Board expects to submit to the Director, the Secretary, and the appropriate congressional committees a report that will include recommendations regarding any actions that may be taken to enhance the ability of the Institute to carry out its priorities and mission, including recommendations for the reauthorization of ESRA.

The contribution that evidence-based research provides is essential to policymakers, educators, and parents for improving education and meeting the academic needs of students. As IES continues to make significant progress in transforming education into an evidence-based endeavor, the Board continues to urge Congress to support the role of IES as the primary agency for federal education research.

The Board notes with regret the language in the 2008 House Appropriations bill, H.R. 2764, that halts the use of funds to conduct the evaluation of the Upward Bound Program until September 30, 2008.¹ It would be very unfortunate to selectively exclude federally funded programs from scientific evaluation.

Chair's Message
 

Consistent with the highlights and OMB's high marks noted in this report, IES had an excellent year. With Russ Whitehurst's term as Director of IES ending, and the pending reauthorizations of NCLB and ESRA, we are entering a period during which important decisions will be made about the future of IES. In 6 short years, my view is that IES has transformed the rigor of federally funded education research. The National Board for Education Sciences is committed to seeing that the next period builds on IES's exceptional performance during its first phase.

Robert C. Granger 

Mr. Jonathan Baron
Coalition for Evidence-Based Policy Council for Excellence in Government
Washington, DC
Term expires November 28, 2011

Ms. Elizabeth C. Bryan, Vice-Chair
Akin Gump Strauss Hauer & Feld
Austin, TX
Term expires November 28, 2008

Dr. Carol A. D'Amico
Conexus Indiana
Indianapolis, IN
Term expires November 28, 2010

Dr. David C. Geary
University of Missouri
Columbia, MO
Term expires November 20, 2010

Dr. Robert C. Granger, Chair
William T. Grant Foundation
New York, NY
Term expires November 28, 2008

Mr. F. Philip Handy
Strategic Industries, LLC
Winter Park, FL
Term expires November 28, 2011

Dr. Eric A. Hanushek
Hoover Institution
Stanford University
Stanford, CA
Term expires November 28, 2010

Dr. Caroline M. Hoxby
Hoover Institution
Stanford University
Stanford, CA
Term expires November 28, 2008

Mr. Gerald (Jerry) Lee
WBEB 101.1 FM Philadelphia
Bala Cynwyd, PA
Term expires November 28, 2008

Dr. Sally E. Shaywitz
Department of Pediatrics
Yale University School of Medicine
New Haven, CT
Term expires November 28, 2011

Dr. Joseph K. Torgesen
Florida Center for Reading Research
Tallahassee, FL
Term expires November 28, 2008

Nominated Member of the National Board for Education Sciences

Ms. Joanne Weiss
NewSchools Venture Fund
San Francisco, CA
Nomination January 22, 2008

Ex-Officio Members

Director of the Institute of Education Sciences
Each of the Commissioners of the IES National Education Centers
Director of the National Institute of Child Health and Human Development
Director of the Census Bureau
Director of the National Science Foundation
Director of the Bureau of Labor Statistics

1. Congress should designate the Institute of Education Sciences, in statute, as the lead agency for all congressionally authorized evaluations of U.S. Department of Education programs, responsible for all operations, contracts, and reports associated with such evaluations. (September 2006)
2. Congress should allow the U.S. Department of Education to pool funds generated by the 0.5 percent evaluation set-aside from smaller programs. (September 2006)
3. The U.S. Department of Education should use its "waiver" authority to build scientifically valid knowledge about what works in K-12 education. (September 2006)
4. Congress, in authorizing and funding evaluations of federal education programs, should require [program] grantees, as a condition of grant award, to participate in the evaluation if asked, including the random assignment to intervention and control groups as appropriate. (April 2005)
5. Congress should create, in statute, effective incentives for federal education program grantees to adopt practices or strategies meeting the highest standard of evidence of sizeable, sustained effects on important educational outcomes. (May 2007)
6. Congress and the U.S. Department of Education should ensure that individual student data can be used by researchers (with appropriate safeguards for confidentiality) in order to provide evaluations and analyses to improve our schools. (September 2006)
7. Congress revise the statutory definition of "scientifically based research" so that it includes studies likely to produce valid conclusions about a program's effectiveness, and excludes studies that often produce erroneous conclusions. (October 2007)
8. The Board will review and advise the IES Director on grant awards where the proposed grantee is selected out of rank order of applicant scores that result from peer review for scientific merit. (January 2008)
9. The Board commends the Secretary and the Department of Education for moving forward in developing new regulations and guidance about how to maintain confidentiality of educational data under FERPA while also providing for research uses of student and school data. The Department should finalize these regulations quickly, incorporating the major clarifications that have been submitted in comments. (May 2008)
10. Congress should expand on the program of supporting statewide longitudinal data systems by requiring that states accepting funding under this program agree to make data in these systems available to qualified researchers (subject to FERPA) for the purpose of research that is intended to help improve student achievement. (May 2008)