May 23rd   May 24th

May 23rd, 2007

Location:
Room 100
80 F St., NW
Washington, DC 20208

Members Present:
Jonathan Baron
Beth Ann Bryan
Carol A. D'Amico
Robert Granger
Philip Handy
Jerry Lee
Craig T. Ramey, Chairman
Sally E. Shaywitz
Joseph K. Torgesen
Herbert J. Walberg

Members Absent:
Eric A. Hanushek
Caroline Hoxby
James Milgram

Ex Officio Members Present:
Grover J. Whitehurst, Director, IES
Phoebe H. Cottingham, Commissioner, National Center for Education Evaluation
Robert A. Kominski, Bureau of the Census
Lynn Okagaki, Commissioner, National Center for Education Research
Mark Schneider, Commissioner, National Center for Education Statistics
Duane Alexander, M.D., Director, National Institute for Child Health and Human Development

Designated Federal Official: Mary Grace Lucier

IES Staff Present:
Sue Betka, Deputy for Administration and Policy, IES
Mike Bowler
Amy Feldman
Ellie McCutcheon
Ann Ricciuti

Members of the Public Present:
Kate Bannum
James Kohlmoos, Knowledge Alliance
Lauren Gibbs, Success for All
Lin Liu, OMB
Gerald Sroufe, AERA
Cheryl White, Washington Partners

The Chairman called the meeting to order at 2 p.m. He called for a motion to approve the minutes of the last meeting; the minutes were approved without opposition along with the present meeting's agenda. The Chairman noted that this was his inaugural meeting as chairman and thanked his predecessor, Dr. Granger, Mr. Handy, the vice chairman, and Mary Grace Lucier for their assistance in preparing for the meeting.

Report of the Executive Director
Dr. Whitehurst brought the members up to date on IES activities since the last meeting in January. Among the topics covered were:

• Administration's proposed increase in 2008 budget for IES, the increase to cover longitudinal studies on high school and college and university students and to expand the 12th grade NAEP testing in reading and mathematics;
• The IES Biennial Report to Congress, about to be released;
• Reports issued by the What Works Clearinghouse (60 intervention reports issued to date, another 34 to be released over the summer);
• The revamped ERIC system, which responded to 62 million separate searches in 2006 at a substantial decrease in costs since 2003;
• The timing and release of upcoming reports, including evaluations of Early Reading First, the DC school choice scholarship program, and Title I of the Elementary and Secondary Education Act for Low Income Children.

There followed a discussion with the members about leveraging the information gained from these initiatives and using the results to help practitioners and policy makers and by including such avenues as radio talk shows to reach the general public.

Dr. Whitehurst concluded his presentation with a discussion of the Program Assessment Rating Tool (PART), a program of the Office of Management and Budget (OMB), which oversees the effectiveness of Federal programs and the metrics/progress measures which will determine the effectiveness rating of IES. He invited input from the Board on the Program Assessment Rating Tool, about which IES and the Office of Management and Budget have been in negotiations over the past year. OMB oversees the effectiveness of Federal programs and assigns a score based on a number of metrics, including the following:

• Does the program have a limited number of specific long-term performance measures?
• Are these measures ambitious?
• Are there annual performance measures towards long-term goals?
• Are there baselines and targets for the performance measures?

Dr. Whitehurst said that IES was aiming for a score of 37 for the year 2013-2014. The number is based on a review of the investments IES has made since since it was established in 2002 and projecting the results of the grants it has funded since that time. Another goal is the training of a new generation of researchers who can compete for and obtain IES funding and carry out the kind of work IES supports. A reasonable projection would be in the area of one or two thousand. The final measure would be that decision makers—perhaps as many as 25%—would routinely consult the What Works Clearinghouse before making decisions.

He went on to describe the specifics of how the progress measures might play out in the domain of reading, one of the IES research priorities and invited the Board to express its views with respect to the measures that have been proposed.

Mr. Lee offered a motion to endorse the proposal, Dr. Shaywitz offered a second. The motion passed unanimously.

Reports from IES Commissioners

Report of the Commissioner of the National Center for Education Statistics
Dr. Schneider discussed some of the work in progress at NCES, including

• The release of the history and civics report cards from the National Assessment of Educational Progress;
• The Release of the Condition of Education 2007 and other reports;
• A study of teacher certification and training by discipline in public high schools, due for release later in the summer;
• A website called College Opportunities On-Line, which will improve the ability of parents and students to compare post-secondary institutions.

Report of the Commissioner of the National Center for Education Research
Dr. Okagaki outlined a framework for the research programs now underway: The desired student outcomes are improvements in readiness for reading, writing, math, science, social skills and behaviors that support learning, and enrollment and completion of post-secondary education. The conditions of education that are under the control of the school system are curriculum, instructional practices, assessments, quality of the education work force, and system level policies, programs, and practices. These intersections have prompted research efforts in several broad domains, such as reading and writing research that addresses curriculum, instructional practice, and assessment from kindergarten to adult education. Given IES' efforts over the last 5 years, there is now a critical mass of projects in key areas that focus on the development and evaluation of curricula, instructional practices, and teacher professional development for English language learners.

NCER supports a research and development center that focuses on the education of English language learners in the middle grades, and in addition it has nearly 20 other programs for English learners.

In the area of math and science, a particular strength is early math curriculum. Early exposure to math concepts and skills during early childhood is now seen as important as early language and reading. Research is also centering on algebra as a gateway course for students, and ten projects are being devoted to algebra instruction.

Areas in which NCER is seeking to stimulate more research activity are

• Early childhood programs and practices;
• Educational technologies;
• High school reform, and
• Interventions for struggling adolescent and adult readers and writers.

Two new research programs will be devoted to cognition and science instruction and instructional technology.

As for what researchers are doing that is showing some success, three projects were cited:

• A teacher professional development training program using software that uses reading scores to develop instructional profiles that help teachers customize instruction;
• A teacher development program that uses on-site and remote coaching;
• An intervention that strengthens young children's numerical estimation skills.

Report of the Commissioner of the National Center for Education Evaluation and Regional Assistance
Dr. Cottingham described the Center's four main lines of work, the evaluation division that conducts studies requested by Congress and by programs in the Department of Education, the What Works Clearinghouse, ERIC, and the Regional Educational Laboratory program. Eight evaluation reports are coming out in 2007. The What Works Clearinghouse now offers 60-some reports in seven review areas. ERIC is in the process of obtaining authors' permission to digitize all the work presently on microfiche. To date 63,000 have responded and another 275,000 remain. Librarians are enthusiastic about this program since microfiche is prone to deteriorate. The Regional Educational Laboratories are in effect the Department of Education's extension program. They have their own regional governing boards and now have a common set of standards on what constitutes good science. Instead of the former practice of undertaking five-year research projects, there is now an arrangement that allows for fast-response work, 12 months or less turnaround, and other work that does longer studies using the best methods recognized by IES. The labs are now closely networked and have one national website, and overall have made dramatic improvements in the transparency of their work. The challenge for the labs is that the demand for their services is outstripping the supply, but they are on a fast learning curve and connecting with experts in the research centers. The labs now are becoming a virtual reference system enabling their clients to receive the best advice based on the best evidence.

The chairman thanked all the presenters for their comprehensive surveys of ongoing work and declared the meeting adjourned at 4:47 p.m.

May 24th, 2007

Location:
Room 100
80 F St., NW
Washington, DC 20208

Members Present:
Jonathan Baron
Beth Ann Bryan
Carol A. D'Amico
Robert Granger
Caroline M. Hoxby (by teleconference)
Jerry Lee
Craig T. Ramey, Chairman
Sally E. Shaywitz
Joseph K. Torgesen
Herbert J. Walberg

Members Absent:
Philip Handy
Eric A. Hanushek
James Milgram

Ex Officio Members Present:
Grover J. Whitehurst, Director, IES
Phoebe H. Cottingham, Commissioner, National Center for Education Evaluation
Lynn Okagaki, Commissioner, National Center for Education Research
Mark Schneider, Commissioner, National Center for Education Statistics

Designated Federal Official: Mary Grace Lucier

IES Staff Present:
Sue Betka, Deputy for Administration and Policy, IES
Mike Bowler
Amy Feldman
Ellie McCutcheon
Ann Ricciuti

Members of the Public Present:
Andrea Browning, SRCD
Connie Citro, CNSTAT/NAS
Kim Krocker, CEC
Gerald Sroufe, AERA

The Chairman called the meeting to order at 9:08 a.m.

The Chairman briefly reviewed the day's agenda and, in anticipation of later discussions, asked the members to consider augmenting the present committee structure with some ad hoc arrangements and to consider giving attention to issues concerning upcoming authorizations of No Child Left Behind and other key pieces of legislation.

Executive Director Search
Dr. Whitehurst said the IES statute needed to be amended to provide hiring authority to the Board. For the former executive director, the Board communicated its choice to Dr. Whitehurst who then used his excepted service authority to hire the person. There followed a short discussion that recapped the steps that were followed to find and recruit this person. Several members expressed a sense of urgency to get a director in place. Dr. Walberg moved to authorize the chairman of the Board in consultation with the Director of IES to recruit and hire an executive director so as to expedite the work of the Board. This recruitment should be done with due speed and diligence. Dr. Torgesen seconded the motion. The Chairman called for a vote and the motion carried.

Board Reporting Requirements
The Board's is required to submit a report five years after the enactment of its legislation, i.e., in November 2007, which includes its recommendations regarding actions that would enhance the ability of the Institute to carry out its priorities and mission. Due to the fact that the Board had no director in place since January 2007, contract actions for this report were unable to be completed in a timely manner and so the report most likely will encounter some delay, as will the Board's annual report assessing the effectiveness of the Institute. A discussion ensued as to ways to expedite the reporting process, possibly a small ad hoc committee to work with the executive director and the chair to produce a working draft for submission to the Board. Dr. Torgesen and Dr. Shaywitz agreed to work with Mr. Baron and the chairman to this end. Dr. Walberg offered to review the document(s) and make editorial suggestions.

Presentation of Mark Dynarski
The Board had an interactive presentation from researcher Mark Dynarski from Mathematica on his recently released, IES-funded study of the Effectiveness of Reading and Math Software Products. The study may be accessed at the IES website at http://ies.ed.gov/ncee/pubs/20074005/index.asp.

Committee Reports:

The Board heard the following committee reports:

NCEE
John Baron offered a recommendation to promote effective incentives for Federal education program grantees to adopt practices or strategies meeting the highest standard of evident of sizable, sustained efforts on important educational outcomes. See attachment for full text.

The recommendation was offered as a motion by Dr. Walberg and seconded. After a short colloquy on how it would be communicated to the Congress and possibly applied, the question was called and the motion passed without opposition.

NCER
Dr. Walberg expressed gratification about the education practice series and expressed the desire of the committee to supplement their committee with a person who has expertise in science education. The convener of the committee, Dr. Milgram, would report to the committee at greater length at the next meeting. A discussion ensued as to the policy ramifications and legal requirements, and a decision was made to pursue this issue at the next meeting.

NCSER
Dr. Shaywitz said that forthcoming competitions for centers in special education would focus on serious behavior disorders at varying grade levels and on responses to intervention in older children. In the coming year, the Center will field nine research competition topics and a postdoctoral training grant, and it is expected that programs designed for interventions in special education would be added to the What Works Clearinghouse. Papers have been commissioned on methodological statistical support. She expressed the committee's thanks to Dr. Ed Kame'enui, the departing commissioner of the Center, for the many wide-ranging projects he created and for encouraging a young and promising staff.

Further Business
The Chairman asked the Board if it would like to entertain the idea of structuring some task forces or working groups or other cross-cutting entities that would further the Board's mission in light of the upcoming reauthorizations and identify areas where there is an opportunity to provide recommendations. Dr. Granger suggested a topic that would lend itself to such an arrangement, the use of evidence in policy and practice. Dr. Hoxby suggested that there might be an opportunity for input in the nexus of IES and the reauthorization of No Child Left Behind. In this regard, Dr. Baron would like to see an improved definition of "scientifically based research." Dr. Shaywitz felt that a working group on dissemination would be desirable. Ms. Bryan expressed an interest in receiving bimonthly updates from the commissioners on what is happening in the Centers. Mr. Lee thought it would be advisable to reach out and talk to research organizations to get their views on where they feel the biggest "payoff" is in the field of education.

Dr. Granger made the point that it would not be necessary specifically to authorize the chair to reflect the Board's views. He recommended that the Board authorize the chair to establish two new committees, one on Federal legislation and the other on the connection between scientific evidence and policy makers and practitioners, with the understanding that a more formal motion and discussion be made at the next meeting. This was moved and seconded; the motion carried.

Dr. Ramey asked for names of persons to act as chairs or conveners of the new committees. Dr. D'Amico nominated Ms. Bryan to head up the Federal legislative group. Dr. Shaywitz nominated Mr. Lee for the dissemination group. The chairman then appointed these individuals to take on these duties.

The Chairman adjourned the meeting at 2:00 p.m.

Attachment: Full text of Policy Recommendation

Policy Recommendation
Approved by the National Board for Education Sciences
May 24, 2007

That Congress create, in statute, effective incentives for federal education program grantees to adopt practices or strategies meeting the highest standard of evidence of sizeable, sustained effects on important educational outcomes.

The Problem: Federal education programs, set up to address important problems, often fall short by funding specific practices or strategies ("interventions") that are not effective.

When federally-funded educational interventions have been evaluated in scientifically-rigorous studies, the studies typically find many ineffective or marginally effective, and a few even harmful. Those interventions found in rigorous studies to produce meaningful, sustained effects on important outcomes—such as academic achievement, grade retention, dropout rates, post-secondary enrollment, and employment and earnings—tend to be the exception. This general pattern occurs in many diverse areas of education— such as dropout prevention, literacy programs, after-school programs, educational technology, school choice, and substance-abuse prevention—as well as other fields in which rigorous studies have been carried out (e.g., medicine, psychology, welfare and employment, crime and justice).

The Opportunity: Research has identified a few interventions meeting the "top tier" of evidence—i.e., well-designed randomized controlled trials showing sizeable, sustained effects on important outcomes.

Perhaps only about 10 such top-tier interventions now exist in the field of education, in areas such as dropout prevention, early reading, schoolwide reform, school-based substance-abuse prevention, and vocational and adult education.

A Possible Incentive: Establishment of a modest-sized competitive grant program to replicate and scale up top-tier interventions, and leverage other funds to support such replication.

1. This recommendation is patterned on the evidence-based nurse visitation initiative in the President's FY 08 budget request. That initiative provides $10 million for a new competitive grant program at the Department of Health and Human Services to fund nurse visitation activities that meet the top tier of evidence.
2. The competitive grant program would award funds to organizations that:
1. Implement a top-tier intervention in any area of education—where "top-tier" might be defined in statute as including interventions shown, in well-designed randomized controlled trials conducted in typical school or community settings, to produce sizeable, sustained improvements in important educational or life outcomes. (Such a showing could be based on a What Works Clearinghouse review or other evidence.)
2. Adhere closely to the specific elements of the intervention.
3. Obtain sizeable matching funds for their project from other federal or non-federal sources that can appropriately fund the intervention, such as federal Title I, Special Education, Safe and Drug-Free Schools, Career and Technical Education, or Adult Education grants, or state or local funding sources. (Congress may need to make clear that funds from these larger federal programs can be used for this purpose.)
The program would thus be designed to provide seed funding for the replication and scale-up of top-tier interventions—funding which would leverage money from the larger sources described above.
3. The program would include rigorous evaluations of the funded projects, where appropriate, to ensure that the interventions remain effective when replicated on a large scale.

Another Possible Incentive: Give priority consideration, in competitive grant programs, to applicants that propose to implement a top-tier intervention (defined as above).

Specifically, in areas of education were top-tier interventions exist, Department programs that make competitive grant awards would give priority consideration (such as 10 additional points out of a possible 100) to grant applicants that propose to implement such an intervention, and ensure close replication of its specific elements.

Conclusion: Rigorous research has identified a few interventions that are very effective in preventing reading failure, substance abuse, dropping out of school, workforce failure, and other outcomes that damage millions of American lives each year. We recommend that Congress provide effective incentives to replicate such interventions, and put them into widespread use.

The National Board for Education Sciences is a Federal advisory committee chartered by Congress, operating under the Federal Advisory Committee Act (FACA); 5 U.S.C., App.2). The Board provides advice to the Director on the policies of the Institute of Education Sciences. The findings and recommendations of the Board do not represent the views of the Agency, and this document does not represent information approved or disseminated by the Department of Education.