Skip Navigation
A gavel National Board for Education Sciences Members | Director's Priorities | Reports | Agendas | Minutes | Resolutions| Briefing Materials
National Board for Education Sciences
October 30-31, 2007 Minutes of Meeting
October 30th   October 31st

October 30th, 2007

Location:
The Washington Court Hotel
525 New Jersey Ave., NW
Washington, DC 20001

Board Members Present:
Jonathan Baron, Council for Excellence in Government
Robert C. Granger, William T. Grant Foundation
F. Philip Handy, Strategic Industries
Caroline Hoxby, Stanford University
Gerald Lee, WBEB 101.1 FM Philadelphia
Richard James Milgram, Stanford University
Craig T. Ramey, Georgetown University
Sally E. Shaywitz, M.D., Yale University
Joseph K. Torgeson, Florida Center for Reading Research
Herbert J. Walberg, Stanford University

Ex Officio Members Present:
Grover J. Whitehurst, Director, Institute of Education Sciences
Phoebe H. Cottingham, Commissioner, National Center for Education Evaluation
Lynn Okagaki, Commissioner, National Center for Education Research
Arden L. Bement, Jr., National Science Foundation

Staff Present:
Norma Garza, Executive Director, NBES
Mary Grace Lucier, Designated Federal Official

IES Staff Present:
Sue Betka
Jack Buckley
Raji Jakai
Bruce Friedland
Allen Ruby
Linda Marshall
Mike Bowler
Anne Ricciuti

Also Present:
Mark Dynarski, Mathematica Policy Research, Inc.
Cora Marrett, National Science Foundation
Lucy Gettman, Reading Recovery Council of North America
Robert Spillane, CNA Corporation
Tiffany Taber, U.S. Department of Education
Kate Bannan, Knowledge Alliance
Karen Studwell, APA
David Van Dyke, Washington Partners
Jim Kohlmoos, Knowledge Alliance
Larry Showhite, Houghton Mifflin
Andrea Browning, Society for Research in Child Development
Allison Cole, Office of Management and Budget
Margie Mancillas, University of Texas Brownsville

Call to Order

Chairman Ramey called the meeting to order at 2:09 p.m. and called for a motion to approve the minutes of the previous meeting. The motion was made and seconded, and it carried unanimously. He called for a motion to approve the meeting agenda. The motion was made and seconded, and it carried unanimously.

Chairman Ramey commented that he had met with Secretary Spellings to discuss the Board and the IES research and evaluation agenda and had suggested to her that individual children be tracked in the No Child Left Behind data, so developmental changes can be differentiated from demographic changes. The Board has restructured, now consisting of 3 committees: Communication, Legislation, and IES Evaluation. Norma Garza is the new NBES Executive Director. The members expressed interest in having former Board members participate in a consulting capacity.

Report of the Executive Director

Norma Gaza thanked the Board and reported on recent NBES activities. Since the last meeting, she helped publish and disseminate the July 2007 Board report, which was released in October. It was sent to Congress, the IES director, and the Secretary of Education. She and Board members have arranged meetings with Congress members and staff. There have been challenges to the evaluation process: security clearances for contractors and conflict of interest (COI) clearance for experts. Additionally, she has been representing the Board at events, including the U.S. Chamber's Institute for a Competitive Workforce, and she will attend a dyslexia conference in Dallas in November.

Update on Institute of Education Sciences

Grover J. Whitehurst, Director of the Institute of Education Sciences (IES), highlighted IES-wide issues. IES has seen many successes recently. First was an effective rating from the Office of Management and Budget (OMB) for research and dissemination. Only 18 percent of programs are rated effective by the OMB, and IES has 3 of the 5 total effective-rated programs in the Department of Education. IES has the highest rating in the Department. He recognized Deputy Director Sue Betka's management specifically in accomplishing this goal.

A task force on urban education research, chaired by Michael Casserly, has been established and had its second meeting in Atlanta. The task force is establishing a list of priorities for engagement in collaborations in urban districts. Issues include increasing rigor, identifying high-performing teachers, and low-performing schools. The goal is to establish consensus on a research agenda and collaborate with urban districts in pursuing the agenda.

The annual IES research conference in June was widely attended and had good presentations by IES grantees. A biennial conference on education research for practitioners is in planning stages.

The American Competitiveness Council issued a report involving federal support for STEM education and a review of the effectiveness of the investments. Very little research demonstrates that the investment is making gains. This caused the formation of an education subcommittee to evaluate STEM investments, and progress has been substantial. The Department continues to work on the What Works Clearinghouse (WWC). The Department's Office of Planning, Evaluation, and Policy Development (OPEPD) is using IES' practice guides for the Doing What Works website, which is directed toward practitioners.

Peer review for the research competitions' second round of submissions is completed. There were 1,000 applications this year. Dr. Ramey suggested that the Board take an active and constructive role in the competitions, pursuant to the legislation. Dr. Whitehurst said the Board has review authority over all grants. He offered to discuss restructuring the Board to address the work if the Board so chose.

College Navigator, a research tool for students investigating college opportunities, is now online. Ms. Hoxby commented on the value of the site.

Early Reading First, a discretionary grant program for preschools, was rigorously evaluated. Also evaluated was DC Choice, the District of Columbia voucher program, which was enthusiastically received but has not yet, in the first year, shown an advantage. The annual NCES report, On the Condition of Education, was released June 1. The National Assessment on Reading and Math received coverage and interest, since people want to see if No Child Left Behind is working.

Three Practice guides were released: on English Learners, on girls in math and science, and on using cognitive psychology principles to manage instruction. More practice guides are in the works. WWC has issued about 30 intervention reports. There followed a discussion of IES' role in the practice guides. IES does not make NAEP recommendations for the National Assessment of Educational Progress (NAEP). The National Assessment Governing Board (NAGB) sets policy, and NCES assesses and generates a report. Dr. Milgram requested that Board members receive copies of the validity studies prior to release. The Board discussed NAEP. Dr. Hoxby noted that researchers have complained about NAGB's choice of covariates. Dr. Buckley, associate commissioner of NCES, said NAGB and NCES could work together on what should be reported. Dr. Ramey suggested that IES closely establish its expectations prior to writing and disseminating findings, which would influence the field. Dr. Whitehurst discussed the power analysis required of all studies. It is important to understand in advance what question is to be answered, rather than searching for outcome measures to find results. Mr. Lee suggested implementing special research on changing behaviors, using the practice guides. Dr. Ramey suggested that IES publicize criteria and standards for research. Dr. Whitehurst accepted the charge.

Challenges facing IES are primarily legislative. Longitudinal postsecondary data remains difficult to collect, and there was no legislative support for a multi-state pilot to track individual students to get better data. Suspected reasons are privacy, cost to institutions, and fear of government intrusion in private universities. There is not yet an FY08 budget. The proposed budget increase is about $580 million, a $30-35 million increase. House Appropriations has suggested that funding will be reduced for the WWC, which they consider duplicative. Under the America Competes Act, Education Data for the 21st Century will competitively fund states to establish P-16 data systems. There is no research provision and no appropriation. IES has already administered Statewide Longitudinal Data Grants in 27 states.

Congress is in the process of defining "scientifically-based research." There are currently two definitions, one from NCLB (No Child Left Behind) and the other from ESRA (Education Sciences Reform Act). The ESRA definition is the more restrictive of the two. In the Upward Bound evaluation, Senate passed legislation saying the Secretary is to spend no more money on the project until Congress approves the design. A previous evaluation showed no effect from the program, and the Department questioned the focus and implementation.

Dr. Ramey noted that Dr. Whitehurst was slated to receive the Peter H. Rossi Award from the Association for Public Policy Analysis and Management for contributions in the evolution of evaluation science.

Update on IES Center Activity, NCES

Jack Buckley, associate commissioner, reported on the work of the National Center for Education Statistics (NCES). In September, NCES released the results of the 2007 National Assessment of Education Progress for Reading and Mathematics, grades 4 and 8. Students showed higher improvements in Math (a 27 point improvement for 4th graders and 17 point improvement for 8th graders since 1990) than in Reading (2 points since 2005and 4 points since 1992 for 4th graders, 3 points since 1992 for 8th graders). Dr. Milgram noted that many math exam questions are problematic. Mr. Buckley said a validity study is looking at that but is not yet released. Changing the content frameworks in the NAEP would alter the trend data. However, general trends in altered and unaltered content areas remain the same.

NCES released Mapping the 2005 State Proficiency Standards onto the NAEP Scale in June. The report compared state standards, and the variation among the states was wide. Most state standards fell between basic and proficient on the NAEP scale. In August, NCES released the results of the first economic assessment, looking at 12th graders. The 2007 NAEP Trial Urban District Assessment (TUDA) in mathematics and reading will be released in November. The writing assessment will be released in the spring. Dr. Hoxby said the College Navigator website should include selectivity data, which is information highly sought by students. Dr. Buckley noted that NCES does not have a selectivity index, though the site is constantly being updated to improve data and user friendliness. Dr. Whitehurst noted that SAT and ACT scores are included, so selectivity can be compared in that manner. Mr. Lee suggested an optional toolbar that the users can put on their sites.

The day of the meeting, NCES released the first report on preschool from the Early Childhood Longitudinal Study Birth (ECLSB) cohort. NCES will release two key international assessments in the next two months: PIRLS, assessing reading literacy of fourth graders, and PISA, the OECD assessment of science literacy primarily in 15-yearolds. Findings on the Trends in International Math and Science (TIMSS) have been completed, and findings will be released in December of 2008.

"Interpreting Twelfth Graders' NAEP Scaled Mathematics Performance Using High School Predictors and Postsecondary Outcomes from the National Education Longitudinal Study of 1988" has attracted attention. The report relates math results on the NAEP metric to correlative math achievements. It is one of the first attempts to assess the predictive value of NAEP achievement levels. Dr. Walberg commented on flaws in the NAEP, including the time to get results, the lack of incentive for the schools, and other psychological motives. Dr. Buckley said a study on motivation in 12th graders is underway. Dr. Whitehurst noted that many 12th graders opt out of the test.

In June, the Statewide Longitudinal Data System Grant Program was expanded to add 13 more states. Total funding for the round was $62 million.

NCES staff and commissioners met with researchers and staff from the National Center for the Analysis of Longitudinal Data and Education Research (CALDER) at the Urban Institute to look at linking state K-12 longitudinal data with survey data from a planned study (a high school longitudinal study that starts in 2009) and to ECLS-K.

Update on IES Center Activity, NCEE

Dr. Phoebe Cottingham reported from the National Center for Education Evaluation and Regional Assistance (NCEE). NCEE consists of two divisions: Evaluation and Knowledge Utilization. Since the last NBES meeting, the Evaluation Center has published two reports: the final report on the National Evaluation of Early Reading First and a report on the DC Opportunity Scholarship Program, looking at impacts after one year.

Eleven evaluation reports are expected to be released in FY08: the National Assessment of Title I No Child Left Behind, which will go to Congress; the Reading First Interim Report; the Enhanced Reading Opportunities: Early Impact and Implementation report, looking at reading programs for 9th graders; the After School Programs: Academic Instruction report, a study in remedial math; the final Even Start Classroom Literacy Interventions and Outcomes Study report, a four-arm Study; a report on Teacher Induction; a Teacher Preparation report looking at alternative certification; the Professional Development in Early Reading Instruction report; the DC Scholarship Fund Program: Two Years report, looking at the voucher program; Enhanced Reading Opportunities: Two Years, looking at adolescent literacy; and the final report on Reading and Math Software.

A new contract was awarded to Mathematica to continue the WWC for five years. Under the first contract, there were 89 reports covering all seven areas and the standards and review system was established. There are four IES Practice Guides from that contract to be released soon: English Language Learners in Elementary Grades, Dropout Prevention, Adolescent Literacy, and Turnaround School Strategies. Since June 26th, the Regional Educational Laboratory Program has released 26 Issues and Answers reports from the Fast Response Projects.

Dr. Walberg suggested that Director and Commissioner reports be delivered prior to the meeting in a written form with hyperlinks to related documents so that the Board members could be better prepared for discussion.

Update on IES Center Activity, Research Centers

Dr. Okagaki reported on the research centers, starting with the Special Education Research Center, which funded 38 new research projects in FY07. The Early Intervention/Early Childhood Special Education Research Program has 14 projects. Nearly 300,000 infants and toddlers received early intervention services under IDEA, and five research teams are working on assessments and interventions for toddlers with disabilities. Five research teams are developing and evaluating the efficacy of speech and language interventions for preschool children with disabilities.

Another research program focuses on improving the quality of Individualized Education Programs and Individualized Family Service Plans through practitioner training and interventions. There are six projects in the program. Additionally, there are single subject research projects developing and testing interventions. An IES research institute is being planned to train researchers.

The National Center for Education Research held its first research training institute in June on cluster randomized trials. It was well-attended and rigorous. There are plans to hold another next year. Dr. Hoxby asked at what level NCER was trying to train researchers. Dr. Okagaki said the researchers have basic statistical skills, so the teaching is at a conceptual level. Dr. Hoxby asked to see the curriculum.

CALDER held its first public conference in October in Washington. The Center for Research on Educational Achievement in Teaching of English Language Learners held its first conference in October, targeting primarily practitioners. The National Center on Performance Initiatives posted three new working papers over the summer, including one on performance incentives.

The FY07 awards have been finalized. There is a growing body of research on vocabulary instruction and text comprehension, two areas identified as being in need by the National Reading Panel. There are programs on preschool children, elementary, middle, and high school children, as well as adult learners in basic education settings. Many projects have a specific English language learner focus. The teacher quality program has 21 projects and focuses on professional development. Dr. Okagaki stressed the importance of having well-trained researchers as IES program officers for their technical assistance to applicants and grantees and their ability to link projects and programs. The officers are very proactive toward getting applications and the best researchers and to develop successful research teams.

A new initiative, growing out of discussions with the Urban Education Research Task Force, to evaluate intervention models for turning around low performing schools is in development. Mr. Lee suggested looking into training school boards. Dr. Okagaki said it will be considered in the next education policy financing systems request for applications.

Adjournment

Dr. Ramey thanked the commissioners and called for a motion to adjourn. It was moved and seconded and carried unanimously, and the meeting adjourned at 4:57 p.m.

October 31st, 2007

Minutes of Meeting, Open Session

Location:
The Washington Court Hotel
525 New Jersey Ave., NW
Washington, DC 20001

Board Members Present:
Jonathan Baron, Council for Excellence in Government
Robert C. Granger, William T. Grant Foundation
F. Philip Handy, Strategic Industries
Caroline Hoxby, Stanford University
Gerald Lee, WBEB 101.1 FM Philadelphia
Richard James Milgram, Stanford University
Craig T. Ramey, Georgetown University
Sally E. Shaywitz, M.D., Yale University
Joseph K. Torgeson, Florida Center for Reading Research
Herbert J. Walberg, Stanford University
David Geary, University of Missouri

Ex Officio Members Present:
Grover J. Whitehurst, Institute of Education Sciences
Phoebe H. Cottingham, Institute of Education Sciences
Lynn Okagaki, Institute of Education Sciences
Duane Alexander, M.D., National Institute of Child Health and Human Development
Arden L. Bement, Jr., National Science Foundation

Staff Present:
Norma Garza, Executive Director, NBES
Mary Grace Lucier, Designated Federal Official

IES Staff Present:
Jack Buckley
Bruce Friedland
Sue Betka
Linda Marshall

Also Present:
Secretary Margaret Spellings, Department of Education Mark Dynarski, Mathematica Policy Research, Inc.
Cora Marrett, National Science Foundation
Joan Ferrini-Mundy, National Science Foundation
Holly Kuzmich, Department of Education
Stephen Baldwin, Synergy Enterprises, Inc.
*Patricia Muller, Center for Evaluation and Education Policy
Carmen Garcia Caceres, University of Texas, Brownsville
Reynaldo Ramirez, University of Texas, Brownsville
Lucy Gettman, Reading Recovery Council of North America
Ashley White, American University
Larry Showhite, Houghton Mifflin
Allison Cole, Office of Management and Budget
Margia Mancillas, UT Brownsville
Amanda Gilmore, American University
Gerald Sroufe, AERA
Jim Kohlmoos, Knowledge Alliance
Ashley Kranz, SRCD
Michelle White, Washington Partners, LLC
Bridget Yodens, American University
*present via teleconference

Call to Order

Dr. Ramey called the meeting to order at 8:43 a.m. and welcomed the new member, David Geary.

Discussion of the Evaluation of IES

Dr. Baldwin presented on the background of his company, Synergy Enterprises Inc. (SEI), a contractor primarily to HHS and the Department of Education. It provides IT support, logistics management, research and evaluation, and other support activities. SEI is working on the IES evaluation and a project on policy issues and community colleges for the Office for Vocational and Adult Education. He introduced Dr. Patricia Muller.

Dr. Muller gave background on the Center for Evaluation and Education Policy (CEEP), which is at Indiana University. Dr. Baldwin said assessment will focus on IES' performance in three categories: rigor, relevance, and utilization. The utilization data is lacking, since hits to the What Works Clearinghouse (WWC) website do not indicate actual adoption. However, the evaluation is limited to existing data and will use data from IES and the national centers. SEI is trying to assemble an advisory committee but is having conflict of interest clearance problems. Security screening for SEI and CEEP staff is underway, and no sensitive data can be handled until those clearances are finished. The evaluation plan has been revised, especially the timeline, due to these delays. The draft report will be released in February, the final report on April 1.

There followed discussion by the Board members. Dr. Ramey commented that the large number of conflicts of interest are due largely to IES' success in attracting researchers. Dr. Granger questioned the role of an advisory committee in the core activities and said it needn't contribute substantially to delay. Dr. Muller commented that the security issues were the real problem, since clearances are needed to see 75 percent of the necessary data. Mr. Lee asked about stakeholder input, and Dr. Muller said they are casting a broad net. Mr. Handy asked about the definition of rigor and the relevance of creating more researchers. Dr. Baldwin said rigor was closely tied to methodological approaches. Dr. Muller pointed out that increasing the number and quality of post doctorate scientists is part of the OMB Performance Assessment Rating Tool assessment, and that's why it's included. Mr. Handy said that went to capacity, not rigor.

After clarification from Dr. Ramey, Dr. Baldwin said that the paper written would be the project final report, which the Board would use to produce the Board's own final report. Dr. Granger noted that the original purpose of the evaluation was to inform the November report to Congress on the reauthorization of the Education Sciences Reform Act and that the evaluation would come too late for that. Mr. Baron noted that IES is set to expire in October of 2008, so April would be an appropriate time. Dr. Whitehurst noted that with the other bills before Congress, IES may be extended another year. Dr. Walberg volunteered to help edit the final Board report but noted that he may not be reappointed and noted that a brief summary will be necessary. Mr. Baron noted that the Board may not be considered fully independent of IES, so an outside evaluation may be preferred. There was further discussion of the formatting of the three areas of focus: rigor, relevance, and utilization. Members wanted research funded by IES as well as NCES reports to be part of the evaluation. Dr. Ramey said the Board would reexamine the issue in January.

Presentation on the What Works Clearinghouse

Dr. Mark Dynarski, principal investigator, said the three guiding principles of the Clearinghouse are to provide useful information to practitioners by answering the questions educators, school communities, and policymakers are asking about what works in education; to focus on quality, using high and transparent scientific standards that raise the bar for education research; and operate efficiently by keeping processes simple and timely. The ultimate goal is to provide evidence-based interventions and practices that result in improved student outcomes. The research community produces the evidence and the Clearinghouse summarizes, disseminates, and synthesizes evidence.

He gave the example of a Board request for information about Fast ForWord. Of 118 studies, 6 met the standards for quality, validity, attrition standards, and reporting results. There followed a discussion of the standards. Dr. Hoxby said that many researchers do not know WWC's standards. She recommended more communication with the researchers. Dr. Dynarski said WWC could have a more visible presence at conferences and produce clear and simple documents describing the standards. Dr. Hoxby said WWC could intervene when papers are published or in draft form. Dr. Whitehurst suggested that requests for disaggregated data could be sent to researchers so to avoid exclusion. Dr. Milgram said it is important that the effect sizes be made clear, since the effect sizes are often very small. Dr. Shaywitz agreed about the small effect size, adding that the evidence size is often small as well. She added that WWC should be careful about positive comments being used by the companies behind the products.

There was general consensus that practitioners have difficulty understanding research results. Mr. Baron suggested a change in the presentation to separate interventions with strong evidence from those with weak evidence and those with an effect on an important outcome from an effect on an unimportant outcome. Dr. Hoxby said the answer was to get practitioners to understand the research, not to simplify the data for the practitioners; she also felt that the cost of the intervention should be included in the information. Dr. Dynarski said that IES created a task for WWC to investigate implementation of resources, since researchers often don't include that information in the articles. Dr. Shaywitz said that the practical use by the educators is the important effect, and the effects of interventions must be made clear. Mr. Lee suggested focusing on long-term effects and getting feedback from small customer panels.

Dr. Dynarski returned to his presentation. WWC's strengths include a clear framework and review process, popular reading and math products, and experienced reviewers and principal investigators, both from the previous version of the website and currently in the certification process. Among WWC's weaknesses are the perception that standards are too high, resulting in a narrow base of evidence and the "Nothing Works Database" backlash; that the reports are technical; that the website is static and navigation is unclear; that setting standards and resolving issues is slow; and that there are no standards yet for some causal designs.

The Board further discussed the standards. Dr. Ramey suggested forwarding the standards to journalists. Dr. Hoxby said objections should be expected when standards are raised. She said it is more important that the standards are not arbitrary and that studies that are excluded for being poor studies should be distinguished from studies that are merely not applicable to the Clearinghouse. Dr. Dynarski described the new quick review assessment product. Dr. Ramey avoiding the appearance of arbitrariness is important and urged IES to disseminate the criteria vigorously.

Dr. Dynarski continued his presentation. WWC's long term goals are to be accepted as a trusted source of evidence, to be a heavily-used resource for educators and researchers, and to incorporate continuous improvement. Short term goals are to identify new topics, certify reviewers and initiate reviews, set up a timely process for setting standards and resolving issues, redesign the website to incorporate dynamic reports and smart searches, and to create more mechanisms for continuous improvement. Outreach efforts include a plan to post on the site and email to the WWC update list an invitation to give feedback, phone interviews with heavy users and decision-makers, meetings with constituent representatives, and meetings with other stakeholders.

WWC's focus is on six new developments. There will be four new practice guides promoting college attendance for low-income students, helping teachers deal with behavioral problems in the classroom, response to intervention in special education, and recruiting and retaining quality teachers. The second focus is quick reviews. There will be a clearinghouse within the Clearinghouse focusing on special education topics. The website will be redesigned for better navigation. The standards will be revised to employprogression, discontinuity, and single subject designs. The final focus is the continued training and certification of reviewers to ensure sensitivity to the standards.

Dr. Torgesen asked what happens when a regional education lab comes up with a quick response that conflicts with WWC's answer. Dr. Cottingham said WCC standards and the lab standards are getting closer together. Dr. Walberg warned that increased rigor in meta-analyses leads to smaller datasets and biased evidence bases. Dr. Ramey concluded that the field needs the WWC, that the Board endorses it, and that the project will improve the field.

Communication Committee Report

Jerry Lee gave an update from his Committee. Since the last Board meeting, the Committee invited Dr. Dynarski to present on WWC and got the IEA evaluation plan to the Board for review. Third, the Committee requested that the Executive Director engage in strategies to increase Board meeting attendance in the field. The final recommendation was that the Board authorize the Executive Director to include public speaking on the role of NBES as part of her functions, to increase the Board's visibility.

Dr. Shaywitz commented on the value of the meeting with staff. Mr. Lee added that there is a weekly email on what is happening in IES, which has been helpful. Dr. Ramey said that the members' talents could be focused toward explaining IES and the state of educational research in symposia, panel discussions, and other venues. Mr. Lee suggested that the Executive Director assemble talking points so that the message is consistent. Dr. Bement suggested that the National Science Board be contacted and informed of NBES.

Legislation Committee Report

Because the Committee Chair was absent, Mr. Baron reported. Holly Kuzmich from the Office of the Secretary discussed the latest legislative developments. No Child Left Behind (NCLB) reauthorization is moving slowly. The House put out a discussion draft for comment from the education community in August. Secretary Spellings sent a letter with the Department's concerns. Feedback has been vast and conflicting, and a bill has not yet been introduced. New legislation must be introduced before Congress addresses appropriations. A reauthorization bill is unlikely to appear in time. Senators Kennedy and Enzi are working to draft bipartisan legislation, but they have not yet drafted a bill. Senate is trying to get a definition of scientifically-based research for the Head Start reauthorization and to use that definition in future bills. The Department is assisting them in crafting the language, hoping to maintain the strength of the definition. The Department has implemented all of the Investigator General's recommendations on Reading First. Without reauthorization, NCLB will continue unchanged, so no reauthorization is better than a bad reauthorization.

There followed a Board discussion. Mr. Lee suggested using the analogy of medical research to encourage maintaining high standards. Dr. Ramey expressed concern that Head Start research is not up to the current standards; he recommended that the Board discuss taking action in the Head Start Reauthorization. Dr. Granger asked about Committee hearings or forums where the issue of scientifically based research would be discussed. Ms. Kuzmich said she knew of none. Dr. Granger said the Head Start discussions should be made publicly.

Mr. Baron read the Committee's recommendation to the Board: "That Congress revise the standard definition of 'scientifically based research' so that it includes studies likely to produce valid conclusions about a program's effectiveness, and excludes studies that often produce erroneous results." The background of the recommendation is that the current definition is too broad. As a result, less-rigorous studies are used, which produce less-rigorous evidence. The current draft bill loosens the definition further. The Committee recommended that the phrase be tightly defined with the specific recommendation: "That Congress revise the statutory definition of 'scientifically based research' and 'scientifically-based reading research' to clarify that such research makes claims about an activities impact on educational outcomes only in well-designed and implemented random assignment experiments, when feasible, and other methods (such as well-matched comparison group studies) that allow for the strongest possible causal inferences when random assignment is not feasible.'" This definition is similar to that in the Education Sciences Reform Act and should be the model for NCLB and other legislation. The language has been endorsed by the Society for Prevention Research. The language is consistent with WWC standards as well as the standards of OMB, FDA, the Academic Competitiveness Council, and the National Academy of Sciences Institute of Medicine.

Mr. Handy moved that the Board adopt the recommendation. The motion was seconded, and discussion followed. Dr. Bement asked about double blind testing in social systems. Mr. Baron said the design of a study would have to be different for different evaluations. Dr. Ramey added that double blinding is often impossible. Ramey added that double blinding is often impossible. Institutional Review BoardRB approval usually requires that parents be informed. Generally, assessors are blinded to mitigate bias. Dr. Hoxby noted that teacher and administrators were another source of bias to be mitigated. She asked if "regression discontinuity" should be included in the language. Mr. Baron said there would be greater discussion and examples in the recommendation document but that the specific language should be streamlined. Dr. Whitehurst stressed the importance of multiple studies, since the language did not address that. He suggested an alternative that, rather than draft new language, to refer Congress to the definition in ESRA. Mr. Baron noted that, in discussion with Congressional staff, citing the precedents for the language in other agencies and in ESRA inspired confidence. The motion carried unanimously. Dr. Granger was not present, not voting.

Mr. Baron said the next move would be follow-up with Congress, utilizing current contacts. The Senate Appropriations Bill has a $20 million increase for education research and language supporting the Department's work evaluating education programs using rigorous methodologies. The bill recommends that IES lead the design and implementation of the evaluations. There were discussions with Rep. Hinojosa, who is willing to help. Meetings with members and staff will continue, and interest in these ideas is being generated. Dr. Ramey recommended that the Board members use their networks to gain support. Mr. Baron added that there is value in partnering with groups and societies as well.

NBES Report on IES Event Schedule

Dr. Walberg gave the Board an overview of the draft event schedule. The NBES Report on IES for Congress is behind schedule, so he planned a schedule to get it on track. First, Board members can make suggestions for the outline and content. He noted that the Board had expressed a desire that results be emphasized over training. The second event was the draft vendor report, which will be distributed to the Board members for suggestions. Dr. Walberg at this time would draft the Board report in approximately two weeks. A draft of this would go to the Board for suggestions. He would meet with the Committee before submitting the second draft to the Board members for approval at a meeting or remotely. Submission to Congress would occur upon approval.

Dr. Whitehurst suggested a markup of the existing legislation to deal with areas where NBES can be improved, such as granting it the authority to compensate an executive director and gift authority. He said the report should include technical and policy issues as well as scientific issues. Dr. Walberg asked that the Legislative Committee start on that project and submit it to be included in the report or appended to it. Dr. Whitehurst said he had provided suggested changes to the Legislation Committee. Dr. Bement stressed the importance of getting any such proposed language to Congress and staff early in the reauthorization process. Dr. Whitehurst said he would communicate with the Legislative Committee Chairperson to get a draft by the next Board meeting. Dr. Ramey said he would meet with Congressional staff when a consensus is reached.

Old Business

Mr. Handy proposed that it be made a rule that reports and presentations be submitted in writing in advance. Dr. Walberg agreed. Board consensus was that a week in advance would be an appropriate time for the members to see the reports.

Discussion with Secretary Margaret Spellings

Dr. Ramey presented the IES Annual Report to Secretary Spellings. Secretary Spellings administered the oath of office to Dr. David Geary. Secretary Spellings then addressed the panel. She said the administration had done a lot for research-based practices, due to NCLB requirements. She urged the Board to protect research from opposition and proposed legislation. The goals set by NCLB make the Department responsible to help the districts meet the goal by identifying and disseminating what works. She urged the Board to address issues in the order of urgency. Dr. Ramey said the Board would welcome a list of topics. Secretary Spellings said she would provide that list to the Board.

Dr. Walberg asked about the status of member reappointments, and the Board discussed the delays in Senate confirmations and the possibility of bringing non-confirmed members on as advisors. Mr. Handy informed Secretary Spellings of the Board resolution on the definition of scientifically-based research. Secretary Spellings said the language could be used in the Head Start Legislation, though it may be too late.

Dr. Bement asked how National Governors Association ideas and initiatives match with Department and Administration priorities. Secretary Spellings said the governors are more interested in high school reform than Congress is. There is interest in early childhood education, but more around access than quality.

Dr. Milgram asked about STEM (Science, Technology, Engineering, and Mathematics) programs and suggested a joint project between the Department and NASA, focusing on math. Dr. Granger asked about the practice guides and WWC. Secretary Spellings said dissemination projects of that sort would be of increasing priority. Dr. Hoxby suggested that the Department work toward getting the media in the habit of consulting the WWC or IES. Dr. Hoxby said the quick review process could prove valuable at times then studies are coming out and getting media attention. Secretary Spellings suggested that Dr. Hoxby give a presentation to the Education Reporters Association to get the word out.

Summary Views and Next Steps

Chairman Ramey invited members to join the committees. Dr. Milgram discussed a Board resolution to modify the language used on the WWC website from technical to ordinary language, since the technical language is often misapplied and misunderstood in school districts. The resolution was put on the agenda for the next meeting. Dr. Milgram raised some points on the Board's duty to advise the Director on the funding of applications after the completion of peer review. He suggested that the Board discuss at the next meeting the size of grants that should go to the Board for review and make a resolution on the matter. Mr. Lee suggested for discussion of a demonstration project at the next meeting.

Dr. Walberg moved to adjourn. The motion was seconded and the meeting was adjourned at 1:21 p.m.

The National Board for Education Sciences is a Federal advisory committee chartered by Congress, operating under the Federal Advisory Committee Act (FACA); 5 U.S.C., App. 2). The Board provides advice to the Director on the policies of the Institute of Education Sciences. The findings and recommendations of the Board do not represent the views of the Agency, and this document does not represent information approved or disseminated by the Department of Education.