National Board for Education Sciences: Meeting Minutes September 20-21, 2006

National Board for Education Sciences
September 20-21, 2006 Minutes of Meeting

September 20th, 2006

Location:
The Washington Court Hotel
525 New Jersey Ave., NW
Washington, DC 20001

Members Present:
Jonathan Baron
Carol D'Amico
Robert Granger, Chairman
Philip Handy
Eric A. Hanushek
Caroline M. Hoxby
Jerry Lee
Richard Milgram
Sally E. Shaywitz
Joseph K. Torgesen
Herbert J. Walberg

Members Absent:
Beth Ann Bryan
Craig Ramey

Ex Officio Members Present:
Grover J. Whitehurst, Director, IES
Phoebe H. Cottingham, Commissioner, National Center for Education Evaluation
Lynn Okagaki, Commissioner, National Center for Education Research
Edward J. Kame'enui, Commissioner, National Center for Special Education
Mark Schneider, Commissioner, National Center for Education Statistics
Arden L. Bement, Jr., Director of the National Science Foundation
Dixie Sommers, delegate of the Director, Bureau of Labor Statistics

Executive Director: Sonia Chessen

Designated Federal Official (Acting): Wilma Greene

IES Staff Present:
Sue Betka, Deputy for Administration and Policy, IES
Andrew White, Deputy for Science
Ricky Takai
Mike Bowler
Ellie McCutcheon
Susan Sanchez
Morgan Stair

Members of the Public Present:
Andrea Browning, Society for Research in Child Development
La Tosha C. Lewis, Consortium of Social Sciences Associations
Jady Johnson, Reading Recovery Council
Annette Zehler, Center for Applied Linguistics
James Kohlmoos, National Education Knowledge Industry Association
Chuck Russell, Southwest Educational Development Laboratory
Allison Dobbs
Gerald Sroufe, AERA
Max McConkey, WestEd
David Glenn
Fritz Mosher, CPRE
Lin Liu, OMB
Rebecca Powell, What Works Clearinghouse
Sara Sparks, Ed Daily
Jane West, Higher Education Consortium for Special Education
Larry Snowhite, Houghton Mifflin
Allison Trepod, SRI International
Samantha Gaiber
Lucy Gettman, RRCNA
Vaughn K. Lauer, ETS
Kati Haycock, Education Trust

The Chairman called the meeting to order at 2 p.m. He called for a motion to approve the minutes of the last meeting; the minutes were approved along with the present meeting's agenda.

Report of the Executive Director
Dr. Granger informed the Board that he had been a speaker at a plenary session of IES' Inaugural Research Conference, June 15 and 16, 2006, which brought together hundreds of IES-supported research grantees as well as prominent practitioners and policy makers. The purpose of the conference was to provide a forum for dialogue and interaction and to advance the productivity of the research community. In July, the Board filed its annual report to Congress for 2005, which is available on the NBES web site. The Chairman participated in meetings with staff of the Office of Management and Budget, which is involved in performance assessment and review of all federal programs and agencies, including IES. OMB has agreed to postpone the review of IES until the 07-08 cycle in order to take advantage of impending evaluation work to be discussed at today's meeting. He informed the members that Education Week is doing a story about IES in its fourth year and that a reporter and photographer would be in attendance at the meeting. Finally, Dr. Granger reported that he and the Executive Director had met with the Secretary's senior staff to bring them up to date on the work of the Board and the agenda for the present meeting.

Report of the Executive Director
Ms. Chessen discussed her efforts in the preparation and submission of the 2006 Annual Report and the preliminary work on a "final report" on IES, which is due in 2007, five years after the passage of the Education Sciences Reform Act and the creation of IES. She spoke of her interactions with the NBES subcommittees in advance of this meeting and in the preparation of a scope of work for an evaluation of IES.

Report of the IES Director
Dr. Whitehurst thanked the members and ex officio members for their attendance and participation in the work of the Board. IES is now at a point where the main building blocks are in place and staff and processes are functioning and beginning to get yield. Attention may now turn from rigor in research to relevance and utilization. This task is underway with the establishment of the Urban Education Research Task Force, which will help to generate projects and collaborations across districts to bring researcher and superintendents together to take on tasks that might otherwise not be addressed. A grant has been awarded to the Council of Great City Schools to establish fellowships for senior researchers to work with urban districts. Another grant will bring together legislators and researchers to discuss the policy decisions at the state level. The first of these institutes will be held in January 2007.

IES is close to completing a survey of research needs of the Chief State School Officers and leading district superintendents, and a collaboration is underway to provide technical assistance to states that have received grants to establish a statewide longitudinal database. Also under development for practitioners is a set of coherent guidelines known as practice guides to address particular problems in such areas as learning English or enhancing the achievement of girls in mathematics and science.

Other activities center on the work of the Regional Educational Laboratories, which have been recompeted, resulting in a turnover of four of the ten labs, and aligning their work more closely with the work of IES. The new lab contract will require peer review of their products. The labs now share a common web site, which will identify and differentiate IES-funded work. The What Works Clearinghouse has overcome technical issues and established processes to expedite reports. Approximately 50 reports are now in the pipeline.

There is heightened interest in the topic of postsecondary research and data, and Secretary Spellings has established a Commission on the Future of Higher Education. A major redesign of a website which students and the public can use to access information about postsecondary institutions is also underway. This feature is called College Navigator and will be an alternative to commercial sources which rank the quality of institutions of higher learning. This information becomes increasingly unreliable, given the proliferation of non-traditional students who start in one institution, transfer to another, and may take additional years to graduate. This situation points up the need for longitudinal data on individual students in order to track graduation rates as a measure of institutional accountability. Concerns have been raised about the privacy of student data, and a method has been devised to address these concerns. This method involves encryption of data and the cooperation of a "trusted third party" to provide linkage of student records from different institutions.

IES has funded a national research and development center on Postsecondary Education at Columbia University. It will focus on community colleges, and a new discretionary grant program on Postsecondary Education will address some of the issues raised by the Spellings Commission. IES has provided seed money for a new professional society, the Society for Research on Education Effectiveness, which will be a home for those who are doing research on the impact of education interventions on outcomes. In addition, IES is funding summer institutes for professional development related to research designs and other issues that come up in grant applications. Finally, IES has had major involvement in the National Mathematics Advisory Panel, the Secretary's Commission on the Future of Higher Education, the Secretary's Growth Model Pilot Program, and in the White House Conference on Global Literacy, among others.

The Director discussed with the Board the comments received on the plan for addressing research priorities. Reactions cited an insufficient emphasis on basic understanding, the role of contextual variables, and on capacity and infrastructure. Dr. Whitehurst said that these suggestions would be given serious consideration and would be incorporated as possible in the planning process.

Finally, Dr. Whitehurst gave a brief description of the budget process to date. The Congress has not yet passed a budget for education, but it appears likely that the IES budget for FY 2007 will be about $535 million, a 3.4 percent increase from the previous year.

Presentation by Kati Haycock, Director, The Education Trust
Ms. Haycock is a leading child advocate in the field of education whose organization works to get all children, especially poor and minority children, to high levels of achievement. On the basis of much time spent in schools, her perception is that more and better research may or may not influence practitioners, whereas policy makers are more inclined to respond quickly. Critical to both groups is that research respond to the questions most pressing to them. A critical factor in student achievement is teacher quality, and the effects of good and poor teachers are cumulative. Studies show that teachers in their first and second years in the classroom are not as effective as more senior teachers, but for some teachers there is not a longer term growth in effectiveness, which has led some to question their pre-service training and their route of entry into the profession. Thus, while there is consensus that standards, assessment, and accountability are all important, attention must now focus on the quality of the teacher force.

In various parts of the country there are high achieving schools in high poverty areas. The question is how can we learn from those that are doing well and what can education science tell us about how to make the better ones great. Better information is needed also in the case of English learners and disabled students and what investments should be made to support high achievement in those populations. A series of questions and answers followed.

Presentation of the Commissioner of the Center on Education Evaluation and Technical Assistance
Dr. Cottingham gave an overview of the major activities of the Center, which has two divisions, Evaluation, which does high quality experiments in real school settings, and Knowledge Utilization, which works on improving access to research and getting research findings out to audiences.

The Evaluation Center is currently conducting 23 evaluations, six of which are on various aspects of improving teaching-recruitment, induction, retention, two on professional development and another on teacher performance.

The Center has in its portfolio the Regional Educational Laboratory program consisting of ten labs, which have been tasked with focusing on questions their regions bring to them. They now have over 90 projects underway and every year they will add another 60. They do targeted research, gathering the best evidence available, and serve as a kind of extension service for people who need advice. The Center also includes the Education Resources Information Center (ERIC), which is a bibliographic database containing over one million abstracts of journal articles and reports in education. Now fully digitized, it has a new website and search engine, and is used all over the globe.

The What Works Clearinghouse is now entering its fifth year of operation online and is focused on practical solutions or interventions in a broad range of topic areas that meet certain standards of evidence and receive a rating from teams of reviewers. At this point, the system is well developed, and the task ahead is to make it work more efficiently with a focus on interventions, which, Dr. Cottingham points out, is what the practitioner wants.

The Chairman adjourned the meeting at 5 p.m.

Top

September 21st, 2006

Minutes of Meeting, Open Session

Location:
The Washington Court Hotel
525 New Jersey Ave., NW
Washington, DC 20001

Members Absent:
Beth Ann Bryan
Craig Ramey

Executive Director: Sonia Chessen

Designated Federal Official (Acting): Wilma Greene

IES Staff Present:
Sue Betka, Deputy for Administration and Policy
Amdrew White, Deputy for Science
Ricky Takai
Mike Bowler
Ellie McCutcheon
Susan Sanchez

Members of the Public Present:
Vaughn K. Lauer, ETS
Bernice Anderson, NSF
Lucy Gettman, RRCNA
Debra Viadero, Education Week

The open session was called to order at 10:50 a.m. The Chairman called on the subcommittee on the National Center for Education Evaluation to give its report and put its recommendations forward.

Jon Baron, convenor of the subcommittee, read and then gave background for three recommendations from the subcommittees that were each considered individually.

First recommendation: Congress should allow the Education Department to pool funds generated by the half percent evaluation set-aside. The recommendation states that Congress make a modest legislative revision to Section 9601 of the Elementary and Secondary Education Act, explicitly to allow the Department to pool evaluation set-aside amounts from several smaller programs within the Department to fund meaningful evaluations in one or a few such programs. Mr. Baron explained that the Elementary and Secondary Education Act gives the Department the authority in many of its K-12 grant programs to set aside one half of one percent of the program's funds to conduct evaluations of that program. The problem this recommendation seeks to address is that the Department does not use this set aside authority in many of its smaller grant programs, because the set-aside does not generate sufficient funds for a meaningful evaluation of the program.

The law, as currently drafted, does not give the Department the flexibility to pool set-aside amounts from several smaller programs to generate sufficient funds for a meaningful evaluation of one or a few of these programs. As a result, the Department generally does not use the set-aside, thereby denying the policy community the opportunity to learn which programs and interventions they find are effective and which are not. Thus, a recommendation is that Congress allow the Department to pool the funds in this way.

This recommendation was approved.

Second recommendation: The Department should use its waiver authority to build scientifically valid knowledge about which educational interventions are effective in enabling schools to make adequate yearly progress and that it also use its waiver authority rigorously to evaluate the effect on student achievement of variations that the Department allows and the No Child Left Behind accountability rule.

Mr. Baron explained that the recommendation asked the Department to put in place an official policy of granting waivers for states and districts to conduct demonstration projects that have two key elements. First, the demonstration project would implement an intervention, such as a specific reading or math program that shows major promise in smaller studies, and second, that the project include an independent, randomized control trial to evaluate the effectiveness of that intervention.

The Department's waiver would then allow the schools and the districts that are carrying out the demonstration project to calculate their adequate yearly progress, either with the students who are participating in the study, or without, whichever calculation yields the higher score. Mr. Baron explained that this kind of policy would be attractive to many troubled schools and districts, giving them the flexibility to conduct demonstration projects designed to make adequate yearly progress in a way that builds scientifically valid knowledge that many schools and districts can then use to make adequate yearly progress.

The second part of the recommendation is that when the Department waives its accountability rules, for example, when it allows variations in its formula for calculating adequate yearly progress, that it include a requirement that the state or district receiving the waiver conduct a randomized control trial to evaluate its effectiveness -- the effect of that waiver on student achievement.

The recommendation passed with the understanding that it would be edited to replace the growth models example with a different example of possible reasons for a waiver.

Third Recommendation: Congress designate IES in statute as the lead Agency for all Congressionally authorized evaluations of Education Department programs, responsible for all operations, contracts and reports associate with such evaluations.

The recommendation passed with the understanding that Committee would work with IES staff to clarify in the background material that this recommendation would not preclude the Secretary or any other executive branch member from hiring a contractor to develop data on how a particular office was operating. That type of evaluation would not be something that had been specifically authorized by Congress and indeed, it would be paid for out of the Department's administrative account, over which the Secretary has a very broad authority in terms of usage and which does not specifically mention evaluations.

Fourth Recommendation: The recommendation is that Congress and the Department of Education should ensure that individual student data might be used by researchers with appropriate safeguards for confidentiality in order to provide evaluations and analyses to improve our schools. NCLB and accountability in general, has as a by-product at the state level, produced a lot of longitudinal data on student achievement, which has lead to dramatic improvements in research abilities and capacities for people who have had access to these data. Unfortunately, it has been interpreted in some place as interacting badly with the Family Educational Rights and Privacy Act (FERPA), previously known as the Buckley Amendment, which was designed to prevent schools from releasing data on their students. The proposal is that the Secretary make clear that what conditions meet the requirements of FERPA, and that she encourage access for researchers to these longitudinal data bases in various forms.

The recommendation passed with the small amendment that the following sentence be deleted: "Others have found it convenient to hide behind FERPA so that they can avoid releasing data about their programs and performance."

Fifth Recommendation: NBES recommends that the National Center for Education Statistics, or NCES, focus its resources on the collection and timely release of statistical data and that it not publish or commission studies that purport to describe the causal effects of policy variables.

There was significant discussion about clarifying the language of this recommendation in different ways, in particular to indicate that other IES centers should be responsible for studies that assess the causal effects of policy variables. In the end, it was decided to pass the recommendation as presented but edit the background slightly to convey better the desire to allow NCES to produce policy-relevant descriptive analyses but avoid causal analyses.

Committee Reports:

NCSER:Dr. Shaywitz reported that the NCSER committee had a productive meeting in which they heard quantitative information about what NCSER was doing in terms of where the resources, research topics, and grant competitions have been and are headed. The committee then discussed future directions based on this information. How can we use what we have learned from these investments to inform the types, scope and depth of future investments?

Dr. Torgesen added that there are special directions or special initiatives that could be developed to have a larger impact in terms of the practice of special education out in the field. He suggested that there are two things we do not know enough about in special education. First of all, there are lots of bits and pieces of effective practices. They come in studies at level two, level three and maybe even occasionally a level four study or a goal four study. But we are lacking information about systems that work effectively for students in special education. Second, Dr. Torgesen suggested that Dr. Kame'enui and his group think about ways that we could study the question of implementation a little more productively and suggested that the initial step was to think about identifying some really successful programs and trying to learn something from them.

NCER: Dr. Milgram congratulated Commissioner Okagaki on behalf of the entire committee on the job she has done at NCER in the six months she has been there. Dr. Milgram applauded Dr. Okagaki's job of restructuring NCER, clarifying the structure, analyzing and putting together a readable and coherent description of the portfolio, which is the content of pamphlet that was distributed to the members. Dr. Milgram that Dr. Okagaki has hired new Project Directors and clarified the responsibilities of the Project Directors, in particular, their responsibilities for creating the requests for proposals or requests for applications and publicizing them to various parties. He cautioned that these project directors should reach out to encourage a broader pool of applicants beyond the usually grantee academic areas.

Dr. Milgram also raised a concern about the representation of various disciplines on the grant evaluation panels and urged the Board to continue its monitoring of the implementation of the peer review procedures. Dr. Walberg concurred.

Dr. Shaywitz concluded by commending Mark Schneider on the work NCES has done to improve the data collection on teachers' salaries.

Mr. Baron commended Phoebe Cottingham and Ricky Takai on the outstanding progress NCEE has made, and Dr. Shaywitz reiterated her approval of NCSER's accomplishments.

The Chairman adjourned the meeting at 2:00pm

* The National Board for Education Sciences is a Federal advisory committee chartered by Congress, operating under the Federal Advisory Committee Act (FACA); 5 U.S.C., App. 2). The Board provides advice to the Director on the policies of the Institute of Education Sciences. The findings and recommendations of the Board do not represent the views of the Agency, and this document does not represent information approved or disseminated by the Department of Education.

Top