National Board for Education Sciences: Meeting Minutes January 23-24, 2006

National Board for Education Sciences
January 23, 2006 Minutes of Meeting

January 23rd, 2006

Location:
Atrium Ballroom
Washington Court Hotel
525 New Jersey Ave., NW
Washington, DC 20001

Members Present:
Jonathan Baron
Beth Ann Bryan
Carol A. D'Amico
James Davis
Robert Granger, Chairman
Philip Handy, Vice Chairman
Eric A. Hanushek
Caroline M. Hoxby (by telephone)
Jerry Lee
Richard Milgram
Sally E. Shaywitz
Joseph K. Torgesen

Members Absent:
Roberto Lopez
Herbert Walberg

Ex Officio Members Present:
Grover J. Whitehurst, Director, IES
Phoebe H. Cottingham, Commissioner, National Center for Education Evaluation
Lynn Okagaki, Commissioner, National Center for Education Research
Edward J. Kame'enui, Commissioner, National Center for Special Education
Mark Schneider, Commissioner, National Center for Education Statistics
Arden L. Bement, Jr., Director of the National Science Foundation
Duane Alexander, Director, National Institute for Child Health and Human Development
Kathleen Utgoff, Director, Department of Labor Statistics
Robert Kominski, delegate of the Director of the Census

Executive Director: Sonia Chessen

Designated Federal Official: Mary Grace Lucier

IES Staff Present:
Sue Betka, Deputy for Administration and Policy, IES
Andrew White, Deputy for Science
Ricky Takai
Mike Bowler
Ellie McCutcheon

Members of the Public Present:
Marcia Knutson, NEKIA
Sandi Wurtz, AERA
Myrna Maldlawitz, Learning Disabilities Association of America
Debra Viadero, Ed Week
James Kohlmoss, NEKIA
Tanya Shuy, NIFL
Sarah Sparks, Ed Daily
Mary Ann Mc Cabe, SRCD
J. Egermeyer, COSSA
Kim Krocker, CEC
Frederic A. Mosher, CPRE

The Chairman called the meeting to order at 2 p.m. The roll was called and a quorum established. The minutes of the previous meeting (September 6 and 7, 2005) were approved as submitted. The meeting agenda was approved.

Chairman's Remarks
The Chairman reviewed the mandated responsibilities of the Board, namely to approve the research priorities for the Institute of Education Sciences (IES) and to approve the agency's peer review procedures for grant applications and reports. He reminded Board members and the public that the IES research priorities had been approved during the September Board meeting and that the focus of this meeting would be the review and approval of the peer review procedures. In addition, the Chairman explained that having completed this, the balance of the meeting would be devoted to learning about the education research being conducted in the other federal research agencies, and hearing from the field about what sort of research questions are important to practitioners.

The Chairman concluded that since the previous meeting, IES had released of the findings of the National Assessment of Educational Progress and the Department has continued to work on the operational details of No Child Left Behind. He explained that there continues to be pressure to improve educational achievement and attainment, and to find ways to bring research findings to bear on closing the achievement gap. The issue is how to improve policy and practice with the evidence that is available now while awaiting better evidence to become available.

Director's Remarks
Director Whitehurst discussed the following personnel changes:

Mark Schneider was confirmed by the Senate on October 21, 2005, as the Commissioner of Education Statistics;
Lynn Okagaki has been appointed Commissioner for Education Research;
Andrew White is the new Deputy Director for Science; and
Jerry Malitz has been named as Chief Information Technology Officer, a new position at IES.

IES will be taking over its own website, and its listserv has been growing exponentially. IES will soon have a new logo and thus will be able to move ahead with a distinctive identity on electronic and print formats.

The Director reiterated that NCES had released the National Assessment of Educational Progress and the National Assessment of Adult Literacy. He also reported that awards were made in the competition for grants for Statewide Longitudinal Databases. In the evaluation arena, a help desk has been opened for the What Works Clearinghouse, and awards are due to be made in March for the competitions for Regional Educational Laboratories. Finally, the Director mentioned that his office has been involved with the preparation for the FY 2007 budget and with providing advice to policy makers.

Peer Review
Deputy Director for Science Andrew White gave a Power Point presentation on IES' scientific peer review processes for grant applications and reports.

Peer Review of Grant Applications
Dr. White summarized the conduct of the peer review process for grant applications. He explained the timelines for submission and review of grants, including the processes of screening of applications and the recruitment of panel members. Dr. White discussed the numbers of applications that have been reviewed over the last few years and the number of grants funded. He also explained how the panels were constructed and matched to the appropriate grants. The system has now been in place for over two years and IES is on the road to having a predictable schedule of grant announcements and semi-annual reviews by standing panels of reviewers.

Peer Review of Reports
Dr. White then explained the review process for reports written by IES staff or contractors. Reports are reviewed in two stages. The first phase is a review by the originating IES Center. The IES Standards and Review Office conducts the second review. There are three categories of review: external review, internal review, and no review-this last category applies to administrative reports, users' manuals, or survey descriptions. Reports are checked for significance to the field of education, quality of scholarship, appropriateness of conclusions, and clarity of presentation. Reports cleared through the Standards and Review Office can then be published by IES and disseminated through the web and through press releases. Dr. Whitehurst made the point that the typical readership for an NCES report on its web page is larger than would be the case for any American journal.

In the course of Dr. White's presentation, the Board had many questions related to the panel composition and the alignment of expertise, as well as how IES ensures that they are getting strong and varied applications and that researchers understand the goal system under which they must apply. The Board asked questions about how reviewers are selected how their qualifications and biases are assessed. Drs Whitehurst, White and Okagaki fielded these questions and committed to making some alterations to the documents overnight. Board members were told that in the morning, there would be time for more extensive discussion of these changes prior to voting to approve the peer review processes.

The chairman adjourned the meeting at 5:10 p.m.

Top

January 24, 2006

Location:
Atrium Ballroom
Washington Court Hotel
525 New Jersey Ave., NW
Washington, DC 20001

Members Present:
Jonathan Baron
Beth Ann Bryan
Carol A. D'Amico
James Davis
Robert Granger, Chairman
Philip Handy, Vice Chairman
Eric A. Hanushek
Caroline M. Hoxby
Jerry Lee
Richard Milgram
Sally E. Shaywitz
Joseph K. Torgesen

Members Absent:
Roberto Lopez
Herbert Walberg

Executive Director: Sonia Chessen

Designated Federal Official: Mary Grace Lucier

IES Staff Present:
Andrew White, Deputy for Science
Ricky Takai
Mike Bowler
Ellie McCutcheon

Department of Education Staff Present: W. Stephen Wilson

Other Federal Staff Present:
Daniel Berch, NICHD/NIH
Travis Thompson, NSF
Donald Thompson, NSF

Members of the Public Present:
Vaughan Lauer, ETS
Sandi Wurtz, AERA
Gerald Sroufe, AERA
Kenna Shaw, ASHG
Debra Viadero, Ed Week
James Kohlmoss, NEKIA
Marcia Knutson, NEKIA
Mary Ann Mc Cabe, SRCD
J. Egermeyer, COSSA
Karen Studwell, APA
Frederic A. Mosher, CPRE
Michael Casserly, CGCS

The Chairman called the meeting to order at 9 a.m. He reviewed the previous day's discussion of the peer review policies and procedures and opened the floor for further discussion.

Further Discussion/Action on Peer Review Policy and Procedure
The Board discussion of the peer review process was primarily focused in the areas of research methodology, bias and reviewer qualifications. First, Board members were interested in providing better guidance to reviewers about IES expectations of research methodology when establishing causality. This kind of guidance is provided to researchers in the call for proposals, but Dr. Whitehurst agreed that it was worth developing some guidance for reviewers of all reports that address the issue of causality, asking reviewers to pay particular attention to the analysis, but also to language that might imply causality when it has not been proven.

Dr. White then presented the changes that were being proposed to the peer review of grant applications. The Board agreed to add a clause that asked researchers to disclose any personal points of view or personal perspectives, positive or negative before the presentation of proposals. The Board also agreed to add language about who would receive summaries of the primary reviewers comments.

The Board had an extensive discussion whether language was needed to formally describe the required qualifications of panel members, in particular the need for content knowledge in the area of mathematics and economics. This discussion culminated in a vote to approve the draft document for peer review of grants as it was, but with the understanding that staff would come back to the Board with some information about the quality assurance procedures they will implement to assure or test the fact that policies and procedures result in panels and decisions that are appropriate given the guidance.

Dr. White presented proposed modifications to the procedures for peer review of reports. These were primarily aimed at codifying a timely review process. However, in response to the discussion of causality, there was also new language to the effect that reviewers will receive guidance on assuring that language that advances causal claims is adequately supported by the methods and analysis use in the report. The Board voted to approve the procedures including these modifications.

Presentation by Michael Casserly, Council of Great City Schools
The Council of Great City Schools is a coalition of 66 of the nation's largest urban public school systems. Mr. Casserly discussed his organization's efforts to improve student achievement and to meet the challenges of greater accountability and transparency. The Council is looking to research to explain why some districts perform better than others and to translate that knowledge into technical assistance for urban school systems that need help. Mr. Casserly provided potential research questions in six categories of research the Council found most salient to its work: student achievement, teacher quality and professional development, high school reform, funding and the use of resources, leadership and governance, and choice and supplemental services.

Mr. Casserly also commented that more tactical questions involving implementation and practice have proved to be of little interest to researchers, though they are "make-or-break" issues for practitioners. Practitioners often do not know where to find or use good research. Due to budget cuts over the years and the increasing dominance of the states, big city school systems now have diminished capacity to conduct their own research and evaluation programs.

Mr. Casserly reported that the Council had convened researcher and superintendents to develop a study design with a set of questions that both sides considered important. The results were published in a report, "Foundations for Success," which had a highly positive effect on reform efforts. Mr. Casserly was interested in doing more of this kind of partnership and suggested that IES might consider investing in capacity building for research in the largest urban school systems and in closer collaborations between the producers and users of research. The Board welcomed this suggestion and discussed possible areas for collaboration. Members suggested that Mr. Casserly meet with Dr. Whitehurst to discuss the possibilities and perhaps return in May to report on their progress in outlining some mutually beneficial work.

Presentation by Dr. Donald Thompson, NSF
Dr. Thompson spoke about the mission of the Education and Human Resources Directorate at the National Science Foundation, namely, the integration of science and research and the preparation of people who will teach others. The Directorate shares IES' goal of developing a vibrant research and development community able to conduct rigorous research and evaluations that will support excellence in education. Other agencies such as Defense, Energy, NASA, and NIH work closely with NSF and share information about attracting more U.S. students to science and engineering and retaining them in those fields. Current collaborations with the Department of Education include the Math and Science Partnership Program and the Research, Evaluation, and Technical Assistance Program, and the Title One Mathematics Toolkit. Future collaborations include developing common data elements for monitoring systems and qualitative measurement workshops to address evaluation concerns.

Presentation by Dr. Duane Alexander, NICHD
The National Institute of Child Health and Human Development (NICHD) was added to the National Institutes of Health in 1962. NICHD broadened the concept of health to include the cognitive, social, and behavioral aspects of human development. This perspective includes learning, a vital part of which is reading, and NICHD has had a program on reading since its inception. The Institute's initial focus was retardation and developmental disabilities, but since the 1980s, its efforts have delved into basic research on reading. Dr. Alexander cited the work of Reid Lyon, who directed the research program in learning and who set up clinical trials in the classroom and subsequently established successful collaborative relationships with the Department of Education, notably an interagency research initiative that involved the NICHD, the Department of Education, and the National Science Foundation.

NICHD is currently working on several collaborative projects related in education including language development; the development of English for Spanish-speaking children; mathematics cognition in learning disabilities; and science learning in K-8. Dr. Alexander hoped that in partnership with IES work could proceed on joint activities in science, improved means of instruction, and the scaling up to the classroom of what has been learned in laboratory settings.

Concluding Remarks, Topics for Next Meeting
The Board has scheduled its next meeting for May 8 and 9, 2006. During a round-table discussion of topics for discussion/action at subsequent meetings, Board members expressed a consensus for a focus on continuing the research-policy-practice continuum. Suggested topics, in addition to follow up discussions by Dr. Whitehurst with the Great City Schools included:

Research on teacher training and effectiveness, class size, the reauthorization of No Child Left Behind, Board report to Congress in 2007 [Handy];
Extending the continuum of evidence for the What Works Clearinghouse [Whitehurst]; *
Teacher quality [Bryan];
Reviews of large randomized controlled trials, and Board reports and recommendations to Congress [Baron];
Consultations with researchers from AFT and NEA [Davis];
Longitudinal studies of the National Center for Special Educational Research [Shaywitz];
Capacity building in the research field [Lee];
Discussions with State commissioners of education and state school boards [Hoxby]; and
Reporting of negative research findings [Milgram].

The chairman indicated that members would be polled for their availability for future meetings.

The meeting adjourned at 2:03 p.m.

*Mr. Baron recused himself from discussions of the What Works Clearinghouse.

The National Board for Education Sciences is a Federal advisory committee chartered by Congress, operating under the Federal Advisory Committee Act (FACA); 5 U.S.C., App.2). The Board provides advice to the Director on the policies of the Institute of Education Sciences. The findings and recommendations of the Board do not represent the views of the Agency, and this document does not represent information approved or disseminated by the Department of Education.

Top