Location:
Institute of Education Sciences Board Room
80 F Street NW
Washington, DC

Members Present:
Duane Alexander, M.D.
Jonathan (Jon) Baron
*Ms. Elizabeth (Beth) Ann Bryan
*Phoebe Cottingham (National Center for Education Evaluation and Regional Assistance (NCEE) (Commissioner), Ex Officio
Carol D'Amico
David C. Geary
Robert C. Granger, Chair
Frank Philip Handy
Eric A. Hanushek
Caroline Hoxby
Robert Kominski (US Census Bureau), Ex Officio
Gerald (Jerry) Lee
Lynn Okagaki, (National Center for Education Research (NCER) Commissioner and National Center for Special Education Research (NCSER) Interim Director),Ex Officio
Mark Schneider, (National Center for Education Statistics (NCES) Commissioner), Ex Officio
Kusum Singh, for Arden L. Bement, Jr., National Science Foundation, Ex Officio
Dixie Sommers, for Keith Hall, (US Bureau of Labor Statistics), Ex Officio
Joseph K. Torgesen
Grover (Russ) J. Whitehurst, IES Director, Ex Officio

*Present via teleconference

Staff:
Norma Garza, Executive Director, NBES
Wilma Greene for Mary Grace Lucier, Designated Federal Official

Family Educational Rights and Privacy Act (FERPA) Panel
Constance Citro, Committee on National Statistics, National Academy of Sciences
Chrys Dougherty, Just for Kids, National Center for Educational Accountability
Williamson Evers, Assistant Secretary for Planning, Evaluation, and Policy Development, ED
Jane Hannaway, Education Policy Center, Center for Analysis of Longitudinal Data in Education Research (CALDER Center), Urban Institute

IES Staff:
Sue Betka
Bruce Friedland
Mary Grace Lucier
Ellie McCutcheon
Anne Ricciuti, Ph.D.
Ricky Takai

Other Federal Staff:
Lynda Carlson
Leroy Rooker
Kelly Scott
Marcia Sprague

Synergy Enterprises, Inc. (SEI):
Steve Baldwin
John McManus
Allison Trepod

Members of the Public:
Kate Bannan
Andrea Browning
Mike Brodie
Karen Douglas
Lucy Gettman
Aimee Gudera
Ashley Kranz
Jim Kohlmoos
Tom Lindsley
Stacey Novelli
Ana Ottman
Tom Plewes
Chuck Russell
Paula Skedsvold
Gerald Sroufe
Debra Viadero
David Van Dyke

Call to Order
Bob Granger

Action: Approval of October 30–31, 2007 minutes

Dr. Granger, National Board for Education Science chair, called the meeting to order at 10:04 a.m. on May 21, 2008. During roll call, Dr. Granger noted that Ms. Bryan and Dr. Cottingham were present via teleconference. Dr. Granger mentioned two upcoming reports—the Board's annual report to ED and to Congress, due in July, and a 5-year report on the Educational Sciences Reform Act (ESRA), due in fall 2008.

Action: Approval of January 24, 2008 minutes with the addition, for clarification, of the words "draft of" before the words "proposed changes" in the first sentence on page 11 under "Legislative Committee Report."

Institute of Education Sciences Update
Dr. Russ Whitehurst

Dr. Whitehurst highlighted IES line items for which the President George W. Bush's budget proposes increases. These IES increases total $112.1 million, which is a 20.5 percent increase. The line items being increased include research on school restructuring ($7.5 million), statistics surveys ($16.2 million), regional educational laboratories' (REL) evaluation (required by contract/$2.0 million), 12^th grade state National Assessment of Educational Progress (NAEP) and other assessments ($32 million), National Assessment Governing Board (NAGB) ($2.8 million), and statewide data systems (the largest proposed increase with the aim of funding the remaining states/$51.7 million).

The number of research grant applications is increasing, and the number of grants approved was 12 percent this year, a higher percentage than in previous years. Replying to a board member's remark suggesting methodological "nit-picking" during grant reviews, Dr. Anne Ricciuti, acting deputy director for science at IES, reported that an analysis of panelists' scores shows that the methodologists are generally more positive than the content reviewers.

The annual IES research conference is scheduled for June 10–12, 2008.

National Center for Education Evaluation and Regional Assistance (NCEE) Update
Dr. Russ Whitehurst (on behalf of Phoebe Cottingham)

Dr. Whitehurst spoke briefly about Senator Edward Kennedy, recently diagnosed with a grave illness, noting that the senator was an architect of an educational reform act. Dr. Whitehurst then provided the board with bulleted lists of What Works Clearinghouse (WWC) and REL reports that have been released since the last NBES board meeting.

The department's portal, the Education Resources Information Center (ERIC), is much improved and well used (e.g., 28 million searches in the last quarter). Accessing full text of articles is a future challenge. Perhaps the government will be able to arrange for free availability of 1-year-old articles.

The media have been active in interpreting the interim report from the Reading First Impact Study that was released on May 1, 2008. Dr. Whitehurst said that IES tries to conduct the best science, then provides cautions at briefings before letting others interpret its reports. Mr. Baron complimented the Reading First report as well done and well designed, and observed the report's inability to identify an approach that had a sizable impact on reading comprehension is an important finding. Dr. Granger pointed out that the final report will incorporate another year of data. Dr. Hoxby suggested distinguishing evaluations of policy from evaluations of the science underlying the policy. Dr. Whitehurst indicated that Congress had determined the evaluation questions in the legislation for the Reading First program.

Action: The Board wishes to schedule a briefing on the Reading First final report.

After receiving a recent interim report on Upward Bound, Congress stopped further evaluation of that program. Dr. Whitehurst explained that the Upward Bound evaluation plan was discretionary, not required by Congress, and that a prior evaluation had questioned the effectiveness of the Upward Bound program. In retrospect, if IES had talked during the design phase to people likely to be affected by the evaluation, a a different series of questions might have been asked and the evaluation might have been better received. Congress's decision to terminate prevented any design changes further along in the process.

Another interim report on the Enhanced Reading Opportunities Study was released on January 28, 2008.

National Center for Education Research (NCER) and National Center for Special Education Research (NCSER)Update
Lynn Okagaki

Dr. Okagaki briefed the Board on the activities of these two centers since the last meeting, emphasizing linkages to Board priorities. The efforts for stimulating new areas of research include (1) getting cognitive psychologists interested in what happens in schools; (2) adapting the popular September 7, 2007 practice guide to apply also to special education; and thinking about (3) interventions to create and evaluate and (4) how to aggressively attract researchers to special education in order to build capacity in the field.

Dr. Hoxby noted that comparability of outcomes is challenging in the special education field, which can discourage researchers. Nevertheless, more data on special education are needed.

When Dr. Okagaki proposed involving states and districts in evaluating their own programs, Mr. Baron said that involving practitioners is a good idea that has worked well in other fields (e.g., welfare).

Dr. Granger asked how NCER is creating incentives for practitioner involvement in planning and implementing research studies. Dr. Okagaki replied that while grants may be an incentive to help educators to think about their job(s) differently, last summer IES was not able to commit to providing such support. Mr. Handy agreed that grant awards can be an incentive.

Action: Dr. Granger invited other Board members to communicate their ideas on capacity building to Dr. Okagaki.

National Center for Education Statistics (NCES) Update
Mark Schneider

Dr. Schneider, NCES director, stated that he would summarize several key items from his written report to the Board. Two of these are (1) international benchmarking and (2) improving access to and usability of NCES data. Two improvements for access and usability are the changed interface that makes "College Navigator" more usable (now receiving 30,000 hits per day) and the new availability of a Spanish version.

Computer assessments will soon be needed for science and 8^th grade writing and are now being fieldtested. Going into schools is challenging because of the huge variation in types of equipment (e.g., PC vs. MAC, multiple software programs, and different levels of computer security) and qualified information technology (IT) staff.

A national government report has recommended how to calculate graduation rates, for which reporting will be required soon. To date, only Florida has the capability for this method.

NCES has initiated efforts to coordinate longitudinal studies that were developed independently. This coordination would make the data more valuable, with the same students then being studied at different periods in their academic development. The first goal is to coordinate high school and postgraduate longitudinal studies, but eventually even kindergarten would be added.

Currently, there are three major assessment tools that are used internationally: (1) the Program for International Student Assessment (PISA), (2) Progress in International Reading Literacy (PIRLS), and (3) the Trends in International Mathematics and Science Study (TIMSS). TIMSS testing is conducted in 4th and 8^th grades. PISA testing, which is done in many countries (including some affluent ones), is under consideration for state benchmarking. One concern of the NCES staff is a framework issue—that state curricula may not necessarily align with what is being tested internationally. Where alignment is good—as with the alignment of Massachusetts with the National Assessment of Educational Progress (NAEP)—the results are good. Another problem is what to do with analyses of international data that make causal policy claims when the analytic methods cannot support those claims. For state level participation in international assessments,, a base of 1,500 students would be needed. This could be an expensive process.

Dr. Hanushek said that data should not be suppressed because of potential bad uses, that the TIMSS data would benefit from longitudinal studies, and that higher standards are desirable. Dr. Schneider noted that the U.S. has little control over actions taken by the Organization for Economic Cooperation and Development (OECD), even though the United States pays a significant portion of the PISA cost. Some of Dr. Schneider's questions are whether the international benchmarking is worthwhile; whether there are less expensive ways to obtain the data; whether it is practical to have statewide assessments of TIMSS and PISA, and whether the scientific basis is sufficient for good analyses.

Dr. Schneider reported on plans for a meeting scheduled for May 23, 2008 that will provide information to states and interested organizations about the benefits and costs of international benchmarking and international studies including what kinds of analyses the data will support, the costs of each type of assessment, and ways to link studies.

When Dr. Schneider mentioned students taking the state-based NAEP, Dr. Hanushek wondered why different states, because of different standards, might end up having different relationships to PISA and NAEP. Dr. Schneider commented that the closeness of PISA and NAEP is a fundamental question. Another fundamental question is whether states want a PISA score for national comparisons.

Dr. Hoxby commended the idea of linkage but brought up two relevant issues: (1) serious attrition of test-takers and (2) a tendency to "refresh" tests with new questions so they are no longer similar to the original one. She also suggested that since few states have their own institutions for conducting educational research, NCES could arrange to gather and archive state data.

Action: Dr. Granger suggested that Drs. Schneider and Hoxby meet to discuss the archiving idea further.

When Dr. Hanushek wondered about states having different frameworks for PISA, NAEP, or other tests, Dr. Schneider said that the main difference is that NAEP is already a national test; in fact, it is called the "gold standard." Dr. Whitehurst observed that state policymakers and education leaders already know how to align curricula with the NAEP; this information is not yet known for PISA. Dr. Granger commented that NCES could have the role of explaining the frameworks that are available and providing criteria-related information; then state personnel could make informed choices.

Dr. Schneider stated that while there have been cross-sectional studies, these do not provide any evidence concerning whether high scoring on any of the tests is associated with desirable life outcomes. Dr. Hanushek then noted that this is a good reason to promote obtaining longitudinal data.

Mr. Lee commented on the importance of finding out the costs (of testing and analyzing the resultant data).

Action: Dr. Granger requested that Dr. Schneider brief NBES at its September meeting on the questions raised during this meeting's NCES discussion.

Evaluation of IES
Joe Torgesen

Dr. Torgesen reported for the committee reviewing the evaluation of IES, which is being conducted by the contractor, Synergy Enterprises, Inc. (SEI). The evaluation involves three issues. These issues are (1) assessing the impact of IES on the rigor and relevance of education research, (2) laying the groundwork for future evaluations, and (3) development of policy recommendations that could improve IES' effectiveness. The evaluation process includes using existing data and obtaining new data through interviews.

SEI presented an interim report on April 15, 2008. Dr. Torgesen and his colleagues on the evaluation committee had concerns, given that the report was limited by missing data. The evaluation committee then prepared a memo with questions and a request to be informed about how SEI plans to pursue additional data. SEI, in turn, has sent a memo addressing the latter point. The evaluation committee now believes that the final report will reflect the full, agreed upon scope of work.

Communication Committee Report
Norma Garza for Jerry Lee

Ms. Garza distributed a handout recommended by Mr. Lee—A Critical Missing Piece in American Education Policy. Mr. Baron reported on an effort by a group of Board and non-Board members to act on Dr. Sally Shaywitz's idea to make the case in the press for rigorous scientific assessments in education. However, to date, the group has not been able to get its statement into major newspapers, such as The New York Times and The Washington Post.

Legislation Committee Report
Jon Baron for Beth Ann Bryan

Dr. Granger noted that the draft ESRA legislation in the meeting packet included recommendations developed by the legislative committee since the Board's January 2008 meeting. Mr. Baron said that the changes respond to the input of the Board and others. The main goal is to obtain a reauthorization without major substantive changes that could be controversial enough to delay the reauthorization.

The recommendations include language and technical changes, a process to avoid long-time vacancies, buffering IES from outside influences (to achieve integrity and independence), and keeping the peer review process managed by the IES director. The changes also include the Board's recommendations for a definition of scientifically based research and a lead position for IES for congressionally authorized scientific educational research under various education laws. There is also a change in how research and development centers are treated.

Mr. Baron then reviewed the draft, focusing on the major changes shown in the draft in red and blue ink:

Section 102. Definitions (pp.5–6): The principles of scientific research presented here as item (18) are similar to language that Congress has used elsewhere. The change is in Item IV regarding appropriate types of experimental methods.

Section 111 Item (b). Mission: The proposed revision makes the IES mission statement more succinct: "to provide national leadership in expanding reliable evidence on which to ground education practice and policy, and to encourage its use by parents, educators, students, researchers, policymakers, and the general public."

Section 113. Delegation: Language has been improved on pages 9–10. Item (b) is consistent with buffering IES and bolstering its priorities.

Section 114. Office of the Director: A technical correction provides continuity between directors by allowing the current director to remain up to a year beyond the end of his or her appointment until a new director has been appointed. Another change is intended to clarify when an appointment term begins. Some other federal offices set terms by the calendar, and some, by the individual, as is being recommended here.

Action: Dr. Whitehurst will confer with attorneys to make sure that the meaning is clear regarding when terms start.

Item (e) (9) has some additional language that specifies coordination with the ED secretary to ensure that IES' work is used.

Section 115. Priorities: As was discussed in January, this section now establishes a schedule for updating the Board's priorities for IES at least every 6 years.

After further consideration of Board powers, Board members also concluded that reviewing priorities more often than every 6 years is a good idea, but that this can be done as part of Board procedures rather than by statutory change. Dr. Whitehurst suggested that the Board could require the IES director to present a biennial plan for carrying out Board priorities. Dr. Baron and others then followed up by developing and presenting the following two proposals to the Board:

1. To add under the duties of the IES director—Submit to the Board, every 2 years, a biennial plan of activities consistent with the Board-approved priorities under Section 115 (of ERSA).
2. To add under the duties of the Board-Review biennial plan and advise the director on the biennial plan submitted under Section 114 Item (f) (2).

Board members voted unanimously to include these changes.

Section 116. NBES: Changes to this section broaden who can participate on the Board; recommend how long Board members may serve; provide more details about the Board's executive director; and simplify the language on standing committees.

Mr. Handy asked if other Board members are comfortable with the rules and duties that the statute specifies for the Board. He also asked whether the statute needs to spell out more clearly that NBES can speak not only to Congress, but also more widely on the status of education research. The consensus was that the Board was comfortable with the language as presented.

Section 117. Commissioners of the National Education Center: The legislative committee recommended keeping the Commissioner for Education Statistics as a Presidential appointee, but during discussion at this meeting, Board members decided to insert instead that all commissioners would be appointed by the IES director. The latter language had been recommended at the January 2008 NBES meeting.

Regarding how center commissioners are appointed, Dr. Granger reviewed how the Board had been weighing the balance between the prestige of a presidential appointment against the potential management challenges created by having three commissioners appointed by the IES Director and one by the President. He reported there was also some concern about the politicization involved in presidential appointments. Members also stated that it was important that the NCES commissioner not serve "at the pleasure" of the President or the Director, but that the commissioner be given a fixed term to protect against removal without cause. Several Board members voiced preferences for insulating appointments from politics. After discussion, the Board agreed to recommend appointment by the Director with a term, instead of appointment by the President.

To protect the commissioners from removal without cause, the Board also reached a consensus that the language in ESRA should include a statement noting that while the IES director can remove center commissioners for cause, he must first review the action with the Board. This language was added.

Section 133. Duties: Language changes sharpen the concept that peer review is at the director's level. This is also ratified in Section 134 (b). When Dr. D'Amico wondered about the $100,000 threshold, Dr. Whitehurst observed that that is actually a rather low amount for grants.

Other language changes provide that the IES director may, but does not have to, use research and development centers (or other appropriate vehicles) to build capacity.

Responding to an inquiry about "state and local policy" as a research topic, Dr. Whitehurst indicated that this phrase had been included in the original legislation. Leaving it out now might be too dramatic of a change.

Section 172. Commissioner for Education Evaluation and Regional Assistance: Outdated items were eliminated.

Section 173. Evaluations: Language is broadened to allow evaluations under other educational acts besides ERSA.

Section 183. Confidentiality: In the section about protections for students, the legislative committee decided to omit extending these protections to information with respect to individual schools.

Section 186. Authority To Publish: The changes involve provision of advance copies of publications.

Section 194. Authorization of Appropriation: This section increases the monetary figure for the maximum possible appropriation. Ms. Bryan discussed the receipt of some external advice to ask for higher funding.

Section 208. Grant Program for Statewide, Longitudinal Data Systems: An outdated section on providing an earlier report was removed.

Section 303. NAEP: The intent of the changes is to clarify how the NCES can release reports, but during discussion at this meeting, Board members suggested that the new language was still ambiguous. Dr. Whitehurst said that the intent is to clarify that the NCES commissioner is responsible for the content of the report, and that NAGB simply arranges for the public event for its release.

The Board then voted on the final package of recommended changes to ESRA, as developed by the Legislation Committee and revised by the Board during the meeting. The package was approved unanimously (a copy is attached for public release).

Dr. Granger noted that the recommended changes would become part of the Board's 5-year report to Congress regarding the reauthorization of ESRA.

Family Educational Rights and Privacy Act (FERPA) Panel Presentations

Dr. Hanushek explained that while the states now have an extraordinary ability to track students' achievement, ED funding currently has no requirement that anyone have access to the states' databases. After 5 years of developing regulations concerning who can look at data, and how, ED has issued draft new regulations and received comments on them. Since the regulations are not yet final, comments by the NBES board could definitely be useful. The key issues relate to who can have access to data on student performance and how to maintain a balance between research needs and student privacy.

Jane Hannaway

Dr. Hannaway indicated that her role on the panel is as a cheerleader for longitudinal data. State administrative data systems that allow for access to these data are important, enabling researchers to ask new questions and obtain better answers. Almost every state has unique student identifiers, and 18 states link teachers with students.

Four features of analytic systems are important. They include the ability to (1) follow students over time, (2) link teacher records with student records, (3) have "census" files that can match all students and teachers, and (4) provide history. The student-linked records can measure gains in learning. Teacher-linked records may be the most important factor in student achievement. Analyzing these (e.g., student performance vs. teacher credentials; factors improving the performance of a previously low-performing school) can help identify teacher effectiveness. The census files that can link all students and teachers allow researchers to conduct multiple comparisons, which can lead to tracking policy dynamics. For example, does a policy shift affect student gains, and, if so, how?

Connie Citro

Dr. Citro said that her topic is "Balancing Privacy Protection With Policy and Research". She noted that while the need for educational research has grown exponentially due to the No Child Left Behind Act and other initiatives, state and local education agencies are naturally wary concerning how FERPA applies to protect data until ED provides clarification. NAS has a history of more than 30 years of looking at and publishing reports on how to obtain and use research data while maintaining privacy and confidentiality, including—most recently—an April 2008 workshop on FERPA and student data. The newest area in which to examine these issues is biological specimens, including DNA.

Dr. Hoxby asked whether the ratio of returns to risk is higher in education than elsewhere. Dr. Citro responded that she is not aware of any rankings of relative risk.

When Dr. Hanushek inquired about whether the April 2008 workshop revealed any evidence of risky data releases, Dr. Citro said that there was no evidence of harm from any releases. Dr. Schneider agreed that there was no evidence of harm from any releases.

Chris Dougherty

Dr. Dougherty said that he also would address "Allowing Data Access for Research While Meeting Federal Privacy Requirements." He observed that the desire is not to waste databases, but that it is hard to quantify future benefits for policymakers; consequently, it is important to remind policymakers repeatedly of the research benefits. The benefits of school system and statewide research can be seen when one considers such alternatives as having to badger 100 individual school districts for data.

Dr. Dougherty reviewed three issues: (1) whether states can contract with third-party researchers without parental consent; (2) whether states can create databases combining early childhood, K–12, and higher education (already done in Texas); and (3) whether states (vs. local education agencies) can create databases combining education and non-education (e.g., employment) records.

Dr. Dougherty recommended (1) authorizing SEAs to enter agreements with research organizations under the "organizations conducting studies" exception; (2) allowing SEAs to record redisclosures of data for studies at the time the data are released for the study, not when the district first submits the data; and (3) clarifying that data can flow back from higher to lower levels of education (e.g., higher education to K–12) not just for record verification, but for evaluation.

Dr. Sommers observed that no one had mentioned yet the possibility of linking datasets across states (e.g., for students who move). Noting the requirement for an entity legally established as custodian of the data, Dr. Dougherty indicated that this is workable across states if the participating states develop a memorandum of understanding to identify the custodial agency. Dr. Whitehurst stated that FERPA does not address sharing data across state borders so this may need clarification in the final regulations. There is a relevant statement in the research grant process giving priority to grant proposals that link across states. Dr. Hanushek noted that the CALDER Center is able to do crossstate research, for which it can release results without releasing data.

Bill Evers

Dr. Evers introduced a colleague accompanying him, Mr. Rooker, and then indicated that as public officials overseeing federal regulations, they would need to limit their public comments. Nevertheless, Dr. Evers underscored the importance of enabling research.

Dr. Evers noted that FERPA is a records confidentiality and nondisclosure law with clear and explicit exceptions. The focus of the proposed new regulations is on access to data for research and evaluation. The regulations can detail what is allowed, but for each state, what agencies can do depends as well on what is authorized under state law. There are two major categories for data release—(1) deidentifiable data, for which small groups present masking problems, and (2) identifiable data, which can involve outsource organizations and require identification as to who is authorized and who controls the data. The new regulations propose substituting new language for the previous term, "easily traceable."

When Dr. Hanushek inquired about a date for putting the draft revised FERPA regulations into effect, Dr. Evers reported that at least 125 comments have been received, and Mr. Rooker predicted a late fall release of regulations.

Dr. Granger also asked about any particular aspects of the current law that could stand in the way of the balance between confidentiality and research. Mr. Rooker indicated that the preamble to the final regulations will comment on any statutory barriers.

The Board unanimously voted in favor of the following two action statements, as presented by Dr. Hanushek and amended during the subsequent Board discussion:

Action #1: The Board commends the Secretary and ED for moving forward in developing new regulations and guidance about how to maintain confidentiality of educational data under FERPA, while also providing for research uses of student and school data. We urge the Department to finalize these published draft regulations quickly, incorporating clarifications consistent with the Board's prior resolution to make longitudinal data more readily available for beneficial research purposes.

Action #2: We urge Congress to expand on the program of supporting statewide longitudinal data systems by requiring that states accepting funding under this program agree to make data in these systems available to qualified researchers (subject to FERPA) for the purpose of research that is intended to help improve student achievement.

Final Remarks

Dr. Granger requested guidance for the next Board meeting, scheduled for September 2008. He expects the agenda to continue to include commissioner reports, and continued work to prepare a report to Congress on the reauthorization of ERSA.

Mr. Lee proposed bringing in some of the best teachers to help the Board address the topic of raising the level of teacher quality. Dr. Cottingham indicated that the report from grants on staff development probably will not be ready by September; however, another impact evaluation might be ready then. Some other reports might be ready, (e.g., a school remediation strategy is due in June, and an afterschool evaluation is due for release soon).

Dr. Hanushek suggested including two or three 1-hour discussions of IES research (in-house or by grantees) so long as these are not redundant with other topics suggested here. Dr. Hoxby noted the discussions should include both a consideration of policy relevance and important methodological issues.

The meeting was adjourned at 12:40 p.m. on May 22, 2008.

The National Board for Education Sciences is a federal advisory committee chartered by Congress, operating under the Federal Advisory Committee Act (FACA); 5 U.S.C., App.2. The Board provides advice to the director on the policies of the Institute of Education Sciences. The findings and recommendations of the Board do not represent the views of the agency, and this document does not represent information approved or disseminated by the Department of Education.