Dr. Feuer said IES should be proud of the results of this analysis, because it is taking the lead despite being new to the funding arena.

Standards and Review Office
Anne Ricciuti, Ph.D., Deputy Director for Science

Dr. Ricciuti described the responsibilities of the Standards and Review Office (SRO) and where it is situated within IES. Dr. Ricciuti pointed out that the SRO is not an official office in IES but rather the name used to refer to peer review activities and staff under the Deputy Director for Science. All peer review activities and staff are situated outside of the four research centers so that peer review of reports and research grant competitions is as objective as possible.

Dr. Ricciuti said the SRO reviews reports from all four centers using a process modeled on that of journals. Reports undergo one of two types of review: external or internal. In general, reports that present new data analyses in a form other than basic cross-tabulations are sent to external experts for peer review. Research reviews and syntheses also receive external review. Reports with limited data analyses and indicator reports general receive internal review. SRO staff works with the centers to ensure reports meet SRO standards before they are released. Every year, about 80 reports are in some stage of the review process.

Most of the research grant competitions come from NCER and NCSER, although NCES holds the Statewide Longitudinal Data competition. The grant review process is modeled on that of NIH and NSF. In FY 2015, the SRO processed 1,099 applications; 992 applications were reviewed by 425 reviewers on 23 review panels.

Dr. Singer wondered whether journals would be better suited to review reports, because they already have a well-developed system for reviewing data and would better disseminate research findings. She noted that journal publication factors into tenure review for academic researchers. Dr. Ricciuti responded that some reports have to be released by IES, such as Congressionally mandated evaluations. She said IES established a rigorous peer review process to elevate the standing of its reports.

Dr. Neild added that the cost of internal peer review is relatively small, particularly in comparison to the costs of large-scale data collections and studies, and substantially improves the quality of reports. The SRO has demonstrated that it is scrupulously fair, nonpartisan, and very careful in its review. Dr. Neild also noted that NCEE contractors are encouraged to submit articles to journals for publication. She hoped academia would recognize the extent of the IES peer review process and weigh IES reports more heavily in tenure and promotion decisions.

Dr. Gamoran supported the role of the SRO in quality control and peer review of reports. All reports, including those of contractors, should be reviewed by staff separate from the research centers and should carry the IES "stamp of approval." The basic descriptive information published by NCES should be disseminated through reports that are reviewed by an impartial party like the SRO. Dr. Gamoran said reports are more likely to be produced by contractors than academics in tenure-track positions; he agreed that more reports should be submitted to journals so they become part of the scientific literature. However, said Dr. Gamoran, there is room for debate over how universities should evaluate peer reviewed products written for government agencies. Organizations that seek to promote the use of research evidence in policymaking should recognize that policymakers are more likely to turn to IES reports than journals, Dr. Gamoran noted.

Dr. Feuer agreed that the SRO should continue to play a role in publishing independently reviewed reports that carry deserved weight, especially because of the growing number of "advice givers" in the field (e.g., think tanks). Dr. Ricciuti said SRO staff looks not only at the technical quality of reports but also at the objectivity of the interpretation and presentation of findings.

Returning to the research grant process, Dr. Ricciuti described the typical timeline for review. She pointed out that screening the applications involves multiple offices. The SRO staff must assess whether the application meets the technical requirements of the competition. From start to finish, the process takes 8–9 months, comparable to that of the NIH.

Dr. Ricciuti described how the number and timing of NCER and NCSER competitions and submission dates changed because of dramatic funding cuts in FY 2013. The SRO adjusted to accommodate the changes. The timeline continues to be about 8.5 months from the time the RFA is published to the release of regret letters to applicants who did not receive awards. In FY 2015, despite an increase in applications, the SRO managed to allow more time for processing and reviewer assessment and earlier panel reviews. It was also able to release review summaries and scores through the new online review system before funding decisions were made (as NIH does).

Dr. Ricciuti said two FY 2015 off-cycle competitions, both with accelerated timelines, contributed to an extremely demanding year for the SRO. For one, the time from receipt of applications to release of regret letters was 6 weeks; for the other, 13 weeks.

In FY 2016, the SRO aims to shorten the application processing time by 3 weeks, give reviewers more time to evaluate applications, and hold review panels earlier in the year. It also plans to release scores online within a few days of the meeting and summary reviews about 1 month after the panel meets. Dr. Ricciuti said FY 2016 also brings two new off-cycle competitions, both of which differ from other types of IES competitions. Both will require substantial strategic planning, and both have accelerated timelines.

Dr. Ricciuti described some of the strategies used to accelerate the review process. For example, for the evaluation of state and local education programs and policies, the SRO was able to recruit reviewers before the applications were submitted because the focus was specific and limited and the number of applications anticipated was small. In addition, SRO used videoconferencing technology for the panel review.

Dr. Ricciuti welcomed suggestions to shorten the timeline for grant review while maintaining the quality of the process, avoiding overburdening reviewers, and providing adequate time for reviewer evaluation. Because she often hears complaints that the process takes too long, she asked what the target timeline should be.

Discussion

Dr. Feuer said the credibility of an organization is largely a function of the rigor of its review process. Once that credibility is compromised, the rest of the organizational investment is compromised. He recommended the SRO weigh any suggestions to shorten the review process very carefully to assess the risks and benefits. Identifying and addressing conflicts of interest is a very important part of the process, he added. Dr. Ricciuti said the SRO has an extensive, proactive process for assessing conflicts of interest.

Dr. Gamoran said the timeline seems appropriate given the volume of applications and the quality of the feedback to applicants. He said the resubmission process has improved greatly, to the credit of Dr. Ricciuti and the SRO, but continued vigilance is warranted. Dr. Ricciuti summarized the results of resubmissions (see the table). For both NCER and NCSER, resubmissions are funded at higher rates than first submissions. Dr. Ricciuti noted that the new IES website will include such data.

1 Number of Applications Funded and Funding Rates by Submission Status for NCER and NCSER Main Education Research Competitions; FY2009 through FY2015

Dr. Singer wondered if the overall process would benefit from having multiple submission deadlines throughout the year. She noted that the panel review of applications serves as a triage process, because most applications are not funded on the first submission. A more formal triage process would not speed up the process but could allow for decisionmaking in a more timely fashion.

Dr. Singer pointed out that most organizations do a poor job of screening out reviewers with a conflict of interest that stems from having had their own applications for similar research rejected. Such conflicts are not declared and may be hard to find. Dr. Singer also said IES has a treasure trove of data in its reviewer scores, and she suggested Dr. Ricciuti think about how to delve into that. For example, if some reviewers' scores highly correlate with those of others, it may not be necessary to have both reviewers. Sometimes, dissonance is better than consonance.

Regarding whether the same reviewers should evaluate a resubmission, Dr. Singer said fresh eyes should review it. The current triage system effectively guarantees continuity. She said there is a role for both approaches (continuity vs. fresh eyes).

Dr. McLaughlin noted that because of the limited funding, NCER and NCSER must continue to have one submission deadline so that they can ensure sufficient funds for awards.

Dr. Chard encouraged the SRO to keep working with NSF and NICHD and to continue putting more information online. The data show that the resubmission process has been working since 2009, despite some perceptions to the contrary.

Lunch

The Board adjourned for lunch at noon, during which members reviewed the Board charter in a closed session. The public meeting resumed at 1:03 p.m.

Overview of the National Center for Education Statistics
Peggy Carr, Ph.D., Acting NCES Commissioner

Dr. Carr gave a brief history of the NCES, which traces its origins to Congressional language from 1867 directing the collection of data on the status of education in the states and territories. The commissioner meets monthly with the heads of 12 other federal statistical agencies. The NCES staff numbers 112, and Dr. Carr is seeking to increase that number, because NCES carries a very heavy load.

According to legislation, NCES' mandate is as follows:

• Produce and disseminate relevant and timely information
• Conduct credible and accurate statistical activities
• Conduct objective statistical activities to collect data that are impartial, clear, and complete
• Protect the trust of information providers by ensuring the confidentiality and exclusive statistical use of their responses

The mission of NCES is to collect, analyze, report, and disseminate education information and statistics in a manner that:

• meets the highest methodological standards;
• is timely, relevant, and useful to practitioners, researchers, policymakers, and the public;
• is objective, secular, neutral, and nonideological; and
• is free of partisan political influence and racial, cultural, gender, or regional bias.

In addition, NCES data collection is guided by four core values: quality, usefulness, predictability, and timeliness. For example, a 2004 push to improve timeliness and predictability led NCES to decrease the timeline dramatically for reporting results of the National Assessment of Educational Progress (NAEP). Every NCES commissioner has made quality a high priority. Dr. Carr said she is also focusing on usefulness of data.

Outlining the organization of NCES, Dr. Carr noted that the Statistical Standards and Data Confidentiality unit was created to enhance NCES' credibility. She pointed out that 85% of the staff has advanced degrees. Data collection efforts cover the lifespan (see Figure 2). Despite the wide range, Dr. Carr said NCES does not collect all data as often as she would like.

2 Selected NCES Data Collections

Products created by NCES range from First Looks—short, initial reports on data—to detailed statistical analyses and R&D reports. The center also offers numerous services, such as providing data management expertise to other ED staff, leading data work in the field, providing leadership for education assessment, and offering technical data collection and systems expertise. It organizes conferences on best practices and facilitates training, which boost relationships with individuals at the school and district level.

The NCES budget is divided into two separate lines. For FY 2015, the statistics budget (including international assessments) is about $100 million, and the national assessments budget is about $129 million.

Dr. Carr described the numerous types of contracts and agreements NCES relies on to do its work. The most challenging and costly component of the work is the technology that NCES brings to schools to conduct assessments. Dr. Carr said NCES is affected by the pervasive decline in survey response rates; in some cases, response rates are so low that results cannot be reported. Other challenges include complex internal processes (such as clearance and confidentiality issues), the quality of external data provided to NCES, programmatic vs. statistical use of data, funding uncertainty, and understaffing.

Discussion

Hirokazu Yoshikawa, Ph.D., asked why the NCES national assessment budget increased substantially from 2008 to 2009. Dr. Carr responded that when the NAEP was tied to No Child Left Behind, NCES received an additional $68 million. Also, in 2009, the NCES budget was increased to fund a full schedule of assessments. In response to Dr. Feuer, Dr. Carr clarified that the money from the statistics budget line can be used to support national assessments, but not the other way around. She added that NCES accounts for nearly half of the IES budget, and NAEP is a big part of the NCES budget.

NCES Assessment Division

Dr. Carr said that combining the NAEP and international assessments into one division was a good move, because the assessments share the same designs, vision, and goals. The NCES assessments monitor programs in the United States and internationally to provide indicators and to measure trends over time. Dr. Carr said NCES informs the world about what students know and what they can do.

Within the Assessment Division, NAEP accounts for about 60% of the work, which includes developing test items, collecting data, and managing NAEP state coordinators and NAEP ambassadors. The international branch has grown tremendously in the past 10 years, starting with the Trends in International Mathematics and Science Study (TIMSS) and more recently with strong interest in the Program for International Student Assessment (PISA). Both the NAEP and international assessments are led by governing bodies. A separate branch of NCES provides administrative support to coordinate contracts and budgets, for example.

Dr. Carr said NAEP is mandated by Congress, and states are required to participate in certain assessments every 2 years. Results of NAEP assessments are reported within 6–12 months of data collection via the First Look reports. The assessments are supported by NAEP state coordinators, who are not only liaisons with schools but also play an integral role in data collection and quality assurance. The first long-term trend assessments in reading and mathematics began in 1968.

Dr. Carr described the NAEP functions as a three-legged stool that relies on the National Assessment Governing Board (NAGB) for policymaking, external contractors for services ranging from item development to scoring, and NCES for operations support. At the peak of the effort, 7,000 people collect NAEP data over the course of 6 weeks.

International Assessments

Daniel McGrath, chief of the Reporting and Dissemination Branch, said international assessments focus on children beginning in fourth grade and continue to adulthood. Assessments from elementary through high school include TIMSS, PISA, the Progress in International Reading Literacy Study (PIRLS), and the Teaching and Learning International Survey (TALIS). The Program for the International Assessment of Adult Competencies (PIAAC) assesses literacy.

The International Association for the Evaluation of Educational Achievement conducts TIMSS and PIRLS. It has a general assembly in which NCES participates to develop policy and strategy. The Organization for Economic Cooperation and Development (OECD) sponsors PISA, PIAAC, and TALIS. Mr. McGrath and his colleagues represent NCES on various OECD boards that guide policy.

The International Assessment branch contributes to study design and development and leads data collection and operations. It also advocates for the development of educational measures, with NAEP as the gold standard and model for most international studies. Finally, the branch provides analysis, reporting, and dissemination of results.

Study design and development includes matrix sampling used by NAEP and all the international assessments. All the IES centers use the evidence-centered design created by the branch. The development process is extensive and can take up to 4 years. A number of task forces provide input on study design.

Developmental research to advance educational assessment is conducted through five contracts. Other efforts to advance assessment include the Survey Assessment Innovation Lab, a virtual laboratory to assess student skills and knowledge. Through PISA, the branch is looking at student behaviors during computer-based assessment to determine, for example, whether there are optimal strategies and what contributes to low performance.

In support of dissemination, the branch has created many tools and reports targeted to various audiences. They are available on the NCES website.

The changing landscape of education poses challenges. For example, the rapid, uneven transition to digital devices in classrooms and homes means habits are changing. There is an opportunity to collect a lot of data through digital devices. The variability in transitions to new standards, curricula, and assessments also poses a challenge.

Discussion

In response to Dr. Gamoran, Dr. Carr explained that NAEP and TIMSS measure the same age ranges. She believed the two studies could be integrated without abandoning either. Mr. McGrath added that PISA and TIMSS provide different information.

Dr. Singer pointed out that the National Academy of Education, under the direction of Dr. Feuer, is evaluating NCES efforts, including international assessments. One initiative will address the number of assessments and opportunities to combine them. Dr. Gamoran said the initiative reflects a refreshing openness to researchers on the part of NCES, and he hoped NCES would continue to support such initiatives. Dr. Feuer added that there may be some collaboration with the National Academy of Sciences and opportunities to bring in experts in demography and other social sciences. The most powerful rhetoric to support education reform comes from headlines comparing U.S. students with those of other countries, he said.

Mr. McGrath noted that TIMSS provides an international angle and is not very expensive; NCES has already saved money on administrative support by leaning on NAEP. Dr. Carr said the burden of testing is more significant than the cost. However, she concurred that TIMSS acts as an indicator and provides needed information.

Sample Surveys Division

Chris Chapman, associate commissioner, said the division designs and conducts sample surveys around individuals, schools, and school systems, then disseminates the findings. The surveys seek to gather data not found in administrative records, such as individual attitudes and motivations. The staff is divided according to the type of data collection—longitudinal surveys and cross-sectional surveys.

Longitudinal surveys have a range of cohorts, from newborns to college-age students. In early education, data come from parents and providers; older students provide data directly. The longest-duration studies collect data for up to 10 years.

Regular cross-sectional surveys include the National Household Education Surveys, the National Teacher and Principal Survey (NTPS), the Private School Survey, the School Crime Supplement, and the School Survey of Crime and Safety. As-needed studies include short, topical NTPS modules; Fast Response Surveys and Postsecondary Quick Information Surveys; and Career and Technical Education Statistics. In addition, the branch develops cross-sectional surveys for ED's Education School Climate Survey and the Recent Graduate Employment and Earnings Survey.

The division spends a lot of time disseminating results. Online, the Datalab tool allows users to sort through data while maintaining correct statistical weights and variances. With eDAT, users can download raw data for analysis.

Discussion

Dr. Gamoran appreciated the openness of the branch and NCES in general. He asked what the survey branch needed most; Mr. Chapman said the branch needs more staff.

Administrative Data Division

Ross Santy, associate commissioner, said the mission of the division is to oversee the collection of "universe" data from state and local education agencies and postsecondary institutions efficiently and with minimum burden. Much of the division's efforts involve working with others in ED and users of the data. The division's three branches correlate with the data sources. The State Longitudinal Data Systems Branch supports state capacity for data system management. The Postsecondary Branch and the Elementary and Secondary Branch operate data collection and reporting in schools.

Data collected include numbers of students (meeting specific criteria), administrators, and faculty; levels and amounts of resources; and counts by status. These data are important to establish a strong, reliable data chain.

The Postsecondary Branch runs the Integrated Postsecondary Education Data System, which collects data from all institutions that participate in federal student financial assistance programs. Data are released through First Look reports, methodology reports, data files, and technical review panel reports. The branch also maintains online tools; it is currently reconfiguring its website to better support mobile access.

The Elementary and Secondary Branch runs the Common Core of Data (CCD) and EDFacts and supports civil rights data collection. The branch works with other programs to determine what they need and what data they can provide. The branch seeks to improve CCD data management and is working with EDFacts Datamarts for ED programs.

The State Longitudinal Data Systems Branch focuses on the Statewide Longitudinal Data Systems, which recently added 16 new teams. In addition to annual papers, updates, and online tools, the branch helps states improve data collection.

The Administrative Data Division also supports developing work. For example, it is ramping up its mapping and geography efforts to incorporate more geographic census data. A new mapping tool, MapED allows users to explore information at national and local levels. Data integration projects include the Wage Index (which incorporates teacher salary and geographic data). The ED Data Inventory is an example of the division's development contributions to data use and transparency.

The division seeks to develop and maintain a national network, which involves a lot of relationship-building and maintenance. Mr. Santy concluded by pointing out that NCES does not directly control any of the systems or organizations that manage the data needed to create high-quality education statistics.

Annual Reports and Information Unit

Thomas D. Snyder, program director, said the unit oversees numerous reports, including the annual Condition of Education report mandated by Congress. The Digest of Education Statistics is widely used, especially by graduate students. Examples of topic-specific reports are the Indicators of School Crime and Safety, which was compiled with data from the Department of Justice, and Trends in High School Dropout and Completion Rates in the United States. The Projections of Education Statistics report is often incorporated into other reports and used by businesses, educators, and policymakers. This year, the unit revitalized its series Status and Trends in the Education of Racial and Ethnic Groups. It also oversees production of other special reports using new NCES data.

The unit responds to data requests from policymakers, ED program staff, and others. Mr. Snyder said NCES often gets media coverage because it is able to respond to media requests for information quickly. The unit also coordinates the publication planning and review system and provides formula grant allocation services. It supports various popular ED websites, including Fast Facts. The unit works with the Federal Interagency Forum on Child and Family Statistics and the OECD.

The unit coordinates social media and outreach for NCES. With the growing use of mobile devices, NCES increasingly relies on mobile-friendly videos, its blog, and Twitter to provide updates and announcements and to reach a broader audience.

Statistical Standards and Data Confidentiality Unit

Andrew White, senior research statistician, sitting in for Chief Statistician and Program Director Marilyn Seastrom, said the unit's mission is to support NCES in the collection and analysis of education statistics in a manner that meets the highest methodological standards, while ensuring that the data are timely, objective, secular, neutral, nonideological, relevant, and useful. Activities include promoting statistical standards and protecting data confidentiality. The unit offers technical support to NCES and other statistics organizations and coordinates quality review.

The unit provides training through the NCES Distance Learning Dataset Training System, which helps users identify and use relevant NCES data. Training also explains how to find and use online data tools. To support training, the unit produces the NCES Statistical Standards, Tabular Guidelines, and Handbook of Survey Methods. It also supports the ED Data Inventory and the National Forum on Education Statistics.

NCES Stakeholders
Nancy Potok, Ph.D., Deputy Director, U.S. Census Bureau

Dr. Potok said the Census Bureau and NCES are not only partners and colleagues, but NCES is also a client of the Census Bureau, in that it relies on the Bureau's data collection efforts for some initiatives. Both organizations are part of the federal statistics system, and both participate in the Interagency Council on Statistical Policy and its many working groups.

The Census Bureau is among the largest statistical agencies, with an annual budget over $1 billion and 17,000 employees. Much discussion among the statistical agencies revolves around independence and the relationship of statistics to policy, the need for objective data, the stress of changing administrations, and the influence of department leaders on statistical agencies. The agencies tend to band together to uphold the standards and independence of federal statistics and to move methodology forward.

The Census Bureau is involved in at least a dozen NCES surveys. The two agencies are working together to improve data collection, advance methodology, make data collection more efficient for respondents (especially for school districts taking multiple surveys), and improve the quality and timeliness of data. In response to Dr. Chard, Dr. Potok explained that the Census Bureau has interagency agreements with NCES to support data collection implementation and processing, survey design, timing, and other areas. In some cases, the Census Bureau acts as a contractor for NCES.

Brian Harris-Kojetin, Ph.D., Deputy Director, Committee on National Statistics, National Academy of Sciences

Dr. Harris-Kojetin formerly served in the Office of Management and Budget (OMB) and worked closely with the 13 statistical agencies. In that position, he reviewed numerous ED and NCES evaluations and surveys, and he offered his perspectives on the basis of his interactions with NCES staff and programs as well as interagency interactions. Dr. Harris-Kojetin said the federal statistical system can be fragmented but works well.

All the statistical agencies face challenges from declining cooperation, flat budgets, increasing costs, and growing concern about the burden on respondents. Dr. Harris-Kojetin said NCES is a key innovator in the field. Thanks to NCES, the OMB disseminated a memo on increasing the use of administrative records for statistical purposes, establishing a broader, government-wide foundation for the NCES approach.

Dr. Harris-Kojetin raised the issue of a potential loss of stature and autonomy for NCES since it became part of IES. He said IES is a model for other agencies, but he was not sure IES has been good for NCES. The statistical community is concerned about the effect of IES on the mission of NCES, he noted.

Michael Casserly, Executive Director, Council of the Great City Schools

Mr. Casserly said CGCS is a coalition of large, urban, public school systems and is an avid user of NCES data. He expressed appreciation for Dr. Carr's leadership and said he believes she is one of the most valuable public servants in federal government. The CGCS is an NCES stakeholder at the ground level, mostly around NAEP data. It works closely with NCES and NAGB on the Trial Urban District Assessment (TUDA).

Mr. Casserly said the CGCS suggested to NAGB 15 years ago that it create TUDA to demonstrate that urban public education is committed to high standards. The assessment enabled comparison of urban school districts across state lines, which other assessments did not allow. It also provided a mechanism for evaluating reforms.

Data from NCES enabled the CGCS to answer questions about academic attainment and other issues in urban public schools. Mr. Casserly said he is grateful for NAEP's contribution and enormously pleased with NCES' continuing collaboration with the CGCS. Other NCES products, such as data on staffing, budget, finance, and salaries, provide unique databases allowing comparisons across urban school districts. Mr. Casserly said NCES is a reliable partner, particularly for NAEP.

Mr. Casserly acknowledged that it takes a long time to analyze data and get it right, but the lack of timeliness can be a barrier to utility. He suggested establishing an ongoing collaboration between NCES and large urban school districts on issues such as data quality and usefulness. In the 1970s, the CGCS had a standing advisory group of urban school researchers and testing directors who met with NCES staff to discuss details and utility of the data. It was a mechanism for NCES and stakeholders to share information.

Discussion

Asked for an example of combining data activities, Dr. Potok said the Census Bureau is doing a lot to link records. The Bureau is required to use data available from existing records whenever possible instead of collecting the same data again. The American Community Survey (ACS) is a rich data set that NCES might use to look at education programs in impoverished geographic areas. More can be done to link data, said Dr. Potok.

Dr. Potok noted that bipartisan legislation introduced in the U.S. Senate would create a clearinghouse for federal records, which the Census Bureau supports. Education records are the only dataset the Census Bureau cannot access because privacy protections override the Bureau's mandate. A mechanism that combines ED data, Census data, and ACS data could be very useful, said Dr. Potok.

The Census Bureau has research data centers across the country that allow researchers to delve into records linked by the Bureau staff. Most of the data centers are located at universities, while some are at agency offices. The Bureau recently proposed expanding the data centers to include all the federal statistical research agencies. By doing so, data could be linked in a secure environment, with privacy and confidentiality protected, opening up new doors for research, said Dr. Potok. Dr. Feuer said education researchers are very sensitive to privacy and confidentiality issues.

Dr. Feuer asked Mr. Casserly for advice on how to protect TUDA data against the inevitable desire to use those data as a scorecard. Mr. Casserly said the CGCS knew that asking for comparable data and public reporting would pose that risk. However, the ability to collect data to improve education remains critical. Mr. Casserly said that NCES gets better every year at presenting the data in a manner that does not invite the scorecard mentality or feed the urge to draw simplistic conclusions. Mr. Casserly believes TUDA is the way to address the enormous pressure to improve academic attainment, close achievement gaps, and respond to public concerns.

Dr. Gamoran asked whether a data clearinghouse is needed if the Census Bureau is already working on expanding its data centers. Dr. Potok said the Census Bureau has a legislative mandate to use administrative records when possible but no corresponding legislation that requires other agencies to provide that data to the Bureau. Gaining access to records can take years of negotiation, said Dr. Potok. A clear Congressional mandate could set an expectation for sharing records.

Dr. Potok continued that when data are collected, they become the Census Bureau's property, which triggers stringent privacy protections and makes it very hard to get the data back to those who provided them or to researchers. Any mechanism that addresses privacy, security, and informed consent in a way that the public finds satisfactory would be welcome, said Dr. Potok. A clearinghouse with standards, expectations, and protections that allows research access would be a good step.

Dr. Harris-Kojetin said the research community is very enthusiastic about the proposed legislation, which raises the profile of statistical data. He hoped there was an opportunity to leverage existing structures and current protection methods.

Asked to expand on his contention that IES may not be good for NCES, Dr. Harris-Kojetin explained that the Education Sciences Reform Act (ESRA), which is up for reauthorization, transferred some of the autonomy and independence of NCES to IES. Dr. Harris-Kojetin said that IES has helped build a strong scientific research arm for ED, and the consolidation of various research components in ED has been helpful. However, NCES has lost some of its autonomy and clout. A recent law that gave fixed-term confirmation to the leadership of the Census Bureau took away the Senate confirmation requirement for the NCES commissioner. The proposed legislation to renew ESRA would demote the NCES commissioner to an appointee of the IES director, which Dr. Harris-Kojetin thinks is a major threat. The federal statistical agencies should have independent heads, he said.

Dr. Gamoran said the Board used to advise Congress on how to treat IES but the members were split on the issue of appointment of leadership. Half did not want to give up the rare position that afforded someone in education such high status, while the other half argued that IES would function better if the centers were all on equal footing. Dr. Gamoran suggested that at a future meeting, the Board should discuss whether the NCES commissioner should be a Presidential appointee.

Dr. Ford asked if there were other concerns beyond the loss of stature. Dr. Harris-Kojetin said NCES could become buried further down in the bureaucracy, as is true for other federal statistical agencies. Dr. Neild disagreed, saying that it is entirely different when a statistical agency is placed within a non-independent policy office, as opposed to being part of a larger scientific agency that has statutory independence and publication authority. By virtue of authorizing legislation, all of IES, including NCES, is granted this independence. She said that a compelling case had not been made for actual harm to the work of NCES.

Because the Census Bureau is responsible for sensitive economic data, the potential for political manipulation is always a concern, said Dr. Potok. It is important to have a person of stature at the helm who can remain independent from politics. Without sufficient stature, an agency's products can be manipulated by others to suit a specific agenda. When the agency head is confirmed by the Senate, that individual has more "muscle" and can take findings directly to the Secretary. Dr. Harris-Kojetin emphasized that the concern about independence is real. He said ED could reinforce its position by ensuring that there is no mechanism for manipulation or control of NCES data.

Mr. Casserly urged the Board to give more attention to stakeholders at the local level who would like to use more NCES data. Local school administrators, researchers, and assessment directors, for example, even in big cities, are unable to take full advantage of NCES data. There are steps NCES should consider to be more relevant to people in the field, which would also increase the number of people who advocate for NCES when it seeks higher appropriations.

Dr. Potok strongly agreed with Mr. Casserly and pointed out that responsiveness will continue to decline unless data collection agencies can give respondents back something of value. The more NCES and other agencies can put data out in formats that people can use to help them do their jobs and inform their decisions, the more the agencies can justify asking for the data. Linking administrative records would lead to more timely information at narrower geographic levels that could be fed back to the primary data users, not just researchers, Dr. Potok added.

Mr. Casserly said TUDA provides very detailed data that allow users to look into all kinds of questions of immediate interest to those who participated. With other efforts, NCES should aim to provide more data on useful topics, such as financing and staffing. However, Mr. Casserly noted that the more granular the data, the more suspicious users are of the accuracy of the data, and school districts will not use them. When NCES gets it right, it does well, said Mr. Casserly, but when studies are problematic, the utility of the data decreases.

Deborah Phillips, Ph.D., pointed out that low response rates contribute to all sorts of data collection problems, such as utility and timeliness. They may also reflect survey fatigue or personnel turnover. Mr. Casserly agreed that responsiveness is a serious concern, but when school districts want real, useful data, the burden is not an issue. For example, the CGCS collects substantial academic and operational data annually, yet school districts do not complain, because they appreciate the frequency, granularity, timeliness, and nature of the data.

Dr. Harris-Kojetin said household survey responses are declining in every rich country. Dr. Potok said a "bad" response rate for Census Bureau is about 70%, but she also agreed that response rates are declining. The level of effort required to get responses is becoming more intense and expensive. Dr. Potok said education survey fatigue could be addressed with a more holistic view of the survey landscape for schools. That is, researchers should look at the schools in the sample and consider how often they are asked to respond and also the timing of surveys. For schools that must take part in multiple surveys, researchers could at least acknowledge the burden and work with the schools to provide meaningful data back to them. She conceded that doing so is difficult when various contractors conduct the surveys.

Dr. Feuer said some political resistance increases the perception of the burden of government-sponsored surveys and leads to dismissal of their utility. He suggested that the CGCS may get better responses because it is not a government agency. Mr. Casserly pointed out that his organization cannot mandate compliance. Stakeholders tell the CGCS what data they need to inform practice. Mr. Casserly again encouraged the Board to create a mechanism that would give local schools and communities more opportunities to share information with NCES and a better sense of what NCES is doing.

Dr. Phillips agreed that steps can be taken to lessen the imposition, such as engaging participants so they feel more ownership, providing timely results, and avoiding overlap between state tests and other surveys, for example. Dr. Harris-Kojetin said some of the problems are structural. Specifically, PISA was inconvenient and offered no payoff to schools, so it frequently did not meet thresholds for reporting. However, in countries where PISA serves as the equivalent of the NAEP, there are no problems.

Dr. Yoshikawa said a National Academy of Sciences panel on integrating immigrants into U.S. society recommended collecting data on the children of immigrants by parents' birthplace through the ACS. He asked what efforts are underway to gather more data on youth. He also asked whether NCES provides input into OECD initiatives other than PISA. Dr. Carr responded that NCES is active in many OECD initiatives. Dana Kelly, chief of the International Assessment Branch and chair of an OECD committee on early childhood education, said NCES works with various entities to avoid duplicating efforts and to ensure that survey items are appropriate for U.S. populations.

Mr. Chapman said NCES does collect data on parents' birthplaces, but the samples are not big enough to provide meaningful data by country or region of origin, because it is a nationally representative sample. He said NCES could work with the Census Bureau on the ACS, but it would need a legislative mandate to add questions. Mr. Chapman thought that, potentially, some data could be collected from administrative records. Mr. Santy added that school systems that get Title III funding (to address students with limited English proficiency) might have data on parents' birthplace. Other programs may have academic achievement results for English language learners, but the two datasets do not overlap. Mr. Sandy said NCES has not yet found a way to get detailed data about the children of immigrants that is not burdensome.

Dr. Phillips asked where data on children with special needs falls into the survey landscape. Dr. Carr replied that NAEP has the largest sample of such children but cannot break down the data too far. Administrative data collections capture some students with special needs, and so does the middle school survey. There is a representative set of indicators for students with special needs, Dr. Carr added. Dr. McLaughlin pointed out that a future iteration of the middle school survey will have a section on students with special needs. Mr. Casserly noted that broader data are available from ED's Office of Special Education Programs and Office for Civil Rights. Mr. Santy said a lot of data could be gathered from various existing sources with more resources and time. He suggested a restricted-use license could be granted to a researcher who would like to work with the data.

Dr. Chard and Dr. Carr thanked the stakeholders for bringing new ideas to the table.

Closing Remarks & Adjournment
David Chard, Ph.D., NBES Chair

Invited to give her impressions of the day, Dr. Neild said she felt a key takeaway is the need for IES to work with NIH and NSF on how the organizations' funding streams influence one another. She said NCES is incredibly productive and creative, despite its very small staff, as is the SRO. Although government is not known for being nimble and creative, said Dr. Neild, IES is both, doing a tremendous amount to support the research infrastructure. She reiterated her request that the Board consider how IES should position itself to preserve its importance and the high quality of its work in the next administration.

Dr. Chard thanked all the Board members for their participation and gave special thanks to Dr. Gamoran and Anthony S. Bryk, Ed.D., who have completed their final terms. He said both have always brought good questions and insight to the table. Dr. Chard, Larry V. Hedges, Ph.D., and Dr. Yoshikawa have completed their first terms on the Board and do not yet know if they will be reappointed. If Dr. Chard is not reappointed, Vice Chair Dr. Loeb will lead the next meeting, and elections for a new chair and vice chair will be held.

Dr. Chard said the Board has been looking for ways to better integrate the Board into IES' work constructively. The Board charter has provisions for standing committees, which could include committees that align with each of the centers. If such committees were developed, the commissioners would work closely with the Board to identify which Board members should serve on which committees. Such an arrangement could mean that the committees and the commissioners hold their own discussions and present jointly at Board meetings. An ad hoc committee could be formed to address special issues of interest to IES, allowing Board members to dig deeper into IES projects and offer a more productive role for Board members.

Dr. Chard said the next NBES meeting is scheduled for February but no date has been determined. He adjourned the meeting at 4:22 p.m.

Report prepared for NBES by Dana Trevas, Shea & Trevas, Inc.

The National Board for Education Sciences is a Federal advisory committee chartered by Congress, operating under the Federal Advisory Committee Act (FACA); 5 U.S.C., App. 2). The Board provides advice to the Director on the policies of the Institute of Education Sciences. The findings and recommendations of the Board do not represent the views of the Agency, and this document does not represent information approved or disseminated by the Department of Education.