2011 National Board for Education Sciences Annual Report
National Center for Education Statistics

The National Center for Education Statistics is moving in new directions to make the most of the data it is collecting. NCES is redesigning the content, collection modes, starting points, and periodicity of its studies to keep up with current policy priorities and research needs. In addition, NCES is exploring ways of integrating data from its studies with administrative data in all areas of education.

Program Highlights

Redesign of Longitudinal Studies

The Center has incorporated redesign work into the development of its longitudinal studies to keep their content current with changes in the educational experiences of our children and youth. For example, in the Early Childhood Longitudinal Study: Kindergarten Class of 2010–11 (ECLS-K:2011), NCES developed a new reading assessment for children who speak Spanish, but not English. It is intended to address a problem identified in the first longitudinal kindergarten study in 1998–99, which could not capture information about these children's reading skills, making the development of growth models in reading a challenge.

Redesign has also been incorporated into NCES's newest secondary school longitudinal study. The design of the High School Longitudinal Study of 2009 (HSLS:09) is similar to past studies but with several innovations: all surveys and assessments were administered via computer; school counselor and parent input into students' decision-making about courses and postsecondary choices was and will continue to be collected; and the mathematics assessment developed specifically for this study focuses on algebra skills critical to success in secondary and postsecondary education.

NCES also has redesigned the timing and frequency of data collection in these longitudinal studies. The first longitudinal kindergarten study from 1998–99 has provided a wealth of useful information. However, because of budget constraints, the study had to skip the second and fourth grade data collections originally scheduled. As research has shown, the learning curves for children are steepest in the earliest years of their lives. The lack of data from the second and fourth grade years has limited analyses useful for understanding when children start to experience problems in school or when learning problems are ameliorated. To address this shortcoming, the new ECLS-K:2011 has collected data in the fall and spring of kindergarten, and the plans are to continue to collect in the spring of every year for cohort members from first through fifth grade. NCES also intends to include at least limited collections in the fall from first and second grades to better understand academic gains and losses associated with summer breaks and gains during a single academic year and how they relate to instruction and other aspects of schooling. NCES has also for the first time taken the assessment of science down to the kindergarten level in order to capture the beginning scientific knowledge and skills of young children.

For HSLS:09, NCES moved the first year of collection to the beginning of ninth grade, when most youth start high school. The most recent previous high school longitudinal study, the Education Longitudinal Study or ELS, began at the end of tenth grade with the goal of studying the transition from high school into college or the workforce. However, by starting at the end of tenth grade as opposed to the beginning of ninth grade, the study missed data for most of the first two years of this cohort's high school experiences, years that research suggests are critical to decisions about dropping out or pursuing further schooling. Moreover, in HSLS:09 students will be followed in eleventh grade and again directly after their intended high school graduation date to determine if and where they applied to postsecondary education, financial aid offers, acceptances, intended institution of matriculation, and immediate post-high school plans. This cohort will continue to be followed through their post secondary education and entry into the workforce.

At the same time, NCES is also working to establish new links between HSLS:09 and longitudinal administrative data from state data systems. The Center has worked with 10 states to include representative samples for these states in HSLS. NCES is currently working with these states to develop Memorandums of Understanding that will provide key information about the sample of students and schools included in HSLS from the states' administrative data systems. The combination of these state data with HSLS data will provide information about student academic achievement prior to ninth grade.

Working in collaboration with education economists, the postsecondary staff at NCES has put the Beginning Postsecondary Students Longitudinal Study (BPS) on a new conceptual footing, grounding its study of student persistence and degree attainment in human capital theory, and developing innovative instrumentation to elicit key study concepts, such as wage expectations and discount rates. Postsecondary, Adult, and Career Education (PACE) is also exploring opportunities to expand their collaboration with Federal Student Aid (FSA). NCES is initiating work to match historical federal financial aid records to existing longitudinal data sets. These matching opportunities will not only expand the usefulness of older data sets for researchers, but meet the need of FSA to understand the relationship between student aid and student outcomes.

NAEP

The National Assessment of Educational Progress (NAEP) is transitioning to eventual computer administration of all assessments for upcoming NAEP cycles. The first subject to go completely online was a national sample of eighth- and twelfth-graders who responded to writing prompts administered electronically in winter 2011. Electronic administration of a writing assessment provides a number of benefits. First, it eliminates the errors inherent in scorers trying to decipher illegible handwriting. Second, it captures information not previously available in group administration, such as the number of keystrokes, time spent on task, and number of revisions. Third, it allows students to use the mode which many now prefer for writing, the use of a keyboard. Fourth, it easily permits revisions without the need for pencil erasures. Fifth, it provides an opportunity for more engaging prompts. Sixth, it allows for easily incorporating universal design principles to accommodate student needs. Finally, it opens up the possibility of computer-aided intelligent scoring, which conceivably could make scoring much less expensive.

NCES piloted an adaptive version of the NAEP mathematics assessment in 2011. As a result of this tailoring of the assessment, more precise ability estimates can be obtained. Also, the computer delivery allows analysts to identify those items where the student was not engaged, based upon time taken to respond. NAEP is currently developing a computer-delivered Technology and Engineering Literacy (TEL) assessment. Computer delivery will be leveraged to offer innovative, interactive assessment tools allowing for meaningful realistic problems to be solved and offering a rich context for demonstration of TEL skills. Realistic problems create better evidence of student ability because TEL requires them to do a meaningful piece of work online. Computer delivery allows for more direct measures of TEL skills, new types of measures, a more engaging assessment environment, and multimodal information presentation.

International Studies

In international studies, NCES is conducting or facilitating linking studies to obtain greater efficiency and enhance the usefulness of its statistical portfolio. The intent is to learn more about how the international assessments relate to assessments used more regularly in U.S. schools and to enable states to benchmark their performance internationally without the cost and burden of fielding international assessments themselves. The largest effort links the National Assessment of Educational Progress (NAEP) and Trends in International Mathematics and Science Study (TIMSS) in mathematics and science at eighth grade in 2011. To create the link, during the NAEP administration window, a subsample of students was administered special booklets that contained both NAEP and TIMSS items. Likewise, during the TIMSS administration window, a subsample of students was also given special booklets. By analyzing the relationship between student performance on NAEP and TIMSS items, a linking function will be created that will allow the projection of TIMSS-like scores for all states. In addition to the national data collection for TIMSS, eight states were invited to participate in TIMSS with independent state representative samples to provide the data needed to validate the linking function.

Through NCES, the United States is connecting internationally by participating in the new international assessment of adult literacy, called the Program for the International Assessment of Adult Competencies (PIAAC). PIAAC is being fielded in 2011, and results will be released in 2013. PIAAC builds on previous work but is much larger than prior international assessments of adults in terms of the number and variety of countries. Some 26 countries are participating, and they include nearly all the advanced economies in the Organization for Economic Cooperation and Development, which coordinates the Program for International Student Assessment (PISA). PIAAC will also include adults' reports of the skills they use on the job. This component of the study is based on similar national work done in the United Kingdom and the United States, but is new to international assessments of adults and is being conducted in collaboration with the U.S. Department of Labor, as well as representatives of labor ministries internationally. PIAAC will be administered on computers, except in cases in which respondents are not familiar enough with computers to use them, or have literacy skills so limited that a paper-and-pencil assessment makes more sense for them. In addition, PIAAC will include an assessment of problem-solving in a technology rich environment, which is also new to adult assessments. One of the benefits of administering PIAAC on computers is the opportunity to adapt assessment items to responses in real-time—that is, to make the assessment easier or harder depending on the items each respondent is able to successfully complete.

NCES is also improving the accessibility of international data, including assessment results and additional contextual information about education systems around the world. In 2010, NCES launched the International Data Explorer, an online analysis tool on the NCES website that enables users to create and download their own tables and charts with international assessment results. And for the first time, in an effort to improve school participation rates in its international assessments, NCES provided participating schools with school-level reports on their performance relative to international and U.S. averages. This is another way of connecting with practitioners and improving their access to information they can use.

Higher Education Data and P-20W Data

NCES' Integrated Postsecondary Education Data (IPEDS) program has spent much of the last few years implementing data collection requirements in the Higher Education Opportunity Act of 2008. Several new data items, such as the net price of attendance after grant and scholarship aid, were collected and have been made available in online data tools, including College Navigator, a college search site for prospective students and parents. IPEDS is currently seeking approval from the Office of Management and Budget (OMB) to make additional data changes to future IPEDS collections, including data that distinguishes online enrollments from other enrollments and programs.

IES and the Department have spent approximately $514 million and invested considerable staff resources to help states establish or improve P-20W (early childhood through workforce) longitudinal data systems. In order to improve data quality within these state data systems, NCES has initiated a comprehensive Common Education Data Standards initiative to help define data definitions and standards from early childhood to postsecondary state data systems. NCES also launched two national programs in 2011 to assist states with their data system development: the Privacy Technical Assistance Center (PTAC) and the Educational Technical Assistance Program (EDTAP). These initiatives serve as a "one-stop" resource for states and districts to learn about best practices and receive expert technical assistance on all matters of data system development.

Statewide Longitudinal Data Systems

The development of statewide longitudinal data systems (SLDS), including their extensions into postsecondary education and the workforce, has focused attention on the need for high-quality, consistent guidance on issues of data stewardship, such as privacy, confidentiality, and data security. Requirements to protect personally identifiable information (PII) are delineated in the Family Educational Rights and Privacy Act (FERPA) and related regulations, other legislation, and guidance from OMB and the National Institute of Standards and Technology. Understanding and complying with privacy regulations can be a complex task for the organizations and individuals tasked with assimilating and using student-level data, especially as they balance these regulations with the goal of using the richness of the data to improve education at local and even individual levels.

NCES—in consultation with the Department of Education's Chief Privacy Officer, the Family Policy Compliance Office (which oversees FERPA), the Office of Planning, Evaluation, and Policy Development, and the Office of the General Counsel—is providing technical assistance for states and organizations that are engaged in building and using student-level longitudinal education data systems. NCES began this process with the release of three technical briefs in the last year on such topics as definitions and concepts; data stewardship and managing PII; and statistical methods for protecting PII in aggregate reports. The newly established Privacy Technical Assistance Center (PTAC) will extend efforts in the areas of privacy, confidentiality, and security by disseminating information, answering individual questions, conducting training and, as appropriate, referring questions to experts in the Department.

Top