IES Blog

Just over five years ago the Institute of Education Sciences (IES) and the National Science Foundation (NSF) released the Common Guidelines for Education Research and Development. The Guidelines provided the expected purposes, justifications, and contributions of various types of research aimed at improving our knowledge of interventions and strategies for improving teaching and learning.  Since 2013, there has been increased attention to replication and reproducibility studies and their role in building the evidence base. In response to this interest and the importance of this work, the two organizations jointly issued the new Companion Guidelines on Replication and Reproducibility in Education Research to supplement the Common Guidelines for Education Research and Development. The companion document provides guidance on the steps researchers can take to promote corroboration, ensure the integrity of research, and extend the evidence base.

The Companion Guidelines identify principles to help education stakeholders design and report reproducibility and replication studies. These principles are consistent with and draw from guidelines provided by scientific and professional organizations, advisory committees, and have emerged in consultation with the field (e.g., Dettmer, Taylor, and Chhin, 2017; Subcommittee on Replicability and Science, 2015). The principles address three main areas – (1) replication and reproducibility at the proposal stage, (2) promoting transparency and openness in designing studies, and (3) considerations in the reporting of results. 

Although the importance of reproducibility and replication studies for advancing scientific knowledge has been widely acknowledged, there are several challenges for researchers in our field, including actual or perceived disincentives (e.g., publication bias; reputation and career advancement norms; emphases on novel, potentially transformative lines of inquiry), implementation difficulties (especially for direct replications), and complexities of interpreting results (e.g., lack of consensus on what it means to “replicate” findings, low statistical power for replications). Grant funding agencies such as IES and NSF as well as education researchers have a role to play in addressing these challenges, promoting reproducibility and replication studies, and ultimately moving the field forward.

Why focus on replication and reproducibility?

The original Common Guidelines document did not substantively address issues pertaining to replication and reproducibility of research.  Given the interest in and importance of this work, IES and NSF are providing additional clarity to the field in terms of common definitions and principles around replication and reproducibility.

Who is the audience for the Companion Guidelines on Replication and Reproducibility? 

The primary audience for this document is education researchers; however, education research funding agencies and reviewers of grant applications are additional audiences for this document.

How should this document be used by researchers intending to apply for grants to conduct a reproducibility or replication study?

This document is meant to highlight the importance of replication and reproducibility studies and to offer guidelines to education stakeholders for thinking about and promoting reproducibility and replication in education research. It does not supersede the guidance provided in the requests for applications provided by IES and NSF. 

What are the guiding principles for proposing replication and reproducibility studies?

The overarching principles at the grant proposal stage are as follows:

1. Clarify how reproducibility or replication studies would build on prior studies and contribute to the knowledge base.
2. Clearly specify any variations from prior studies and the rationale for such variations.
3. Ensure objectivity (e.g., by conducting an independent investigation, or by putting safeguards in place if the original investigator(s) is involved).

In addition to these principles, the document also lays out principles for promoting transparency, open science, and reporting results.

Read the full Companion Guidelines here.

The 21st century American classroom continues to evolve, particularly through the incorporation of technology into K-12 learning. In response to these changes, the National Center for Education Statistics (NCES) consistently works to ensure our data collections include information on how these changes affect U.S. education.

Technology is changing how teachers teach, as well as what, how, and where students learn. As a tool, technology has the potential to improve our education system by creating more equitable circumstances for all. However, while technology has assisted in improving educational experiences and outcomes for some, inequities persist. That’s why the NCES Ed Tech Equity Initiative was created—to better inform the condition of American education by giving greater focus to the relationship technology has with K-12 students’ education.

Within the framework, we define technology as digital resources (e.g., internet, phones, laptops, tablets, and software). Ed Tech Equity, or education technology and equity, refers to fairness regarding the relationship of technology and students’ educational experiences and outcomes.

THE FRAMEWORK

The Ed Tech Equity Framework serves as the conceptual anchor for the Initiative—it captures the most critical factors that influence ed tech equity as it relates to K-12 education. The framework was created following extensive research and feedback. NCES reviewed existing NCES data collections and reports, as well as relevant research external to NCES. Additionally, we consulted NCES staff and stakeholders, including teachers, principals, and researchers. Stay tuned for a more in-depth look at the framework in our next blog post.

EXISTING TECHNOLOGY-RELATED EFFORTS

Another critical step in advancing this work included completing a comprehensive internal review of NCES’ current tech-related efforts to understand what tech-related items are already collected, reported, and disseminated. Through this review, we found that a number of NCES surveys collect and report tech-related information. However, there is room for NCES to improve upon these existing efforts. As one of the first steps in this direction, NCES convened a panel of experts to share their insights and recommendations for ed tech equity data collection, reporting, and dissemination.

OUR VISION

It is important that NCES remains agile in its pursuit of comprehensive and timely data on condition of education across the country. Through this Initiative, we intend to provide researchers, policymakers, educators, parents, and students with user-friendly data that informs the relationship between technology and K-12 education.

While we’ve accomplished a great deal thus far, we’re excited to continue to advance this Initiative and to share our results!

By Halima Adenegan, NCES and Emily Martin, Hager Sharp

The expanding use of technology affects the lives of students both inside and outside the classroom. While exposure to learning technology inside schools and classrooms is important, access can also differ once those students are in their homes. It’s important for educators to be aware of the potential barriers to technology and internet access that students may face. A recent report from NCES, Student Access to Digital Learning Resources Outside the Classroom, highlighted some differences in home internet access for students.

The percentage of 5- to 17-year-old students with either no internet access or only dial-up access differed by students’ race/ethnicity.

Access also differed geographically. Remote rural locales had the highest percentage of students with either no internet access or only dial up access at home. Within these remote rural areas, the percentage of students lacking access differed by students’ race/ethnicity. Forty-one percent of Black students and 26 percent of Hispanic students living in remote rural areas had either no internet access or only dial up access at home. This was higher than the percentage of White students (13 percent) and Asian students (11 percent) living in remote rural areas who had either no internet access or only dial up access at home.   

The percentage of students who had no access to the Internet or only dial-up access was higher for students living below the poverty threshold (26 percent) than for students living between 100 and 185 percent of the poverty threshold (15 percent) and at greater than 185 percent of the poverty threshold (4 percent).

In 2015, the two most common main reasons for children ages 3 to 18 to not have home internet access were that it was too expensive or that the family did not believe they needed it/ were not interested in having it (38 percent each). Other main reasons for not having home internet access included that the home lacked a computer or a computer adequate for internet use (8 percent), internet service was not available in the area (5 percent), the Internet could be used somewhere else (3 percent), and privacy and security concerns (i.e., online privacy and cybersecurity and personal safety concerns) (2 percent). 

Browse the full report for more data on additional topics relating to differences in access to technology and the internet.

By Lauren Musu

In the 2015-16 school year, there were approximately 90,400 principals and 3,827,100 teachers in public elementary and secondary schools in the United States. Knowledge about the characteristics and experiences of these key school staff can help inform decisions about education.  The National Teacher and Principal Survey (NTPS) supports these decisions by providing data on a variety of topics from the perspective of teachers, principals and other school staff. Results from these questionnaires provide information such as:

• Principals’ education. Among public schools, a majority of principals held a master’s degree (61 percent) as their highest degree, compared to an education specialist/professional diploma at least one year beyond the master’s level (27 percent), a doctorate/first professional degree (10 percent), or a bachelor’s degree or less (2 percent).
• Hours worked by teachers. On average, regular full-time teachers in public schools spent 53 hours per week on all school-related activities. That includes 27 hours that they were paid to deliver instruction to students during a typical full week. Public school teachers were required to work an average of 38 hours per week to receive their base pay.
• Online courses. Nationwide, about 21 percent of public schools offered at least one course entirely online. This was more common among public charter schools (29 percent) than it was among traditional public schools (20 percent). A greater percentage of high (58 percent) and combined (64 percent) schools offered one or more courses entirely online than all public schools. It was also more common for schools with fewer than 100 students (45 percent) and schools with 1,000 or more students (44 percent). Among schools offering online courses, relatively more public charter schools offered all of their classes online (14 percent) than traditional public schools (5 percent).

More examples of the type of information collected in the 2015-16 NTPS can be seen in the video below:

More information is available in the NTPS online table library. In addition, analysts can access the data using DataLab or obtain a restricted-use license to conduct their own analyses of NTPS restricted-use data files.

By Maura Spiegelman

Charter school enrollment has grown significantly over time. Between fall 2000 and fall 2015, overall public charter school enrollment increased from 0.4 million to 2.8 million students, and the percentage of public school students who attended charter schools increased from 1 to 6 percent. The number of charter schools also increased during this period, from 1,990 to 6,860.

The characteristics of charter schools and traditional public schools differ in some ways. A higher percentage of charter schools are located in cities and a lower percentage are located in rural areas as compared to traditional public schools.

There are also some differences in the characteristics of students who attend charter schools and traditional public schools. A higher percentage of charter schools had higher percentages of minority students enrolled. In school year 2015–16, more than half of the students were White in 58 percent of traditional public schools. In comparison, 34 percent of charter schools had more than 50 percent White enrollment. In 9 percent of traditional public schools more than half of students were Black compared to 23 percent for charter schools. In 16 percent of traditional public schools, more than half of students were Hispanic compared to 25 percent for charter schools.

High-poverty schools are those in which more than 75 percent of students qualify for free or reduced-price lunch (FRPL) under the National School Lunch Program. In the 2015–16 school year, 24 percent of traditional public schools were high-poverty compared with 35 percent of public charter schools. In contrast, low-poverty schools–in which less than 25 percent of students qualify for FRPL–accounted for 16 percent of traditional public schools and 21 percent of public charter schools.

Access indicators from the Condition of Education on the characteristics of traditional public schools and public charter schools as well as charter school enrollment for more data!

By Lauren Musu