IES Blog

by Erin Pollard, ERIC Project Officer

IES funds approximately $237 million of research a year in order to provide practitioners and policymakers with the information they need to improve education. But what good is that research if it sits behind a paywall and the people who need it the most can’t access it? That is the thought behind IES’s Public Access Policy. It requires all of our grantees and contractors to make the full text of any peer reviewed work that we funded freely available through ERIC within a year of publication.

Before the policy was adopted in 2011, the majority of the work funded by the National Center for Education Research and National Center for Special Education Research appeared in peer reviewed journals. These journals are largely subscription based; meaning only those who had access to a library with a subscription could access the articles. Given that journal subscriptions are frequently over $1,000 per journal per year and annual subscriptions for academic databases start at close to $10,000 per year, many smaller schools and districts simply cannot afford to purchase access to high quality research. Their teachers and administrators must rely on freely available resources.

IES wanted to change the model. We believe that because IES-supported research is publicly funded, the results should be publicly available. We have worked with publishers, editors, and grantees to find a way to make our policy mutually beneficial.

We are not alone in this effort. IES is part of a larger federal initiative to make most research findings publicly available. Federal agencies are coordinating to adopt similar policies all across the government to increase the availability of good science and to improve evidence-based decision making.

Since we adopted the policy 5 years ago we have been able to make more than 600 publications freely available in ERIC and will be able to release 250 more in the next year. This is just the beginning. We expect more and more work to become available each year. Because ERIC powers other search engines and academic databases with its metadata, we are disseminating these full text articles widely, wherever users are looking for their research. By giving teachers, administrators and policymakers access to high quality research, we are able to get our work into the hands of the people who can use it to build a brighter future for our Nation’s students.

Since 2014, IES has funded the Center for the Analysis of Postsecondary Readiness (CAPR) to answer questions about the rapidly evolving landscape of developmental education at community colleges and open-access four-year institutions. CAPR is providing new insights into how colleges are reforming developmental education and how their reforms are impacting student outcomes through three major studies:

• A survey and interviews about developmental education practices and reform initiatives
• An evaluation of the use of multiple measures for assessing college readiness
• An evaluation of math pathways.

Preliminary results from these studies indicate that some reforms help more students finish their developmental requirements and go on to do well in college-level math and English.

National Study of Developmental Education Policies and Practices

CAPR has documented widespread reform in developmental education at two- and four-year colleges through a national survey and interviews on developmental education practices and reforms. Early results from the survey show that colleges are moving away from relying solely on standardized tests for placing students into developmental courses. Colleges are also using new approaches to delivering developmental education including shortening developmental sequences by compressing or combining courses, using technology to deliver self-paced instruction, and placing developmental students into college-level courses with extra supports, often called corequisite remediation.

Developmental Math Instructional Methods in Public Two-Year Colleges (Percentages of Colleges Implementing Specific Reform Strategies)

Notes: Percentages among two-year public colleges that reported offering developmental courses. Colleges were counted as using an instructional method if they used it in at least two course sections. Categories are not mutually exclusive.

Evaluation of Developmental Math Pathways and Student Outcomes

CAPR has teamed up with the Charles A. Dana Center at the University of Texas at Austin to evaluate the Dana Center Mathematics Pathways (DCMP) curriculum at four community colleges in Texas. The math pathways model tailors math courses to particular majors, with a statistics pathway for social science majors, a quantitative reasoning pathway for humanities majors, and an algebra-to-calculus pathway for STEM majors. DCMP originally compressed developmental math into one semester, though now the Dana Center is recommending corequisite models. Instructors seek to engage students by delving deeply into math concepts, focusing on real-world problems, and having students work together to develop solutions.

Interim results show that larger percentages of students assigned to DCMP (versus the traditional developmental sequence) enrolled in and passed developmental math. More of the DCMP students also took and passed college-level math, fulfilling an important graduation requirement. After three semesters, 25 percent of program group students passed a college-level math course, compared with 17 percent of students assigned to traditional remediation.

Evaluation of Alternative Placement Systems and Student Outcomes (aka Multiple Measures)

CAPR is also studying the impact of using a combination of measures—such as high school GPA, years out of high school, and placement test scores—to predict whether students belong in developmental or college-level courses. Early results from the multiple measures study show that, in English and to a lesser extent in math, the multiple measures algorithms placed more students into college-level courses, and more students passed those courses (compared to students placed with a single test score).

College-Level English Course Placement, Enrollment, and Completion in CAPR’s Multiple Measures Study (Percentages Compared Across Placement Conditions)

College-Level Math Course Placement and Completion in CAPR’s Multiple Measures Study

Looking Ahead to the Future of Developmental Education

These early results from CAPR’s evaluations of multiple measures and math pathways suggest that those reforms are likely to be important pieces of future developmental education systems. CAPR will release final results from its three studies in 2019 and 2020.

Guest blog by Nikki Edgecombe and Alexander Mayer

Nikki Edgecombe is the principal investigator of the Center for the Analysis of Postsecondary Readiness, an IES-funded center led by the Community College Research Center (CCRC) and MDRC, and a senior research scientist at CCRC. Alexander Mayer is the co-principal investigator of CAPR and deputy director of postsecondary education at MDRC.

Spending on elementary and secondary education increased in school year 2015–16 (Fiscal Year 2016). At the national level, education spending has recovered from the economic downturn between 2009 and 2013. This is the third consecutive year spending increased, reversing a decline in spending for the prior four years after adjusting for inflation. These findings come from a recently released report from the National Center for Education Statistics (NCES).

The First Look report, Revenues and Expenditures for Public Elementary and Secondary Education: School Year 2015–16 (Fiscal Year 2016) is based on data from the National Public Education Finance Survey (NPEFS), a component of the Common Core of Data (CCD).  

The amount spent per student for the day-to-day operation of public elementary and secondary schools rose to $11,841 in Fiscal Year (FY) 16.^[1] Current expenditures per student increased by 2.9 percent between FY 15 and 16, following on the heels of an increase of 3.2 percent from the prior year, after adjusting for inflation.^[2]  Although spending per student was higher in FY 16 than in FY 07, it had decreased each year from FY 09 to FY 13.

NOTE: Spending is reported in constant FY 16 dollars, based on the Consumer Price Index (CPI).
SOURCE: U.S. Department of Education, National Center for Education Statistics, Common Core of Data (CCD), "National Public Education Financial Survey," fiscal years 2007 through 2016.

Salaries and wages make up the largest proportion of current expenditures, but have increased at a much slower rate than employee benefits or other expenditures.^[3]  After peaking at $7,047 per pupil in FY 09, salaries and wages declined to a low of $6,452 per pupil in FY 2013, and increased to $6,748 per pupil in FY 16.

The proportion of salaries and wages in current expenditures per pupil decreased from 60.2 percent in FY 09 to 57.0 in FY 16. In contrast, the proportion of employee benefits in current expenditures per pupil increased from 20.4 percent in FY 09 to 22.9 percent in FY 16.

NOTE: Spending is reported in constant FY 16 dollars, based on the Consumer Price Index (CPI).
SOURCE: U.S. Department of Education, National Center for Education Statistics, Common Core of Data (CCD), "National Public Education Financial Survey," fiscal years 2007 through 2016.

At the state level, spending on current expenditures per student ranged from a low of $7,006 in Utah to a high of $22,231 in New York. Current expenditures per student were at least 40 percent higher than the national average in the following states and jurisdictions:

• New York ($22,231)
• District of Columbia ($21,135)
• Connecticut ($19,615)
• New Jersey ($19,041)
• Vermont ($19,023)
• Alaska ($17,510)
• Massachusetts ($16,986)

Current expenditures per student for public elementary and secondary education, by state: Fiscal year 2016

SOURCE: U.S. Department of Education, National Center for Education Statistics, Common Core of Data (CCD), "National Public Education Financial Survey," fiscal year 2016

Between FY 14 and FY 16, current expenditures per student increased by 3 percent or more in 29 states, and by 1 to less than 3 percent in 15 states. Increases in current expenditures per student from FY 14 to FY 16 were highest in California (16.4 percent), Washington (9.9 percent), Hawaii (9.3 percent), New York (8.8 percent), and Pennsylvania (8.2 percent).  

NOTE: Spending is reported in constant FY 16 dollars, based on the Consumer Price Index (CPI).
SOURCE: U.S. Department of Education, National Center for Education Statistics, Common Core of Data (CCD), "National Public Education Financial Survey," fiscal years 2014 and 2016. 

The recently released report also presents national and state data on public school funding by source.^[4] Total education funding increased by 4.0 percent (from $652.1 to $678.4 billion) from FY 15 to FY 16 following an increase of 3.3 percent from FY 14 to FY 15.  Local funding increased by 3.7 percent (from $291.1 to $303.8 billion), state funding increased by 4.9 percent (from $303.6 to $318.6 billion), and federal funding slightly increased by 1.1 percent (from $55.4 to $56.0 billion).

SOURCE: U.S. Department of Education, National Center for Education Statistics, Common Core of Data (CCD), "National Public Education Financial Survey," fiscal years 2007 through 2016.

The percentage of total funding from federal sources accounted for approximately 9 percent of total funding in both FY 07 and FY 16; however, there were notable fluctuations during this period. The federal percentage increased from 8.2 percent of funding in FY 08 to 12.5 percent of funding in FY 11. In part, this increase reflects the impact of the American Recovery and Reinvestment Act (ARRA). As the funds from the program were spent, the federal percentage decreased from 10.2 percent of total funding in FY 12 to 8.3 percent in FY 16.

Local sources accounted for 44.8 percent of total funding in FY 16, and have been relatively stable over the past 10 years. The percentage of total funding from state sources decreased from a high of 48.3 percent in FY 08 to 43.4 percent in FY 10, and has since increased to 47 percent in FY 16.

Just over five years ago the Institute of Education Sciences (IES) and the National Science Foundation (NSF) released the Common Guidelines for Education Research and Development. The Guidelines provided the expected purposes, justifications, and contributions of various types of research aimed at improving our knowledge of interventions and strategies for improving teaching and learning.  Since 2013, there has been increased attention to replication and reproducibility studies and their role in building the evidence base. In response to this interest and the importance of this work, the two organizations jointly issued the new Companion Guidelines on Replication and Reproducibility in Education Research to supplement the Common Guidelines for Education Research and Development. The companion document provides guidance on the steps researchers can take to promote corroboration, ensure the integrity of research, and extend the evidence base.

The Companion Guidelines identify principles to help education stakeholders design and report reproducibility and replication studies. These principles are consistent with and draw from guidelines provided by scientific and professional organizations, advisory committees, and have emerged in consultation with the field (e.g., Dettmer, Taylor, and Chhin, 2017; Subcommittee on Replicability and Science, 2015). The principles address three main areas – (1) replication and reproducibility at the proposal stage, (2) promoting transparency and openness in designing studies, and (3) considerations in the reporting of results. 

Although the importance of reproducibility and replication studies for advancing scientific knowledge has been widely acknowledged, there are several challenges for researchers in our field, including actual or perceived disincentives (e.g., publication bias; reputation and career advancement norms; emphases on novel, potentially transformative lines of inquiry), implementation difficulties (especially for direct replications), and complexities of interpreting results (e.g., lack of consensus on what it means to “replicate” findings, low statistical power for replications). Grant funding agencies such as IES and NSF as well as education researchers have a role to play in addressing these challenges, promoting reproducibility and replication studies, and ultimately moving the field forward.

Why focus on replication and reproducibility?

The original Common Guidelines document did not substantively address issues pertaining to replication and reproducibility of research.  Given the interest in and importance of this work, IES and NSF are providing additional clarity to the field in terms of common definitions and principles around replication and reproducibility.

Who is the audience for the Companion Guidelines on Replication and Reproducibility? 

The primary audience for this document is education researchers; however, education research funding agencies and reviewers of grant applications are additional audiences for this document.

How should this document be used by researchers intending to apply for grants to conduct a reproducibility or replication study?

This document is meant to highlight the importance of replication and reproducibility studies and to offer guidelines to education stakeholders for thinking about and promoting reproducibility and replication in education research. It does not supersede the guidance provided in the requests for applications provided by IES and NSF. 

What are the guiding principles for proposing replication and reproducibility studies?

The overarching principles at the grant proposal stage are as follows:

1. Clarify how reproducibility or replication studies would build on prior studies and contribute to the knowledge base.
2. Clearly specify any variations from prior studies and the rationale for such variations.
3. Ensure objectivity (e.g., by conducting an independent investigation, or by putting safeguards in place if the original investigator(s) is involved).

In addition to these principles, the document also lays out principles for promoting transparency, open science, and reporting results.

Read the full Companion Guidelines here.

I came to the Institute of Education Sciences (IES) in 2002 to build connections between education and the basic science of learning and development. The weak links between these two fields were surprising to me, given how foundational such science is to the very purpose of education.

IES had just launched the Cognition and Student Learning program[1], and researchers were invited to submit applications to examine whether principles of learning established in basic science were robust when examined in education settings.  Six years later, we launched the Social and Behavioral Context to Support Academic Learning to understand the ways in which the social environment of classrooms and school affected learning. Together, IES has invested over $445M, an investment that has contributed substantially to our foundational knowledge of teaching and learning.

I was surprised by this recent blog by Bob Pianta and Tara Hofkens. While they acknowledge the research that IES has supported to transform education practice, they did not seem to realize our substantial, ongoing investments in the basic science of teaching and learning—both in and out of classrooms.

In part, this may reflect their perception of what types of work we support under our Exploration goal – which is not limited to “scouring databases” but instead involves all types of research, including small-scale experiments and longitudinal studies. These projects generate foundational knowledge about what factors are associated with learning outcomes and can potentially be changed through education. In fact, the questions that Pianta and Hofkens want answered by the basic science of education are the same questions that some IES grantees have been examining over the course of the last 15 years. 

Here are just a few examples.

1. What factors regulate children's attention in a classroom setting? Anna Fisher and her team found that cluttered classroom walls in kindergarten led to greater distraction and less learning – a finding that captured the imagination of the nation and the nation’s educators.
2. What roles do the capabilities of peers play in advancing children's cognitive capabilities? A new study led by Adrienne Nishina is examining how student’s ability to think about situations from different perspectives is related to their day-to-day interactions with peers from diverse backgrounds.
3. What factors promote or inhibit teachers' responses to children's perceived misbehavior? Teachers’ expertise and teachers’ emotional competencies are two factors that IES-funded researchers have found to relate to their responses to children’s behavior.
4. What role do social and emotional experiences and affective processes play in fostering learning? Shannon Suldo and her team find that the coping strategies that high school students choose to manage their responses to stressors are linked to learning outcomes.
5. What are the components of school climate that matter the most for different forms of student success? Two recent projects, one in Cleveland, and one in Virginia, are using survey data to explore the relationship between school climate, social behavioral competencies and academic outcomes. The teams are also exploring how those relationships vary within student subgroups.

Funding the basic science of teaching and learning—in and out of classrooms—has been and will continue to be a cornerstone of the work that IES funds. The IES investment in this area is broad, and is shared in books such as Make It Stick: The Science of Successful Learning, Becoming Brilliant: What Science Tells us About Raising Successful Children, and Educator Stress: An Occupational Health Perspective. 

Importantly, IES is not the only funder in this area. The National Science Foundation invests substantially in their Science of Learning portfolio, the McDonnell Foundation’s Understanding Human Cognition portfolio includes an explicit request for projects at the intersection of cognition and education, and the Child Development and Behavior Branch of the Eunice Kennedy Shriver National Institute of Child Health and Human Development (NICHD) supports a variety of relevant research programs.  I agree that we need systematic investment in the basic science of teaching and learning. But we must build on what we have already learned.  

We are grateful that Pianta and Hofkens recognize the importance of investing in this area. Perhaps the fact that they did not acknowledge the substantial investments and contributions IES has made in exploring the important questions they pose is an IES problem. While we have invested heavily in the science of learning, we have skimped on brand development and self-promoting. If someone as central to the field such as Pianta, who has received several IES grants, including research training grants, doesn’t know what IES has done, that is a red flag that we will need to attend to.

In the meantime, we hope that this brief glimpse into our investment to date has illustrated some of the questions that the basic science of teaching and learning within education can answer. More importantly here’s where you can seek funding for this type of work.

Elizabeth Albro

Commissioner, National Center for Education Research