IES Blog

Institute of Education Sciences

CAPR: Answers to Pressing Questions in Developmental Education

Since 2014, IES has funded the Center for the Analysis of Postsecondary Readiness (CAPR) to answer questions about the rapidly evolving landscape of developmental education at community colleges and open-access four-year institutions. CAPR is providing new insights into how colleges are reforming developmental education and how their reforms are impacting student outcomes through three major studies:

  • A survey and interviews about developmental education practices and reform initiatives
  • An evaluation of the use of multiple measures for assessing college readiness
  • An evaluation of math pathways.

Preliminary results from these studies indicate that some reforms help more students finish their developmental requirements and go on to do well in college-level math and English.

National Study of Developmental Education Policies and Practices

CAPR has documented widespread reform in developmental education at two- and four-year colleges through a national survey and interviews on developmental education practices and reforms. Early results from the survey show that colleges are moving away from relying solely on standardized tests for placing students into developmental courses. Colleges are also using new approaches to delivering developmental education including shortening developmental sequences by compressing or combining courses, using technology to deliver self-paced instruction, and placing developmental students into college-level courses with extra supports, often called corequisite remediation.

Developmental Math Instructional Methods in Public Two-Year Colleges (Percentages of Colleges Implementing Specific Reform Strategies)

Notes: Percentages among two-year public colleges that reported offering developmental courses. Colleges were counted as using an instructional method if they used it in at least two course sections. Categories are not mutually exclusive.

Evaluation of Developmental Math Pathways and Student Outcomes

CAPR has teamed up with the Charles A. Dana Center at the University of Texas at Austin to evaluate the Dana Center Mathematics Pathways (DCMP) curriculum at four community colleges in Texas. The math pathways model tailors math courses to particular majors, with a statistics pathway for social science majors, a quantitative reasoning pathway for humanities majors, and an algebra-to-calculus pathway for STEM majors. DCMP originally compressed developmental math into one semester, though now the Dana Center is recommending corequisite models. Instructors seek to engage students by delving deeply into math concepts, focusing on real-world problems, and having students work together to develop solutions.

Interim results show that larger percentages of students assigned to DCMP (versus the traditional developmental sequence) enrolled in and passed developmental math. More of the DCMP students also took and passed college-level math, fulfilling an important graduation requirement. After three semesters, 25 percent of program group students passed a college-level math course, compared with 17 percent of students assigned to traditional remediation.

Evaluation of Alternative Placement Systems and Student Outcomes (aka Multiple Measures)

CAPR is also studying the impact of using a combination of measures—such as high school GPA, years out of high school, and placement test scores—to predict whether students belong in developmental or college-level courses. Early results from the multiple measures study show that, in English and to a lesser extent in math, the multiple measures algorithms placed more students into college-level courses, and more students passed those courses (compared to students placed with a single test score).

 

College-Level English Course Placement, Enrollment, and Completion in CAPR’s Multiple Measures Study (Percentages Compared Across Placement Conditions)

 

College-Level Math Course Placement and Completion in CAPR’s Multiple Measures Study

Looking Ahead to the Future of Developmental Education

These early results from CAPR’s evaluations of multiple measures and math pathways suggest that those reforms are likely to be important pieces of future developmental education systems. CAPR will release final results from its three studies in 2019 and 2020.

Guest blog by Nikki Edgecombe and Alexander Mayer

Nikki Edgecombe is the principal investigator of the Center for the Analysis of Postsecondary Readiness, an IES-funded center led by the Community College Research Center (CCRC) and MDRC, and a senior research scientist at CCRC. Alexander Mayer is the co-principal investigator of CAPR and deputy director of postsecondary education at MDRC.

National Spending for Public Schools Increases for Third Consecutive Year in School Year 2015-16

Spending on elementary and secondary education increased in school year 2015–16 (Fiscal Year 2016). At the national level, education spending has recovered from the economic downturn between 2009 and 2013. This is the third consecutive year spending increased, reversing a decline in spending for the prior four years after adjusting for inflation. These findings come from a recently released report from the National Center for Education Statistics (NCES).

The First Look report, Revenues and Expenditures for Public Elementary and Secondary Education: School Year 2015–16 (Fiscal Year 2016) is based on data from the National Public Education Finance Survey (NPEFS), a component of the Common Core of Data (CCD).  

The amount spent per student for the day-to-day operation of public elementary and secondary schools rose to $11,841 in Fiscal Year (FY) 16.[1] Current expenditures per student increased by 2.9 percent between FY 15 and 16, following on the heels of an increase of 3.2 percent from the prior year, after adjusting for inflation.[2]  Although spending per student was higher in FY 16 than in FY 07, it had decreased each year from FY 09 to FY 13.


NOTE: Spending is reported in constant FY 16 dollars, based on the Consumer Price Index (CPI).
SOURCE: U.S. Department of Education, National Center for Education Statistics, Common Core of Data (CCD), "National Public Education Financial Survey," fiscal years 2007 through 2016.


Salaries and wages make up the largest proportion of current expenditures, but have increased at a much slower rate than employee benefits or other expenditures.[3]  After peaking at $7,047 per pupil in FY 09, salaries and wages declined to a low of $6,452 per pupil in FY 2013, and increased to $6,748 per pupil in FY 16.

The proportion of salaries and wages in current expenditures per pupil decreased from 60.2 percent in FY 09 to 57.0 in FY 16. In contrast, the proportion of employee benefits in current expenditures per pupil increased from 20.4 percent in FY 09 to 22.9 percent in FY 16.


NOTE: Spending is reported in constant FY 16 dollars, based on the Consumer Price Index (CPI).
SOURCE: U.S. Department of Education, National Center for Education Statistics, Common Core of Data (CCD), "National Public Education Financial Survey," fiscal years 2007 through 2016.


At the state level, spending on current expenditures per student ranged from a low of $7,006 in Utah to a high of $22,231 in New York. Current expenditures per student were at least 40 percent higher than the national average in the following states and jurisdictions:

  • New York ($22,231)
  • District of Columbia ($21,135)
  • Connecticut ($19,615)
  • New Jersey ($19,041)
  • Vermont ($19,023)
  • Alaska ($17,510)
  • Massachusetts ($16,986)

Current expenditures per student for public elementary and secondary education, by state: Fiscal year 2016

SOURCE: U.S. Department of Education, National Center for Education Statistics, Common Core of Data (CCD), "National Public Education Financial Survey," fiscal year 2016


Between FY 14 and FY 16, current expenditures per student increased by 3 percent or more in 29 states, and by 1 to less than 3 percent in 15 states. Increases in current expenditures per student from FY 14 to FY 16 were highest in California (16.4 percent), Washington (9.9 percent), Hawaii (9.3 percent), New York (8.8 percent), and Pennsylvania (8.2 percent).  


NOTE: Spending is reported in constant FY 16 dollars, based on the Consumer Price Index (CPI).
SOURCE: U.S. Department of Education, National Center for Education Statistics, Common Core of Data (CCD), "National Public Education Financial Survey," fiscal years 2014 and 2016. 


The recently released report also presents national and state data on public school funding by source.[4] Total education funding increased by 4.0 percent (from $652.1 to $678.4 billion) from FY 15 to FY 16 following an increase of 3.3 percent from FY 14 to FY 15.  Local funding increased by 3.7 percent (from $291.1 to $303.8 billion), state funding increased by 4.9 percent (from $303.6 to $318.6 billion), and federal funding slightly increased by 1.1 percent (from $55.4 to $56.0 billion).


SOURCE: U.S. Department of Education, National Center for Education Statistics, Common Core of Data (CCD), "National Public Education Financial Survey," fiscal years 2007 through 2016.


The percentage of total funding from federal sources accounted for approximately 9 percent of total funding in both FY 07 and FY 16; however, there were notable fluctuations during this period. The federal percentage increased from 8.2 percent of funding in FY 08 to 12.5 percent of funding in FY 11. In part, this increase reflects the impact of the American Recovery and Reinvestment Act (ARRA). As the funds from the program were spent, the federal percentage decreased from 10.2 percent of total funding in FY 12 to 8.3 percent in FY 16.

Local sources accounted for 44.8 percent of total funding in FY 16, and have been relatively stable over the past 10 years. The percentage of total funding from state sources decreased from a high of 48.3 percent in FY 08 to 43.4 percent in FY 10, and has since increased to 47 percent in FY 16.

 


[1] Spending refers to current expenditures. Current expenditures are comprised of expenditures for the day-to-day operation of schools and school districts for public elementary and secondary education, including expenditures for staff salaries and benefits, supplies, and purchased services. Current expenditures include instruction, instruction-related, support services (e.g., social work, health, and psychological services), and other elementary/secondary current expenditures, but exclude expenditures on capital outlay, other programs, and interest on long-term debt. 

[2] In order to compare spending from one year to the next, expenditures are converted to constant dollars, which adjusts figures for inflation.

[3] Other expenditures include current expenditures other than salaries, wages, and employee benefits, such as purchased services, tuition, supplies, etc.

[4] Funding refers to revenues. Revenues are comprised of all funds received from external sources, net of refunds, and correcting transactions. Noncash transactions, such as receipt of services, commodities, or other receipts in kind are excluded, as are funds received from the issuance of debt, liquidation of investments, and nonroutine sale of property.

 

Companion Guidelines on Replication and Reproducibility in Education Research

Just over five years ago the Institute of Education Sciences (IES) and the National Science Foundation (NSF) released the Common Guidelines for Education Research and Development. The Guidelines provided the expected purposes, justifications, and contributions of various types of research aimed at improving our knowledge of interventions and strategies for improving teaching and learning.  Since 2013, there has been increased attention to replication and reproducibility studies and their role in building the evidence base. In response to this interest and the importance of this work, the two organizations jointly issued the new Companion Guidelines on Replication and Reproducibility in Education Research to supplement the Common Guidelines for Education Research and Development. The companion document provides guidance on the steps researchers can take to promote corroboration, ensure the integrity of research, and extend the evidence base.

The Companion Guidelines identify principles to help education stakeholders design and report reproducibility and replication studies. These principles are consistent with and draw from guidelines provided by scientific and professional organizations, advisory committees, and have emerged in consultation with the field (e.g., Dettmer, Taylor, and Chhin, 2017; Subcommittee on Replicability and Science, 2015). The principles address three main areas – (1) replication and reproducibility at the proposal stage, (2) promoting transparency and openness in designing studies, and (3) considerations in the reporting of results. 

Although the importance of reproducibility and replication studies for advancing scientific knowledge has been widely acknowledged, there are several challenges for researchers in our field, including actual or perceived disincentives (e.g., publication bias; reputation and career advancement norms; emphases on novel, potentially transformative lines of inquiry), implementation difficulties (especially for direct replications), and complexities of interpreting results (e.g., lack of consensus on what it means to “replicate” findings, low statistical power for replications). Grant funding agencies such as IES and NSF as well as education researchers have a role to play in addressing these challenges, promoting reproducibility and replication studies, and ultimately moving the field forward.

Why focus on replication and reproducibility?

The original Common Guidelines document did not substantively address issues pertaining to replication and reproducibility of research.  Given the interest in and importance of this work, IES and NSF are providing additional clarity to the field in terms of common definitions and principles around replication and reproducibility.

Who is the audience for the Companion Guidelines on Replication and Reproducibility? 

The primary audience for this document is education researchers; however, education research funding agencies and reviewers of grant applications are additional audiences for this document.

How should this document be used by researchers intending to apply for grants to conduct a reproducibility or replication study?

This document is meant to highlight the importance of replication and reproducibility studies and to offer guidelines to education stakeholders for thinking about and promoting reproducibility and replication in education research. It does not supersede the guidance provided in the requests for applications provided by IES and NSF. 

What are the guiding principles for proposing replication and reproducibility studies?

The overarching principles at the grant proposal stage are as follows:

  1. Clarify how reproducibility or replication studies would build on prior studies and contribute to the knowledge base.
  2. Clearly specify any variations from prior studies and the rationale for such variations.
  3. Ensure objectivity (e.g., by conducting an independent investigation, or by putting safeguards in place if the original investigator(s) is involved).

In addition to these principles, the document also lays out principles for promoting transparency, open science, and reporting results.

Read the full Companion Guidelines here.

 

Basic Science of Learning and Development Within Education: The IES Investment

I came to the Institute of Education Sciences (IES) in 2002 to build connections between education and the basic science of learning and development. The weak links between these two fields were surprising to me, given how foundational such science is to the very purpose of education.

IES had just launched the Cognition and Student Learning program[1], and researchers were invited to submit applications to examine whether principles of learning established in basic science were robust when examined in education settings.  Six years later, we launched the Social and Behavioral Context to Support Academic Learning to understand the ways in which the social environment of classrooms and school affected learning. Together, IES has invested over $445M, an investment that has contributed substantially to our foundational knowledge of teaching and learning.

I was surprised by this recent blog by Bob Pianta and Tara Hofkens. While they acknowledge the research that IES has supported to transform education practice, they did not seem to realize our substantial, ongoing investments in the basic science of teaching and learning—both in and out of classrooms.

In part, this may reflect their perception of what types of work we support under our Exploration goal – which is not limited to “scouring databases” but instead involves all types of research, including small-scale experiments and longitudinal studies. These projects generate foundational knowledge about what factors are associated with learning outcomes and can potentially be changed through education. In fact, the questions that Pianta and Hofkens want answered by the basic science of education are the same questions that some IES grantees have been examining over the course of the last 15 years. 

Here are just a few examples.

  1. What factors regulate children's attention in a classroom setting? Anna Fisher and her team found that cluttered classroom walls in kindergarten led to greater distraction and less learning – a finding that captured the imagination of the nation and the nation’s educators.
  2. What roles do the capabilities of peers play in advancing children's cognitive capabilities? A new study led by Adrienne Nishina is examining how student’s ability to think about situations from different perspectives is related to their day-to-day interactions with peers from diverse backgrounds.
  3. What factors promote or inhibit teachers' responses to children's perceived misbehavior? Teachers’ expertise and teachers’ emotional competencies are two factors that IES-funded researchers have found to relate to their responses to children’s behavior.
  4. What role do social and emotional experiences and affective processes play in fostering learning? Shannon Suldo and her team find that the coping strategies that high school students choose to manage their responses to stressors are linked to learning outcomes.
  5. What are the components of school climate that matter the most for different forms of student success? Two recent projects, one in Cleveland, and one in Virginia, are using survey data to explore the relationship between school climate, social behavioral competencies and academic outcomes. The teams are also exploring how those relationships vary within student subgroups.

Funding the basic science of teaching and learning—in and out of classrooms—has been and will continue to be a cornerstone of the work that IES funds. The IES investment in this area is broad, and is shared in books such as Make It Stick: The Science of Successful Learning, Becoming Brilliant: What Science Tells us About Raising Successful Children, and Educator Stress: An Occupational Health Perspective

Importantly, IES is not the only funder in this area. The National Science Foundation invests substantially in their Science of Learning portfolio, the McDonnell Foundation’s Understanding Human Cognition portfolio includes an explicit request for projects at the intersection of cognition and education, and the Child Development and Behavior Branch of the Eunice Kennedy Shriver National Institute of Child Health and Human Development (NICHD) supports a variety of relevant research programs.  I agree that we need systematic investment in the basic science of teaching and learning. But we must build on what we have already learned.  

We are grateful that Pianta and Hofkens recognize the importance of investing in this area. Perhaps the fact that they did not acknowledge the substantial investments and contributions IES has made in exploring the important questions they pose is an IES problem. While we have invested heavily in the science of learning, we have skimped on brand development and self-promoting. If someone as central to the field such as Pianta, who has received several IES grants, including research training grants, doesn’t know what IES has done, that is a red flag that we will need to attend to.

In the meantime, we hope that this brief glimpse into our investment to date has illustrated some of the questions that the basic science of teaching and learning within education can answer. More importantly here’s where you can seek funding for this type of work.

Elizabeth Albro

Commissioner, National Center for Education Research

 

[1] IES was authorized in November 2002. The Cognition and Student Learning research program was launched by the Office of Educational Research and Improvement, the office from which IES was created.

Sharing strategies to increase research-based educational practices

By Cora Goldston, REL Midwest

Highlighted Resources

How can states, districts, and schools identify effective practices to address challenges and achieve their goals? Education research can point the way, but sometimes finding and accessing relevant research can be a frustrating and time-consuming process. And even when practitioners can find research, it can be difficult to determine a study’s rigor and the strength of research evidence supporting interventions.

Equipping practitioners to use research evidence

Through the Midwest Alliance to Improve Knowledge Utilization (MAIKU), the Regional Educational Laboratory (REL) Midwest is partnering with practitioners to help states, districts, and schools use research to inform practice. The goal is to make it easier for educators to find research relevant to their priorities, assess the level of evidence that supports potential practices, and implement those practices that are based on strong evidence.

REL Midwest and MAIKU are supporting the use of research in education practice in several ways. For example, REL Midwest provided coaching sessions for the Ohio Department of Education (ODE) on understanding the Every Student Succeeds Act (ESSA) tiers of evidence. In addition, REL Midwest created a crosswalk that shows how the ESSA evidence tiers align with ratings from research clearinghouses, such as the What Works Clearinghouse. In turn, ODE is using this information to help Ohio districts that are applying for Striving Readers grants. To receive the grants, districts must demonstrate that they plan to use research-based practices to improve student literacy. As a result of REL Midwest’s support, ODE has strengthened its capacity to help districts determine the level of evidence supporting certain practices and, thus, to submit stronger grant applications.

REL Midwest is providing similar support across the region. In Michigan, we are conducting coaching sessions for the state Department of Education to help agency leadership choose priorities from the state’s Top 10 in 10 plan, identify research-based practices that support those priorities, and collaborate to implement new state-level practices. In Wisconsin, REL Midwest hosted a training series for the Department of Public Instruction to increase the agency’s capacity to collect, analyze, and use data to adjust state-level policies and practices. And in Illinois, REL Midwest is holding a training series for the State Board of Education on research methods, data collection, and data analysis and how to use the findings to inform agency practices.

June webinar on increasing evidence use

MAIKU is also working with researchers to support evidence use in education practice. On June 19, 2018, REL Midwest and MAIKU hosted a webinar to discuss how researchers can share evidence with practitioners in useful and accessible ways.

The webinar featured a presentation by Alan J. Daly, Ph.D., of the University of California at San Diego, and Kara Finnigan, Ph.D., of the University of Rochester. Dr. Daly and Dr. Finnigan discussed how information-sharing networks are structured among school and district staff and the challenges for practitioners in accessing and using research-based practices.   

Building on this context, Dr. Daly and Dr. Finnigan shared insights about the most effective ways to maximize the reach of research. One of their key findings is that the pattern of people’s social ties makes a difference for sharing and using research-based practices. Finnigan and Daly noted that the set of relationships we have can increase access to research evidence if the right ties are present but can constrain access to resources when those ties are not present. The quality of relationships also matters; high levels of trust are essential for more in-depth exchanges of information. The takeaway: fostering both the quantity and quality of social relationships is important for sharing research evidence.  

During the webinar, Jaime Singer, senior technical assistance consultant at REL Midwest, also shared actionable strategies that researchers can use to support evidence use in practice, including training and coaching sessions, checklists, blog posts, and clearinghouses of effective practices.

The webinar included a panel discussion about REL Midwest’s ESSA evidence tiers coaching sessions and crosswalk for ODE. REL Midwest researcher Lyzz Davis, Ph.D., provided a researcher perspective on developing resources to meet ODE’s needs. Heather Boughton, Ph.D., and Melissa Weber-Mayrer, Ph.D., at ODE provided practitioner perspectives on how REL Midwest’s work has strengthened the agency’s capacity to help districts find and use evidence-based interventions.