Inside IES Research

The college pipeline from start to finish is an extraordinarily complex process with numerous decision points, options, and obstacles. Students from advantaged social backgrounds are more likely than those from disadvantaged backgrounds to attend schools and colleges staffed with advisers and support staff that have time and resources to assist them. They may also draw on relationships with family, adults in their communities, or knowledgeable peers for assistance in navigation decisions. For students without such supports, the sequence of choices may become so overwhelming that they respond by delaying decisions or making poor choices that lead to sizable delays in their degree progression. With these challenges in mind, IES has funded three information and advising initiatives that draw on insights from researchers, practitioners, and the research literature.

Technical Working Group Meeting

In July 2019, NCER convened a technical working group of 14 researchers and practitioners for a set of conversations structured around three intervention strategies that have garnered substantial attention over the last 5 years: nudges and other light-touch informational campaigns; intensive, proactive coaching and advising; and comprehensive approaches that comprise advising and other supports such as technology and financial incentives. Researchers and practitioners shared perceptions about the effectiveness of each strategy, its relevance to targeted student populations, and conditions for implementation. At the end of the day, working group members provided recommendations (see the Technical Working Group Meeting Summary for a full list), including the following:  

• Institutions should help determine what strategies get tested, apply for research grants, and participate in the research as it progresses.
• Research is needed that addresses the large amount of information that students face and that identifies the types of information that students respond to and act on.
• Replication studies should be designed to measure the effectiveness of promising intervention strategies for specific student groups, with the goal of enhancing effectiveness and cost-effectiveness.

Practice Guide on Effective Advising for Postsecondary Students

In October 2021, the What Works Clearinghouse (WWC) released a Practice Guide on Effective Advising for Postsecondary Students. The practice guide includes four evidence-based recommendations designed for an audience of administrators and staff at community colleges, 4-year institutions, and other public or private technical colleges who are responsible for designing and/or delivering advising to students:

• Intentionally design and deliver comprehensive, integrated advising that incorporates academic and non-academic supports to empower students to reach their educational goals.
• Transform advising to focus on the development of sustained, personalized relationships with individual students throughout their college career.
• Use mentoring and coaching to enhance comprehensive, integrated advising in ways that support students’ achievement and progression.
• Embed positive incentives in intentionally designed advising structures to encourage student participation and continued engagement.

Gap Analysis of Information and Advising Research and Practice

In March 2020, the Lead Team of the College Completion Network began a project aimed at identifying gaps in the research evidence base for information and advising strategies. The project is organized into three parts:

1. A systematic review of the research literature, documenting evidence of the effect of information and advising policies, practices, and programs on student outcomes
2. A scan of information and advising policies, practices, and programs that colleges use to improve student outcomes
3. A gap analysis to compare the findings from the scan to the findings from the systematic review to look for effective practices that are not widely implemented and promising practices in the field that have not been evaluated

The team plans to report its full set of findings by December 2021. College Completion Network study descriptions are available here: https://collegecompletionnetwork.org/studies.

This blog is the second in a blog series on Effective Postsecondary Interventions that highlights interventions with evidence of effectiveness generated through IES-funded research. For the first blog in the series, please see here.

Written by James Benson (James.Benson@ed.gov), a Program Officer for Postsecondary Education within NCER’s Policy and Systems Division, and Felicia Sanders (Felicia.Sanders@ed.gov), a Program Officer for the What Works Clearinghouse within NCEE’s Knowledge Use Division.

Accessibility to job opportunities has long been a challenge for individuals with disabilities. In 2020, people with disabilities had a much higher unemployment rate (12.6%) than those without a disability (7.9%). To raise awareness about disability employment issues and celebrate the contributions of workers with disabilities, National Disability Employment Awareness Month (NDEAM) is held every October. The 2021 theme for NDEAM is “America’s Recovery: Powered by Inclusion,” which underscores the importance of ensuring that people with disabilities have access to employment as the nation recovers from the pandemic.

NCSER supports research related to career readiness and employment for students with disabilities through the Transition to Postsecondary Education, Career, and/or Independent Living (Transition) research program. Since its inception in 2006, NCSER has invested over $58 million in the Transition program and the portfolio has grown to 38 research grants. To encourage additional research in the important but under-studied area, NCSER also competed the Career and Technical Education for Students with Disabilities (CTE) special topic in FY 2019 and FY 2020, awarding three research grants totaling around $3 million.

Below are examples of recently funded Transition and CTE studies that are exploring, developing, or evaluating programs, policies, and practices aimed to help improve career readiness and employment outcomes for students with disabilities.

• Exploring relationships between college and career readiness, self-determination, and transition planning among adolescents with and without disabilities (PI: Allison Lombardi, University of Connecticut) This project aims to establish measurable constructs of college and career readiness by exploring extant data from the National Longitudinal Transition Study 2012.
• Supported College and Career Readiness (SCCR) for Secondary Students with Emotional and Behavioral Problems (PI: Lee Kern, Lehigh University) Through this project, the research team is developing and pilot testing a multi-component program that augments typical school-based college and career readiness activities.
• CTE Teachers and Long-Term Outcomes for Students with Disabilities (PI: Dan Goldhaber, University of Washington) This project aims to examine the relationship between effectiveness in career and technical education for students with disabilities and student outcomes, such as postsecondary enrollment and employment.
• Understanding the Antecedents of STEM Career and Technical Education Coursetaking for High School Students with Learning Disabilities (PI: Michael Gottfried, University of Pennsylvania) Through this project, the research team is investigating whether participation in applied STEM career and technical education courses in high school is related to students with learning disabilities pursuing and persisting in STEM majors and/or careers.
• Moving Transition Forward: Exploration of College-Based and Conventional Transition Practices for Students with Intellectual Disability and Autism (PI: Meg Grigal, University of Massachusetts, Boston) This study aims to examine and compare critical aspects of two transition approaches—a college-based transition experience and a conventional high school or community-based transition experience—and examine the associations between involvement in those programs and student employment outcomes for students with intellectual disability and/or autism spectrum disorder.\
• College and Career Readiness for Transition (CCR4T): Development and Validation of a Student Measure (PI: Allison Lombardi, University of Connecticut) The research team in this project is developing and validating the College and Career Readiness for Transition (CCR4T), a measure of the perceptions of high school students with disabilities of their own college and career readiness.

NCSER also plans to support additional research for students with disabilities with a new grant competition, Research to Accelerate Pandemic Recovery in Special Education, funded through the American Rescue Plan. This grant competition will fund research that addresses pandemic recovery, including recovery relevant to career readiness.

For more information about NDEAM as well as ideas for celebrating this month, please see https://www.dol.gov/agencies/odep/initiatives/ndeam.

Written by Akilah Swinton Nelson, Program Officer at National Center for Special Education Research. For more information about the Transition to Postsecondary Education, Career, and/or Independent Living and Career and Technical Education for Students with Disabilities topic areas, contact Akilah at akilah.nelson@ed.gov.

IES supports rigorous research that can provide scientific evidence on how best to address our nation’s most pressing education needs. As part of the Standards for Excellence in Education Research (SEER) principles, IES-funded researchers are encouraged, and in some cases required, to conduct a cost analysis for their projects with the intended goal of supporting education agencies’ decision-making around the adoption of programs, policies, or practices. 

The Cost Analysis in Practice (CAP) Project is a 3-year initiative funded by IES to support researchers and practitioners who are planning or conducting a cost analysis of educational programs and practices. This support includes the following freely available resources.

• Resources developed by the CAP Project
  • Introductory resources on cost analysis including Standards and Guidelines 1.1, an infographic, a video lecture, and FAQs.
  • Tools for planning your cost analysis, collecting and analyzing cost data, and reporting your results.
  • A Help Desk for you to submit inquiries about conducting a cost analysis with a response from a member of the CAP Project Team within two business days.
• Other resources recommended by the CAP Project
  • Background materials on cost analysis
  • Guidance on carrying out a cost analysis
  • Standards for the Economic Evaluation of Educational and Social Programs
  • Cost analysis software

The CAP Project is also involved in longer-term collaborations with IES-funded evaluation projects to better understand their cost analysis needs. As part of this work, the CAP Project will be producing a set of three blogs to discuss practical details regarding cost studies based on its collaboration with a replication project evaluating an intervention that integrates literacy instruction into the teaching of American history. These blogs will discuss the following:

• Common cost analysis challenges that researchers encounter and recommendations to address them
• The development of a timeline resource for planning a cost study
• Data collection for a cost study

The CAP Project is interested in your feedback on any of the CAP Project resources and welcomes suggestions for additional resources to support cost analysis. If you have any feedback, please fill out a suggestion form at the bottom of the Resources web page.

In response to a call from IES for research on how to best support postsecondary teachers and students to thrive in online environment, NCER is establishing a new research and development (R&D) center. This center, led by SRI International (SRI) and the Community College Research Center (CCRC) at Teachers College (Columbia University), aims to help faculty embed support for self-directed learning skills into their online and hybrid courses.

This R&D center will support postsecondary instructors in making optimal use of technology features often available in online course tools to bolster student self-management strategies. Through its research and capacity-building programs, the center aims to strengthen teaching and learning, improve student outcomes, and ensure all students—regardless of race, ethnicity, or socioeconomic status—have equitable learning opportunities and attainment in broad-access institutions.

“Lack of self-directed learning skills can hinder a student’s success in any college course,” says SRI’s Rebecca Griffiths, a lead researcher in the new center, “but the challenge is greater in online courses, which typically place more responsibility on students to manage their own learning.”

Self-directed learning skills, also known as self-regulated learning skills, encompass three interrelated and mutually reinforcing areas:

• Affect, which includes self-efficacy and the motivation to learn
• Strategic actions, which include planning, goal setting, strategies to organize, code, and rehearse
• Metacognition, which includes self-monitoring, self-evaluating and self-correction

These three areas can form a virtuous cycle. When students believe that studying helps them learn important and useful knowledge, they are more likely to study strategically and effectively. Effective study habits in turn enhance academic performance and build positive mindsets including confidence, motivation around learning, and a sense of personal growth.

SRI and CCRC will partner with Achieving the Dream and nine broad-access, public colleges, and universities across the U.S. to conduct these research program activities.

The research goals of the R&D center are to—

• Generate new knowledge about how faculty can effectively use technology features and instructional practices in online STEM courses to create a positive feedback loop for students
• Shed light on institutional policies and practices and instructional environments needed to support a coherent, intentional, and sustainable approach to helping students build self-directed learning skills across their coursework
• Develop and pilot a technology-enabled, skills development model that will use technology features already widely available in learning management systems, adaptive courseware, and mobile apps to deliver instruction on these skills
• Using research findings to inform the development of a rich, interactive toolkit to support institutions and faculty as they implement self-directed learning skills instruction at scale in online programs.

In addition to carrying out the research activities, the center will provide national leadership and capacity-building activities for postsecondary teaching and learning. Through partnership with Achieving the Dream, technology developers, researchers, education equity advocates, and others, the center will establish the critical importance of integrating self-directed learning into instruction to improve teaching and learning and improve equity in postsecondary student outcomes. They will also engage faculty, instructional designers, and educational technology developers to share knowledge and to co-develop and disseminate capacity-building resources that support teaching these skills and strategies. 

The center is led by Dr. Deborah Jonas (PI, SRI International, top photo), Dr. Nicole Edgecombe (Co-PI, Teachers College, Columbia University), Dr. Rebecca Griffiths (Co-PI, SRI International), and Dr. Neil Seftor (Co-PI, SRI International).

This blog was written by the R&D center team. For further information about the grant, contact the program officer: Dr. Meredith Larson.

Across a set of research grant programs, IES generates knowledge of how to increase students’ access to, progress through, and completion of postsecondary credentials and degrees. Funded projects develop and test a range of interventions from state-level policies to classroom practices, with an emphasis on strategies that promote college attainment for students historically underrepresented in postsecondary education. This new blog series called Effective Postsecondary Interventions highlights interventions with evidence of effectiveness generated through IES-funded research.

The Early College High School Model

The Early College High School (ECHS) model addresses barriers to college attainment commonly experienced by students historically underrepresented in higher education. Students from low-income families and minoritized racial and ethnic groups often attend high schools that lack rigorous pre-college courses, strong support for college enrollment, and established connections to colleges and universities. For those students, cost is an additional barrier to enrollment and persistence in college. The ECHS model addresses these barriers by combining secondary and postsecondary instruction within the same school, prioritizing college-preparatory high school courses, offering opportunities to enroll in college courses while in high school, and providing comprehensive supports for academic and social-emotional development—all at little or no cost to the students. Although concurrent enrollment in high school and college courses (dual enrollment) is a core component of the model, the ECHS model is more expansive than dual enrollment because it includes a broader set of intervention components and has a clear equity objective.

All early colleges reflect these four design principles:

• Enrollment of students historically underrepresented in higher education
• Partnerships including a local education agency, a higher education institution, and the surrounding community
• An integrated program of secondary and postsecondary education with the goal of all students earning 1 to 2 years of college credit prior to high school graduation
• A comprehensive support system for students to develop academic skills as well as social and behavioral skills.

Attending Early Colleges Increases Postsecondary Attainment

Four IES-funded projects have evaluated impacts of the ECHS model. Prior studies within these projects found that significantly larger percentages of early college students completed a college preparatory course of study during high school and enrolled in postsecondary education within six years of entering high school. The two most recent projects assess postsecondary attainment:

• A follow-up efficacy study of 19 North Carolina early colleges led by Dr. Julie Edmunds (University of North Carolina) finds that 42 percent of early college students attained an associate or bachelor’s degree within 10 years of high school entry, compared with 31 percent of their peers in traditional high schools. The WWC review for this study indicates that the study meets WWC standards without reservations, and this finding meets the ESSA Tier 1 evidence standard.
• A follow-up efficacy study of 10 early colleges in five states led by Dr. Mengli Song (American Institutes for Research) finds that 45 percent of early college students attained an associate or bachelor’s degree during the same 10-year period, compared with 33 percent of their peers in traditional high schools. The WWC review for this study also indicates that the study meets WWC standards without reservations, and this finding meets the ESSA Tier 1 evidence standard.

In addition, early college students earned associate degrees at rates that exceeded their counterparts in traditional high schools by 22% and 18%, respectively, while earning bachelor’s degrees at equal or higher rates. These impacts are substantial, and especially noteworthy because both evaluations studied early colleges across a range of settings. Moreover, the impacts are similar for different student subgroups, regardless of gender, race/ethnicity, or family income.

Why have early colleges been so effective?

Early colleges set high expectations, provide high-quality interactions between staff and students, and encourage college access and success for all students. Several studies confirm that early colleges substantially improve high school experiences and outcomes. Students in North Carolina early colleges reported higher expectations, more rigorous and relevant instruction, stronger academic and social supports, and better relationships with teachers than their counterparts in other high schools. Early college students in AIR’s five-state sample reported significantly higher levels of college-going culture and instructor support than their counterparts in traditional high schools. The combination of high expectations and supportive relationships promotes better outcomes for early college students beginning in ninth grade (compared with students in other high schools). For instance, early college students are more likely to persist in college-preparatory math courses, attain a significant number of college credits during high school, and graduate from high school. Importantly, these positive results hold for students from all racial and ethnic groups, including students who enter high school at low levels of math proficiency.

For more information about the studies, the SERVE Center at the University of North Carolina at Greensboro has posted a brief of impact findings from their evaluation of North Carolina early colleges. The American Institutes of Research has posted a brief of impact findings from their five-state evaluation of early colleges.  

Written by James Benson (James.Benson@ed.gov), a Program Officer for Postsecondary Education within NCER’s Policy and Systems Division.