Co-Principal Investigators: Carrell, Scott; Hetts, John; Johnson, Hans; Lundy-Wagner, Valerie; Martorell, Paco; Rodriguez, Olga

Education Agency: California Community Colleges (CCC)

Related Network Teams: The Community College Recovery Research Network, part of the Improving Pandemic Recovery grant program, includes the following projects — Accelerating Recovery in Community Colleges Network Lead (R305X220022); Leveraging Technology and Engaging Students: Evaluating Covid-19 Recovery Efforts in the Los Angeles Community College District (R305X220018); Strengthening Virginia's Pandemic Recovery Efforts: Providing High-Quality Community College Workforce Education to Underserved Adults (R305X220024)

A Research Network involves several teams of researchers who are working together to address a critical education problem or issue. The objective is to encourage information sharing, build new knowledge, and assist policymakers and practitioners to strengthen education policies and programs and improve student education outcomes. The Community College Recovery Research Network was established in September 2022 to conduct research to support strategies for counteracting declines in postsecondary enrollment and academic progress that occurred for many learners during the COVID-19 pandemic. This Network addresses the needs of student subgroups identified in the American Rescue Plan Act of 2021, many of whom were acutely impacted by disruptions resulting from the COVID-19 pandemic. Each project within the network includes at least one education agency. The Network Lead and Research Teams will work collaboratively with the agencies to identify, explore, evaluate, and improve programs and policies designed to accelerate re-engagement and academic progress for all learners, with a specific focus on impacted subgroups.

Purpose: The purpose of this Network project is to assess pandemic recovery efforts undertaken by the California Community Colleges (CCC) through a set of research and dissemination activities aiming to: (1) determine what recovery activities CCC have engaged in, and how supplemental federal, state, and philanthropic dollars were spent in those efforts; (2) examine which of these recovery activities have contributed to improving student outcomes; and (3) engage community college leaders and state policymakers on best practices for improving student outcomes post-pandemic. Enrollments at CCCs have declined by 14 percent since 2019, leading to concerns about whether and how students may return to their intended degree paths. Using substantial resources from federal, state, and philanthropic entities, CCCs have engaged in a host of recovery efforts to address continuing declines in enrollment and to support student persistence and degree attainment. To date, there has been no formal investigation to understand whether and which of these recovery efforts have been effective in addressing the reengagement of community college students, their performance, persistence, and completion. This project will address this need through a combination of descriptive and exploratory research carried out in partnership with the CCC.

Project Activities: During the first phase of this project, the research team will conduct a survey of college leaders to uncover how supplemental federal, state, and other philanthropic dollars were spent on recovery efforts, and what these recovery activities consisted of. The survey instrument will inquire about supplemental financial aid, basic needs support, technology access, changes to enrollment and grading practices, alternative modes of remote instruction (i.e., synchronous/asynchronous courses), and investments in professional development for online instruction. During the second phase of the project the research team will engage in descriptive and quasi-experimental analysis of survey data as well as administrative data from the California Community Colleges Chancellor's Office (CCCCO). Researchers will begin this phase with correlational analyses of college recovery activities and student engagement outcomes. Next, they will analyze the impact of two specific recovery activities—supplemental financial aid, and the introduction of short course formats—as well as two additional recovery activities identified as promising during phase 1. During phase three, the research team will engage community college leaders across California in a set of continuous improvement-focused activities that directly build upon the research findings from research phases one and two.

Products: Through all phases of the project, researchers will prepare presentations of emerging results in partnership with the CCCCO through their regular webinar series and other professional learning opportunities aimed at community college practitioners and leaders. Other infographics, or research and policy briefs will be prepared for national audiences and the field at large and published through Wheelhouse: The Center for Community College Leadership and Research at UC Davis or through the Public Policy Institute of California (PPIC). In addition, the research team will present and publish results from descriptive and quasi-experimental analyses in peer-reviewed conferences and publications.

Structured Abstract

Setting: This project takes place in 116 community colleges in California.

Sample: The sample will be a census of students enrolled in California community colleges from the 2014–15 through the 2023–24 academic year, with nearly two million enrollees each year.

Factors: The project will focus on at least four recovery activities undertaken by CCC to support student reengagement and academic success including supplemental student relief aid, shortened course formats, other reforms aiming to increase enrollment flexibility, and grading practices.

Research Design and Methods: To understand what actions CCC have taken to address enrollment declines and student outcomes, the research team will analyze data from the CCCCO administrative database and a project-administered statewide survey of recovery activities at the colleges. In addition, correlational analyses will focus on the relationship between these college-level recovery activities and student outcomes and assess whether and what types of recovery efforts may be supporting student reengagement and success. The research team will also conduct quasi-experimental analyses that leverage differences in the implementation of specific recovery strategies across the system, including the timing of those activities and details of their implementation. All analyses will include tests for potential heterogeneity by race/ethnicity, gender, age, and a proxy for income.

Control Condition: In the quasi-experimental analysis of the impact of supplemental aid, students who do not receive supplemental aid, or who receive smaller amounts of aid, will serve as the control condition. For the analysis of short course formats, students in standard-length courses will serve as the control condition.

Key Measures: From administrative data provided by CCC, the research team will construct a variety of short- and long-term measures of postsecondary success at both the individual and college level including enrollment, persistence (e.g., fall-to-fall enrollment), credit accumulation, course performance, degree and certificate attainment, and transfer to a four-year university. From the survey of college administrators, researchers will construct measures that specify the recovery activities undertaken by the colleges such as supplemental student relief aid and short course formats, as well as variation in the features and implementation of these activities. Also from administrative data provided by CCC, the research team will construct student subgroup measures (race, gender, age, first time vs returning, part-time vs full-time).

Data Analytic Strategy: The research team will begin their analysis by employing descriptive statistics and correlations to summarize and assess the types of recovery activities employed by colleges and the extent to which these activities are associated with improved student outcomes. Next, to assess plausible causal relationships between recovery activities and student outcomes, the research team will employ quasi-experimental modeling strategies to evaluate relations between the recovery activities and short- and long-term postsecondary measures of postsecondary success. To assess the effects of short courses, researchers will use student fixed effects to control for unobserved factors at the student level that are correlated with academic outcomes, and campus-by-course fixed effects to control for differences in the availability of short courses across campuses and courses. To assess the effects of financial aid funded through recovery programs, researchers will employ a difference-in-difference design to identify the effect of the relief aid by comparing the change in outcomes between eligible and ineligible students before and after the pandemic. The model specification will also include campus fixed effects to distinguish effects of the relief aid from effects of other programs that may vary across campuses. All analyses will also include a focus on key student subgroups as well as region and urbanicity of campuses.

Cost Analysis: Using administrative data and data collected from the Phase 1 survey, the research team will calculate costs for recovery activities that entail spending (e.g., relief aid paid to students). They will report both per-recipient and total program costs.