|Title:||Leveraging Technology and Engaging Students: Evaluating Covid-19 Recovery Efforts in the Los Angeles Community College District|
|Principal Investigator:||Avery, Christopher||Awardee:||Harvard University|
|Program:||Community College Recovery Research Network [Program Details]|
|Award Period:||3 years (09/01/2022 – 08/31/2025)||Award Amount:||$2,889,797|
|Type:||Exploration and Efficacy||Award Number:||R305X220018|
Co-Principal Investigators: Harrington, Deborah; Melguizo, Tatiana; Pearl, Maury
Education Agency: Los Angeles Community College District (LACCD)
Related Network Teams: The Community College Recovery Research Network, part of the Improving Pandemic Recovery grant program, includes the following other projects — Accelerating Recovery in Community Colleges Network Lead (R305X220022); Evidence to Inform Improvement: Supporting California Community Colleges in Pandemic Recovery (R305X220016); Strengthening Virginia's Pandemic Recovery Efforts: Providing High-Quality Community College Workforce Education to Underserved Adults (R305X220024)
A Research Network involves several teams of researchers who are working together to address a critical education problem or issue. The objective is to encourage information sharing, build new knowledge, and assist policymakers and practitioners to strengthen education policies and programs and improve student education outcomes. The Community College Recovery Research Network was established in September 2022 to conduct research to support strategies for counteracting declines in postsecondary enrollment and academic progress that occurred for many learners during the COVID-19 pandemic. This Network addresses the needs of student subgroups identified in the American Rescue Plan Act of 2021, many of whom were acutely impacted by disruptions resulting from the COVID-19 pandemic. Each project within the network includes at least one education agency. The Network Lead and Research Teams will work collaboratively with the agencies to identify, explore, evaluate, and improve programs and policies designed to accelerate re-engagement and academic progress for all learners, with a specific focus on impacted subgroups.
Purpose: The purpose of this Network project is to explore and evaluate the effects and impacts of online and hybrid course modalities on psychosocial and academic outcomes for Los Angeles Community College District (LACCD) students and to develop supports for modalities that improve academic outcomes. At the onset of the COVID-19 pandemic from Fall 2019 to Fall 2020 enrollment in the nine colleges in LACCD fell by 30%, jeopardizing the progress of reform efforts that had helped to produce an increase in the completion rate from 16% to 26% over the previous decade. In response, LACCD invested heavily in improving the quality of its remote and hybrid course offerings. LACCD plans to build on these investments as part of a longer-term shift in its course offerings designed to increase scheduling options, promote student engagement, and improve student outcomes. This project seeks to understand the innovations that LACCD has implemented to date, which students are accessing them, what effects they are having on students' academic progress, and to use the research findings to inform LACCD's reform efforts.
Project Activities: The project team will pursue four lines of inquiry. The first line of inquiry will use historical administrative data and local labor market indicators to describe the influences of the COVID-19 pandemic on LACCD students, focusing on patterns of enrollment, academic success, and decision making about engaging in community colleges. These analyses will be supported by LACCD's rich administrative data infrastructure. The second line of inquiry will consist of a mixed methods evaluation of the response of LACCD to the challenges posed by the pandemic, especially its shift towards online and hybrid course modalities and the relationship of those modalities with student and instructor psychosocial wellbeing. This stage will draw on observations, focus groups, and student and instructor surveys. The third line of inquiry will consist of quantitative assessment of the impact of these online and hybrid course modalities on students' academic outcomes, using administrative data to estimate the influence of dosage, student characteristics, academic preparation, and course subject on grades and credit accumulation. This stage of the project will include both a retrospective quasi-experimental evaluation and a prospective evaluation of the expansion of a novel course modality, hy-flex, that maximizes flexibility for students to access multiple learning modalities within the same course. The fourth and final line of inquiry will draw on interviews and survey responses to explore the role of advising supports and assess the costs and benefits of implementing online and hybrid modalities at scale.
Products: The research team will produce an annual report summarizing new findings for LACCD and stakeholders across its 9 institutions. The project team will also hold an annual convening with LACCD and institutional stakeholders. This convening will be an opportunity to present new findings, solicit feedback, discuss implications for policy and practice, and affirm next steps. The team will release public briefs summarizing new findings and recommendations for policy and practice in years 2 and 3 of the project. A final public report will be released at the end of the project summarizing the findings across the duration of the grant. The team will submit manuscripts of its key findings to peer-reviewed research journals.
Setting: The nine colleges of the Los Angeles Community College District (LACCD). This is the largest community college district in California. All 9 campuses meet the criteria for Hispanic Serving Institutions, and one is also an Asian American and Native American Pacific Islander-Serving Institution.
Population: The population of interest for this study is LACCD students enrolled between Fall 2017 and Fall 2024. Descriptive analyses of trends in enrollment and enrollment intensity will include all entering students in these cohorts. The quasi-experimental evaluation of online learning will examine entering students between Fall 2017 and Fall 2019, as well as entering students in Fall 2021 and beyond. These analyses will be limited to degree-seeking students, excluding students solely enrolled in dual enrollment and non-credit programs, and ELL students. The evaluation of the expansion of hy-flex courses will also be limited to degree-seeking students.
Factors: This project examines how LACCD has expanded and adapted online and hybrid course modalities to facilitate student access to community colleges. LACCD offers asynchronous and synchronous online courses, as well as hybrid courses and a novel hy-flex structure. Hy-flex courses allow students to dynamically choose how to attend each class session while maximizing interactions between in-person and remote students. The research team will first examine the impact of asynchronous online learning on student outcomes, both prior to and after the onset of the COVID-19 pandemic to document the impact of this modality on students as well as whether this impact has shifted following LACCD's substantial investment in instructor training for online instruction. The research team will also evaluate an expansion of hy-flex options in the district to see whether this novel structure successfully combines the flexibility students appreciate with enough support to facilitate student success.
Research Design and Methods: This project draws on a variety of quantitative, qualitative, and mixed methods, guided by the specific lines of inquiry and research questions. For the early quantitative phase of the project, the research team will use descriptive quantitative analyses of administrative and survey data to analyze patterns of student enrollment, academic success, and decision-making in the wake of the COVID-19 pandemic. During the later phase, the team will use a quasi-experimental design to evaluate the impact of hy-flex courses on students' academic outcomes. The team will use mixed methods, drawing on survey, observational, focus group, and administrative records to explore the response of LACCD to the challenge of COVID-19 and the relationship between the shift to online and hybrid learning with student psychosocial outcomes, instructor wellbeing, and instructor workload. The research team will synthesize findings and implications from prior qualitative, quantitative, and mixed methods analyses to develop practical guidance for LACCD when making decisions about course modality offerings and advising students choosing between modalities.
Control Condition: In the causal impact analysis for hy-flex course modalities, students not included in courses using these alternative modalities will serve as the control condition.
Key Measures: The research team will construct measures of student academic outcomes including course performance, persistence, and completion from LACCD administrative records. Instructor well-being and student psychosocial outcomes such as feelings of stress, sense of belonging, and engagement will be measured using previously validated scales. Scores for each construct will be computed following the guidance for the specific scale, whether the average of item responses or a factor score combing all responses included in the scale. Constructs will be checked for reliability in the study sample. Instructional modality will be measured using administrative records from LACCD indicating course modality.
Data Analytic Strategy: The research team will use linear regression models to estimate the causal impacts of online and hy-flex modalities on student academic success and the heterogeneity of those impacts. Instrumental variables and fixed effects parameters will be included in the models to estimate the impact of online learning. When analyzing qualitative data, the team will use deductive analytic strategies to analyze responses from interviews and focus groups using pre-defined question protocols. To address the final line of inquiry related to continued improvements and scaling of online and hybrid modalities, the research team will combine survey, interview, and administrative data to assess the cost of implementing and scaling online and hy-flex modalities.
Cost Analysis: This project will include a cost analysis of LACCD's efforts to create and scale online and hybrid offerings, as well as a cost-effectiveness analysis of the novel hy-flex modality. The team will gather cost information from budget records as well as interviews with key stakeholders and instructor surveys to capture non-monetary costs. The project team will further examine the scalability and sustainability of these modalities by examining instructor turnover as a key mediator for sustainability.