|Title:||California's College and Career Readiness Standards in the Era of Common Core Assessments|
|Principal Investigator:||Kurlaender, Michal||Awardee:||University of California, Davis|
|Program:||Evaluation of State and Local Education Programs and Policies [Program Details]|
|Award Period:||5 years (8/1/15-7/31/20)||Award Amount:||$4,974,387|
Co-Principal Investigators: Jonathan Isler (California Department of Education), Scott Carrell and Paco Matorell (University of California, Davis)
Purpose: In this project, researchers will describe and evaluate the impact of the revised California Early Assessment Program (EAP) that signals 11th grade students about their readiness to take postsecondary for-credit courses (based on their 11th grade state test scores in English and math) and provides advice on improving their readiness during 12th grade. In a previously funded IES study, researchers found that the Early Assessment Program had modest effects of college readiness for students attending the California State University system. The program has undergone two major changes justifying further evaluation. First, the program switched from voluntary to mandatory student participation. Second, California's adoption of the Common Core State Standards was followed by its adoption of the standards-aligned Smarter Balanced Summative Assessments (Smarter Balanced) in 2014–2015, which are now used to determine students' college readiness status.
Project Activities: The research team will carry out both a descriptive analysis and an evaluation of the EAP. The descriptive analysis will identify the characteristics of students identified as college ready and the schools they attend, and will document how these schools have changed with California's switch to Smarter Balanced. The descriptive analysis will also track students in Career and Technical Education (CTE) pathways in K–12 into postsecondary schooling to examine general postsecondary outcomes, including persistence, degree attainment, and employment. Researchers will employ a regression discontinuity design to estimate the impact of being placed into the categories of college readiness and the concurrent advice provided by category on 12th grade coursetaking and academic performance, college application and enrollment, and postsecondary remedial coursetaking and for-credit coursetaking. Researchers will also examine whether any impact differs by student demographics or school characteristics.
Products: The research team will provide evidence of the efficacy of California's EAP. Researchers will provide this evidence directly to the California Department of Education and to the three public higher education systems in California — California Community Colleges(CCC), California State University (CSU), and University of California(UC) — through ongoing joint meetings, briefings, and policy papers. Researchers will present results to the research community through presentations and peer-reviewed publications and to the practitioner and policymaker communities through a series of policy briefs, reports, seminars, and conferences supported by Policy Analysis for California Education (PACE).
Setting: The project will take place in the state of California.
Sample: The study will include all California 11th graders in public schools during the academic years 2009–2010 through 2019–2020.
Intervention: Starting in in 2014–2015, 11th grade students take the Smarter Balanced computer adaptive assessments for English—language arts/literacy and mathematics. These tests are based on the Common Core State Standards, which California began implementing in 2010. Based on their scores, students are categorized into one of four levels of achievement, with Level 4 identified as “college ready” and Level 3 as “conditionally ready.” Level 4 students can directly enroll in college-level courses at a California college without having to take a remedial placement test. Level 3 students receive a letter advising them that they can directly enroll in college-level courses if they complete an approved course (an advanced English course or a math course that requires Algebra II as a prerequisite) in 12th grade with a grade of C or above. Level 2 students receive general advice on preparing for college and interpreting the test results, but do not receive specific advice on 12th grade coursetaking, and are expected to have to take the postsecondary remedial placement test. California began the EAP in 2004 using the California Standards Tests augmented with a set of items that students could voluntarily complete and a focus on student entry in the CSU system. Since then, the community college system joined the program, the testing has become mandatory, and the Smarter Balanced Summative Assessments have replaced the augmented California Standards Tests.
Research Design and Methods: The team will use a regression discontinuity design to compare impacts of the students just above and just below each cutoff in the Smarter Balanced Summative Assessments that is used to place students into the four college readiness categories (e.g., comparing students who scored just high enough to be identified as “college ready” with those just missing that cutoff). Researchers will estimate impacts for 12th grade coursetaking and academic performance, college application and enrollment, and postsecondary remedial coursetaking and for-credit coursetaking. Researchers will also examine how the effect of these signals on these outcomes differs by student and school characteristics (e.g., student demographics, student prior achievement, high school characteristics).
Control Condition: For each college readiness category, students in the adjacent category representing a lower stage of college readiness will be the comparison group (i.e., Level 4 compared to Level 3, Level 3 to Level 2, and Level 2 to Level 1).
Key Measures: In this study, measures of outcomes are 12th grade coursetaking and academic performance, college application and enrollment, and postsecondary remedial coursetaking and for-credit coursetaking. The measures of college readiness are the scores on 11th grade English language arts tests and a mathematics test. Covariates include student demographics, prior student achievement, and high school characteristics.
Data Analytic Strategy: For the descriptive analysis, researchers will use density distributions of student characteristics, a linear random effects model to compare schools, and correlations between the test and postsecondary success. In addition, researchers will use difference-in-difference regression to compare student characteristics and test score to postsecondary success before and after the adoption of the Smarter Balanced Summative assessments. For the evaluation study, researchers will use a regression discontinuity design using the test cutoffs used to place students into the four college readiness categories. The research team will examine scores around the cutoffs for manipulation and observable covariates for differences around the cutoffs.
Related IES Project: Ready or Not? California's Early Assessment Program and the Transition to College (R305A100971)
*This grant was originally funded under the FY 2015 Evaluation of State Education Programs and Policies competition.