|Title:||Center on Standards, Alignment, Instruction and Learning (C-SAIL)|
|Principal Investigator:||Porter, Andrew||Awardee:||University of Pennsylvania|
|Program:||National Research and Development Centers [Program Details]|
|Award Period:||5 yeas (7/1/2015–6/30/2020)||Award Amount:||$9,999,999|
|Goal:||Multiple Goals||Award Number:||R305C150007|
Topic: Standards in Schools
Purpose: The Research and Development Center on Standards, Alignment, Instruction, and Learning (C-SAIL) will examine the implementation of college- and career-ready standards in 5 states, assess how they are related to student outcomes in 50 states, and test an intervention that supports standards-aligned instruction in 5 states. C-SAIL's launch occurs at a time when states and districts across the country are engaged in a variety of initiatives aimed at implementing college- and career-ready standards. During this time of reform, C-SAIL addresses the need for timely research that will catalogue the full range of state and district efforts to implement the standards, and assess the effects of these activities on instruction and student outcomes. Reaching beyond describing and assessing the effects of naturally-occurring variation in standards implementation, C-SAIL will also evaluate the Feedback on Alignment and Support for Teachers (FAST) intervention. This scalable intervention will provide high-quality guidance and support to teachers, as a means of maximizing the positive potential of college- and career-ready standards to improve learning for all groups of students.
C-SAIL begins with the premise that, under favorable conditions, college- and career-ready standards can lead to substantial improvements in student outcomes, and these improvements can be shared across groups including students with disabilities, English learners, low-income students, and students from racial/ethnic minority groups. To explore and test the most favorable conditions for standards, C-SAIL engages the policy attributes theory. Within this theoretical framework, five key policy attributes translate into successful standards implementation:
C-SAIL will employ this theoretical framework as a means of describing and explaining specific activities undertaken by states, districts, schools, and teachers in its studies.
The Center will conduct four studies: an implementation study, a longitudinal analysis, a measurement study, and an intervention study. The two studies of natural variation—the implementation and longitudinal study—seek to understand state, district, and school efforts to implement college- and career-ready standards since the publication of the Common Core State Standards (CCSS) and other college- and career-ready standards. The implementation study will be conducted in five partner states that present sharp contrasts in their approaches to implementation, to see how the new standards have or have not led to different approaches to implementation. A complementary study of natural variation, the longitudinal study will look at all 50 states, plus urban districts in the NAEP Trial Urban District Assessment (TUDA), to assess how timing of adoption of college- and career-ready standards and approaches to their implementation have or have not resulted in improvements to NAEP test scores. The measurement study will revise and modify instruments used to measure the alignment of standards, assessments, instruction, and curriculum materials, for use in the longitudinal and intervention studies. The intervention study will test a scalable professional development strategy intended to support teachers as they align their practice to the standards.
College- and Career-Ready Standards Implementation Study
The Implementation study will provide a detailed picture of how states and districts are supporting implementation of the standards, how educators are interpreting the standards and changing their practice in response to them, and how these changes are related to measures of student learning.
Data collection for the implementation study will be based upon a representative sample of districts, schools, and teachers within 5 states (Kentucky, Massachusetts, Missouri, Ohio, and Texas). Researchers will select approximately 70 districts from each state. Within each district, they will select two elementary and two high schools, and then within each school, two math and two ELA teachers, as well as a teacher of English learning students and a teacher of students with disabilities in the high schools. Annual interviews with state and district administrators during years 1–4 of the project will collect in-depth information on standards implementation, including their alignment with the policy attributes theory. Annual reviews of state and district implementation documents will provide additional information on guidance provided to teachers as they implement the standards. During years one and four, surveys of district officers will collect information about the district's policy, support, and guidance for implementation of college- and career-ready standards. Principal surveys will collect information about principals' leadership and guidance for standards implementation. Teacher surveys will address teachers' instructional practice, how they implement standards in the classroom, and their access to professional development opportunities and standards-supporting materials from their states and districts.
Data collection activities will yield detailed quantitative and qualitative portrayals of standards implementation in each of the states, grounded in the Center's policy attributes theory. By merging annual student assessment data with the quantitative implementation data, researchers will examine relations between standards implementation and student learning. They will assess whether differences in states' policy systems are related to trends in implementation and student achievement, and whether some standards are easier to implement effectively than others. The research team will share the results of this work broadly with policymakers, practitioners, and other stakeholder in each state. Annual implementation reports will also draw lessons from across the five states and target a broader national audience looking to learn from the work being done in the five states.
Longitudinal Outcomes Study
The longitudinal outcomes study will examine the overall effects of implementing college- and career-ready standards and assessments as rigorously as possible, given the national scale of implementation. The research team will assess whether states' adoption of college- and career-ready standards, as well as assessments aligned to the standards, is associated with increases in achievement test scores, high school graduation, and college success. The research team will explore whether the effect of college- and career-ready standards and assessment adoption is moderated by the specificity, consistency, authority, power, and stability of state implementation; and the extent to which state standards are aligned with assessments in math and ELA in elementary and high school. Researchers will also assess whether the effect of college- and career-ready standards on student outcomes varies significantly across student sub-groups including students with disabilities and English Learners.
The research team will employ a comparative interrupted time series (CITS) design for the longitudinal outcomes study, which will include students in all 50 states. The CITS design will exploit natural variation in the timing of standards and assessment implementation, as well as variation in the cognitive demand of individual state standards prior to implementation, to examine as rigorously as possible the effect of implementing college- and career-ready standards and assessments. The model will facilitate annual comparisons between states implementing college- and career-ready standards and those not implementing them, as well as comparisons between implementing states that had weak and strong standards prior to college- and career-ready standards. Outcomes assessed will include National Assessment of Educational Progress (NAEP) test scores in grades 4, 8, and 12; high school graduation rates from NCES; and college enrollment and persistence records from the National Student Clearinghouse, as well as college graduation rates from NCES. The team will measure policy attributes by constructing a 50-state database that describes each state's policies along each of the five policy dimensions, and the alignment of its standards and assessments. Policy attributes will be measured annually for four years, and alignment will be measured annually in grades 4 and 8 for ELA and math.
The longitudinal outcomes study will result in a rigorous appraisal of the early effects of college- and career-ready standards. This research will give educators early information regarding whether their efforts to implement college- and career-ready standards are making a difference, and will also point to potential trouble spots if effects differ across student sub-groups. The outcomes study will also provide needed information to policymakers, who will be interested in knowing whether specific types of policies surrounding the standards enhance or detract from their effects on student outcomes.
The Measurement Study
The measurement study will generate a set of measurement instruments that the project team will use to analyze the academic content and cognitive demand of teachers' instruction and their classroom assessments (i.e. quizzes or exams), as well as curriculum materials. The primary objective of the measurement instruments will be assessing alignment of instruction and materials to college- and career-ready standards. Whereas previous generations of academic standards emphasized instructional topics, college- and career-ready standards place greater emphasis on reasoning and communication abilities, including analysis, investigation, explanation, and demonstration. These higher-order skills place cognitive demand upon students. The research team will accurately account for the cognitive demand of instruction and materials by revising the Surveys of Enacted Curriculum (SEC) framework to account for this dimension of college- and career-ready standards.
The measurement study will produce a revised SEC rubric for rating standards, assessments, and curriculum materials; revised weekly and semester-long teacher surveys; and a new observation instrument that can be used to measure the quality of teachers' instruction in terms of its alignment with college- and career-ready standards. The Center will use these products in its studies, and will make these instruments available to other researchers seeking to assess alignment of instruction and curriculum materials.
The Intervention Study
The Feedback on Alignment and Support for Teachers (FAST) intervention will focus on providing instructional guidance to teachers, as a crucial ingredient for aligning instruction to standards and in turn improving student achievement. This focus responds to previous waves of standards-based reform, in which teachers received limited guidance for improving their instruction. The Center will focus on helping teachers align their instruction to both the content and cognitive demand requirements within college- and career-ready standards. To accomplish this, the Center will provide real-time guidance to teachers in concert with school-based learning communities and engagement from school principals. To assess the impacts of FAST on student outcomes, Center researchers will randomly assign schools to receive the intervention, and compare test scores for students at these schools to those of students in schools not receiving the intervention.
Center researchers will combine timely feedback to teachers on their instruction with instructional guidance in the form of feedback, examples, materials, and example lesson plans. Timely feedback will occur through an online portal and an offsite FAST coach. Teachers will receive regular online feedback on the alignment of their instruction after filling out weekly surveys developed in the Measurement Study. To provide feedback relevant to all students, teachers will fill out surveys for each of three target students: a student who is an English Learner, a student with a disability, and a student at the median in class achievement. Teachers will also scan and upload their assessments (tests and quizzes) on a weekly basis, and a videotaped lesson on a monthly basis. Taken together, this rich set of instructional data will inform input from each teacher's offsite FAST coach. Teachers will be able to access FAST coaches through email or internet chat on an ongoing basis, and during scheduled biweekly coaching sessions via video conference. The FAST coaches will provide standards-aligned materials and examples calibrated to the areas in which the teacher's instruction is misaligned. Input from FAST coaches will be reinforced through regular learning community meetings at intervention schools, attended by Center staff and school principals.
To test the intervention, the Center will implement a randomized controlled trial (RCT) in the 5 states also participating in the implementation study, with 2 districts in each state and 6 schools per district, resulting in a total of 240–480 teachers in the study. The Center chose the school as the unit of assignment because the FAST intervention encourages collaboration among teachers. Researchers will compare outcomes for schools within each district randomly assigned to the treatment (FAST) and control conditions. The main outcomes of interest will be students' scores on states' end-of-year state assessments (in math and ELA), and a Center-created assessment that will be common across the five participating states. Researchers will employ multi-level models to measure the impact of the intervention and impact differences across teachers and schools. To compute the final impact estimates, researchers will use fixed-effects models with binary coefficients for each district, to eliminate the possibility of unrelated district activities biasing the results.
Results from the Intervention Study will provide trustworthy evidence regarding whether or not the FAST intervention strategy leads to improvements in student achievement for students in general, and for key student sub-groups. The Center will produce a manual describing how to implement the FAST intervention. Through its website, the Center will make the FAST intervention available to states and districts interested in implementing it for their students.
Leadership and Dissemination Activities:
The Center will host panels, webinars, and other interactive activities starting in year one. These activities will be designed to foster conversations where states and districts can share implementation successes and failures, and devise plans for improving implementation in the near future. The Center will actively share information with national organizations by hosting seminars and panels timed at the beginning or end of national conferences. The Center will convene two meetings per year for all staff and affiliates.
The Center will maintain a state-of-the-art website, which will serve as a distribution vehicle for research papers, policy briefs, and standards-aligned practices, to serve a wide audience including practitioners and policymakers at all levels of the education system. The Center will regularly publish a C-SAILnotes newsletter, publicize its work via social media, and produce DVDs to communicate successful practices. Center researchers will produce research papers for submission to peer-reviewed publications.
Key Personnel:Andrew Porter (University of Pennsylvania), Laura M. Desimone (University of Pennsylvania), Nelson L. Flores (University of Pennsylvania), Amy Stornaiulo (University of Pennsylvania), Morgan S. Polikoff (University of Southern California), Lynn S. Fuchs (Vanderbilt University), Douglas H. Fuchs (Vanderbilt University), Michael Garet (AIR), Mengli Song (AIR), Scott Buckley (AIR), Michael P. Cohen (AIR), Kathryn Drummond (AIR), Kerstin Le Floch (AIR), Terry Salinger (AIR), Toni Smith (AIR), Kirk Walters (AIR).