Project Activities
The researchers developed the curriculum module and related technology Prytaneum in year 1, conducted the feasibility study among six diverse schools in years 1 and 2, carried out a pilot test of the intervention in diverse locations in Illinois and Florida in year 3, and then analyzed and wrote up the results in year 4. At the time of the submission of the final report, the team was engaged in the ongoing process of writing, publishing and disseminating their findings.
Structured Abstract
Setting
The researchers implemented the study in California, Illinois, Florida, Ohio and Washington State in urban, suburban, and rural schools, and with Democratic and Republican members of Congress. The researchers recruited schools in collaboration with the Chicago Public Schools, the McCormick Foundation Democracy Schools in Illinois, and the Lou Frey Center in Florida.
Sample
In our second year, we recruited a total of 11 teachers in 9 different high schools in California, Florida, and Illinois. All but 2 of the 9 high schools participated in an online deliberative town hall with their Member of Congress. In our third year of data collection, we had 33 classrooms and 12 teachers; 692 students that filled out either a pre- or post-survey (most filled out both) and attended a town hall with their member of Congress. Six members of Congress each hosted one town hall attended by the participating schools located within their congressional district. These schools and members were from California, Illinois, Ohio and Washington State. In addition, we had one teacher from California with four class periods and 149 students who filled out pre-and post- surveys without participating in the curriculum or town hall to provide useful control data that we use below. In all, we exceeded our recruitment goals, though we made adjustments as needed given the challenges presented by recruitment.
The curriculum module is aligned to the Common Core State Standards, the College, Career, and Civic Life (C3) Framework for Social Studies State Standards, as well as state specific standards for Illinois and Florida. The module is structured so that it addresses issue areas that involve controversial topics that are relevant to students' interests, allowing teachers to choose which one is the best fit for their course and students' interests. The curriculum module unfolds over four phases: pre-assessment of student writing and survey outcomes; preparation for deliberative town hall including issue research; the online Deliberative Town Hall; and a reflection, structured writing, and post-assessment. The module provides the opportunity for a deliberation with a member of Congress wherein students develop and explain their own positions on a controversial issue.
Research design and methods
In year 1, the researchers developed the first draft of the module and professional development materials for teachers and began collaborating with them on the feasibility study with careful attention to fidelity. The feasibility study continued through year 2, providing a basis to revise the curriculum, professional development materials, and technology. Researchers also recruited new school sites and teachers to expand the sample. In year 3, continued collecting data (observations, interviews and focus groups, administrative data collection) and finalized the analysis plans for each phase of the research process. In year 4, the researchers conducted all data analysis, and subsequently wrote up, and disseminated findings and information on the intervention to diverse audiences.
Control condition
The researchers adjusted their strategy not to include a control condition. Because of the enormous effort required to recruit schools, districts and teachers to participate, that happen to also be located in the congressional district of members of Congress who also were willing to participate, they were unable to recruit a sufficient number of schools to implement this evaluation as a randomized control trial. They found it difficult to recruit teachers to participate in evaluating a new, experimental curriculum. Further, there were ethical concerns in asking a large number of students to participate in an RCT control condition for which they receive no direct benefit. Instead, the team relied on a Staggered Rollout Design and Difference-in-Differences Design to help estimate the causal impact of the intervention, two analytic approaches that do not require a control group.
Key measures
The researchers measured students' analytical writing ability using the Civically Engaged Writing Analysis Continuum and integrative complexity, which measures demonstrated understanding of competing aspects and trade-offs for a policy topic. The researchers assessed student pre-post outcome surveys that include measures of policy knowledge, argument repertoires, civic dispositions such as political internal and external efficacy as well as political interest, and civic participation.
Data analytic strategy
The researchers leveraged a mixed methods data analytic strategy. They analyzed qualitative data alongside the field notes from classroom observations and quantitative data from surveys and administration data. They also analyzed the quantitative survey data using two research designs: a difference-in-differences analysis and a staggered rollout design, which together allow us to evaluate the robustness of our results and warrant stronger, though not unimpeachable, claims of causality.
People and institutions involved
IES program contact(s)
Products and publications
The research team produced a scalable, 14-day, writing-focused module that fit within existing 12th grade social studies curricula. They also created a custom, web-based platform that packaged the curriculum, online student writing notebooks, and easy-to-use web-based tools for horizontal deliberation. The project team gathered data on the success and pain points of implementation of this curricular module, and produced a teacher-facing website. The team produced updated and accessible curriculum materials along with issue guides. The team also produced town hall videos to learn about how these are done. They will also provide their findings on the effect of exposure to the curriculum module along with thoughts on how to improve future applications of it to teachers, school administrators, policy makers, and to academic audiences in education and in political science through reports and peer-reviewed publications.
Project website:
Publications:
ERIC Citations: Publications from this project available in ERIC can be found here.
Supplemental information
Co-Principal Investigator: Esterling, Kevin
Questions about this project?
To answer additional questions about this project or provide feedback, please contact the program officer.
