Skip Navigation
Funding Opportunities | Search Funded Research Grants and Contracts

IES Grant

Title: Catalyzing Comprehension Through Discussion and Debate
Center: NCER Year: 2010
Principal Investigator: Donovan, Suzanne Awardee: Strategic Education Research Partnership (SERP) Institute
Program: Reading for Understanding Research Initiative      [Program Details]
Award Period: 5 years Award Amount: $19,352,384
Type: Multiple Goals Award Number: R305F100026
Description:

Overview: The Reading for Understanding Initiative was created to develop effective approaches to improving reading comprehension for all students in prekindergarten through grade 12. This grant is one of six awarded as part of this Initiative. The purpose of this grant was to explore the contributions of perspective taking, complex reasoning, and academic language skills to reading for understanding for upper elementary and middle school students; to develop, refine, and test the efficacy of two literacy programs that target the development of these skills for a general population of students and for struggling readers; and to develop professional learning opportunities for science teachers that could serve as a model for other content areas regarding discipline-specific challenges of reading and argumentation. The grant covered four major activities: basic studies, development of interventions, professional development, and efficacy studies.

Basic Studies: The research team hypothesized that deep reading comprehension is a function of simple comprehension (i.e., understanding the meaning of individual words and sentences), background knowledge, and three independent, malleable capacities that had been given little attention previously: perspective taking, academic language skills, and complex reasoning. In order to test the theory, the team developed instruments to measure each of the three capacities: the Social Perspective Taking Acts Measure (SPTAM), Core Academic Language Skills Instrument (CALS-I), and Reflective Judgment Test (RFJ) for complex reasoning. Each of the instruments was independently validated (Diazgranados, Selman, & Dionne, 2016; Uccelli, Barr, Dobbs, Phillips Galloway, Meneses, & Sánchez, 2015).

The team collected longitudinal data over three years in order to explore developmental trajectories for each of the three capacities. The team also conducted studies to explore whether perspective taking, complex reasoning, and academic language: (a) are independent of each other, and (b) can account for variation in comprehension outcomes. The team explored whether the predictive value of the measures varies with different comprehension assessments and in different classroom contexts.

To analyze how these capacities relate to deep comprehension, the team used longitudinal data collected with the Global Integrated Scenario-based Assessment (GISA), a newly developed computer-based assessment designed to reflect students' abilities to evaluate texts, integrate information from multiple texts, and use textual evidence to formulate a position, all features of deep reading comprehension. Utilizing data collected in fall and spring of one academic year with fourth-through-seventh grade students, researchers tested the role of academic language, perspective taking, and complex reasoning in explaining variance in end-of-year GISA scores, controlling for beginning-of-year scores and student demographics. All three predictors explained small, but significant, amounts of additional variance. (LaRusso, Kim, Jones, Selman, Uccelli, Jones, Donovan, and Snow, 2016).

Key Personnel: Selman, Robert; Uccelli, Paola; Dawson, Theo; Fischer, Kurt; LaRusso, Maria; Donovan, Suzanne; Snow, Catherine

Development of Interventions: The research team developed multiple programs that rely on discussion and debate to catalyze the growth of academic language skills, perspective taking ability, and complex reasoning. The first suite of programs, Word Generation (WG), is a set of tier 1, cross content-area programs for students in grades 4–8 (wordgen.serpmedia.org). Researchers also developed a tier 2 program—the Strategic Adolescent Reading Intervention (STARI) — that targets middle school students reading several grade levels below expectation (stari.serpmedia.org). It is intended to build their deep comprehension skills at the same time that more basic reading skills are addressed.

The Word Generation suite is comprised of WordGen Weekly, Science Generation, Social Studies Generation, and WordGen Elementary. WordGen Weekly is a middle school program that exposes students to academic vocabulary, builds perspective-taking skills by providing multiple viewpoints on high-interest, controversial topics, and motivates complex reasoning through the demands of discussion, debate, and writing. This project supported updating, revising, and/or replacing the 72 previously developed units that constituted the original Word Generation program, now called WordGen Weekly . To extend WG into more in-depth treatment of content-area topics in middle school, the project developed 36 entirely new units. Eighteen of the units constitute Social Science Generation (SoGen), exploring ancient civilizations, geography, and civics. The other 18 units comprise Science Generation (SciGen), treating scientific processes and methods, scientific measurement, ecology, force, energy, and genetics. The project also developed WordGen Elementary, a new program that builds academic language skills, perspective taking, and complex reasoning in fourth and fifth grades. Like the middle-school units, the 24 two-week WordGen Elementary units focus on grade-appropriate, high-interest topics, incorporate instructional materials and tasks that cross content areas, and rely on discussion as a mechanism to catalyze the development of the target capacities.

The STARI intervention has a familial relationship with Word Generation; it too relies on high-interest topics, and uses discussion and debate to actively engage students in perspective-taking, complex reasoning, and the use of academic language. STARI is highly structured to support students' word reading, fluency, and simple comprehension skills simultaneously. It uses full-length novels of high interest to students, related nonfiction readings, and project-developed fluency passages at four levels of reading difficulty that relate to the unit theme and build relevant background knowledge. STARI provides materials for two years of instruction, thus offering an option for students to continue if they do not make adequate progress for grade-level work in one year. Level 1 (for sixth/seventh graders) targets the same skill set as level 2 (for seventh/eighth graders), but the topics treated in level 2 are somewhat more sophisticated.

Key Personnel: Snow, Catherine; Hemphill, Lowry; Ellinger, Matthew; Donovan, Suzanne

Professional Development Model: The research team developed a set of strategies and materials to support the professional learning required to implement WG and STARI . For WG, the challenge of leading effective discussion and debate is a central goal of the professional development (PD). STARI shares that goal, but also provides more intensive PD on building basic reading skills, and on program-specific strategies such as partnered reading.

The challenges of discussion and debate, and of reading for understanding more generally, are magnified in science for two reasons: (1) scientific knowledge is considered by many teachers to be settled, leaving little to discuss; and (2) science texts use disciplinary conventions that, until mastered, make comprehension difficult. The project therefore developed and pilot-tested a model of professional development to enhance the skills of middle and elementary teachers of science to teach students how to construct meaning from science texts, with a particular focus on the use of discussion to support comprehension in science classrooms. Video and other data were collected to examine the nature of the change achieved in teaching practice at three points across the two years. Successful strategies that emerged from the work were incorporated into a project website on Reading to Learn in Science (serpmedia.org/rtls), and the Stanford team has offered two iterations of a Reading to Learn in Science MOOC that has reached thousands of participants.

Key Personnel: Osborne, Jonathan; O'Connor, Catherine; Hemphill, Lowry; Snow, Catherine; Donovan, Suzanne

Efficacy Studies: The suite of Word Generation programs was tested in a school-randomized controlled trial in four districts, located in two states in the northeastern region of the United States. The districts include two major cities and one small city serving ethnically diverse, primarily low-income students, and one suburban district serving a primarily white, low-to-middle income population. The study measured the efficacy of the programs under realistic rather than high fidelity conditions. Summer institutes were offered, but many participating teachers were unable to attend. Coaching was made available, but in accordance with standard practice, teachers' engagement with coaches was voluntary. The level of implementation varied, with average student workbook completion ranging between 46% and 57% across years and grades.

The team conducted analyses with a sample of 7,773 fourth-through-seventh grade students nested in 25 schools randomized to treatment and control conditions over two years. Results indicate that the program improved students' knowledge of taught vocabulary for both the elementary cohort and middle grades cohort. Students in fourth and fifth grades who received the program in year 2 improved on measures of academic language skills, perspective taking and deep reading comprehension, with larger effects in high implementing classrooms. In sixth and seventh grade classrooms, students showed significant gains in perspective taking and deep comprehension in year 2 (Jones, LaRusso, Kim, Kim, Selman, Uccelli, Barnes, Donovan & Snow, 2019).

To examine the impact of WG on the quality of classroom interactions, the team conducted observations twice per year for two years and coded on six dimensions of the Classroom Assessment Scoring System (CLASS): Regard for Adolescent Perspectives, Instructional Learning Formats, Content Understanding, Analysis and Inquiry, Quality of Feedback, and Instructional Dialogue. The team hypothesized that WG classrooms would be rated higher on Regard for Adolescent Perspectives, Analysis and Inquiry, and Instructional Dialogue—dimensions that are closely aligned with WG's theory of change and program design. In both Years 1 and 2, WG classrooms observed implementing WG had significantly higher quality interactions than control classrooms for all dimensions, with stronger impacts for the dimensions most closely aligned with program theory. WG classrooms were also observed during regular practice (not implementing the WG program) and in year 1, the quality of classroom interactions was not statistically different from interactions observed in control classrooms. However, in year 2, there was evidence that changes in classroom interactions of WG classrooms carried over into regular practice, with positive, significant treatment differences for the three dimensions most closely aligned with program theory: Regard for Adolescent Perspectives, Analysis and Inquiry, and Instructional Dialogue (LaRusso, Jones, Kim, Kim, Barnes, Donovan & Snow, results presented at the 2016 meeting for the Society for Research on Educational Effectiveness).

In order to conduct a close examination of structured discussion in classrooms, the team developed a new tool: the Low Inference Discourse Observation tool (LIDO). Preliminary analyses provide evidence of its reliability and validity. IRT analyses produced two scores: one for teacher conversational moves, and one for student conversational moves. Researchers found statistically significant, positive treatment effects for both teacher and student conversational moves (O'Connor, LaRusso, Harbaugh, results presented at the 2016 meeting of the American Education Research Association).

STARI was evaluated under realistic rather than ideal conditions in a student-randomized controlled trial (n=483) in four districts in diverse communities in a single state. The program's impact was analyzed using both intent-to-treat (ITT) and treatment-on-the-treated (TOT) models. The ITT effects were positive and statistically significant for four out of the six outcomes measured by the RISE assessment: word recognition and decoding, vocabulary, morphology, and efficiency of basic reading. The TOT effects were nearly double the ITT effects, suggesting that greater exposure to the STARI curriculum improved student outcomes. The GISA was administered separately as a measure of reading for understanding, and demonstrated a statistically significant impact of STARI for both the ITT analysis and the TOT analysis (Kim, Hemphill, Troyer, Thomson, Jones, LaRusso & Donovan, 2016).

Key Personnel: Jones, Stephanie; Kim, James; LaRusso, Maria; Snow, Catherine; Hemphill, Lowry; O'Connor, Catherine; Donovan, Suzanne

Related Projects: Word Generation: An Efficacy Trial (R305A090555); Digitizing STARI: Strategic Adolescent Reading Intervention (91990023C0040); Adapting STARI to Accelerate Student Reading Achievement in Middle Grades (R305N220025)

PRODUCTS AND PUBLICATIONS

ERIC Citations:  Find available citations in ERIC for this award here.

Select Publications:

Diazgranados, S., Selman, R. L., & Dionne, M. (2016). Acts of Social Perspective Taking: A Functional Construct and the Validation of a Performance Measure for Early Adolescents. Social Development, 25, 572–601.

Dobbs, C. L. (2014). Signaling Organization and Stance: Academic Language Use in Middle Grade Persuasive Writing. Reading and Writing, 27, 1327–1352.

Dobbs, C. L. & Kearns, D. (2016). Using New Vocabulary in Writing: Exploring how Word and Learner Characteristics Relate to the Likelihood that Writers use Newly Taught Vocabulary. Reading and Writing, 29, 1817–1844.

Duhaylongsod, L. J. (2016). Promoting Argumentation Skills in Urban Middle Schools: Studies of Teachers and Students Using a Debate-Based Social Studies Curriculum (Doctoral dissertation). Harvard University, Cambridge.

Duhaylongsod, L., Snow, C. E., Selman, R., & Donovan, S. (2015). Toward Disciplinary Literacy: Dilemmas and Challenges in Designing History Curriculum to Support Middle School Students. Harvard Educational Review, 85, 587–608.

Elizabeth, T., Anderson, T., Snow, C. E., & Selman, R. (2012). Academic Discussions: An Analysis of Instructional Discourse and an Argument for an Integrative Assessment Framework. American Education Research Journal, 49 (6): 1214–1250. doi: 10.3102/0002831212456066

Goldman, S., & Snow, C. E. (2015). Adolescent Literacy: Development and Instruction. In A. Pollatsek & R. Treiman (Eds.), Handbook on Reading (463–478). Oxford University Press.

Goldman, S., Snow, C. E., & Vaughn, S. (2016). Common Themes in Teaching Reading for Understanding: Lessons from Three Projects. Journal of Adolescent and Adult Literacy. doi: 10.1002/jaal.586

Hsiao, L. (2015). Moving the Discussion Forward Through Surprises and Dilemmas: Teacher Learning in Academic Discussion (Doctoral dissertation). Harvard University, Cambridge.

Hsin, L., & Snow, C. E. (2017). Social Perspective Taking: A Benefit of Bilingualism in Academic Writing. Reading and Writing: An Interdisciplinary Journal, 30 (6): 1193–1214.

Jones, S. M., LaRusso, M., Kim, J., Kim, H. Y., Selman, R., Uccelli, P., . . . Snow, C. E. (2019). Experimental Effects of Word Generation on Vocabulary, Academic, Language, Perspective Taking, and Reading Comprehension In High Poverty Schools. Journal of Research on Educational Effectiveness, 12 (3): 448–483.

Kim, J. S., Hemphill, L., Troyer, M. T., Thomson, J. M., Jones, S. M., LaRusso, M. & Donovan, S. (2016). Engaging Struggling Adolescent Readers to Improve Reading Skills. Reading Research Quarterly, 1–26.doi:10.1002/rrq.171

LaRusso, M. D., Donovan, S., & Snow, C. (2016). Implementation Challenges for Tier One and Tier Two School-Based Programs for Early Adolescents. In B. Foorman (Ed.), Challenges to Implementing Effective Reading Intervention in Schools. New Directions for Child and Adolescent Development, 154, 11—30.

LaRusso, M., Kim, H. Y., Selman, R., Uccelli, P., Dawson, T., Jones, S., . . . Snow, C. (2016). Contributions of Academic Language, Perspective Taking, and Complex Reasoning to Deep Reading Comprehension. Journal of Research on Educational Effectiveness, 9 (2), 201–222.

LaRusso, M., Jones, S. M., Kim, H. Y., Kim, J., Barnes, S., Donovan, S., & Snow, C. (2016). Impacts of a Discussion-Based Academic Language Program on Classroom Interactions in 4th Through 7th Grades. Presented at the Society for Research on Educational Effectiveness, Washington, D.C.

Li, J., Snow, C. E., Jiang, J., & Edwards, N. (2015). Technology Use and Self-Perceptions of English Language Skills Among Urban Adolescents. Computer Assisted Language Learning, 28 (5), 450–478.doi: 10.1080/09588221.2014.881387

Li, J., Snow, C. E., & White, C. (2015). Teen Culture, Technology and Literacy Instruction: Urban Adolescent Students' Perspectives. Canadian Journal of Learning and Technology / La revue canadienne de l'apprentissage et de la technologie, 41 (3). Canadian Network for Innovation in Education. doi: http://dx.doi.org/10.21432/T2004H

Li, J., Snow, C. E., & White, C. (2015). Urban Adolescent Students and Technology: Access, Use and Interest in Learning Language and Literacy. Innovation in Language Learning and Teaching, 9 (2), 143–162. doi: 10.1080/17501229.2014.882929

O'Connor, C., & Snow, C. E. (2017). Classroom Discourse: What do we Need to Know for Research and For Practice? In M. Schober, A. Britt, & D. Rapp, (Eds.) The Routledge Handbook of Discourse Processes, 2nd Edition.

O'Connor, C., LaRusso, M., Harbaugh, A. G. (2016). Measuring Classroom Conversational Moves With a New Observation Tool: Insights Into Teacher Behavior, Student Behavior, and Classroom Intervention Impacts. Presented at the American Educational Research Association Annual Conference, Washington, D.C.

Osborne, J., Donovan, B. M., Henderson, J. B., MacPherson, A. C., & Andrew, W. (2016). Arguing From Evidence in Middle School Science: 24 Activities for Productive Talk and Deeper Learning. Corwin.

Phillips Galloway, E. (2016). The Development of Core Academic Language and Reading Comprehension in Pre-Adolescent and Adolescent Learners (Doctoral dissertation). Harvard University, Cambridge.

Phillips Galloway, E., Stude, J., & Uccelli, P. (2015). Negotiating Communicative Practices in Schools: Student's Oral and Written Reflections on the Academic Register. Linguistics & Education, 31, 221–237.

Phillips Galloway, E., & Uccelli, P. (2015). Modeling the Relationship Between Lexico-Syntactic and Discourse Organization Skills in Middle Grade Writers: Insights into Later Productive Language Skills that Support Academic Writing. Reading and Writing, 28 (6), 797–828.

Snow, C. E. (2014). Extended Discourse in First and Second Language Acquisition: A Challenge and an Opportunity. Journal of Japanese Linguistics, 30.

Snow, C. E. (2014). Input to Interaction to Instruction: Three Key Shifts in the History of Child Language Research. Journal of Child Language, 41 (Supplement 1), 117–124.

Snow, C. E. (2014). Language, Literacy, and the Needs of the Multilingual Child. Perspectives in Education, 32, 11–20.

Snow, C. E. (in press). Motivation and Engagement in Language and Literacy Development. In E. Segers & P. van den Broek (Eds.), Developmental Perspectives in Written Language and Literacy. Amsterdam: John Benjamins Publishing.

Snow, C. E., Uccelli, P., & White, C. (2013). Bedingungen und Bedeutung des Erwerbs von Bildungssprache Durch Heranwachsende. (The Conditions for and Significance of Children's Acquisition of Academic Language). In C. Rosebrook & A. Bertschi-Kaufmann (Eds.), Literalitat Erfassen: Bildungspolitisch, Kulturell, Individuell (29–39). Base: Beltz Juventa.

Surrain, S., Duhaylongsod, L., Selman, R., & Snow, C. E. (2019). Using Narrative Thinking in Argumentative Writing. In E. Veneziano & A. Nicolopoulou (Eds.), Narrative, literacy and other skills: Studies in intervention . Philadelphia, PA: John Benjamins.

Uccelli, P., Barr, C. D., Dobbs, C. L., Phillips Galloway, E., Meneses, A., & Sánchez, E. (2015). Core Academic Language Skills (CALS): An Expanded Operational Construct and a Novel Instrument to Chart School-Relevant Language Proficiency in Pre-Adolescent and Adolescent Learners. Applied Psycholinguistics, 36 (5), 1077–1109. doi: 10.1017/S0142716400006X.

Uccelli, P., Galloway, E. P., Barr, C., Meneses, A., & Dobbs, C. (2015). Beyond Vocabulary: Exploring Cross-Disciplinary Academic Language Proficiency and its Association with Reading Comprehension. Reading Research Quarterly, 50 (3), 261–359. doi: 10.1002/rrq.104

Uccelli, P., & Meneses, A. (2015). Lenguaje académico: construcción y validación de un instrumento para medir habilidades de lenguaje académico en estudiantes de primaria. Invited paper in Special Edited Volume, M. Brisk & P. Proctor, Miríada Hispánica, 10, 177–202.

Uccelli, P., & Phillips Galloway, E. (2016). Academic Language Across Content Areas: Lessons from an Innovative Assessment and from Students' Reflections About Language. Journal of Adolescent & Adult Literacy. doi: 10.1002/jaal.553


Back