Skip Navigation
archived information
REL Appalachia

[Return to Ask A REL]

REL Appalachia Ask A REL Response

Educator Effectiveness, Literacy
PDF icon

January 2019

Question

What are the school- and classroom-level components necessary for an effective middle school literacy program?

Response

Thank you for your request to our REL Reference Desk regarding evidence-based information about effective middle school literacy programs. Ask A REL is a collaborative reference desk service provided by the 10 Regional Educational Laboratories (RELs) that, by design, functions much in the same way as a technical reference library. Ask A REL provides references, referrals, and brief responses in the form of citations in response to questions about available education research.

Following an established REL Appalachia research protocol, we searched for peer-reviewed articles and other research reports on school and classroom components of middle school literacy programs. We focused on identifying resources that specifically addressed the effects of middle school literacy programs on student outcomes. Our search yielded very few references pertaining specifically to school level components. Therefore, the references below focus primarily on classroom level components of middle school literacy programs. The sources included ERIC and other federally funded databases and organizations, research institutions, academic research databases, and general Internet search engines. For more details, please see the methods section at the end of this document.

The research team did not evaluate the quality of the resources provided in this response; we offer them only for your reference. Also, the search included the most commonly used research databases and search engines to produce the references presented here, but the references are not necessarily comprehensive, and other relevant references and resources may exist. References are listed in alphabetical order, not necessarily in order of relevance.

Research References

Herlihy, C. M., & Kemple, J. J. (2004). The Talent Development Middle School model: Context, components, and initial impacts on students' performance and attendance. New York, NY: MDRC. Retrieved from https://eric.ed.gov/?id=ED484744.

From the abstract:
The Talent Development Middle School model was created to make a difference in struggling urban middle schools. The model is part of a trend in school improvement strategies whereby whole-school reform projects aim to improve performance and attendance outcomes for students through the use of major changes in both the organizational structure and the educational processes of middle schools. The models that function in this way—broadly referred to as ‘comprehensive school reform (CSR) models’—have been developed both nationally and locally, and they receive support from a combination of federal, state, and local funding as well as from private foundations. Talent Development has been a key target of federal resources earmarked for expanding the use of CSR initiatives in middle schools. The model reflects many of the core principles embedded in the CSR movement. School-level structural changes, for example, create more personalized learning environments for students and teachers; curricular changes improve the rigor of coursework and raise teachers' and students' expectations; and professional development for teachers fills gaps in both content knowledge and pedagogy. The findings in this report—which offers an initial assessment of the first and most intensive effort at scaling up the use of the Talent Development Middle School model—indicate that Talent Development had a positive impact on eighth-grade math achievement and exhibited modest impacts on attendance rates. At the same time, the model produced an inconsistent pattern of impacts on eighth-grade reading and had few significant impacts on outcomes for seventh-grade students. This assessment is based on an innovative analytic methodology that relies on a combination of before-and-after and comparison-schools methods.

Although the findings offer hope that the Talent Development model can improve academic outcomes, at least in math, for middle school students, more data collection and analysis are needed before definite conclusions can be drawn. A subsequent report will track outcomes for two additional years of implementation and will provide a clearer picture of the potential for improvements in middle school achievement to lead to greater persistence in high school and, eventually, to graduation.

Herrera, S., Truckenmiller, A. J., & Foorman, B. R. (2016). Summary of 20 years of research on the effectiveness of adolescent literacy programs and practices. Washington, DC: U.S. Department of Education, Institute of Education Sciences, National Center for Education Evaluation and Regional Assistance, Regional Educational Laboratory Southeast. Retrieved from https://eric.ed.gov/?id=ED568684.

From the abstract:
The importance of adolescent literacy is well established, and the topic continues to be of both local and national interest. Practitioners need to know not only which programs and practices appear effective, but which have the scientific evidence to support that claim. To identify effective programs and practices for general education students in grades 6–12, this review examined 33 studies of adolescent literacy programs and practices published over the past 20 years using What Works Clearinghouse (WWC) standards to evaluate the scientific rigor of their research design. This review presents key findings from 12 programs and practices demonstrating positive or potentially positive effects on reading comprehension, vocabulary, or general literacy. The review found: (1) Most of the identified programs and practices included instructional elements such as explicit instruction in reading comprehension or use of instructional routines; (2) These programs and practices can be implemented within the structure of a typical middle-school language arts or content-area classroom. In most cases implementation involved ongoing support for teachers; (3) None of the 12 identified programs and practices was conducted in a high school setting; and (4) Some of the programs and practices were identified as having potentially positive effects on high-stakes outcome assessments, such as state accountability reading assessments. The following are appended: (1) The search, screening, and review process; (2) The 111 studies reviewed using What Works Clearinghouse standards; (3) Details of the 33 studies that the review team found had met What Works Clearinghouse evidence standards; (4) Description of the interventions used in the 33 studies that the review team found had met What Works Clearinghouse evidence standards.

Kamil, M. L., Borman, G. D., Dole, J., Kral, C. C., Salinger, T., & Torgesen, J. (2008). Improving adolescent literacy: Effective classroom and intervention practices: A practice guide (NCEE #2008–4027). Washington, DC: National Center for Education Evaluation and Regional Assistance, Institute of Education Sciences, U.S. Department of Education. Retrieved from https://eric.ed.gov/?id=ED502398.

From the abstract:
The goal of this practice guide is to formulate specific and coherent evidence-based recommendations that educators can use to improve literacy levels among adolescents in upper elementary, middle, and high schools. The target audience is teachers and other school personnel with direct contact with students, such as coaches, counselors, and principals. The guide includes specific recommendations for educators and the quality of evidence that supports these recommendations. The first three recommendations are strategies that classroom teachers can incorporate into their instruction to help students gain more from their reading tasks in content-area classes. The fourth recommendation offers strategies for improving student motivation for and engagement with learning.

Together, the recommendations are designed to address the literacy needs of all adolescent learners. The fifth recommendation refers specifically to adolescent struggling readers, those students whose poor literacy skills weaken their ability to make sense of written material.

Little, C. A., McCoach, D. B., & Reis, S. M. (2014). Effects of differentiated reading instruction on student achievement in middle school. Journal of Advanced Academics, 25(4), 384–402. Abstract retrieved from https://eric.ed.gov/?id=EJ1043639.

From the abstract:
Reading instruction often does not focus on appealing to student interests, offering choice, or responding to the needs of advanced readers. In this experimental study, we examined the effects on achievement of an instructional approach involving choice, differentiated instruction, and extensive, supported, independent reading, with corresponding elimination of regular reading instruction. The study, which incorporated multi-site cluster-randomized design, was conducted in four middle schools with 2,150 students and 47 teachers. Pretest and posttest data were collected on reading fluency and comprehension, with Hierarchical Linear Modeling (HLM) procedures used to investigate the effects of the intervention. Results indicated similar results overall for treatment and control group students, with treatment outperforming control on reading fluency at two of the schools. The findings demonstrate that the intervention resulted in similar or higher scores for fluency and similar scores for comprehension, despite the diminished whole-group and small-group instruction provided in the intervention as compared with regular reading classes.

Mohammed, S. S., Swanson, E., Roberts, G., Vaughn, S., Klingner, J. K., & Boardman, A. G. (2010). The effects of Collaborative Strategic Reading instruction on the reading comprehension of middle school students: Year 1. Evanston, IL: Society for Research on Educational Effectiveness. Retrieved from https://eric.ed.gov/?id=ED513345.

From the abstract:
This project is a multi-site, multi-year study designed to test the efficacy of a fully developed intervention, Collaborative Strategic Reading (CSR), with adolescent readers. In year 1, the research questions were: (1) ‘Does CSR improve reading comprehension for adolescent readers attending relatively low SES schools?’; and (2) ‘Does CSR improve reading comprehension for adolescent struggling readers attending relatively low SES schools?’ This study was conducted in 6 middle schools in Texas and Colorado.

Based on the current analysis, the authors conclude that there is a small, significant main effect of CSR on reading comprehension as measured by the Gates-MacGinitie assessment; but that this effect is not statistically significant for struggling readers. The authors believe that the findings from this study suggest that CSR is a feasible and effective practice that can be readily integrated into reading and language arts instruction with positive impact.

They are encouraged about the potential effectiveness of this practice because the positive findings from this efficacy study resulted from treatment implementation conditions that are readily replicable.

Slavin, R. E., Cheung, A., Groff, C., & Lake, C. (2008). Effective reading programs for middle and high schools: A best-evidence synthesis. Reading Research Quarterly, 43(3), 290–322. Abstract retrieved from https://eric.ed.gov/?id=EJ817089; full text available at: https://onlinelibrary.wiley.com/doi/abs/10.1002/
tea.20212
.

From the abstract:
This article systematically reviews research on the achievement outcomes of four types of approaches to improving the reading of middle and high school students: (1) reading curricula; (2) mixed-method models (methods that combine large-and small-group instruction with computer activities); (3) computer-assisted instruction; and (4) instructional-process programs (methods that focus on providing teachers with extensive professional development to implement specific instructional methods). Criteria for inclusion in the study were use of randomized or matched control groups, a study duration of at least 12 weeks, and valid achievement measures that were independent of the experimental treatments. A total of 33 studies met these criteria. The review concludes that programs designed to change daily teaching practices have substantially greater research support than those focused on curriculum or technology alone. Positive achievement effects were found for instructional-process programs, especially for those involving cooperative learning, and for mixed-method programs. The effective approaches provided extensive professional development and significantly affected teaching practices. In contrast, no studies of reading curricula met the inclusion criteria, and the effects of supplementary computer-assisted instruction were small.

U.S. Department of Education, Institute of Education Sciences, What Works Clearinghouse. (2011). Adolescent literacy intervention report: Student team reading and writing. Retrieved from https://eric.ed.gov/?id=ED526132.

From the abstract:
‘Student team reading and writing’ refers to two cooperative learning programs for secondary students included in this intervention report: (1) Student Team Reading and Writing and (2) Student Team Reading. The Student Team Reading and Writing program (Stevens, 2003) is an integrated approach to reading and language arts for early adolescents. The program incorporates (1) cooperative learning classroom processes; (2) a literature anthology for high-interest reading material; (3) explicit instruction in reading comprehension; (4) integrated reading, writing, and language arts instruction; and (5) a writing process approach to language arts. Student Team Reading (Stevens, 1989; Stevens & Durkin, 1992) comprises the reading part of Student Team Reading and Writing and consists of two principal elements: (1) literature-related activities (including partner reading, treasure hunts, word mastery, story retelling, story-related writing, and quizzes) and (2) direct instruction in reading comprehension strategies (such as identifying main ideas and themes, drawing conclusions, making predictions, and understanding figurative language). The writing part of the Student Team Reading and Writing program includes selection-related writing. As part of the two programs that are the focus of this report, students work in heterogeneous learning teams, and activities are designed to follow a regular cycle that involves teacher presentation, team practice, independent practice, peer pre-assessment, and individual assessments that form the basis for team scores. The cooperative learning teams used in the programs are intended to engage students in academic interactions. Two studies of student team reading and writing that fall within the scope of the Adolescent Literacy review protocol meet What Works Clearinghouse (WWC) evidence standards with reservations. The two studies included more than 5,200 adolescent learners from grades 6 through 8 in urban middle schools in the eastern United States. Based on these two studies, the WWC considers the extent of evidence for student team reading and writing on adolescent learners to be medium to large for the comprehension domain and small for the general literacy achievement domain. The two studies that meet WWC evidence standards with reservations did not examine the effectiveness of ‘student team reading and writing’ on adolescent learners in the alphabetics and reading fluency domains. ‘Student team reading and writing’ was found to have potentially positive effects on comprehension and no discernible effects on general literacy achievement for adolescent learners. Appended are: (1) Research details for Stevens (2003); (2) Research details for Stevens & Durkin (1992); (3) Summary of findings; (4) Outcome measures for each domain; (5) Findings included in the rating for the comprehension domain; (6) Findings included in the rating for the general literacy achievement domain; (7) Criteria used to determine the rating of a study; (8) Criteria used to determine the rating of effectiveness for an intervention; and (9) Criteria used to determine the extent of evidence for an intervention.

U.S. Department of Education, Institute of Education Sciences, What Works Clearinghouse. (2012). Adolescent literacy intervention report: Reading Edge. Retrieved from https://eric.ed.gov/?id=ED533259 ;full text of the study by Chamberlain, Daniels, Madden, & Slavin (2007), which met WWC standards without reservations, is available at: https://www.researchgate.net/profile/Robert_Slavin/publication/33041930_A_randomized_
evaluation_of_the_Success_for_All_Middle_School_reading_program/links/5498122b0cf2519f5a1db579/A-randomized-evaluation-of-the-Success-for-All-Middle-School-reading- program.pdf
.

From the abstract:
Reading Edge is a middle school literacy program that emphasizes cooperative learning, goal setting, feedback, classroom management techniques, and the use of metacognitive strategy, whereby students assess their own skills and learn to apply new ones. The program is a component of the Success for All (SFA)® whole-school reform model and provides eight levels of instruction, from beginning through eighth-grade reading levels. Students are grouped into classes based on ability, and whole-class reading instruction is delivered in daily 60-minute blocks. Instruction at the early levels uses fiction, nonfiction, and simple scripts to help students develop basic decoding skills, reading fluency, vocabulary, and comprehension. At reading level 3 and higher, students focus on developing comprehension strategies using both narrative and expository texts. All levels focus on building background knowledge and developing study skills. Although the program is often implemented in the context of the SFA® whole-school reform, this report focuses on Reading Edge as a stand-alone program in grades 4 and higher. Thirty-seven studies reviewed by the What Works Clearinghouse (WWC) investigated the effects of Reading Edge on adolescent readers. One study (Chamberlain, Daniels, Madden, & Slavin, 2007) is a randomized controlled trial that meets WWC evidence standards without reservations. That one study is summarized in this report. The remaining 36 studies do not meet either WWC eligibility screens or evidence standards. Appended are: (1) Research details for Chamberlain et al. (2007); (2) Outcome measures for each domain; (3) Findings included in the rating for the comprehension domain; and (4) Supplemental subtest findings for the comprehension domain.

Additional Ask A REL Responses to Consult

Ask A REL Midwest. (2018). What does the research say about the effects of leadership practices, such as classroom observations, on literacy outcomes for secondary school students? Retrieved from https://ies.ed.gov/ncee/edlabs/regions/midwest/askarel/2018/secondary-literacy-leadership-practices.aspx.

Ask A REL Southwest. (2018). Literacy: Effective reading interventions for secondary students. Retrieved from https://ncee/edlabs/regions/southwest/ask-a-rel/secondary-reading-interventions.aspx.

Additional Organizations to Consult

ASCD: https://www.ascd.org/

From the website:
ASCD is dedicated to excellence in learning, teaching, and leading so that every child is healthy, safe, engaged, supported, and challenged.

Evidence for ESSA: https://www.evidenceforessa.org/

From the website:
To maximize the impact on practice, educational leaders must have a simple, straightforward way to identify programs and practices that meet the ESSA evidence standards. This website was created to help identify these programs. It provides a free, authoritative, user-centered database to help anyone—school, district, or state leaders, teachers, parents, or concerned citizens—easily find programs and practices that align to the ESSA evidence standards and meet their local needs.

What Works Clearinghouse: https://ies.ed.gov/ncee/wwc/

From the website:
The What Works Clearinghouse (WWC) reviews the existing research on different programs, products, practices, and policies in education. Our goal is to provide educators with the information they need to make evidence-based decisions. We focus on the results from high-quality research to answer the question ‘What works in education?’

Methods

Keywords and Search Strings

The following keywords and search strings were used to search the reference databases and other sources:

  • (“middle school” OR “middle grade”) AND (literacy OR reading) AND (“school-wide” OR schoolwide OR “whole school” OR “whole-school” OR “school improvement” OR “comprehensive school”) AND (effectiv* OR impact)
  • (“middle school” OR “middle grade”) AND (literacy OR reading) AND (“class-level” OR “class-wide” OR classroom) AND (effectiv* OR impact)

Databases and Resources

We searched ERIC, a free online library of more than 1.6 million citations of education research sponsored by the Institute of Education Sciences (IES), for relevant resources. Additionally, we searched the academic database ProQuest, Google Scholar, and the commercial search engine Google.

Reference Search and Selection Criteria

In reviewing resources, Reference Desk researchers consider—among other things—these four factors:

  • Date of the publication: Searches cover information available within the last 10 years, except in the case of nationally known seminal resources.
  • Reference sources: IES, nationally funded, and certain other vetted sources known for strict attention to research protocols receive highest priority. Applicable resources must be publicly available online and in English.
  • Methodology: The following methodological priorities/considerations guide the review and selection of the references: (a) study types—randomized controlled trials, quasi experiments, surveys, descriptive data analyses, literature reviews, policy briefs, etc., generally in this order; (b) target population, samples (representativeness of the target population, sample size, volunteered or randomly selected), study duration, etc.; (c) limitations, generalizability of the findings and conclusions, etc.
  • Existing knowledge base: Vetted resources (e.g., peer-reviewed research journals) are the primary focus, but the research base is occasionally slim or nonexistent. In those cases, the best resources available may include, for example, reports, white papers, guides, reviews in non-peer-reviewed journals, newspaper articles, interviews with content specialists, and organization website.

Resources included in this document were last accessed on January 15, 2019. URLs, descriptions, and content included here were current at that time.


This memorandum is one in a series of quick-turnaround responses to specific questions posed by educational stakeholders in the Appalachian Region (Kentucky, Tennessee, Virginia, and West Virginia), which is served by the Regional Educational Laboratory Appalachia (REL AP) at SRI International. This Ask A REL response was developed by REL AP under Contract ED-IES-17-C-0004 from the U.S. Department of Education, Institute of Education Sciences, administered by SRI International. The content does not necessarily reflect the views or policies of IES or the U.S. Department of Education, nor does mention of trade names, commercial products, or organizations imply endorsement by the U.S. government.