Research References

Selecting Curriculum Materials

Eslami, H. (2014). The Effect of Syntactic Simplicity and Complexity on the Readability of the Text. Journal of Language Teaching and Research, 5(5), 1185–1191. Available from: https://www.researchgate.net/publication/276248122_The_Effect_of_Syntactic_Simplicity_and_Complexity_on_the_
Readability_of_the_Text.

From the abstract:

The purpose of this study is to investigate the effect of syntactic simplicity and complexity on the readability of the text. To achieve this, a set of standard reading comprehension passages were syntactically modified to develop three different versions of the same text (i.e., reduced, original, expanded) at different readability levels. A number of 257 senior Iranian EFL students participated in the study. The participants were divided into three proficiency levels of high, mid, and low, each taking the three different versions of the same text. The results revealed that there was no significant difference among the high proficient students' performance on the three versions. However, there were significant differences among the mid and low proficient students' performance on these versions. The results, therefore, indicate that syntactic complexity may create comprehension problems for mid and low proficient students, but not for high proficient ones. The results of this study can be useful for language teachers, syllabus designers and test developers in selecting suitable texts matched to the learners' ability level.

Gersten, R., Baker, S.K., Shanahan, T., Linan-Thompson, S. Collins, P., & Scarcella, R. (2007). Effective Literacy and English Language Instruction for English Learners in the Elementary Grades: A Practice Guide (NCEE 2007-4011). Washington, DC: National Center for Education, Evaluation and Regional Assistance, Institute of Education Sciences, U.S. Department of Education. Available from https://eric.ed.gov/?id=ED497258.

From the ERIC sabstract:

This Practice Guide is the first in a series of the Institute of Education Sciences (IES) guides in education that are developed by a panel of experts. The guides are intended to bring the best available evidence and expertise to bear on the types of systemic challenges that cannot currently be addressed by single intervention or programs. This first guide addresses the challenge of providing effective literacy instruction for English learners in the elementary grades. Although the target audience is a broad spectrum of school practitioners such as administrators, curriculum specialists, coaches, staff development specialists and teachers, the more specific objective is to reach district-level administrators with a Practice Guide that will help them develop practice and policy options for their schools. The Guide offers five specific recommendations for district administrators and indicates the quality of the evidence that supports these recommendations. The recommendations are: (1) Screen for reading problems and monitor progress; (2) Provide intensive small-group reading interventions; (3) Provide extensive and varied vocabulary instruction; (4) Develop academic English; and (5) Schedule regular peer-assisted learning opportunities. The following are appended: (1) Technical Information on the Studies; and (2) Levels of Evidence for the Recommendations in the Practice Guide.

Hetherington, A. (1985). Assessing the Suitability of Reading Materials for ESL Students. TESL Canada Journal, 3(1). 37–52. Retrieved from https://eric.ed.gov/?id=EJ328234.

From the abstract:

The paper begins with an examination of the criteria by which first and second language reading texts have traditionally been graded, criteria which focus primarily on the linguistic characteristics of a text. It is proposed that if reading is viewed as interaction between a text and a reader, there are other variables to consider, those related to the reader side of the process: the readers' interests, background knowledge and purposes for reading. Within this interactive framework and after the reader variables have been considered, the subject matter, format, organization and discourse and linguistic variables of a text can be assessed. Implications of recent research in these areas are discussed. In conclusion, a set of guidelines is proposed for assessing the suitability of both graded and ungraded texts for ESL students.

Effective Readability Formulas

Aziz, A., Fook, C.Y., & Alsree, Z. (2010). Computational Text Analysis: A More Comprehensive Approach to Determine Readability of Reading Materials. Advances in Language and Literary Studies, 1(1), 200–219. Available from https://eric.ed.gov/?id=EJ1131712.

From the ERIC abstract:

Reading materials are considered having high readability if readers are interested to read the materials, understand the content of the materials and able to read the materials fluently. In contrast, reading materials with low readability discourage readers from reading the materials, create difficulties for readers to understand the content of the materials and prevent readers to read the materials fluently. The study presented in this article intends to propose a comprehensive approach to analyze reading materials so that not only the overall readability of the materials can be determined, but information about sentence and word difficulty as well. This study was interested to analyze reading materials at three levels: (1) text; (2) sentence; and (3) and word levels. Three computational tools were used to extract information related to readability of reading materials at the three levels: (1) A readability formula was used to estimate materials difficulty at text level; (2) writing enhancement software was used to estimate materials difficulty at sentence level; and (3) concordance software was used to estimate materials difficulty at word level. It was found that conventional readability formulas usually provide estimates of overall readability of reading materials. However, the composite computational tools proposed in this study, are able to provide more information about readability of reading materials at sentence and word levels. These tools enable estimation of materials difficulty to be performed objectively and reliably.

Council of Chief State School Officers (2017). Supplemental Information for Appendix A of the Common Core State Standards for English Language Arts and Literacy: New Research on Text Complexity. Washington, DC: Council of Chief State School Officers. Available from https://eric.ed.gov/?id=ED576695.

From the ERIC abstract:

Appendix A of the Common Core State Standards (hereafter CCSS) contains a review of the research stressing the importance of being able to read complex text for success in college and career. The research shows that while the complexity of reading demands for college, career, and citizenship have held steady or risen over the past half century, the complexity of texts students are exposed to has steadily decreased in that same interval. In order to address this gap, the CCSS emphasize increasing the complexity of texts students read as a key element in improving reading comprehension. The importance of text complexity to student success had been known for many years prior to the release of the CCSS, but its release spurred subsequent research that holds implications for how the CCSS define and measure text complexity. As a result of new research on the quantitative dimensions of text complexity called for at the time of the standards' release, this report expands upon the three-part model outlined in Appendix A of the CCSS in ELA/Literacy that blends quantitative and qualitative measures of text complexity with reader and task considerations. It also presents new field-tested tools for helping educators assess the qualitative features of text complexity. [For "Common Core State Standards for English Language Arts & Literacy in History/Social Studies, Science, and Technical Subjects. Appendix A: Research Supporting Key Elements of the Standards, Glossary of Key Terms," see ED522007.]

Janan, D., & Wray, D. (2014). Reassessing the Accuracy and Use of Readability Formulae. Malaysian Journal of Learning and Instruction, 11(1), 127–145. Available from https://eric.ed.gov/?id=EJ1137275.

From the ERIC abstract:

Purpose: The purpose of the study is to review readability formulae and offer a critique, based on a comparison of the grading of a variety of texts given by six well-known formulae.

Methodology: A total of 64 texts in English were selected either by or for native English speaking children aged between six and 11 years. Each text was assessed using six commonly used readability formulae via the Words Count website (http://www.wordscount.info/) which provides automated readability indices using FOG, Spache, SMOG, Flesh-Kincaid and Dale-Chall. For the ATOS formula, the Renaissance Learning website was used (http://www.renlearn.com/ar/overview/atos/). Statistical tests were then carried out to check the consistency among the six formulae in terms of their predictions of levels of text difficulty.

Findings: The analysis demonstrated significantly different readability indices for the same text using different formulae. It appeared that some of the formulae (but not all) were consistent in their ranking of texts in order of difficulty but were not consistent in their grading of each text. This finding suggests that readability formulae need to be used carefully to support teachers' judgements about text difficulty rather than as the sole mechanism for text assessment. Significance: Making decisions about matching texts to learners is something regularly required from teachers at all levels. Making such decisions about text suitability is described as measuring the "readability" of texts, and for a long time, this measurement has been treated as unproblematic and achieved using formulae which use such features as vocabulary difficulty and sentence length. This study suggests that the use of such readability formulae is more problematic than may at first appear. Although the study was carried out with native English speaking children using texts in English, it is argued that the lessons learnt apply equally to Malay speakers reading Malay language texts.

Readability Formulas for English Learners

Aziz, A., Fook, C.Y., & Alsree, Z. (2010). Scientific Structural Changes within Texts of Adapted Reading Materials. English Language Teaching, 4(2), 216–223. Available from https://eric.ed.gov/?id=EJ1081968.

From the ERIC abstract:

ESL instructors generally use ready-made reading materials in commercially published coursebooks. However, it would be more effective for learners if ESL instructors are involved in developing reading materials for them. This is because they are closer, more sensitive and responsive to the needs of their learners and they are also aware of the reading ability of the learners. ESL instructors can adapt reading materials taken from non-instructional sources like magazines and newspapers and adapt them to suit the needs and reading ability of their learners. Adaptation enables ESL instructors to adjust the difficulty level of the materials. Adjustment can be made by making changes at sentence and word levels and this study intends to demonstrate the structural changes that take place when the materials are being adapted. This study uses a tool named "IDL Suite" to measure the difficulty level of the materials before and after adaptation.

Baldauf, R.B., Propst, I.K. (1981). Assessing the Readability of Materials for Elementary ESL Pupils, Annual Association for Asian and Pacific American Education, Honolulu, 24 April 1981. Available from https://eric.ed.gov/?id=ED209417.

From the abstract:

This paper assesses the effectiveness of readability indices, standard cloze procedure, and the matching cloze procedure as determinants of the readability of supplementary English materials for elementary ESL students in a Pacific Island context. A review of readability indices and the standard cloze procedure indicated that neither procedure is an adequate measure of text readability, according to the study. Readability indices are identified as not adequately considering differences in sentence and paragraph structure, which are considered a major factor in difficulty levels of elementary ESL texts. The paper states that the standard cloze procedure is too difficult for most beginning ESL pupils because of the production skills necessary to complete the task. Matching cloze procedure, which requires primarily recognition skills, was given to elementary ESL pupils in Saipan to see whether it could be used to evaluate the readability of elementary reading materials. The matching cloze procedure is said to show excellent reliability, and generally correlated more highly with criterion measures for readability.

Crossley, S.A., Allen, D.B., McNamara, D.S. (2011). Text Readability and Intuitive Simplification: A Comparison of Readability Formulas. Reading in a Foreign Language, 23(1), 84–101. Available from: https://eric.ed.gov/?id=EJ926371.

From the ERIC abstract:

Texts are routinely simplified for language learners with authors relying on a variety of approaches and materials to assist them in making the texts more comprehensible. Readability measures are one such tool that authors can use when evaluating text comprehensibility. This study compares the Coh-Metrix Second Language (L2) Reading Index, a readability formula based on psycholinguistic and cognitive models of reading, to traditional readability formulas on a large corpus of texts intuitively simplified for language learners. The goal of this study is to determine which formula best classifies text level (advanced, intermediate, beginner) with the prediction that text classification relates to the formulas' capacity to measure text comprehensibility. The results demonstrate that the Coh-Metrix L2 Reading Index performs significantly better than traditional readability formulas, suggesting that the variables used in this index are more closely aligned to the intuitive text processing employed by authors when simplifying texts.

Greenfield, J. (2004). Readability Formulas for EFL. Japanese Association for Language Teaching, 26(1), 5–24. Available from: http://jalt-publications.org/jj/articles/2622-readability-formulas-efl.

From the abstract:

EFL/ESL teachers use English readability formulas to match texts to their students' reading levels. However, the formulas' validity for EFL/ESL use has gone largely untested. Two studies have now addressed this issue, with divergent results. Brown (1998) found that classic formulas were not very accurate predictors of EFL difficulty, while Greenfield (1999) found that they predicted for EFL about as well as they did for native English readers. Both studies produced accurate EFL readability formulas. In the analysis presented here, the difference in the two studies' findings is attributed to Brown's random passage set. Brown's formula proves more accurate with the other study's passages than with his own, agreeing with observed EFL difficulty and predictions by classic formulas. This supports the finding that the classic formulas are valid for EFL use.

Methods

Keywords and Search Strings

The following keywords and search strings were used to search the reference databases and other sources:

• “Curriculum” AND “Readability” AND “English Language” NOT “Dissertations & Theses”
• “Curriculum Readability” AND “English Language Learners” AND “Pacific” NOT Dissertations & Theses”
• “Curriculum Materials” AND “Readability Strategies” AND “English Language Learners” NOT “Dissertations & Theses”
• “English Language Learners” AND “Curriculum” NOT “Dissertations & Theses”
• “Materials Readability” NOT “Dissertations & Theses”
• “Readability” AND “Pacific” NOT “Dissertations & Theses”
• “Readability” AND “Materials Selection” AND “ESL” NOT “Dissertations & Theses”

Databases and Resources

ERIC, EBSCO Host, ProQuest Education Journals, Google/Google Scholar

Reference Search and Selection Criteria

REL Pacific searched ERIC and other academic journal databases for studies that were published in English-language peer-reviewed research journals within the last 20 years, with the exception of two older articles due to their relevancy to the Pacific context or EL readability measures. REL Pacific prioritized documents that are accessible online and publicly available, and prioritized references that provide practical information based on peer-reviewed research for the teachers and leaders who requested this Ask A REL. Resources included in this document—including URLs, descriptions, and content—were last accessed in June 2018.

This memorandum is one in a series of quick-turnaround responses to specific questions posed by educational stakeholders in the Pacific Region (American Samoa, the Commonwealth of the Northern Mariana Islands, the Federated States of Micronesia, Guam, Hawai'i, the Republic of the Marshall Islands, and the Republic of Palau), which is served by the Regional Educational Laboratory (REL Pacific) at McREL International. This memorandum was prepared by REL Pacific under a contract with the U.S. Department of Education's Institute of Education Sciences (IES), Contract ED-IES-17-C-0010, administered by McREL International. Its content does not necessarily reflect the views or policies of IES or the U.S. Department of Education, nor does mention of trade names, commercial products, or organizations imply endorsement by the U.S. Government.