Research References

Abedi, J. (2009). Computer testing as a form of accommodation for English language learners. Educational Assessment, 14(3–4), 195–211. Abstract retrieved from https://eric.ed.gov/?id=EJ870415; full text available at: https://pdfs.semanticscholar.org/33eb/a17e722122c9596c7de0a3cade18100123cf.pdf.

This study compared performance of both English language learners (ELLs) and non-ELL students in Grades 4 and 8 under accommodated and nonaccommodated testing conditions. The accommodations used in this study included a computerized administration of a math test with a pop-up glossary, a customized English dictionary, extra testing time, and small-group testing. Extra time and small-group testing were included only for Grade 4 students. A reading latent composite score was used as a covariate. Results indicated that computer testing was the most effective accommodation in providing valid and accessible assessments for ELL students for both Grades 4 and 8. It is an alternative test item delivery and an easy-to-access gloss of non-math lexicon. This accommodation did not impact the validity of assessments.

Abedi, J. (2013). Validity issues in designing accommodations for English language learners. In G. Fulcher & F. Davidson (Eds.), The Routledge handbook of language testing (pp. 62–76). Abingdon, United Kingdom: Routledge. Retrieved from https://www.routledgehandbooks.com/doi/10.4324/9780203181287.ch3.

Different accommodations are used in the assessment of students who are English language learners (ELLs). However, many of these accommodations were originally created and used for students with disabilities (see Oller, this volume); therefore, the utility of such accommodations for ELL students is questionable. To help reduce the impact of construct-irrelevant sources in the assessment of ELL students, accommodations should be used that (1) are effective in making assessments more accessible for ELL students, (2) provide assessment outcomes that are valid, i.e., comparable with those of non-ELLs, and (3) are sensitive to student background. These three characteristics are vital in determining the validity of accommodated outcomes. First, if an accommodation is not helpful in reducing the performance gap which is partly due to the impact of construct—irrelevant sources, then the accommodation is not doing what it is intended to do. For example, if an accommodation does not help ELL students with understanding the complex linguistic structure of the assessment, which is unrelated to the content being measured, then that accommodation may not be relevant. Second, if an accommodation alters the construct being measured, then the validity of the accommodated assessment could be at risk. An accommodation that provides unfair advantage to the recipients may impact the measurement of the construct. Third, if an accommodation does not address each individual student's educational needs, then the outcome of the accommodated assessment many not present a good picture of what the student truly knows and is able to do. For instance, providing a native language assessment to an ELL student who has been predominantly instructed in English may not produce a desirable outcome. In this chapter, we introduce the concept of accommodation for ELL students, discuss major issues concerning accommodations, and elaborate on the limitations of the currently used accommodations for ELL students. The chapter also discusses the methodology for conducting studies to examine the effectiveness and validity of accommodated assessments and provides recommendations on best practices of accommodations.

Kieffer, M. J., Lesaux, N. K., Rivera, M., & Francis, D. J. (2009). Accommodations for English language learners taking large-scale assessments: A meta-analysis on effectiveness and validity. Review of Educational Research, 79(3), 1168–1201. Abstract retrieved from https://eric.ed.gov/?id=EJ879419; full text available at: https://www.jstor.org/stable/pdf/40469092.pdf.

Including English language learners (ELLs) in large-scale assessments raises questions about the validity of inferences based on their scores. Test accommodations for ELLs are intended to reduce the impact of limited English proficiency on the assessment of the target construct, most often mathematic or science proficiency. This meta-analysis synthesizes research on the effectiveness and validity of such accommodations for ELLs. Findings indicate that none of the seven accommodations studied threaten the validity of inferences. However, only one accommodation—providing English dictionaries or glossaries—has a statistically significant effect on ELLs' performance, and this effect equates to only a small reduction in the achievement score gap between ELLs and native English speakers. Findings suggest that accommodations to reduce the impact of limited language proficiency on academic skill assessment are not particularly effective. Given this, we posit a hypothesis about the necessary role of academic language skills in mathematics and science assessments.

Kieffer, M. J., Rivera, M., & Francis, D. J. (2012). Practical guidelines for the education of English language learners: Research-based recommendations for the use of accommodations in large-scale assessments: 2012 update. Portsmouth, NH: RMC Research Corporation, Center on Instruction. Retrieved from https://eric.ed.gov/?id=ED537635.

This report presents results from a new quantitative synthesis of research on the effectiveness and validity of test accommodations for English language learners (ELLs) taking large-scale assessments. In 2006, the Center on Instruction published a review of the literature on test accommodations for ELLs titled ‘Practical Guidelines for the Education of English Language Learners: Research-based Recommendations for the Use of Accommodations in Large-Scale Assessments’ (Francis, Rivera, Lesaux, Kieffer, & Rivera, 2006). This new publication provides an update to the 2006 report, incorporating evidence from nine studies not previously included and providing updated recommendations for educators and policy-makers. Results drawn from 20 studies (including, in total, more than 33,000 students, of whom more than 9,400 were ELLs) were aggregated using meta- analysis. The studies were primarily conducted using researcher-created tests of mathematics and science with items drawn from the National Assessment of Educational Progress (NAEP) in grades 4 and 8. Drawing on the existing evidence, the authors suggest the following recommendations, ordered by the strength of the available evidence: (1) Use simplified English in test design, eliminating irrelevant language demands for all students; (2) Provide English dictionaries/glossaries to ELLs; (3) Match the language of tests and accommodations to the language of instruction; and (4) Provide extended time to ELLs or use untimed tests for all students.

Kopriva, R. J., Emick, J. E., Hipolito-Delgado, C. P., & Cameron, C. A. (2007). Do proper accommodation assignments make a difference? Examining the impact of improved decision making on scores for English language learners. Educational Measurement: Issues and Practice, 26(3), 11–20. Abstract retrieved from https://eric.ed.gov/?id=EJ775388; full text available at: https://pdfs.semanticscholar.org/6795/3f100d301d
08a381359df94d2bd758952f5c.pdf.

Does it matter if students are appropriately assigned to test accommodations? Using a randomized method, this study found that individual students assigned accommodations keyed to their particular needs were significantly more efficacious for English language learners (ELLs) and that little difference was reported between students receiving incomplete or not recommended accommodations and no accommodations whatsoever. A sample of third and fourth grade ELLs in South Carolina (n = 272) were randomly assigned to various types of test accommodations on a mathematics assessment. Results indicated that those students who received the appropriate test accommodations, as recommended by a version of a computerized accommodation taxonomy for ELLs (the selection taxonomy for English language learners accommodations; STELLA), had significantly higher test scores than ELLs who received no accommodations or those who received incomplete or not recommended accommodation packages.

Additionally, students who were given no test accommodations scored no differently than those students that received accommodation packages that were incomplete or not recommended, given the students' particular needs and challenges. These findings are important in light of research and anecdotal reports that suggest a general lack of systematicity in the current system of assigning accommodations and a tendency to give all available accommodations regardless of individual child characteristics. The results also have important implications for how future accommodation research should be structured to determine the benefits of particular accommodations and accommodation packages.

This study would suggest that control and treatment groups should be assembled based on specific student needs in order for direct comparisons to be made.

Koran, J., & Kopriva, R. J. (2017). Framing appropriate accommodations in terms of individual need: Examining the fit of four approaches to selecting test accommodations of English language learners. Applied Measurement in Education, 30(2), 71–81. Abstract retrieved from https://eric.ed.gov/?id=EJ1132357.

Providing appropriate test accommodations to most English language learners (ELLs) is important to facilitate meaningful inferences about learning. This study compared teacher large-scale test accommodation recommendations to those from a literature- and practitioner-grounded accommodation selection taxonomy. The taxonomy links student-specific needs, strengths, and schooling experiences to large-scale test accommodation recommendations that differentially minimize barriers of access for students with different profiles. A blind panel of experts rated four sets of recommendations for each of 114 ELLs. Results found the taxonomy was a significantly better fit for distinguishing accommodations by student need than teacher recommendations. Further, the fit of teacher recommendations showed no difference when the teacher used a structured data collection procedure to gather profile information about each of their ELLs and when they did not, and teachers' recommendations were not found to differ significantly from a random set of accommodations. Findings are consistent with previous literature that suggests the task of matching specific accommodations to individual needs, rather than the task of identifying individual needs, is where teachers struggle in recommending appropriate test accommodations.

Kos̆ak-Babuder, M., Kormos, J., Ratajczak, M., & Piz̆orn, K. (2018). The effect of read-aloud assistance on the text comprehension of dyslexic and non-dyslexic English language learners. Language Testing, 36(1), 51–75. Abstract retrieved from https://eric.ed.gov/?id=EJ1200824; full text available at: http://eprints.lancs.ac.uk/90124/
1/LT_read_aloud_assistance_final.pdf.

One of the special arrangements in testing contexts is to allow dyslexic students to listen to the text while they read. In our study, we investigated the effect of read-aloud assistance on young English learners' language comprehension scores. We also examined whether students with dyslexia identification benefit from this assistance differently from their peers with no official identification of dyslexia.

Our research was conducted with young Slovenian learners of English who performed four language assessment tasks adapted from a standardized battery of Slovenian national English language tests. In a counter-balanced design, 233 students with no identified dyslexia and 47 students with dyslexia identification completed two language comprehension tasks in a reading-only condition, one task with read-aloud assistance and one task in listening-only mode. We used Generalized Linear Mixed-Effects Modelling (GLMM) to estimate accurately the effects of the mode of administration, dyslexia status, and input text difficulty, while accounting for error variance owing to random differences between students, texts, and questions.

The results of our study revealed that young L2 learners with no dyslexia identification performed similarly in the three conditions. The read-aloud assistance, however, was found to increase the comprehension scores of dyslexic participants when reading difficult texts, allowing them to perform at the level of their non-dyslexic peers. Therefore, our study suggests that this modification of the test administration mode might assist dyslexic students in demonstrating their text comprehension abilities.

Li, H., & Suen, H. K. (2012). The effects of test accommodations for English language learners: A meta-analysis. Applied Measurement in Education, 25(4), 327–346. Abstract retrieved from https://eric.ed.gov/?id=EJ981547.

A meta-analysis using Hierarchical Linear Modeling (HLM) was conducted to examine the effects of test accommodations on the test performance of English language learners (ELLs). The results indicated that test accommodations improve ELLs' test performance by about 0.157 standard deviations—a relatively small but statistically significant increase. Once the potential predictors that may have contributed to the variance of the effect sizes across studies had been accounted for, only English proficiency was found to be significant. Further, the results indicated that ELLs with a low level of English proficiency benefited much more from test accommodations than did those with a high level of English proficiency. Little difference was observed in regard to other factors such as students' ethnicity, students' grade level, or the subject for which they were being examined. Although previous studies have suggested that linguistic simplification may be more effective than other methods, results from this meta-analysis offered no support for that suggestion.

Sato, E., Rabinowitz, S., Gallagher, C., & Huang, C. W. (2010). Accommodations for English language learner students: The effect of linguistic modification of math test item sets. (NCEE 2009–4079). Washington, DC: National Center for Education Evaluation and Regional Assistance, Institute of Education Sciences, U.S. Department of Education. Retrieved from https://eric.ed.gov/?id=ED510556.

This study examined the effect of linguistic modification on middle school students' ability to show what they know and can do on math assessments. REL West's study on middle school math assessment accommodations found that simplifying the language—or linguistic modification—on standardized math test items made it easier for English language learners to focus on and grasp math concepts, and thus was a more accurate assessment of their math skills. The results contribute to the body of knowledge informing assessment practices and accommodations appropriate for English language learner students. The study examined students' performance on two sets of math items—both the originally worded items and those that had been modified. Researchers analyzed results from three subgroups of students—English learners (EL), non-English language arts proficient (NEP), and English language arts proficient (EP) students. Key results include: (1) Linguistically modifying the language of mathematics test items did not change the math knowledge being assessed; (2) The effect of linguistic modification on students' math performance varied between the three student subgroups. The results also varied depending on how scores were calculated for each student; and (3) For each of the four scoring approaches analyzed, the effect of linguistic modification was greatest for EL students, followed by NEP and EP students. The report is structured as follows. Following an Executive Summary and a Study Overview, Chapter 2 describes the study design, sample selection and recruitment, item set development processes, and standardized administration procedures. Chapter 3 describes the implementation of the accommodation (linguistic modification), including discussion of considerations and methods for data analysis. Chapter 4 presents findings from data analyses. Chapter 5 summarizes and interprets key findings, describes study challenges, comments on implications of the findings, and offers recommendations for future research.

Sireci, S. G., & Faulkner-Bond, M. (2015). Promoting validity in the assessment of English learners. Review of Research in Education, 39(1), 215–252. Abstract retrieved from https://eric.ed.gov/?id=EJ1053558; full text available at https://www.jstor.org/stable/pdf/44668658.pdf?seq=1#page_scan_tab_contents.

Across the globe, educational tests are being used at a rapidly increasing rate. More recently, educational tests are being used to inform educational policy and for holding educators accountable for student learning. One reason educational assessments are used for these important purposes is that they are considered to provide reliable and objective information regarding students' achievement. The fact that these tests are ‘standardized,’ meaning the content, test administration conditions, and scoring are uniform (consistent) across all test takers, supports this perception of objectivity.

However, no test is perfectly suited for all students and so educational tests typically do the best they can for the majority of the population tested (Geisinger, 2000). For this reason, it is important to consider identifiable subgroups of the examinee population when considering the fairness and appropriateness of educational tests. In the United States, one important subgroup that presents particularly difficult challenges to valid measurement is English learners (ELs). In this article, we define ELs and the difficult challenges inherent in assessing their academic knowledge, skills, and abilities.

Turkan, S., & Oliveri, M. E. (2014). Considerations for providing test translation accommodations to English language learners on Common Core standards-based assessments. (ETS Research Report No. RR–14–05). Princeton, NJ: Educational Testing Service. Retrieved from https://eric.ed.gov/?id=EJ1109283.

In this article, we review translation, adaptation policies, and practices in providing test accommodation for English language learners (ELLs) in the United States. We collected documents and conducted interviews with officials in the 12 states that provide translation accommodations to ELLs on content assessments. We then summarized challenges to ensuring fair and valid accommodations to ELLs and provided recommendations to address the challenges involved in translating a content test while preserving its validity.

Wolf, M. K., Kim, J., Kao, J. C., & Rivera, N. M. (2009). Examining the effectiveness and validity of glossary and read-aloud accommodations for English language learners in a math assessment. (CRESST Report 766). Los Angeles, CA: University of California, National Center for Research on Evaluation, Standards, and Student Testing (CRESST). Retrieved from https://eric.ed.gov/?id=ED507754.

Glossary and reading aloud test items are often listed as allowed in many states' accommodation policies for ELL students, when taking states' large-scale mathematics assessments. However, little empirical research has been conducted on the effects of these two accommodations on ELL students' test performance. Furthermore, no research is available to examine how students use the provided accommodations. The present study employed a randomized experimental design and a think-aloud procedure to delve into the effects of the two accommodations. A total of 605 ELL and non-ELL students from two states participated in the experimental component and a subset of 68 ELL students participated in the think-aloud component of the study. Results showed no significant effect of glossary, and mixed effects of read aloud on ELL students' performance. Read aloud was found to have a significant effect for the ELL sample in one state, but not the other. Significant interaction effects between students' prior content knowledge and accommodations were found, suggesting the given accommodation was effective for the students who had acquired content knowledge. During the think-aloud analysis, students did not actively utilize the provided glossary, indicating lack of familiarity with the accommodation. Implications for the effective use of accommodations and future research agendas are discussed. Three appendices are included: (1) Example of Read-Aloud Script; (2) Glossary Terms Used in Math Test; and (3) The Five Think-Aloud Items.

Young, J. W., & King, T. C. (2008). Testing accommodations for English language learners: A review of state and district policies. (College Board Research Report No. 2008-6). New York, NY: The College Board. Retrieved from https://eric.ed.gov/?id=ED563044.

This report is a review and summary of current information regarding test accommodations currently used in different states and districts for English language learners (ELL). Similarities and differences among states regarding ELL accommodation are documented. Five appendixes are included: (1) Accommodations Designated for ELLs in States' Policies, Classified by Traditional Accommodation Categories (Rivera et al., 2006); (2) Presentation Accommodations and Modifications for ELLs for States with High School Exit Exams; (3) Response, Timing, and Scheduling Accommodations and Modifications for ELLs for States with High School Exit Exams; (4) Setting and Other Accommodations and Modifications for ELLs for States with High School Exit Exams; and (5) Sources for States’ Accommodations Policies.

Additional Ask A REL Responses to Consult

Ask A REL West at WestEd. (2014). What are testing accommodations for English language learner (ELL) students in the PARCC and Smarter Balanced tests? How are they similar and different? Retrieved from https://relwest.wested.org/system/documents/pdfs/293/original/REL_West_website.Test_Accomms_for_ELs.03.2014-1.pdf?1404937440.

Additional Organizations to Consult

ASCD: http://www.ascd.org/

ASCD is dedicated to excellence in learning, teaching, and leading so that every child is healthy, safe, engaged, supported, and challenged.

• Fair and Square Assessment for ELLs: http://www.ascd.org/publications/educational-leadership/feb16/vol73/num05/Fair-And-Square-Assessments-for-ELLs.aspx

Center for Applied Linguistics (CAL): http://www.cal.org/

The Center for Applied Linguistics (CAL) is a non-profit organization founded in 1959. Headquartered in Washington DC, CAL has earned an international reputation for its contributions to the fields of bilingual and dual language education, English as a second language, world languages education, language policy, assessment, immigrant and refugee integration, literacy, dialect studies, and the education of linguistically and culturally diverse adults and children. CAL's mission is to promote language learning and cultural understanding by serving as a trusted source for research, resources, and policy analysis. Through its work, CAL seeks solutions to issues involving language and culture as they relate to access and equity in education and society around the globe.

• Area of Impact: English learners: http://www.cal.org/areas-of-impact/english-learners

Methods

Keywords and Search Strings

The following keywords and search strings were used to search the reference databases and other sources:

• (“assessment accommodation*” OR “test* accommodation*”) AND (math* OR read*) AND (“English Language Learner” OR “EL” OR “ELL” OR “dual language learner” OR “DLL” OR “emergent bilingual”) AND (appropriate OR construct OR compare OR valid* OR fair)
• (“assessment accommodation*” OR “test* accommodation*”) AND (math* OR read*) AND (“English Language Learner” OR “EL” OR “ELL” OR “dual language learner” OR “DLL” OR “emergent bilingual”) AND (appropriate OR construct OR compare OR valid* OR fair) AND (“text to speech” OR “read aloud” OR “extended time” OR “assistive technology” OR “adult transcription” OR “rest break*” OR “visual representation”)

Databases and Resources

We searched ERIC, a free online library of more than 1.6 million citations of education research sponsored by the Institute of Education Sciences (IES), for relevant resources. Additionally, we searched the academic database ProQuest, Google Scholar, and the commercial search engine Google.

Reference Search and Selection Criteria

In reviewing resources, Reference Desk researchers consider—among other things—these four factors:

• Date of the publication: Searches cover information available within the last 10 years, except in the case of nationally known seminal resources.
• Reference sources: IES, nationally funded, and certain other vetted sources known for strict attention to research protocols receive highest priority. Applicable resources must be publicly available online and in English.
• Methodology: The following methodological priorities/considerations guide the review and selection of the references: (a) study types—randomized controlled trials, quasi experiments, surveys, descriptive data analyses, literature reviews, policy briefs, etc., generally in this order; (b) target population, samples (representativeness of the target population, sample size, volunteered or randomly selected), study duration, etc.; (c) limitations, generalizability of the findings and conclusions, etc.
• Existing knowledge base: Vetted resources (e.g., peer-reviewed research journals) are the primary focus, but the research base is occasionally slim or nonexistent. In those cases, the best resources available may include, for example, reports, white papers, guides, reviews in non-peer-reviewed journals, newspaper articles, interviews with content specialists, and organization website.

Resources included in this document were last accessed on January 22, 2019. URLs, descriptions, and content included here were current at that time.

This memorandum is one in a series of quick-turnaround responses to specific questions posed by educational stakeholders in the Appalachian Region (Kentucky, Tennessee, Virginia, and West Virginia), which is served by the Regional Educational Laboratory Appalachia (REL AP) at SRI International. This Ask A REL response was developed by REL AP under Contract ED-IES-17-C-0004 from the U.S. Department of Education, Institute of Education Sciences, administered by SRI International. The content does not necessarily reflect the views or policies of IES or the U.S. Department of Education, nor does mention of trade names, commercial products, or organizations imply endorsement by the U.S. government.