REL Midwest Ask A REL Response

Data Use

October 2020

Allbright, T., & Hough, H. (2020). Measures of SEL and school climate in California. State Education Standard, 20(2), 28–32 and 49–50. https://eric.ed.gov/?id=EJ1257764

From the ERIC abstract: “California’s CORE Districts—a consortium of eight school districts serving a racially and socioeconomically diverse population of over one million students—have since 2014 led the way in deploying measures of social and emotional learning (SEL) and school climate and culture. Influenced by surging interest and research support over the past decade, these districts have collected data in hopes of continuously improving how their K-12 schools address the social and emotional dimensions of student development. In recent years, many advocates have called for schools to pay greater attention to holistic aspects of schooling, arguing for whole-child education, attention to noncognitive factors, and programming to support student SEL. In this article, the authors describe how the CORE Districts plumbed the possibilities of using holistic measures to improve schools.”

Bruch, J., Gellar, J., Cattell, L., Hotchkiss, J., & Killewald, P. (2020). Using data from schools and child welfare agencies to predict near-term academic risks (REL 2020-027). Washington, DC: U.S. Department of Education, Institute of Education Sciences, National Center for Education Evaluation and Regional Assistance, Regional Educational Laboratory Mid-Atlantic. https://eric.ed.gov/?id=ED606230

From the ERIC abstract: “This report provides information for administrators, researchers, and student support staff in local education agencies who are interested in identifying students who are likely to have near-term academic problems such as absenteeism, suspensions, poor grades, and low performance on state tests. The report describes an approach for developing a predictive model and assesses how well the model identifies at-risk students using data from two local education agencies in Allegheny County, Pennsylvania: a large local education agency and a smaller charter school network. It also examines which types of predictors—in-school variables (performance, behavior, and consequences) and out-of-school variables (human services involvement and public benefit receipt)—are individually related to each type of near-term academic problem to better understand why the model might flag students as at risk and how best to support these students. The study finds that predictive models using machine learning algorithms identify at-risk students with moderate to high accuracy. In-school variables drawing on school data are the strongest predictors across all outcomes, and predictive performance is not reduced much when out-of-school variables drawing on human services data are excluded and only school data are used. However, some out-of-school events and services—including child welfare involvement, emergency homeless services, and juvenile justice system involvement—are individually related to near-term academic problems. The models are more accurate for the large local education agency than for the smaller charter school network. The models are better at predicting low grade point average, course failure, and scores below the basic level on state tests in grades 3-8 than at predicting chronic absenteeism, suspensions, and scores below the basic level on high school end-of-course standardized tests. The findings suggest that many local education agencies could apply machine learning algorithms to existing school data to identify students who are at risk of near-term academic problems that are known to be precursors to school dropout.”

Buttaro, A., Jr., & Catsambis, S. (2019). Ability grouping in the early grades: Long-term consequences for educational equity in the United States. Teachers College Record, 121(2). https://eric.ed.gov/?id=EJ1200514

From the ERIC abstract: “Background: Ability grouping has resurged in U.S. schools despite long-standing debates over its consequences for educational equity. Proponents argue that it is the best response to variation in academic skills because it allows teachers to customize the content and pace of instruction to students’ diverse needs. Critics answer that this practice places students in divergent educational paths that reproduce educational and social inequalities. Despite the contested nature of ability grouping, research has yet to produce reliable longitudinal evidence to evaluate critics’ claims. Objective: We examine the degree to which exposure to within-class grouping for reading instruction from kindergarten to third grade is predictive of students’ reading test scores and English coursework in the middle grades. Research Design: We use multilevel achievement growth models predicting average reading achievement from kindergarten to eighth grade as a function of years of exposure in low, average, or high ability groups in kindergarten through third grade and control variables relevant to each grade. We evaluate the achievement differences between students who are grouped in these ability groups for one or more years and those who were never ability grouped. We use multinomial logistic regression models to estimate the degree to which number of years in each ability group in K-3 grades predicts placements in eighth-grade English classes (below grade or honors, as opposed to regular English classes). Data: We use data from the Early Childhood Longitudinal Study-Kindergarten Cohort (ECLS-K), a national panel study of the 1998 U.S. kindergarten cohort sponsored by National Center for Education Statistics, U.S. Department of Education. Our sample consists of 7,800 students with data for fall of kindergarten, and spring of kindergarten and first, third, fifth, and eighth grades. Findings: Compared with similar students who were ungrouped in the early grades, those in high-ability reading groups have higher test scores, whereas those in low-ability groups have lower test scores in every grade from kindergarten to the eighth grade. In addition, compared with their ungrouped counterparts, students in low-ability groups in the early grades are more likely to enroll in eighth grade English classes that are below grade level, whereas those in high-ability groups in these grades are more likely to enroll in honors eighth-grade English classes. Achievement gaps between previously grouped and ungrouped students increase with every additional year of exposure to ability grouping. Conclusions: Students’ ability group placements in the early grades evolve into divergent educational paths that grow further apart with multiple years of grouping. These findings provide the first longitudinal evidence linking ability grouping to the reproduction of educational inequalities.”

Note: REL Midwest was unable to locate a link to the full-text version of this resource. Although REL Midwest tries to provide publicly available resources whenever possible, it was determined that this resource may be of interest to you. It may be found through university or public library systems.

Center on Great Teachers and Leaders. (2020). Talent development framework: Improving access to excellent educators for all students. Washington, DC: American Institutes for Research. https://eric.ed.gov/?id=ED607031

From the ERIC abstract: “Highly effective teachers and school leaders are of utmost importance to improve student achievement and ensure the success of the education system. Unfortunately, students in underserved schools are not provided equitable access to effective teachers and leaders. For this reason, many states and districts have implemented evidence-based talent management strategies at various points across the full educator career continuum to strengthen the quality of the educator workforce. The Center on Great Teachers and Leaders (GTL Center) at the American Institutes for Research (AIR) has developed the ‘Talent Development Framework,’ which provides an opportunity and a roadmap for state- and district-level policymakers to systemically address and improve educator quality. Beyond promoting a comprehensive, systemic approach to improving educator quality, the ‘Talent Development Framework’ aids state and district leaders in proactively and purposefully addressing talent challenges per local context. It also helps policymakers make sense of talent challenges across the educator career continuum by illustrating and systematizing the complexity of developing human talent in education.”

Chikwe, M., & Cooper, R. (2020). School leaders’ sense-making and use of equity-related data to disrupt patterns of inequality. Journal of Educational Leadership and Policy Studies, 3(3), Special Issue 3. https://eric.ed.gov/?id=EJ1252030

From the ERIC abstract: “This qualitative phenomenological study explored how 19 school leaders in seven California comprehensive high schools were making sense of data and using data to transform their schools by closing achievement gap for historically underrepresented students. Data is a powerful tool in the hands of school leaders to transform patterns of inequality and bring about change in teaching practices, outreach to parents, and student-centered activities at the schools where school leaders are motivated by social justice. This study identified two different kinds of data — objective and subjective — that school leaders used. The researchers found that school leaders interpreted and used data in three different ways: as a diagnostic tool, as a critical road map, and as a reference point for crucial conversations. School leaders’ interpretation and use of such data lead to transformative changes that promote equalization of educational opportunities for all students.”

Datnow, A., & Hubbard, L. (2015). Teachers’ use of assessment data to inform instruction: Lessons from the past and prospects for the future. Teachers College Record, 117(4). https://eric.ed.gov/?id=EJ1056748

From the ERIC abstract: “Background: Data use has been promoted as a panacea for instructional improvement. However, the field lacks a detailed understanding of how teachers actually use assessment data to inform instruction and the factors that shape this process. Purpose: This article provides a review of literature on teachers’ use of assessment data to inform instruction. We draw primarily on empirical studies of data use that have been published in the past decade, most of which have been conducted as data-driven decision making came into more widespread use. The article reviews research on the types of assessment data teachers use to inform instruction, how teachers analyze data, and how their instruction is impacted. Research Design: Review of research. Findings: In the current accountability context, benchmark assessment data predominate in teachers’ work with data. Although teachers are often asked to analyze data in a consistent way, agendas for data use, the nature of the assessments, and teacher beliefs all come into play, leading to variability in how they use data. Instructional changes on the basis of data often focus on struggling students, raising some equity concerns. The general absence of professional development has hampered teachers’ efforts to use data, as well as their confidence in doing so. Conclusions: Given that interim benchmark assessment data predominate in teachers’ work with data, we need to think more deeply about the content of those assessments, as well as how we can create conditions for teachers to use assessment to inform instruction. This review of research underscores the need for further research in this area, as well teacher professional development on how to translate assessment data into information that can inform instructional planning.”

Note: REL Midwest was unable to locate a link to the full-text version of this resource. Although REL Midwest tries to provide publicly available resources whenever possible, it was determined that this resource may be of interest to you. It may be found through university or public library systems.

Datnow, A., & Park, V. (2015). Data use—For equity. Educational Leadership, 72(5), 48–54. https://eric.ed.gov/?id=EJ1051704

From the ERIC abstract: “School leaders are often drowning in data but are unsure which forms of data will help them create a portrait of student achievement that will motivate staff to look beyond simple trends and delve deeper into root causes. Teachers often wish for more guidance on the kinds of data analysis and teaching strategies that will help them move the needle in classroom instruction. The authors offer five key principles of data use that can guide leaders in promoting deeper inquiry around data in their schools and districts so all students have the opportunity to achieve at high levels. These principles include articulating your purpose and commitment to equity, emphasizing the importance of pausing and reflecting before you make decisions on the basis of the data, being careful when using data to differentiate instruction, focusing on student engagement, and using your professional judgment. A key lesson? Note the authors, ‘Focus on the principles of effective data use, and not just the practices. Centering on the goals of improving education for all students, promoting an inquiry mind-set, and supporting teacher professionalism will help leaders use data for long-term continual improvement.’”

Note: REL Midwest was unable to locate a link to the full-text version of this resource. Although REL Midwest tries to provide publicly available resources whenever possible, it was determined that this resource may be of interest to you. It may be found through university or public library systems.

Datnow, A., & Park, V. (2018). Opening or closing doors for students? Equity and data use in schools. Journal of Educational Change, 19(2), 131–152. https://eric.ed.gov/?id=EJ1179402

From the ERIC abstract: “Ensuring equitable opportunities and outcomes for all students is a top priority of many educators and policymakers across the globe. Data use can be an important lever for achieving equity, but how this can occur is not well understood. In this article, we draw upon knowledge gained in a decade of in-depth qualitative research to examine the ways in which data use helps to open or close doors for students. We discuss data use practices that influence equity goals: (1) accountability-driven data use and data use for continuous improvement; (2) using data to confirm assumptions and using data to challenge beliefs, and (3) tracking and flexible grouping to promote student growth. Along each of these dimensions, there are active decision makers, complex processes of data use at play, and a great deal of variation both within and across contexts. Ultimately, educators and policymakers are faced with critical choices regarding data use that can profoundly affect students’ daily educational experiences and trajectories.”

Note: REL Midwest was unable to locate a link to the full-text version of this resource. Although REL Midwest tries to provide publicly available resources whenever possible, it was determined that this resource may be of interest to you. It may be found through university or public library systems.

Fabillar, E. (2018). Systemic equity review framework: A practical approach to achieving high educational outcomes for all students. Newton, MA: Education Development Center. https://eric.ed.gov/?id=ED593451

From the ERIC abstract: “Education Development Center’s (EDC) systemic equity review framework helps education leaders take a deeper look at inequities in order to understand the complex systems that affect student performance. While equity audits have historically played a role in curriculum assessments and state accountability systems, achievement gaps by race and class persist in U.S. public schools. Our equity review is distinctive in that we integrate traditional forms of data collection with educational ethnography, a human-centered method that allows for a holistic perspective on equity assets and challenges. In our efforts to re-conceptualize equity assessments, EDC has created a three-phase model to guide the work: (1) Review equity assets and challenge; (2) Develop a theory of action; and (3) Develop and implement an equity improvement plan.”

Farley, C. (2019). Better evidence for better schools: Insights from the first 10 years of the Research Alliance for New York City Schools. New York, NY: Research Alliance for New York City Schools. https://eric.ed.gov/?id=ED594872

From the ERIC abstract: “In 2008, a diverse group of New York City leaders—including representatives from the NYC Department of Education, the teacher and administrator unions, the philanthropic and business communities, and New York University—came together to establish the Research Alliance for New York City Schools. Since that time, the Research Alliance has worked closely with policymakers, educators, and other stakeholders to identify pressing research questions and carry out rigorous studies on a wide range of topics that that matter to the City’s public schools. ‘Better Evidence for Better Schools: Insights from the First 10 Years of the Research Alliance for New York City Schools’ reflects on what the Research Alliance has learned to date. The brief addresses three central questions: (1) What have we learned about system-wide progress and challenges? This section of the brief highlights system-wide improvements, including substantial increases in graduation and college enrollment rates, as well as persistent inequality in student opportunities and outcomes. It also describes a heavy concentration of vulnerable students in particular neighborhoods and schools; (2) What have we learned about district policies to promote more effective schools? This section reviews evidence on NYC’s sweeping high school reform efforts, which included closing many large low-performing high schools, opening new smaller schools, and extending high school choice to all students throughout the district. Other studies described in this section highlight the importance of an explicit theory of action for district programs and initiatives; and (3) What have we learned about efforts to support strong teaching and learning? This section offers a number of key lessons, including ‘measure what matters’ (e.g., school leadership, collaboration, safety, and the creation of a warm, supportive learning environment), ‘monitor student progress early and often’ (in terms of both academic indicators and social and emotional learning), and ‘build educator capacity to use data effectively.’ The brief also outlines several priorities for future research, including studies of the location and sources of educational inequality, studies examining the implementation and impact of specific strategies to promote more equitable outcomes, and research designed to directly and continuously inform practitioners who are implementing and striving to improve programs.”

Flannery, K. B., Kato, M. M., & Horner, R. H. (2019). Using outcome data to implement multi-tiered behavior support (PBIS) in high schools. Eugene, OR: Technical Assistance Center on Positive Behavioral Interventions and Supports. https://eric.ed.gov/?id=ED600632

From the ERIC abstract: “Using data for decision-making is critical for schoolwide leadership teams and has been shown to enhance both social and academic outcomes for students (Faria et al., 2017). Using data effectively, however, requires that teams have a clear vision about the type of data, format of data presentation, and process for using data. To avoid expending resources on data collection that is not well used, we recommend building decision-systems rather than data systems. Start with the decisions a team will make, provide the team with the relevant data, and establish a protocol for using data in making team decisions. Teams need to have the right data in the right format at the right time in order to make efficient and effective decisions. In this Practice Brief we propose that there are at least four core types of data needed by high school [Positive Behavioral Interventions and Supports] PBIS Leadership Teams and that these data can be used to problem-solve at the (a) whole school, (b) at-risk group, or (c) individual student levels. We encourage each school team to review the data currently available in their school for effective decision-making and consider possible revisions to their information systems, as appropriate or if needed.”

Hunter, G. P., Williamson, S., Wilks, A., Hanley, J. M., & Stecher, B. M. (2020). Using data to support the intensive partnerships for effective teaching initiative: Data collection, metric and dashboard creation, and lessons learned. Santa Monica, CA: RAND Corporation. https://eric.ed.gov/?id=ED606013

From the ERIC abstract: “The Intensive Partnerships for Effective Teaching initiative, which was funded by the Bill & Melinda Gates Foundation, was a multiyear effort to improve student outcomes—particularly high school graduation and college attendance among low-income minority students—by increasing student access to effective teaching. The RAND Corporation worked with the foundation to collect and warehouse data from participating sites and to produce annual data dashboards that presented quantitative information about key indicators of the progress of the reforms. During the course of this project, the RAND data team conducted four key activities: (1) defining the metrics that would be used to monitor and assess annual progress and that would appear in the dashboard, (2) collecting the data from the sites to compute the metrics, (3) managing and standardizing the data, and (4) creating the dashboard and reporting the metrics to the sites and the foundation. This report discusses the challenges in defining metrics and collecting data. It also describes how the RAND data team addressed those challenges. Specifically, the authors examine challenges and recommendations in four areas: (1) issues related to defining metrics used to track system performance; (2) issues related to data collection; (3) issues related to managing and standardizing data across sites; and (4) issues related to data confidentiality, data sensitivity, and partnerships. The authors also draw overarching lessons related to the systematic use of education data for periodic program monitoring.”

Knoeppel, R. C., & Della Sala, M. R. (2013). Measuring equity: Creating a new standard for inputs and outputs. Educational Considerations, 40(2), 45–53. https://eric.ed.gov/?id=EJ1006217; full text available from https://newprairiepress.org/cgi/viewcontent.cgi?article=1088&context=edconsiderations#page=50.

From the ERIC abstract: “The purpose of this article is to introduce a new statistic to capture the ratio of equitable student outcomes given equitable inputs. Given the fact that finance structures should be aligned to outcome standards according to judicial interpretation, a ratio of outputs to inputs, or ‘equity ratio,’ is introduced to discern if conclusions can be drawn with regard to the equity of both the financial resources and educational opportunity. In developing this ratio, the authors were interested in knowing if educational outcomes were equitable given equitable inputs. Previous analyses of the equity of finance systems made use of measures of dispersion; yet a more complete understanding of the equity of the system must also include measures of distribution. As such, part of the discussion of the equity ratio will include both an analysis of both the dispersion and the distribution of the results.”

Knudson, J. (2020). Leveraging data for a culture of improvement: Priorities for district leaders (Policy and Practice Brief). Washington, DC: California Collaborative on District Reform at American Institutes for Research. https://eric.ed.gov/?id=ED606510

From the ERIC abstract: “School closures in response to the COVID-19 pandemic have dramatically changed the conditions in which students learn and experience schooling. Disparities in students’ access to learning and in their academic outcomes are likely to exacerbate longstanding challenges and inequities. Now more than ever, educators need information that will help them address student needs, support improvement, and address these inequities. This brief draws on the experiences and lessons of local districts who have taken up the charge to improve access, use, and communication of data throughout their systems. The brief also shares insights for using data in the spirit of continuous improvement and in the service of all California students.”

Palmer, D. L., Almager, I. L., Valle, F., Gabro, C., & deLeon, V. (2019). Using equity audits to create a support system for marginalized students. School Leadership Review, 14(2), 9. https://eric.ed.gov/?id=EJ1269673

From the ERIC abstract: “This qualitative content analysis study examined the framing of equity audits and the Texas Accountability Intervention System (TAIS) plans implemented by aspiring principal fellows to develop a support system for marginalized students. The purpose of this study was to demonstrate how a principal preparation program leverages equity-driven data to support the learning and engagement of all students, with an emphasis on supporting English Language Learners (ELL)s and Special Education (SPED) students. The findings revealed that using equity-driven data and progress monitoring quarterly goals did impact student learning, specifically with the ELL and SPED population.”

Peltzman, A., & Curl, C. (2019). Communicating performance: A best practice resource for encouraging use of state and school report cards. Washington, DC: Council of Chief State School Officers. https://eric.ed.gov/?id=ED593476

From the ERIC abstract: “State and school report cards provide a powerful avenue for states to reach families and the broader public as essential partners in improving student outcomes. The federal Every Student Succeeds Act (ESSA) and many state legislatures require states to publish an array of education data including measures at the state, district, and school levels. The report cards also go deeper, illuminating how these measures vary for students by race and ethnicity, income, language, disability, and other characteristics. State and school report cards that effectively communicate these data to the public can inform educators and families, help them ask better questions, and ultimately, drive school improvement to support all students. To answer questions about student performance, state education agencies have increased their capacity to collect, manage, analyze, and make decisions based on data over the last 15 years. While states have made substantial progress, too few families, community leaders, and other stakeholders regularly review and act on states’ education data. The next frontier for state leadership is to advance beyond providing access to data to driving the use of data. Effective use of data is critical to more effectively support educators and students. When educators have comprehensive information about student performance and can consider that information for all students and based on different student populations, they can start to take critical steps towards addressing achievement gaps. This report includes several specific examples of how data can be used effectively to address equity issues: (1) Initiating conversations about equity with diverse stakeholders; (2) Publicly examining data on current performance and trends; (3) Disaggregating data in meaningful ways to identify disparities in opportunity and outcomes; and (4) Publicly sharing data on measures of students’ progress after graduation and long-term success. With this resource, the authors highlight practices and questions to help state education agencies increase use of state and school report cards for decision-making and continuous school improvement.”

Schueler, B. (2014). Measuring family-school relations for school reform and improvement. Evanston, IL: Society for Research on Educational Effectiveness. https://eric.ed.gov/?id=ED562821

From the ERIC abstract: “To the extent that family engagement does indeed improve students’ chances for academic success, class-based differences in family-school relations could contribute to class-based achievement gaps. Pro-engagement policy efforts could mitigate inequality if they successfully encourage involvement among the least engaged parent populations. However, universal family involvement promotion efforts have the potential to reinforce educational inequality if they simply provide more support for already engaged parents to either stay involved or increase their involvement (Lareau & Shumar, 1996; Fine, 1993). Therefore, additional research is needed to examine whether these programs decrease or reinforce inequality and to identify what policies and practices best encourage engagement among those parents whose children would benefit most. At the school level, to effectively target and promote engagement, educators need to understand how parents perceive the degree to which they engage, whether parents’ perceptions align with the school’s view, as well as the barriers that parents believe prevent them from getting involved (Hoover-Dempsey et al., 2005; Hoover-Dempsey & Sandler, 1997). Unfortunately, there are a limited number of existing tools designed to measure family-school engagement, and particularly barriers to engagement. This presentation and paper describe the process one research team used to develop a set of survey tools that assess parents’ perceptions of their engagement with their children’s schools and the barriers they perceive that prevent them from becoming more involved. Findings suggest that educational leaders and researchers alike can use the survey tools developed to measure parent perceptions of their engagement with the school and the barriers they face to becoming more involved. The author believes that the tools described here can aid educators in identifying groups of parents that are less engaged and in need of targeted outreach efforts, designing family-engagement strategies that are tailored to their communities, and tracking their progress at encouraging engagement over time. The author also supports the opinion that researchers can rely on these tools to better understand the ways in which family-school engagement can be harnessed to alleviate, rather than reinforce, educational inequality.”

UnidosUS. (2018). Educational fairness and Latino student success in Arizona (White paper). Washington, DC: Author. https://eric.ed.gov/?id=ED593639

From the ERIC abstract: “This report provides an overview of key provisions in Every Student Succeeds Act (ESSA), discusses how Arizona’s ESSA plan addresses accountability for Latino students and English Learners (ELs), and provides recommendations to Arizona’s accountability system to better ensure that Latino and EL students in Arizona are receiving a high-quality education that prepares them for both college and career.”

UnidosUS. (2018). Educational fairness and Latino student success in Florida (White paper). Washington, DC: Author. https://eric.ed.gov/?id=ED593642

From the ERIC abstract: “This report provides an overview of key provisions in Every Student Succeeds Act (ESSA), discusses how Florida’s ESSA plan addresses accountability for Latino students and English Learners (ELs), and provides recommendations to Florida’s accountability system to better ensure that Latino and EL students in Florida are receiving a high-quality education that prepares them for both college and career.”

Villani, S. (2018). Educators use data and find solutions to improve equity. Bethesda, MD: Mid-Atlantic Equity Consortium. https://maec.org/resource/educators-use-data-find-solutions-improve-equity/

From the abstract: “Part of CEE’s Exploring Equity Issues series, this paper provides an overview of how education leaders collect and use data to address questions related to equity, identify the root causes of their problems, and make decisions to implement change. Finally, there is a discussion on the need to examine how school district’s policies and practices may limit the access and opportunities for some students, particularly students of color and those living in poverty.”