This blog helps state and local education agencies turn literacy learning questions into a practical measurement plan by inventorying existing data sources and mapping them to questions about rollout, implementation quality, data use, and outcomes.
In the critical effort to support children’s early literacy development, many state education agencies and local education agencies have adopted new literacy policies and made major investments, yet turning literacy policy into improved classroom practice and student reading outcomes is rarely straightforward. Drawing on Regional Educational Laboratory (REL) Northwest research and technical support partnerships with education agencies in Alaska, Oregon, and Washington, this three-blog series examines the questions that state and local education agencies often ask as they strengthen, align, measure, and evaluate literacy efforts.
The series focuses on actions leaders can take to evaluate literacy efforts, with practical examples of how to:
- Develop learning questions that prioritize systems and quality of literacy programming while staying anchored in student outcomes.
- Build a measurement plan that makes smart use of existing data.
- Use the data to evaluate literacy initiatives.
The series does not prescribe a single model, but rather highlights promising practices and questions to support ongoing evaluation and improvement of literacy programming. See the Using Existing Data to Evaluate and Measure Literacy Policies and Programming resource for more.
Building a measurement plan that makes smart use of existing data
The first blog in this series described how state and local education agencies can use focused learning questions to evaluate whether literacy policies and programming are being implemented as intended, supported with sufficient quality, and contributing to stronger teaching practices and student reading outcomes. This blog focuses on how state and local education agencies can turn learning questions into a practical measurement plan using data that they likely already have.
After state and local education agencies determine the set of learning questions they want to answer about literacy programming, the next task is figuring out how to answer them with accessible data and information. Some may assume they either need to launch new data collection efforts or commission studies before they can begin evaluating their literacy efforts. However, funding and bandwidth for these efforts may be limited, and state and local education agencies may be able to leverage existing data. The key is cataloging existing data sources, then connecting those data deliberately to learning questions.
Start with a data inventory
A data inventory is a simple but powerful first step: a structured effort to identify what data already exist and where they live. Before connecting data to learning questions, state and local education agencies need a clear picture of what data are available.
Data might come from:
- Student information systems.
- Assessment platforms.
- Human resources and staffing records.
- Professional learning attendance records and coaching logs.
- State report card data.
- Curriculum vendor dashboards.
- Program evaluations.
With an inventory in hand, the next step is connecting each data source to the learning questions it can support. Building a simple matrix helps clarify what data are available for each question, what is missing, and what would be needed to answer it with confidence.
A useful first move is to classify each data source by the type of question it can support. Some sources are well suited to answer:
- “Did it happen?” questions about rollout and reach, such as whether a screening was administered to all kindergarteners as intended.
- “How well?” questions about implementation quality and enabling conditions, such as whether students receive the full amount of expected tutoring time.
- “So what?” questions about signals in teacher practice and student outcomes, such as whether teachers’ core classroom instructional practices change after training and coaching has been provided.
Being clear about what each source can and cannot answer prevents overreliance on convenient but misaligned data. This classification also helps identify gaps. Mapping the inventory to learning questions will reveal which questions remain unanswered, and whether additional data could be collected, existing measures could be adapted, or certain questions should be deprioritized for the time being.
The goal of this mapping exercise is not to build a perfect measurement system but to create a practical plan that identifies the best available data for each priority question, acknowledges where gaps exist, and is realistic about what can be tracked given available capacity.
A practical measurement plan
The components that follow illustrate what this mapping looks like in practice. Each component and set of learning questions includes common data sources, many of which state and local education agencies already have access to, to help staff connect common learning questions about literacy programming to data they can realistically use.
Component A. Literacy screening and data use
Rollout: Which local education agencies and schools are administering a high-quality screener at the intended grades and time points? Are participation rates sufficient to draw meaningful conclusions?
Data sources: Local education agency assessment plans, state screener approval list compared to local education agency selections, vendor rosters, local education agency submissions to the state data system, and administration calendars.
Quality: Are there common administration or training issues that may undermine data quality (accommodations, make-up tests, administration conditions, scoring timelines)?
Data sources: Training attendance and completion logs, administration fidelity checklists, school testing coordinator logs, accommodation reports, and vendor test session records (timing, pauses, flagged irregularities).
Data use: To what extent do state screening expectations connect to the next steps schools are expected to take when students are identified as at risk for reading difficulties?
Data sources: Multi-tiered system of supports (MTSS) meeting notes and decision logs, intervention placement records, progress monitoring data reports, diagnostic assessment records, and student information system records indicating tier or level of support.
Outcomes: Are local education agencies using screening data to prompt timely follow-up actions (further diagnostics, connecting students to support)? Is that use becoming more consistent over time?
Data sources: Intervention start dates and service minutes, schedules, weekly or biweekly progress monitoring data, student growth reports from vendors, and special program participation records.
- Rollout: Are all eligible students screened on schedule? Where are students being consistently missed by school, grade, or student group?
Data sources: Screener participation files, student enrollment files, attendance during testing windows, and student group data in student information systems and state longitudinal systems (grade level, school, student population). - Quality: Do educators have clear, shared guidance on next steps for follow-up assessment and interventions? Are those used consistently across schools?
Data sources: Local education agency MTSS handbook, decision rule documentation, MTSS logs (records of intervention tracking and team decisions), sample of student files for documented decisions, and data meeting notes and records. - Data use: Do schools have routines for reviewing screening data and documenting decisions (grade-level meetings, team notes)?
Data sources: Meeting calendars and minutes, intervention tracking system records, principal checklists, and coaching notes. - Outcomes: For students identified as at risk, how quickly are supports provided? Do ongoing assessment data show expected growth once support begins?
Data sources: Dates of assessment in testing platforms, intervention rosters with start dates, dates students were referred for additional support, and scheduling and service delivery logs.
Component B. High-quality instructional materials (HQIM)
- Rollout: Which local education agencies have adopted materials for literacy that meet the state education agency’s definition of quality (vetted list, rubric, ratings)?
Data sources: Local education agency curriculum adoption reports, state curriculum survey, procurement and contracts, state-vetted list selections, and textbook and data system reporting. - Quality: What state education agency supports are reaching local education agencies (implementation guidance, leader tools, curriculum-aligned professional learning) for reading and writing? Where do local education agencies still need more support?
Data sources: State education agency technical assistance and professional learning participation rosters, webinar attendance, coaching and technical assistance logs, distribution and download analytics for guidance documents, and local education agency implementation plans. - Quality: What barriers are local education agencies reporting (planning time, staffing, competing initiatives)? Which barriers can the state education agency most realistically help address?
Data sources: Local education agency surveys and interviews, technical assistance needs assessments, implementation plan reviews, principal surveys, and staffing and vacancy reports. - Outcomes: Where implementation supports are strongest, are there more consistent changes in how teachers instruct reading and writing? Over time, are there stronger student literacy outcomes?
Data sources: State or local education agency classroom observation and walkthrough tools, coaching logs, teacher surveys on materials use, interim assessment results, state assessment results, and early literacy benchmark trends.
Rollout: Has the team defined what literacy material adoption means beyond simply purchasing materials (access, use, and expectations for consistency)? Do leaders and educators share that definition?
Data sources: Teacher surveys, lesson plan audits, learning management system and curriculum platform usage analytics, principal walkthrough logs, and grade-level planning artifacts.
Quality: Are teachers using core materials with consistency (intended scope and sequence, routines, text complexity)? Where are adaptations helping or hurting implementation?
Data sources: Unit and lesson plan submissions, professional learning community notes, walkthrough observation notes, teacher self-reports, coaching cycle documentation, and sample artifact review.
- Quality: Is professional learning directly connected to the adopted materials (unit planning, lessons, using student work)? To what degree does the professional learning also integrate state literacy standards and proficiency expectations?
Data sources: Professional learning agendas and materials, attendance and completion records, coaching logs, teacher feedback surveys, observation of professional learning sessions, and standards-to-curriculum alignment documentation mapped against standards. - Outcomes: Are there early indicators that instruction is changing in the ways the team expected (walkthrough data, coaching notes, instructional artifacts)? Are students’ results reflecting those changes?
Data sources: Interim and benchmark assessments, running records, formative assessment artifacts, student work protocols, and student group trend reports.
Component C. Multi-tiered system of supports for reading (MTSS-R)
- Rollout: To what extent do local education agencies have a clear framework for identifying and supporting struggling readers, including how students move in and out of different levels of support?
Data sources: Local education agency MTSS-R handbooks, policy submissions, technical assistance artifacts, state education agency monitoring reviews, and rubric-based document review. - Quality: How well does MTSS-R guidance connect screening, core classroom instruction, and additional intervention? Or are these treated as separate efforts?
Data sources: Local education agency guidance documents, crosswalk of assessment-to-intervention guidance documents, interviews and focus groups, technical assistance notes, and implementation rubrics. - Quality: Do local education agencies have the staffing, schedules, materials, and training needed to deliver intervention as designed?
Data sources: Staffing assignments (interventionists and coaches), master schedules, intervention program inventories, budget and expenditure records, and caseload reports. - Outcomes: Are students getting additional help when assessment data show they need it? Do ongoing assessment data show growth for students receiving intervention?
Data sources: Tier placement records, service minutes, progress monitoring, special education referral rates (contextual), and intervention exit data.
- Rollout: Can the team clearly describe and demonstrate how it identifies students who need extra support, what that support looks like, and whether support is working?
Data sources: MTSS platform exports (data reports from intervention tracking systems), tier rosters, student support plans, and intervention schedules. - Quality: How often are additional and intensive supports delivered? Are they in small enough groups and matched closely enough to what students need?
Data sources: Schedules, service logs, intervention attendance, group rosters, and time-on-intervention reports. - Quality: What is the process by which team meetings result in clear, documented next steps, such as adjusting instruction, changing a student’s level of support, or following up on progress?
Data sources: Meeting minutes, decision logs, followup task trackers, and case audits. - Outcomes: For students receiving intervention, are students making progress? What is the plan when students are not making expected progress (adjust intensity, change approach, improve core classroom instruction)?
Data sources: Progress monitoring trend lines, documented instructional changes, intervention change logs, and referral documentation.
Component D. Teacher professional learning
- Rollout: Who is participating in state-supported literacy professional learning (by role, grade band, region)? Is participation frequent and intensive enough to meaningfully improve instruction? Are followup supports in place to meaningfully improve instruction?
Data sources: Registration and attendance records, completion and certification records, coaching assignment rosters, and records of participation frequency and duration. - Quality: To what extent is professional learning grounded in evidence-based reading instruction and connected to the materials educators are expected to use?
Data sources: Professional learning curricula and agendas, alignment rubrics, coaching program documentation, participant surveys, and facilitator observation notes. Quality: To what extent are literacy coaching, peer support, and administrator support in place so educators can implement what they learn?
Data sources: Coaching logs and assignment records, administrator observation and feedback records, teacher surveys on support received, and coaching program documentation.
- Systems: To what degree do licensure and educator preparation program requirements align with evidence-based reading instruction?
Data sources: Licensure standards, educator preparation program syllabi audits, program approval documentation, and candidate assessment results. - Outcomes: Do available indicators suggest that teachers are continually improving how they teach literacy? Are those changes most visible where professional learning is strongest?
Data sources: Preassessments and postassessments, teacher surveys, observation tools, coaching notes, retention rates, and turnover rates.
Rollout: Are teachers, interventionists, and principals receiving professional learning that is relevant to their specific role in supporting early literacy?
Data sources: Professional learning catalogs, role-based pathways, attendance rosters by role, and professional learning plans.
- Quality: Is literacy professional learning connected to teachers’ daily work (planning, practice, feedback) with followup coaching and opportunities to troubleshoot?
Data sources: Coaching logs, coaching cycle frequency records, teacher participation records, and professional learning community agendas. Quality: Are principals and literacy leaders prepared to support teachers’ literacy instruction through classroom observation and meaningful feedback?
Data sources: Leader training records, walkthrough tool use rates, feedback artifacts, and leader self-assessments.
Outcomes: How is the team tracking the changes in literacy teaching practices (walkthrough tools, teacher planning materials, coaching cycles)? Do those indicators align with improvements in student outcomes?
Data sources: Walkthrough data trends, teacher artifacts, observation ratings, and student work samples.
Component E. Family engagement
- Rollout: What family-facing literacy resources are available statewide? How widely are local education agencies actually using them?
Data sources: Distribution logs, resource webpage traffic and download analytics, local education agency self-reports, and partner program participation rosters. - Quality: Do resources reach families where they are, in their language, in accessible formats, and with clear guidance on what they can do at home?
Data sources: Accessibility reviews, translation inventories, family surveys, usability testing, and feedback from family liaisons. Systems: Do local education agencies have support to help families understand screening results and connect them to next steps for their child?
Data sources: Sample family communication templates, local education agency guidance, technical assistance logs, and family notification compliance.
- Outcomes: Do families report understanding reading development goals and how to support reading at home? Are more families participating in literacy efforts over time?
Data sources: Family surveys and polls, participation counts in events and workshops, library partnership metrics, and resource utilization metrics.
- Rollout: Are families receiving timely, clear communication about what their child is working on in reading, screening results, and available supports?
Data sources: Communication logs, sample family letters and texts, translation records, family-facing portal analytics, and parent–teacher conference records. - Quality: Which families are not being reached? What is getting in the way?
Data sources: Participation rates by student group, records of families who were unreached (returned mail, undelivered texts), survey nonresponse patterns, and community partner feedback. - Quality: How does the team gather input from families on the family-facing resources and how they use them? What does the feedback reveal about additional support the team could create?
Data sources: Short feedback forms, event exit tickets, family liaison notes, and continuous improvement trackers. - Outcomes: Do family feedback and resource use data suggest increased capacity to support early literacy?
Data sources: Family survey results, resource download and utilization data, event participation trends, family liaison observational notes.
Rollout, quality, and outcomes questions about literacy initiatives may not require new data collection efforts to answer, however they do require a plan. The work is in finding the data, organizing it, and connecting it to the right question. The final blog in this series explores what that looks like in practice.
