Skip Navigation
Print Evaluations

Race to the Top—Early Learning Challenge: Descriptive Study of Tiered Quality Rating and Improvement Systems in Nine Round 1 States

Contract Information

Current Status:

This study has been completed.

Duration:

December 2012 – March 2019

Cost:

$2,968,330

Contract Number:

ED-IES-10-C-0077

Contractor(s):

Mathematica Policy Research

Contact:

The Race to the Top—Early Learning Challenge Program (RTT-ELC) aimed to improve early learning and development programs so children enter kindergarten ready to succeed. The program awarded $1 billion in four-year grants to 20 states in three rounds between 2011 and 2013. The program promoted reform in five key areas: (1) state systems; (2) high-quality, accountable programs; (3) early learning and development outcomes for children; (4) workforce; and (5) measurement. The second area focused on the design and implementation of Tiered Quality Rating and Improvement Systems (TQRIS) that can provide parents and other stakeholders with information on the quality of early learning programs. To better understand the TQRIS that RTT-ELC grantees developed, this descriptive study: (1) examined the structure and characteristics of grantee states' TQRIS, (2) analyzed state administrative data to examine patterns in the participation and distribution of programs in TQRIS, and (3) synthesized findings from grantee states with completed TQRIS validation studies. This study focused on the nine Round 1 states, which received over $500 million in 2011.

  • How were TQRIS structured and implemented in the nine Round 1 RTT-ELC states?
  • How were TQRIS ratings defined, collected, and generated in the nine Round 1 RTT-ELC states?
  • What were the patterns in TQRIS participation, the distribution of programs and children across the rating levels and the movement of programs and children across the rating levels since 2011?
  • How do TQRIS characteristics (including structure, policies, and practices) relate to program movement up the rating levels and achievement of the highest rating level?
  • What have completed RTT-ELC state validation studies found about the relationship between TQRIS tiers and program quality and the relationship between children's development and TQRIS ratings?

The descriptive study collected and analyzed various data from the nine Round 1 RTT-ELC grantee states. In fall 2014, the study collected and conducted a targeted review of documents describing the structure of TQRIS, including component measures and the quality indicators used to evaluate preschool programs, and how these were combined to generate overall ratings. In fall 2014 through winter 2015, the study also conducted interviews with state administrators to confirm and clarify the information obtained from documents and gather information that could not be obtained from the document reviews.

To address the third and fourth research questions above, the study collected administrative data from the Round 1 RTT-ELC grantee states through mid-2017. The study addressed the last question by conducting a systematic review of TQRIS validation studies completed by seven Round 1 states and two Round 2 states.

Overall, the study found that:

  • There were substantial differences in the ways that states structured and implemented, promoted participation in, and rated and monitored early learning programs in their rating systems (Progress and Challenges in Developing Tiered Quality Rating and Improvement Systems in the Round 1 Race to the Top-Early Learning Challenge States).
  • While most states increased the percentage of programs rated at top TQRIS levels, a goal of RTT-ELC, 68 percent of programs remained at the same rating level during the study period. None of the states in the study could provide data needed to assess whether the number of high-needs children enrolled in top-rated TQRIS programs increased, another RTT-ELC goal (Moving on Up? Program Quality Ratings Under Race to the Top—Early Learning Challenge).
  • States that implemented TQRIS longer and whose rating structures allowed more flexibility for early learning programs to demonstrate quality had a higher percentage of programs at the top rating levels (Quality Ratings and System Characteristics: Patterns in the Round 1 Race to the Top—Early Learning Challenge States).
  • State reports found that programs with higher TQRIS ratings also had higher scores on independent measures of quality. However, children attending programs with higher TQRIS ratings generally did not have better developmental outcomes than those attending programs with lower TQRIS ratings (Are Ratings from Tiered Quality Rating and Improvement Systems Valid Measures of Program Quality?).