Skip Navigation
Print Evaluations

Race to the Top—Early Learning Challenge: Descriptive Study of Tiered Quality Rating and Improvement Systems in Nine Round 1 States

Contract Information

Current Status:

Report Preparation is underway.

Duration:

December 2012 – September 2018

Cost:

$2,968,330

Contract Number:

ED-IES-10-C-0077

Contractor(s):

Mathematica Policy Research

Contact:

The Race to the Top–Early Learning Challenge Program (RTT-ELC) aimed to improve early learning and development programs so children enter kindergarten ready to succeed. The program awarded $1 billion in four-year grants to 20 states in three rounds between 2011 and 2013. The program promoted reform in five key areas: (1) state systems; (2) high-quality, accountable programs; (3) early learning and development outcomes for children; (4) workforce; and (5) measurement. The second area focused on the design and implementation of Tiered Quality Rating and Improvement Systems (TQRIS) that can provide parents and other stakeholders with information on the quality of early learning programs. To better understand the TQRIS that RTT-ELC grantees developed, this descriptive study: (1) examined the structure and characteristics of grantee states' TQRIS, (2) analyzed state administrative data to examine patterns in the participation and distribution of programs in TQRIS, and (3) synthesized findings from grantee states with completed TQRIS validation studies. This study focused on the nine Round 1 states, which received over $500 million in 2011.

  • How were TQRIS structured and implemented in the nine Round 1 RTT-ELC states?
  • How were TQRIS ratings defined, collected, and generated in the nine Round 1 RTT-ELC states?
  • What were the patterns in TQRIS participation, the distribution of programs and children across the rating levels and the movement of programs and children across the rating levels since 2011?
  • How do TQRIS characteristics (including structure, policies, and practices) relate to program movement up the rating levels and achievement of the highest rating level?
  • What have completed RTT-ELC state validation studies found about the relationship between TQRIS tiers and program quality and the relationship between children's development and TQRIS ratings?

The descriptive study collected and analyzed various data from the nine Round 1 RTT-ELC grantee states. In fall 2014, the study collected and conducted a targeted review of documents describing the structure of TQRIS, including component measures and the quality indicators used to evaluate preschool programs, and how these were combined to generate overall ratings. In fall 2014 through winter 2015, the study also conducted interviews with state administrators to confirm and clarify the information obtained from documents and gather information that could not be obtained from the document reviews.

To address the third and fourth research questions above, the study collected administrative data from the Round 1 RTT-ELC grantee states through mid-2017. The study will answer the last question by conducting a systematic review of up to nine completed RTT-ELC grantee state validation studies available in 2017.

The first report found that:

  • Eight of the nine states had a statewide TQRIS in place by 2015.
  • States promoted participation in TQRIS by making it mandatory for certain programs (such as public prekindergarten programs or Head Start programs), providing alternative pathways into higher rating levels (so programs could avoid the full data collection and verification process), and offering financial incentives tied to higher ratings.
  • States used different methods for calculating ratings, some of which changed over time.
  • States used various sources of evidence to collect the information needed to rate programs, and they developed processes and standards to help ensure the reliability of this information.

Subsequent reports are anticipated. The key findings will be updated when those reports are released.

The first report addressing the first two research questions, titled Progress and Challenges in Developing Tiered Quality Rating and Improvement Systems (TQRIS) in the Round 1 Race to the Top-Early Learning Challenge (RTT-ELC) States, was released in November 2017.

Three additional reports (to address the third, fourth, and fifth research questions) are expected in 2018 and will be announced on http://ies.ed.gov/ncee/.