Co-Principal Investigators: Williford, Amanda; Hamre, Bridget; Gibbs, Chloe; Pianta, Robert C.

Purpose: The purpose of this project is to validate a new outcomes-based approach to Quality Rating and Improvement Systems (QRIS) and explore assumptions about the role of parental selection in early childhood programs. High quality early childhood education (ECE) programs can narrow school-readiness gaps, alter children’s life trajectories and yield substantial social returns. However, there is substantial variation in quality, both across and within early childhood sectors (e.g., licensed childcare centers, Head Start, state pre-kindergarten programs). QRIS have emerged as a popular policy strategy to improve quality through standards, accountability, and supports. Although there is strong federal and state commitment to QRIS as an improvement strategy in ECE, the rapid roll-out of states’ QRIS systems has outpaced the research base around measuring quality in early childhood settings. In this project, the researchers will work in partnership with Louisiana Department of Education (LDE) to examine QRIS as a malleable factor that has the potential to affect the ECE landscape, improve the quality of early learning environments, and lead to improvements in children’s school readiness skills.

Project Activities: Over a three-year period, the researchers will collect primary data and use state-provided secondary data to examine associations between quality and child outcomes and parental selection of early childhood programming for their children. The researchers will   validate the two key components of Louisiana's QRIS, the Kindergarten Readiness System, by investigating associations with gains on direct child assessments and exploring optimal strategies for building composite measures of quality. In addition, they will link parent-reported ECE preferences and perceptions of quality to measures of observed classroom quality to examine the extent to which parents are able to distinguish higher quality programs. The team will answer two research questions: (1) what are the best ways to combine data collected through classroom observations and teacher-conducted child assessments to create a validated rating system linked to children’s learning gains? and (2) how do families in Louisiana currently identify and select early childhood settings for their four-year-old children? This study will provide the state with recommendations about how best to combine the measures they are piloting into a rating system with predictive validity, and will also guide their development of “single point of entry” systems to provide parents with information about the availability and quality of ECE options.

Products: The products of this project will be preliminary evidence about the relationship between QRIS measures of classroom quality and children’s school readiness skills, and the role that QRIS plays in influencing parents’ choice of ECE programs for their children.  Peer reviewed publications will also be produced.

Structured Abstract

Setting: This study will take place in publicly-funded preschool programs across the state of Louisiana.

Sample: For the primary data collection, the study sample includes 90 ECE classrooms, child care centers, Head Start and state-funded pre-kindergarten programs that began piloting key components of Louisiana’s Kindergarten Readiness System in 2013. Study participants will include 1,080 four-year old children (12 children per classroom) and 1,620 parents. Approximately 62 percent of children in the proposed sample qualify for free or reduced price lunch. The sample is about 52 percent White and 44 percent Black.

Intervention: The Louisiana Department of Education is piloting and scaling up its Kindergarten Readiness System, a novel QRIS focused on (1) measures of children’s learning and (2) the quality of teacher-child interactions. All publicly-funded ECE programs in Louisiana are required to use Teaching Strategies GOLD to assess children’s school readiness skills.  In addition, programs are expected to use the Classroom Assessment Scoring System to measure teacher-child interactions.

Research Design and Methods: The researchers will explore the predictive validity of the key components of Louisiana’s Kindergarten Readiness System by linking program data collected in Louisiana with primary data that will be collected in the initial year of the project. Specifically, the researchers will examine the extent to which “internally-conducted” CLASS ratings align both with scores from externally conducted CLASS observations and with children’s gains in school readiness skills. They will also explore the extent to which parents can accurately assess program quality as defined by QRIS, as well as the extent to which parents define quality in ways that are distinct from QRIS. In Year 1, the researchers will conduct direct child assessments, CLASS observations, and administer direct, teacher and parent surveys in fall 2014 and spring 2015. LDE staff will provide the University of Virginia (UVA) research team with access to state-collected Teaching Strategies GOLD and CLASS data for the 2013-2014 school year. The UVA research team will develop data systems, prepare data for analysis and conduct preliminary analyses. In Year 2, LDE staff will provide the UVA research team with access to 2014-2015 Teaching Strategies GOLD and CLASS data collected by the LDE teachers and staff. Preliminary validation and parental selection data analyses will be conducted.  In Year 3, LDE staff will provide the University of Virginia (UVA) research team with access to state-collected Teaching Strategies GOLD and CLASSdata for the 2015-2016 school year. The researchers will compile three years of state-collected data and conduct a descriptive analysis of quality distribution across the publicly-funded ECE programs in the state. They will also complete validation and parental choice analyses.

Control Condition: Due to the nature of this research design, there is no control condition.

Key Measures:

LDE-provided Secondary Data: The LDE will provide the researchers with access to secondary data collected by LDE staff and teachers.  The LDE mandates the use of the Classroom Assessment Scoring System (CLASS) and Teaching Strategies GOLD in preschool programs that participate in Louisiana’ Kindergarten Readiness System.  Teachers in participating programs use Teaching Strategies GOLD three times throughout the year (fall, winter, and spring).  CLASSobservations are conducted in fall and spring of each year

Primary Data Collection: For the primary data collection study, the researchers will conduct direct assessments of children’s school readiness skills in the areas of receptive and expressive vocabulary, phonological awareness, print knowledge, math and self-regulation in fall 2014 and spring 2015. They will also conduct independent observations of the quality of teacher-child interactions. The Peabody Picture Vocabulary Test-3rd edition (PPTV-III) will be used to measure children’s receptive vocabulary skills. The Picture Vocabulary Subtest of the Woodcock-Johnson-III (WJ-III) will be used to assess children’s expressive vocabulary skills. The Phonological Awareness and Print Knowledge subtests of the Test of Preschool Early Literacy will be used to assess children’s emergent literacy skills. The WJ-III Applied Problems subtest and the Tools for Early Assessment in Mathematics-Short Form will be used to assess children’s early mathematics skills. Inhibitory control will be assessed using the Pencil Tap Test. Children’s behavioral self-regulation, including inhibitory control, working memory and attention focusing will be assessed using the Head Toes Knees Shoulders Task. Data collectors will conduct CLASS observations in the fall and spring. Data will be collected to examine the fidelity of implementation (e.g., teachers’ “buy-in” and use of the measure as a formative assessment tool) of Teaching Strategies GOLD. In fall 2014, program directors will complete a survey providing information about the children enrolled in their program and the services provided. Directors will indicate the program’s type (private child care center, Head Start, school-based prekindergarten) as well as the number of children enrolled, the number of classrooms and number of teachers and aides. Participating teachers and parents will complete surveys in the fall 2014 and spring 2015.

Data Analytic Strategy: Researchers will conduct analyses to explore associations between various proxies of quality and direct assessments of children’s learning, and to explore parental processes in selecting ECE programs. This information will be used to generate testable hypotheses about the effective design of QRIS. The quality proxy variables include: 1) Average CLASS score as measured internally by Community Networks; (2) Developmental gains measured using Teaching Strategies GOLD; (3) Average CLASS score as measured by reliable assessors from our research team; (4) Fidelity of implementation measures of Teaching Strategies GOLD; and (5) Parental ratings of program quality and engagement. The main analytic tool will be multivariate linear regression analysis. To answer the primary research question, the researchers will examine the relationships between each quality measure and gains over the program year as measured by direct child assessments. In addition, multivariate analyses will be conducted to examine which composites of quality proxies are most strongly related to children’s learning gains. To answer the second research question, the researchers will describe parents’ survey responses and explore whether ECE selection processes differ statistically depending on family characteristics such as race, income, and parental education, as well as community characteristics such as urbanicity.