Project Activities
This evaluation will use several causal analyses to determine if the TN-VPK program is meeting its intended aims of improved school readiness of students entering kindergarten, improved achievement test scores of students in the elementary grades, a decrease in grade retentions, and a decrease in special education placements.
Structured Abstract
Setting
The evaluation will take place across the state of Tennessee.
Sample
The Tennessee Voluntary Pre-k Program (TN-VPK) serves at-risk students. This evaluation will follow at-risk pre-k students through the end of third grade. The total number of students involved in the entire study will depend on the number of TN-VPK-supported schools with a waiting list of student and a willingness to participate. For the randomized sub-study (described below) approximately 25 sites will take part providing 530 treatment students and 176 control students. For the regression discontinuity study (described below) 128 classrooms will take part providing 1,280 treatment students and 1280 control students.
The Tennessee Voluntary Pre-k Program (TN-VPK) is a statewide program administered by the Office of Early Learning in the Tennessee Department of Education. It began as a pilot program in 1998 and expanded rapidly after 2005 with legislation that increased its funding, which reached $83 million in FY 2008â09. Currently there are 934 pre-k classrooms in 133 of the 136 school districts across Tennessee serving more than 18,000 at-risk students. By statute, the program gives priority to four-year-old students eligible for free and reduced price lunch and, secondarily, to students with disabilities, identified as ELL, or otherwise at-risk. TN-VPK provides competitive grants to local schools systems to implement an approved pre-k program while also meeting standards set by the State Board of Education that require each classroom to have a licensed teacher with a pre-k endorsement, a high adult-student ratio (1:10), a small class size (maximum of 20), an approved age-appropriate curriculum, and a minimum of 5.5 hours a day for a minimum of 180 days.
Initial research
Two interleaved research designs will be used to evaluate the TN-VPK program. The first is a randomized experiment encompassing school systems across Tennessee with more eligible applicants than can be accommodated. A sample of these schools will use a lottery to select the children to be enrolled, leaving a randomized control group that will not be admitted and will have access only to the community resources that would be available if there were no TN-VPK program. Grade retention, special education placement, and third-grade achievement test outcomes will be obtained from the Department of Education data system. An intensive sub-study of this randomized sample will also collect literacy, math, and social-emotional outcome measures annually through the end of third grade. Second, a regression-discontinuity design (RDD) that capitalizes on the legally mandated age cutoff for pre-k eligibility (slightly younger students who must wait a year to enter pre-k will serve as the control group) will be implemented in 128 representatively sampled classrooms. The RDD will include the same literacy, math, and social-emotional outcome measures used in the randomized sub-study. The academic assessments will be given to the treatment students at the start of kindergarten and to the control students at the start of pre-k. For both groups, the pre-k teacher will provide the social-emotional ratings.
Control condition
For the randomization study, students who were not randomly chosen to attend a TN-VPK pre-k site (control students) may remain at home, be placed in child care, or attend another type of preschool depending on their families' decision.
For the regression discontinuity study, the control students are too young to attend the TN-VPK site in the study year but will attend it in the next year. Where they spend the study year will be their families' choice.
Key measures
The randomization study will make use of third-grade student scores on the Tennessee Comprehensive Assessment Program (TCAP) in reading/language arts, math, science and social studies. For the randomization sub-study, students in pre-k through grade 3 will be assessed annually using 5 Woodcock Johnson III subtests: Understanding Directions, Letter-Word Identification, Spelling, Applied Problems, and Quantitative Concepts. Students in grades Kâ3 will receive two additional WJ III subtests: Passage Comprehension and Mathematics Comprehension. Teachers will annually rate each student using the Social Skills scale of the Social Skills Rating System, Problem Behaviors scale of SSRS, Work-Related Skills from Cooper-Farran Behavior Rating Scales, and Adaptive Language Inventory.
For the regression discontinuity study, pre-k students will be assessed using 5 Woodcock Johnson III subtests: Understanding Directions, Letter-Word Identification, Spelling, Applied Problems, and Quantitative Concepts. Their teachers will rate them on the Social Skills scale and Problem Behaviors scale of the Social Skills Rating System. In addition, an outside observer will visit the classroom one time and provide a rating using the Early Childhood Environment Rating Scale-Revised (6 subscales), Early Language and Literacy Classroom Observation, and Classroom Observation of Early Mathematics-Environment and Teaching.
Data analytic strategy
The results from the longitudinal randomized experiment will be analyzed with multilevel statistical models to assess the effects of TN-VPK at successive grade levels (using outcomes of grade retention, special education status, and assessment scores) with particular attention to its impact on third-grade state achievement test scores. Moderator analyses will determine whether differential effects are associated with children's gender, ethnicity, baseline scores, or family characteristics. Results from multilevel analysis of the data from the regression discontinuity design will provide a representative statewide assessment of the effects of TN-VPK classrooms on school readiness at entry to kindergarten, but will be oriented chiefly toward identifying the teacher, classroom, and school variables that are associated with the most positive outcomes.
People and institutions involved
IES program contact(s)
Project contributors
Products and publications
Project website:
Publications:
Journal article, monograph, or newsletter
Farran. D.C. and Lipsey, M.W. (2015). Expectations of Sustained Effects from Scaled up pre-K: Challenges from the Tennessee study. Evidence Speaks Report, 1(3): 1-7.
Farran, D.C. and Lipsey, M. W. (2016). Evidence for the Benefits of State Prekindergarten Programs: Myth and Misrepresentation. Behavioral Science and Policy Association, 2(1): 9-18.
Fuhs, M.W., Farran, D.C., and Nesbitt, K.T. (2013). Preschool Classroom Processes as Predictors of Children's Cognitive Self-Regulation Skills Development. School Psychology Quarterly, 28(4), 347.
Fuhs, M.W., Farran, D.C., and Nesbitt, K.T. (2015). Prekindergarten Children's Executive Functioning Skills and Achievement Gains: The Utility of Direct Assessments and Teacher Ratings. Journal of Educational Psychology, 107(1), 207.
Gordon, R. A., Hofer, K.G., Fujimoto, K.A., Risk, N., Kaestner, R., and Korenman, S. (2015). Identifying High Quality Preschool Programs: New Evidence on the Validity of the ECERS-R in Relation to School Readiness Goals. Early Education and Development, 26(8), 1086-1110.
Lipsey, M.W., Farran, D.C., and Durkin, K. (2018). Effects of the Tennessee Prekindergarten Program on Children's Achievement and Behavior Through Third Grade. Early Childhood Research Quarterly, 45, 155-176.
Spivak, A.L., and Farran, D.C. (2016). Predicting First Graders' Social Competence From Their Preschool Classroom Interpersonal Context. Early Education and Development, 27(6), 735-750.
Swain, W.A., Springer, M.G., and Hofer, K.G. (2015). Early Grade Teacher Effectiveness and pre-K Effect Persistence: Evidence From Tennessee. AERA Open, 1(4): 1-17.
Nongovernment report, issue brief, or practice guide
Lipsey, M.W., Farran, D.C., and Hofer, K.G. (2015). A Randomized Control Trial of a Statewide Voluntary Prekindergarten Program on Children's Skills and Behaviors through Third Grade. Research Report. Nashville, TN: Peabody Research Institute.
Lipsey, M.W., Hofer, K.G., Dong, N., Farran, D.C., and Bilbrey, C. (2013). Evaluation of The Tennessee Voluntary Prekindergarten Program: End of Pre-K Results From the Randomized Control Design. Research Report. Nashville, TN: Vanderbilt University, Peabody Research Institute.
Working paper
Lipsey, M.W., Farran, D.C., and Hofer, K.G. (2016). Effects of a State Prekindergarten Program on Children's Achievement and Behavior through Third Grade. Working Paper. Nashville, TN: Peabody Research Institute.
Watts, T.W., Duncan, G.J., and Rivas, M.A. (2019) A Reanalysis of Impacts of the Tennessee Voluntary Prekindergarten Program. (EdWorkingPaper: 19-28). Retrieved from Annenberg Institute at Brown University: https://doi.org/10.26300/mzk4-jk96
Supplemental information
Co-Principal Investigator: Bobbi Lussier (Tennessee Department of Education) and Dale C. Farran (Vanderbilt University)
Questions about this project?
To answer additional questions about this project or provide feedback, please contact the program officer.
