Co-Principal Investigator: Doss, Christopher

Purpose: In this project, the team  aims to replicate an effectiveness study of TipsByText (TbT), a text messaging intervention for parents of prekindergartners that serves as a tool schools and districts can use to improve the home learning environment, resulting in improved school readiness and better education outcomes in kindergarten (K) and first grade. TbT is designed to address barriers to behavior change using behavioral economics theories so that parents can support the literacy, math, and social-emotional (SEL) development of their children in the home. Previous studies have shown its effectiveness in urban areas, and this study will investigate whether equal, or potentially larger, effects can be seen in rural areas. Understanding the effectiveness of TbT in rural areas is particularly valuable because rural students enter K with lower levels of school readiness than their non-rural peers, and students who enter K behind are unlikely to catch up. TbT is uniquely suited to rural communities as text-messaging is a ubiquitous platform and the program minimizes time and material costs by leveraging everyday routines and household materials. The University of New Mexico and RAND will lead the evaluation, New Mexico Regional Education Cooperatives are education agencies that will manage TbT, and Brown University will deliver the intervention.

Project Activities: The researchers will randomly assign students of prekindergarten (PreK) age to receive either TbT or control texts. Data collection activities include baseline parent surveys of the home environment and parent perceptions of child skills, monthly parent surveys on implementation, follow-up parent surveys of the home environment and parent perceptions of child skills, child assessment of literacy, math, and SEL skills with the BRIGANCE, and parent focus groups and interviews. Secondary data sources will include K and first grade state-mandated assessments on child literacy and math, grade repetition, special education program participation, and attendance from New Mexico (NM) Public Education Department (PED). Brown University will provide texting platform administrative data. Impact analysis will include effects on mediators and outcomes. Implementation analysis will capture parent interactions with TbT and the facilitators and barriers to its success. The researchers will also estimate costs and cost-effectiveness.

Products: The expected products of the project include a version of TbT that is adapted to the rural context, peer-reviewed journal articles on implementation analysis, impact analysis, and cost analysis articles, and a research brief for a non-researcher audience. In addition, the research team will execute a strategic dissemination plan that includes e-newsletter outreach to professional organizations, media placement, targeted policymaker delivery, presentations at policymaker, researcher, and practitioner conferences, op-eds and articles for general interest, and industry-based publications.

Structured Abstract

Setting: The study will take place in 60 rural elementary schools from 34 districts in New Mexico.

Sample: Approximately 2,230 children of PreK age (~4 years old) from the catchment area of 60 rural elementary schools in NM and their parents will participate in the study. The majority will be students of color (Hispanic, Native American, other non-white race) and economically-disadvantaged (69% of students qualify for free or reduced-price lunch).

Intervention: TbT sends parents three texts each week focused on a particular literacy, math, or SEL skill. The texting period will last for 10 months, or approximately the PreK year before students enter K. This systematic effectiveness replication will be executed under routine conditions that do not deviate from ongoing public-school practices and collaborates with the existing education resources and infrastructure in NM.

Research Design and Methods: This randomized control trial is designed to meet What Works Clearinghouse (WWC) Standards without reservations. The researchers will employ a blocked randomization design where a block is defined by elementary school catchment area and preschool attendance. Impact analyses will examine the effects of the program on mediators and parent and child outcomes. Implementation analyses include qualitative data analysis of parents focus groups, interviews with parents, text developers, and an advisory panel, and parent surveys. Finally, a cost analysis will estimate the costs and cost-effectiveness.

Control Condition: The control texts in this study will provide information on deadlines and requirements for K (e.g., vaccinations) but no academic related content.

Key Measures: The BRIGANCE, which measures literacy, math and SEL skills, will be used as the key measure of K readiness and will be administered at K entry. The research team will also obtain the following outcomes at the student level in K and first grade from NM PED administrative data: the Kindergarten Observation Tool (state K entry assessment), iStation reading and math formative assessments for K and first grade (administered 3 times per year), grade repetition, special education services, attendance, and demographics. Baseline measures will also be obtained of home learning environment and student literacy, math and SEL (using the PEDS-DM) at enrollment. Measures of parents' activities that promote learning come from items in the original TbT study parent survey and items from the Early Childhood Longitudinal Study (ECLS) home environment module. Parent self-efficacy related to supporting student learning will be assessed with the Parent Reading Belief Inventory. Through parent surveys during the course of the intervention and immediately following the intervention, data for the implementation analysis and cost analysis will be obtained. These data will be supplemented by administrative data from the texting platform on text delivery (all participants), parent focus groups (64 parents), and interviews of a variety of other stakeholders (up to 64 participants).

Data Analytic Strategy: The research team will assess whether the treatment group has received TbT with fidelity using data from the parent surveys, texting platform administrative data, and parent focus groups and interviews. The comparison of treatment and control outcomes includes moderation and mediation analysis. There are specific plans for comparing results of this replication to the results from the original study, specifying the generalizability of the findings, and expressing the findings in ways that are meaningful to multiple audiences.

Cost Analysis: The researcher will estimate the total costs of TbT at the district level and also at the student level. They will also estimate cost-effectiveness in the form of student-level cost per incremental gain in measures of K readiness. The team will compare the  latter to cost-effectiveness estimates for other early childhood interventions that aim to promote K readiness.