WWC review of this study

Texting Parents: Evaluation Report and Executive Summary

Miller, Sarah; Davison, Jenny; Yohanis, Jamie; Sloan, Seaneen; Gildea, Aideen; Thurston, Allen (2017). Education Endowment Foundation. Retrieved from: https://eric.ed.gov/?id=ED581121

  • Randomized Controlled Trial
     examining 
    15,355
     Students
    , grades
    6-11

Reviewed: March 2017

At least one finding shows strong evidence of effectiveness
At least one statistically significant positive finding
Meets WWC standards without reservations
General Mathematics Achievement outcomes—Statistically significant positive effect found for the domain
Outcome
measure
Comparison Period Sample Intervention
mean
Comparison
mean
Significant? Improvement
    index
Evidence
tier

Combination of standardized scores on Hodder Access Maths Test and General Certificate of Secondary Education Maths test

Parent Engagement Project (PEP) vs. Business as usual

0 Days

Full sample;
11,587 students

0.05

0.02

Yes

 
 
1
 
More Outcomes
Show Supplemental Findings

Combination of standardized scores on Hodder Access Maths Test and General Certificate of Secondary Education Maths test

Parent Engagement Project (PEP) vs. Business as usual

0 Days

Low achievement;
1,902 students

-1.16

-1.27

Yes

 
 
7

Combination of standardized scores on Hodder Access Maths Test and General Certificate of Secondary Education Maths test

Parent Engagement Project (PEP) vs. Business as usual

0 Days

Small schools;
2,387 students

-0.01

-0.15

Yes

 
 
6

Combination of standardized scores on Hodder Access Maths Test and General Certificate of Secondary Education Maths test

Parent Engagement Project (PEP) vs. Business as usual

0 Days

Female;
5,233 students

0.04

-0.06

Yes

 
 
4

Combination of standardized scores on Hodder Access Maths Test and General Certificate of Secondary Education Maths test

Parent Engagement Project (PEP) vs. Business as usual

0 Days

Key stage 3 (years 7 and 9);
8,736 students

0.08

0.00

Yes

 
 
3

Combination of standardized scores on Hodder Access Maths Test and General Certificate of Secondary Education Maths test

Parent Engagement Project (PEP) vs. Business as usual

0 Days

Non-English learners;
9,942 students

0.12

0.04

Yes

 
 
3

Combination of standardized scores on Hodder Access Maths Test and General Certificate of Secondary Education Maths test

Parent Engagement Project (PEP) vs. Business as usual

0 Days

High achievement;
1,998 students

0.98

0.95

No

--

Combination of standardized scores on Hodder Access Maths Test and General Certificate of Secondary Education Maths test

Parent Engagement Project (PEP) vs. Business as usual

0 Days

Male;
6,354 students

0.13

0.09

No

--

Combination of standardized scores on Hodder Access Maths Test and General Certificate of Secondary Education Maths test

Parent Engagement Project (PEP) vs. Business as usual

0 Days

Free school meals;
1,683 students

-0.27

-0.28

No

--

Combination of standardized scores on Hodder Access Maths Test and General Certificate of Secondary Education Maths test

Parent Engagement Project (PEP) vs. Business as usual

0 Days

Key stage 4 (year 11);
2,851 students

0.11

0.11

No

--

Combination of standardized scores on Hodder Access Maths Test and General Certificate of Secondary Education Maths test

Parent Engagement Project (PEP) vs. Business as usual

0 Days

English learners;
1,642 students

-0.17

-0.05

Yes

-5
 
 

Combination of standardized scores on Hodder Access Maths Test and General Certificate of Secondary Education Maths test

Parent Engagement Project (PEP) vs. Business as usual

0 Days

Large schools;
671 students

0.18

0.32

No

--
Literacy Achievement outcomes—Indeterminate effect found for the domain
Outcome
measure
Comparison Period Sample Intervention
mean
Comparison
mean
Significant? Improvement
    index
Evidence
tier

Combination of standardized scores on Hodder Access Reading Test and General Certificate of Secondary Education English test

Parent Engagement Project (PEP) vs. Business as usual

0 Days

Full sample;
11,413 students

0.05

0.06

No

--
More Outcomes
Show Supplemental Findings

Combination of standardized scores on Hodder Access Reading Test and General Certificate of Secondary Education English test

Parent Engagement Project (PEP) vs. Business as usual

0 Days

Low achievement;
1,975 students

-0.91

-0.97

Yes

 
 
3

Combination of standardized scores on Hodder Access Reading Test and General Certificate of Secondary Education English test

Parent Engagement Project (PEP) vs. Business as usual

0 Days

Key stage 3 (years 7 and 9);
8,485 students

0.08

0.03

No

--

Combination of standardized scores on Hodder Access Reading Test and General Certificate of Secondary Education English test

Parent Engagement Project (PEP) vs. Business as usual

0 Days

Female;
5,181 students

0.14

0.08

No

--

Combination of standardized scores on Hodder Access Reading Test and General Certificate of Secondary Education English test

Parent Engagement Project (PEP) vs. Business as usual

0 Days

Large schools;
664 students

0.26

0.22

Yes

 
 
2

Combination of standardized scores on Hodder Access Reading Test and General Certificate of Secondary Education English test

Parent Engagement Project (PEP) vs. Business as usual

0 Days

Non-English learners;
9,812 students

0.15

0.11

No

--

Combination of standardized scores on Hodder Access Reading Test and General Certificate of Secondary Education English test

Parent Engagement Project (PEP) vs. Business as usual

0 Days

Male;
6,232 students

0.01

0.01

No

--

Combination of standardized scores on Hodder Access Reading Test and General Certificate of Secondary Education English test

Parent Engagement Project (PEP) vs. Business as usual

0 Days

Small schools;
2,299 students

-0.05

-0.03

No

--

Combination of standardized scores on Hodder Access Reading Test and General Certificate of Secondary Education English test

Parent Engagement Project (PEP) vs. Business as usual

0 Days

Key stage 4 (year 11);
2,928 students

0.12

0.17

No

--

Combination of standardized scores on Hodder Access Reading Test and General Certificate of Secondary Education English test

Parent Engagement Project (PEP) vs. Business as usual

0 Days

High achievement;
1,964 students

0.87

0.91

No

--

Combination of standardized scores on Hodder Access Reading Test and General Certificate of Secondary Education English test

Parent Engagement Project (PEP) vs. Business as usual

0 Days

Free school meals;
1,747 students

-0.29

-0.18

No

--

Combination of standardized scores on Hodder Access Reading Test and General Certificate of Secondary Education English test

Parent Engagement Project (PEP) vs. Business as usual

0 Days

English learners;
1,598 students

-0.31

-0.21

No

--
Staying in school outcomes—Statistically significant positive effect found for the domain
Outcome
measure
Comparison Period Sample Intervention
mean
Comparison
mean
Significant? Improvement
    index
Evidence
tier

Number of days present (standardized)

Parent Engagement Project (PEP) vs. Business as usual

0 Days

Full sample;
15,350 students

0.04

-0.01

Yes

 
 
2
 
More Outcomes
Show Supplemental Findings

Number of days present (standardized)

Parent Engagement Project (PEP) vs. Business as usual

0 Days

Key stage 4 (year 11);
5,229 students

-0.05

-0.17

Yes

 
 
4

Number of days present (standardized)

Parent Engagement Project (PEP) vs. Business as usual

0 Days

Female;
6,906 students

-0.02

-0.04

No

--

Number of days present (standardized)

Parent Engagement Project (PEP) vs. Business as usual

0 Days

Non-English learners;
12,702 students

0.00

-0.03

No

--

Number of days present (standardized)

Parent Engagement Project (PEP) vs. Business as usual

0 Days

Male;
8,444 students

0.02

0.01

No

--

Number of days present (standardized)

Parent Engagement Project (PEP) vs. Business as usual

0 Days

High achievement;
4,922 students

0.03

0.00

No

--

Number of days present (standardized)

Parent Engagement Project (PEP) vs. Business as usual

0 Days

Free school meals;
2,469 students

-0.42

-0.37

No

--

Number of days present (standardized)

Parent Engagement Project (PEP) vs. Business as usual

0 Days

Key stage 3 (years 7 and 9);
10,121 students

0.03

0.05

No

--

Number of days present (standardized)

Parent Engagement Project (PEP) vs. Business as usual

0 Days

English learners;
2,645 students

0.03

0.08

No

--


Evidence Tier rating based solely on this study. This intervention may achieve a higher tier when combined with the full body of evidence.

Characteristics of study sample as reported by study author.


  • 17% English language learners

  • Female: 45%
    Male: 55%
    • B
    • A
    • C
    • D
    • E
    • F
    • G
    • I
    • H
    • J
    • K
    • L
    • P
    • M
    • N
    • O
    • Q
    • R
    • S
    • V
    • U
    • T
    • W
    • X
    • Z
    • Y
    • a
    • h
    • i
    • b
    • d
    • e
    • f
    • c
    • g
    • j
    • k
    • l
    • m
    • n
    • o
    • p
    • q
    • r
    • s
    • t
    • u
    • x
    • w
    • y

    International
  • Race
    Other or unknown
    30%
    White
    70%

Setting

The study was conducted in 36 geographically dispersed secondary schools in England. Each school contained students from Key Stages (KS) 3 and 4 of the English educational system. Within participating schools, students in Years 7, 9, and 11 were eligible for inclusion in the study. KS3 includes Years 7 and 9, and students in these years are typically between 11 and 14 years old. KS4 includes Year 11, and students in Year 11 are typically 15 to 16 years old.

Study sample

The analytic sample included 29 schools and 58 KS groups (29 KS groups in the intervention condition and 29 KS groups in the comparison condition). For the absenteeism outcome, the analytic sample included 7,436 students in the intervention group and 7,919 in the comparison group. For the general mathematics achievement outcome, the analytic sample included 5,613 students in the intervention group and 5,977 students in the comparison group. For the general literacy achievement outcome, the analytic sample included 5,376 students in the intervention group and 6,037 students in the comparison group. In both the intervention and comparison groups, 55% of students were male. 14% of students in the intervention group and 18% of students in the comparison group were eligible for free school meals. 17% of students in the intervention group and 19% of students in the comparison group were English as an Additional Language (EAL) status. 18% of students in the intervention group and 20% of students in the comparison group were characterized as having Special Educational Needs (SEN). 73% of students in the intervention group and 67% of students in the comparison group were "White British." 57% of students in the intervention group and 77% of students in the comparison group were in KS3.

Intervention Group

The intervention is the Parent Engagement Project (PEP), an intervention designed to improve student outcomes through increased parent engagement by text message. It is a year-long, school-level intervention. The intervention consists of text messages sent directly by schools to parents at specific times and intervals during the school year. The types of text messages sent included advance notice of tests and important deadlines (4 days and 1 day in advance); notice of missing homework; and summaries of daily lessons, with conversational prompts (rotated weekly across the subjects of math, reading, and science). The text messages were designed to be sent by schools via their existing school information management systems. The intervention was intended to send approximately 65 texts to parents throughout the year; the average number texts actually sent to each parent was 30, with a range from 15 to 77. An average of 22.19 texts were sent about upcoming assessments, 11.39 texts were send regarding missing homework, and 5.08 texts were sent with conversational prompts.

Comparison Group

The comparison condition was "business as usual." Parents received standard school-level communication via existing school information management systems, email, telephone calls, letters, and student homework diaries/planners. The comparison condition did not receive any text messaging.

Support for implementation

The intervention is intended to be used with existing school information management systems. Implementing schools and teachers received an initial training for a full day in September 2014 and ongoing support throughout the evaluation. An additional half-day of training was held in January 2015. Detailed instructions were provided on the content of the text messages. Each school appointed one staff member to be the project liaison officer, who was responsible for coordinating the text messages within the school. The study team also employed research assistants who were responsible for overseeing the implementation of PEP across schools. Each research assistant worked with a group of project liaison officers to facilitate consistent implementation across schools, including obtaining test dates and conversation prompts. More than 70% of school personnel reported positive reactions to the training received and being confident about implementing the intervention. Video recordings of the initial training were made available to all schools.

 

Your export should download shortly as a zip archive.

This download will include data files for study and findings review data and a data dictionary.

Connect With the WWC

loading
back to top