The Effects of In-School Virtual Tutoring on Student Reading Development: Evidence from a Short-Cycle Randomized Controlled Trial. EdWorkingPaper No. 24-942
Douglas D. Ready; Sierra G. McCormick; Rebecca J. Shmoys (2024). Annenberg Institute for School Reform at Brown University. Retrieved from: https://eric.ed.gov/?id=ED672510
-
examining1,777Students, grades1-4
Single Study Review
Review Details
Reviewed: May 2026
- Single Study Review (findings for BookNook)
- Randomized Controlled Trial
- Meets WWC standards without reservations because it is a cluster randomized controlled trial with low cluster-level attrition and individual-level non-response.
This review may not reflect the full body of research evidence for this intervention.
Evidence Tier rating based solely on this study. This intervention may achieve a higher tier when combined with the full body of evidence.
Findings
|
Outcome measure |
Comparison | Period | Sample |
Intervention mean |
Comparison mean |
Significant? |
Improvement index |
||
|---|---|---|---|---|---|---|---|---|---|
|
Northwest Evaluation Association (NWEA) Measures of Academic Progress (MAP): Reading |
BookNook vs. Business as usual |
0 Days |
Full sample;
|
N/A |
N/A |
Yes |
|
||
| Show Supplemental Findings | |||||||||
|
Northwest Evaluation Association (NWEA) Measures of Academic Progress (MAP): Reading |
BookNook vs. Business as usual |
0 Days |
English learners;
|
N/A |
N/A |
No |
-- | ||
Evidence Tier rating based solely on this study. This intervention may achieve a higher tier when combined with the full body of evidence.
Sample Characteristics
Characteristics of study sample as reported by study author.
-
51% English language learners -
Female: 48%
Male: 52% -
- B
- A
- C
- D
- E
- F
- G
- I
- H
- J
- K
- L
- P
- M
- N
- O
- Q
- R
- S
- V
- U
- T
- W
- X
- Z
- Y
- a
- h
- i
- b
- d
- e
- f
- c
- g
- j
- k
- l
- m
- n
- o
- p
- q
- r
- s
- t
- u
- v
- x
- w
- y
California
-
Race Asian 9% Black 9% Other or unknown 80% White 2% -
Ethnicity Hispanic 79% Not Hispanic or Latino 21% -
Eligible for Free and Reduced Price Lunch Other or unknown 100%
Study Details
Setting
The study took place in six public charter schools in Northern California. Students were grouped with other students in the same grade, and the school day included a Learning Lab period when BookNook tutoring was scheduled.
Study sample
The study analyzed results for 1,777 students in grades 1–4. Students were assigned within 77 same-grade student groups formed within schools; 42 groups with 959 students were assigned to the intervention group and 35 groups with 818 students were assigned to the comparison group. The study included students who had the demographic information and reading test scores needed for the analysis. About 9.5% of students received special education services.
Intervention Group
BookNook is a supplemental tutoring program designed to support students who need extra help in reading, although in this study it was offered to all students assigned to the intervention group. It uses a virtual platform and structured literacy lessons to build skills such as phonological awareness, phonics, fluency, vocabulary, and comprehension. In this study, students in the intervention group received 30-minute BookNook tutoring sessions during Learning Lab two to three times per week. The implementation lasted about 12 weeks, including two weeks of school breaks, which meant students had about 10 weeks of instruction. Many students did not receive the full intended amount of tutoring, defined in the study as at least 20 sessions: about 20% completed 20 or more sessions, about 45% completed 10 to 19 sessions, and about 34% completed fewer than 10 sessions.
Comparison Group
Students in the comparison group continued to receive the regular reading supports and activities that the charter schools usually provided during Learning Lab. The study did not give more detail about the content, staff, or structure of these regular supports. The authors reported that comparison students were not given access to BookNook during the implementation period.
Support for implementation
BookNook provided the virtual tutors, and the manuscript reported that most tutors had prior K–12 teaching experience, at least 3 years of tutoring experience, or were enrolled in a teaching certification program. The study did not provide additional detail about formal training or implementation support for school staff beyond describing the qualifications of the BookNook tutors.
Additional Sources
In the case of multiple manuscripts that report on one study, the WWC selects one manuscript as the primary citation and lists other manuscripts that describe the study as additional sources.
-
Ready, D. (2023). BookNook in Rocketship Schools: Final Impact Results. Consortium for Policy Research in Education, Teachers College, Columbia University.
An indicator of the effect of the intervention, the improvement index can be interpreted as the expected change in percentile rank for an average comparison group student if that student had received the intervention.
For more, please see the WWC Glossary entry for improvement index.
An outcome is the knowledge, skills, and attitudes that are attained as a result of an activity. An outcome measures is an instrument, device, or method that provides data on the outcome.
A finding that is included in the effectiveness rating. Excluded findings may include subgroups and subscales.
The sample on which the analysis was conducted.
The group to which the intervention group is compared, which may include a different intervention, business as usual, or no services.
The timing of the post-intervention outcome measure.
The number of students included in the analysis.
The mean score of students in the intervention group.
The mean score of students in the comparison group.
The WWC considers a finding to be statistically significant if the likelihood that the finding is due to chance alone, rather than a real difference, is less than five percent.
The WWC reviews studies for WWC products, Department of Education grant competitions, and IES performance measures.
The name and version of the document used to guide the review of the study.
The version of the WWC design standards used to guide the review of the study.
The result of the WWC assessment of the study. The rating is based on the strength of evidence of the effectiveness of the intervention. Studies are given a rating of Meets WWC Design Standards without Reservations, Meets WWC Design Standards with Reservations, or >Does Not Meet WWC Design Standards.
A related publication that was reviewed alongside the main study of interest.
Study findings for this report.
Based on the direction, magnitude, statistical significance, and sample size of the findings within a domain, the WWC assigns effectiveness ratings as one of the following: Tier 1 (strong evidence), Tier 2 (moderate evidence), Tier 3 (promising evidence), uncertain effects, and negative effects. For more detail, please see the WWC Handbook.
The WWC may review studies for multiple purposes, including different reports and re-reviews using updated standards. Each WWC review of this study is listed in the dropdown. Details on any review may be accessed by making a selection from the drop down list.
Tier 1 Strong indicates strong evidence of effectiveness,
Tier 2 Moderate indicates moderate evidence of effectiveness, and
Tier 3 Promising indicates promising evidence of effectiveness,
as defined in the
non-regulatory guidance for ESSA
and the regulations for ED discretionary grants (EDGAR Part 77).