When Less May Be More: A 2-Year Longitudinal Evaluation of a Volunteer Tutoring Program Requiring Minimal Training.
Baker, Scott; Gersten, Russell; Keating, Thomas (2000). Reading Research Quarterly, v35 n4 p494-519. Retrieved from: https://eric.ed.gov/?id=EJ616173
-
examining84Students, grades1-2
Grant Competition
Review Details
Reviewed: October 2016
- Grant Competition (findings for Start Making a Reader Today (SMART))
- Randomized Controlled Trial
- Meets WWC standards without reservations because it is a randomized controlled trial with low attrition.
This review may not reflect the full body of research evidence for this intervention.
Evidence Tier rating based solely on this study. This intervention may achieve a higher tier when combined with the full body of evidence.
Findings
|
Outcome measure |
Comparison | Period | Sample |
Intervention mean |
Comparison mean |
Significant? |
Improvement index |
Evidence tier |
|---|---|---|---|---|---|---|---|---|
|
Woodcock Reading Mastery Test- Revised (WRMT-R): Word Identification subtest |
Start Making a Reader Today (SMART) vs. Business as usual |
2 Years |
Full sample;
|
449.40 |
437.90 |
No |
-- |
|
Outcome measure |
Comparison | Period | Sample |
Intervention mean |
Comparison mean |
Significant? |
Improvement index |
Evidence tier |
|---|---|---|---|---|---|---|---|---|
|
Woodcock Reading Mastery Test-Revised (WRMT-R): Word Comprehension subtest |
Start Making a Reader Today (SMART) vs. Business as usual |
2 Years |
Full sample;
|
472.30 |
465.40 |
No |
-- | |
|
Woodcock Reading Mastery Test-Revised (WRMT-R): Passage Comprehension subtest |
Start Making a Reader Today (SMART) vs. Business as usual |
2 Years |
Full sample;
|
468.90 |
464.70 |
No |
-- |
|
Outcome measure |
Comparison | Period | Sample |
Intervention mean |
Comparison mean |
Significant? |
Improvement index |
Evidence tier |
|---|---|---|---|---|---|---|---|---|
|
Oral Reading Fluency: Second-Grade Passage |
Start Making a Reader Today (SMART) vs. Business as usual |
2 Years |
Full sample;
|
61.50 |
45.90 |
No |
-- | |
|
Oral Reading Fluency: First-Grade Passage |
Start Making a Reader Today (SMART) vs. Business as usual |
2 Years |
Full sample;
|
71.30 |
55.90 |
No |
-- |
Evidence Tier rating based solely on this study. This intervention may achieve a higher tier when combined with the full body of evidence.
Sample Characteristics
Characteristics of study sample as reported by study author.
-
Female: 52%
Male: 48% -
Rural, Suburban, Urban
-
- B
- A
- C
- D
- E
- F
- G
- I
- H
- J
- K
- L
- P
- M
- N
- O
- Q
- R
- S
- V
- U
- T
- W
- X
- Z
- Y
- a
- h
- i
- b
- d
- e
- f
- c
- g
- j
- k
- l
- m
- n
- o
- p
- q
- r
- s
- t
- u
- v
- x
- w
- y
Oregon
-
Race Asian 6% Black 30% Native American 10% White 47% -
Ethnicity Hispanic 6% Not Hispanic or Latino 94%
Study Details
Setting
The study was set in six schools in four Oregon school districts, and 24 classrooms. All of the schools were participating in the SMART program for the first time, therefore, none of the 1st graders in the study had tutoring in kindergarten. The participating schools were Title I schools in two of the largest counties in Oregon.
Study sample
The schools were all Title I schools and represented a mix of large city, moderate-sized city, and rural schools in Oregon. Students were European American (47%), African American (30%), American Indian (10%), Asian American (6%), and Latino (6%). There were 44 females and 40 males in the analytic sample (Total = 84).
Intervention Group
In addition to regular reading instruction during the day, SMART intervention group students received tutoring. The Start Making a Reader Today (SMART) program is a two-year volunteer tutoring program, in which low-performing readers receive one-hour of tutoring from an adult volunteer per week for six months each year in the first and the second grade. The number of 30-minute tutoring sessions ranged from 49-98 per student (mean of 73, SD of 10.9). Volunteers are asked to read to the child, to read together with the child, to have the child reread a section just read by the volunteer, and to ask questions of the child based on the reading. Volunteer training is 1-2 hours, and follow a Volunteer Handbook to guide the tutoring process. The tutors were adults from the local business community, and were older than 30 years (33% aged 30-45; 29% aged 45-65; and 20% over 65 years). The authors described that a half-time SMART coordinator at each school managed the program and described that they were typically AmeriCorps volunteers or instructional assistants with no formal training in reading instruction or elementary education, and training lasted about 1 full day per year for them.
Comparison Group
The comparison condition received all usual services offered by the school. The only noted difference for SMART students was the 1 hour of tutoring per week.
Support for implementation
Volunteer training is 1-2 hours, and follow a Volunteer Handbook to guide the tutoring process. The training occurs in impromptu sessions and consists of instructional modeling by the coordinator. The authors described that a half-time SMART coordinator at each school managed the program and described that they were typically AmeriCorps volunteers or instructional assistants with no formal training in reading instruction or elementary education, and training lasted about 1 full day per year for them. The volunteer tutors work through the volunteer handbook and use four primary reading strategies: 1) reading to the child, 2) reading with the child, 3) volunteer reads a section and the child re-reads, and 4) the volunteer asks questions during reading tutoring.
Structuring Out-of-School Time to Improve Academic Achievement
Review Details
Reviewed: July 2009
Structuring Out-of-School Time to Improve Academic Achievement Practice Guide- Randomized Controlled Trial
- Meets WWC standards with reservations
This review may not reflect the full body of research evidence for this intervention.
Evidence Tier rating based solely on this study. This intervention may achieve a higher tier when combined with the full body of evidence.
Findings
Evidence Tier rating based solely on this study. This intervention may achieve a higher tier when combined with the full body of evidence.
Sample Characteristics
Characteristics of study sample as reported by study author.
-
Female: 52%
Male: 48% -
Race Asian 6% Black 30% Native American 10% White 47% -
Ethnicity Hispanic 6% Not Hispanic or Latino 94%
Start Making a Reader Today (SMART) Intervention Report - Beginning Reading
Review Details
Reviewed: June 2007
- Randomized Controlled Trial
- Meets WWC standards without reservations
This review may not reflect the full body of research evidence for this intervention.
Evidence Tier rating based solely on this study. This intervention may achieve a higher tier when combined with the full body of evidence.
Please see the WWC summary of evidence for Start Making a Reader Today (SMART).
Findings
|
Outcome measure |
Comparison | Period | Sample |
Intervention mean |
Comparison mean |
Significant? |
Improvement index |
Evidence tier |
|---|---|---|---|---|---|---|---|---|
|
Woodcock Reading Mastery Tests- Revised (WRMT-R): Word Identification subtest |
Start Making a Reader Today (SMART) vs. business as usual |
posttest |
Grade 1;
|
449.40 |
437.90 |
No |
-- |
|
Outcome measure |
Comparison | Period | Sample |
Intervention mean |
Comparison mean |
Significant? |
Improvement index |
Evidence tier |
|---|---|---|---|---|---|---|---|---|
|
Woodcock Reading Mastery Test-Revised (WRMT-R): Word Comprehension subtest |
Start Making a Reader Today (SMART) vs. business as usual |
posttest |
Grade 1;
|
472.30 |
456.40 |
No |
-- | |
|
Woodcock Reading Mastery Test-Revised (WRMT-R): Passage Comprehension subtest |
Start Making a Reader Today (SMART) vs. business as usual |
posttest |
Grade 1;
|
468.90 |
464.70 |
No |
-- |
|
Outcome measure |
Comparison | Period | Sample |
Intervention mean |
Comparison mean |
Significant? |
Improvement index |
Evidence tier |
|---|---|---|---|---|---|---|---|---|
|
Oral Reading Fluency: Second-Grade Passage |
Start Making a Reader Today (SMART) vs. business as usual |
posttest |
Grade 1;
|
61.50 |
45.90 |
No |
-- | |
|
Oral Reading Fluency: First-Grade Passage |
Start Making a Reader Today (SMART) vs. business as usual |
posttest |
Grade 1;
|
71.30 |
55.90 |
No |
-- |
Evidence Tier rating based solely on this study. This intervention may achieve a higher tier when combined with the full body of evidence.
Sample Characteristics
Characteristics of study sample as reported by study author.
-
Female: 52%
Male: 48% -
- B
- A
- C
- D
- E
- F
- G
- I
- H
- J
- K
- L
- P
- M
- N
- O
- Q
- R
- S
- V
- U
- T
- W
- X
- Z
- Y
- a
- h
- i
- b
- d
- e
- f
- c
- g
- j
- k
- l
- m
- n
- o
- p
- q
- r
- s
- t
- u
- v
- x
- w
- y
Oregon
-
Race Asian 6% Black 30% Native American 10% White 47% -
Ethnicity Hispanic 6% Not Hispanic or Latino 94%
Study Details
Setting
The study took place in two large counties in western Oregon. The schools represented a diverse range of communities, from low income/large city to working class/moderate size-city to rural settings.
Study sample
Participants were 127 first-grade students from 24 classrooms in six Title I schools in four districts. Participants were nominated by their teachers as needing supplemental reading assistance based on two criteria: low reading skills and relatively little reading experience with adults or others at home. The students were randomly assigned to intervention and comparison conditions within classrooms after being matched on the Rapid Letter Naming pretest. The study presented findings after the intervention students completed two years of the program. At the end of second grade, 84 students of the original sample remained (43 students in the intervention and 41 students in the comparison group). The study included an additional comparison group of 36 average-achieving readers from the same schools. Analysis involving these comparison groups was not eligible for WWC review because the WWC considers only comparisons of students with similar achievement backgrounds in assessing the effectiveness of SMART®. Student ethnicity was 47% European-American, 30% African-American, 10% American Indian, 6% Asian-American, and 6% Latino.
Intervention Group
Students received one-to-one tutoring for six months each year while they were in first and second grade. The program consisted of two 30-minute sessions a week. Students could also take home two books a month. The number of sessions per student ranged from 49 to 98 with a mean of 73 sessions.
Comparison Group
Students in the comparison group received the same regular classroom reading instruction as students in the intervention group, but did not receive the tutoring program.
Outcome descriptions
The Woodcock Reading Mastery Tests–Revised (WRMT-R) word identification subtest was used to test students’ knowledge of alphabetics. First- and second-grade passages from the Oral Reading Fluency were used to test fluency. The WRMT-R passage comprehension subtest was used to test comprehension. Authors also looked at referral rates for special education; however this is not an outcome specified for the beginning reading topic (see Appendices A2.1–2.3 for more detailed descriptions of outcome measures).
Support for implementation
The SMART® program intentionally places minimal demands on volunteer tutors and classroom teachers. Volunteer tutors are given 1-2 hours of training, preferably before the school year begins, but occasionally in an “on the job” setting. The training focuses as much on the logistics of tutoring as it does on reading instruction techniques. A key resource for the volunteers is a volunteer handbook, which describes four reading strategies that they can use with students: reading to the child, reading with the child, re-reading with the child, and asking the child questions about what has been read. Volunteers rely on their own judgment for any other needs.
An indicator of the effect of the intervention, the improvement index can be interpreted as the expected change in percentile rank for an average comparison group student if that student had received the intervention.
For more, please see the WWC Glossary entry for improvement index.
An outcome is the knowledge, skills, and attitudes that are attained as a result of an activity. An outcome measures is an instrument, device, or method that provides data on the outcome.
A finding that is included in the effectiveness rating. Excluded findings may include subgroups and subscales.
The sample on which the analysis was conducted.
The group to which the intervention group is compared, which may include a different intervention, business as usual, or no services.
The timing of the post-intervention outcome measure.
The number of students included in the analysis.
The mean score of students in the intervention group.
The mean score of students in the comparison group.
The WWC considers a finding to be statistically significant if the likelihood that the finding is due to chance alone, rather than a real difference, is less than five percent.
The WWC reviews studies for WWC products, Department of Education grant competitions, and IES performance measures.
The name and version of the document used to guide the review of the study.
The version of the WWC design standards used to guide the review of the study.
The result of the WWC assessment of the study. The rating is based on the strength of evidence of the effectiveness of the intervention. Studies are given a rating of Meets WWC Design Standards without Reservations, Meets WWC Design Standards with Reservations, or >Does Not Meet WWC Design Standards.
A related publication that was reviewed alongside the main study of interest.
Study findings for this report.
Based on the direction, magnitude, and statistical significance of the findings within a domain, the WWC characterizes the findings from a study as one of the following: statistically significant positive effects, substantively important positive effects, indeterminate effects, substantively important negative effects, and statistically significant negative effects. For more, please see the WWC Handbook.
The WWC may review studies for multiple purposes, including different reports and re-reviews using updated standards. Each WWC review of this study is listed in the dropdown. Details on any review may be accessed by making a selection from the drop down list.
Tier 1 Strong indicates strong evidence of effectiveness,
Tier 2 Moderate indicates moderate evidence of effectiveness, and
Tier 3 Promising indicates promising evidence of effectiveness,
as defined in the
non-regulatory guidance for ESSA
and the regulations for ED discretionary grants (EDGAR Part 77).