Achievement Network's Investing in Innovation Expansion: Impacts on Educator Practice and Student Achievement
West, Martin R.; Morton, Beth A.; Herlihy, Corinne M. (2016). Retrieved from: https://eric.ed.gov/?id=ED565458
-
examining21,345Students, grades3-8
Department-funded evaluation
Review Details
Reviewed: January 2017
- Department-funded evaluation (findings for Achievement Network (ANet))
- Randomized Controlled Trial
- Meets WWC standards without reservations because it is a randomized controlled trial with low attrition.
This review may not reflect the full body of research evidence for this intervention.
Evidence Tier rating based solely on this study. This intervention may achieve a higher tier when combined with the full body of evidence.
Findings
|
Outcome measure |
Comparison | Period | Sample |
Intervention mean |
Comparison mean |
Significant? |
Improvement index |
Evidence tier |
|
|---|---|---|---|---|---|---|---|---|---|
|
Reading test score |
Achievement Network (ANet) vs. Business as usual |
1 Year |
Full sample;
|
N/A |
N/A |
No |
-- | ||
| Show Supplemental Findings | |||||||||
|
Reading test score |
Achievement Network (ANet) vs. Business as usual |
1 Year |
Non-Free or reduced price lunch;
|
N/A |
N/A |
No |
-- | ||
|
Reading test score |
Achievement Network (ANet) vs. Business as usual |
1 Year |
Grade: 6, 7, 8;
|
N/A |
N/A |
No |
-- | ||
|
Reading test score |
Achievement Network (ANet) vs. Business as usual |
1 Year |
Free or reduced price lunch;
|
N/A |
N/A |
Yes |
-- | ||
|
Reading test score |
Achievement Network (ANet) vs. Business as usual |
1 Year |
Grade: 3, 4, 5;
|
N/A |
N/A |
Yes |
-- | ||
|
Outcome measure |
Comparison | Period | Sample |
Intervention mean |
Comparison mean |
Significant? |
Improvement index |
Evidence tier |
|
|---|---|---|---|---|---|---|---|---|---|
|
Math test score |
Achievement Network (ANet) vs. Business as usual |
1 Year |
Full sample;
|
N/A |
N/A |
No |
-- | ||
| Show Supplemental Findings | |||||||||
|
Math test score |
Achievement Network (ANet) vs. Business as usual |
1 Year |
Grade: 6, 7, 8;
|
N/A |
N/A |
No |
-- | ||
|
Math test score |
Achievement Network (ANet) vs. Business as usual |
1 Year |
Non-Free or reduced price lunch;
|
N/A |
N/A |
No |
-- | ||
|
Math test score |
Achievement Network (ANet) vs. Business as usual |
1 Year |
Free or reduced price lunch;
|
N/A |
N/A |
No |
-- | ||
|
Math test score |
Achievement Network (ANet) vs. Business as usual |
1 Year |
Grade: 3, 4, 5;
|
N/A |
N/A |
Yes |
-- | ||
Evidence Tier rating based solely on this study. This intervention may achieve a higher tier when combined with the full body of evidence.
Sample Characteristics
Characteristics of study sample as reported by study author.
-
- B
- A
- C
- D
- E
- F
- G
- I
- H
- J
- K
- L
- P
- M
- N
- O
- Q
- R
- S
- V
- U
- T
- W
- X
- Z
- Y
- a
- h
- i
- b
- d
- e
- f
- c
- g
- j
- k
- l
- m
- n
- o
- p
- q
- r
- s
- t
- u
- v
- x
- w
- y
Illinois, Louisiana, Massachusetts
Study Details
Setting
The intervention occurred in five school districts in the United States: Boston, Massachusetts; Chelsea, Massachusetts; Springfield, Massachusetts; Jefferson Parish, Louisiana; and Chicago, Illinois.
Study sample
Although the unit of assignment was schools, and the intervention was geared toward teachers, eligible outcomes were measured for students. 21,335 students were included in the analyses of the ELA outcome and 21,258 students were included in the analyses of the Math outcome. 62% were in grades 3-5 (elementary school), 38% were in grades 6-8 (middle school), and 87% were eligible for free or reduced price lunch.
Intervention Group
ANet focuses on improving teacher use of data in order to improve both teaching and learning. It is a data-based instructional program with four key components: (1) quarterly interim assessments aligned to state content standards in English language arts and math for Grades 3 through 8; (2) data tools such as reports on individual student progress; (3) school leader coaching to support teacher use of assessment data to improve instruction; and (4) a peer schools network for joint professional development and sharing results. Prior to the beginning of the school year, ANet works with the intervention schools or districts to align their assessment tool with the existing "curriculum and curricular scope and sequence." Throughout the year, there are scheduled coaching visits and assessment administration meetings focused on reviewing data and action planning. During the school year, students are given quarterly interim assessments. These interim assessments are "the core source of data to inform teachers' instructional decision making." Results are provided within two business days to teachers and school leaders and meetings are head to discuss results, identify gaps in learning, and formulate plans for action. ANet works with school leaders throughout the year, coaching them to build capacity to support teacher implementation of data-based instruction al practices. The program model "focuses on strengthening leaders' engagement with interim assessment data and their prioritization of the use of data to inform instruction." As part of the program, school leaders are supposed to build school-wide cultures of achievement, develop the skills and practices of their teachers around instructional data use, manage the data cycle, and reflect on school and student progress, making changes when necessary. A data leadership team is formed, which is normally comprised of the school leader (usually the principal), the assistant principal, and a content-level or grade-level instructional leader. This team is coached by an ANet coach and provides data leadership for the program.
Comparison Group
Schools in the comparison group were on the wait list to receive the intervention in the following year. However, there were no restrictions for the comparison schools in terms of using other programs. All of the comparison group schools administered interim assessments in some subjects or grades. The educators in the comparison group schools likely received some kind of data-related support.
Support for implementation
Schools in the intervention condition received coaching and other support from the ANet program both before and during program implementation. Before the school year begins, ANet works with districts or schools to align the ANet interim assessments (which will be given to the students during the school year) with the school's existing curriculum and curricular sequence and scope. ANet coaches also make visits throughout the year. For example, they work closely with school leaders (usually principals) to build capacity to support teacher implementation of data-based instructional practices. They also meet with the data leadership team, which consists of the school leader, the assistant principal, and a content-area or grade-level instructional leader. ANet coaches assist schools with planning both before and after the interim assessments are completed by the students.
An indicator of the effect of the intervention, the improvement index can be interpreted as the expected change in percentile rank for an average comparison group student if that student had received the intervention.
For more, please see the WWC Glossary entry for improvement index.
An outcome is the knowledge, skills, and attitudes that are attained as a result of an activity. An outcome measures is an instrument, device, or method that provides data on the outcome.
A finding that is included in the effectiveness rating. Excluded findings may include subgroups and subscales.
The sample on which the analysis was conducted.
The group to which the intervention group is compared, which may include a different intervention, business as usual, or no services.
The timing of the post-intervention outcome measure.
The number of students included in the analysis.
The mean score of students in the intervention group.
The mean score of students in the comparison group.
The WWC considers a finding to be statistically significant if the likelihood that the finding is due to chance alone, rather than a real difference, is less than five percent.
The WWC reviews studies for WWC products, Department of Education grant competitions, and IES performance measures.
The name and version of the document used to guide the review of the study.
The version of the WWC design standards used to guide the review of the study.
The result of the WWC assessment of the study. The rating is based on the strength of evidence of the effectiveness of the intervention. Studies are given a rating of Meets WWC Design Standards without Reservations, Meets WWC Design Standards with Reservations, or >Does Not Meet WWC Design Standards.
A related publication that was reviewed alongside the main study of interest.
Study findings for this report.
Based on the direction, magnitude, and statistical significance of the findings within a domain, the WWC characterizes the findings from a study as one of the following: statistically significant positive effects, substantively important positive effects, indeterminate effects, substantively important negative effects, and statistically significant negative effects. For more, please see the WWC Handbook.
The WWC may review studies for multiple purposes, including different reports and re-reviews using updated standards. Each WWC review of this study is listed in the dropdown. Details on any review may be accessed by making a selection from the drop down list.
Tier 1 Strong indicates strong evidence of effectiveness,
Tier 2 Moderate indicates moderate evidence of effectiveness, and
Tier 3 Promising indicates promising evidence of effectiveness,
as defined in the
non-regulatory guidance for ESSA
and the regulations for ED discretionary grants (EDGAR Part 77).