Mentoring Early Career Teachers in Urban Alaska: Impact Findings from the Investing in Innovation (i3) Evaluation of the Alaska Statewide Mentor Project Urban Growth Opportunity
Ault, Phyllis Campbell; Roccograndi, Angela; Burke, Art (2017). Education Northwest. Retrieved from: https://eric.ed.gov/?id=ED607173
-
examining556Teachers, grades4-10
Department-funded evaluation
Review Details
Reviewed: July 2022
- Department-funded evaluation (findings for Alaska Statewide Mentor Project (ASMP) Urban Growth Opportunity (UGO) Program)
- Randomized Controlled Trial
- Meets WWC standards without reservations because it is a randomized controlled trial with low attrition.
This review may not reflect the full body of research evidence for this intervention.
Evidence Tier rating based solely on this study. This intervention may achieve a higher tier when combined with the full body of evidence.
Findings
|
Outcome measure |
Comparison | Period | Sample |
Intervention mean |
Comparison mean |
Significant? |
Improvement index |
Evidence tier |
|
|---|---|---|---|---|---|---|---|---|---|
|
Alaska state assessment for math |
Alaska Statewide Mentor Project (ASMP) Urban Growth Opportunity (UGO) Program vs. Business as usual |
0 Years |
Students of study teachers teaching grades 4-6 (cohorts 1-2); Year 2 of teaching;
|
0.10 |
0.04 |
No |
-- | ||
|
Alaska state assessment for math |
Alaska Statewide Mentor Project (ASMP) Urban Growth Opportunity (UGO) Program vs. Business as usual |
0 Years |
Students of study teachers teaching grades 7-10 (cohorts 1-2); Year 2 of teaching;
|
-0.13 |
-0.29 |
No |
-- | ||
| Show Supplemental Findings | |||||||||
|
Alaska state assessment for math |
Alaska Statewide Mentor Project (ASMP) Urban Growth Opportunity (UGO) Program vs. Business as usual |
-1 Years |
White students of study teachers teaching grades 7-10 (cohorts 1-3); Year 1 of teaching;
|
0.13 |
-0.13 |
Yes |
|
||
|
Alaska state assessment for math |
Alaska Statewide Mentor Project (ASMP) Urban Growth Opportunity (UGO) Program vs. Business as usual |
-1 Years |
Hispanic students of study teachers teaching grades 7-10 (cohorts 1-3); Year 1 of teaching;
|
-0.37 |
-0.57 |
Yes |
|
||
|
Alaska state assessment for math |
Alaska Statewide Mentor Project (ASMP) Urban Growth Opportunity (UGO) Program vs. Business as usual |
-1 Years |
Alaska Native students of study teachers teaching grades 7-10 (cohorts 1-3); Year 1 of teaching;
|
-0.30 |
-0.55 |
Yes |
|
||
|
Alaska state assessment for math |
Alaska Statewide Mentor Project (ASMP) Urban Growth Opportunity (UGO) Program vs. Business as usual |
-1 Years |
Students of two or more races of study teachers teaching grades 7-10 (cohorts 1-3); Year 1 of teaching;
|
-0.14 |
-0.38 |
Yes |
|
||
|
Alaska state assessment for math |
Alaska Statewide Mentor Project (ASMP) Urban Growth Opportunity (UGO) Program vs. Business as usual |
-1 Years |
Students of study teachers teaching grades 4-6 (cohorts 1-3); Year 1 of teaching;
|
0.13 |
0.02 |
No |
-- | ||
|
Alaska state assessment for math |
Alaska Statewide Mentor Project (ASMP) Urban Growth Opportunity (UGO) Program vs. Business as usual |
-1 Years |
Students of study teachers teaching grades 7-10 (cohorts 1-3); Year 1 of teaching;
|
-0.09 |
-0.31 |
No |
-- | ||
|
Outcome measure |
Comparison | Period | Sample |
Intervention mean |
Comparison mean |
Significant? |
Improvement index |
Evidence tier |
|
|---|---|---|---|---|---|---|---|---|---|
|
CLASS classroom organization |
Alaska Statewide Mentor Project (ASMP) Urban Growth Opportunity (UGO) Program vs. Business as usual |
0 Years |
Random subsample (cohorts 2-3); Year 2 of teaching;
|
5.88 |
5.99 |
No |
-- | ||
|
CLASS emotional support |
Alaska Statewide Mentor Project (ASMP) Urban Growth Opportunity (UGO) Program vs. Business as usual |
0 Years |
Random subsample (cohorts 2-3); Year 2 of teaching;
|
4.48 |
4.59 |
No |
-- | ||
|
CLASS instructional support |
Alaska Statewide Mentor Project (ASMP) Urban Growth Opportunity (UGO) Program vs. Business as usual |
0 Years |
Random subsample (cohorts 2-3); Year 2 of teaching;
|
3.05 |
3.21 |
No |
-- | ||
| Show Supplemental Findings | |||||||||
|
CLASS emotional support |
Alaska Statewide Mentor Project (ASMP) Urban Growth Opportunity (UGO) Program vs. Business as usual |
-1 Years |
Random subsample (cohorts 2-3); Year 1 of teaching;
|
4.48 |
4.72 |
No |
-- | ||
|
CLASS classroom organization |
Alaska Statewide Mentor Project (ASMP) Urban Growth Opportunity (UGO) Program vs. Business as usual |
-1 Years |
Random subsample (cohorts 2-3); Year 1 of teaching;
|
5.78 |
6.00 |
No |
-- | ||
|
CLASS instructional support |
Alaska Statewide Mentor Project (ASMP) Urban Growth Opportunity (UGO) Program vs. Business as usual |
-1 Years |
Random subsample (cohorts 2-3); Year 1 of teaching;
|
3.11 |
3.25 |
No |
-- | ||
|
Outcome measure |
Comparison | Period | Sample |
Intervention mean |
Comparison mean |
Significant? |
Improvement index |
Evidence tier |
|
|---|---|---|---|---|---|---|---|---|---|
|
Alaska state assessment for reading |
Alaska Statewide Mentor Project (ASMP) Urban Growth Opportunity (UGO) Program vs. Business as usual |
0 Years |
Students of study teachers teaching grades 4-6 (cohorts 1-2); Year 2 of teaching;
|
0.05 |
0.09 |
No |
-- | ||
| Show Supplemental Findings | |||||||||
|
Alaska state assessment for reading |
Alaska Statewide Mentor Project (ASMP) Urban Growth Opportunity (UGO) Program vs. Business as usual |
-1 Years |
Students of study teachers teaching grades 4-6 (cohorts 1-3); UGO vs. no mentorship comparison group; Year 1 of teaching;
|
0.28 |
0.11 |
Yes |
|
||
|
Alaska state assessment for reading |
Alaska Statewide Mentor Project (ASMP) Urban Growth Opportunity (UGO) Program vs. Business as usual |
-1 Years |
Students of study teachers teaching grades 4-6 (cohorts 1-3); Year 1 of teaching;
|
0.14 |
0.08 |
No |
-- | ||
|
Outcome measure |
Comparison | Period | Sample |
Intervention mean |
Comparison mean |
Significant? |
Improvement index |
Evidence tier |
|
|---|---|---|---|---|---|---|---|---|---|
|
Teacher retention in the state |
Alaska Statewide Mentor Project (ASMP) Urban Growth Opportunity (UGO) Program vs. Business as usual |
0 Years |
Full sample (cohorts 1-3); Year 2 of teaching;
|
80.50 |
76.60 |
No |
-- | ||
| Show Supplemental Findings | |||||||||
|
Teacher retention in the state |
Alaska Statewide Mentor Project (ASMP) Urban Growth Opportunity (UGO) Program vs. Business as usual |
2 Years |
Full sample (cohort 1); Year 4 of teaching;
|
72.70 |
62.90 |
No |
-- | ||
|
Teacher retention in the state |
Alaska Statewide Mentor Project (ASMP) Urban Growth Opportunity (UGO) Program vs. Business as usual |
1 Year |
Full sample (cohort 1); Year 3 of teaching;
|
80.50 |
74.30 |
No |
-- | ||
|
Teacher retention in the state |
Alaska Statewide Mentor Project (ASMP) Urban Growth Opportunity (UGO) Program vs. Business as usual |
3 Years |
Full sample (cohort 1); Year 5 of teaching;
|
68.80 |
65.70 |
No |
-- | ||
|
Outcome measure |
Comparison | Period | Sample |
Intervention mean |
Comparison mean |
Significant? |
Improvement index |
Evidence tier |
|
|---|---|---|---|---|---|---|---|---|---|
|
Alaska state assessment for writing |
Alaska Statewide Mentor Project (ASMP) Urban Growth Opportunity (UGO) Program vs. Business as usual |
0 Years |
Students of study teachers teaching grades 4-6 (cohorts 1-2); Year 2 of teaching;
|
0.15 |
0.07 |
No |
-- | ||
| Show Supplemental Findings | |||||||||
|
Alaska state assessment for writing |
Alaska Statewide Mentor Project (ASMP) Urban Growth Opportunity (UGO) Program vs. Business as usual |
-1 Years |
Students of study teachers teaching grades 4-6 (cohorts 1-3); Year 1 of teaching;
|
0.02 |
0.05 |
No |
-- | ||
Evidence Tier rating based solely on this study. This intervention may achieve a higher tier when combined with the full body of evidence.
Sample Characteristics
Characteristics of study sample as reported by study author.
-
Female: 77%
Male: 23% -
Urban
-
- B
- A
- C
- D
- E
- F
- G
- I
- H
- J
- K
- L
- P
- M
- N
- O
- Q
- R
- S
- V
- U
- T
- W
- X
- Z
- Y
- a
- h
- i
- b
- d
- e
- f
- c
- g
- j
- k
- l
- m
- n
- o
- p
- q
- r
- s
- t
- u
- v
- x
- w
- y
Alaska
-
Race Other or unknown 15% White 85% -
Ethnicity Other or unknown 100% -
Eligible for Free and Reduced Price Lunch Other or unknown 100%
Study Details
Setting
The study took place in five urban school districts in Alaska. The intervention focuses on early career teachers who taught 4th through 10th grade.
Study sample
Eighty-five percent of the teachers were White and 77% were female.
Intervention Group
This grant evaluated the effectiveness of the Alaska Statewide Mentor Project (ASMP) Urban Growth Opportunity (UGO) program. Like the New Teacher Center (NTC) model on which the ASMP mentoring model was based, the ASMP model uses a rigorous selection process for mentors, who then participate in ongoing professional development, conduct weekly interactions with early career teachers (ECTs), collect and analyze classroom data using formative assessment tools, and collaborate with ECTs to develop plans for using reflective practices. However, ASMP adapted the NTC model to respond to the particular needs of teachers in Alaska, most of whom work with children from Alaska Native families that have unique needs. The ASMP model was further adapted for the UGO program to accommodate implementation in an urban setting. The study examined the effects of ECTs’ participation in the UGO program on three outcomes: their retention as a teacher in Alaska, their instructional practice, and the academic performance of their students.
Comparison Group
Early career teachers in the comparison group received their school district's standard support for novice teachers. Two of the five districts had formal mentoring programs for early career teachers.
Support for implementation
No implementation support was described separate from the intervention components, which included ongoing professional development, weekly interactions with early career teachers (ECTs), and formative assessment tools.
Grant Competition
Review Details
Reviewed: September 2018
- Grant Competition (findings for Alaska Statewide Mentor Project (ASMP) Urban Growth Opportunity (UGO) Program)
- Randomized Controlled Trial
- Meets WWC standards without reservations because it is a randomized controlled trial with low attrition.
This review may not reflect the full body of research evidence for this intervention.
Evidence Tier rating based solely on this study. This intervention may achieve a higher tier when combined with the full body of evidence.
Findings
|
Outcome measure |
Comparison | Period | Sample |
Intervention mean |
Comparison mean |
Significant? |
Improvement index |
Evidence tier |
|---|---|---|---|---|---|---|---|---|
|
Writing assessment |
Alaska Statewide Mentor Project (ASMP) Urban Growth Opportunity (UGO) Program vs. Business as usual |
2 Years |
Primary Grades;
|
0.15 |
0.07 |
No |
-- | |
|
Reading assessment |
Alaska Statewide Mentor Project (ASMP) Urban Growth Opportunity (UGO) Program vs. Business as usual |
2 Years |
Primary grades;
|
0.05 |
0.09 |
No |
-- |
|
Outcome measure |
Comparison | Period | Sample |
Intervention mean |
Comparison mean |
Significant? |
Improvement index |
Evidence tier |
|---|---|---|---|---|---|---|---|---|
|
Math assessment |
Alaska Statewide Mentor Project (ASMP) Urban Growth Opportunity (UGO) Program vs. Business as usual |
2 Years |
Primary Grades;
|
0.10 |
0.04 |
No |
-- | |
|
Math assessment |
Alaska Statewide Mentor Project (ASMP) Urban Growth Opportunity (UGO) Program vs. Business as usual |
2 Years |
Secondary Grades;
|
N/A |
N/A |
No |
-- |
|
Outcome measure |
Comparison | Period | Sample |
Intervention mean |
Comparison mean |
Significant? |
Improvement index |
Evidence tier |
|---|---|---|---|---|---|---|---|---|
|
Classroom Assessment Scoring System (CLASS): Emotional Support |
Alaska Statewide Mentor Project (ASMP) Urban Growth Opportunity (UGO) Program vs. Business as usual |
2 Years |
Full sample;
|
4.48 |
4.59 |
No |
-- | |
|
Classroom Assessment Scoring System (CLASS): Classroom Organization |
Alaska Statewide Mentor Project (ASMP) Urban Growth Opportunity (UGO) Program vs. Business as usual |
2 Years |
Full sample;
|
5.88 |
5.99 |
No |
-- | |
|
Classroom Assessment Scoring System: Instructional Support |
Alaska Statewide Mentor Project (ASMP) Urban Growth Opportunity (UGO) Program vs. Business as usual |
2 Years |
Full sample;
|
3.05 |
3.21 |
No |
-- |
|
Outcome measure |
Comparison | Period | Sample |
Intervention mean |
Comparison mean |
Significant? |
Improvement index |
Evidence tier |
|
|---|---|---|---|---|---|---|---|---|---|
|
Teacher retention rate |
Alaska Statewide Mentor Project (ASMP) Urban Growth Opportunity (UGO) Program vs. Business as usual |
3 Years |
Full sample;
|
0.83 |
0.77 |
No |
-- | ||
| Show Supplemental Findings | |||||||||
|
Teacher retention rate |
Alaska Statewide Mentor Project (ASMP) Urban Growth Opportunity (UGO) Program vs. Business as usual |
4 Years |
Cohort 1;
|
0.76 |
0.63 |
No |
-- | ||
|
Teacher retention rate |
Alaska Statewide Mentor Project (ASMP) Urban Growth Opportunity (UGO) Program vs. Business as usual |
3 Years |
Cohort 1;
|
0.84 |
0.74 |
No |
-- | ||
|
Teacher retention rate |
Alaska Statewide Mentor Project (ASMP) Urban Growth Opportunity (UGO) Program vs. Business as usual |
5 Years |
Cohort 1;
|
0.70 |
0.66 |
No |
-- | ||
Evidence Tier rating based solely on this study. This intervention may achieve a higher tier when combined with the full body of evidence.
Sample Characteristics
Characteristics of study sample as reported by study author.
-
Female: 76% -
Urban
-
- B
- A
- C
- D
- E
- F
- G
- I
- H
- J
- K
- L
- P
- M
- N
- O
- Q
- R
- S
- V
- U
- T
- W
- X
- Z
- Y
- a
- h
- i
- b
- d
- e
- f
- c
- g
- j
- k
- l
- m
- n
- o
- p
- q
- r
- s
- t
- u
- v
- x
- w
- y
Alaska
-
Race White 82%
Study Details
Setting
This study took place in 5 urban school districts in the state of Alaska.
Study sample
Among the participating teachers in the sample, all teachers were first year teachers, and more than three-forths (78.2%) were female. The majority of the teachers had at least bachelor's degrees (61.7%) and about one-quarter had master's degrees (25.6%). Nearly two-thirds of the participating teachers received their teaching degrees in Alaska (62.9%). The large majority (85%) did not relocate to Alaska for their teaching job. Among students in the sample (in Year 2), about half the sample was White (48%), 5% were African-American, 12% were Alaska Native, 10% were Asian, 10% were Hispanic, 5% were Native Hawaiian or Pacific Islander, 10% were two or more races, and less than 5% were American Indian. About half of the student sample was female, and about two thirds (68.4%) of the student sample qualified for free or reduced price lunch. About one-fifth (20.7%) of the sample were English language learners. About one-fourth of the sample (23.7%) were special education students.
Intervention Group
The Alaska Statewide Mentor Project (ASMP) Urban Growth Opportunity (UGO) Program provided professional development for early career teachers in five high-need district in Alaska, and offered two years of support from fully released, highly trained mentors. Full-time mentors had a caseload of no more than 15 early career teachers. Mentor qualifications were having at least eight years of teaching experience in Alaska, having recent classroom teaching experience or other relevant work in education (within past two years), having strong content knowledge in core subjects, and recognition as an excellent classroom practitioner. Over the two years, mentors participate in intensive professional development which included: 1) 2-days per year of orientation, 2) 3 days per year of Wrap Up sessions, 3) four academy-style training sessions for mentors, 4) participation in at least ten Friday Forums, 5) monthly communication with coach, 6) participation in two shadowing sessions with coach in Year 1, and at least one shadowing session in Year 2, and 7) utilization of mentor formative assessment tools during coaching. Early career teachers communicated with their mentors at least weekly, and mentors provided at least 3.5 hours per month of face-to-face interaction for each Early Career Teacher. The Formative Assessment tools for mentors included: 1) documentation of conversations through the Collaborative Assessment log, 2) use of formative assessment tools to support teachers and gathering of classroom data, and 3) support for reflective practice through an Individual Learning Plan, Mid-Year Review, and Professional Growth Reflection.
Comparison Group
While the intervention teachers were mentored for 2 years, the comparison group did not receive mentoring or had formal mentoring typically offered in their district. In two of the five participating districts, the districts offered formal district mentoring which varied in quality and duration/intensity of support, which was a typical service. In three of the five participating districts, no formal mentoring was offered to the comparison group. The comparison group services would be considered "business as usual."
Support for implementation
The Alaska Statewide Mentor Project (ASMP) Urban Growth Opportunity (UGO) Program was funded by a federal Investing in Innovation (i3) validation grant. Support for project implementation of this two-year mentoring program for early career teachers came from earlier implementation of the ASMP in Alaska's rural school districts. The New Teacher Center (NTC) model formed the foundation for the ASMP program structure and professional development curriculum. The program was adapted for the state of Alaska and and serving a high proportion of Alaska Native students, many of whom live in rural/bush communities. The model was adapted for Alaskan urban settings which involved less interaction with principals, more district coordination, and greater flexibility to meet with Early Career Teachers more often and for shorter periods of time. The program builds from research on teacher professional development practices, mentoring practice, mentoring qualifications, and the use of formative feedback in educative mentoring.
An indicator of the effect of the intervention, the improvement index can be interpreted as the expected change in percentile rank for an average comparison group student if that student had received the intervention.
For more, please see the WWC Glossary entry for improvement index.
An outcome is the knowledge, skills, and attitudes that are attained as a result of an activity. An outcome measures is an instrument, device, or method that provides data on the outcome.
A finding that is included in the effectiveness rating. Excluded findings may include subgroups and subscales.
The sample on which the analysis was conducted.
The group to which the intervention group is compared, which may include a different intervention, business as usual, or no services.
The timing of the post-intervention outcome measure.
The number of students included in the analysis.
The mean score of students in the intervention group.
The mean score of students in the comparison group.
The WWC considers a finding to be statistically significant if the likelihood that the finding is due to chance alone, rather than a real difference, is less than five percent.
The WWC reviews studies for WWC products, Department of Education grant competitions, and IES performance measures.
The name and version of the document used to guide the review of the study.
The version of the WWC design standards used to guide the review of the study.
The result of the WWC assessment of the study. The rating is based on the strength of evidence of the effectiveness of the intervention. Studies are given a rating of Meets WWC Design Standards without Reservations, Meets WWC Design Standards with Reservations, or >Does Not Meet WWC Design Standards.
A related publication that was reviewed alongside the main study of interest.
Study findings for this report.
Based on the direction, magnitude, and statistical significance of the findings within a domain, the WWC characterizes the findings from a study as one of the following: statistically significant positive effects, substantively important positive effects, indeterminate effects, substantively important negative effects, and statistically significant negative effects. For more, please see the WWC Handbook.
The WWC may review studies for multiple purposes, including different reports and re-reviews using updated standards. Each WWC review of this study is listed in the dropdown. Details on any review may be accessed by making a selection from the drop down list.
Tier 1 Strong indicates strong evidence of effectiveness,
Tier 2 Moderate indicates moderate evidence of effectiveness, and
Tier 3 Promising indicates promising evidence of effectiveness,
as defined in the
non-regulatory guidance for ESSA
and the regulations for ED discretionary grants (EDGAR Part 77).