Kemple, J. J., Herlihy, C. M., & Smith, T. J. (2005). Evidence from the Talent Development High School model. New York, NY: MDRC. . Retrieved from: https://eric.ed.gov/?id=ED485348
Reviewed: January 2023
This review may not reflect the full body of research evidence for this intervention.
Evidence Tier rating based solely on this study. This intervention may achieve a higher tier when combined with the full body of evidence.
|
Outcome measure |
Comparison | Period | Sample |
Intervention mean |
Comparison mean |
Significant? |
Improvement index |
Evidence tier |
|
|---|---|---|---|---|---|---|---|---|---|
|
Pennsylvania System of School Assessment (PSSA) math -- at or above proficient level |
Talent Development High School (TDHS) model vs. Business as usual |
3 Years |
Year 1 cohort, first-time ninth graders (measured in 11th grade) ;
|
10.00 |
7.00 |
No |
-- | ||
|
Pennsylvania System of School Assessment (PSSA) math -- at basic proficiency level |
Talent Development High School (TDHS) model vs. Business as usual |
3 Years |
Year 1 cohort, first-time ninth graders (measured in 11th grade);
|
10.80 |
14.00 |
No |
-- | ||
|
Pennsylvania System of School Assessment (PSSA) math -- at below-basic proficiency |
Talent Development High School (TDHS) model vs. Business as usual |
3 Years |
Year 1 cohort, first-time ninth graders (measured in 11th grade);
|
73.80 |
80.00 |
No |
-- | ||
|
Pennsylvania System of School Assessment (PSSA) math scaled score |
Talent Development High School (TDHS) model vs. Business as usual |
3 Years |
Year 1 cohort; first-time ninth graders (measured in 11th grade);
|
N/A |
N/A |
No |
-- | ||
| Show Supplemental Findings | |||||||||
|
Pennsylvania System of School Assessment (PSSA) math scaled score |
Talent Development High School (TDHS) model vs. Business as usual |
3 Years |
All first-time ninth graders, early implementing TDHS schools (measured in 11th grade);
|
N/A |
N/A |
No |
-- | ||
|
Pennsylvania System of School Assessment (PSSA) math -- at or above proficient level |
Talent Development High School (TDHS) model vs. Business as usual |
3 Years |
All first-time ninth graders, early implementing TDHS schools (measured in 11th grade);
|
N/A |
N/A |
No |
-- | ||
|
Pennsylvania System of School Assessment (PSSA) math -- at basic proficiency level |
Talent Development High School (TDHS) model vs. Business as usual |
3 Years |
All first-time ninth graders, early implementing TDHS schools (measured in 11th grade);
|
N/A |
N/A |
No |
-- | ||
|
Pennsylvania System of School Assessment (PSSA) math -- at below-basic proficiency |
Talent Development High School (TDHS) model vs. Business as usual |
3 Years |
All first-time ninth graders, early implementing TDHS schools (measured in 11th grade);
|
N/A |
N/A |
No |
-- | ||
|
Outcome measure |
Comparison | Period | Sample |
Intervention mean |
Comparison mean |
Significant? |
Improvement index |
Evidence tier |
|
|---|---|---|---|---|---|---|---|---|---|
|
Took 11th grade PSSA test on time |
Talent Development High School (TDHS) model vs. Business as usual |
3 Years |
Year 1 cohort, first time ninth graders (measured in 11th grade);
|
45.04 |
45.00 |
No |
-- | ||
| Show Supplemental Findings | |||||||||
|
Took 11th grade PSSA test on time |
Talent Development High School (TDHS) model vs. Business as usual |
3 Years |
All first-time ninth graders, early implementing TDHS schools (measured in 11th grade);
|
N/A |
N/A |
No |
-- | ||
|
Outcome measure |
Comparison | Period | Sample |
Intervention mean |
Comparison mean |
Significant? |
Improvement index |
Evidence tier |
|
|---|---|---|---|---|---|---|---|---|---|
|
Pennsylvania System of School Assessment (PSSA) reading -- at below-basic proficiency |
Talent Development High School (TDHS) model vs. Business as usual |
3 Years |
Year 1 cohort, first-time ninth graders (measured in 11th grade);
|
68.40 |
68.00 |
No |
-- | ||
|
Pennsylvania System of School Assessment (PSSA) reading -- at basic proficiency level |
Talent Development High School (TDHS) model vs. Business as usual |
3 Years |
Year 1 cohort, first-time ninth graders (measured in 11th grade);
|
22.30 |
22.00 |
No |
-- | ||
|
Pennsylvania System of School Assessment (PSSA) reading -- at or above proficient level |
Talent Development High School (TDHS) model vs. Business as usual |
3 Years |
Year 1 cohort, first-time ninth graders (measured in 11th grade);
|
9.30 |
10.00 |
No |
-- | ||
|
Pennsylvania System of School Assessment (PSSA) reading scaled score |
Talent Development High School (TDHS) model vs. Business as usual |
3 Years |
Year 1 cohort, first-time ninth graders (measured in 11th grade) ;
|
N/A |
N/A |
No |
-- | ||
| Show Supplemental Findings | |||||||||
|
Pennsylvania System of School Assessment (PSSA) reading scaled score |
Talent Development High School (TDHS) model vs. Business as usual |
3 Years |
All first-time ninth graders, early implementing TDHS schools (measured in 11th grade);
|
N/A |
N/A |
No |
-- | ||
|
Pennsylvania System of School Assessment (PSSA) reading -- at or above proficient level |
Talent Development High School (TDHS) model vs. Business as usual |
3 Years |
All first-time ninth graders, early implementing TDHS schools (measured in 11th grade);
|
N/A |
N/A |
No |
-- | ||
|
Pennsylvania System of School Assessment (PSSA) reading -- at below-basic proficiency |
Talent Development High School (TDHS) model vs. Business as usual |
3 Years |
All first-time ninth graders, early implementing TDHS schools (measured in 11th grade);
|
N/A |
N/A |
No |
-- | ||
|
Pennsylvania System of School Assessment (PSSA) reading -- at basic proficiency level |
Talent Development High School (TDHS) model vs. Business as usual |
3 Years |
All first-time ninth graders, early implementing TDHS schools (measured in 11th grade);
|
N/A |
N/A |
No |
-- | ||
|
Outcome measure |
Comparison | Period | Sample |
Intervention mean |
Comparison mean |
Significant? |
Improvement index |
Evidence tier |
|---|---|---|---|---|---|---|---|---|
|
Attendance rate |
Talent Development High School (TDHS) model vs. Business as usual |
1 Year |
First-time ninth graders; early implementing TDHS schools;
|
N/A |
N/A |
No |
-- | |
|
Attendance rate of 90% or higher |
Talent Development High School (TDHS) model vs. Business as usual |
1 Year |
First-time ninth graders, early implementing TDHS schools;
|
N/A |
N/A |
No |
-- | |
|
Attendance rate of 80% or lower |
Talent Development High School (TDHS) model vs. Business as usual |
1 Year |
First-time ninth graders; early implementing TDHS schools;
|
N/A |
N/A |
No |
-- |
Characteristics of study sample as reported by study author.
The study uses data on 20 cohorts of students in "non-selective, comprehensive high schools" in Philadelphia. The main analysis includes five schools that implemented the TDHS intervention model and six matched comparison schools that did not implement the model.
This information was not provided.
TDHS is a school reform model for restructuring large high schools with persistent attendance and discipline problems, poor student achievement, and high dropout rates. The model includes both structural and curricular reforms. It calls for schools to reorganize into small "learning communities"--including ninth-grade academies for first-year students and career academies for students in upper grades--to reduce student isolation and anonymity. It also emphasizes high academic standards, provides all students with a college-preparatory academic sequence, and provides staff with professional development to support implementation.
Standard high school curricula and organization.
Schools received funding in support of the TDHS model from federal grants, Center for Research on the Education of Students Placed At Risk, and district funds. Districts provided additional teachers to the schools implementing the model to support changes in scheduling and team teaching. A small implementation team was also sent to each school to facilitate development of the TDHS model. The team worked one-on-one with teachers, in addition to coordinating school-level activities.
Reviewed: July 2007
This review may not reflect the full body of research evidence for this intervention.
Evidence Tier rating based solely on this study. This intervention may achieve a higher tier when combined with the full body of evidence.
Please see the WWC summary of evidence for Talent Development High Schools.
|
Outcome measure |
Comparison | Period | Sample |
Intervention mean |
Comparison mean |
Significant? |
Improvement index |
Evidence tier |
|---|---|---|---|---|---|---|---|---|
|
Enrolled in tenth grade by end of second year (%) |
Talent Development High Schools vs. unknown |
end of second year of high school |
Cohorts 1 and 2;
|
68.00 |
60.00 |
No |
-- | |
|
Total credits earned by end of second year |
Talent Development High Schools vs. unknown |
end of third year of high school |
Cohorts 1 and 2;
|
9.50 |
8.60 |
No |
-- |
Characteristics of study sample as reported by study author.
| Black | 75% |
| Hispanic | 17% | |
| Not Hispanic or Latino | 83% |
The impact study was conducted in 11 nonselective public high schools in Philadelphia.
The main analysis sample included first-time ninth-grade students1 from five high schools that began implementing Talent Development High Schools between 1999 and 2001 and six matched comparison high schools. Between two and four comparison schools were matched to each of the five intervention schools based on the racial/ethnic composition and promotion rates of the schools’ ninth-grade students (Kemple & Herlihy, 2004). A comparison school could be matched to multiple Talent Development High Schools. The study compared the outcomes of ninth graders who entered Talent Development High Schools in the three years immediately after the program was implemented with those of ninth graders from these schools in the three years just before program implementation and with the outcome differences over the same time period for the matched comparison schools. Many students selected for Talent Development High Schools had low test scores and were overage for their grade. More than three-quarters were African-American and about one in six were Hispanic. Poor attendance was common, with two-thirds missing at least 20% of scheduled school days during their ninth-grade year. In addition, many did not make regular progress toward graduation, with just half promoted to tenth grade at the end of their ninth-grade year. Students in the matched comparison schools were generally similar to Talent Development High Schools students on these characteristics (Kemple & Herlihy, 2004). The study examined three cohorts of students. Cohort 1 included students in the intervention and matched comparison schools who enrolled in the ninth grade during the first year of Talent Development High Schools implementation at the intervention schools. Similarly, Cohort 2 and Cohort 3 included students who were enrolled in the ninth grade during the second and third years of implementation, respectively. Given the fixed period for data collection, later cohorts had shorter follow-up periods. To ensure both an adequate follow-up and an adequate sample size for assessing program effectiveness, the WWC used second-year results based on Cohorts 1 and 2 to rate the effectiveness of Talent Development High Schools. Longer-term results based only on Cohort 1 and shorter-term results based on all three cohorts are reported in Appendix A4.
The Philadelphia public school district implemented the Talent Development High Schools model in seven high schools. The district began to roll out the program in 1998, with one or two high schools launching Talent Development High Schools each year over a five-year period. School administrators volunteered their schools as candidates for implementing the new program. To allow for adequate follow-up, the impact study excluded the two Philadelphia high schools that implemented Talent Development High Schools last. All the Philadelphia Talent Development High Schools created ninth-grade academies on a separate floor or wing of the building, which were taught by teams of four to five teachers. Each school introduced block scheduling with 80- to 90-minute class sessions, introducing “double dose” math and English courses for ninth and tenth graders. These double sections of English and math allowed students to both prepare for and take college preparatory classes over the course of one academic year. Six of the seven schools offered “Twilight School” for new or repeating ninth graders with serious attendance or discipline problems. The model for students in grades 10 through 12 centered around career academies, in which students were divided into smaller “learning communities” around a broad career interest and the curriculum was organized around a career theme. Many Philadelphia high schools already had career academies before Talent Development High Schools was implemented, including many non-Talent Development schools. The study authors concluded that “(i)t is likely, therefore, that the upper-grade experience of students in Talent Development schools did not greatly differ from that of students in non-Talent Development schools” (Kemple, Herlihy, & Smith, 2005, p. 27). The study authors reported some variation in how the program was implemented across schools (Kemple, Herlihy, & Smith, 2005). In particular, they noted considerable variation across the intervention schools in the amount of technical assistance and support schools received from the intervention developer, as well as the amount of interventionspecific training school staff received.
Matched comparison schools were nonselective Philadelphia high schools that did not implement Talent Development High Schools. The authors compared the intervention group both with students in the comparison schools and with students who attended the intervention schools prior to the implementation of Talent Development High Schools.
Two relevant outcomes are included in this review: total credits earned and enrollment in the tenth grade by the end of the second year of high school (see Appendix A2 for more detailed descriptions of these outcome measures). The study also examined Talent Development High Schools’ effects on attendance and student achievement. These outcomes are not included in this report because they do not fall within the three domains (staying in school, progressing in school, and completing school) examined by the WWC’s review of dropout prevention interventions. Effects on the percentage of students who exited the school district were also estimated. However, the WWC had concerns about the validity of this measure and did not include it in the review. The study also examined the effects of Talent Development High Schools on graduation on the two earliest implementing schools. Since these results are only available for a small subset of the full research sample, they are not considered for the effectiveness rating and improvement index.
Teachers at Talent Development High Schools were regular teachers employed by the Philadelphia Public Schools. “Curriculum coaches” who had been trained by the intervention developer provided on-site technical assistance with implementing the Talent Development High Schools model. The developer also provided summer training institutes for staff.
In the case of multiple manuscripts that report on one study, the WWC selects one manuscript as the primary citation and lists other manuscripts that describe the study as additional sources.
Kemple, James J.; Herlihy, Corinne M. (2004). Context, Components, and Initial Impacts on Ninth-Grade Students? Engagement and Performance. The Talent Development High School Model. MDRC.
An indicator of the effect of the intervention, the improvement index can be interpreted as the expected change in percentile rank for an average comparison group student if that student had received the intervention.
For more, please see the WWC Glossary entry for improvement index.
An outcome is the knowledge, skills, and attitudes that are attained as a result of an activity. An outcome measures is an instrument, device, or method that provides data on the outcome.
A finding that is included in the effectiveness rating. Excluded findings may include subgroups and subscales.
The sample on which the analysis was conducted.
The group to which the intervention group is compared, which may include a different intervention, business as usual, or no services.
The timing of the post-intervention outcome measure.
The number of students included in the analysis.
The mean score of students in the intervention group.
The mean score of students in the comparison group.
The WWC considers a finding to be statistically significant if the likelihood that the finding is due to chance alone, rather than a real difference, is less than five percent.
The WWC reviews studies for WWC products, Department of Education grant competitions, and IES performance measures.
The name and version of the document used to guide the review of the study.
The version of the WWC design standards used to guide the review of the study.
The result of the WWC assessment of the study. The rating is based on the strength of evidence of the effectiveness of the intervention. Studies are given a rating of Meets WWC Design Standards without Reservations, Meets WWC Design Standards with Reservations, or >Does Not Meet WWC Design Standards.
A related publication that was reviewed alongside the main study of interest.
Study findings for this report.
Based on the direction, magnitude, and statistical significance of the findings within a domain, the WWC characterizes the findings from a study as one of the following: statistically significant positive effects, substantively important positive effects, indeterminate effects, substantively important negative effects, and statistically significant negative effects. For more, please see the WWC Handbook.
The WWC may review studies for multiple purposes, including different reports and re-reviews using updated standards. Each WWC review of this study is listed in the dropdown. Details on any review may be accessed by making a selection from the drop down list.
Tier 1 Strong indicates strong evidence of effectiveness,
Tier 2 Moderate indicates moderate evidence of effectiveness, and
Tier 3 Promising indicates promising evidence of effectiveness,
as defined in the
non-regulatory guidance for ESSA
and the regulations for ED discretionary grants (EDGAR Part 77).
Resources
Newsflash
Contact Us
Facebook
Twitter
YouTube