WWC review of this study

San Francisco Bay Area KIPP schools: A study of early implementation and achievement. Final report.

Woodworth, K. R., David, J. L., Guha, R., Wang, H., & Lopez-Torkos, A. (2008). Menlo Park, CA: SRI International.

examining

2,725

Students

, grades

5-8

Grant Competition

Review Details

Reviewed: February 2018

No statistically significant positive
findings

Meets WWC standards with reservations

For:

Grant Competition (findings for San Francisco Bay Area Knowledge is Power Program (KIPP) Schools)

Using:

Research Design:

Quasi-Experimental Design

Rating:

Meets WWC standards with reservations because it uses a quasi-experimental design in which the analytic intervention and comparison groups satisfy the baseline equivalence requirement.

This review may not reflect the full body of research evidence for this intervention.

Evidence Tier rating based solely on this study. This intervention may achieve a higher tier when combined with the full body of evidence.

Findings

General Mathematics Achievement outcomes—Indeterminate effect found for the domain
Outcome measure	Comparison	Period	Sample	Intervention mean	Comparison mean	Significant?	Improvement index
California Standards Test: Mathematics	San Francisco Bay Area Knowledge is Power Program (KIPP) Schools vs. Business as usual	1 Year	Grade: 5; Combined schools A, B, and C; 2004 cohort.; 884 students	342.27	301.83	No	--
California Standards Test: Mathematics	San Francisco Bay Area Knowledge is Power Program (KIPP) Schools vs. Business as usual	1 Year	Grade: 5; Combined schools A, B, and C; 2003 cohort.; 1,841 students	323.99	297.36	No	--
California Standards Test: Mathematics	San Francisco Bay Area Knowledge is Power Program (KIPP) Schools vs. Business as usual	1 Year	Grade: 6; Combined schools A, B, and C; 907 students	351.44	312.09	No	--

Literacy Achievement outcomes—Indeterminate effect found for the domain
Outcome measure	Comparison	Period	Sample	Intervention mean	Comparison mean	Significant?	Improvement index
California Standards Test English Language Arts (CST-ELA)	San Francisco Bay Area Knowledge is Power Program (KIPP) Schools vs. Business as usual	1 Year	Grade: 6; Schools A, B, and C, combined; 907 students	338.74	314.12	No	--
California Standards Test English Language Arts (CST-ELA)	San Francisco Bay Area Knowledge is Power Program (KIPP) Schools vs. Business as usual	1 Year	Grade: 5; Combined schools A, B, and C. 2004 cohort.; 884 students	328.41	313.75	No	--
California Standards Test English Language Arts (CST-ELA)	San Francisco Bay Area Knowledge is Power Program (KIPP) Schools vs. Business as usual	1 Year	Grade: 5; Combined Schools A, B, and C; 2003 cohort; 1,841 students	323.22	317.59	No	--

Evidence Tier rating based solely on this study. This intervention may achieve a higher tier when combined with the full body of evidence.

Sample Characteristics

Characteristics of study sample as reported by study author.

17% English language learners
Female: 54%
Male: 46%
Urban
- B
- A
- C
- D
- E
- F
- G
- I
- H
- J
- K
- L
- P
- M
- N
- O
- Q
- R
- S
- V
- U
- T
- W
- X
- Z
- Y
- a
- h
- i
- b
- d
- e
- f
- c
- g
- j
- k
- l
- m
- n
- o
- p
- q
- r
- s
- t
- u
- v
- x
- w
- y
California

Race
Black		69%
Other or unknown		32%

Ethnicity
Hispanic		19%
Not Hispanic or Latino		81%

Study Details

Setting

The study was conducted in two school districts in the San Francisco Bay Area in California. Three Knowledge is Power Program (KIPP) schools were included in the intervention group in the analysis. The two districts that these three schools belonged to were kept anonymous for the impact analysis section of the study.

Study sample

There were 263 fifth grade KIPP students included in the analytic sample. On average 11% were Latino, 78% were African American, 8% were English learners, 14% were special education students, 81% were eligible for free or reduced-price lunches, and 49% were female. The average age in months of these students was 123.67. The fifth grade comparison group students had similar demographic characteristics, as these characteristics were used in the propensity score matching process. The sixth grade students included in the analytic sample were broadly similar to the fifth grade students in terms of demographic characteristics. At School A, 26 sixth graders were included in the analysis. On average 19% were Latino, 73% were African American, 15% were English learners, 4% were special education students, 81% were eligible for free or reduced-price lunches, 62% were female, and the average starting age in months was 122.94. At School B, 25 sixth graders were included in the analysis. On average 44% were Latino, 36% were African American, 44% were English learners, 8% were special education students, 84% were eligible for free or reduced-price lunches, 52% were female, and the average starting age in months was 123.62. At School C, 19 sixth graders were included in the analysis. On average 16% were Latino, 68% were African American, 16% were English learners, 5% were special education students, 74% were eligible for free or reduced-price lunches, 58% were female, and the average starting age in months was 123.95. The sixth grade comparison group students had similar demographic characteristics, as these characteristics were used in the propensity score matching process.

Intervention Group

The intervention consisted of one year of attendance at a KIPP school in the Bay Area in California. Only students who attended the full year were included in the treatment sample. Each KIPP school adheres to a set of five principals: "Choice and commitment" highlights that students and parents choose to attend a KIPP school and follow its program. "High expectations" are maintained for each student through a system of rewards and consequences for behavior. KIPP schools have "more time" than other schools, with school days lasting at least 9 hours and Saturday school during the summer and at least 85 minutes daily is spent on ELA and mathematics. KIPP school leaders have the "power to lead" meaning that they have total control over their budgets and personnel. Students at KIPP schools are supposed to have "focus on results" by performing well on standardized tests and preparing for high school and college. Teachers joining KIPP schools in the sample generally came from highly selective colleges, were alternatively certified, and had a median of 3 years of classroom experience.

Comparison Group

Students in the comparison group experienced business-as-usual instruction at other schools in the district. Students who attended a KIPP school but transferred to another school in the district are excluded from the comparison group.

Support for implementation

The Bay Area KIPP schools raise between $400,000 and $700,000 each year to cover the gap between operating costs and the money they receive from state and local funds. The KIPP Foundation provides support by helping with teacher recruitment, fundraising, and other logistics. KIPP school leaders have substantial control over teacher hiring and their schools in general.

Knowledge is Power Program (KIPP) Intervention Report - Charter Schools

Review Details

Reviewed: January 2018

At least one finding shows moderate evidence of effectiveness

At least one statistically significant positive finding

Meets WWC standards with reservations

For:

Knowledge is Power Program (KIPP) Intervention Report - Charter Schools

Using:

Research Design:

Quasi-Experimental Design

Rating:

Meets WWC standards with reservations because it uses a quasi-experimental design in which the analytic intervention and comparison groups satisfy the baseline equivalence requirement.

Findings

English language arts achievement outcomes—Statistically significant positive effect found for the domain
Outcome measure	Comparison	Period	Sample	Intervention mean	Comparison mean	Significant?	Improvement index
California Standards Test English/Language Arts (CST-ELA)- scaled score	Knowledge is Power Program (KIPP) vs. Business as usual	1 Year	Aggregated sample; Grade 6; 810 students	339.85	313.33	Yes	17
California Standards Test English/Language Arts (CST-ELA)- scaled score	Knowledge is Power Program (KIPP) vs. Business as usual	1 Year	Aggregated sample; grade 5; 2,725 students	325.47	316.39	Yes	8
Show Supplemental Findings
California Standards Test English/Language Arts (CST-ELA)- scaled score	Knowledge is Power Program (KIPP) vs. Business as usual	0 Years	Aggregated sample; Grades 5 and 6; 3,535 students	328.49	315.68	Yes	11

General Mathematics Achievement outcomes—Statistically significant positive effect found for the domain
Outcome measure	Comparison	Period	Sample	Intervention mean	Comparison mean	Significant?	Improvement index	Evidence tier
California's State Standardized Assessment in Mathematics	Knowledge is Power Program (KIPP) vs. Business as usual	1 Year	Aggregated sample; Grade 5; 2,725 students	331.91	298.76	Yes	21
California's State Standardized Assessment in Mathematics	Knowledge is Power Program (KIPP) vs. Business as usual	1 Year	Aggregated sample; Grade 6; 810 students	349.91	311.29	Yes	19

Evidence Tier rating based solely on this study. This intervention may achieve a higher tier when combined with the full body of evidence.

Sample Characteristics

Characteristics of study sample as reported by study author.

12% English language learners
Female: 53%
Male: 47%
Urban
- B
- A
- C
- D
- E
- F
- G
- I
- H
- J
- K
- L
- P
- M
- N
- O
- Q
- R
- S
- V
- U
- T
- W
- X
- Z
- Y
- a
- h
- i
- b
- d
- e
- f
- c
- g
- j
- k
- l
- m
- n
- o
- p
- q
- r
- s
- t
- u
- v
- x
- w
- y
California

Race
Black		75%
Other or unknown		25%

Ethnicity
Hispanic		13%
Not Hispanic or Latino		87%

Study Details

Setting

The study was conducted in two unnamed school districts in the San Francisco Bay Area in California. Three KIPP schools were included in the intervention group in the analysis.

Study sample

The study used a matched-student quasi-experimental design, where the intervention group was comprised of students at five KIPP middle schools, and the comparison group was a sample of students matched on baseline reading and math test scores; gender, race, special education, limited English proficiency, and free or reduced-price lunch status; and whether the student repeated a grade in the baseline year. There were 263 fifth-grade KIPP students included in the analytic sample. On average, 11% were Latino, 78% were African American, 8% were English learners, 14% were special education students, 81% were eligible for free or reduced-price lunches, and 49% were female. The average age in years of these students was 10.3. Among the 810 sixth-grade students (70 KIPP students and 740 comparison students) included in the analytic sample, 26% were Latino, 56% were African American, 28% were English learners, 7% were special education students, 86% were eligible for free or reduced-price lunches, and 59% were female.

Intervention Group

The intervention consisted of 1 year of attendance at one of three KIPP schools in the Bay Area in California. Only students who attended the full year were included in the intervention group sample. Each KIPP school adheres to a set of five principals: “Choice and commitment” highlights that students and parents choose to attend a KIPP school and follow its program. “High expectations” are maintained for each student through a system of rewards and consequences for behavior. KIPP schools have “more time” than other schools, with school days lasting at least 9 hours and Saturday school during the summer, and at least 85 minutes daily is spent on English language arts and mathematics. KIPP school leaders have the “power to lead” meaning that they have total control over their budgets and personnel. Students at KIPP schools are supposed to have “focus on results” by performing well on standardized tests and preparing for high school and college. Teachers joining KIPP schools in the sample generally came from highly selective colleges, were alternatively certified, and had a median of 3 years of classroom experience. The sixth-grade analysis includes students who joined one of the three study schools during their sixth-grade year.

Comparison Group

Support for implementation