Table B.1. NCEE-Funded Studies Using State Assessments: Study Background Information
| Num | Study Name (Short Form) | Organization(s) Conducting the Study | Nature of Intervention | Study Design/Unit of Random Assessment | Target Population/Subjects | Findings Available |
|---|---|---|---|---|---|---|
| 1 | Evaluation of the Impact of Charter School Strategies (Charter Schools)a | Mathematica Policy Research, Inc. (Mathematica) | Charter schools are public schools that have been granted autonomy over their operations and freedom from state and district regulations that govern other public schools; however, they must meet accountability standards. | Individual students are randomly assigned by the charter schools through their admission lotteries. Control students are those who applied but were not admitted. | Middle School - entering 5th, 6th, or 7th grade Cohort 1 has 20 schools and Cohort 2 has 23 schools | Not yet reported |
| 2 | Impact Evaluation of Enhanced Academic Instruction (EAI) For After-School Programs (After-School EAI)b | Manpower Development Research Corporation (MDRC), Public/Private Ventures, Bloom Associates, Survey Research Management | Two models providing 45 minutes of formal academic instruction in reading (Success for All's Adventure Island) or math (Harcourt Athletics) in after-school programs. | Random assignment is at the student level. Students were randomly assigned by grade within each after-school center. Control students receive regular after-school services. | 2nd to 5th grades | Year 1—2008 |
| 3 | An Impact Evaluation of Comprehensive Teacher Induction Programs (Teacher Induction)c | Mathematica; Center for Education Leadership, WestEd | Two high intensity induction programs were chosen—the Santa Cruz New Teacher Project by the New Teacher Center at the University of California-Santa Cruz, and the Pathwise Framework Induction Model by the Educational Testing Service in Princeton, NJ. A prominent feature of the models is the use of mentors who are trained extensively and released from teaching to devote an entire year to supporting new teachers. | Schools were randomly assigned to treatment group (received comprehensive teacher induction services) or control (took part in district's usual teacher induction program). | Analysis for math program includes 1,961 students; analysis for reading program includes 1,828 students 2nd to 6th grades Total sample is 210 treatment schools and 208 control schools | Year 1—2008 |
| 4 | Improving Adolescent Literacy Across the Curriculum in High Schools: An Evaluation of the Strategic Instruction Model’s Content Literacy Continuum (CLC)d | Midwest REL | CLC is a school-wide literacy-across-the-curriculum model that includes teaching practices to help teachers organize and present information and a literacy-focused curriculum that embeds literacy instruction in content-area instruction. | Random assignment was done at the school level, separately within each school district. | 9th grade There are 33 schools from 8 districts | Not yet reported |
| 5 | Evaluation of Principles-Based Professional Development to Improve Reading Comprehension for English Language Learners (Pacific CHILD)e | Pacific REL | Pacific CHILD adapted to teachers of English language arts for 4th and 5th grade ELL. Two-year professional development intervention. | Random assignment was done at the school level, with blocking to ensure that treatment and control groups were balanced in terms of size and location. | 4th and 5th grades 23 treatment schools and 23 control schools | Not yet reported |
| 6 | The Impact of Professional Development Models and Strategies on Teacher Practice and Student Achievement in Early Reading (Early Reading PD)f | American Institutes for Research (AIR), MDRC | Two professional development methods: a 5-day, content-focused summer institute with three days of follow-up through the school year; and the institute and follow-up days plus coaching by an in-school reading specialist trained in a particular coaching approach. | Random assignment was done at the school level, with equal numbers of schools assigned to a treatment A group, a treatment B group, and a control group that participated only in the district’s usual PD services. | 2nd grade There are 90 schools in six districts, with a total of 270 second grade teachers | Final—2008 |
| 7 | Lessons in Character Study (LIC)g | West REL | LIC is a comprehensive, schoolwide English language arts-based character education program that includes core reading and writing curricula and support materials that reinforce good character and language arts learning standards. | Random assignment is at the school level. | 2nd to 5th grades 15,000 students and their teachers at 50 schools (25 of which are receiving the intervention) | Not yet reported |
| 8 | A Study of the Effectiveness of the Success in Sight School Improvement Intervention (Success in Sight)h | Central REL | Success in Sight is a whole-school reform model emphasizing data-based decision making, research-based strategies for school improvement, community building, and shared leadership. | Schools will be randomly assigned within districts. | 3rd to 5th grades | Not yet reported |
| 9 | Eighth-Grade Access to Algebra I: A Study of Virtual Algebra (Virtual Algebra)i | Northeast REL | An online course in Algebra I taught by a certified algebra instructor. | Random assignment occurred at the school level, and schools were blocked based on state and size. | 8th grade 70 schools with a total of 479 Algebra-ready students and 2,081 total 8th grade students | Not yet reported |
| 10 | Impact Evaluation of the U. S. Department of Education’s Student Mentoring Program (Mentoring)j | Abt Associates; Branch Associates, Moore & Associates, Center for Resource Management (CRM) | The U.S. Department of Education's Student Mentoring Program is designed to assist local educational agencies and community-based organizations to promote mentoring programs for children with greatest need in the 4th through 8th grades. | The study uses random assignment at the student level. | 4th to 8th grades | Final – 2009 |
| Studies Using Data from One State | ||||||
| 11 | The Efficacy of the Measures of Academic Progress (MAP) and Its Associated Training on Differentiated Instruction and Student Achievement (MAP Assessment)k | Midwest REL | The MAP system includes a portfolio of assessment tests designed to be predictive of student performance on standardized tests and training in use of the tests to help teachers tailor instruction to individual student needs. | The study uses a cluster-randomized design with a delayed-treatment control group. Grade within school is the unit of assignment. Teachers in the control group will receive their schools’ customary form of PD and will receive the treatment two years later. | 4th and 5th grades 32 schools with a total of about 128 teachers | Not yet reported |
| 12 | The Effects of a Hybrid Secondary School Course in Algebra I on Teacher Practices, Classroom Quality and Adolescent Learning (Hybrid Algebra)l | Appalachia REL | Online Algebra 1 module customized to be combined with a traditional Algebra 1 course. Professional development and instructional supports provided by Kentucky Virtual High School for its hybrid teachers. | Random assignment is at the school level. | 9th grade 41 schools | Not yet reported |
| 13 | A Study of the Effects of Using Classroom Assessment for Student Learning (CASL)m | Central REL | CASL is a content-neutral professional development program to help teachers improve their use of formative assessment to help students succeed in school. | Schools will be randomly assigned to an intervention group or a control group that will continue with regular PD. | 4th and 5th grades 64 schools with three to six teachers in each school. | Not yet reported |
| 14 | The Effectiveness of the Alabama Math, Science, and Technology Initiative (AMSTI)n | Southeast REL | AMSTI promotes student achievement in math and science through improving pedagogical practices of teachers, integration of technology in science and math curricula, and provision of technology to schools. Alabama Dept. of Ed. awards grants for two years of professional development, technology-oriented tools and resources, and ongoing technical support from local colleges. | Schools were paired on the basis of grade configuration, math scores, percentage of students qualifying for free or reduced-price lunch, and the percentage of minority students. A coin toss determined which school in each pair would be in the treatment group. | 4th to 8th grades | Not yet reported |
| 15 | Understanding Science and the Academic Literacy of English Learners (Understanding Science)o | West REL | Understanding Force and Motion, a professional development course aimed at improving teacher science content knowledge and pedagogy. | Randomization will occur at the teacher level, nested within district/region. | 8th grade 120 total teachers, each teaching two physical science classes of 20-25 students, for a total sample of 4,800 students | Not yet reported |
| 16 | Quality Teaching for English Learners (QTEL)p | West REL | QTEL aims to improve the capacity of teachers to support the linguistic, conceptual, and academic development of ELLs by providing summer professional development sessions, individual coaching and classroom support, and collaborative planning sessions. | The study uses a cluster random assignment design with schools as the unit of randomization. | 6th to 8th grades 52 schools (26 treatment); approximately 600 teachers and 55,000 students | Not yet reported |
| 17 | Assessment Accommodations for English Language Learners (ELL Accommodations)q | West REL | The development of a modified math test to better assess the skills of ELL students without affecting assessment for non-ELL students (linguistic modification). | 7th and 8th grades | Not yet reported | |
| 18 | Closing the Reading Gap (Striving Readers)r | Florida State University (FSU), Corporation for the Advancement of Policy Evaluation | Four reading program interventions were selected: Corrective Reading, Failure Free Reading, Spell Read P.A.T., and Wilson Reading. | The study uses random assignment at two levels: schools were randomly assigned to one of the four interventions and within each school students were randomly assigned to the treatment or control groups. | 3rd and 5th grades 779 students | Final—2007 |
| Other Studies | ||||||
| 19 | Impact Evaluation of a School-Based Violence Prevention Program (Violence Prevention)s | RTI International, Pacific Institute for Research (PIRE), Tanglewood Research, Inc. | A curriculum-based violence-prevention program, Responding in Peaceful and Positive Ways (RIPP), and a whole-school program, Best Behavior, are implemented together. | 4th to 8th grades | Data collected for students entering the program in 2005-2006 and 2006-2007 | |
| 20 | The Impact of Professional Development Strategies on Teacher Practice and Student Achievement in Math (Math PD)t | AIR, MDRC | A math professional development model that focuses on student misconceptions in the areas of fractions, decimals, ratios, percentages, and proportions. The professional development consists of a three-day summer institute, five day-long seminars during the school year, and 10 days of additional coaching support | Not available | Not yet reported | |
| 21 | Impact Evaluation of the Upward Bound’s Increased Focus on Higher-Risk Studentsu | Abt Associates; Urban Institute; Berkeley Policy Associates | Study was cancelled in March 2008 | |||
|
NOTES: The sources of information reviewed on these studies include publicly available reports and summaries, as well as unpublished study design documents that were current as of November of 2008. Publicly available sources of information for each study are noted below. NCEE = National Center for Education Evaluation and Regional Assistance. SOURCES: a. U.S. Department of Education. "Evaluation of the Impact of Charter School Strategies." Institute of Education Sciences, National Center for Education Evaluation and Regional Assistance. Retrieved from http://ies.ed.gov/ncee/projects/evaluation/choice_charter.asp on October 6, 2009. b. Black, Alison R., Fred Doolittle, Pei Zhu, and others (2008). The Evaluation of Enhanced Academic Instruction After-School Programs: Findings After the First Year of Implementation (NCEE 2008-4022). Washington, DC: National Center for Education Evaluation and Regional Assistance, Institute of Education Sciences, U.S. Department of Education. c. Glazerman, Steven, Sarah Dolfin, Martha Bleeker, and others. (2008). Impacts of Comprehensive Teacher Induction: Results from the First Year of a Randomized Controlled Study (NCEE 2009-4034). Washington, DC: National Center for Education Evaluation and Regional Assistance, Institute of Education Science, U.S. Department of Education. d. U.S. Department of Education. "Improving Adolescent Literacy Across the Curriculum in High Schools (Content Literacy Continuum, CLC)." Institute of Education Sciences, National Center for Education Evaluation and Regional Assistance. Retrieved from http://ies.ed.gov/ncee/edlabs/projects/project.asp?projectID=34 on October 6, 2009. e. U.S. Department of Education. "Evaluation of Principles-Based Professional Development to Improve Reading Comprehension for English Language Learners." Institute of Education Sciences, National Center for Education Evaluation and Regional Assistance. Retrieved from http://ies.ed.gov/ncee/edlabs/projects/project.asp?projectID=61 on October 6, 2009. f. Garet, Michael S., Stephanie Cronen, Marian Eaton, Anja Kurki, Meredith Ludwig, Wehmah Jones, Kazuaki Uekawa, Audrey Falk, Howard Bloom, Fred Doolittle, Pei Zhu, and Laura Sztejnberg. The Impact of Two Professional Development Interventions on Early Reading Instruction and Achievement (NCEE 2008-4030). Washington, DC: National Center for Education Evaluation and Regional Assistance, Institute of Education Sciences, U.S. Department of Education. g. U.S. Department of Education. "Effects of the Lessons in Character English Language Arts Character Education Program on Behavior and Academic Outcomes." Institute of Education Sciences, National Center for Education Evaluation and Regional Assistance. Retrieved from http://ies.ed.gov/ncee/edlabs/projects/rct_91.asp on October 6, 2009. h. U.S. Department of Education. "The Effects of Success in Sight as a School Improvement Intervention." Institute of Education Sciences, National Center for Education Evaluation and Regional Assistance. Retrieved from on October 6, 2009. i. REL Northeast and Islands. "Eighth Grade Access to Algebra I: A Study of Virtual Algebra." Retrieved from http://www.virtualalgebrastudy.org/documents/Virtual_Algebra_Study_033108.pdf on October 6, 2009. j. Bernstein, L., Dun Rappaport, C., Olsho, L., Hunt, D., and Levin, M. (2009). Impact Evaluation of the U.S. Department of Education's Student Mentoring Program (NCEE 2009-4047). Washington, DC: National Center for Education Evaluation and Regional Assistance, Institute of Education Sciences, U.S. Department of Education. k U.S. Department of Education. "Efficacy of Frequent Formative Assessment for Improving Instructional Practice and Student Performance, Given Variations in Training to Use Assessment Results." Institute of Education Sciences, National Center for Education Evaluation and Regional Assistance. Retrieved from http://ies.ed.gov/ncee/edlabs/projects/project.asp?projectID=36 on October 6, 2009. l. U.S. Department of Education. "The Effects of Hybrid Algebra I on Teaching Practices, Classroom Quality, and Adolescent Learning." Institute of Education Sciences, National Center for Education Evaluation and Regional Assistance. Retrieved from http://ies.ed.gov/ncee/edlabs/projects/rct_8.asp on October 6, 2009. m. U.S. Department of Education. "The Effects of Classroom Assessment for Student Learning (CASL) on Student Achievement." Institute of Education Sciences, National Center for Education Evaluation and Regional Assistance. Retrieved from http://ies.ed.gov/ncee/edlabs/projects/rct_18.asp on October 6, 2009. n. U.S. Department of Education. "The Effectiveness of the Alabama Math, Science, and Technology Initiative (AMSTI)." Institute of Education Sciences, National Center for Education Evaluation and Regional Assistance. Retrieved from http://ies.ed.gov/ncee/edlabs/projects/project.asp?projectID=69 on October 6, 2009. o. U.S. Department of Education. "Impact of the Understanding Science Professional Development Model on Science Achievement of English Language Learner Students." Institute of Education Sciences, National Center for Education Evaluation and Regional Assistance. Retrieved from http://ies.ed.gov/ncee/edlabs/projects/rct_87.asp on October 6, 2009. p. U.S. Department of Education. "Quality Teaching for English Learners (QTEL)." Institute of Education Sciences, National Center for Education Evaluation and Regional Assistance. Retrieved from http://ies.ed.gov/ncee/edlabs/projects/project.asp?projectID=88 on October 6, 2009. q. U.S. Department of Education. "Effect of Linguistic Modification of Math Assessment Items on English Language Learner Students." Institute of Education Sciences, National Center for Education Evaluation and Regional Assistance. Retrieved from http://ies.ed.gov/ncee/edlabs/projects/rct_92.asp on October 6, 2009. r. Torgesen, J., Schirm, A., Castner, L., Vartivarian, S., Mansfield, W., Myers, D., Stancavage, F., Durno, D., Javorsky, R., and Haan, C. (2007). National Assessment of Title I, Final Report: Volume II: Closing the Reading Gap, Findings from a Randomized Trial of Four Reading Interventions for Striving Readers (NCEE 2008-4013). Washington, DC: National Center for Education Evaluation and Regional Assistance, Institute of Education Sciences, U.S. Department of Education. s. U.S. Department of Education. "Impact Evaluation of a School-Based Violence Prevention Program." Institute of Education Sciences, National Center for Education Evaluation and Regional Assistance. Retrieved from http://ies.ed.gov/ncee/projects/evaluation/other_violence.asp on October 6, 2009. t. U.S. Department of Education. "The Impact of Professional Development Strategies on Teacher Practice and Student Achievement in Math." Institute of Education Sciences, National Center for Education Evaluation and Regional Assistance. Retrieved from http://ies.ed.gov/ncee/projects/evaluation/tq_mathematics.asp on October 6, 2009. u. U.S. Department of Education. "Impact Evaluation of the Upward Bound's Increased Focus on Higher-Risk Students." Institute of Education Sciences, National Center for Education Evaluation and Regional Assistance. Retrieved from http://ies.ed.gov/ncee/projects/evaluation/upward.asp on October 6, 2009. | ||||||