Project Activities
The research team completed three interconnected, exploratory studies.
- In Study 1, the researchers assessed and described Boston Public School's P-3 policies and practices that may be associated with student experiences. The researchers examined the intended program (e.g., goals and outcomes, instructional content and dosage, teacher practices, and instructional approach), the implementation system (e.g., staff recruitment/compensation, training and supervision, and assessment), aspects of the school district's culture and climate (e.g., staff autonomy, respect and leadership), and attributes and characteristics of staff, families, and students being served.
- For Study 2, the researchers addressed curricular alignment across the early grades; the quality of classroom interactions between teachers, students, and peers; the quality of the language, literacy and math environment; and teacher characteristics (such as buy-in and internalization of district policies, beliefs about children, morale and stress) that may be related to student outcomes. The researchers also examined a set of classroom population characteristics (percentage of dual language learners, low-income students, students who attended pre-k, and students with average or above early skills).
- For Study 3, the researchers examined associations between classroom processes and practices, family engagement, the home environment, afterschool and summer activities, and children's developmental skill trajectory and third grade outcomes. The research team also worked with the Assessment Team to develop a new classroom observation tool to assess structural and process features of pre-k and kindergarten to third grade classrooms that are associated with child outcomes.
Structured Abstract
Setting
This project took place in Boston, Massachusetts in the Boston Public Schools (BPS) early elementary schools and community-based (CB) pre-k programs.
Sample
For Study 1, the sample included up to 333 teachers and 30 school administrators from selected elementary schools and childcare providers, as well as key informants from the school district. For Studies 2 and 3, the pre-kindergarten sample included 20 elementary schools (40 pre-k classrooms), 10 community-based pre-k programs (10 classrooms), and 305 students. For Studies 2 and 3, the kindergarten to third grade sample included 80 classrooms and 567 students (305 from pre-k classrooms and 262 children who did not attend a pre-k program) each year.
The study examined existing policies and programs. The COVID-19 pandemic interrupted follow-up data collection in students’ 2nd- 4th grade years.
Research design and methods
The researchers conducted three complementary studies over a five-year period. For Study 1 (descriptive study of policies and practices), the research team reviewed documents and survey administrative staff in year 1, surveyed parent in years 1 and 2, reviewed administrative records in years 3 and 5, interviewed district personnel in years 1–5, and surveyed coaches and teachers in years 1–5. In Study 2, the research team observed classrooms each school year. For Studies 2 and 3, the researchers directly assessed children in fall and spring of the pre-k (year 1) and kindergarten years (year 2), and in spring of each school year in grades 1 (year 3) and 5 (year 7). For a subset of Spanish-speaking students, the research team assessed children's vocabulary and math skills in both English and Spanish in pre-k and kindergarten. Teachers reported on children's internalizing and externalizing behaviors, social skills, and learning-related skills each year. The researchers obtained school records data, such as grades, grade placement, attendance, and special education services from the school district. In addition to the site-specific program of research, the research team also worked with the Assessment Team to develop, pilot test, and validate a classroom observation tool.
Control condition
Due to the nature of the research design, there was no control condition.
Key measures
The researchers used the Classroom Assessment Scoring System to assess the quality of classroom interactions and the Individualizing Student Instruction and the Teacher Behavior Rating Scale measures to assess instructional aspects of the classroom environment. Teachers completed a demographic questionnaire and several surveys, including the Modernity Scale, the Job-Role Quality scale, the adaptability and autonomy subscales of the TCU Organization Readiness for Change assessment, and items from the NCES Schools and Staffing Survey about teachers' network of support. Teachers also reported on enthusiasm and support for BPS policies and practices. BPS coaches used curriculum fidelity measures from the Building Blocks and Opening the World of Language curricula to collect data on curricular focus, use, and fidelity of implementation. School district collected data using the Phonological Awareness Literacy Screening in pre-k, the Dynamic Indicators of Basic Early Literacy Skills, and the Expressive Vocabulary Test (pre-k and K). The research team used the Applied Problems and Picture Vocabulary subscales of the Woodcock-Johnson, the Tools for Early Assessment in Math, the Renfrew bus story the Hearts and Flowers task, and Forward and Backward Digit Span to assess children's academic and social behavioral skills. Equivalent subscales of the Batería II Woodcock-Muñoz were used at the pre-k and kindergarten time points to assess Spanish-speaking dual language learners. The Child Behavior Checklist, the Social Skills Rating System, and the Cooper Farran Behavioral Rating Scale were used to collect teacher-report of students' problem behaviors, social skills, and learning-related skills (e.g., self-regulation and cooperation). The research team also obtained school records information.
Data analytic strategy
For Study 1, the researchers catalogued, compiled and extracted key themes from field notes collected via interviews, discussions with key informants, and document review. The research team used this qualitative data to: 1) characterize the alignment of supports, implementation barriers, and facilitators of BPS policies; 2) describe the content and skills emphasized in the policies, goals, curriculum, and assessments; 3) outline coach and teacher recruitment; 4) describe professional development plans for teachers within and across grades and, specifically, the instructional approach for dual language learners; and 5) determine how students are assigned to classrooms within schools. For Studies 2 and 3, researchers used multilevel modeling to estimate the relative importance of particular malleable factors on students' academic and self-regulatory outcomes, developmental trajectories in these skills over time, and outcomes in fifth grade. This analytic technique was used to account for the nesting of students within classrooms, and classrooms within schools, and adjusted for a pre-specified set of school, classroom, and student characteristics. Researchers addressed questions about the relative importance of particular malleable features on student outcomes using moderation and subgroup analysis for groups of students defined by student background and classroom characteristics of interest.
People and institutions involved
IES program contact(s)
Project contributors
Products and publications
The research team produced study findings and develop databases to inform future policy and practice in early childhood programs and early elementary school classrooms. The researchers disseminated their research findings to a range of audiences, including early childhood practitioners, elementary school personnel, policymakers, and other researchers.
Project website:
Publications:
ERIC Citations: Publications from this project available in ERIC can be found here.
Select Publications:
McCormick, M.P., Hanno, E., MacDowell, C., Weiland, C., Hsueh, J., Pralica, M., Maves, S., Snow, C., Sachs, J. (2025). Moving Beyond Point in Time Estimates: Using Growth Models to Understand When PreK Convergence Happens, How, and for Which Skills. Child Development.
McCormick, M.P., MacDowell, C., Weiland, C., Hsueh, J., Pralica, M., Maier, M., Maves, S., Snow, C., Sachs, J. (2024). Instructional alignment is associated with PreK persistence: Descriptive evidence from the Boston Public Schools. Early Childhood Research Quarterly, 67, 89-100.
Wu, T., Weiland, C., McCormick, M.P., Hsueh, J., Snow, C., & Sachs, J. (2024). One score to rule them all? Comparing the predictive and concurrent validity of 30 Hearts and Flowers scoring approaches. Assessment, 0(0). https://doi.org/10.1177/10731911241229566
Thompson, M.E., Weiland, C., McCormick, M.P., Hsueh, J., Snow, C., & Sachs, J. (2023). The role of early schooling in shaping inequality in academic, executive functioning, and social-emotional skills. Socius, 9. https://doi.org/10.1177/23780231231199375
Weiland, C., McCormick, M.P., Duer, J., Friedman-Krauss, A., Pralica, M., Xia, S., Nores, M., & Mattera, S. (2024). Mixed-delivery public prekindergarten: Differences in demographics, quality, and children’s gains in community-based versus public school programs across five large-scale systems. Early Childhood Research Quarterly, 68, 247-259.
McCormick, M.P., Pralica, M., Hsueh, J., Weiland, C., Weissman, A., Xia, S., Shapiro, A., MacDowell, C., Maves, S., Taylor, A., & Sachs, J. (2023). Going the distance: Exploring variation in access to high-quality PreK by geographic proximity, race/ethnicity, family income, and home language. AERA Open, 9. https://doi.org/10.1177/23328584231168867
Moffett, L., Weiland, C., McCormick, M.P., Hsueh, J., Snow, C., & Sachs, J. (2023). Off-task behavior as a measure of in-classroom executive function skills? Evidence for construct validity and contributions to gains in prekindergartners’ cognitive skills. Early Education and Development, 1 – 24.
Weiland, C., Moffett, L., Guerrero-Rosada, P., Weissman, A., Zhang, K., Maier, M., Snow, C., McCormick, M.P., Hsueh, J., & Sachs, J. (2023). Is child-level measurement the key to improving the predictive validity of observational measures in early education classrooms? Early Childhood Research Quarterly, 313-326.
Moffett, L., Weissman, A., McCormick, M.P., Weiland, C., Hsueh, J., Sachs, J., & Snow. C. (2023). Enrollment in PreK and children’s social-emotional and executive functioning skills: To what extent are associations sustained across time? Journal of Educational Psychology, 115(3), 460-474.
McCormick, M.P., Pralica, M., Weiland, C., Hsueh, J., Moffett, L., Guerrero-Rosada, P., Weissman, A., Zhang,, K., Snow, C., Maier, M., Davies, E., Taylor, A., & Sachs, J. (2022). Does kindergarten instruction matter for sustaining the PreK boost? Evidence from individual- and classroom-level survey and observational data. Developmental Psychology. 58(7), 1298–1317.
Maier, M., McCormick, M.P., Xia, S., Hsueh, J., Weiland, C., Sachs, J., Boni, M., Morales, M., Tonachel, M., & Snow, C. (2022). Content-rich instruction and cognitive demand in PreK: Using systematic observations to predict child gains. Submitted to Early Childhood Research Quarterly, 60(3), 96-109.
Weiland, C., Sachs, J., McCormick, M.P., Hsueh, J., & Snow, C. (2021). Fast response research to answer practice and policy questions. Future of Children, 31(1), 75 – 96.
Moffett, L., Weissman, A., Weiland, C., McCormick, M.P., Hsueh, J., Sachs, J., & Snow, C. (2021). Unpacking Pre-K classroom organization: Types, variation, and links to school readiness gains. Journal of Applied Developmental Psychology, 77, 101346. https://doi.org/10.1016/j.appdev.2021.101346.
McCormick, M.P., Weiland, C., Hsueh, J., Pralica, M., Ketner, A., Snow, C., & Sachs, J. (2021). Is skill type the key to the pre-K fadeout puzzle? Differential associations between enrollment in pre-K and constrained and unconstrained skills across kindergarten. Child Development, 92(4), 599 – 620. doi: 10.1111/cdev.13520.
McCormick, M.P., Pralica, M., Guerrero, P., Weiland, C., Condliffe, B., Hsueh, J., Sachs, J., & Snow, C. (2021). Can center-based care reduce summer slow-down prior to kindergarten? Variation by family income, race/ethnicity, and Dual Language Learner status. American Education Research Journal, 58(2), 420 – 455.doi: 10.3102/0002831220944908.
Guerrero-Rosada, P., Weiland, C., McCormick, M.P., Hsueh, J., Sachs, J., Snow, C., & Maier, M. (2021). Process quality and children’s language, math, and executive function gains in prekindergarten: A replication and extension study. Early Childhood Research Quarterly, 54(1), 1 – 12. doi: 10.1016/j.ecresq.2020.07.009.
McCormick, M.P., Weiland, C., Hsueh, J., Maier, M., Hagos, R., Sachs, J., Snow, C., Leacock, N., & Schick, L. (2020). Promoting content-enriched alignment across the early grades: A review of policies in the Boston Public Schools. Early Childhood Research Quarterly, 52(3), 57 – 73. doi: 10.1016/j.ecresq.2019.06.012.
McCormick, M.P., Ketner, A., Weiland, C., Hsueh, J., Sachs, J., & Snow, C. (2020). Time well spent: Parents’ home-based learning practices and children’s academic skills. Developmental Psychology, 56(4), 710-726. doi: 10.1037/dev0000891.
Harding, J.F., McCoy, D., & McCormick, M.P. (2020). Building policies to support effective transitions from PreK to elementary school. Introduction to the Early Childhood Research Quarterly, 52(3), 1 – 4. doi: 10.1016/j.ecresq.2019.12.007.
Policy briefs and reports:
Wu, T., McCormick, M., Weiland, C., Hsueh, J., Sachs, J., & Snow, C. (2023, January). What sustains the Pre-K boost?: New evidence from Boston Public Schools. University of Michigan Education Policy Initiative Policy Brief. https://edpolicy.umich.edu/research/what-sustains-pre-k-boost-new-evidence-boston-public-schools
Weiland, C., McCormick, M., & Friedman-Krauss, A. (2022, December). Equity in mixed-delivery prekindergarten systems requires new investments and new thinking. Brookings Institute Brown Center Chalkboard. https://www.brookings.edu/blog/brown-center-chalkboard/2022/12/06/equity-in-mixed-delivery-prekindergarten-systems-requires-new-investments-and-new-thinking/
Weiland, C., McCormick, M.P., Duer, J., Friedman-Krauss, A., Pralica, M., Xia, S., Nores, M., & Mattera, S. Mixed-delivery public prekindergarten: Differences in demographics, quality, and children’s gains in community-based versus public school programs across five large-scale systems. Annenberg Working Paper. https://www.edworkingpapers.com/sites/default/files/ai22-651.pdf
Weiland, C. & Guerrero-Rosada, P. (2022, April). Widely used measures of pre-K classroom quality: What we know, gaps in the field, and promising new directions. MDRC policy brief. https://www.mdrc.org/sites/default/files/Widely_Used_Measures.pdf
McCormick, M., & Mattera, S. (2022, February). Learning more by measuring more: Building better evidence on pre-k programs by assessing the full range of children’s skills. MDRC policy brief. https://www.mdrc.org/publication/learning-more-measuring-more.
Weiland, C., Sachs, J., McCormick, M., Hsueh, J., Snow, C., & Taylor, A. (2022, January). Balancing rigor and timeliness in the time of COVID-19. Future of Children Research Briefs Series. https://futureofchildren.princeton.edu/sites/ futureofchildren/files/weiland_et_al.pdf
Pianta, R., McCormick., M.P., Burchinal, M., Peisner-Feinberg, E., Purtell, K., Knoche, L., & Ludvik, D. (2021, May). Sustaining the Pre-K boost: Skill type matters. IES Early Learning Network Policy Brief. https://earlylearningnetwork.unl.edu/wp-content/uploads/2021/05/210527-Policy-Brief_Sustaining_Pre-K_Boost.pdf.
McCormick, M.P. & Sengal, A. (2021, March). How should school districts spend their summer? The case for investing pandemic relief funds in pre-K and kindergarten summer programs. New America Education Policy Blog. https://www.newamerica.org/education-policy/edcentral/districts-spend-summer-k/.
McCormick, M.P. & Huang, S. (2021, March). Analysis: Two strategies for states’ COVID relief funds – Make early childhood programs better and make sure low-income parents know about them. Essay posted by the 74 million. https://www.the74million.org/article/analysis-2-strategies-for-states-covid-relief-funds-make-early-childhood-programs-better-and-make-sure-low-income-parents-know-about-them/.
Sengal, A., McCormick, M.P., & Castleman-Smith, J. (2021, March). Summer learning in early childhood: Opportunities for investing in equity. MDRC Issue Focus.
McCormick, M.P., & Weiland, C. (2021, February). Going big for little kids: Why kindergarten is critical in the COVID-19 recovery. New America Education Policy Blog. https://www.newamerica.org/education-policy/edcentral/kindergarten-critical-recovery/.
Vitiello, V., McCormick, M.P., Purtell, K., Prokasky, A., Iruka, I., & Ludvik, D. (2020, November). Bridging the gap: Easing the transition from pre-K to kindergarten. IES Early Learning Network Policy Brief. https://earlylearningnetwork.unl.edu/wp-content/uploads/2020/11/201105-Policy-Brief-PreK-K-Transition.pdf.
McCormick, M.P. (2020). Investing in equitable access to high-quality early care and education. MDRC Issue Focus.
“Q&A about measuring quality in pre-k classrooms,” New America Foundation, by Aaron Loewenberg, September 9, 2020: https://www.newamerica.org/education-policy/edcentral/q-about-measuring-quality-pre-k-classrooms/
Maier, M.F., Hsueh, J., & McCormick, M.P. (2020, September). Rethinking classroom quality: What we know and what we are learning. MDRC Issue Focus.
McCormick, M.P. & Pralica, M. (2020, August). Individual growth modeling: Application and implications for research in educational settings. MDRC Reflections on Methodology Blog Post.
Weiland, C., McCormick, M.P., Hsueh, J., Sachs, J., Snow, C., Unterman, R., & Shapiro, A. (2019, November). Instructional alignment for sustaining the prekindergarten boost: Early findings from Boston. Post for National Institute of Early Education Research Blog.
McCormick, M.P. & Parise, L. (2019, August). Can aligned instructional prevent PreK fadeout? Evidence Matters Podcast. https://www.mdrc.org/pub/evidence-first-mdrc-podcast.
McCormick, M.P. (2019, August). Linking fidelity of implementation to outcomes in real-world settings. MDRC Reflections on Methodology Blog Post.
McCormick, M.P., Mattera, S., & Hsueh, J. (2019, July). Preschool to third grade alignment: What do we know and what are we learning? MDRC Issue Focus.
McCormick, M.P. & Maier, M. (2018, November). Working with practitioners to develop measures of implementation fidelity. MDRC Reflections on Methodology Blog Post.
McCormick, M.P., Hsueh, J., Weiland, C., & Bangser, M. (2017). The challenge of sustaining preschool impacts: Introducing ExCEL P-3, a study from the Expanding Children’s Early Learning
Additional project information
This project is a research team in the Early Learning Network.
Related projects
Questions about this project?
To answer additional questions about this project or provide feedback, please contact the program officer.
