Inside IES Research

Classroom teachers of young children face a seemingly never-ending challenge – how to manage disruptive behavior while simultaneously teaching effectively and supporting the needs of every student in the classroom. Researchers at Virginia Commonwealth University and the University of Florida have received five IES research grants over the past decade – three through the National Center for Special Education Research (NCSER) and two from the National Center for Education Research (NCER) – to develop and test a model of training and professional development, including coaching, for early childhood and early elementary school teachers on how best to support children who engage in disruptive and otherwise challenging classroom behaviors.

With their first IES grant in 2008, Drs. Maureen Conroy and Kevin Sutherland developed the original BEST in CLASS model for early childhood teachers. The goal of BEST in CLASS - PK is to increase the quantity and quality of specific instructional practices with young children (ages 3-5 years old) who engage in high rates of challenging behaviors with the ultimate goal of preventing and reducing problem behavior. Professional development consists of a six-hour workshop that uses didactic and interactive learning activities supported by video examples and practice opportunities. Following the workshop, teachers receive a training manual and 14 weeks of practice-based coaching in the classroom. 

The results of this promising development work led to a 2011 IES Efficacy study to test the impact of BEST in CLASS - PK on teacher practices and child outcomes. Based on positive findings from that Efficacy study the team was awarded two additional Development and Innovation grants – one in 2016 to develop a web-based version of BEST in CLASS – PK to increase accessibility and scalability and another in 2015 to adapt BEST in CLASS – PK for early elementary school classrooms (BEST in CLASS – Elementary). Drs. Sutherland and Conroy are currently in the second year of an Efficacy study to test the impact of BEST in CLASS - Elementary to determine if the positive effects of BEST in CLASS in preschool settings are replicated in early elementary classrooms.

Written by Emily Doolittle, NCER Team Lead for Social Behavioral Research, and Jacquelyn Buckley, NCSER Team Lead for Disability Research

In today’s diverse classrooms, it is both challenging and critical to gather accurate and meaningful information about student knowledge and skills. Certain populations present unique challenges in this regard – for example, English learners (ELs) often struggle on assessments delivered in English. On “typical” classroom and state assessments, it can be difficult to parse how much of an EL student’s performance stems from content knowledge, and how much from language learner status. This lack of clarity makes it harder to make informed decisions about what students need instructionally, and often results in ELs being excluded from challenging (or even typical) coursework.

Over the past several years, NCER has invested in several grants to design innovative assessments that will collect and deliver better information about what ELs know and can do across the PK-12 spectrum. This work is producing some exciting results and products.

• Jason Anthony and his colleagues at the University of South Florida have developed the School Readiness Curriculum Based Measurement System (SR-CBMS), a collection of measures for English- and Spanish-speaking 3- to 5-year-old children. Over the course of two back-to-back Measurement projects, Dr. Anthony’s team co-developed and co-normed item banks in English and Spanish in 13 different domains covering language, math, and science. The assessments are intended for a variety of uses, including screening, benchmarking, progress monitoring, and evaluation. The team used item development and evaluation procedures designed to assure that both the English and Spanish tests are sociolinguistically appropriate for both monolingual and bilingual speakers.

• Daryl Greenfield and his team at the University of Miami created Enfoque en Ciencia, a computerized-adaptive test (CAT) designed to assess Latino preschoolers’ science knowledge and skills. Enfoque en Ciencia is built on 400 Spanish-language items that cover three science content domains and eight science practices. The items were independently translated into four major Spanish dialects and reviewed by a team of bilingual experts and early childhood researchers to create a consensus translation that would be appropriate for 3 to 5 year olds. The assessment is delivered via touch screen and is equated with an English-language version of the same test, Lens on Science.

​

• A University of Houston team led by David Francis is engaged in a project to study the factors that affect assessment of vocabulary knowledge among ELs in unintended ways. Using a variety of psychometric methods, this team explores data from the Word Generation Academic Vocabulary Test to identify features that affect item difficulty and explore whether these features operate similarly for current, former, as well as students who have never been classified as ELs. The team will also preview a set of test recommendations for improving the accuracy and reliability of extant vocabulary assessments.

• Researchers led by Rebecca Kopriva at the University of Wisconsin recently completed work on a set of technology-based, classroom-embedded formative assessments intended to support and encourage teachers to teach more complex math and science to ELs. The assessments use multiple methods to reduce the overall language load typically associated with challenging content in middle school math and science. The tools use auto-scoring techniques and are capable of providing immediate feedback to students and teachers in the form of specific, individualized, data-driven guidance to improve instruction for ELs.

By leveraging technology, developing new item formats and scoring models, and expanding the linguistic repertoire students may access, these teams have found ways to allow ELs – and all students – to show what really matters: their academic content knowledge and skills.

Written by Molly Faulkner-Bond (former NCER program officer).

The Montessori method of education was developed over 100 years ago by Dr. Maria Montessori. This “whole child” approach centers around the theory that children are capable of initiating learning in a thoughtfully prepared environment that develops children’s physical, social, emotional, and cognitive growth. Core components of Montessori education are mixed age classrooms in three-year groupings (e.g., 3-6 year olds, 6-9, 9-12, etc.), a carefully prepared environment filled with appropriate materials and lessons, student freedom to select lessons and activities each day, and daily uninterrupted 3-hour work blocks.

According to the National Center for Montessori in the Public Sector (NCMPS), there are currently over 5,000 Montessori schools in the U.S., 500 of which are public schools and over 150 of which serve public preschool and kindergarten students.  Despite its growing popularity in public preschools and Head Start schools, no large-scale evaluation of the efficacy of the Montessori model on children’s academic, social, and emotional skills has been conducted. 

This year, IES funded the first such study. A project team led by Dr. Ann-Marie Faria and Ms. Karen Manship (American Institutes for Research) and Dr. Angeline Lillard (University of Virginia) will study more than 650 children for three years, beginning with their entry at age 3 into preschool. Importantly, this study relies on individual random student assignment via lottery entry to compare preschool students who enroll in Montessori at age 3 to those who are assigned to a waitlist control group (and thus are in other settings such as public PreK, daycare, or a home setting). Data will be collected in diverse urban and suburban school districts across the country, including Houston (TX), Hartford and New Haven (CT), and Washington, DC.

Researchers will examine the impact of preschool Montessori education on children’s academic, social, and emotional skills, as well as kindergarten readiness skills. The research team will also conduct a cost effectiveness study of the public Montessori preschool model, and will examine the effect of fidelity of implementation of Montessori on student outcomes. Collectively, the findings from this study will provide valuable evidence of the efficacy of Montessori preschool education. Ultimately, the researchers plan to disseminate their findings to educators, parents, and policymakers through research briefs, infographics, blog posts, and webinars.

By Amanda M. Dettmer, PhD, American Psychological Association Executive Branch Science Fellow/ AAAS Science & Technology Policy Fellowship

Photo credit: Marilyn Horan, Carroll Creek Montessori Public Charter School

Social and emotional learning (SEL) is a key ingredient of high-quality education care, is important for both educators and children, and has been associated with children’s concurrent and later academic and social success.

Over a decade ago, Yale University’s Center for Emotional Intelligence developed and began testing RULER, an SEL program geared toward children and educators (i.e., school leaders, teachers, and staff). RULER stands for five key social and emotional skills: Recognizing emotions in self and others, Understanding the causes of emotions in self and others, Labeling and talking about emotions, Expressing emotions across situations, and Regulating emotions effectively. For children and the key adults in their lives, RULER combines a whole-school professional development approach with a skill-building curriculum targeting educator and student social and emotional skills, school and classroom climate, and educator and student well-being. RULER is currently offered for pre-k–12 and out-of-school-time settings.

IES has supported the development and testing of RULER programs since 2012. The first IES award supported the modification of existing components of the RULER K-8^th grade intervention and creation of new developmentally appropriate content for preschool settings. RULER is currently implemented in over 200 early childhood school- and home-based programs across the country and nearly 2,000 K-12 schools nationwide. Although RULER’s evidence-base has been growing over the years, RULER has not been systematically studied in large-scale, randomized controlled trials in preschool settings nor has it undergone an external evaluation in the later grades.

That is about to change: this year, IES awarded two grants to study the effects of the RULER programs. One will study the efficacy of whole-school RULER implementation for preschool students (under the Early Learning Programs and Policies program), and the other will do so for grades K-6 (under the Social and Behavioral Context for Academic Learning program).

The Preschool RULER grant (PI: Craig Bailey, PhD) will assess school readiness in children aged 3-5, as well as outcomes at the teacher/classroom and school leader/school levels. The researchers will study 72 early childhood centers, including public, private, and Head Start programs from urban areas in Connecticut, using a multisite, cluster-randomized control trial design. Altogether, approximately 216 classrooms, 1,800 staff, and 2,160 children will participate. Children, educators, and school leaders will be assessed for social and emotional skills, and educators/leaders will be assessed for emotionally intelligent pedagogy and leadership. Children will also be assessed for their approaches to learning, pre-literacy, and pre-math skills. This study will provide evidence about the efficacy of RULER in preschool settings and contribute to our understanding of high quality early childhood interventions that promote social emotional learning.

The other grant, for RULER in grades K-6 (PI: Jason Downer, PhD), will be the first large-scale external evaluation of RULER. The study will take place in 60 urban and suburban public elementary schools, including 420 teachers and 2,520 K-6 students in Virginia. Key outcomes for this study will include school climate assessments (assessed by teacher and principal reports), teacher well-being (assessed by self-report), and four student outcomes: social-emotional skills, behavior, academic engagement and academic achievement (assessed by standardized assessments, tests, and attendance records). Ultimately, this study will describe RULER’s effects on school climate, teacher well-being, classroom climate, and student outcomes.

By Amanda M. Dettmer, AAAS Science & Technology Policy Fellow Sponsored by the American Psychological Association Executive Branch Science Fellowship

Photo credits: Yale Center for Emotional Intelligence

Identifying young children with language delays can improve later outcomes

Language is a core ability that children must master for success both in and out of the classroom. Extensive studies have shown that many tasks, including math, depend on linguistic skill, and that early language skills are predictive of school readiness and academic success. Being able to quickly identify children at early ages with language delays is crucial for targeting effective interventions.

Enter the QUILS.

In 2011, the National Center for Education Research (NCER) at IES funded a 4-year grant to Dr. Roberta Golinkoff (University of Delaware) and Drs. Kathy Hirsh-Pasek (Temple University) and Jill de Villiers (Smith College) to develop a valid and reliable computer-based language assessment for preschoolers aged 3-5 years old. The resulting product was the Quick Interactive Language Screener (QUILS), a computerized tool to measure vocabulary, syntax, and language acquisition skills. The assessment ultimately measures what a child knows about language and how a child learns, and automatically provides results and reports to the teacher.

The preschool version of QUILS is now being used by early childhood educators, administrators, reading specialists, speech-language pathologists, and other early childhood professionals working with young children to identify language delays. The QUILS is also being utilized in other learning domains. For example, a new study relied on the QUILS, among other measures, to examine links between approaches to learning and science readiness in over 300 Head Start students aged 3 to 5 years.

QUILS is now being revised for use with toddlers. In 2016, the National Center for Special Education Research (NCSER) funded a 3-year study to revise the QUILS for use with children aged 24-36 months. The researchers have been testing the tool in both laboratory and natural (child care centers, homes, and Early Head Start programs) settings to determine which assessment items to use in the toddler version of QUILS. Ultimately, these researchers aim to develop a valid and reliable assessment to identify children with language delays so that appropriate interventions can begin early.

By Amanda M. Dettmer, AAAS Science & Technology Policy Fellow Sponsored by the American Psychological Association Executive Branch Science Fellowship