Inside IES Research

Notes from NCER & NCSER

Evaluating Oregon’s Adult Basic Skills Transition Planning Process: An Interview with Judith Alamprese

In her 2017 IES grant, Judith Alamprese (Abt Associates) is collaborating with the state of Oregon’s Office Community College and Workforce Development to evaluate a program that aims to help adults earn a GED® and transition into postsecondary education. This project is funded under the Low-Cost, Short-Duration Evaluation of Education Interventions competition, which supports research that aims to produce meaningful results for local and/or state education agencies quickly. Program Officer, Meredith Larson, interviewed Ms. Alamprese about this current work, how it came into being, and what it might mean for Oregon and adult education more broadly.

Tell us about your area of research and why it’s important to Oregon.

My current research is focused on determining effective interventions for assisting low-skilled adults establish and succeed in a career pathway. Oregon was one of the first states to implement a statewide initiative for transitioning adult basic skills learners (henceforth adult learners) to further education and work, and this project expands Oregon’s activities to support adult learners’ success.  

What is your current project studying?

The Transition Planning Process (TPP) project is a collaboration between Oregon’s Office of Community College and Workforce Development (CCWD) and Abt Associates (Abt). We are using a randomized controlled trial (RCT) to test whether text messaging helps adult learners earn a GED® and transition to postsecondary education and training.

What is TPP, and why is this approach innovative?

TPP is a text messaging intervention in which transition facilitators who work with the adult learners send text messages to the learners to help keep them on track to complete their GED® and enroll in postsecondary courses. The intervention is a supplement to the facilitators’ other transition activities to prepare learners for next steps in education and work.

TPP has a standardized list of text messages to prompt learners to take the GED® tests, set college goals, access information on college planning and other college preparation activities. Facilitators can send texts customized to programs’ specific transition activities.

CCWD chose text messaging because it appeared to be a low-cost approach that could support existing transition activities and provide a boost to ABS learners. The TPP project is an exciting opportunity to determine whether texting can be effective with ABS learners, and may be a promising approach for encouraging specific behaviors in learners preparing to go to college.

How did this project come into being?

The TPP project grew out of Abt’s and CCWD’s work together on an IES Researcher-Practitioner Partnership grant. This grant was a longitudinal study of Oregon’s Pathways for Adult Basic Skills Transition to Postsecondary Education and Work initiative. The findings from Abt’s analyses of adult learners’ GED® attainment and postsecondary participation prompted Oregon to want to try some additional strategies to encourage ABS learners to earn a secondary credential and enroll in postsecondary courses.

What is the current status of the project?

The study is underway, and transition facilitators are providing text messages to encourage adult learners to initiate and complete GED® testing, determine next steps, and begin the postsecondary planning process. The facilitators have found that while many treatment group learners respond to the texts, some learners have chosen to increase their face-to-face interaction with their facilitators. The facilitators report that texting is an efficient way to reinforce learners and check on their progress.

Why is this work important?

This research is particularly important because it is a rigorous test of an intervention that could be beneficial to adult basic skill learners nationwide and could leverage such programs’ existing activities in transitioning learners from basic skills programs to further education and training. We will learn more about the types of information and support that are most persuasive in helping learners succeed.

New IES Grantee Focuses on Improving Adult Literacy

In her first IES grant, Dr. Elizabeth Tighe (Georgia State University) is taking expertise honed in both an NCER predoctoral fellowship and PIAAC methods training program to help further adult literacy research. Her earlier work includes developing assessments for adults with low literacy, leveraging statistical approaches to understand these adults’ abilities and difficulties, and using eye-tracking paradigms to explore their ability to self-monitor during reading. 

Program officer, Meredith Larson, interviewed Dr. Tighe about her previous work and new grant.

What is your general area of research, and why is it important?

I focus on adult struggling readers, which comprises roughly 36 million (1 in 6) adults in the U.S. Only a fraction of these adults enroll in adult education programs, which are plagued by insufficient funding, high teacher turnover rates, and a lack of research-based instructional practices and curricula. By better understanding these adults’ strengths and deficits and how best to measure their skills, I aim to inform and improve adult education programs.

What could people do with your research?

My research could directly inform how we help adults become stronger readers, and this can improve educational outcomes, such as GED attainment. I am working towards building better assessments for adult education practitioner and researcher use. My longer-term goal is to design a curriculum to teach morphology (e.g., prefixes and suffixes) and use this to improve adults’ vocabulary and reading comprehension.

What are you trying to learn through your new IES project?

For this grant, I’m using the Program for the International Assessment of Adult Competencies (PIAAC), a large-scale, international assessment of adult literacy, numeracy, and digital problem-solving skills, to create risk profiles of adults with low literacy skills. This sort of information could move us closer to being able to individualize instruction in adult education programs to match the needs of specific learners.

We will use PIAAC data to explore how demographic characteristics (age, race/ethnicity, educational background, employment status) and malleable factors (enjoyment of learning, frequency of computer use, reading and writing behaviors at home and at work) influence low literacy performance.

Further, we are examining whether risk factors differ by whether someone has a high school diploma and whether someone has participated in education or training recently. We will also explore whether reading components and literacy skills are predictive of low-skilled adults’ numeracy skills.

Our findings could have important implications for understanding risk factors and predictors of low literacy as well as low numeracy. As stated previously, 1 in 6 U.S. adults have low literacy skills and nearly 1 in 3 have low numeracy skills. For GED attainment, adults must demonstrate proficiency in both of these areas (along with science, social studies, and writing knowledge). It’s important to have targeted, individualized instruction for these adults because they may have time or resource barriers.

How did this particular research project arise?

I first learned about PIAAC at a summer institute. I was intrigued, in particular, because PIAAC is the first assessment of this size to include a reading component supplement for lower-skilled adults.

I recently attended a 3-day NCER/ETS PIAAC training workshop, which allowed me to work with PIAAC data and network with others. This workshop influenced my decision to apply for an IES grant. I felt that a 2-year grant using extant data would be a great way to combine my interests regarding individual differences in adults’ component skills and get my feet wet with IES as a new investigator. I am excited to bridge my interests and grow as a researcher by learning and working alongside two experts (Drs. Yaacov Petscher and John Sabatini) in the larger reading and education field!

The Value of Partnerships for Studying English Learner Education

English learners (ELs) can be a tricky student population to study. In some ways, these students who are learning English as part of their education are a homogenous population. For example, more than 75% of them speak Spanish, and more than half of the nation's 4.8 million EL students  are concentrated in grades K – 3. On the other hand, EL education can be very context-driven. For example, districts and states vary considerably both in the composition of their particular EL population, and in the specific policies, assessments, and instructional supports they use to guide ELs’ education.

In response to this latter point, some NCER grantees have figured out that one good way to study and support EL education is by working in close partnership with a specific education agency. In other words, don’t fight the place-based nature of EL education – rather, use it as an asset to make your research even more relevant.

There are several examples of IES-funded projects that have successfully worked using this approach. The most obvious are eight Researcher-Practitioner Partnerships we have funded that explicitly focused on ELs. These projects have tackled a range of EL-related issues including science education and trajectories through middle and high school, and have taken place in states like Utah and Oregon, and districts like Saint Paul, MN and Cleveland, OH. As intended for RPP projects, these partners have leveraged their mutual interests to complete rigorous analyses and create products with immediate value for practitioners (read more about one example here).

Partnerships and collaborations are not limited to the RPP competition, however. Some EL researchers have found ways to collaborate closely with agency partners even in the context of a more “typical” research project. One example of this is work done by Peggy Estrada with the Los Angeles Unified School District (LAUSD). Dr. Estrada’s Goal 1 grant focused on longitudinal patterns in EL performance before and after reclassification, or the point at which a student is deemed to no longer need the services and supports associated with EL status.

In LAUSD, reclassification decisions were made based on three criteria, and Dr. Estrada found that many students who are not reclassified miss only one of the three. She also observed that there are only eight different profiles that can describe a student’s status on the three criteria. She reasoned, further, that if teachers knew a student’s profile, they could tailor their supports to help the student meet the criterion on which he or she fell short.

To share these findings with LAUSD, Dr. Estrada created a number of actionable data visualization tools. These proved key in what happened next: LAUSD staff members Kathy Hayes and Hilda Maldonado immediately saw the value in Dr. Estrada’s findings and visualizations, and worked with their colleagues to create an English Learner Dashboard that incorporated both. The Dashboard is an interactive data tool that provides summary information about student reclassification profiles and allows staff to design reports tailored to their needs. LAUSD staff can download the names of students in each profile and generate reports with student names and detailed assessment results for each reclassification criteria.

Creation of this Dashboard is another great example of the value created through collaborations between researchers and practitioners – particularly for EL research. LAUSD staff cited the value of working with an external partner who provided objectivity and helped them to think more critically and deeply about an issue they found important. Dr. Estrada found that working with LAUSD enhanced both the validity and the utility of her research. In the end, both parties win – as do, more importantly, the EL students themselves.

Written by Molly Faulkner-Bond, Program Officer, NCER and Karen Douglas, former Program Officer, NCER

Exploring New Insights and Approaches to Closing the Gender Achievement Gap in STEM

Gender achievement gaps in education, particularly in the science, technology, engineering, and mathematics (STEM) domain continue to persist. On the 2015 NAEP, male students continue to significantly outperform female students in science in Grades 8 and 12, and on the 2017 NAEP, male students outperformed female students in mathematics at Grades 4 and 8. In addition, a recent study by Reardon and colleagues supported by IES found that the gender achievement gap favoring male students in mathematics was related to local socioeconomic conditions, with the gap being more prevalent in more socioeconomically advantaged school districts.

Although we know that the gender achievement gap is pervasive, particularly in STEM, what can we do to help close the gap? Previously, IES released a Practice Guide on Encouraging Girls in Math and Science that included evidence-based recommendations for how practitioners can better support and encourage girls to pursue math- and science-related fields. Adding to this research base, three new FY 2018 IES grants funded under the Education Research Grants program will explore potential causes and correlates of gender differences in achievement that can provide new insights and approaches to closing the gender achievement gap in STEM.  Here is a brief summary of these studies along with their potential contributions to research, practice, and policy.

Gender Stereotypes in STEM – Although many factors influence the gender gap in STEM, research points to gender difference in students' interest and motivation in STEM as a major contributor to later disparities in STEM majors and careers. Allison Master and colleagues will explore how and when gender stereotypes about academic fields emerge, the relationship between stereotypes and motivation in STEM fields, and whether teaching a growth mindset (i.e., the belief that intelligence is malleable) can change stereotypes and improve students' sense of belonging, self-efficacy, interest, and outcomes in STEM in grades 2 to 8. The results from this study will be used to inform the development of future interventions to reduce the impact of STEM-gender stereotypes.

The Relation of Gender-Integrated Classroom Climate to Students' Academic Outcomes - Because boys and girls are typically taught together in classrooms, there is the assumption that boys and girls are cooperative and integrated in their classroom activities, yet evidence suggests this may not be the case.  Some classroom climates facilitate gender integration, while other classroom climates may perpetuate gender segregation where students tend to only work with classmates of the same gender. Carol Lynn Martin and colleagues will examine how gender integration relates to 4th to 6th grade students' school-related engagement and academic perceptions and achievement. The results from this study will provide preliminary evidence of potentially promising practices for gender integration in classrooms that can help girls feel more comfortable working with boys and may encourage persistence in STEM.

Underrepresented Student Learning in Online Introductory STEM College Courses – Online instruction has the potential to make course content more accessible to a larger number of students, thereby strengthening the STEM pipeline. Michelle Perry and colleagues will explore the interaction among various characteristics of online instruction and postsecondary students' persistence in STEM courses. In particular, the researchers will explores how students traditionally underrepresented in STEM (e.g., women, first-generation students, minorities) benefit from or are impeded by online course features (e.g., course videos, discussion boards). The results from this study will provide a theory of postsecondary online STEM instruction that could strengthen persistence in STEM among women and others traditionally underrepresented in STEM. 

Written by Christina Chhin, Education Research Analyst, National Center for Education Research

 

IES Funds First Large-Scale Evaluation Study of Public Preschool Montessori

The Montessori method of education was developed over 100 years ago by Dr. Maria Montessori. This “whole child” approach centers around the theory that children are capable of initiating learning in a thoughtfully prepared environment that develops children’s physical, social, emotional, and cognitive growth. Core components of Montessori education are mixed age classrooms in three-year groupings (e.g., 3-6 year olds, 6-9, 9-12, etc.), a carefully prepared environment filled with appropriate materials and lessons, student freedom to select lessons and activities each day, and daily uninterrupted 3-hour work blocks.

   

According to the National Center for Montessori in the Public Sector (NCMPS), there are currently over 5,000 Montessori schools in the U.S., 500 of which are public schools and over 150 of which serve public preschool and kindergarten students.  Despite its growing popularity in public preschools and Head Start schools, no large-scale evaluation of the efficacy of the Montessori model on children’s academic, social, and emotional skills has been conducted. 

This year, IES funded the first such study. A project team led by Dr. Ann-Marie Faria and Ms. Karen Manship (American Institutes for Research) and Dr. Angeline Lillard (University of Virginia) will study more than 650 children for three years, beginning with their entry at age 3 into preschool. Importantly, this study relies on individual random student assignment via lottery entry to compare preschool students who enroll in Montessori at age 3 to those who are assigned to a waitlist control group (and thus are in other settings such as public PreK, daycare, or a home setting). Data will be collected in diverse urban and suburban school districts across the country, including Houston (TX), Hartford and New Haven (CT), and Washington, DC.

Researchers will examine the impact of preschool Montessori education on children’s academic, social, and emotional skills, as well as kindergarten readiness skills. The research team will also conduct a cost effectiveness study of the public Montessori preschool model, and will examine the effect of fidelity of implementation of Montessori on student outcomes. Collectively, the findings from this study will provide valuable evidence of the efficacy of Montessori preschool education. Ultimately, the researchers plan to disseminate their findings to educators, parents, and policymakers through research briefs, infographics, blog posts, and webinars.

 

By Amanda M. Dettmer, PhD, American Psychological Association Executive Branch Science Fellow/ AAAS Science & Technology Policy Fellowship

Photo credit: Marilyn Horan, Carroll Creek Montessori Public Charter School