Inside IES Research

Notes from NCER & NCSER

IES at the Conference on Computing and Sustainable Societies

Over the summer, researchers, technologists, and policymakers gathered in Accra, Ghana for the Association for Computing Machinery’s Conference on Computing and Sustainable Societies (ACM COMPASS) to discuss the role of information technologies in international development.

Two IES-funded researchers from Carnegie Mellon University’s Program in Interdisciplinary Education Research, Michael Madaio and Dr. Amy Ogan, shared their research on developing voice-based early literacy technologies and evaluating their efficacy with low-literate, bilingual families in the Ivory Coast. 

Their research draws on methods from human-computer interaction, the learning sciences, and information-communication technology for development, to design educational technologies that are culturally and contextually appropriate.

Although the COMPASS conference focused on cross-cultural applications and technology for development, the research presented has implications for U.S. based education researchers, practitioners, and policymakers.

For instance, while research provides evidence for the importance of parental involvement in early literacy, parents with low literacy in the target language – as in many bilingual immigrant communities in the U.S. – may not be able to support their children with the explicit, instrumental help suggested by prior research (for example, letter naming or bookreading). This suggests that there may be opportunities for technology to scaffold low-literate or English Learners (EL) parental support in other ways.

At the conference, researchers described interactive voice-based systems (known as “IVR”) that help low-literate users find out about crop yields, understand local government policies, and engage on social media.  

This body of work has implications for designers of learning technologies in the U.S. Many families may not have a smartphone, but basic feature phones are ubiquitous worldwide, including in low-income, immigrant communities in the U.S. Thus, designers of learning technologies may consider designing SMS- or voice-based (such as IVR) systems, while schools or school districts may consider how to use voice-based systems to engage low-literate or EL families who may not have a smartphone or who may not be able to read SMS information messages.

In a rapidly changing, increasingly globalized world, research at IES may benefit from increased international engagement with international research, both focusing specifically on education, as well as information technology research that has implications for educational research, practice, and policy.

This guest blog was written by Michael Madaio. He is an IES Predoctoral Fellow in the Program in Interdisciplinary Education Research at Carnegie Mellon University. He is placed in the Human-Computer Interaction Institute.

Fieldnotes: Reflections from an Adult Education Instructor on Research and Practice

Approximately 18 percent of US adults are at the lowest levels of literacy and nearly 30 percent are at the lowest levels of numeracy. The adult education system serves adults with low skills, but many education researchers know little about the students or the setting.  Recently, NCER convened a working group of adult education instructors, administrators, and researchers to discuss adult education’s research and dissemination needs.

Mr. Marcus Hall, an adult education instructor at the Community Learning Center and JEVS Human Services in Philadelphia, participated in this working group. He spoke with Meredith Larson, NCER program officer for adult education, about his experiences and interests in research. A copy of the working group meeting summary is available here.

Please describe your adult education classroom.

I once taught a 7-week course with students ranging from 18 to over 60 years old who had low literacy or math scores. I tried to contextualize instruction around their career interests and differentiate it to their learning needs. For example, some students were proficient readers but needed comprehension and math practice while others struggled with one or more of the basic components of reading. Somehow, I needed to help those learning phonics and those struggling with fluency while also challenging those ready for comprehension work. It’s hard to meet student needs in such a short time without teacher aides or adaptive technologies.

Why is research particularly important for adult education?

The challenges we face are monumental. Despite the large number of adults in need, adult education feels under-funded, under-staffed, and under-appreciated. Our students need complex, comprehensive, and well-rounded intervention, but we often have to make the most out of slightly targeted, inexpensive, and difficult-to-implement solutions. We need researchers to provide practical information and recommendations that we can use today to help adults learn and retain information.

Have you used research into your teaching?

Specifically, for reading instruction, I use techniques and activities built on evidence-based reading interventions. I start with tested diagnostic assessments to determine the needs of my students followed by strategies such as Collaborative Oral Reading or Repeated Reading exercises to support my students.

What topic during the meeting stood out to you?

The discussion about the workforce and professional development resonated with me. Many of our educators are part-time, come out of K-12, close to retirement, and may not have specific training for working with adults. They are asked to teach subjects they may not have any certification in, and their programs may not be able to provide the professional development they need. Just as we need supports for our learners, we need research to develop supports for us educators.

What additional research would you like to see?

Many of my students have had traumatic experiences that, when relived in the classroom, can cause them to disengage or struggle. I feel that understanding triggers and signs of discomfort has greatly enhanced my ability to help my students. Many educators want to leverage mental health approaches, like trauma-informed care, but we could use help learning how to integrate these strategies into instruction.

What do you hope researchers and educators keep in mind regarding one another?

It seems that researchers publish and promote their work to other researchers and then move to the next topic. This may be due to time constraints, publishing demands, or institutional requirements. I hope researchers take the time to come into our settings and observe us in action. I want researchers to work with us to help us understand and accept what is and isn’t working.

As for educators, we need to not try things and then stop using them when something unexpected occurs. At times, we revert back to what we know and are most comfortable with in the classroom. We educators can and must think critically about our norms and be ready and willing to enhance our practice with new information. 

Equity Through Innovation: New Models, Methods, and Instruments to Measure What Matters for Diverse Learners

In today’s diverse classrooms, it is both challenging and critical to gather accurate and meaningful information about student knowledge and skills. Certain populations present unique challenges in this regard – for example, English learners (ELs) often struggle on assessments delivered in English. On “typical” classroom and state assessments, it can be difficult to parse how much of an EL student’s performance stems from content knowledge, and how much from language learner status. This lack of clarity makes it harder to make informed decisions about what students need instructionally, and often results in ELs being excluded from challenging (or even typical) coursework.

Over the past several years, NCER has invested in several grants to design innovative assessments that will collect and deliver better information about what ELs know and can do across the PK-12 spectrum. This work is producing some exciting results and products.

  • Jason Anthony and his colleagues at the University of South Florida have developed the School Readiness Curriculum Based Measurement System (SR-CBMS), a collection of measures for English- and Spanish-speaking 3- to 5-year-old children. Over the course of two back-to-back Measurement projects, Dr. Anthony’s team co-developed and co-normed item banks in English and Spanish in 13 different domains covering language, math, and science. The assessments are intended for a variety of uses, including screening, benchmarking, progress monitoring, and evaluation. The team used item development and evaluation procedures designed to assure that both the English and Spanish tests are sociolinguistically appropriate for both monolingual and bilingual speakers.

 

  • Daryl Greenfield and his team at the University of Miami created Enfoque en Ciencia, a computerized-adaptive test (CAT) designed to assess Latino preschoolers’ science knowledge and skills. Enfoque en Ciencia is built on 400 Spanish-language items that cover three science content domains and eight science practices. The items were independently translated into four major Spanish dialects and reviewed by a team of bilingual experts and early childhood researchers to create a consensus translation that would be appropriate for 3 to 5 year olds. The assessment is delivered via touch screen and is equated with an English-language version of the same test, Lens on Science.

  • A University of Houston team led by David Francis is engaged in a project to study the factors that affect assessment of vocabulary knowledge among ELs in unintended ways. Using a variety of psychometric methods, this team explores data from the Word Generation Academic Vocabulary Test to identify features that affect item difficulty and explore whether these features operate similarly for current, former, as well as students who have never been classified as ELs. The team will also preview a set of test recommendations for improving the accuracy and reliability of extant vocabulary assessments.

 

  • Researchers led by Rebecca Kopriva at the University of Wisconsin recently completed work on a set of technology-based, classroom-embedded formative assessments intended to support and encourage teachers to teach more complex math and science to ELs. The assessments use multiple methods to reduce the overall language load typically associated with challenging content in middle school math and science. The tools use auto-scoring techniques and are capable of providing immediate feedback to students and teachers in the form of specific, individualized, data-driven guidance to improve instruction for ELs.

 

By leveraging technology, developing new item formats and scoring models, and expanding the linguistic repertoire students may access, these teams have found ways to allow ELs – and all students – to show what really matters: their academic content knowledge and skills.

 

Written by Molly Faulkner-Bond (former NCER program officer).

 

Inequity Persists in Gifted Programs

The National Center for Research on Gifted Education (NCRGE) at the University of Connecticut, in Phase I of a rigorous research agenda, examined how academically-gifted students are identified and served in three states in order to provide systematic information for the field. The research team focused especially on the representation of historically underserved groups in gifted education.

NCER recently spoke with the Center’s Principal Investigator, Del Siegle, a nationally-recognized expert on gifted education. 

What is the biggest challenge facing gifted educators today?

Unfortunately, many of our nation’s brightest students from underserved populations (e.g., Black, Hispanic, English Learner, and/or free and reduced-price lunch eligible) are not being identified as gifted and do not receive gifted education services. About 80% of states that completed the most recent National Association for Gifted Children’s State of the States survey indicated that underrepresentation of students from underserved populations was an important or very important issue in their state.

What did you find in your study of identification of underserved students for gifted programs?

During Phase I of our work, we analyzed standardized student achievement test data from three states that mandate gifted identification and programming. We found that schools were less likely to identify students from underserved groups as gifted—even in cases where the underserved child had similar achievement test scores. For example, students with similar test scores who received free and reduced price lunch were less than half as likely to be identified as gifted as students who didn’t receive free or reduced price lunch.

What identification practices are schools using?

Cognitive tests and teacher nominations were the most common identification tools across the three states we studied. The majority (90% to 96%) of the districts in all three states used these practices to select students. Identification for gifted services occurs most often in third grade. Districts seldom reassess identified students once they are identified and only about half reassess non-identified students in elementary schools at regular intervals. Screening all children and using a variety of identification criteria showed promise for reducing under-identification in one of our states.

How are students being serviced in gifted programs?

In the three states we studied, schools primarily focused on critical thinking and creativity followed by communication skills, research skills, and self-directed projects.  Mathematics and reading language arts acceleration was much less of a focus and were ranked among the bottom third of focus areas. Gifted students seldom receive gifted programming in core academic areas. Only 29% of the schools provided a separate gifted curriculum in reading/language arts. Only 24% of the schools had a separate gifted curriculum in mathematics. Gifted students spent 5 hours or more each week in regular education mathematics and reading/language arts classrooms. Of the 74% of schools reporting using pull-out services, only 32% offered separate gifted curriculum in reading/language arts and 28% offered separate gifted curriculum in math. 

What about gifted student growth in mathematics and reading?

In 3rd grade, gifted students are approximately 2 grade levels ahead of students not identified as gifted, but gifted students grow more slowly than non-gifted students between 3rd and 5th grade. Most grouping arrangements for gifted students had no impact on the growth of academic achievement. We believe much of this has to do with the limited advanced mathematics and reading instruction gifted students receive in their classrooms and gifted programs.

What is the next step in your research?

We are examining the effect of attending dedicated gifted classes in core content areas on academic achievement in reading/language arts and mathematics in a large, ethnically, economically, and linguistically diverse urban school district. Our research will compare the reading/language arts and mathematics achievement of gifted students in three different settings: schools offering a full-time gifted-only program with gifted classes in all subject areas, schools offering a part-time gifted-only program with gifted classes in mathematics, and schools offering a part-time gifted-only program with gifted classes in reading/language arts.

New IES Grantee Focuses on Improving Adult Literacy

In her first IES grant, Dr. Elizabeth Tighe (Georgia State University) is taking expertise honed in both an NCER predoctoral fellowship and PIAAC methods training program to help further adult literacy research. Her earlier work includes developing assessments for adults with low literacy, leveraging statistical approaches to understand these adults’ abilities and difficulties, and using eye-tracking paradigms to explore their ability to self-monitor during reading. 

Program officer, Meredith Larson, interviewed Dr. Tighe about her previous work and new grant.

What is your general area of research, and why is it important?

I focus on adult struggling readers, which comprises roughly 36 million (1 in 6) adults in the U.S. Only a fraction of these adults enroll in adult education programs, which are plagued by insufficient funding, high teacher turnover rates, and a lack of research-based instructional practices and curricula. By better understanding these adults’ strengths and deficits and how best to measure their skills, I aim to inform and improve adult education programs.

What could people do with your research?

My research could directly inform how we help adults become stronger readers, and this can improve educational outcomes, such as GED attainment. I am working towards building better assessments for adult education practitioner and researcher use. My longer-term goal is to design a curriculum to teach morphology (e.g., prefixes and suffixes) and use this to improve adults’ vocabulary and reading comprehension.

What are you trying to learn through your new IES project?

For this grant, I’m using the Program for the International Assessment of Adult Competencies (PIAAC), a large-scale, international assessment of adult literacy, numeracy, and digital problem-solving skills, to create risk profiles of adults with low literacy skills. This sort of information could move us closer to being able to individualize instruction in adult education programs to match the needs of specific learners.

We will use PIAAC data to explore how demographic characteristics (age, race/ethnicity, educational background, employment status) and malleable factors (enjoyment of learning, frequency of computer use, reading and writing behaviors at home and at work) influence low literacy performance.

Further, we are examining whether risk factors differ by whether someone has a high school diploma and whether someone has participated in education or training recently. We will also explore whether reading components and literacy skills are predictive of low-skilled adults’ numeracy skills.

Our findings could have important implications for understanding risk factors and predictors of low literacy as well as low numeracy. As stated previously, 1 in 6 U.S. adults have low literacy skills and nearly 1 in 3 have low numeracy skills. For GED attainment, adults must demonstrate proficiency in both of these areas (along with science, social studies, and writing knowledge). It’s important to have targeted, individualized instruction for these adults because they may have time or resource barriers.

How did this particular research project arise?

I first learned about PIAAC at a summer institute. I was intrigued, in particular, because PIAAC is the first assessment of this size to include a reading component supplement for lower-skilled adults.

I recently attended a 3-day NCER/ETS PIAAC training workshop, which allowed me to work with PIAAC data and network with others. This workshop influenced my decision to apply for an IES grant. I felt that a 2-year grant using extant data would be a great way to combine my interests regarding individual differences in adults’ component skills and get my feet wet with IES as a new investigator. I am excited to bridge my interests and grow as a researcher by learning and working alongside two experts (Drs. Yaacov Petscher and John Sabatini) in the larger reading and education field!