IES Blog

Institute of Education Sciences

Towards a Better Understanding of Middle-Schoolers’ Argumentation Skills

What is the difference between fact and opinion? How do you find relevant evidence and use it to support a position? Every day, teachers help students practice these skills by fostering critical discussions, a form of argumentation that encourages students to use reasoning to resolve differences of opinion.

In their IES-funded study, Exploring and Assessing the Development of Students' Argumentation Skills, Yi Song and her colleagues are uncovering activities (both teacher led and technology supported) that can improve middle-school students’ ability to generate better oral and written arguments.

This project began in 2019 and is working in classrooms and with teachers and students. The researchers have created a series of videos that describe their work. In this series, Dr. Song and her co-PIs, Dr. Ralph Ferretti and Dr. John Sabatini, discuss why the project is important to education, how they will conduct the research plan, and how educators can apply what they are learning in classrooms.

 

 


For questions and more information, contact Meredith Larson (Meredith.Larson@ed.gov), Program Officer, NCER

"Boys Have It; Girls Have to Work for It": Examining Gender Stereotypes in Mathematics Achievement

In 2020, Andrei Cimpian, along with co-PIs Sapna Cheryan, Joseph Cimpian, and Sarah Lubienski, were awarded a grant for “Boys Have It; Girls Have to Work for It”: The Development and Consequences of Gender Stereotypes About Natural Talent vs. Effort in Mathematics. The goal of this project is threefold: 1) to explore the origins of the gender stereotype that girls achieve in math due to effort and boys achieve in math due to natural talent, 2) to investigate the consequences of these stereotypes, and 3) to identify ways of reducing the negative effects of these stereotypes on mathematics outcomes. In this blog, we interviewed Dr. Andrei Cimpian on his inspiration and insights on this research, as well as his plans to disseminate the findings to education practitioners.

 

Dr. Andrew CimpianWhat spurred your research, and what prior research was foundational for this current study? 

The co-PIs and I were inspired to do this research because we were struck by the contrast between two sets of facts. On the one hand, girls do better in school than boys from kindergarten to grade 12. Women also obtain more bachelor’s and graduate degrees than men. On the other hand, we as a society still think of men as more brilliant and genius-like than women. For example, participants in a 2018 study referred more male than female acquaintances for a job that they were told requires natural smarts1. When the same job was said to require a strong work ethic instead, participants referred equal numbers of women and men.

Of course, societal views of women and men have changed quite a bit over the last century. With respect to general competence, women are now equal with men in the eyes of the American public. But the stereotype that associates “raw,” high-level intellectual talent with men more than women seems to have resisted change. Why?

Our research is testing a promising hypothesis: It is possible that people give different explanations for women’s and men’s intellectual successes, explaining men’s competence as being due primarily to their inborn intellectual talent and women’s as being due to their efforts. This effort-vs.-talent stereotype “explains away” women’s achievements by attributing them to a quality—perseverance—that is less valued in American culture than natural ability.

Versions of this explanatory stereotype have been documented in adults, but our project will provide its first systematic investigation among children. In particular, we will investigate the effort-vs.-talent stereotype in the domain of mathematics because innate ability is particularly valued in this domain1, which might make this stereotype especially consequential.

 

What are some examples of language or behavior that might suggest an individual holds a particular stereotype? Are there potential ways of mitigating the negative effects of stereotypes?

The best example of this stereotype that I can think of—and this is in fact the anecdote that crystallized our team’s interest in this topic—was recounted by co-PI Joseph Cimpian in a recent piece for The Brookings Institution (emphasis is mine):

About five years ago, while Sarah [Lubienski] and I were faculty at the University of Illinois, we gathered a small group of elementary teachers together to help us think through […] how we could intervene on the notion that girls were innately less capable than boys. One of the teachers pulled a stack of papers out of her tote bag, and spreading them on the conference table, said, “Now, I don’t even understand why you’re looking at girls’ math achievement. These are my students’ standardized test scores, and there are absolutely no gender differences. See, the girls can do just as well as the boys if they work hard enough.” Then, without anyone reacting, it was as if a light bulb went on. She gasped and continued, “Oh my gosh, I just did exactly what you said teachers are doing,” which is attributing girls’ success in math to hard work while attributing boys’ success to innate ability. She concluded, “I see now why you’re studying this.”

In terms of what can be done to mitigate the effects of this stereotype, our project will investigate a potential strategy: normalizing effort by making it clear to students that everyone (not just particular groups) needs to work hard to learn math. This message reframes what is viewed as necessary for success in math away from the belief that natural talent is key, thereby undercutting the power of effort-vs.-talent stereotypes.

 

The current study focuses on elementary school students in grades 1 through 4. What was the motivation for choosing this specific age group?

In general, gender stereotypes about intellectual ability seem to emerge quite early. For instance, girls as young as 6 and 7 are less likely than boys to associate being “really, really smart” with members of their own gender. For this reason, we think it is really important to focus on young children—we need to understand when effort-vs.-talent stereotypes first take root!

“Catching” these stereotypes when they first arise is also important for intervention purposes. If left unchecked, the effects of effort-vs.-talent stereotypes may snowball over time (for example, differences in the types of careers that young women and men are motivated to pursue).

 

What plans do you have to disseminate the findings of this research in ways that will be useful for education practitioners? 

We are mindful of the importance of getting this research into the hands of teachers so that they can use it in practice. We hope to write articles on this work for media outlets that draw educationally oriented audiences. To reach parents as well, we will coordinate with popular media outlets to disseminate the results of this work to general audiences. More generally, we will make every effort to ensure that the findings have maximal societal impact, raising awareness of effort-vs.-talent stereotypes among parents, educators, and the general public.

 


Andrei Cimpian, PhD (@AndreiCimpian), Professor of Psychology at New York University, has conducted extensive research on children’s conceptual development, explanations, and motivation in school.

Written by Yuri Lin (ylin010101@g.ucla.edu), intern for the Institute of Education Sciences.

Photo credit: Brian Stauffer


1The full PDF and resources are available at https://www.cimpianlab.com/motivation.

Representation Matters: Exploring the Role of Gender and Race on Educational Outcomes

This year, Inside IES Research is publishing a series of interviews showcasing a diverse group of IES-funded education researchers that are making significant contributions to education research, policy, and practice.

 

The process of education transmits sociocultural values to learners in addition to information and knowledge. How individuals are represented in curricula and instructional materials can teach students about their place in the world. This can either perpetuate existing systemic inequalities or, conversely, provide a crucial counternarrative to them. With an exploration grant from IES, Anjali Adukia (University of Chicago) and Alex Eble (Teachers College, Columbia University) are exploring how  representation and messages about gender and race in elementary school books may influence student’s education outcomes over time. The researchers will develop and use machine-learning tools that leverage text and image analysis techniques to identify gender- and race-based messages in commonly used elementary-school books.

 

Interview with Anjali Adukia, University of Chicago

Tell us how your research contributes to a better understanding of the importance of diversity, equity, and/or inclusion in education.

In my work, I seek to understand how to reduce inequalities such that children from historically (or contemporaneously) marginalized backgrounds have equal opportunities to fully develop their potential. I examine factors that motivate and shape behavior, preferences, and educational decision-making, with a particular focus on early-life influences. Proceeding from the notion that children are less likely to be able to focus on learning until their basic needs are met, my research uses both econometric methods and qualitative approaches to understand the specific roles different basic needs play in making these decisions. My research, for example, has explored the role of safety and health (sanitation, violence), economic security (road construction, workfare), justice (restorative practices), and representation (children’s books), particularly for marginalized groups.

 

As a woman and a minority, how has your background and experiences shaped your career?

My research is informed and influenced by my own experiences. When I was a child, I never understood why there weren’t more characters that looked like me or when there were, why they had such limited storylines. For me personally, the motivation underlying our IES-funded project was borne out of my lived experience of always searching for content that reflected who I was. I think of representation as a fundamental need: if you don’t see yourself represented in the world around you, it can limit what you see as your potential; and similarly, if you don’t see others represented, it can limit what you see as their potential; and if you only see certain people represented, then this shapes your subconscious defaults.

It was a real watershed moment when I realized that academia allowed me to pursue many of my larger goals in life, in which I hope to meaningfully improve access to opportunities and outcomes for children – especially those from disadvantaged backgrounds. I hope to accomplish this in various ways, paying forward the many kindnesses generously given to me by: (1) producing rigorous policy-relevant evidence that expands our understanding of big questions and opens new avenues for inquiry; (2) translating my research such that it helps inform policymakers and practitioners in the design of school policies and practices; (3) understanding issues with a depth and sophistication that comes from “on-the-ground” insights, knowledge cultivated in multiple disciplines using different methodologies, introspection, humility, and courage; (4) directly working with government agencies, non-profit organizations, and community groups to positively inform policies; (5) contributing to the larger public discourse; and (6) by training and advising students to have the fortitude to ask hard questions, to be able to defend different perspectives on issues, to learn that knowledge brings more questions than answers, and to be willing to take risks and fail (and in the process, I will certainly learn more from them than they will ever be able to learn from me).

 

What has been the biggest challenge you encountered and how did you overcome the challenge?

Life is always filled with challenges, but one challenge starting from when I was young was to feel comfortable in my own skin and to find legitimacy in my own voice. I grew up as an Indian-American daughter of Hindu immigrants in a rural, predominantly white and Christian setting. I was different from the other kids and did not always feel like I fit in. I remember literally trying to erase my skin hoping that it would make it lighter. I found the helpers, as my parents (and Mr. Rogers) would suggest, and tried to focus on the voices that lifted me up – my family, teachers, other mentors, those friends who loved me no matter my differences. My mother always told me to find the kindness, the good, the love in people; to find the common ground and to embrace and learn from the differences. I surrounded myself with love, focusing on what I had and on what I could do rather than what society was telling me I couldn’t do. I turned to concentrating on things that mattered to me, that drove me. I don’t think there is a single challenge in life that I overcame alone. I have been very lucky, and I am deeply grateful for the many gifts in my life, the many loved ones – family, friends, colleagues, mentors, healthcare workers – who have lifted me up, and the opportunities that came my way.

 

How can the broader education research community better support the needs of underrepresented, minority scholars?

The notion of what is considered to be an important question is often driven by the senior scholars in a field, for example, the people considered to be “giants.” Demographically, this small set of leading scholars has historically consisted of people from the most highly represented groups (particularly in economics). And because the field is thus shaped mainly by researchers from a “dominant” group background, the key questions being pursued may not always reflect the experiences or concerns of people from underrepresented backgrounds. Education research has pockets where these different perspectives are being considered, but it can continue to evolve by becoming more open to approaches thought to be less traditional or to questions not typically asked (or asked from a different point of view). Expanding the notion of what is considered important, rigorous research can be difficult and cause growing pains, but it will help expand our knowledge to incorporate more voices.

 

What advice would you give to emerging scholars from underrepresented, minority backgrounds that are pursuing a career in education research?

Keep a journal of questions that arise and topics that pique your curiosity and interest. Soon, you will find questions in the fabric of everyday life, and you will start to articulate the wonder you see in the world around you and what inspires you to action, to understand the universe further. I find that when I return to past writings and journal entries, I am reminded of questions that have ignited my fires and see some of the common themes that emerge over time. Find your voice and know that your voice and views will grow and evolve over time. There are so many interesting and important questions one can pursue. Most importantly, you have to be true to yourself, your own truth. Find circles of trust in which you can be vulnerable. Draw strength from your struggle. There is deep truth and knowledge within you.

 


Dr. Anjali Adukia is an Assistant Professor at the University of Chicago Harris School of Public Policy and the College.

This interview was produced by Christina Chhin (Christina.Chhin@ed.gov), Program Officer, National Center for Education Research.

 

Better Reading Comprehension When You Know That You Don’t Know

The more you already know about a topic, the easier it may be to comprehend and learn from texts about that topic. But knowledge has to start somewhere. So how can we help students learn from texts when they may have low background knowledge?

In their exploratory study, researchers from ETS found that lack of knowledge is not necessarily a barrier to comprehension. Rather, they suggest that students who can identify their lack of background knowledge are more likely to comprehend and learn new information than students who do not acknowledge they lack background knowledge. In other words, knowing that you might not know may lead to better outcomes.

To determine the role of background knowledge, the researchers pretested middle and high school students’ background knowledge through questions related to topics the students may have some but not complete knowledge of, such as ecology, immigration, and wind power. The pretest included an “I don’t know” option, along with correct and incorrect responses.

Students then took a scenario-based assessment in which they read multiple sources about each of the topics. This type of assessment mirrors real-world learning by encouraging readers to build their own interpretations of a topic, which helps researchers determine whether students comprehend what they read.

They found that students who selected “I don’t know” when answering background knowledge questions had better understanding of the content than those who provided wrong answers on these questions. In fact, students who selected “I don’t know” rather than answering incorrectly were nearly three times as likely to learn from sources that provided the correct information than students who had answered the pretest incorrectly. Students who selected “I don’t know” may also learn more than students who had a comparable level of weak background knowledge. The researchers suggest that the “I don’t know” readers may have set different reading goals prior to engaging with the sources than those who guessed incorrectly.

 

Possible Implications for Teaching and Learning

The results from this work support the idea that having and building background knowledge is key. Thus, teachers may want to assess existing knowledge and address knowledge gaps prior to instruction.

Teachers may also want to provide an “I don’t know” option or options that allow students to rate their level of certainty. Doing so may help teachers distinguish between students who recognize their own gaps in knowledge from those who may not be aware that they are wrong or that they simply do not know. This latter group of students may need more help in determining the accuracy of their judgments or may have incorrect knowledge that could interfere with learning.

The researchers further suggest that teachers may want to go beyond the role of background knowledge by teaching students how to set appropriate reading goals and use strategic reading approaches to learn new facts or correct existing misunderstandings.

 


The research reported here was conducted under NCER grant R305A150176: What Types of Knowledge Matters for What Types of Comprehension? Exploring the Role of Background Knowledge on Students' Ability to Learn from Multiple Texts.

This blog was written by Dr. Meredith Larson. Contact her for more information about this project.

Addressing COVID-19’s Disruption of Student Assessment

Under an IES grant, the RAND Corporation, in collaboration with NWEA, is developing strategies for schools and districts to address the impacts of COVID-19 disruptions on student assessment programs. The goal is to provide empirical evidence of the strengths and limitations of strategies for making decisions in the absence of assessment data. Jonathan Schweig, Andrew McEachin, and Megan Kuhfeld describe early findings from surveys and structured interviews regarding key concerns of districts and schools. 

 

As a first step, we surveyed assessment and research coordinators from 23 school districts (from a sample of 100 districts) and completed follow-up interviews with seven of them on a variety of topics, including the re-entry scenario for their district, the planning activities that they were not able to perform this year due to coronavirus-based disruptions to spring 2020 assessments, and the strategies they were employing to support instructional planning in the absence of assessment data. While the research is preliminary and the sample of respondents is not nationally representative, the survey and interview responses identified two key concerns arising from the lack of spring 2020 assessment data which has made it challenging to examine student or school status and change over time, especially as COVID-19 has differential impacts on student subgroups:

 

  • Making course placement decisions. Administrators typically rely on spring assessment scores—often in conjunction with other assessment information, course grades, and teacher recommendations—to make determinations for course placements, such as who should enroll in accelerated or advanced mathematics classes. 
  • Evaluating programs or district-wide initiatives. Many districts monitor the success of these programs internally by looking at year-to-year change or growth for schools or subgroups of interest. 

 

How are school systems responding to these challenges? Not surprisingly, the responses vary depending on local contexts and resources. Where online assessments were not feasible in spring 2020, some school districts used older testing data to make course recommendations, either from the winter or from the previous school year. Some districts relaxed typical practice and provided more autonomy to individual schools, relying on school staff to exercise local judgment around course placements and using metrics like grades and teacher recommendations. Other districts reported projecting student scores based on student assessment histories. Relatedly, some districts were already prepared for this decision because they had recently experienced difficulties with adopting an online assessment system and had to address similar problems caused by large numbers of missing or invalid tests.

 

School districts also raised concerns about whether assessments administered during the 2020-21 school year would be valid and comparable so that they could be used in student placement and program evaluation decisions. These concerns included the following:

  • Several respondents raised concerns about the trustworthiness of remote assessment data collected this fall and the extent to which results could be interpreted as valid indicators of student achievement or understanding.
  • Particularly for districts that started the 2020-21 school year remotely, respondents were concerned about student engagement and motivation and the possibility of students rushing assessments, running into technological or internet barriers, or seeking assistance from guardians or other resources. 
  • Respondents raised questions about the extent to which available assessment scores are representative of school or district performance as a whole. Given that vulnerable students (for example, students with disabilities, students experiencing homelessness) may be the least likely to have access to remote instruction and assessments, it is likely that the students who are not assessed this year are different from students who are able to be assessed.
  • Other respondents noted that they encountered resistance from parents around fall assessment because they prioritized student well-being (for example, safety, sense of community, and social and emotional well-being) more so than academics. This is a perspective that resonates with recent findings from a nationally representative sample of teachers and school leaders drawn from RAND’s American Educator Panel (AEP).

 

In the next phase of the work, the research team plans to:

  • Conduct a series of simulation and empirical studies regarding the most common strategies that the district respondents indicated they were using to make course placement decisions and to evaluate programs or district-wide initiatives.
  • Provide a framework to help guide local research on the intended (and unintended) consequences for school and school system decision making when standardized test scores are not available.

 

We welcome individuals to reach out to RAND with additional recommendations or considerations. We are also interested in hearing how districts are approaching course placement, accountability, and program evaluation across the country. Connect with the research team via email at jschweig@rand.org.

 


Jonathan Schweig is a social scientist at the nonprofit, nonpartisan RAND Corporation.

Andrew McEachin is a senior policy researcher at the nonprofit, nonpartisan RAND Corporation.

Megan Kuhfeld is a researcher at NWEA.