Inside IES Research

Notes from NCER & NCSER

Real-World Responses in Real Time: Helping Rural Schools Navigate Rising Mental Health Needs due to COVID-19

Photo of a mother consoling her daughter on a sofa

The United States has observed Mental Health Awareness Month every May since 1949 to raise awareness and educate the public about mental illnesses, including strategies and resources for supporting mental health and wellness. Mental health needs prior to the coronavirus pandemic were already enormous with 1 in 6 school age youth needing mental health support but unlikely to receive it. In fact, a recent study found that half of the estimated 7.7 million U.S. children with a treatable mental health disorder did not receive the necessary treatment from mental health professionals. This service gap is even greater in rural areas. How can rural schools support students, families, and staff during a global pandemic that has shut down school buildings and increased demand for mental health supports?

The IES-funded National Center for Rural School Mental Health is supporting partnerships with rural school districts in three states (Missouri, Virginia, and Montana) to develop and test ways to support the mental health needs of their students. In response to the COVID-19 crisis, the center has compiled a set of resources for families, schools, teachers, and youth on a wide range of pandemic-related challenges.

Visit https://www.ruralsmh.org/covid19/ for information ranging from how to navigate online learning to resources for suicide prevention and protecting children exposed to drug abuse at home. Among the many resources you can find here are tips for parents to encourage cooperative behavior at home, stress management tools for educators, and telehealth tips for youth and teens. For more information about mental health needs in rural settings and how Dr. Wendy Reinke, the Center’s director, and her colleagues are working on approaches to support the mental health needs of their students, please see this previous blog post.  


Written by Emily Doolittle (Emily.Doolittle@ed.gov), National Center for Education Research (NCER) Team Lead for Social Behavioral Research

 

Closing the Opportunity Gap Through Instructional Alternatives to Exclusionary Discipline

According to the most recent GAO analysis of the U.S. Department of Education Civil Rights Data Collection, Black students, boys, and students with disabilities are disproportionately suspended or expelled in K-12 public schools. The reasons for these disparities may not always be clear, but the consequences are stark—suspended or expelled students miss out on opportunities to learn. What can be done to minimize this opportunity gap?

In 2018, researchers at the University of Oregon received a grant to develop an alternative to exclusionary discipline for middle schools. The Inclusive Skill Building Learning Approach (ISLA) will function as a Tier I universal intervention in middle schools that use Positive Behavioral Interventions and Supports (PBIS). ISLA systems and practices will give teachers other options for dealing with misbehaving students along with strategies to support students when they return to the classroom following a trip to the principal’s office. I recently spoke with Dr. Rhonda Nese, the principal investigator for Project ISLA, about how she became interested in this work and how she and her colleagues are tackling this challenge of narrowing the opportunity gap in middle school classrooms.

A photo of the University of Oregon research team

How did you become interested in the issue of disproportionate discipline?

I had been deeply interested in the school-to-prison pipeline research for many years, but the light switch went on for me when I was spending time in a middle school through my work on another project. I started noticing a pattern of students, mostly boys and students of color, sitting in the office every time I walked into this school. And I’m talking about lots of students! This office would be flooded with kids; not learning, not speaking with anyone, just sitting and looking downcast. And it was disturbing.

When I asked the assistant principal what the students were doing in the office, she shared that, for whatever reason, the students were sent out of class and needed to meet with an administrator. So, I became curious. On average how much class time were they missing? I was floored to learn that the average was three days of missed instruction, which is the equivalent of over 1200 minutes of learning. And the deeper I dug the more I realized how pervasive the problem was. In addition to the racial disparities I saw in the kids being excluded, it was also clear that the students who were missing instruction were those who needed to be in class the most: students living in poverty, students struggling academically, and students receiving special education services. And the process of sitting and waiting was doing the students a tremendous disservice academically, behaviorally, and emotionally. I saw firsthand the issues I needed to begin addressing immediately, and I knew I found my passion.

How does Project ISLA extend or build on your earlier research?

I started developing ISLA during my postdoc years when I was deep in the PBIS literature, examining predictors of sustained implementation of evidence-based practices, and beginning to explore interventions to address implicit biases in discipline disproportionality. So, I was able to combine what I was learning from practitioners and from scientific findings to craft an intervention that was rooted in behavioral theory, embedded in preventative practices, and incorporated teacher and student voice.

I also became clearer with myself and others that ISLA is not about “fixing” kids: it is about changing adult behavior to improve student outcomes and relationships. Now through our iterative development process, our team is learning so much about what it takes to support school staff with making this work their own, how we get buy-in from the school community, and how we braid the ISLA work with other preventative practices they already have in place.

What are the core components of the ISLA intervention? What are its essential practices? What have you learned so far about what it takes to implement ISLA in middle schools?

One of our greatest goals is to help educators make this philosophical shift where they view sending a student out of class as a really big deal, and thus, should be reserved for situations in which the teachers and students need support with problem solving, skill building, and making amends. In order to accomplish this, we begin with spending a lot of time with our educators developing and revisiting preventative practices to improve the classroom environment, and in turn, reduce the need for exclusion. This includes working with educators to develop and implement universal relationship-building strategies, graduated discipline practices within the classroom, neutralizing routines to reduce the impact of implicit biases on their decision making, and mechanisms for supporting students in effective and respectful ways. We then layer on a systematized process for students and teachers to request breaks, and then on top of this we have our processes that are provided to students in the event that they are sent out to help them get back to class faster and with the skills to make amends with their teacher. This includes a debrief, skills coaching, and reconnection supports with a front office staff member and a process for their teacher to listen reflectively and agree on how they will problem solve with the student if there’s an issue in the future.  

Getting folks to move away from exclusionary discipline practices takes a lot of time and a lot of patience, because suspensions and other forms of exclusion are deeply tied to systems of oppression that have been prevalent in the United States. And especially in middle school, there’s this pervasive myth that students should know how to behave by this point, and so anything to the contrary is seen as willful defiance as opposed to a skill gap. Unfortunately, there’s no quick fix, and ISLA is certainly not a silver bullet. In fact, we call ISLA Tier I+ because it starts with universal preventative practices and then adds supports for students and teachers who need more. Because of all the myth busting and support layering we’re doing, working with a team of educators in each school has been critical for buy-in and implementation. They help guide our iterative changes, give us strategies to consider, and are the voice to their colleagues. They are invested in the work because they are helping to develop it for their schools. And our work is so much more meaningful because of them.

 

Dr. Nese and her team are mid-way through their project. Now that they have completed the iterative development process they are testing the usability and feasibility of ISLA in new middle schools this year. In their pilot study of promise next year, they will see if ISLA increases instructional time for students and improves student-teacher relationships and school climate. In addition to creating ISLA user guides and materials, the team plans to develop technical reports, video tutorials, trainings, and webinars that will be available through the Office for Special Education Programs (OSEP) Center on Positive Behavioral Interventions and Supports website.


Written by Emily Doolittle, National Center for Education Research Team Lead for Social Behavioral Research. This is the third in a series of blog posts that stems from the 2020 Annual Principal Investigators Meeting. The theme of the meeting was Closing the Gaps for All Learners and focused on IES’s objective to support research that improves equity in access to education and education outcomes. Other posts in this series include Why I Want to Become an Education Researcher and Diversify Education Sciences? Yes, We Can!

 

SELweb: From Research to Practice at Scale in Education

With a 2011 IES development grant, researchers at Rush University Medical Center, led by Clark McKown, created SELweb, a web-based system to assess the social-emotional skills in children in Kindergarten to Grade 3. The system (watch the video demo) includes illustrated and narrated modules that gauge children’s social acceptance with peers and assess their ability to understand others’ emotions and perspectives, solve social problems, and self-regulate. The system generates teacher reports with norm-referenced scores and classroom social network maps. Field trials with 8,881 children in seven states demonstrate that system produces reliable and valid measures of social-emotional skills. Findings from all publications on SELweb are posted here.

In 2016, with support from the university, McKown launched a company called xSEL Labs, to further develop and ready SELweb for use at scale and to facilitate the launch SELweb into the school marketplace. SELweb is currently used in 21 school districts in 16 states by over 90,000 students per year.

Interview with Clark McKown of Rush University Medical Center and xSEL Labs

 

From the start of the project, was it always a goal for SELweb to one day be ready to be used widely in schools?

CM: When we started our aspiration was to build a usable, feasible, scientifically sound assessment and it could be done. When the end of the grant got closer, we knew that unless we figured out another way to support the work, this would be yet another good idea that would wither on the vine after showing evidence of promise. In the last year and a half of the grant, I started thinking about how to get this into the hands of educators to support teaching and learning, and how to do it in a large-scale way.

 

By the conclusion of your IES grant to develop SELweb, how close were you to the version that is being used now in schools? How much more time and money was it going to take?

CM: Let me answer that in two ways. First is how close I thought we were to a scalable version. I thought we were pretty close. Then let me answer how close we really were. Not very close. We had built SELweb in a Flash based application that was perfectly suited to small-scale data collection and was economical to build. But for a number of reasons, there was no way that it would work at scale. So we needed capital, time, and a new platform. We found an outstanding technology partner, the 3C Institute, who have a terrific ed tech platform well-suited to our needs, robust, and scalable. And we received funding from the Wallace Foundation to migrate the assessment from the original platform to 3C’s. The other thing I have learned is that technology is not one and done. It requires continued investment, upkeep, and improvement.

What experiences led you to start a company? How were you able to do this as an academic researcher?

CM: I could tell you that I ran a children’s center, had a lot of program development experience, had raised funds, and all that would be true, and some of the skills I developed in those roles have transferred. But starting a company is really different than anything I’d done before. It’s exciting and terrifying. It requires constant effort, a willingness to change course, rapid decision-making, collaboration, and a different kind of creativity than the academy. Turns out I really like it. I probably wouldn’t have made the leap except that the research led me to something that I felt required the marketplace to develop further and to realize its potential. There was really only so far I could take SELweb in the academic context. And universities recognize the limitations of doing business through the university—that’s why they have offices of technology transfer—to spin off good ideas from the academy to the market. And it’s a feather in their cap when they help a faculty member commercialize an invention. So really, it was about finding out how to use the resources at my disposal to migrate to an ecosystem suited to continuing to improve SELweb and to get it into the hands of educators.

How did xSEL Labs pay for the full development of the version of SELweb ready for use at scale?

CM: Just as we were getting off the ground, we developed

 a partnership with a research funder (the Wallace Foundation) who was interested in using SELweb as an outcome measure in a large-scale field trial of an SEL initiative. They really liked SELweb, but it was clear that in its original form, it simply wouldn’t work at the scale they required. So we worked out a contract that included financial support for improving the system in exchange for discounted fees in the out years of the project.

What agreement did you make with the university in order to start your company and commercial SELweb?

CM: I negotiated a license for the intellectual property from Rush University with the university getting a royalty and a small equity stake in the company.

Did anyone provide you guidance on the business side?

CM: Yes. I lucked into a group of in-laws who happen to be entrepreneurs, some in the education space. And my wife has a sharp business mind. They were helpful. I also sought and found advisors with relevant expertise to help me think through the initial licensing terms, and then pricing, marketing, sales, product development, and the like. One of the nice things about business is that you aren’t expected to know everything. You do need to know how and when to reach out to others for guidance, and how to frame the issues so that guidance is relevant and helpful.

How do you describe the experience of commercializing SELWeb?

CM: Commercialization is, in my experience, an exercise in experimentation and successive approximations. How will you find time and money to test the waters? Commercialization is an exciting and challenging leap from the lab to the marketplace. In my experience, you can’t do it alone, and even with great partners, competitive forces and chance factors make success scale hard to accomplish. Knowing what you don’t know, and finding partners who can help, is critical.

I forgot who described a startup as a temporary organization designed to test whether a business idea is replicable and sustainable. That really rings true. The experience has been about leaving the safe confines of the university and entering the dynamic and endlessly interesting bazaar beyond the ivory tower to see if what I have to offer can solve a problem of practice.

In one sentence (or two!), what would say is most needed for gaining traction in the marketplace?

CM: Figure out who the customer is, what the customer needs, and how what you have to offer addresses those needs. Until you get that down, all the evidence in the world won’t lead to scale.

Do you have advice for university researchers seeking to move their laboratory research into wide-spread practice?

CM: It’s not really practical for most university researchers to shift gears and become an entrepreneur. So I don’t advise doing what I did, although I’m so glad I did. For most university researchers, they should continue doing great science, and when they recognize a scalable idea, consider commercialization as an important option for bringing the idea to scale. My impression is that academic culture often finds commerce to be alien and somewhat grubby, which can get in the way. The truth is, there are whip-smart people in business who have tremendous expertise. The biggest hurdle for many university researchers will be to recognize that they lack expertise in bringing ideas to market, they will need to find that expertise, respect it, and let go of some control as the idea, program, or product is shaped by market forces. It’s also a hard truth for researchers, but most of the world doesn’t care very much about evidence of efficacy. They have much more pressing problems of practice to attend to. Don’t get me wrong—evidence of efficacy is crucial. But for an efficacious idea to go to scale, usability and feasibility are the biggest considerations.

For academics, getting the product into the marketplace requires a new set of considerations, such as: Universities and granting mechanisms reward solo stars; the marketplace rewards partnerships. That is a big shift in mindset, and not easily accomplished. Think partnerships, not empires; listening more than talking.

Any final words of wisdom in moving your intervention from research to practice?

CM: Proving the concept of an ed tech product gets you to the starting line, not the finish. Going to scale benefits from, probably actually requires, the power of the marketplace. Figuring out how the marketplace works and how to fit your product into it is a big leap for most professors and inventors. Knowing the product is not the same as knowing how to commercialize it.

 ____________________________________________________________________________

Clark McKown is a national expert on social and emotional learning (SEL) assessments. In his role as a university faculty member, Clark has been the lead scientist on several large grants supporting the development and validation of SELweb, Networker, and other assessment systems. Clark is passionate about creating usable, feasible, and scientifically sound tools that help educators and their students.

This interview was produced by Ed Metz of the Institute of Education Sciences. This post is the third in an ongoing series of blog posts examining moving from university research to practice at scale in education.

Building the Evidence Base for BEST in CLASS – Teacher Training to Support Young Learners with the Most Challenging Classroom Behavior

Classroom teachers of young children face a seemingly never-ending challenge – how to manage disruptive behavior while simultaneously teaching effectively and supporting the needs of every student in the classroom. Researchers at Virginia Commonwealth University and the University of Florida have received five IES research grants over the past decade – three through the National Center for Special Education Research (NCSER) and two from the National Center for Education Research (NCER) – to develop and test a model of training and professional development, including coaching, for early childhood and early elementary school teachers on how best to support children who engage in disruptive and otherwise challenging classroom behaviors.

A group of young students, several with their hands raised sit cross legged on the floor

With their first IES grant in 2008, Drs. Maureen Conroy and Kevin Sutherland developed the original BEST in CLASS model for early childhood teachers. The goal of BEST in CLASS - PK is to increase the quantity and quality of specific instructional practices with young children (ages 3-5 years old) who engage in high rates of challenging behaviors with the ultimate goal of preventing and reducing problem behavior. Professional development consists of a six-hour workshop that uses didactic and interactive learning activities supported by video examples and practice opportunities. Following the workshop, teachers receive a training manual and 14 weeks of practice-based coaching in the classroom. 

Best in Class logoThe results of this promising development work led to a 2011 IES Efficacy study to test the impact of BEST in CLASS - PK on teacher practices and child outcomes. Based on positive findings from that Efficacy study the team was awarded two additional Development and Innovation grants – one in 2016 to develop a web-based version of BEST in CLASS – PK to increase accessibility and scalability and another in 2015 to adapt BEST in CLASS – PK for early elementary school classrooms (BEST in CLASS – Elementary). Drs. Sutherland and Conroy are currently in the second year of an Efficacy study to test the impact of BEST in CLASS - Elementary to determine if the positive effects of BEST in CLASS in preschool settings are replicated in early elementary classrooms.

Written by Emily Doolittle, NCER Team Lead for Social Behavioral Research, and Jacquelyn Buckley, NCSER Team Lead for Disability Research

Measuring Social and Emotional Learning in Schools

Social and emotional learning (SEL) has been embraced by many schools and districts around the country. Yet in the rush to adopt SEL practices and support student SEL competencies, educators often lack assessment tools that are valid, reliable, and easy to use.

 

Washoe County School District in Nevada has moved the needle on SEL assessment with support from an IES Researcher-Practitioner Partnership grant. The district partnered with the Collaborative for Academic, Social, and Emotional Learning (CASEL) to develop the Social and Emotional Competency Assessments (WCSD-SECAs)—free, open-source instruments that schools can use to measure SEL competencies of students in 5th through 12th grade.

Long and short versions of the SECA are available to download from the school district’s website, along with a bank of 138 items across 8 SEL domains that schools around the country can use to modify SECA assessments for their local context. The long-form version has been validated and aligned to the CASEL 5 SEL competency clusters and WCSD SEL standards (self-awareness, self-management, social awareness, relationship skills, and responsible decision making). The assessment is also available in Spanish, and the Metro Nashville Public schools offer the assessment in 8 additional languages.  

Students complete the long-form SECA as part of Washoe’s Annual Student Climate Survey by rating how easy or difficult SEL skills are for them. Under the Social Awareness domain, students respond to items such as “Knowing what people may be feeling by the look on their face” or “Learning from people with different opinions than me.” Under the Responsible Decision Making domain, students rate themselves on skills such as “Saying ‘no’ to a friend who wants to break the rules” and “Thinking of different ways to solve a problem.”

The SECA is one component of Washoe County’s larger School Climate Survey Project that is marking its 10th anniversary this year. Washoe provides district-level and school-level reports on school climate to support the district’s commitment to providing safe, caring, and engaging school environments for all of Washoe’s students and families.  

Written by Emily Doolittle, NCER’s Team Lead for Social Behavioral Research