IES Blog

Institute of Education Sciences

SELweb: From Research to Practice at Scale in Education

With a 2011 IES development grant, researchers at Rush University Medical Center, led by Clark McKown, created SELweb, a web-based system to assess the social-emotional skills in children in Kindergarten to Grade 3. The system (watch the video demo) includes illustrated and narrated modules that gauge children’s social acceptance with peers and assess their ability to understand others’ emotions and perspectives, solve social problems, and self-regulate. The system generates teacher reports with norm-referenced scores and classroom social network maps. Field trials with 8,881 children in seven states demonstrate that system produces reliable and valid measures of social-emotional skills. Findings from all publications on SELweb are posted here.

In 2016, with support from the university, McKown launched a company called xSEL Labs, to further develop and ready SELweb for use at scale and to facilitate the launch SELweb into the school marketplace. SELweb is currently used in 21 school districts in 16 states by over 90,000 students per year.

Interview with Clark McKown of Rush University Medical Center and xSEL Labs

 

From the start of the project, was it always a goal for SELweb to one day be ready to be used widely in schools?

CM: When we started our aspiration was to build a usable, feasible, scientifically sound assessment and it could be done. When the end of the grant got closer, we knew that unless we figured out another way to support the work, this would be yet another good idea that would wither on the vine after showing evidence of promise. In the last year and a half of the grant, I started thinking about how to get this into the hands of educators to support teaching and learning, and how to do it in a large-scale way.

 

By the conclusion of your IES grant to develop SELweb, how close were you to the version that is being used now in schools? How much more time and money was it going to take?

CM: Let me answer that in two ways. First is how close I thought we were to a scalable version. I thought we were pretty close. Then let me answer how close we really were. Not very close. We had built SELweb in a Flash based application that was perfectly suited to small-scale data collection and was economical to build. But for a number of reasons, there was no way that it would work at scale. So we needed capital, time, and a new platform. We found an outstanding technology partner, the 3C Institute, who have a terrific ed tech platform well-suited to our needs, robust, and scalable. And we received funding from the Wallace Foundation to migrate the assessment from the original platform to 3C’s. The other thing I have learned is that technology is not one and done. It requires continued investment, upkeep, and improvement.

What experiences led you to start a company? How were you able to do this as an academic researcher?

CM: I could tell you that I ran a children’s center, had a lot of program development experience, had raised funds, and all that would be true, and some of the skills I developed in those roles have transferred. But starting a company is really different than anything I’d done before. It’s exciting and terrifying. It requires constant effort, a willingness to change course, rapid decision-making, collaboration, and a different kind of creativity than the academy. Turns out I really like it. I probably wouldn’t have made the leap except that the research led me to something that I felt required the marketplace to develop further and to realize its potential. There was really only so far I could take SELweb in the academic context. And universities recognize the limitations of doing business through the university—that’s why they have offices of technology transfer—to spin off good ideas from the academy to the market. And it’s a feather in their cap when they help a faculty member commercialize an invention. So really, it was about finding out how to use the resources at my disposal to migrate to an ecosystem suited to continuing to improve SELweb and to get it into the hands of educators.

How did xSEL Labs pay for the full development of the version of SELweb ready for use at scale?

CM: Just as we were getting off the ground, we developed

 a partnership with a research funder (the Wallace Foundation) who was interested in using SELweb as an outcome measure in a large-scale field trial of an SEL initiative. They really liked SELweb, but it was clear that in its original form, it simply wouldn’t work at the scale they required. So we worked out a contract that included financial support for improving the system in exchange for discounted fees in the out years of the project.

What agreement did you make with the university in order to start your company and commercial SELweb?

CM: I negotiated a license for the intellectual property from Rush University with the university getting a royalty and a small equity stake in the company.

Did anyone provide you guidance on the business side?

CM: Yes. I lucked into a group of in-laws who happen to be entrepreneurs, some in the education space. And my wife has a sharp business mind. They were helpful. I also sought and found advisors with relevant expertise to help me think through the initial licensing terms, and then pricing, marketing, sales, product development, and the like. One of the nice things about business is that you aren’t expected to know everything. You do need to know how and when to reach out to others for guidance, and how to frame the issues so that guidance is relevant and helpful.

How do you describe the experience of commercializing SELWeb?

CM: Commercialization is, in my experience, an exercise in experimentation and successive approximations. How will you find time and money to test the waters? Commercialization is an exciting and challenging leap from the lab to the marketplace. In my experience, you can’t do it alone, and even with great partners, competitive forces and chance factors make success scale hard to accomplish. Knowing what you don’t know, and finding partners who can help, is critical.

I forgot who described a startup as a temporary organization designed to test whether a business idea is replicable and sustainable. That really rings true. The experience has been about leaving the safe confines of the university and entering the dynamic and endlessly interesting bazaar beyond the ivory tower to see if what I have to offer can solve a problem of practice.

In one sentence (or two!), what would say is most needed for gaining traction in the marketplace?

CM: Figure out who the customer is, what the customer needs, and how what you have to offer addresses those needs. Until you get that down, all the evidence in the world won’t lead to scale.

Do you have advice for university researchers seeking to move their laboratory research into wide-spread practice?

CM: It’s not really practical for most university researchers to shift gears and become an entrepreneur. So I don’t advise doing what I did, although I’m so glad I did. For most university researchers, they should continue doing great science, and when they recognize a scalable idea, consider commercialization as an important option for bringing the idea to scale. My impression is that academic culture often finds commerce to be alien and somewhat grubby, which can get in the way. The truth is, there are whip-smart people in business who have tremendous expertise. The biggest hurdle for many university researchers will be to recognize that they lack expertise in bringing ideas to market, they will need to find that expertise, respect it, and let go of some control as the idea, program, or product is shaped by market forces. It’s also a hard truth for researchers, but most of the world doesn’t care very much about evidence of efficacy. They have much more pressing problems of practice to attend to. Don’t get me wrong—evidence of efficacy is crucial. But for an efficacious idea to go to scale, usability and feasibility are the biggest considerations.

For academics, getting the product into the marketplace requires a new set of considerations, such as: Universities and granting mechanisms reward solo stars; the marketplace rewards partnerships. That is a big shift in mindset, and not easily accomplished. Think partnerships, not empires; listening more than talking.

Any final words of wisdom in moving your intervention from research to practice?

CM: Proving the concept of an ed tech product gets you to the starting line, not the finish. Going to scale benefits from, probably actually requires, the power of the marketplace. Figuring out how the marketplace works and how to fit your product into it is a big leap for most professors and inventors. Knowing the product is not the same as knowing how to commercialize it.

 ____________________________________________________________________________

Clark McKown is a national expert on social and emotional learning (SEL) assessments. In his role as a university faculty member, Clark has been the lead scientist on several large grants supporting the development and validation of SELweb, Networker, and other assessment systems. Clark is passionate about creating usable, feasible, and scientifically sound tools that help educators and their students.

This interview was produced by Ed Metz of the Institute of Education Sciences. This post is the third in an ongoing series of blog posts examining moving from university research to practice at scale in education.

Building the Evidence Base for BEST in CLASS – Teacher Training to Support Young Learners with the Most Challenging Classroom Behavior

Classroom teachers of young children face a seemingly never-ending challenge – how to manage disruptive behavior while simultaneously teaching effectively and supporting the needs of every student in the classroom. Researchers at Virginia Commonwealth University and the University of Florida have received five IES research grants over the past decade – three through the National Center for Special Education Research (NCSER) and two from the National Center for Education Research (NCER) – to develop and test a model of training and professional development, including coaching, for early childhood and early elementary school teachers on how best to support children who engage in disruptive and otherwise challenging classroom behaviors.

With their first IES grant in 2008, Drs. Maureen Conroy and Kevin Sutherland developed the original BEST in CLASS model for early childhood teachers. The goal of BEST in CLASS - PK is to increase the quantity and quality of specific instructional practices with young children (ages 3-5 years old) who engage in high rates of challenging behaviors with the ultimate goal of preventing and reducing problem behavior. Professional development consists of a six-hour workshop that uses didactic and interactive learning activities supported by video examples and practice opportunities. Following the workshop, teachers receive a training manual and 14 weeks of practice-based coaching in the classroom. 

The results of this promising development work led to a 2011 IES Efficacy study to test the impact of BEST in CLASS - PK on teacher practices and child outcomes. Based on positive findings from that Efficacy study the team was awarded two additional Development and Innovation grants – one in 2016 to develop a web-based version of BEST in CLASS – PK to increase accessibility and scalability and another in 2015 to adapt BEST in CLASS – PK for early elementary school classrooms (BEST in CLASS – Elementary). Drs. Sutherland and Conroy are currently in the second year of an Efficacy study to test the impact of BEST in CLASS - Elementary to determine if the positive effects of BEST in CLASS in preschool settings are replicated in early elementary classrooms.

Written by Emily Doolittle, NCER Team Lead for Social Behavioral Research, and Jacquelyn Buckley, NCSER Team Lead for Disability Research

Measuring Social and Emotional Learning in Schools

Social and emotional learning (SEL) has been embraced by many schools and districts around the country. Yet in the rush to adopt SEL practices and support student SEL competencies, educators often lack assessment tools that are valid, reliable, and easy to use.

 

Washoe County School District in Nevada has moved the needle on SEL assessment with support from an IES Researcher-Practitioner Partnership grant. The district partnered with the Collaborative for Academic, Social, and Emotional Learning (CASEL) to develop the Social and Emotional Competency Assessments (WCSD-SECAs)—free, open-source instruments that schools can use to measure SEL competencies of students in 5th through 12th grade.

Long and short versions of the SECA are available to download from the school district’s website, along with a bank of 138 items across 8 SEL domains that schools around the country can use to modify SECA assessments for their local context. The long-form version has been validated and aligned to the CASEL 5 SEL competency clusters and WCSD SEL standards (self-awareness, self-management, social awareness, relationship skills, and responsible decision making). The assessment is also available in Spanish, and the Metro Nashville Public schools offer the assessment in 8 additional languages.  

Students complete the long-form SECA as part of Washoe’s Annual Student Climate Survey by rating how easy or difficult SEL skills are for them. Under the Social Awareness domain, students respond to items such as “Knowing what people may be feeling by the look on their face” or “Learning from people with different opinions than me.” Under the Responsible Decision Making domain, students rate themselves on skills such as “Saying ‘no’ to a friend who wants to break the rules” and “Thinking of different ways to solve a problem.”

The SECA is one component of Washoe County’s larger School Climate Survey Project that is marking its 10th anniversary this year. Washoe provides district-level and school-level reports on school climate to support the district’s commitment to providing safe, caring, and engaging school environments for all of Washoe’s students and families.  

Written by Emily Doolittle, NCER’s Team Lead for Social Behavioral Research

Building Partnerships to Support Mental Health Needs in Diverse Rural Schools: The National Center for Rural School Mental Health

About 1 in 5 school-age children experience serious mental health issues yet few receive services. In rural schools, geographic isolation and limited resources make receiving services even more difficult. The IES-funded National Center for Rural School Mental Health is addressing this challenge.

The 5 year, $10 million National R&D Center is supporting partnerships with a wide variety of rural school districts in three states (Missouri, Virginia, and Montana) to develop and test ways to support the mental health needs of their students. I recently spoke with Dr. Wendy Reinke, the Center’s director, about the unique mental health needs in the rural settings where the center is working and how she and her colleagues are approaching this work.  

Tell us a little bit about the rural communities you are partnering with in Missouri, Virginia, and Montana.

Each state provides a unique geological context that we anticipate will inform the tools and interventions we are developing for wide use in rural schools. For instance, Missouri sits in the middle of the country where half of the school districts are considered rural and another third or so are considered small towns. Virginia encompasses central Appalachia which struggles with issues of under-employment, mental health, and school dropout. In the northwest, rural residents are scattered across Montana’s 56 counties, 30 of which are classified as “frontier” counties with three or fewer persons per square mile.  The tools and interventions we develop will need to be feasible and effective across these very different contexts.

What are the most common mental health challenges being faced in the different rural communities you are partnering with?

Part of the work of the Rural Mental Health Center will be learning more about the types of  mental health challenges faced by rural communities. From my current work in Missouri’s rural schools, common areas of concern include youth with depression, anxiety, conduct problems, substance abuse, and suicidality.  Identifying youth early can help to prevent or reduce the burden of these problems.  Accordingly, we plan to not only offer interventions for youth facing mental health challenges but work with schools to prevent and identify early, youth who would benefit from supports.

The work you have planned for the center builds on prior IES-funded work. Tell us more about how this work provides a foundation for launching the work of the center.

A cornerstone of the Center is the use of an assessment tool that will allow schools to gather data to determine their needs for school-level prevention, group-based interventions, and individualized interventions.  This tool was developed in partnership with six school districts (five of which are rural) and University of Missouri researchers.  Through the IES partnership grant we were able to validate the measure and gather stakeholder input to improve the tool and the overall intervention model.  These data collected using this tool will be linked to evidence-based interventions, several of which have been developed and evaluated through IES funding.  It is very exciting to have the opportunity to pull all of these projects together to support our rural schools.

Much of your earlier research has been done in urban school districts. How did you become interested in research with rural schools? What would you recommend to other researchers interested in doing research with rural schools?

I grew up and attended school in a rural coal-mining town in Pennsylvania. When I moved to Missouri, I had access and opportunity in working alongside rural school districts.  One recommendation, which I think goes for research in any schools, is to operate as a partner with them. For instance, the six school districts we worked with formed a Coalition, and we include the Coalition as co-authors on any publication or presentation that comes from this work.  Further, we present with partners at conferences and report back findings to the community.  I think an open and collaborative relationship gains trust, allowing for additional opportunities to conduct research alongside our school partners. Additionally, our ideas for studies are nearly always driven by the needs expressed by our schools based on the pressing challenges they report to us.

The Center also has a policy focus with work that will be led by your Montana partners. Tell us more about this aspect of the Center’s work and the types of policy issues the Center will address.

We will be working with rural school district partners across the three states to identify important issues facing rural schools.  Dr. Ryan Tolleson-Knee from the University of Montana will be leading this initiative.  At the Center kick-off meeting held in June, a subgroup of rural school partners interested in policy was formed.  The plan is for this subgroup to develop a toolkit that can be readily used by public school personnel and state and national policymakers to improve outcomes for youth.  One topic of interest is how might rural school districts partner with one another (similar to the Coalition described earlier) to maximize and share resources across the communities.  Over the next five years, the toolkit will expand and connect to issues faced by our rural schools.

Written by Emily Doolittle, NCER Team Lead for Social Behavioral Research

Are You What You Eat? Understanding the Links Between Diet, Behavior, and Achievement During Middle School

We’ve all heard the phrase “you are what you eat,” but what exactly does it mean for student learning and achievement in middle school? In 2018, researchers from the University of Alabama at Birmingham received an IES Exploration grant to investigate the direction and nature of the relationships between middle school students’ diet, behavior, and academic achievement. These relationships have not been fully studied in the United States, nor have longitudinal designs been used (most existing studies are cross-sectional) making it hard to determine the precise nature of the links between what adolescents eat and potential implications for learning and achievement.  

Because children in the United States consume about half of their nutrients at school, the need to identify school nutrition policies and practices that benefit student behavior and achievement is great, especially given newly published findings that motivated this IES research and that have attracted lots of media interest in recent days (see this story from CNN and this press release). The Alabama researchers found that specific nutrients (high sodium, low potassium) predicted depression over a year later in a sample of 84 urban, primarily African American adolescents (mean age 13 years). In the IES study, these researchers are expanding their work with a larger and more diverse sample of 300 students. In the first year of this 4-year study, the researchers recruited about two thirds of their sample (186 students across 10 schools) who completed the first of three week-long assessments as 6th graders and who will complete assessments again in the 7th and 8th grades. During each week-long assessment period, each student reports on their own diet and academic functioning, and on their own and their peers’ emotions and behavior. They also complete objective tests of attention and memory. The researchers observe each child’s actual food and beverage consumption at school and behavior during one academic class period. They also collect school records of grades, test scores, attendance, discipline incidents, and information about each school’s nutrition policies and practices. Parents and teachers also report on student diet, behavior, and academic functioning.

This school year the researchers are recruiting the rest of their sample. If their findings suggest a role for school practices and dietary factors in student behavior and achievement, they can guide future efforts to develop school-based programs targeting students’ diet that could be easily implemented under typical school conditions.

Written by Emily Doolittle, NCER Team Lead for Social Behavioral Research