IES Blog

Institute of Education Sciences

How Remote Data Collection Enhanced One Grantee’s Classroom Research During COVID-19

Under an IES grant, Michigan State University, in collaboration with the Michigan Department of Education, the Michigan Center for Educational Performance and Information, and the University of Michigan, is assessing the implementation, impact, and cost of the Michigan “Read by Grade 3” law intended to increase early literacy outcomes for Michigan students. In this guest blog, Dr. Tanya Wright and Lori Bruner discuss how they were able to quickly pivot to a remote data collection plan when COVID-19 disrupted their initial research plan.  

The COVID-19 pandemic began while we were planning a study of early literacy coaching for the 2020-2021 academic year. It soon became abundantly clear that restrictions to in-person research would pose a major hurdle for our research team. We had planned to enter classrooms and record videos of literacy instruction in the fall. As such, we found ourselves faced with a difficult choice: we could pause our study until it became safer to visit classrooms and miss the opportunity to learn about literacy coaching and in-person classroom instruction during the pandemic, or we could quickly pivot to a remote data collection plan.

Our team chose the second option. We found that there are multiple technologies available to carry out remote data collection. We chose one of them (a device known as the Swivl) that included a robotic mount, where a tablet or smartphone can be placed to take the video, with a 360-degree rotating platform that works in tandem with a handheld or wearable tracker and an app that allows videos to be instantly uploaded to a cloud-based storage system for easy access.

Over the course of the school year, we captured over 100 hours of elementary literacy instruction in 26 classrooms throughout our state. While remote data collection looks and feels very different from visiting a classroom to record video, we learned that it offers many benefits to both researchers and educators alike. We also learned a few important lessons along the way.

First, we learned remote data collection provides greater flexibility for both researchers and educators. In our original study design, we planned to hire data collectors to visit classrooms, which restricted our recruitment of schools to a reasonable driving distance from Michigan State University (MSU). However, recording devices allow us to capture video anywhere, including rural areas of our state that are often excluded from classroom research due to their remote location. Furthermore, we found that the cost of purchasing and shipping equipment to schools is significantly less than paying for travel and people’s time to visit classrooms. In addition, using devices in place of data collectors allowed us to easily adapt to last-minute schedule changes and offer teachers the option to record video over multiple days to accommodate shifts in instruction due to COVID-19.

Second, we discovered that we could capture more classroom talk than when using a typical video camera. After some trial and error, we settled on a device with three external wireless microphones: one for the teacher and two additional microphones to place around the classroom. Not only did the extra microphones record audio beyond what the teacher was saying, but we learned that we can also isolate each microphone during data analysis to hear what is happening in specific areas of the classroom (even when the teacher and children were wearing masks). We also purchased an additional wide-angle lens, which clipped over the camera on our tablet and allowed us to capture a wider video angle.  

Third, we found remote data collection to be less intrusive than sending a research team into schools. The device is compact and can be placed on any flat surface in the classroom or be mounted on a basic tripod. The teacher has the option to wear the microphone on a lanyard to serve as a hands-free tracker that signals the device to rotate to follow the teacher’s movements automatically. At the end of the lesson, the video uploads to a password-protected storage cloud with one touch of a button, making it easy for teachers to share videos with our research team. We then download the videos to the MSU server and delete them from our cloud account. This set-up allowed us to collect data with minimal disruption, especially when compared to sending a person with a video camera to spend time in the classroom.

As with most remote work this year, we ran into a few unexpected hurdles during our first round of data collection. After gathering feedback from teachers and members of our research team, we were able to make adjustments that led to a better experience during the second round of data collection this spring. We hope the following suggestions might help others who are considering such a device to collect classroom data in the future:

  1. Consider providing teachers with a brief informational video or offering after-school training sessions to help answer questions and address concerns ahead of your data collection period. We initially provided teachers with a detailed user guide, but we found that the extra support was key to ensuring teachers had a positive experience with the device. You might also consider appointing a member of your research team to serve as a contact person to answer questions about the remote data collection during data collection periods.
  2. As a research team, it is important to remember that team members will not be collecting the data, so it is critical to provide teachers with clear directions ahead of time: what exactly do you want them to record? Our team found it helpful to send teachers a brief two-minute video outlining our goals and then follow up with a printable checklist they could use on the day they recorded instruction. 
  3. Finally, we found it beneficial to scan the videos for content at the end of each day. By doing so, we were able to spot a few problems, such as missing audio or a device that stopped rotating during a lesson. While these instances were rare, it was helpful to catch them right away, while teachers still had the device in their schools so that they could record missing parts the next day.

Although restrictions to in-person research are beginning to lift, we plan to continue using remote data collection for the remaining three years of our project. Conducting classroom research during the COVID-19 pandemic has proven challenging at every turn, but as we adapted to remote video data collection, we were pleased to find unanticipated benefits for our research team and for our study participants.


This blog is part of a series focusing on conducting education research during COVID-19. For other blog posts related to this topic, please see here.

Tanya S. Wright is an Associate Professor of Language and Literacy in the Department of Teacher Education at Michigan State University.

Lori Bruner is a doctoral candidate in the Curriculum, Instruction, and Teacher Education program at Michigan State University.

Educating English Learner Students During the Pandemic: Remote and In-Person Instruction and Assessment

The IES-funded R&D Center, the Center for the Success of English Learners (C-SEL), is undertaking a focused program of research aimed at improving access and outcomes for English Learners. One of C-SEL’s recent activities has been to develop resources to aid policymakers and practitioners working with middle school and secondary English learners. The research team at the Center, including Drs Diane August and Coleen Carlson, along with Maria Yolanda Cieslak and Kenneth Michael Nieser, recently released a brief on Educating English Learner Students During the Pandemic: Remote & In-person Instruction & Assessment-Recommendations and Resources for State and Districts. In this guest blog, the researchers provide an overview of the brief.

English Learners (ELs) benefit from specialized support to help them acquire second language proficiency and core content knowledge that builds on their cultural and linguistic assets. This specialized support is required by law, and the U.S. Department of Education reminded States that this is the case, even when learning is remote. Based on a review of the existing relevant literature, this brief provides detailed information related to the impact of remote learning on English Learners (ELs) and their teachers during the pandemic and the potential and limitations of using digital learning resources (DLRs) to educate these students. For instance, while DLRs have the potential to support learning and engagement for ELs, districts report barriers to their use, such as lack of home access to DLRs; teachers’ level of expertise and technology skills; and the lack of knowledge around what are the appropriate DLRs for ELs.

The brief also describes current legislation that authorizes funds for a variety of activities that could be used to support ELs and their families when instruction is delivered remotely. Some of these federal resources include:

  • Formula Grant Programs Under the Every Student Succeeds Act
  • CARES Act
  • Coronavirus Response and Relief Supplemental Appropriations Act

In addition, the brief includes information about policies and resources for each of the 50 states in place in December 2020 that support districts and schools in instructing and assessing EL students remotely. It also includes a reading list of recent resources focused on remote learning for ELs, with brief descriptions of each resource and links to the resource.

In March 2021, the Center hosted a webinar to discuss recent recommendations for states and districts. One set of recommendations focuses on methods to enhance the learning and emotional well-being of EL students who have lost ground during the pandemic. Recommendations are also made for assessing ELs when schooling is or has been remote or hybrid. These recommendations can be found in the brief.  

In the upcoming months, C-SEL investigators look forward to preparing future blog posts and research briefs on the research of the Center and the students and teachers we are serving. Next up on the agenda is an overview of the students, highlighting their diversity, and some too often ignored, forgotten, or simply unknown characteristics of this important subgroup. Stay tuned!


Dr. Diane August is Principal at D. August and Associates and a Senior Research Scientist at the Center for Applied Linguistics.

Dr. Coleen Carlson is an Associate Research Professor, and Associate Director at the Texas Institute for Measurement, Evaluation, and Statistics (TIMES) at the University of Houston.

Ms. Maria Yolanda Cieslak is a Professional Development Specialist at the Center for Applied Linguistics.

Mr. Kenneth Michael Nieser is a Researcher at the Texas Institute for Measurement, Evaluation, and Statistics at the University of Houston.

 

Supporting Teachers and their English Learners during Online Learning

Under an IES grant, Drs. Leslie Babinski, Steve Amendum, Steve Knotek, and Marta Sánchez are evaluating the impact of the Developing Consultation and Collaboration Skills (DCCS) program. The DCCS program is a year-long professional development intervention designed to support English-as-a-Second Language (ESL) and classroom teachers to develop their skills in collaboration, literacy instruction, and parent outreach and engagement for their Latino students. Unfortunately, COVID-19 disrupted their study, but rather than calling it quits, the research team used it as an opportunity to explore how teachers are responding to a new normal and learned some important lessons along the way.

 

Last spring, we were in the middle of conducting a randomized control trial in elementary schools working with teaching teams of kindergarten, first grade, and ESL teachers when COVID-19 hit, and schools pivoted to online instruction.

As we paused our efficacy trial and adapted our plans, we continued to stay engaged with teachers despite the rapidly shifting circumstances. We realized that the foundational skills central to our intervention—using high-impact instructional strategies for English learners (ELs), building on families’ cultural wealth, promoting collaboration between ESL and classroom teachers, and ongoing, supportive implementation coaching—could be modified and adapted for different contexts and modes of delivery. These ongoing interactions with teachers provided us with an up-close look at their experiences as they connected with families remotely and adapted to this new and unfamiliar way of teaching. We learned about the challenges, successes, and possibilities for providing online instruction for young ELs.

While online teaching during the current school year looks and feels very different from the emergency shift to remote teaching from last spring, we learned important lessons from the experiences of teachers and students.

 

First, we learned that our implementation coaching model could be productive in a virtual format. In fact, teachers were eager to engage in reflective conversations despite the demands on their time and the stress of the pandemic. Each of the school teams continued to meet with our implementation coach. Together, they found new ways to scaffold instruction for ELs during online learning. Teachers supported one another in finding creative ways to use technology to communicate with families and provide support that reached beyond academics. They also created lessons that intentionally adapted instruction to reach ELs in their classrooms. For example, embedded in the videos the teachers created, they continued to use our core instructional strategies, such as previewing academic vocabulary before reading a new text, supplementing student background knowledge related to the content, providing sentence frames for both oral and written participation, and selecting texts related to student cultures and families.

 

Second, it is clear that there are serious inequities in access to online schooling. In our study, even as teachers made the commendable efforts described above, they also reported that more than half of their EL students required a device from the school district, while one-third of the families also needed a Wi-Fi hotspot to access online learning. This rate is considerably higher than a Pew Research Center poll that found that one in five parents reported difficulties with online learning due to lack of a computer or Wi-Fi access, highlighting the fact that many English Learners may have limited access to the technology necessary for online learning.

 

Third, online learning requires a partnership between teachers, families, and other caregivers. Even with a device and internet access, many young children and their families had difficulty logging onto their school accounts and navigating the technical aspects of the learning platforms for online schooling. Teachers reported that about half of the EL students in their classrooms had a parent or guardian who was able to help them with schoolwork at home. About 20% had help from another adult or a sibling. In one case, a teacher described a family in which a kindergarten student received support for online schooling from her brother in second grade. In our study, support at home was critical for student participation in online lessons. Our work with teachers during remote teaching and learning highlighted how parents and other caregivers are important advocates for their children’s education and are eager to partner with teachers and schools to help their children succeed.

 

Looking to the future, it is clear that instruction for ELs will need to focus on equitable access to high-quality instruction, whether online or in-person. Access to devices and Wi-Fi is essential, not only for online teaching and learning, but also for extending learning into the home. Finally, we note that teacher collaboration and coaching can be effectively adapted for an online environment and is an essential component in providing support to teachers for high-quality instruction for ELs.

Although we could not have anticipated the value of continuing to work with teachers during the pivot to remote instruction, we are grateful for the experience and all that we have learned during the process.

 

For more information, see this EdWeek article and this interview on The TakeAway.


Dr. Leslie Babinski is an associate research professor in the Sanford School of Public Policy and the Director of the Center for Child and Family Policy.

Dr. Steve Amendum is a professor at the School of Education at the University of Delaware.

Dr. Steve Knotek is a professor in the School of Education at the University of North Carolina at Chapel Hill.

Dr. Marta Sánchez is an associate professor in the Watson College of Education at the University of North Carolina Wilmington.

This is part of a series of blog posts focusing on conducting education research during COVID-19.

 

Addressing COVID-19’s Disruption of Student Assessment

Under an IES grant, the RAND Corporation, in collaboration with NWEA, is developing strategies for schools and districts to address the impacts of COVID-19 disruptions on student assessment programs. The goal is to provide empirical evidence of the strengths and limitations of strategies for making decisions in the absence of assessment data. Jonathan Schweig, Andrew McEachin, and Megan Kuhfeld describe early findings from surveys and structured interviews regarding key concerns of districts and schools. 

 

As a first step, we surveyed assessment and research coordinators from 23 school districts (from a sample of 100 districts) and completed follow-up interviews with seven of them on a variety of topics, including the re-entry scenario for their district, the planning activities that they were not able to perform this year due to coronavirus-based disruptions to spring 2020 assessments, and the strategies they were employing to support instructional planning in the absence of assessment data. While the research is preliminary and the sample of respondents is not nationally representative, the survey and interview responses identified two key concerns arising from the lack of spring 2020 assessment data which has made it challenging to examine student or school status and change over time, especially as COVID-19 has differential impacts on student subgroups:

 

  • Making course placement decisions. Administrators typically rely on spring assessment scores—often in conjunction with other assessment information, course grades, and teacher recommendations—to make determinations for course placements, such as who should enroll in accelerated or advanced mathematics classes. 
  • Evaluating programs or district-wide initiatives. Many districts monitor the success of these programs internally by looking at year-to-year change or growth for schools or subgroups of interest. 

 

How are school systems responding to these challenges? Not surprisingly, the responses vary depending on local contexts and resources. Where online assessments were not feasible in spring 2020, some school districts used older testing data to make course recommendations, either from the winter or from the previous school year. Some districts relaxed typical practice and provided more autonomy to individual schools, relying on school staff to exercise local judgment around course placements and using metrics like grades and teacher recommendations. Other districts reported projecting student scores based on student assessment histories. Relatedly, some districts were already prepared for this decision because they had recently experienced difficulties with adopting an online assessment system and had to address similar problems caused by large numbers of missing or invalid tests.

 

School districts also raised concerns about whether assessments administered during the 2020-21 school year would be valid and comparable so that they could be used in student placement and program evaluation decisions. These concerns included the following:

  • Several respondents raised concerns about the trustworthiness of remote assessment data collected this fall and the extent to which results could be interpreted as valid indicators of student achievement or understanding.
  • Particularly for districts that started the 2020-21 school year remotely, respondents were concerned about student engagement and motivation and the possibility of students rushing assessments, running into technological or internet barriers, or seeking assistance from guardians or other resources. 
  • Respondents raised questions about the extent to which available assessment scores are representative of school or district performance as a whole. Given that vulnerable students (for example, students with disabilities, students experiencing homelessness) may be the least likely to have access to remote instruction and assessments, it is likely that the students who are not assessed this year are different from students who are able to be assessed.
  • Other respondents noted that they encountered resistance from parents around fall assessment because they prioritized student well-being (for example, safety, sense of community, and social and emotional well-being) more so than academics. This is a perspective that resonates with recent findings from a nationally representative sample of teachers and school leaders drawn from RAND’s American Educator Panel (AEP).

 

In the next phase of the work, the research team plans to:

  • Conduct a series of simulation and empirical studies regarding the most common strategies that the district respondents indicated they were using to make course placement decisions and to evaluate programs or district-wide initiatives.
  • Provide a framework to help guide local research on the intended (and unintended) consequences for school and school system decision making when standardized test scores are not available.

 

We welcome individuals to reach out to RAND with additional recommendations or considerations. We are also interested in hearing how districts are approaching course placement, accountability, and program evaluation across the country. Connect with the research team via email at jschweig@rand.org.

 


Jonathan Schweig is a social scientist at the nonprofit, nonpartisan RAND Corporation.

Andrew McEachin is a senior policy researcher at the nonprofit, nonpartisan RAND Corporation.

Megan Kuhfeld is a researcher at NWEA.

Webinar Recap: EdTech Resources for Special Education Practitioners

It goes without saying the COVID19 pandemic has and continues to have a profound effect on education. Students are adjusting to hybrid or fully remote learning, and educators are continuing to make complex decisions about how best to support students in the new normal.

On October 28, 2020, InnovateEDU and the Educating All Learners Alliance hosted a webinar focused on education technology resources for special education. More than 1,100 practitioners joined the event in real-time.

 

 

The webinar featured video demonstrations of five special education technology tools that were developed through the IES Small Business Innovation Research Program and ED’s Office of Special Education Educational Technology, Media, and Materials for Individuals with Disabilities Program. The event also included conversations with special education practitioners and researchers who provided perspectives on the role of special education and technology to meet the needs of all students. The webinar involved a variety of resources and opportunities, including:

 

During the webinar, practitioners participated by adding comments in the chat box with a “wish list” of education technology they would like to have now to support teaching and learning. Participants entered dozens of responses, many calling for increased connectivity and access to hardware and software, especially in rural areas. Other responses focused on education technologies for teachers, students with or at-risk for disabilities, and parents and caregivers.

Following are just a few of the entries:

 

For Teachers

  • “More coaching tools to use with children who are learning remotely to provide instantaneous feedback”
  • “Descriptions that allow teachers to at-a-glance identify the features a program offers to match to the features that their students need”
  • “Using data to support teachers and students with decisions that move learning forward.”
  • “Resources that I can use to assist with non-compliant behaviors and keeping their attention in person and virtually.”
  • Making it possible for students to show their work for math so that we can see that rather than just their answers.”
  • “Common share place for all teachers.”
  • “I am looking for a way to deliver instructions to the home distantly”

 

For Students with Disabilities

  • “Teaching students how to be self-determined learners.”
  • “Build this skill set from kindergarten.”
  • “Develop and implement collaborative activities”
  • “My nonverbal students need hands on.”
  • “Engagement and motivation; remote resources.”
  • “Student choice and voice.”

 

For Parents

  • “Make it a family affair / Zoom with family member supporting on other side.”
  • “A resource that we can use to incorporate the parent or group home worker that have to navigate these different learning apps for the student.”
  • “Easy-to-follow videos that we can use to show parents and students how to use these resources when they aren’t in front of us.”

 

Lastly, one of the teachers provided a comment: “We need more of these events.”  From everyone involved in the October 28 webinar, thanks for attending. We are planning for more events like this one soon.

 


Edward Metz (Edward.Metz@ed.gov) is a research scientist at the Institute of Education Sciences in the US Department of Education.

Tara Courchaine (Tara.Courchaine@ed.gov) is a program officer at the Office of Special Education Programs in the US Department of Education.