IES Blog

Note: NCEE will continue to accept study nominations after the April 3^rd deadline, adding them on a regular basis to our growing bibliography found here. Studies received before the deadline will be considered for the June 1 data release. NCEE will use studies received after the deadline to inform our prioritization of studies for review. Awareness of these studies will also allow NCEE to consider them for future activities related to distance and/or online education and remote learning.

In the midst of the coronavirus crisis, we know that families and educators are scrambling for high-quality information about what works in distance education—a term we use here to include both online learning as well as opportunities for students to use technology or other resources to learn while not physically at school.

Leaders in the education technology ecosystem have already begun to respond to the COVID-19 outbreak by creating websites like techforlearners.org, which as of today lists more than 400 online learning products, resources, and services. But too little information is widely available about what works in distance education to improve student outcomes.

If ever there is a time for citizen science, it is now. Starting today, the What Works Clearinghouse™ (WWC) at the U.S. Department of Education’s Institute of Education Sciences is announcing its first-ever cooperative rapid evidence synthesis.

Here is what we have in mind:

• Between now and April 3^rd, we are asking families and educators to share with us questions they have about effective distance education practices and products. We are particularly interested in questions about practices that seem especially relevant today, in which educators are called to adapt their instruction to online formats or send learning materials home to students, and families, not all of whom have internet access, seek to combine available technology with other resources to create a coherent learning experience for their students. Early education, elementary, postsecondary, and adult basic education practices and products are welcome. Submit all nominations to NCEE.Feedback@ed.gov.
• During that same time, we are asking that members of the public, including researchers and technologists, nominate any rigorous research they are aware of or have conducted that evaluates the effectiveness of specific distance education practices or products on student outcomes. As above,  education, elementary, postsecondary, and adult basic education practices and products are welcome.
  • Submit all nominations to NCEE.Feedback@ed.gov. Nominations should include links to publicly available versions of studies wherever possible.
  • Study authors are strongly encouraged to nominate studies as described above and simultaneously submit them to ED’s online repository of education research, ERIC. Learn more about the ERIC submission process here.
  • We will post a link to a list of studies on this page and update it on a regular basis.
     
• By June 1, certified WWC reviewers will have prioritized and screened as many nominated studies as resources allow. Based on the responses received from families, educators, researchers, and technologists, we may narrow the focus of our review; however, nominations will be posted to our website, even those we do not review. Reviews will be entered in the WWC’s Review of Individual Studies Database, which can be downloaded as a flat file.
   
• After June 1, individual meta-analysts, research teams, or others can download screened studies from the WWC and begin their meta-analytic work. As researchers complete their syntheses, they should submit them through the ERIC online submission system and alert IES. Although we cannot review each analysis or endorse their findings, we will do our best to announce each new review via social media—amplifying your work to educators, families, and other interested stakeholders. Let me know at NCEE.Feedback@ed.gov if this part of the work is of interest to you or your colleagues.

Will you help, joining the WWC’s effort to generate high-quality information about what works in distance education? If so, submit your study today, let me know you or your team are interested in lending your meta-analytic skills to the effort, or just provide feedback on how to make this work more effectively. You can reach me directly at matthew.soldner@ed.gov.

Matthew Soldner

Commissioner, National Center for Education Evaluation and
Agency Evaluation Officer, U.S. Department of Education

Over the summer, researchers, technologists, and policymakers gathered in Accra, Ghana for the Association for Computing Machinery’s Conference on Computing and Sustainable Societies (ACM COMPASS) to discuss the role of information technologies in international development.

Two IES-funded researchers from Carnegie Mellon University’s Program in Interdisciplinary Education Research, Michael Madaio and Dr. Amy Ogan, shared their research on developing voice-based early literacy technologies and evaluating their efficacy with low-literate, bilingual families in the Ivory Coast. 

Their research draws on methods from human-computer interaction, the learning sciences, and information-communication technology for development, to design educational technologies that are culturally and contextually appropriate.

Although the COMPASS conference focused on cross-cultural applications and technology for development, the research presented has implications for U.S. based education researchers, practitioners, and policymakers.

For instance, while research provides evidence for the importance of parental involvement in early literacy, parents with low literacy in the target language – as in many bilingual immigrant communities in the U.S. – may not be able to support their children with the explicit, instrumental help suggested by prior research (for example, letter naming or bookreading). This suggests that there may be opportunities for technology to scaffold low-literate or English Learners (EL) parental support in other ways.

At the conference, researchers described interactive voice-based systems (known as “IVR”) that help low-literate users find out about crop yields, understand local government policies, and engage on social media.  

This body of work has implications for designers of learning technologies in the U.S. Many families may not have a smartphone, but basic feature phones are ubiquitous worldwide, including in low-income, immigrant communities in the U.S. Thus, designers of learning technologies may consider designing SMS- or voice-based (such as IVR) systems, while schools or school districts may consider how to use voice-based systems to engage low-literate or EL families who may not have a smartphone or who may not be able to read SMS information messages.

In a rapidly changing, increasingly globalized world, research at IES may benefit from increased international engagement with international research, both focusing specifically on education, as well as information technology research that has implications for educational research, practice, and policy.

This guest blog was written by Michael Madaio. He is an IES Predoctoral Fellow in the Program in Interdisciplinary Education Research at Carnegie Mellon University. He is placed in the Human-Computer Interaction Institute.

By Michael Frye and Sarah Costelloe. Both are part of Abt Associates team working on the What Works Clearinghouse.

Technology is part of almost every aspect of college life. Colleges use technology to improve student retention, offer active and engaging learning, and help students become more successful learners. The What Works Clearinghouse’s latest practice guide, Using Technology to Support Postsecondary Student Learning, offers several evidence-based recommendations to help higher education instructors, instructional designers, and administrators use technology to improve student learning outcomes.

IES practice guides incorporate research, practitioner experience, and expert opinions from a panel of nationally recognized experts. The panel that developed Using Technology to Support Postsecondary Student Learning included five experts with many years of experience leading the adoption, use, and research of technology in postsecondary classrooms.  Together, guided by Abt Associates’ review of the rigorous research on the topic, the Using Technology to Support Postsecondary Student Learning offers five evidence-based recommendations:

Each recommendation is assigned an evidence level of minimal, moderate, or strong. The level of evidence reflects how well the research demonstrates the effectiveness of the recommended practices. For an explanation of how levels of evidence are determined, see the Practice Guide Level of Evidence Video.   The evidence-based recommendations also include research-based strategies and examples for implementation in postsecondary settings. Together, the recommendations highlight five interconnected themes that the practice guide’s authors suggest readers consider:

• Focus on how technology is used, not on the technology itself.

“The basic act of teaching has actually changed very little by the introduction of technology into the classroom,” said panelist MJ Bishop, “and that’s because simply introducing a new technology changes nothing unless we first understand the need it is intended to fill and how to capitalize on its unique capabilities to address that need.” Because technology evolves rapidly, understanding specific technologies is less important than understanding how technology can be used effectively in college settings. “By understanding how a learning outcome can be enhanced and supported by technologies,” said panelist Jennifer Sparrow, “the focus stays on the learner and their learning.”

• Technology should be aligned to specific learning goals.

Every recommendation in this guide is based on one idea: finding ways to use technology to engage students and enhance their learning experiences. Technology can engage students more deeply in learning content, activate their learning processes, and provide the social connections that are key to succeeding in college and beyond. To do this effectively, any use of technology suggested in this guide must be aligned with learning goals or objectives. “Technology is not just a tool,” said Panel Chair Nada Dabbagh. “Rather, technology has specific affordances that must be recognized to use it effectively for designing learning interactions. Aligning technology affordances with learning outcomes and instructional goals is paramount to successful learning designs.”

• Pay attention to potential issues of accessibility.

The Internet is ubiquitous, but many households—particularly low-income households and those of recent immigrants and in rural communities—may not be able to afford or otherwise access digital communications. Course materials that rely heavily on Internet access may put these students at a disadvantage. “Colleges and universities making greater use of online education need to know who their students are and what access they have to technology,” said panelist Anthony Picciano. “This practice guide makes abundantly clear that colleges and universities should be careful not to be creating digital divides.”

Instructional designers must also ensure that learning materials on course websites and course/learning management systems can accommodate students who are visually and/or hearing impaired. “Technology can greatly enhance access to education both in terms of reaching a wide student population and overcoming location barriers and in terms of accommodating students with special needs,” said Dabbagh. “Any learning design should take into consideration the capabilities and limitations of technology in supporting a diverse and inclusive audience.”

• Technology deployments may require significant investment and coordination.

Implementing any new intervention takes training and support from administrators and teaching and learning centers. That is especially true in an environment where resources are scarce. “In reviewing the studies for this practice guide,” said Picciano, “it became abundantly clear that the deployment of technology in our colleges and universities has evolved into a major administrative undertaking. Careful planning that is comprehensive, collaborative, and continuous is needed.”

“Hardware and software infrastructure, professional development, academic and student support services, and ongoing financial investment are testing the wherewithal of even the most seasoned administrators,” said Picciano. “Yet the dynamic and changing nature of technology demands that new strategies be constantly evaluated and modifications made as needed.”

These decisions are never easy. “Decisions need to be made,” said Sparrow, “about investment cost versus opportunity cost. Additionally, when a large investment in a technology has been made, it should not be without investment in faculty development, training, and support resources to ensure that faculty, staff, and students can take full advantage of it.”

• Rigorous research is limited and more is needed.

Despite technology’s ubiquity in college settings, rigorous research on the effects of technological interventions on student outcomes is rather limited. “It’s problematic,” said Bishop, “that research in the instructional design/educational technology field has been so focused on things, such as technologies, theories, and processes, rather than on the problems we’re trying to solve with those things, such as developing critical thinking, enhancing knowledge transfer, and addressing individual differences. It turns out to be very difficult to cross-reference the instructional design/educational technology literature with the questions the broader field of educational research is trying to answer.”

More rigorous research is needed on new technologies and how best to support instructors and administrators in using them. “For experienced researchers as well as newcomers,” said Picciano, “technology in postsecondary teaching and learning is a fertile ground for further inquiry and investigation.”

Readers of this practice guide are encouraged to adapt the advice provided to the varied contexts in which they work. The five themes discussed above serve as a lens to help readers approach the guide and decide whether and how to implement some or all of the recommendations.

Download Using Technology to Support Postsecondary Student Learning from the What Works Clearinghouse^™ website at https://ies.ed.gov/ncee/wwc/PracticeGuide/25.

In a recent blog post, NCES announced the groundbreaking work of the NCES Ed Tech Equity Initiative. The Center’s efforts for this initiative focus on working with stakeholders to identify how NCES data collection, reporting, and dissemination efforts can better inform the relationship between technology and K–12 students’ educational experiences and outcomes. 

THE FRAMEWORK

As part of these efforts, NCES developed a framework to better understand the various facets that influence technology in K–12 education, as well as how these facets interact. The framework was created through extensive research and is designed to be revised over time to align with changes in the ed tech equity space.

The NCES Ed Tech Equity Framework, included below, is comprised of four critical components—Indicators (located in the center of the framework), Dimensions and Environments (the green and purple circle), and Change Agents (shown in the outer gray circle).

HOW IT WORKS

The interaction of the framework elements informs ed tech equity and NCES data collection:

• Indicators represent the broad categories used to measure or assess education technology—relevant NCES survey questions will fit within at least one of the Indicator categories.
• Dimensions are the key perspectives through which NCES focuses its ed tech equity data collection efforts.
• Environments are the settings that facilitate educational experiences.
• Finally, Change Agents are factors that impact or influence students’ educational experiences and outcomes.

Below, a few existing NCES items are mapped to the framework to illustrate how it will be used in NCES data collection:

• TECHNOLOGY RESOURCES AND SUPPORT

• TEACHING IN-SCHOOL: In this school year, did your school offer training for teachers on how to use computers or other digital devices?  —NAEP, 2017

• TECHNOLOGY KNOWLEDGE, SKILLS, AND ATTITUDES

• TEACHING OUT-OF-SCHOOL: During the last 12 months, which of the following activities have you or another family member done with [your 9th grader]?

- Worked or played on a computer together  —HSLS, 2009

• INTEGRATION OF TECHNOLOGY

• LEARNING IN SCHOOL: Do you use the Internet to do any of the following tasks for schoolwork (including classroom tasks, homework, studying outside of class)?

- c) Collaborate with classmates on assignments or projects  —TIMSS, 2015​

NEXT STEPS

NCES recently convened an expert panel to assist with evaluating NCES’ existing technology-related efforts and provide recommendations on priorities for future NCES data collection, reporting, and dissemination. Feedback from the panel will assist us in our efforts to provide greater focus on the relationship between technology and K–12 students’ educational experiences and outcomes. We plan to share insights from the expert panel meeting in an upcoming blog post.

By Halima Adenegan, NCES, and Emily Martin, Hager Sharp

The expanding use of technology affects the lives of students both inside and outside the classroom. While exposure to learning technology inside schools and classrooms is important, access can also differ once those students are in their homes. It’s important for educators to be aware of the potential barriers to technology and internet access that students may face. A recent report from NCES, Student Access to Digital Learning Resources Outside the Classroom, highlighted some differences in home internet access for students.

The percentage of 5- to 17-year-old students with either no internet access or only dial-up access differed by students’ race/ethnicity.

Access also differed geographically. Remote rural locales had the highest percentage of students with either no internet access or only dial up access at home. Within these remote rural areas, the percentage of students lacking access differed by students’ race/ethnicity. Forty-one percent of Black students and 26 percent of Hispanic students living in remote rural areas had either no internet access or only dial up access at home. This was higher than the percentage of White students (13 percent) and Asian students (11 percent) living in remote rural areas who had either no internet access or only dial up access at home.   

The percentage of students who had no access to the Internet or only dial-up access was higher for students living below the poverty threshold (26 percent) than for students living between 100 and 185 percent of the poverty threshold (15 percent) and at greater than 185 percent of the poverty threshold (4 percent).

In 2015, the two most common main reasons for children ages 3 to 18 to not have home internet access were that it was too expensive or that the family did not believe they needed it/ were not interested in having it (38 percent each). Other main reasons for not having home internet access included that the home lacked a computer or a computer adequate for internet use (8 percent), internet service was not available in the area (5 percent), the Internet could be used somewhere else (3 percent), and privacy and security concerns (i.e., online privacy and cybersecurity and personal safety concerns) (2 percent). 

Browse the full report for more data on additional topics relating to differences in access to technology and the internet.

By Lauren Musu