Inside IES Research

Notes from NCER & NCSER

See How IES is Supporting Technology-Delivered Assessments

For decades, student assessments have looked the same: multiple-choice or short-answer questions administered with pencil and paper, with all students receiving a common group of questions. Today, innovations in assessment design, greater understanding in the learning sciences, and new technology have all contributed to the way that assessments are administered and taken, and how the resulting information is shared with teachers, students, and families.

Since 2002, the Institute of Education Sciences (IES) has made more than 100 awards for the development of new assessments that are driven and delivered through technology.  The awards were made to a mix of academic researchers, entrepreneurial firms, and larger education research organizations. All of the projects included a rigorous research and development process, with studies to validate that assessments are measuring what is intended and pilots to test the promise of the technologies for improving student learning outcomes.

To highlight some of the technology-delivered assessments, IES has created YouTube Playlists that feature 57 videos in seven areas:

The assessments highlighted are delivered via mobile apps or through web-based computers and administered for different purposes. Some are diagnostic assessments used to screen students at the start of a new unit or year to identify areas where students struggle or areas to target with intervention. Many serve as performance assessments to determine how well students analyze information and draw conclusions when engaging in complex scenario-based activities. Others are summative assessments used to measure student performance during and at the end of the school year. Many also include a formative assessment component that adjusts based on the level of performance, and are designed to provide feedback and cues to students to inform the learning process.

A good number of the assessments are administered as simulations, games, scenarios, and puzzles, allowing for complex challenges where students can demonstrate mastery of knowledge and skills. Several enable new opportunities for assessment through the application of technological advances, such as natural language processing and machine-learning, read-aloud stories, fast-paced tasks that require students to respond, and speech recognition programs. Many save classroom time because the assessments are self-administered, and teachers benefit from the automatic grading as students go.  Most of the assessments also provide teachers information to guide practice through data dashboards or generated reports. Several of the assessments are already being used in school around the country.

Below are highlights from each of the playlists. It is important to note that none of assessments highlighted are wide enough in scope or configured to measure the full depth and breadth of State learning standards. Therefore, they are not sufficient to replace statewide summative assessments used for accountability and reporting purposes.  Collectively, however, the examples highlight the promise of technology-delivered assessments to improve and expand on existing approaches for measuring student learning and social and emotional skills, and for informing teacher instruction.  

Mathematics and Science

ASSISTments is web-based mathematics platform that assesses and then provides immediate feedback to students in grades 3-12, and generates teacher reports use to inform instruction. 

SimScientists is a simulation platform that formatively and summatively assesses science inquiry skills and knowledge aligned to middle school Next Generation Science Standards. 

Reading and Writing

RAPID is an adaptive literacy diagnostic and summative assessment system for students in Kindergarten through grade 12. 

Revision Assistant provides automated sentence-level in-line feedback to students during writing tasks aligned to Common Core State Standards. 

Social and Emotional Development

VESIP is a web-based simulated environment that measures the ability of students in grade 3-7 to interpret social cues which research demonstrates are needed to resolve conflicts. 

Early Learning

The School Readiness Curriculum Based Measurement System provides universal screening, benchmarking, and progress monitoring in language, literacy, mathematics, and science, for students in Pre-K and Kindergarten students. 

English Learning

ONPAR assesses the science and mathematics content knowledge and skills of English- and Spanish- speaking students using hyperlinks and animations to make questions accessible to all students. 

Tools for Teacher Practice

CLASS 5.0 automatically analyzes classroom discourse (student and teacher talking during class) and provides reliable profiles to guide and optimize how teachers lead instruction. 

Students With Disabilities or At Risk for Disabilities

NumberShire is a game-based mathematics intervention for students with, or at risk for, disabilities in Kindergarten through Grade 2. The game embeds instructional supports such as providing explicit, systematic, and frequent instruction, goal setting, and allowing students to work at their own pace. 

AnimalWatchVi Suite is an iPad app covering pre-algebra mathematics for middle and high school students with visual impairments. The app includes accommodation tools such as problem narration, audio hints, braille, and tactile graphics to provide accessible assessment. 

Written by Edward Metz, ED/IES SBIR program manager and IES Education Technology topic program officer.

Awards to Accelerate Research in Education Technology

The Institute of Education Sciences (IES), in partnership with the Small Business Administration (SBA), has made awards to two U.S.-based accelerators – organizations that provide support to technology developers and researchers in the development and launch of new innovations. They are among 68 awards made under SBA’s 2016 Growth Accelerator Fund Competition

The $50,000, one-year awards are designed to support accelerators in building the capacity of education technology small businesses to conduct research in the development and evaluation of new innovative products.

The Role of Accelerators in Education

Accelerators fill an important role in the fast-growing, education technology ecosystem by offering a “one-stop shop” for small businesses looking to advance their research and development, and commercialization processes. For example, accelerators assist with idea formation and market analysis, software development, licensing and copyright planning, networking, manufacturing, raising investment funds, and getting the products tested by schools.

However, not many education accelerators provide assistance to developers in conducting rigorous and relevant education research. Because start-ups typically do not employ or partner with education researchers, many new technologies are often not iteratively developed and refined based on feedback from students and teachers or evaluated for promise or efficacy in improving education outcomes. The lack of research-based information often leaves school practitioners without the information they need to guide decision making on whether to adopt a new intervention.

To help strengthen these areas, the U.S. Department of Education/IES partnered with SBA to create a new topic in this year’s Growth Accelerator Fund Competition—Education Technology Research. This topic called for applications from accelerators with plans to support developers in conducting education research, including research to inform concept idea development, research to test prototypes and inform refinements, and pilot studies to evaluate the promise and efficacy of fully developed technologies. Applications were reviewed by a panel of expert judges. The two awards were made to the following accelerators:

  • Virginia-based Jefferson Education, which will build a foundation for a national education researcher database to connect entrepreneurial developers to a qualified education research partners. The database will facilitate the creation of partnerships to carry out a wide range of research studies, from case studies to inform the development of a new intervention to experimentally designed studies of the education outcomes of fully developed technology interventions.
  • California-based New Schools Ignite/WestED Research Partnership, which will create a website with free resources and information on the role of different forms of research across the lifespan of a technology intervention, training and assistance in research methods, and with opportunities to conduct research in education settings. The funding will also be used develop data collection and analysis systems to help entrepreneurs understand the impact and effectiveness of the technologies housed within the accelerator.

Along with building the research capacity of developers, the awardees will disseminate information to developers about IES programs that fund research, development and evaluation, including the ED/IES Small Business Innovation Research program and the IES Research Grants Programs in Education Technology for Education and Special Education.

For more information about how IES is supporting the development of education technology, visit the IES website or follow us on Facebook and Twitter.

Developer of IES-Funded App Named an “Innovator to Watch”

By Dana Tofig, Communications Director, IES

Congratulations to one of our IES Small Business Innovation Research (SBIR) grantees, who has been named one of eight innovators to watch by Smithsonian Magazine.

Grace Wardhana, of Kiko Labs, developed the idea for Kiko’s Thinking Time, while watching her child, who she says was “obsessed with the iPad.” She wanted to turn that tablet time into learning time. With experience at Microsoft and McKinsey, a Master’s degree in science and engineering from Stanford, and a MBA from Harvard Business School, Grace got to work developing the app.

In 2014, she applied for and received a Phase I SBIR grant to develop a prototype of Thinking Time and in 2015, was awarded Phase II funding.  Smithsonian Magazine included Wardhana as one of eight “scientists, musicians, artists, and educators” to “expect great things from” in 2016. About Thinking Time, author Emily Matchar wrote that it “was developed in collaboration with Harvard and Berkeley neuroscientists and is aimed at helping children develop executive function—the skills of memory, focus and self-regulation necessary for success in school, work and beyond.”

You can watch a YouTube video to learn more about Thinking Time and also get more information on the IES website

PHOTOS
- Top of page--Grace Wardhana, of Kiko Labs. Source: https://www.kikolabs.com/about-us 
- Screenshot of Kiko's Thinking Time app

A Night to Play and Learn

By Dana Tofig, Communications Director, IES

It was an event that had the feel of an arcade, but the heart of a schoolhouse.

The Ed Games Expo on Wednesday, December 9, 2015, hosted 45 developers who are building games for learning that are designed to engage students across a variety of topics and subject matters. One-third of the games on display at the Expo are supported by the Institute of Education Sciences Small Business Innovation Research (ED/IES SBIR) program, which provides awards to companies to improve the use of technology in education.

Photo by Lauren Kleissas

(Photo by Lauren Kleissas)

"Games are inherently engaging," said Brooke Morrill, an educational researcher at Schell Games. "A student may or may not be interested in a topic matter, but it doesn't matter. They are engaged in the game."

Schell Games was demonstrating a prototype of "Happy Atoms," a game that combines hand-on resources with technology to create an interactive learning experience. Users can use create atom models with balls and sticks; similar to the way it’s been done in Chemistry classes for generations. But using an app equipped with vision recognition software, users can scan the model they've built to see what they've created or if they've made any mistakes. The app then connects to curriculum-aligned content about the molecule and how it is used in the real world.

Happy Atoms (pictured below) was a long-time pet project of the company's CEO, Jesse Schell, who is a vanguard in the educational gaming industry. However, the company didn't have funding to put a full-time team on the game's development. In 2014, Schell received a Phase I funding from ED/IES SBIR to develop a prototype and, earlier this year, received a Phase II award to further develop and evaluate Happy Atoms.

"We wouldn't be where we are without our IES funding," Morrill said.

A few tables away, the Attainment Company was demonstrating ED/IES SBIR-supported technology that is designed to build the reading, comprehension, and writing skills of special education students. For instance, Access: Language Arts is an app and software designed to allow middle school students with intellectual disabilities to read adapted versions of the books their peers are reading (like The Outsiders and the Diary of Anne Frank) while building their writing skills and even engaging in research.

 

"We know many students, especially those with autism, are motivated by technology because of the consistency it provides," said Pamela J. Mims, an assistant professor of special education at East Tennessee State University, who is working with Attainment on Access: Language Arts.  "We collected a lot of data on this and we see a lot of engagement."

Engagement is a big part of what drove John Krajewski, of Strange Loop Games, to develop ECO, another game that has received Phase I and II funding through ED/IES SBIR. Krajewski calls ECO a “global survival game,” in which students work together to build and maintain a virtual world for 30 days.

“You are trying to build up enough technology for your society, but in the process you can pollute and damage this world to the point it could die on its own,” Krajewski said. “You have to make decisions as a group about what has to be done in this world.”

In order to preserve the world, players have to pass laws that will protect resources while allowing society to continue to develop. For example, students might decide to limit the number of trees that can be cut down each day. Then, they can use graphs and data to see the impact of their laws and the health of their world. ECO not only builds an understanding about ecology and environmental science, but it builds real-world skills, like collaboration, communication, and scientific conversation and debate.

Krajewski said the funding from ED/IES SBIR not only allowed the project to be developed, but was a vote of confidence that allows ECO to be accepted in the field.

“IES has given us total runway to make this thing happen, which is awesome,” he said.

ED/IES SBIR is now soliciting Phase I proposals from firms and their partners for the research, development, and evaluation of commercially viable education technology products. You can learn more on the IES website

 

ED Games Expo 2015: Showcasing ED/IES SBIR-Supported Games for Learning

Game-based learning is gaining popularity as more and more young people and adults play and learn from games in and out of the classroom. Well-designed games can motivate learners to actively engage in challenging tasks, master content, and sharpen critical thinking and problem solving skills. The meteoric rise and popularity of mobile handheld and tablet devices has enabled game-playing anywhere and at any time, providing expanded opportunities for game developers.

In recent years, the Small Business Innovation Research program at the Institute of Education Sciences has supported the R&D and evaluation of many games for learning.  Other SBIR programs across the Federal government are also investing in games for learning across education, health, and the military.

The ED Games Expo is an annual event. This year the Expo is showcasing 45 developers (including 15 supported by ED/IES SBIR) who created games for learning. At the Expo, attendees will meet the developers while playing games that cover a range of topics – including earth science, ecology, chemistry, math, early learning, social and emotional learning, smoking cessation, mental health, stress resilience, or cultural awareness.

Event Details

  • Date: Wednesday, December 9, 2015, from 6PM to 8PM 
  • Location: In Washington DC, at 1776, 1133 15th St. 12th floor
  • RVSP: rsvp@theESA.com
  • Note: The Expo is sponsored by 1776 and the Entertainment Software Association and is free to attend. Please RVSP to ensure entry to the event, as space is limited. 

 

 

Questions? Comments? Please email us at IESResearch@ed.gov.