Success Story out of the ED/IES SBIR Program: Inq-ITS by Apprendis

Product:
Inq-ITS (Inquiry Intelligent Tutoring System) virtual labs for science are designed for students in grades 4 through 10 to practice and learn the competencies scientists use. When engaging in the labs, students form questions, collect data, analyze data using mathematics, and communicate scientific findings in writing. Inq-ITS automatically assesses student competencies based on what they do and write within the labs, rather than using multiple-choice items. An artificial intelligence engine provides real-time feedback to students as they work on problems and provides teachers with reports that tracks student progress, thereby reducing grading time. Teachers also receive alerts via a dashboard with information on how to support struggling students. Inq-ITS runs within web browsers with no installation necessary and works across all computing platforms. Teacher resources are embedded to support the alignment of the labs to learning goals and standards. Inq-ITS generates downloadable reports, allows for assignment sharing, integrates with Google Classroom and its Gradebook, and has a co-teacher functionality for multiple teachers to use Inq-ITS and receive alerts for the same class.

Research and Development:
Inq-ITS and its teacher alerting platform, Inq-Blotter, have been developed over a 15-year period with grant funding from the Institute of Education Sciences, the National Science Foundation, and the ED/IES SBIR program. All components were developed iteratively, with feedback from educators and students at key milestone points, including—

• Algorithms that automatically track and score proficiencies at "doing" science and generate feedback based on work.
• A lab authoring framework for new labs across 150 topics in physical, life and earth science that align to the algorithms.
• An AI virtual coach that uses assessment data to deliver proactive, increasingly targeted, personalized help to students when they struggle.
• Teacher reports that track student growth in science competencies over time.
• Student reports that provide feedback on science competencies they have mastered and where they need to improve.
• Inq-Blotter Alerting Dashboard that alerts teachers when a student struggles, with information on how the student struggles, as well as conversation-starters for teachers to provide guidance.

Inq-ITS was validated and evaluated for learning outcomes using a combination of Evidence-Centered Design, iterative UX design, one-on-one structured interviews, classroom prototype testing, and randomized controlled experiments. Research articles from all studies are available here: https://www.inqits.com/research. A summary from this research includes:

• Usability and feasibility: Multiple studies in hundreds of classrooms and with thousands of students demonstrate that the system functions as intended for teachers and students. Inq-ITS can be used successfully in classroom, hybrid, or remote settings.
• Rigor of autoscoring algorithms: Several studies with 600 students in total showed that automated scoring was consistent and reliable across labs and diverse student populations, matching human coders with high precision (84 to 99 percent).
• Research on the AI virtual coach: Across multiple randomized controlled studies with approximately 400 students, Inq-ITS' AI Virtual Coach helped students improve at performing science practices across different science topics. In a separate longitudinal study with over 100 students, students' learning gains remained over 170 days.
• Research on the teacher alerting dashboard: In a randomized controlled study with 8 teachers and 280 students, teachers who used Inq-Blotter were able to help twice as many students than teachers who did not. The help teachers provided improved student performance on their next Inq-ITS lab, particularly for interpreting data and warranting claims competencies. In a pilot study with 4 teachers and 25 students, increased competency in students was predicted based on the feedback teachers provided, driven by information from the alerts. Research on the alerting dashboard also showed that help by teachers triggered by alerts leads to improvement in students' inquiry for the practice on which they received help.

Commercialization:
Inq-ITS was initially developed as an academic research program beginning in 2007 at Worcester Polytechnic Institute in Worcester, MA. Apprendis was founded in 2012 to develop a commercially viable education technology product that can be used at scale in education practice. Apprendis uses a direct sales model and has been purchased by schools and districts in New York, New Jersey, Kentucky, and California. As of January 2023, over 1.33 million labs have been completed by 166,000 students in almost 13,000 schools across the United States.

Awards / Recognition:

• 2020 – Listed by Tech & Learning Magazine as a top virtual lab software
• 2020 – Listed by Commonsense Education on a list of Top Apps, Games, and Websites for Scientific Inquiry
• 2020 – Listed by Commonsense Education as a Top 10 Best Science Tools for Middle School.
• 2018 – Winner of the MassVentures SBIR START Phase I
• 2017 – Commonsense Education "Top Pick" Best of Edtech
• 2017 – SIIA Innovation 2nd Place for most innovative educational technology
• 2017 – eSchoolNews App of the Week
• 2016 – Named a Top Product for Teacher Needs by Digital Promise
• 2016 – Winner of a Civic Innovation Scholarship from Microsoft

Patents:
US Patents 9230221, 9564057, 10186168

Related Government Projects that Supported the R&D and Evaluation:
University researchers at Worcester Polytechnic Institute and Rutgers Graduate School of Education created and evaluated the efficacy of Inq-ITS for middle school life, earth, and physical science on prior ED/IES (R305A090170, R305A120778) and National Science Foundation (DRL0733286, DGE0742503, DRL1008649, DRL1252477, DRL1643673) grants. Apprendis developed and tested the efficacy of Inq-Blotter (ED-IES-15C0018, ED-IES-16C0014), a dashboard to provide teachers real-time diagnostic alerts on how students struggle to conduct science inquiry practices. University researchers further examined Inq-Blotter's classroom impacts (NSF-IIS1629045). Apprendis then expanded Inq-ITS and Inq-Blotter to high school physical science by incorporating "model critiquing" and "using mathematics" science practices (ED-IES-91990018C0022, ED-IES-91990019C0037). Currently, further research is being conducted at Rutgers University to incorporate and test the efficacy of personalized feedback on how students explore, critique, and revise mathematical models of physical science phenomena (R305A210432).