|Title:||Challenge Feedback Tool|
|Principal Investigator:||Cleghorn, Marisa||Awardee:||Future Engineers, LLC|
|Program:||Small Business Innovation Research [Program Details]|
|Award Period:||2 years (05/15/2022 – 05/14/2024)||Award Amount:||$1,000,000|
|Type:||Phase II Development||Award Number:||91990022C0046|
Company Website: https://www.futureengineers.org
Video Demonstration of the Phase I Prototype: https://www.youtube.com/watch?v=BofmVIK7oXQ
Purpose: This project team will develop a product to improve the feedback that adult judges provide to students who have entered online educational challenges. Online challenges provide students the opportunity to present proposals in different areas of education, including STEM. Despite the potential of challenges as an educational experience, the large volume of submissions often prevents judges from providing formative feedback to students to help improve a concept and deepen learning.
Project Activities: In the Phase I project in 2021, the team developed a new prototype portal for adult judges to provide formative feedback and to safely engage with students who submitted entries in response to challenges. The prototype also included a machine learning algorithm to identify themes in student entries and provided judges with suggestions on what feedback to provide. At the end of Phase I, researchers completed a pilot study with 11 educators, 291 students in Grades 3 to 8, and 218 judges who participated in student challenges. Researchers found that judges successfully provided formative feedback to students and used the machine learning-generated insights to help identify themes, and that students reported being able to integrate the feedback as they refined the submissions.
In Phase II, the team will fully develop the portal, including optimizing the machine learning engine to organize student submissions into themes and to flag any inappropriate student entries, finalize the user-experience and user-interface design, and add time-saving tools. The team will conduct iterative refinements and include feedback from students, judges, and educators at major production milestones until the product is fully functional. After development is complete, researchers will conduct a pilot study with 44 middle school classrooms (15 per treatment condition) with approximately 25 students per classroom (for a total of 1100 students) to test the feasibility, usability, fidelity of implementation, and the promise of the new portal to support outcomes. All classes will participate in a student challenge, half will receive feedback from judges, and half will not. Researchers will examine changes in students' perceptions of their ability to be successful with challenges using items from the Competency Beliefs in STEM measure, and a researcher developed measure for engagement in learning.
Product: Through prior IES-funded SBIR projects in 2016 and 2017, the developer created an online platform, Future Engineers, to administer educational challenges for students in or outside of K-12 classrooms. This project will develop a new portal for adult judges to evaluate and provide written feedback to students in response to their submissions and an interface for students to safely engage with mentors to facilitate an iterative process to refine and improve the submission. Additional features will include a machine learning engine to generate automatic feedback based on text analysis and natural language processing. The website will require educator or parent approval for student involvement, and will generate additional teacher-assigned class awards, superlatives, or badges that get displayed in the gallery to provide more acknowledgment.
Related IES Projects: Challenge Feedback Tool for Mentor and Students (91990021C00150); Development of an Online, Multi-Challenge Platform for K–12 Students (EDIES17C0033); Development of an Online, Multi-Challenge Platform that can Host Multiple K–12 Engineering Design Challenge Competitions Simultaneously (EDIES16C0002)