Project Activities
The researchers will iteratively refine and test the CourseMIRROR application for use in postsecondary introductory physics and gateway math courses. They will integrate student and instructor feedback as they develop both the application and the supporting tools (e.g., dashboard) and materials (e.g., professional development materials). Once they have a fully functional version of CourseMIRROR, they will pilot it in classes at a large university and a community college.
Structured Abstract
Setting
The research will take place at universities in Indiana and Pennsylvania and in a community college in Indiana.
Sample
Approximately 120 students will participate during the development process and 600 undergraduate students will participate in the pilot studies. Approximately 14 university and community college instructors will also take part in the development process with additional instructors helping with the pilot study.
Intervention
CourseMIRROR is a mobile application for smartphones, tablets, or computers that prompts students to write about course lecture concepts at the end of each class. It uses a natural language processing algorithm to analyze students' input and generate summaries of their reflections. Students can use these reflections to identify lecture themes, identify their misunderstandings, and gauge their knowledge. Instructors can leverage the reflections to identify where the class may be struggling, thus allowing them to intervene more effectively.
Research design and methods
Building off an existing prototype of CourseMIRROR, the research team will further develop and test the mobile technologies, web tools, instructor interfaces, and natural language processing algorithms. The final product will also include a professional development module to help instructors implement CourseMIRROR. The design process includes multiple quasi-experimental studies to refine elements of the reflection process (e.g., improving the prompts and feedback students get while writing their reflections) as well as qualitative work such as classroom observations and interviews and focus groups with teachers and students. This iterative process will include gathering both student and instructor feedback, both quantitative and qualitative (e.g., observation data). The final pilot study will include sections of a university introductory physics course and a community college gateway math course. Instructors of these courses will have a treatment class, which uses CourseMIRROR; and a business-as-usual course, which does not. The researchers will collect student baseline data on their achievement, motivation, and learning strategies and pre- and post-test data.
Control condition
The comparison course in the pilot study will be a course taught by the same instructor as the treatment course, but the students in it will not use CourseMIRROR.
Key measures
Student outcome measures include course exam scores, standardized conceptual tests (e.g., the ACT Compass tests), questionnaires (e.g., the Motivated Strategies for Learning Questionnaire), and classroom observations and instructor and student interview data. The researchers will use student demographic data and administrative data (e.g., course-taking patterns) to determine whether impacts vary by student characteristics.
Data analytic strategy
The research team will analyze the student outcome data using analysis of variance models for each dependent measure and a multivariate analysis of variance model with multiple dependent measures.
People and institutions involved
IES program contact(s)
Products and publications
Products: Researchers will produce a fully developed CourseMIRROR application for postsecondary students and peer-reviewed publications.
Journal article, monograph, or newsletter
Menekse, M. (2019). The Reflection-Informed Learning and Instruction to Improve Students' Academic Success in Undergraduate Classroom. The Journal of Experimental Education, 1-17.
Proceedings
Anwar, S. (2019). Impact of Educational Technology-Based Learning Environment on Students' Achievement Goals, Motivational Constructs, and Engagement. In Proceedings of the 2019 ACM Conference on International Computing Education Research (pp. 321-322). ACM.
Supplemental information
Co-Principal Investigator: Samarapungavan, Ala; Litman, Diane
Questions about this project?
To answer additional questions about this project or provide feedback, please contact the program officer.
