Project Activities
The initial 3-year phase of the project will adopt a design-based research methodology. In phase 1, researcher-instructor led development teams will build SBA exemplars (not using the authoring tool) to support development of authoring and support system modules, while a first version of the framework is drafted. Phase 2 will include iterative refinements to the framework, SBAs, and authoring system design; a prototype of the instructor user interface will be developed; new instructors will be recruited to develop SBAs using them; and design-based research training workshops will be conducted to refine the system. In phase 3, the project team will build functionalities to enhance the user SBA authoring experience of novice instructors; integrate SBA exemplars as design templates; integrate secondary functionalities for expert users; and refine professional support systems.
Structured Abstract
Setting
The research activities will take place in universities in Tennessee and Georgia.
Sample
The sample will be instructors and students involved in the courses for which the SBAs will be developed.
The project team will develop an SBA authoring and professional support system to enable college instructors to create their own gen-AI enabled SBAs. Other deliverables include (a) multimodal literacy framework that defines the constructs that SBAs measure, to supports task design through an evidence-centered design process, and to increase the transparency of the work for all stakeholders; (b) a minimum of 12 fully operable SBAs for instructor and student use; and (c) a professional support system that demonstrates how to design, tailor, and deliver SBAs in college course.
Research design and methods
A design-based research methods paradigm will support the iterative development and refinement of the products and associated support services to users. Evaluation of SBA tasks will be guided by evidence-centered design principles and psychometric methods to assure that items and tasks meet appropriate standards for reliability, validity, and. The project team will also conduct analyses of potential market fit, research process integrity, and cost analyses.
Key measures
Data analytic strategy
The project team will take a mixed methodology approach, with most design-based research studies of usability, feasibility, and appeal collecting and analyzing qualitative data sources. They will analyze the student SBA task data using quantitative, psychometric/statistical models.
Cost analysis strategy
The cost analysis will follow SEER standards and be guided by the IES Cost Analysis Starter Kit (IES, 2020).
People and institutions involved
IES program contact(s)
Project contributors
Partner institutions
Georgia State University
Educational Testing Service (ETS)
Middle Tennessee State University
MindTrust
Products and publications
Products: The project team will contribute to publications and professional conferences as data and evidence accrue from SBA studies, instructor experiences, and implementations. Starting in phase 2, the team will address higher-education users and clients by conducting training workshops at conferences and meetings or through synchronously online settings where they will (a) discuss and share the multimodal literacy framework, (b) demo the SBA prototype, (c) conduct hands on tutorials using the gen-AI SBA authoring system, and (d) provide access to the other professional support system resources.
Additional project information
Partner Institutions: Georgia State University; Educational Testing Service; Middle Tennessee State University; Capti; Mindtrust; Caimber; Workbay
Related projects
Questions about this project?
To answer additional questions about this project or provide feedback, please contact the program officer.
