Project Activities
The researchers will test the impact of UDSN in students across three states using a randomized controlled trial. They will also test for moderation of student characteristics with subgroup analyses and test for the mediation of mechanisms by which UDSN supports students in improving their science knowledge.
Structured Abstract
Setting
The study will be conducted in elementary schools in Texas and Indiana. A school in Massachusetts will test quality assurance (to ensure the technology is working properly) prior to the efficacy study.
Sample
There will be 36 fourth-grade teachers and their students participating in this efficacy study. Each class is expected to include approximately 20 students, including 3 students with learning disabilities. Approximately 33 teachers and their students will participate in the initial quality assurance testing.
Intervention
UDSN is as an intervention that provides students with the space to collect, organize, and display their scientific observations and data they collect, as well as reflections and thoughts about their inquiry experiences. Teachers use the information as ongoing formative assessment and provide feedback to the students. The intervention also includes accessibility features such as text-to-speech technology, real time highlighting, keyboard accessible actions, multimedia glossary, and word-by-word English-to-Spanish translation. In addition, the design incorporates pedagogy to support active science learning through conceptual anchors (words and pictures) and navigation structures (plan, get data, explain).
Research design and methods
Before the start of the efficacy trial, initial quality assurance testing will determine if any technology bug fixes are needed or if infrastructure support is needed to scale up of the implementation of UDSN. The researchers will then use a randomized controlled trial to compare the outcomes of students in classes using the UDSN versus the traditional paper-based science notebooks. Teachers will be randomly assigned to either the UDSN or the control condition in Year 1, and in Year 2 their classes will be assigned the opposite condition. Only students in classes assigned to the UDSN condition will have logins to use the program. All teachers will be trained in both UDSN and the traditional paper notebook methods. Data will be collected to monitor contamination (use of intervention-related approaches with traditional paper notebook classes) and intervene as necessary. The researchers have included a moderation and subgroup analysis in their design to determine if effect sizes are influenced by UDSN differentially for students with learning disabilities or other subgroups (e.g., gender, ethnicity, free and reduced lunch status).
Control condition
Students in the control condition will use traditional paper-based science notebooks.
Key measures
Student outcome measures will include two measures of student characteristics—student record abstract (e.g., student aptitude and reading scores used for special education eligibility and demographics) and the Learning Disability Diagnostic Inventory (teacher rating scale for listening, speaking, reading, writing, mathematics and reasoning skills of students). Four measures of science content learning will be used: Pearson Interactive Science and STEMscopes Unit Tests (aligned to curriculum and completed at the end of each unit), ASK Science Content Tests (measure of knowledge of science content), Measures of Academic Progress (broader measure of science knowledge), and an assessment of student notebooks using the Assessment of Student Notebook Work rubric. Fidelity will be measured through a researcher-developed assessment, as well as data on dosage, adherence and quality data gathered through the UDSN usage log information and a log that teachers complete. Three measures of social validity will be used to understand the general support, perceived beneficial effects, and therapeutic alliance—classroom survey of teachers, teacher log of the use of class time, and student surveys.
Data analytic strategy
Hierarchical linear model regression analyses will be used to understand the impact of the intervention on student outcomes, the presence of any subgroup moderating variables, and any mediation effects (e.g., notebook usage and science engagement). In addition, a treatment on the treated analysis will be conducted to link fidelity of implementation with student outcomes using an instrumental variable approach (two-stage least-square model).
People and institutions involved
IES program contact(s)
Products and publications
Products: The products of this project include evidence of the efficacy of UDSN for improving student science knowledge across a broad range of science content, peer-reviewed publications and presentations.
Related projects
Supplemental information
Co-Principal Investigator: Jennifer Yu (SRI International)
Questions about this project?
To answer additional questions about this project or provide feedback, please contact the program officer.
