Project Activities
The MASS project will adhere to systematic design principles to develop formative assessments to be used during and at the end of science curriculum units. The researchers will use systematic design principles to create a coherent, multilevel state science assessment system by aligning the designs of the embedded assessments with the designs of the benchmark assessments. Next, they will align the designs and items of the embedded and benchmark assessments with relevant specifications and items on the state science test. The researchers will study the effects of the formative assessments on middle school science student learning, validate the use of data from the embedded and benchmark assessments for interpreting student performance on the targeted science standards, and describe the components of the formative assessments and their implementation so that they can serve as scalable models. They will do this by adapting and integrating the powerful assessments and technical infrastructure developed in the National Science Foundation project Calipers: Using Simulations to Assess Complex Science Learning.
Structured Abstract
Setting
The setting for this study is the state of Washington.
Sample
Participants in this project will include approximately 400 eighth-grade students enrolled in science classes in 16 middle school classrooms. Sixteen eighth-grade science teachers will be identified to participate in the field test that will be conducted in Year 3.
Research design and methods
Embedded and benchmark formative assessments will be developed for two middle school science topics: Forces and Motion, and Ecosystems. After test design, prototype development, and feasibility testing, a set of validity studies will be conducted in a field test to document the technical quality and implementation of the assessments. The MASS assessment development process will proceed in two phases. Phase 1 will include the design and development of the MASS assessments, including feasibility testing with individual students and pilot testing in a few classrooms. In Phase 2, the researchers will conduct a field test in a larger number of classrooms in the state of Washington to gather evidence of the technical quality of the embedded and benchmark assessments, examine the relationship of the MASS assessments to the students' state science test performance, and document the features of the implementations of the MASS assessments.
Key measures
Key measures include students' state test scores in science, external expert interviews, and teacher questionnaires and interviews.
Data analytic strategy
Analyses will include basic psychometric information, multiple regressions relating teacher judgments with the assessment data, comparisons of state test data from teachers' classes in the prior year with test scores from the field test year, and correlational and cross-case study analyses of the implementation and effectiveness of the components of the formative assessments and activities.
People and institutions involved
IES program contact(s)
Products and publications
Products: The expected outcomes of this research include published reports on the development and impact of formative assessments on middle school science student learning.
Book chapter
Quellmalz, E.S., and Silberglitt, M.D. (2017). Simscientists: Affordances of Science Simulations for Formative and Summative Assessment. In H. Jiao and R.W. Lissitz (Eds.), Technology Enhanced Innovative Assessment: Development, Modeling, and Scoring From an Interdisciplinary Perspective (pp. 71-94). Information Age Publishing.
Supplemental information
Purpose: The purpose of the Multilevel Assessment of Science Standards (MASS) project is to create a new generation of technology-enhanced formative assessments that bring best formative assessment practices into classrooms, transforming what, how, when, and where science learning is assessed. They will do this by adapting and integrating the powerful assessments and technical infrastructure developed in the National Science Foundation project Calipers: Using Simulations to Assess Complex Science Learning.
Questions about this project?
To answer additional questions about this project or provide feedback, please contact the program officer.
