Skip Navigation

Science, Technology, Engineering, and Mathematics (STEM) Education

Grantees

- OR -

Investigator

- OR -

Goals

- OR -

FY Awards

- OR -

Multilevel Assessments of Science Standards (MASS)

Year: 2008
Name of Institution:
WestEd
Goal: Measurement
Principal Investigator:
Quellmalz, Edys
Award Amount: $1,599,998
Award Period: 4 years
Award Number: R305A080225

Description:

Purpose: Large-scale state-level tests and classroom assessments are often not well aligned, and often do not support good instructional decisions. In addition, the technical quality of classroom assessments developed by teachers or those that are embedded in published curriculum materials is often unknown or inadequate. The purpose of the Multilevel Assessment of Science Standards (MASS) project is to create a new generation of technology-enhanced formative assessments that bring best formative assessment practices into classrooms, transforming what, how, when, and where science learning is assessed. Simulation-based tasks with immediate, individualized feedback and a hint system will provide evidence that embedded and benchmark classroom assessments with technical quality can gather, document, and promote students’ learning of connected science knowledge and extended inquiry not measured by large-scale tests.

Project: The MASS project will adhere to systematic design principles to develop formative assessments to be used during and at the end of science curriculum units. The researchers will use systematic design principles to create a coherent, multilevel state science assessment system by aligning the designs of the embedded assessments with the designs of the benchmark assessments. Next, they will align the designs and items of the embedded and benchmark assessments with relevant specifications and items on the state science test. The researchers will study the effects of the formative assessments on middle school science student learning, validate the use of data from the embedded and benchmark assessments for interpreting student performance on the targeted science standards, and describe the components of the formative assessments and their implementation so that they can serve as scalable models. They will do this by adapting and integrating the powerful assessments and technical infrastructure developed in the National Science Foundation project Calipers: Using Simulations to Assess Complex Science Learning.

Products: The expected outcomes of this research include published reports on the development and impact of formative assessments on middle school science student learning.

Structured Abstract

Purpose: The purpose of the Multilevel Assessment of Science Standards (MASS) project is to create a new generation of technology-enhanced formative assessments that bring best formative assessment practices into classrooms, transforming what, how, when, and where science learning is assessed. They will do this by adapting and integrating the powerful assessments and technical infrastructure developed in the National Science Foundation project Calipers: Using Simulations to Assess Complex Science Learning.

Setting: The setting for this study is the state of Washington.

Population: Participants in this project will include approximately 400 eighth-grade students enrolled in science classes in 16 middle school classrooms. Sixteen eighth-grade science teachers will be identified to participate in the field test that will be conducted in Year 3.

Research Design and Methods: Embedded and benchmark formative assessments will be developed for two middle school science topics: Forces and Motion, and Ecosystems. After test design, prototype development, and feasibility testing, a set of validity studies will be conducted in a field test to document the technical quality and implementation of the assessments. The MASS assessment development process will proceed in two phases. Phase 1 will include the design and development of the MASS assessments, including feasibility testing with individual students and pilot testing in a few classrooms. In Phase 2, the researchers will conduct a field test in a larger number of classrooms in the state of Washington to gather evidence of the technical quality of the embedded and benchmark assessments, examine the relationship of the MASS assessments to the students’ state science test performance, and document the features of the implementations of the MASS assessments.

Key Measures: Key measures include students’ state test scores in science, external expert interviews, and teacher questionnaires and interviews.

Data Analytic Strategy: Analyses will include basic psychometric information, multiple regressions relating teacher judgments with the assessment data, comparisons of state test data from teachers’ classes in the prior year with test scores from the field test year, and correlational and cross-case study analyses of the implementation and effectiveness of the components of the formative assessments and activities.

Project Website: http://simscientists.org/

Publications

Book chapter

Quellmalz, E.S., and Silberglitt, M.D. (2017). Simscientists: Affordances of Science Simulations for Formative and Summative Assessment. In H. Jiao and R.W. Lissitz (Eds.), Technology Enhanced Innovative Assessment: Development, Modeling, and Scoring From an Interdisciplinary Perspective (pp. 71–94). Information Age Publishing.