|Assessing Data Modeling and Statistical Reasoning
|Science, Technology, Engineering, and Mathematics (STEM) Education [Program Details]
Purpose: In this study, the researchers proposed to develop an assessment system to evaluate elementary and middle school students' skills and understandings related to data modeling and statistical reasoning. Data modeling includes deciding which aspects of the world are relevant to a conceptual model, how best to measure them, how to structure and represent the resulting measures, and how to make relevant inferences. This area of instruction was beginning to grow in importance in the early 2000s, giving rise to the need for new assessments.
Project Activities: In this project, the researchers planned to develop an assessment system aimed at grades 5 through 8 that featured both formative and summative assessment of data modeling skills and practices. The formative components of the system was to focus on supporting instruction with items designed for teachers to use as instructional tools and diagnostic aides. The summative components of the system were to profile students' skills and knowledge as it developed over the targeted grades. The researchers aimed to design a system that would evaluate growth over time based on research that showed how students' knowledge and proficiency with these skills characteristically develops.
THE FOLLOWING CONTENT DESCRIBES THE PROJECT AT THE TIME OF FUNDING
Setting: The schools will be located in three urban sites in Tennessee, Massachusetts, and California.
Sample: All of the schools include diverse and underserved fifth to eighth grade student populations.
Intervention: The researchers will work with teachers to design formative and summative assessments that diagnose students' skills and knowledge in data modeling. The formative assessments feature contexts for instruction as well as observation. Each assessment includes teacher notes that suggest ways to leverage the assessment as an opportunity for instruction. Teacher notes are based on current research-based understandings of student reasoning and on new research that the researchers will conduct. The researchers will employ the Berkeley Evaluation and Assessment Research model to develop construct maps (progress variables) for each of four strands of data modeling: (a) measurement, (b) representation, (c) data structures, and (d) statistical inference. Progress variables are hypothetical developmental trajectories of learning that reflect an emerging research base about how students in this age band typically reason about these concepts. Construct maps guide the development of formative and summative items, which will be tested in fifth through eigth grade classrooms.
Research Design and Methods: The initial phase of the work will focus on developing progress variables indicating four components of data modeling: measurement, data structures, data display, and statistical inference. The second phase will encompass larger samples of students and teachers to obtain evidence for reliability and validity and to test the viability of the assessment system at a greater scale. Specifically, in the first phase the researchers will conduct small-scale work with teachers to iteratively refine items based on student responses (written, interviews, and think-alouds) and teacher responses to the accompanying scoring guides and teacher notes. During the phase 2, the researchers will transition toward larger sample sizes (500 students at each grade) appropriate to determining psychometric characteristics of item functioning. Finally, a second large-scale data collection will be conducted with the revised items during the following year.
Key Measures Analytic Strategy: Outcome variables will be scaled with a multidimensional item response model.
Related IES Projects: Data Modeling Supports the Development of Statistical Reasoning (R305A110685), Innovative Computer-Based Formative Assessment via a Development, Delivery, Scoring, and Report-Generative System (R305A120217)
ERIC Citations: Find available citations in ERIC for this award here.
Lehrer, R., Kim, M.J., Ayers, E., and Wilson, M. (2013). Toward Establishing a Learning Progression to Support the Development of Statistical Reasoning. In J. Confrey, and A. Maloney (Eds.), Learning Over Time: Learning Trajectories in Mathematics Education. Charlotte, NC: Information Age Publishers.
Lehrer, R., Kim, M.J., and Jones, S. (2011). Developing Conceptions of Statistics by Designing Measures of Distribution. ZDM-The International Journal on Mathematics Education, 43(5): 723–736.