|Title:||Assessing Data Modeling and Statistical Reasoning|
|Principal Investigator:||Lehrer, Richard||Awardee:||Vanderbilt University|
|Program:||Science, Technology, Engineering, and Mathematics (STEM) Education [Program Details]|
|Award Period:||4 years||Award Amount:||$1,599,946|
Purpose: The recent explosion of research on statistical reasoning reflects the widespread recognition of its importance for all students and future citizens. For example, the ability to reason soundly about economics, history, politics, and current events requires the ability to both produce and understand arguments based on data. Data modeling is, in fact, what professionals actually do when they reason statistically, and it is central to a wide variety of disciplines, including engineering, economics, medicine, and natural science. Data modeling includes deciding which aspects of the world are relevant to a conceptual model, how best to measure them, how to structure and represent the resulting measures, and how to make relevant inferences. The purpose of this project is to develop an assessment system to evaluate elementary and middle school students' skills and understandings related to data modeling and statistical reasoning.
Project Activities: In this project, the researchers will develop an assessment system aimed at grades 5-8 that features both formative and summative assessment of data modeling skills and practices. The formative components of the system will be focused on supporting instruction; items will be designed so that teachers can employ them as instructional tools and as diagnostic aides. The summative components of the system will profile students' skills and knowledge with an eye toward characterizing development over the targeted grades. Informed by recent research on how students' knowledge and proficiency with these skills characteristically develops, the system will be designed to evaluate growth over time.
Products: The products from this study include a system aimed at assessing data modeling skills and practices among students in late elementary and middle school, and published reports.
Setting: The schools will be located in three urban sites in Tennessee, Massachusetts, and California.
Population: All of the schools include diverse and underserved fifth to eighth grade student populations.
Intervention: The researchers will work with teachers to design formative and summative assessments that diagnose students' skills and knowledge in data modeling. The formative assessments feature contexts for instruction as well as observation: Each assessment includes Teacher Notes that suggest ways to leverage the assessment as an opportunity for instruction. Teacher Notes are based on current research-based understandings of student reasoning and on new research that the researchers will conduct. The researchers will employ the Berkeley Evaluation and Assessment Research model to develop construct maps (progress variables) for each of four strands of data modeling: (a) measurement, (b) representation, (c) data structures, and (d) statistical inference. Progress variables are hypothetical developmental trajectories of learning that reflect an emerging research base about how students in this age band typically reason about these concepts. Construct maps guide the development of formative and summative items, which will be tested in grades 5-8 classrooms.
Research Design and Methods: The initial phase of the work will focus on developing progress variables indicating four components of data modeling: measurement, data structures, data display, and statistical inference. The second phase will encompass larger samples of students and teachers to obtain evidence for reliability and validity and to test the viability of the assessment system at a greater scale. Specifically, in the first two years the researchers will conduct small-scale work with teachers to iteratively refine items based on student responses (written, interviews, think-alouds) to items, and teacher responses to the accompanying scoring guides and Teacher Notes. During the third year, the researchers will transition toward larger sample sizes (500 students at each grade) appropriate to determining psychometric characteristics of item functioning. During Year 4, a second large-scale data collection will be conducted with the revised items.
Key Measures Analytic Strategy: Outcome variables will be scaled with a multidimensional item response model.
Project Website: http://modelingdata.org/
Related IES Projects: Data Modeling Supports the Development of Statistical Reasoning (R305A110685) and Innovative Computer-Based Formative Assessment via a Development, Delivery, Scoring, and Report-Generative System (R305A120217)
Publications from this project:
Lehrer, R., Kim, M.J., and Jones, S. (2011). Developing Conceptions Of Statistics By Designing Measures Of Distribution. International Journal on Mathematics Education(ZDM), 43 (5): 723–736.
Lehrer, R., Kim, M.J., Ayers, E., and Wilson, M. (in press). Toward Establishing A Learning Progression To Support The Development Of Statistical Reasoning. In J.Confrey and A. Maloney (Eds.), Learning over Time: Learning Trajectories in Mathematics Education. Charlotte, NC: Information Age Publishers.