Skip Navigation

A study of reliability and consequential validity of a mathematics diagnostic assessment system in Georgia

Region:
Southeast
Abstract:

Description: Several school districts in Georgia are currently using the Individual Knowledge Assessment of Number (IKAN; Ministry of Education, 2011) and the Global Strategy Stage (GloSS; Ministry of Education, 2012) for which there is very limited psychometric data. IKAN provides information on students' number knowledge, that is, magnitude comparisons, knowledge of base 10 system and meaning of decimals and fractions. The GloSS provides information on the strategies the students used when solving mathematical problems. When used together, the IKAN and GloSS assessments furnish teachers with information on how students solve mathematics problems and students' understanding of whole and rational number concepts. Teachers can then use the information from the assessments to tailor their instruction to students' levels of understanding and address problems that underlie lack of success with grade-level curriculum.

This study is to determine the inter assessor reliability and consequential validity of the IKAN/GloSS diagnostic assessments. Findings will be of interest to Georgia's mathematics and instructional leadership team from Georgia Department of Education (GaDOE) so that they can make a more informed decision about the use of these assessments across the state.

Research Questions:

  1. What is the inter-assessor reliability of the Global Strategy Stage (GloSS) assessment when administered by two assessors within a one-week period?
  2. What is the inter-assessor reliability of the Individual Knowledge Assessment of Number (IKAN) assessment when administered by two assessors within a one-week period?
  3. What is the consequential validity of using IKAN and GloSS diagnostic assessments (that is, teachers' perception of their instructional utility for providing intervention)?

Study Design: Results from a sample of 60 students, 30 each in grades 1 and 3 from two participating districts in Georgia will be used to answer research questions about inter-assessor reliability (questions 1 and 2 above). Two separate assessors will administer the IKAN and GLoSS assessments in the sequence intended by the developers on two occasions within a one-week period.

The target sample for the consequential validity (question 3) will be 40 teachers of students in Grades 1 and 3 from the two participating districts who are using the IKAN/GLoSS instruments. Surveys will be completed and twenty-four to forty teachers (in groups of six to 10 members) will participate in focus groups.

Projected Release Date: Spring 2020

Partnership or Research Alliance: Improving Mathematics Instruction Research Alliance

Related Products: A technical report for researchers and a summary for educators will be available when the study is complete.

Principal Investigators & Affiliation:
Dr. Russell Gersten, Researcher for Mathematics Alliance, Instructional Research Group partner with REL Southeast
Madhavi Jayanthi, Researcher for Mathematics Alliance, Instructional Research Group partner with REL Southeast
Dr. Daniel Anderson, Research Associate, University of Oregon