Project Activities
Researchers will develop, revise, and validate the next generation of ACES following three major phases: item development, field-testing, and standardization and analysis of psychometric properties. They will develop a secure online administration and scoring platform to facilitate use and dissemination of ACES-2. In addition, they will complete a cost analysis of the use of the assessment system.
Structured Abstract
Setting
The study will recruit participants from a minimum of 25 sites across the United States.
Sample
Researchers will recruit student and teacher participants to ensure that both the field-test sample (N = 3,250) and standardization sample (N = 2,600) are sufficiently large and representative of the national K-12 student population.
Assessment
The original ACES include a teacher version for use with students in K–12 and a student self-report version for use with students in grades 6 through 12. Each of these versions assesses academic skills (reading/language, mathematics, critical thinking) and academic enablers (motivation, interpersonal skills, engagement, study skills). Although scores from the ACES have strong psychometric evidence to support their use, there are several areas (e.g., construct coverage, developmental sensitivity, diversity of standardization sample) where revision will further enhance the utility and validity of the ACES.
Research design and methods
Consistent with the goal of producing user-ready versions of the ACES-2 by the end of the funding period, the project design reflects the three major phases of the scale development process (item development, field-testing, and standardization and analysis of psychometric properties). The project team will complete one phase of the development process annually during each of the first 3 years of the funding period, while simultaneously creating a secure online platform to facilitate dissemination and use of the ACES-2. In the final year, the researchers will disseminate the findings and conduct a cost analysis of the use of the ACES-2 in practice.
Control condition
There is no control condition for this study.
Key measures
The project's measurement battery includes the field-test and final forms of the ACES-2 Teacher and Student measures as well as several additional validity measures (e.g., SSIS Rating Scales; Motivated Strategies for Learning Questionnaire, STAR).
Data analytic strategy
Item response theory will be used to analyze item properties, inform construction of final forms, and vertically scale ACES-2 scores to maximize their developmental sensitivity and utility for assessing change over time. Researchers will carry out additional analyses to answer questions focused on reliability (e.g., internal consistency, generalizability), validity (e.g., factor analysis, correlation, multilevel regression), and fairness (e.g., differential item functioning analyses) of ACES-2 scores.
Cost analysis strategy
They will conduct a cost analysis of use of the ACES-2 in practice in the final year of the project.
People and institutions involved
IES program contact(s)
Products and publications
Products: The researchers will produce a reliable and valid second generation of ACES, the ACES-2 that will be ready for use with teachers and students from kindergarten through 12th grade (K-12). They will report findings from their research activities via conference proceedings and peer-reviewed publications.
Supplemental information
Co-Principal Investigators: Lei, Pui-Wa; Anthony, Christopher
Questions about this project?
To answer additional questions about this project or provide feedback, please contact the program officer.
