|Title:||ACES-2: Development and Validation of the Revised Academic Competence Evaluation Scales|
|Principal Investigator:||DiPerna, James||Awardee:||Pennsylvania State University|
|Program:||Social and Behavioral Context for Academic Learning [Program Details]|
|Award Period:||4 years (07/01/2020 - 06/30/2024)||Award Amount:||$1,399,236|
Co-Principal Investigators: Lei, Pui-Wa; Anthony, Christopher
Purpose: The Academic Competence Evaluation Scales (ACES) are multi-informant rating scales designed to efficiently assess social, behavioral, and academic skills associated with learning in schools. Published 20 years ago, the ACES are even more relevant today given increased recognition of the importance of such social and behavioral variables by parents, teachers, and policy makers. As such, the purpose of this project is to refine and develop the next generation of student and teacher versions of the ACES.
Project Activities: Researchers will develop, revise, and validate the next generation of ACES following three major phases: item development, field-testing, and standardization and analysis of psychometric properties. They will develop a secure online administration and scoring platform to facilitate use and dissemination of ACES-2. In addition, they will complete a cost analysis of the use of the assessment system.
Products: The researchers will produce a reliable and valid second generation of ACES, the ACES-2 that will be ready for use with teachers and students from kindergarten through 12th grade (K–12). They will report findings from their research activities via conference proceedings and peer-reviewed publications.
Setting: The study will recruit participants from a minimum of 25 sites across the United States.
Sample: Researchers will recruit student and teacher participants to ensure that both the field-test sample (N = 3,250) and standardization sample (N = 2,600) are sufficiently large and representative of the national K–12 student population.
Assessment: The original ACES include a teacher version for use with students in K–12 and a student self-report version for use with students in grades 6 through 12. Each of these versions assesses academic skills (reading/language, mathematics, critical thinking) and academic enablers (motivation, interpersonal skills, engagement, study skills). Although scores from the ACES have strong psychometric evidence to support their use, there are several areas (e.g., construct coverage, developmental sensitivity, diversity of standardization sample) where revision will further enhance the utility and validity of the ACES.
Research Design and Methods: Consistent with the goal of producing user-ready versions of the ACES-2 by the end of the funding period, the project design reflects the three major phases of the scale development process (item development, field-testing, and standardization and analysis of psychometric properties). The project team will complete one phase of the development process annually during each of the first 3 years of the funding period, while simultaneously creating a secure online platform to facilitate dissemination and use of the ACES-2. In the final year, the researchers will disseminate the findings and conduct a cost analysis of the use of the ACES-2 in practice.
Control Condition: There is no control condition for this study.
Key Measures: The project's measurement battery includes the field-test and final forms of the ACES-2 Teacher and Student measures as well as several additional validity measures (e.g., SSIS Rating Scales; Motivated Strategies for Learning Questionnaire, STAR).
Data Analytic Strategy: Item response theory will be used to analyze item properties, inform construction of final forms, and vertically scale ACES-2 scores to maximize their developmental sensitivity and utility for assessing change over time. Researchers will carry out additional analyses to answer questions focused on reliability (e.g., internal consistency, generalizability), validity (e.g., factor analysis, correlation, multilevel regression), and fairness (e.g., differential item functioning analyses) of ACES-2 scores.
Cost Analysis: They will conduct a cost analysis of use of the ACES-2 in practice in the final year of the project.