Project Activities
The research will be conducted in five phases. Phase 1 will involve item development. In Phase 2 the research team will use a systematic procedure to review and refine the items to ensure they are aligned with the assessment framework. In Phase 3 the researchers will field test the items and conduct analyses to select the final items for the measure. In Phase 4 they will conduct a large-sample administration of the measure to determine its psychometric properties. In the final phase they will develop a scoring procedure and conduct validation studies.
Structured Abstract
Setting
The research will take place in high schools in Arkansas, Connecticut, Delaware, Massachusetts and North Carolina.
Sample
Approximately 1,500 students in grades 9-12 (including 300 students with high-incidence disabilities) will participate in the field testing of the items. Approximately 5,000 students in grades 9-12 (including 1,500 youth with high-incidence disabilities) will participate in the large-sample administration of the measure.
The CCR4T will be a self-report measure of perceptions of CCR for high school youth with high-incidence disabilities and youth without disabilities. It will be designed to supplement extant school data in order to assess areas of need and inform decisions about CCR supports. The measure will include items that assess student perceptions of academic and nonacademic domains of CCR in the following six areas: (1) academic engagement, (2) mindsets, (3) learning processes, (4) critical thinking, (5) interpersonal engagement, and (6) transition competencies.
Research design and methods
The researchers will use the CCR4T Assessment Framework, based on six domains of CCR, to guide a rigorous, iterative instrument research and development plan to ensure the final measure is of high psychometric quality with strong validity evidence. This plan consists of five phases: (1) item development, based on previous empirical work, literature review, and focus group feedback; (2) item review and refinement using systematic procedures to ensure alignment of the items with the assessment framework; (3) field-testing of items and data analysis to investigate the dimensional structure and select items for the final measure; (4) large-sample administration and data analyses to determine an appropriate psychometric model and demonstrate measurement invariance across subgroups; (5) development of scoring procedures and validation studies to provide support for the intended uses of scores. Validity evidence based on test content is obtained in Phases 1 and 2. Validity evidence based on internal structure is obtained in Phases 3 and 4 through the model-building procedures and assessment of model fit. Convergent and discriminant validity evidence will be obtained by examining the pattern of correlations between CCR4T scores and proximal outcomes measured by academic and behavioral variables in extant school records in Phase 5. Predictive validity related to long-term post-school (distal) outcomes will be obtained using the field-test data. In addition, a series of analyses in Phase 5 will address issues around self-report bias, where school academic and behavioral data will be used to examine relationships between student responses and academic and behavior variables that represent proximal and distal outcomes.
Control condition
Due to the nature of the research design, there is no control condition.
Key measures
In addition to the CCR4T, data will be collected from school records to assess academic indicators (such as grade point average and SAT or ACT scores) and behavioral indicators (such as number of absences, office discipline referrals, and in-school and out-of-school suspensions). Data will also be collected using a survey with items from the State of Connecticut exit survey that capture student demographics, participation in postsecondary education since leaving high school, and participation in post-school employment.
Data analytic strategy
The focus group data from Phase 1 will be analyzed using content analyses to sort excerpts into similar code categories and group into broader themes. This analysis will inform/confirm the initial operational definitions of the CCR domains, subdomains and associated potential items. Content validity ratio methodology will be used to determine the degree to which the item represents the intended domain based on a review by expert panelists. Field-test data will be analyzed using descriptive analyses to identify problematic items to inform preliminary modeling procedures. Confirmatory factor analyses will be used to determine a final measurement model. Tests of measurement invariance will be performed across subgroups of students and item response theory (IRT) will be used for final calibration and scoring of the items. Further tests of model fit and measurement invariance at the item-level (tests for differential item functioning) will be performed using IRT procedures.
People and institutions involved
IES program contact(s)
Project contributors
Products and publications
Products: The primary product of this project is a fully developed and validated assessment of CCR for students with disabilities, the CCR4T. The project will result in peer-reviewed publications and presentations as well as additional dissemination products that reach education stakeholders such as practitioners and policymakers.
ERIC Citations: Find available citations in ERIC for this award here.
Additional project information
This grant was awarded to another institution before being transferred to the current awardee.
Previous award details:
Questions about this project?
To answer additional questions about this project or provide feedback, please contact the program officer.
