Description:

Co-Principal Investigator: Kandace Fleming, University of Kansas

Purpose: The purpose of this study is to modify and validate an existing assessment, the Communication Complexity Scale (CCS), so that it reflects current communication abilities in school-age children with autism spectrum disorder (ASD) and minimal verbal skills, is reliable and valid for measuring outcomes and educational progress, can be learned by teachers and related service personnel within a feasible time period, and is affordable to use. The CCS has been developed and assessed with promising results but requires modification for live-scoring during naturally occurring communication in classroom contexts to provide ongoing progress and outcome data more efficiently than the original tool. These enhancements to the CCS would enable school personnel to have a reliable and valid means of evaluating communication progress and outcomes for students with ASD (ages 3–18 years) who are minimally verbal.

Project Activities: The research team will conduct four overlapping phases to gather data from special educators and their students to be used for iteratively refining the CCS. In Phase 1, the research team will develop live-scoring procedures for scoring the interactive assessment, which will be designed to provide opportunities for students to communicate. Phase 2 involves the development of live-scoring procedures for observations of classroom activities. In Phase 3, the team will evaluate the psychometric properties of the CCS when used to live-score assessments and communication during classroom activities. Finally, Phase 4 involves an assessment of educator usability, the development of training materials for using the CCS for both the assessment and classroom observations, and dissemination to promote the use of the newly modified tool in schools.

Products: The primary product from this project includes the modified CCS to assess communication skills for students with autism who are minimally verbal. In addition, products include peer-reviewed publications and presentations.

Structured Abstract

Setting: The research will take place in elementary schools with preschool programs and secondary schools in Kansas.

Sample: Approximately 100 students between the ages of 3 and 18 years who have ASD and minimal verbal skills (less than 20 words produced independently without prompting) will participate in this research. The researchers will also recruit 20 speech-language pathologists and other special educators who will learn to administer and score the assessment.

Assessment: The Communication Complexity Scale (CCS) is a 12-point scale that can be used to describe how a student with minimal verbal skills communicates (including pre-intentional, intentional, and symbolic modes of communication). The CCS assesses a range of behaviors, including, for example, alerting, changing behaviors in response to a change in the environment, and combining two symbolic forms of communication in a phrase. There is an interactive assessment paradigm that includes six opportunities designed to evoke requests and six opportunities designed to evoke joint attention. Student communication produced in response to this paradigm is scored. Administration of the CSS is expected to take approximately 30 minutes for children with ASD. The current study will determine if communication during authentic classroom activities can also be captured reliably and validly using the scale with some modifications.

Research Design and Methods: In Phase 1, the researchers will adapt the existing procedures that use video coding to enable the live scoring of structured assessments. To obtain data to assess reliability and validity, educators will learn to administer the structured protocol to three students with fidelity and to reliably score the CCS starting with existing video-recorded assessments and moving to live assessments of their students once criteria has been reached. In Phase 2, teachers will nominate four activities that involve the most communication to be videotaped. The videos will be used by research staff to develop coding procedures and clarify code book definitions. Once researchers can reliably score from the video without stopping, they will test the procedures by scoring live in classrooms. Educators will be trained using the revised definitions and protocols. Educators will follow the same process as research staff: initially scoring video, then scoring video without stopping, and finally live scoring the interactions in classrooms. Researchers will live score to examine reliability of the educators using the assessment. In Phase 3, concurrent and discriminant validity and sensitivity to change will be examined. In Phase 4, training materials will be developed for the assessment and classroom observation and their usability evaluated through a survey of educators experiences.

Control Condition: Due to the nature of the research design, there is no control condition.

Key Measures: In addition to the CCS, the Childhood Autism Rating Scale will be used as an indicator of autism severity and as a screener for participants. The Communication Matrix, a parent-report measure of communication skill, and the Communication subscale of the Vineland Adaptive Behavior Scales, Second Edition (Vineland-II) will be used to provide concurrent validity evidence. Socialization and Daily Living subscale scores from the Vineland II will provide discriminant validity evidence. Student progress toward meeting communication goals will be collected and assessed using Goal Attainment Scaling. Educator surveys will obtain data on their characteristics and experiences as well as assess the usability of the CCS live-scoring training manual and materials.

Data Analytic Strategy: Inter-rater reliability will be assessed using intra-class correlation coefficients, percent agreement, and weighted kappa for each communication mode (pre-intentional, intentional, and symbolic). Pearson correlations between CCS scores derived from structured assessments, live classroom observations, and other measures will be used to examine construct validity. Sensitivity to change for both structured and classroom observation scores will be examined using paired-samples t-tests. Fisher's exact test will be used to determine if there are differences between CCS classifications (i.e., improving/not improving communication) and Goal Attainment Scale classifications. Usability data will be analyzed with descriptive statistics and qualitative content analysis.