|Title:||Multiple-choice Online Causal Comprehension Assessment for Postsecondary Students (MOCCA-College): Measuring Individual Differences in Reading Comprehension Ability of Struggling College Readers by Text Type|
|Principal Investigator:||Seipel, Ben||Awardee:||California State University, Chico|
|Program:||Postsecondary and Adult Education [Program Details]|
|Award Period:||3 years (07/01/2018 - 06/30/2021)||Award Amount:||$1,383,123|
Co-Principal Investigator: Carlson, Sarah; Davison, Mark; Clinton, Virginia
Purpose: The research team will refine and validate a reading comprehension assessment originally designed for third- to fifth-grade readers for use with postsecondary students. The refined assessment, MOCCA-College (MOCCA-C) will be a practical, reliable, and valid measure of reading comprehension processes that can help postsecondary practitioners, administrators, and researchers diagnose students reading comprehension difficulties.
Project Activities: The researchers will use some of the narrative items from the existing assessment and develop new narrative and expository items. Through multiple iterations, they will refine these items and create three forms of the assessment before conducting validation studies with a nationally representative sample of postsecondary students.
Products: Researchers will produce a fully developed and validated assessment of postsecondary students' reading comprehension (narrative and expository text) and peer-reviewed publications.
Setting: The research will be conducted in 2- and 4-year postsecondary institutions in California, Georgia, Oregon, and North Dakota.
Sample: Approximately 2100 students will participate in the initial development studies, and over 4000 students will participate in the validation studies. They will also recruit six postsecondary instructors and program directors to serve in a content review panel.
Assessment: The Multiple-choice Online Causal Comprehension Assessment-College (MOCCA-C) will assess postsecondary students' reading comprehension ability and distinguish between common comprehension difficulties. MOCCA-C builds off an earlier measure, MOCCA, which was the product of an IES-funded project (R305A140185) and was designed for and validated with third- through fifth-grade students. The final MOCCA-C will consist of 3 scaled forms with 50 short narrative and informational texts that span grade 5 through 12+ on the Flesch-Kincaid (FK) reading scale. Each seven-sentence story is missing the sixth sentence. Test takers choose a sentence from three options to fill the missing sentence. One of the options is correct and creates a causally coherent connection, one is a paraphrase, and one is a lateral connection (i.e., an inference or association that is based on background knowledge, may be tangential, and isn't causally coherent).
Research Design and Methods: The researchers will take a subset of current MOCCA items and write an additional 100 items. Postsecondary instructors, administrators, and experts will review these items, giving feedback on their readability and cultural sensitivity. In Year 1, a sample of postsecondary students will pilot the MOCCA-C items, with a subset also completing think alouds. Using these data, the researchers will refine the items and develop three forms of MOCCA-C. In Year 2, they will then begin field testing the forms with an additional wave of postsecondary students and revise the forms or items if necessary. In the final year, they will conduct a second field test and evaluate the content, diagnostic, criterion, construct validity and fairness of the assessment.
Control Condition: Due to the nature of this research design, there is no control condition.
Key Measures: Key measures include existing standardized reading measures (e.g., TOWRE-2 and the Nelson Denny Reading Test) and common postsecondary admissions data (e.g., ACT/ SAT scores, high school GPA).
Data Analytic Strategy: The researchers will use item response theory (IRT), choice theory, logistic models, univariate and multivariate analyses, and reliability and validity analyses.
Related IES Projects: Multiple-choice Online Cloze Comprehension Assessment (MOCCA): Refining and Validating a Measure of Individual Differences in Reading Comprehension Processes During Reading (R305A140185)