Description:

Co-Principal Investigator: Hodara, Michelle

Purpose: As community colleges make changes to curriculum and course structures to improve outcomes in development education, postsecondary instructors increasingly need professional development. Yet, most college faculty have limited access to professional development focused on teaching in their discipline. This project iteratively adapted a collaborative-learning model for teacher professional development that has evidence of improving math instruction in K–12 settings called "lesson study" for use among developmental mathematics faculty at three community colleges. The adapted model was implemented with fidelity in a pilot study, and faculty adopted new evidence-based teaching practices after participating in lesson study. Students in some course sections taught by faculty who participated in the pilot study performed better on a researcher-designed assessment of student learning. However, the pilot study was not able to detect a positive relationship between lesson study and students' course grades or progression in college-level math. Findings from this project indicate features of professional development that may help faculty adopt new teaching practices and offer directions for future research on lesson study in higher education.

Project Activities: Lesson study experts from Education Northwest provided training on lesson study to mathematics faculty from three community colleges. The project team and faculty worked collaboratively on implementing and refining the lesson study model over three iterative cycles in 2018 and 2019. Faculty from across the participating institutions convened for three meetings for additional training, cross-college learning, and to provide feedback on the model. During this model development period, the research team collected data in the form of observations, a faculty survey, and interviews with faculty participants. In fall 2019, four teams of faculty teaching developmental quantitative literacy piloted the adapted lesson study model. Faculty who had participated in the model development phase of the project served as lesson study facilitators. During the pilot study, researchers conducted a follow-up faculty survey, collected observation data, and interviewed faculty and administrators to determine usability, feasibility, and fidelity. Students were assessed using a researcher-designed instrument. Researchers also analyzed course outcomes and student progression to college-level math using administrative data from the three colleges.

Key Features of the Intervention:

• Faculty work in teams of 3–6 people to plan a collaborative lesson for a selected course, teach the lesson to students, and observe how students interact with the lesson content.
• Then the team debriefs the first teaching, revises the lesson, and reteaches it to a new group of students.
• Structured protocols guide the team through each phase and direct teams to reflect on how learning from the lesson study cycle can be applied to their teaching more broadly.

Key Outcomes: The main features of the intervention and findings of the project's pilot study are as follows (see Bickerstaff et al., 2021): Implementation, Feasibility, and Usability Testing

• Lesson study is different from typical opportunities for professional development in community colleges, in part because it makes use of a highly structured approach to improving discipline-specific teaching practices.
• Faculty implemented lesson study with fidelity, as measured by an 18-indicator rubric of strong lesson study implementation.
• Faculty who participated in lesson study adopted new teaching practices. Analysis of observation notes and lesson plans from the model development and pilot phases showed that revised lessons included more open-ended, cognitively demanding tasks and new strategies to increase mathematical communication among students.
• To sustain lesson study, institutions must provide support. Lesson study requires administrative coordination and a skilled facilitator.

Pilot Study

• The study found a positive, statistically significant relationship between faculty participation in lesson study participation and student performance on selected items on a researcher-designed assessment for students in sections of one course that was the focus of lesson study.
• The study found no positive relationship between lesson study and students' course grades or progression into college-level math. However, the researchers were unable to establish baseline equivalency between students in the treatment and comparison groups on prior academic achievement.

Structured Abstract

Setting: The research was conducted in one urban, one rural, and one suburban community college in Oregon.

Sample: Twenty-two math faculty from three community colleges participated in the project. Of these faculty, 11 contributed to the development of the lesson study model. During the pilot study, data were collected from 331 developmental math students in a developmental quantitative literacy course taught by 8 participating and 7 nonparticipating faculty members.

Intervention: Lesson study is a structured, collaborative professional development approach that gives instructors a framework for actively investigating how to improve student learning in their classrooms. Lesson study engages a team of educators in iterative cycles of inquiry that focus on lesson design for improved student learning. The adapted lesson study model includes four cycle phases which take about 20 hours to complete: study and plan a lesson; teach, observe, and debrief; revise and reteach the lesson; and reflect and report on professional learning. Among the indicators of strong lesson study implementation are attending to collaboration norms, maintaining an inquiry focus on student learning, applying evidence-based teaching practices in lesson design, and considering broader applications to teaching practice beyond the single lesson. The protocols for each cycle activity prompt lesson study teams to observe and closely examine specific instructional strategies and their impact on student learning. These practices create an environment of experimentation that support faculty to experiment with approaches to teaching they may not otherwise try.

Research Design and Methods: This project used qualitative and quantitative methods to study the implementation and outcomes of the lesson study intervention. During the development phase, the research team interviewed participating and nonparticipating faculty and college administrators, conducted a survey of mathematics faculty, and observed and took field notes at lesson study training and cycle activities. For each lesson study cycle, the research team collected two versions of the lesson plan. During this phase, researchers sought to understand the usability and feasibility of the model, faculty perspectives and experiences, and the ways in which the approach is different from typical faculty development activities. This research informed the adaptations to the model.

During the pilot study, researchers collected and analyzed lesson plans and observed lesson study activities and scored each team's cycle using a fidelity rubric. The research team conducted a follow-up faculty survey and interviewed faculty who participated in the pilot study. Across the development and pilot phases, researchers conducted a total of 47 semi-structured interviews and observed and took field notes during 10 lesson study cycles.

The pilot study also included an analysis of student learning and math course outcomes in developmental quantitative literacy course sections. The research team administered a researcher-designed assessment of student learning and compared the performance of students taught by participating and nonparticipating faculty members. The research team also analyzed student-level administrative data from students at the three participating colleges to assess the association between faculty participation in the intervention and student course outcomes.

Control Condition: Students enrolled in developmental quantitative literacy course sections taught by nonparticipating faculty members comprise the control condition. There were 160 students in the control condition.

Key Measures: The research team collaborated with faculty partners from the three colleges to develop an assessment consisting of four open-ended items focused on percentages—a key concept that underlies many skills and learning outcomes in the quantitative literacy pathway. Researchers scored assessments using a rubric. For the analysis of student-level administrative data, the study examined four outcome measures: grade in the quantitative literacy course, whether students passed the course with a C or better, their persistence to the end of the course (that is, they did not withdraw or have an incomplete grade), and whether they enrolled in a college-level math course in the subsequent term.

Data Analytic Strategy: The researchers thematically coded the qualitative data collected during the model development and pilot phases to determine the contrast between lesson study and other professional development opportunities and to explore barriers to feasibility, usability, and fidelity. They ran pairwise t-tests to identify differences in responses in the faculty survey across time and faculty population. Using faculty time logs during the pilot study and interviews with faculty and administrators, the research team identified key drivers of the cost of implementing the adapted model. For the pilot study, assessment data were matched with student administrative records. The research team compared the performance of students in sections taught by faculty participants with students in sections taught by nonparticipants, as well as students in pre-pilot course sections (in 2018), using Pearson chi-square tests. Second, to control for underlying differences between students in sections taught by faculty participants and those in sections taught by nonparticipants, the research team conducted logistic regressions that accounted for students' demographic characteristics, including race/ethnicity, gender, age, Pell Grant status, and prior enrollment.

Using administrative data on student course outcomes, the research team used propensity score weighting and regression analysis to assess the relationship between enrollment in a math course taught by faculty participating in lesson study and the outcomes of interest. The research team did not have a measure of prior academic achievement for students in the sample but were able to ensure that students taught by lesson study participants and those taught by nonparticipants were demographically equivalent. The study also used a multilevel model to account for college and instructor characteristics that could explain differences in outcomes between students in sections taught by participants and those in sections taught by nonparticipants.

Products

ERIC Citations: Find available citations in ERIC for this award here.

Project Websites:

• https://educationnorthwest.org/resources/adapting-lesson-study-community-college-math-instruction
• https://ccrc.tc.columbia.edu/research-project/adapting-lesson-study-for-community-college-mathematics-instruction.html

Select Publications:

Nongovernment reports

Bickerstaff, S., Raphael, J., Hodara, M., Leasor, L. & Riggs, S. (2021). The implementation and outcomes of lesson study in community college mathematics. Columbia University, Teachers College, Community College Research Center.  Full Text

Bickerstaff, S., Raphael, J., Zamora, D. E. C., & Leong, M. (2019). Adapting Lesson Study for Community College Mathematics Instruction.Community College Research Center, Teachers College, Columbia University. Full Text