About this practice guide

The Institute of Education Sciences (IES) publishes practice guides in education to provide educators with the best available evidence and expertise on current challenges in education. The What Works Clearinghouse (WWC) develops practice guides in conjunction with an expert panel, combining the panel’s expertise with the findings of existing rigorous research to produce specific recommendations for addressing these challenges. The WWC and the panel rate the strength of the research evidence supporting each of their recommendations. See Appendix A for a full description of practice guides.

The goal of this practice guide is to offer educators specific, evidence-based recommendations that address the challenges of teaching algebra to students in grades 6 through 12. This guide synthesizes the best available research and shares practices that are supported by evidence. It is intended to be practical and easy for teachers to use. The guide includes many examples in each recommendation to demonstrate the concepts discussed.

Practice guides published by IES are available on the What Works Clearinghouse website at http://whatworks.ed.gov.

How to use this guide

This guide provides educators with instructional recommendations that can be implemented in conjunction with existing standards or curricula and does not recommend a particular curriculum. Teachers can use the guide when planning instruction to prepare students for future mathematics and post-secondary success. The recommendations are appropriate for use with all students learning algebra in grades 6–12 and in diverse contexts, including for use during both formative and summative assessment.

Administrators, professional development providers, program developers, and researchers can also use this guide. Administrators and professional development providers can use the guide to implement evidence-based instruction and align instruction with state standards or to prompt teacher discussion in professional learning communities. Program developers can use the guide to create more effective algebra curricula and interventions. Finally, researchers may find opportunities to test the effectiveness of various approaches to algebra and explore gaps or variations in the algebra instruction literature.

Teaching Strategies for Improving Algebra Knowledge in Middle and High School Students

April 2015

Panel

Jon R. Star (Chair)

HARVARD UNIVERSITY

Anne Foegen

IOWA STATE UNIVERSITY

Matthew R. Larson

LINCOLN PUBLIC SCHOOLS

William G. McCallum

UNIVERSITY OF ARIZONA

Jane Porath

TRAVERSE CITY AREA PUBLIC SCHOOLS

Rose Mary Zbiek

PENNSYLVANIA STATE UNIVERSITY

Staff

Pia Caronongan

Joshua Furgeson

Betsy Keating

Julia Lyskawa

MATHEMATICA POLICY RESEARCH

Project Officers

Diana McCallum

Vanessa Anderson

INSTITUTE OF EDUCATION SCIENCES

NCEE 2015-4010

U.S. DEPARTMENT OF EDUCATION

This report was prepared for the National Center for Education Evaluation and Regional Assistance, Institute of Education Sciences, under the What Works Clearinghouse contract to Mathematica Policy Research (Contract ED-IES-13-C-0010).

Disclaimer

The opinions and positions expressed in this practice guide are those of the authors and do not necessarily represent the opinions and positions of the Institute of Education Sciences or the U.S. Department of Education. This practice guide should be reviewed and applied according to the specific needs of the educators and education agency using it, and with full realization that it represents the judgments of the review panel regarding what constitutes sensible practice, based on the research that was available at the time of publication. This practice guide should be used as a tool to assist in decision making rather than as a “cookbook.” Any references within the document to specific education products are illustrative and do not imply endorsement of these products to the exclusion of other products that are not referenced.

U.S. Department of Education

Arne Duncan Secretary

Institute of Education Sciences

Sue Betka Acting Director

National Center for Education Evaluation and Regional Assistance

Ruth Neild Commissioner

Joy Lesnick Associate Commissioner

April 2015

This report is in the public domain. Although permission to reprint this publication is not necessary, the citation should be as follows:

Star, J. R., Caronongan, P., Foegen, A., Furgeson, J., Keating, B., Larson, M. R., Lyskawa, J., McCallum, W. G., Porath, J., & Zbiek, R. M. (2015). Teaching strategies for improving algebra knowledge in middle and high school students (NCEE 2014-4333). Washington, DC: National Center for Education Evaluation and Regional Assistance (NCEE), Institute of Education Sciences, U.S. Department of Education. Retrieved from the NCEE website: http://whatworks.ed.gov.

The citation for this What Works Clearinghouse practice guide begins with the panel chair, followed by the names of the panelists and staff listed in alphabetical order.

This report is available on the IES website at http://whatworks.ed.gov.

Alternate Formats

On request, this publication can be made available in alternate formats, such as Braille, large print, or CD. For more information, contact the Alternate Format Center at (202) 260-0852 or (202) 260-0818.

Teaching Strategies for Improving Algebra Knowledge in Middle and High School Students

Table of Contents

Introduction to the Teaching Strategies for Improving Algebra Knowledge in Middle and High School Students Practice Guide

Recommendation 1. Use solved problems to engage students in analyzing algebraic reasoning and strategies

Recommendation 2. Teach students to utilize the structure of algebraic representations

Recommendation 3. Teach students to intentionally choose from alternative algebraic strategies when solving problems

Glossary

Appendix A. Postscript from the Institute of Education Sciences

Appendix B. About the Authors

Appendix C. Disclosure of Potential Conflicts of Interest

Appendix D. Rationale for Evidence Ratings

References

Endnotes

List of Tables

Table 1. Recommendations and corresponding levels of evidence

Table A.1. Institute of Education Sciences levels of evidence for What Works Clearinghouse practice guides

Table D.1. Outcome domain definitions and sample items

Table D.2. Studies providing evidence for Recommendation 1

Table D.3. Studies providing evidence for Recommendation 2

Table D.4. Studies providing evidence for Recommendation 3

List of Examples

Recommendation 1

Example 1.1. Questions to facilitate discussion of solved problems

Example 1.2. Questions to facilitate discussion of the structure of problems

Example 1.3. Examples of solved problems for different learning objectives

Example 1.4. Presenting multiple solved problems with varying degrees of difficulty

Example 1.5. One way to introduce incorrect solved problems

Example 1.6. Parallel correct and incorrect solved problems, factoring

Example 1.7. Parallel correct and incorrect solved problems, completing the square

Example 1.8. Using solved problems in whole-class discussion

Example 1.9. Solved problems in small-group work

Example 1.10. Incomplete solved problems

Example 1.11. Independent practice that incorporates solved problems

Recommendation 2

Example 2.1. Seeing structure in algebraic representations

Example 2.2. Modeling precise mathematical language, the distributive property

Example 2.3. Imprecise vs. precise mathematical language

Example 2.4. Student using reflective questioning

Example 2.5. Reflective questions for noticing structure

Example 2.6. Equations of the same line in different forms

Example 2.7. Equations of the same parabola in different forms

Example 2.8. Multiple algebraic representations

Example 2.9. Using diagrams to compare algebraic representations

Example 2.10. Examples of cooperative learning strategies

Recommendation 3

Example 3.1. Alternative and conventional solution strategies

Example 3.2. The same problem solved using two different solution strategies

Example 3.3. Two different problems, each solved with the same strategy

Example 3.4. Reflective questions for selecting and considering solution strategies

Example 3.5. Some possible strategies for solving quadratic equations

Example 3.6. Some possible strategies for solving linear systems

Example 3.7. Prompts to encourage students to articulate their reasoning

Example 3.8. Sample student dialogue discussing two different solution strategies

Example 3.9. Small-group comparison and discussion activity

Introduction to the Teaching Strategies for Improving Algebra Knowledge in Middle and High School Students Practice Guide

Algebra is often the first mathematics subject that requires extensive abstract thinking, a challenging new skill for many students. Algebra moves students beyond an emphasis on arithmetic operations to focus on the use of symbols to represent numbers and express mathematical relationships. Understanding algebra is a key for success in future mathematics courses, including geometry and calculus. Many mathematics experts also consider algebra knowledge and skills important for post-secondary success as well as for producing a skilled workforce for scientific and technical careers.1 Algebra requires proficiency with multiple representations, including symbols, equations, and graphs, as well as the ability to reason logically, both of which play crucial roles in advanced mathematics courses.

Most states have standards for student knowledge in high school algebra. However, these standards do not typically provide evidence-based^2 instructional guidance for implementing the standards. This practice guide provides educators with recommendations and resources to help students achieve success in algebra.

This practice guide presents evidenced-based suggestions for how to improve algebra skills and knowledge for students in grades 6–12. The guide offers three recommendations that provide teachers with specific, actionable guidance for implementing these practices in their classrooms. It also provides a level of supporting evidence for each recommendation, examples to use in class, and solutions to potential implementation challenges.

Overarching themes

This practice guide highlights three general and interrelated themes for improving the teaching and learning of algebra.

• Developing a deeper understanding of algebra. Although proficiency in arithmetic operations is important to becoming proficient in algebra, the recommendations advocate algebra instruction that moves students beyond superficial mathematics knowledge and toward a deeper understanding of algebra. This includes encouraging students to make connections between algebraic concepts and the procedures present in problems, and helping students recognize how the placement of the quantities relative to the operations in problems impacts the solution strategy. Teachers can prompt students to consider: What am I being asked to do in this problem? What do I know about the form of this expression or equation? What are the relationships between the quantities in this expression or equation? How can I check that my solution is correct?

• Promoting process-oriented thinking. The guide emphasizes moving beyond a primary focus on the correct final answer to algebra problems to also promoting the understanding of the processes by which one arrives at an answer. For example, the guide encourages students to consider questions such as the following: What decisions did you make to solve the problem? What steps did you take to solve the problem? Was this a good strategy? Why or why not? Are there other ways to solve the problem? Can you show (through manipulatives, pictures, or number-lines) how you solved the problem?

• Encouraging precise communication. The guide prompts teachers to provide frequent opportunities for students to reason with and talk about mathematical concepts, procedures, and strategies using precise mathematical language. This communication plays a key role in helping students develop mathematical understanding. For example, the guide encourages teachers to ask students: How would you describe this problem using precise mathematical language? How would you describe your strategy for solving this problem using precise mathematical language?

Overview of the recommendations

Recommendation 1. Use solved problems to engage students in analyzing algebraic reasoning and strategies.

1. Have students discuss solved problem structures and solutions to make connections among strategies and reasoning.

2. Select solved problems that reflect the lesson’s instructional aim, including problems that illustrate common errors.

3. Use whole-class discussions, small-group work, and independent practice activities to introduce, elaborate on, and practice working with solved problems.

Recommendation 2. Teach students to utilize the structure of algebraic representations.

1. Promote the use of language that reflects mathematical structure.

2. Encourage students to use reflective questioning to notice structure as they solve problems.

3. Teach students that different algebraic representations can convey different information about an algebra problem.

Recommendation 3. Teach students to intentionally choose from alternative algebraic strategies when solving problems.

1. Teach students to recognize and generate strategies for solving problems.

2. Encourage students to articulate the reasoning behind their choice of strategy and the mathematical validity of their strategy when solving problems.

3. Have students evaluate and compare different strategies for solving problems.

Summary of supporting research

Practice guide staff conducted a thorough literature search, identified studies that met protocol requirements, and then reviewed those studies using the What Works Clearinghouse (WWC) design standards. This literature search focused on studies published within 20 years from the start of the review, as is standard on WWC literature searches. The time frame was established to help ensure that the guide characterizes effectiveness from the research base relative to conditions similar to those in schools today and to define a realistic scope of work for the review.

A search for literature related to algebra instruction published between 1993 and 2013 yielded more than 2,800 citations. These studies were all screened for relevance according to eligibility criteria described in the practice guide protocol.3 Studies that did not include populations of interest, measure relevant outcomes, or assess the effectiveness of replicable practices used to teach algebra were excluded. Consistent with the protocol, the literature search and screening excluded general policies, such as an extra period of algebra, that are typically determined by administrators and not teachers. Of the eligible studies, 30 studies used experimental and quasi-experimental designs to examine the effectiveness of the practices found in this guide’s recommendations. From this subset, 15 studies met the WWC’s rigorous evidence group design standards and were related to the panel’s recommendations. Studies were classified as having a positive or negative effect when the result was either statistically significant (unlikely to occur by chance) or substantively important (producing considerable differences in outcomes).

In this guide, the different types of knowledge and skills that algebra students are expected to master are classified into three domains: conceptual knowledge, procedural knowledge, and procedural flexibility. Conceptual knowledge includes understanding algebraic ideas, operations, procedures, and notation. Procedural knowledge includes choosing operations and procedures to solve algebra problems, as well as applying operations and procedures to arrive at the correct solution to problems. Procedural flexibility includes identifying and implementing multiple methods to solve algebra problems, as well as choosing the most appropriate method. (For more information about the domains and how outcomes were classified into the domains, see Appendix D.) Some of the recommendations in this guide are more effective in improving achievement in certain domains than others. For example, most of the supporting evidence for Recommendation 3 suggested that the practices in this recommendation are most effective in improving procedural flexibility. The evidence linking Recommendation 3 to improvement in conceptual knowledge and procedural knowledge was not as strong.

The recommendations can be implemented individually in any order or together at the same time with one exception: as the evidence suggests and as the panel also believes, the practices proposed in Recommendation 3 are most effective if they are implemented once students have some fluency with algebra procedures and strategies. One recommended practice is no more or less important than another. A practice from one recommendation can also be used to implement another recommendation. For example, solved problems (Recommendation 1) can be used to teach algebraic structure (Recommendation 2) and multiple solution strategies (Recommendation 3). The panel believes that each recommendation can be used to develop conceptual knowledge, procedural knowledge, and procedural flexibility. For example, solved problems can help students learn concepts, strategies, reasoning, and algebraic structure.

The evidence level for each recommendation is based on an assessment of the relevant evidence supporting each recommendation. (Appendix A describes the criteria for each level of evidence.) Table 1 shows the level of evidence rating for each recommendation as determined by WWC guidelines outlined in Table A.1 in Appendix A. (Appendix D presents more information on the body of evidence supporting each recommendation.)

Table 1. Recommendations and corresponding levels of evidence

How to use this guide

This guide provides educators with instructional recommendations that can be implemented in conjunction with existing standards or curriculum and does not prescribe the use of a particular curricula. Teachers can use the guide when planning instruction to prepare students for future mathematics and post-secondary success. The recommendations are appropriate for use with all students learning algebra in grades 6–12 and in diverse contexts, including for use during both formative and summative assessment.

Administrators, professional development providers, program developers, and researchers can also use this guide. Administrators and professional development providers can use the guide to implement evidence-based instruction and align instruction with state standards or to prompt teacher discussion in professional learning communities. Developers can use the guide to develop more effective algebra curricula and interventions. Finally, researchers may find opportunities to test the effectiveness of various approaches to algebra and explore gaps or variations in the algebra instruction literature.

Recommendation 1

Use solved problems to engage students in analyzing algebraic reasoning and strategies.

Compared to elementary mathematics work like arithmetic, solving algebra problems often requires students to think more abstractly. Algebraic reasoning requires students to process multiple pieces of complex information simultaneously, which can limit students’ capacity to develop new knowledge. (Such reasoning is sometimes described as imposing high cognitive load or challenging working memory, which can interfere with students’ ability to learn.4) Solved problems can minimize the burden of abstract reasoning by allowing students to see the problem and many solution steps at once—without executing each step—helping students learn more efficiently.5

Analyzing and discussing solved problems can also help students develop a deeper understanding of the logical processes used to solve algebra problems.6 Discussion and the use of incomplete or incorrect solved problems can encourage students to think critically.

Summary of evidence: Minimal Evidence

Four studies examined the effects of using solved problems in algebra instruction and met WWC group design standards without reservations (see Appendix D).7 Of the four, three studies showed positive effects^8 on conceptual knowledge.9 One of these three studies showed a positive effect of providing correct and incorrect solved problems and prompting students to explain the solution, compared to students who were prompted similarly as they solved practice problems.10 Two additional studies demonstrated that providing students with solved problems alongside practice problems had positive effects on student achievement, compared to students who received additional practice problems.11 These three studies examined solved problems with students in remedial, regular, and honors algebra classes. The remaining study found that solved problems had negative effects on conceptual and procedural knowledge.12 This study compared students who studied solved problems to students who used reflective questioning (a practice suggested in Recommendation 2).13 This body of evidence suggests that, compared to asking students to solve practice problems alone, studying solved problems can improve achievement.

How to carry out the recommendation

1. Have students discuss solved problem structures and solutions to make connections among strategies and reasoning.

Create opportunities for students to discuss and analyze solved problems by asking students to describe the steps taken in the solved problem and to explain the reasoning used. Ask students specific questions about the solution strategy, and whether that strategy is logical and mathematically correct. Asking these questions encourages active student engagement. Vary the questions based on the needs of students and the types of problems being discussed. The questions can be asked verbally or written down for students to reference. Example 1.1 presents general questions that could be applicable to many types of algebra problems. (See later examples in this recommendation for ideas about how to incorporate these and other specifically tailored questions to sample solved problems.)

Example 1.1. Questions to facilitate discussion of solved problems

Foster extended analysis of solved problems by asking students to notice and explain different aspects of a problem’s structure (Example 1.2). Carefully reviewing and discussing the structure and each solution step of a solved problem helps students recognize the sequential nature of solutions and anticipate the next step in solving a problem. This can improve students’ ability to understand the reasoning behind different problem-solving strategies.

Example 1.2. Questions to facilitate discussion of the structure of problems

2. Select solved problems that reflect the lesson’s instructional aim, including problems that illustrate common errors.

Use solved problems to accomplish diverse learning objectives (Example 1.3). Choose solved problems that are linked to the learning objectives from student examples (including from current or past students and other teachers’ students) or curricular materials, or make up examples. Specific types of solved problems—different problems solved with similar solution steps and incorrect solved problems—can be used for specific learning objectives. During the lesson, allow students to consult solved problems independently or in groups to understand different ways to solve a problem when they are unsure how to proceed.

Presenting several solved problems that use similar solution steps can help students see how to approach different problems that have similar structures (Example 1.4). To incorporate multiple solved problems into a lesson, consider the following approaches:

• Select problems with varying levels of difficulty and arrange them from simplest to most complex applications of the same concept.

• Display the multiple examples simultaneously to encourage students to recognize patterns in the solution steps across problems.

• Alternatively, show the problems individually, one after the other, to facilitate more detailed discussion on each problem.

Example 1.3. Examples of solved problems for different learning objectives^14

Example 1.4. Presenting multiple solved problems with varying degrees of difficulty

After reviewing correct solved problems, use incorrect solved problems to help students deepen their understanding of concepts and correct solution strategies by analyzing strategic, reasoning, and procedural errors. Students can discuss problems and strategies that they know are incorrect to develop a better understanding of the process used to obtain a correct solution (Example 1.5).

Example 1.5. One way to introduce incorrect solved problems

When analyzing an incorrect solved problem, students should explain why identified errors led to an incorrect answer so they can better understand the correct processes and strategies.

Present an incorrect solved problem by itself, or display it beside the same problem solved correctly and ask students to compare the two strategies used. Clearly label correct and incorrect examples, so students do not confuse correct and incorrect strategies. One option, as shown in Example 1.6, is to show a correct solved problem alongside multiple incorrect solved problems and ask probing questions to draw students’ attention to the errors and help them understand what was done incorrectly.

Example 1.6. Parallel correct and incorrect solved problems, factoring

Include different types of errors—for example, strategic errors and procedural (or computational) errors—next to a correct solved problem (Example 1.7). Errors can arise from using an incorrect strategy (a strategic or reasoning error) or from making a mathematical operations mistake (a procedural or computational error). Encourage students to think critically about how different steps and choices can lead to errors.

Example 1.7. Parallel correct and incorrect solved problems, completing the square

3. Use whole-class discussions, small-group work, and independent practice activities to introduce, elaborate on, and practice working with solved problems.

Although the studies that meet WWC design standards did not examine the effect of solved problems across different situations, the panel believes that solved problems are useful in a variety of contexts. Introduce solved problems during whole-class instruction to provide an overview of a solution strategy. For example, after beginning to explain slope-intercept form, use solved problems to help introduce the concept of graphing linear equations (Example 1.8).

Example 1.8. Using solved problems in whole-class discussion

Create activities for pairs or small groups of students to critically analyze solved problems. For example, present solved problems, including a combination of incorrect and correct solved problems, with accompanying questions for students to answer in small groups of two or three (Example 1.9). The questions can ask about the steps, structures, and strategies in the solved problems. After students work in small groups, bring the whole class together to discuss the problems further using mathematically appropriate language, and engage all students to address relevant components of the problem.

Example 1.9. Solved problems in small-group work^16

Solved problems can replace some exercises in students’ independent practice. After a lesson, give students incomplete solved problems and ask students to complete the solutions (Example 1.10).

Example 1.10. Incomplete solved problems

In the incomplete solved problems, determine which step of the problem should remain blank, depending on the students’ familiarity with the topic and the goal of the lesson.

Another strategy for incorporating solved problems into independent practice activities is alternating solved problems with unsolved problems that are similar to the solved problems in terms of problem structure or solution strategy (Example 1.11). Create exercises that show a solved problem followed by an unsolved problem for students to complete. Students can use the solved problem as an example to guide their independent work, while the unsolved problem provides a way for students to actively engage with the solved problem.

Example 1.11. Independent practice that incorporates solved problems

Potential roadblocks and suggested approaches

Roadblock 1.1. I already use solved problems during whole-class instruction, but I’m not sure students are fully engaged with them.

Suggested Approach. Whole-class discussion and analysis of solved problems can help guide students to notice aspects of the solved problems that are important. Asking questions and fostering discussion encourages students to think critically about solved problems. During whole-class instruction, model engaging with solved problems by using think-aloud questions. See Example 1.1 for examples of the types of questions that can foster discussion and analysis, and see Example 1.2 for important aspects of problems to highlight.

Look for opportunities to incorporate solved problems into many aspects of the lesson beyond whole-class instruction. During small group or independent work, help students engage with solved problems in a more meaningful way (as opposed to simply studying them) by providing students with probing questions (as in Example 1.9), requiring students to fill in missing steps of incomplete solved problems (as in Example 1.10), or asking students to solve problems themselves after studying a similar solved problem (as in Example 1.11). Class assessments can also ask students to analyze solved problems in the same manner in which they were incorporated in the lessons.

Roadblock 1.2. I do not know where to find solved problems to use in my classroom and do not have time to make new examples for my lessons.

Suggested Approach. Solved problems can come from many sources. Often, curriculum materials or textbooks may include sample or solved problems. In addition, student work can provide solved problems. Consider current students’ responses on homework, projects, and assessments, including unique strategies to solve a problem, to find suitable material for solved problems. Past student work can also provide examples for solved problems, particularly related to common strategic or procedural errors. Finally, ask other teachers in your school if they have any solved problems that they can share, or if they have any sample student work that could be adapted into solved problems.

Roadblock 1.3. I’m worried that showing students incorrect solved problems will confuse them.

Suggested Approach. Exposing students to incorrect solved problems can help students understand the difference between correct and incorrect solution strategies and can highlight the reasoning behind common errors. If students are not accustomed to analyzing work and interpreting the math, they may initially resist analyzing incorrect solved problems. Setting the expectation that understanding the process is as important as a final correct answer helps students adjust and become more comfortable with using and discussing incorrect solved problems. Clearly identify and label incorrect solved problems to ensure that students recognize that there is an error within the solution steps. Presenting parallel correct solved problems alongside incorrect solved problems can help students recognize and analyze the error. Engage students in a discussion about each step of the incorrect solved problem, thoroughly explaining and discussing the nature of the error to reduce confusion about which strategies were applied correctly and incorrectly. Students who can identify and discuss the reasons for where and why an error exists may be better able to recognize when and how to use correct strategies as they later solve problems independently. In addition, discussion of incorrect solved problems allows students to confront their own potential errors in a non-threatening way. When evaluating hypothetical student work, students can be more open and honest in their analysis of errors than they might be if critiquing their own work. See Examples 1.5, 1.6, and 1.7 for examples of how to implement incorrect solved problems to minimize student confusion.

Recommendation 2

Teach students to utilize the structure of algebraic representations.

Structure refers to an algebraic representation’s underlying mathematical features and relationships, such as

• the number, type, and position of quantities, including variables

• the number, type, and position of operations

• the presence of an equality or inequality

• the relationships between quantities, operations, and equalities or inequalities

• the range of complexity among expressions, with simpler expressions nested inside more complex ones

Example 2.1. Seeing structure in algebraic representations

Paying attention to structure helps students make connections among problems, solution strategies, and representations that may initially appear different but are actually mathematically similar (Example 2.1). With an understanding of structure, students can focus on the mathematical similarities of problems that may appear to be different, which can simplify solving algebra problems. In particular, recognizing structure helps students understand the characteristics of algebraic expressions and problems regardless of whether the problems are presented in symbolic, numeric, verbal, or graphic forms.

Summary of evidence: Minimal Evidence

Six studies contributed to the level of evidence for this recommendation. Four studies met WWC group design standards without reservations, and two studies did not meet WWC group design standards but adequately controlled for selection bias (see Appendix D).17 Four of the six studies found positive effects^18 on procedural knowledge,19 and three studies found positive effects on con-ceptual knowledge.20 However, none of the studies examined an important component of the recommendation: the use of language that reflects mathematical structure. Two of the six studies taught students to use reflective questioning, one method of focusing on structure.21 One study showed positive effects on procedural and conceptual knowledge for reflective questioning compared to having students study solved problems in small groups, and the other study showed positive effects on procedural knowledge, for reflective questioning compared to regular instruction without reflective questioning.22 The remaining four studies compared using graphical representations—another recommended method of focusing on structure—to similar instruction that did not involve graphical representations.23 Two of the four studies had positive effects on procedural knowledge.24 The other two studies found that using graphical representations had positive effects on conceptual knowledge.25 The populations included in the six studies varied; two of the studies were conducted outside of the United States,26 and two study samples included students with specific learning challenges.27 Overall, the body of evidence indicated that teaching reflective questioning or using graphical representations improves procedural and conceptual knowledge across diverse populations.

How to carry out this recommendation

1. Promote the use of language that reflects mathematical structure.

Although no study that meets WWC group design standards examines the effect of using precise mathematical language, the panel believes that using such language is a key component to understanding structure and sets the foundation for the use of reflective questioning, multiple representations, and diagrams (discussed in more detail below). When talking to students, phrase algebra solution steps in precise mathematical language to communicate the logical meaning of a problem’s structure, operations, solution steps, and strategies (Example 2.2). Use precise mathematical language to help students analyze and verbally describe the specific features that make up the structure of algebraic representations. When introducing a new topic or concept, use and model precise mathematical language to encourage students to describe the structure of algebra problems with accurate and appropriate terms.

During whole-class instruction, teachers can rephrase student solutions and responses to questions using appropriate mathematical language instead of vague, non-mathematical language (Example 2.3). The panel believes that precise mathematical language more accurately and completely describes the mathematical validity of a problem, helping students better understand quantities and operations, along with the relationships between them. In addition, as suggested in Example 2.3, a vague expression might have more than one meaning.

Example 2.2. Modeling precise mathematical language, the distributive property

Example 2.3. Imprecise vs. precise mathematical language

2. Encourage students to use reflective questioning to notice structure as they solve problems.

By asking themselves questions about a problem they are solving, students can think about the structure of the problem and the potential strategies they could use to solve the problem. First, model reflective questioning to students by thinking aloud while solving a problem. Teachers can write down the questions they ask themselves to clearly demonstrate the steps of their thinking processes. Then present a problem during whole-class instruction, and ask students to write down what questions they might ask themselves to solve the problem. Students can practice the think-aloud process while working in pairs or share their written ideas with a partner. This process will help students use reflective questioning on their own during independent practice to explore algebraic structure (Example 2.4).

Example 2.4. Student using reflective questioning

To help students with reflective questioning, hang a poster or distribute a list of common questions students can ask themselves while solving a problem (Example 2.5). Initial question lists can be updated as new questions are used in class, helping students connect the questions to new learning experiences.

Example 2.5. Reflective questions for noticing structure

3. Teach students that different algebraic representations can convey different information about an algebra problem.

Recognizing and explaining corresponding features of the structure of two representations can help students understand the relationships among several algebraic representations, such as equations, graphs, and word problems. Examples 2.6 and 2.7 demonstrate how equations represented in different forms provide different information. Teachers can present students with equations in different forms and ask students to identify the similarities and differences. Working in pairs, students can then discuss the similarities and differences they identified.

Example 2.6. Equations of the same line in different forms

Example 2.7. Equations of the same parabola in different forms^28

Help students see that different representations based on the same information can display the information differently, as in Example 2.8. Specific representations can exhibit some information about the problem’s structure more readily than other representations. For example, students might find it easier to find the x-intercepts of the graph of a quadratic function if it is expressed in factored form rather than in standard form.

As needed, incorporate diagrams into instruction to demonstrate the similarities and differences between representations of algebra problems to students (Example 2.9). Encourage students to use a diagram to visualize the structure of a problem, organize and document the solution steps of the problem, and translate the problem into another representation, as illustrated in several studies reviewed for this guide.29

Example 2.8. Multiple algebraic representations

Example 2.8. Multiple algebraic representations (continued)

Example 2.9. Using diagrams to compare algebraic representations

Potential roadblocks and suggested approaches

Roadblock 2.1. I like to simplify mathematical language, and my students seem to respond positively to my use of simplified and informal language as well. Doesn’t this approach make it easier for students than using complicated mathematical language?

Suggested Approach. Informal language often relies on superficial features such as the position of symbols on the page, rather than on the underlying mathematical operations. Informal language can introduce misconceptions and confusion during standardized assessments where precise language is used, adding unnecessary complexity by giving students another idea to understand. While it is not harmful to use informal language in the classroom, it is necessary for students to know precise mathematical language and understand the logical meaning behind it.

Precise mathematical language is not necessarily more complicated, but it is more mathematically accurate. For example, instructing students to “Add the opposite to each side” is a precise but still simple way to tell them to use inverse operations, instead of the more imprecise phrase “Do the opposite to each side.” Precise language facilitates mathematical communication beyond the classroom by promoting the use of common language across classrooms, grade levels, and assessments. Precision in language serves to highlight mathematical structure, helping students notice problem components such as quantities, operations, and their interrelationships. Referring to these components frequently and with precise language helps make them more accessible to students.

Roadblock 2.2. My students race through problems. How do I get students to slow down, pause to ask themselves questions, and think about the problem?

Suggested Approach. If students are moving through problems quickly without pausing to think or ask themselves questions, this might suggest one of two possibilities. First, the problems may in fact be too familiar and easy for students if they can successfully solve the problems without a lot of reflective thought. If this is the case, consider offering students variations of familiar problems that have similar mathematical structure to what they are familiar with but that may look very different from what they are used to. Challenge students to solve problems in more than one way and to explain the differences between the solution strategies. As a second possibility, students may have developed well-practiced strategies that they prefer to use, so they may not consider alternative strategies, even when their preferred strategies are not optimal. Consider assigning tasks that require students to respond to reflective questions, perhaps even instead of actually solving the problem. In either case, the expert panel believes it might be fruitful to use cooperative learning strategies, such as those presented in Example 2.10, to encourage students to use reflective questioning and carefully think through algebra problems.

Roadblock 2.3. Diagrams don’t seem to be very useful to some of my students.

Suggested Approach. Some students will find the correct answer to a problem without the need for a diagram. However, sometimes a student can work toward the right answer without noticing or attending to the problem’s structure or even without understanding what the problem is about. Diagrams can illuminate the mathematical structure of a problem and facilitate understanding of the mathematics behind the problem. Thus, even when a diagram is not perceived to be needed for getting the right answer, teachers can encourage students to recognize that diagrams continue to be essential tools in highlighting structure and fostering deeper understanding.

Example 2.10. Examples of cooperative learning strategies

Recommendation 3

Teach students to intentionally choose from alternative algebraic strategies when solving problems.

A strategy involves a general approach for accomplishing a task or solving a problem. Unlike an algorithm, which contains a sequence of steps that are intended to be executed in a particular order, a strategy may require students to make choices based on the specifics of the problem as well as their problem-solving goals. A strategy might also include alternative approaches that consider variations of a problem or unexpected results a student might encounter while implementing the steps of the solution. Strategies are general and broadly applicable, making them useful in solving a variety of problems.

By learning from and having access to multiple algebraic strategies, students learn to approach algebra problems with flexibility, recognizing when to apply specific strategies, how to execute different solution strategies correctly, and which strategies are more appropriate for particular tasks. This can help students develop beyond the memorization of one approach, allowing them to extend their knowledge and think more abstractly.

Comparing correct solution strategies can help deepen students’ conceptual understanding and allow students to notice similarities and differences between problem structures and solution strategies. Having students compare strategies may be more conducive to learning than having students study individual strategies, because comparison activities enable students to reference their prior knowledge of one strategy to learn new strategies.

While this recommendation promotes the understanding and use of multiple solution strategies, the recommendation does not advocate that students be fluent in all possible strategies for solving a given problem type. By learning alternative strategies, students can select from different options when they encounter a familiar or unfamiliar problem.

Summary of evidence: Moderate Evidence

Six studies examined the effects of teaching alternative algebraic strategies and met WWC group design standards without reservations (see Appendix D).30 Four studies showed positive effects^31 of teaching alternative algebraic strategies in at least one of the three outcome domains (procedural knowledge, conceptual knowledge, and procedural flexibility),32 and two studies found negative or mixed effects.33 Three of the studies with positive effects examined the effects of studying different solutions—presented side by side—to the same solved problem, compared to students that studied solved problems with methods presented on different pages, or to students that studied two different solved problems solved using the same solution method.34 The fourth study found positive effects of asking students to solve a problem one way and then solve it a second time using a different method or ordering of steps (compared to students asked to just solve a problem one way).35 The two studies with mixed or negative results involved students with no prior knowledge of algebra, and they compared the use of multiple strategies to the use of just one strategy to solve a problem.36 One study found negative effects on all three outcome domains for students with no knowledge of algebra, but found no significant effects in any domain for students who had attempted some algebra on the pretest.37 The other study found a negative effect on procedural knowledge but a positive effect on procedural flexibility.38 These findings indicate that teaching alternative algebraic strategies can improve achievement, especially procedural flexibility, once students have developed some procedural knowledge of algebra.

How to carry out this recommendation

1. Teach students to recognize and generate strategies for solving problems.

Provide students with examples that illustrate the use of multiple algebraic strategies. Include standard strategies that students commonly use, as well as alternative strategies that may be less obvious. Students can observe that strategies vary in their effectiveness and efficiency for solving a problem. Example 3.1 illustrates conventional and alternative solution strategies for several different solved problems. In the linear equations in this example, conventions might be to distribute for multiplication over addition or subtraction before applying properties of equality or to work with linear terms before working with constants.

Solved problems can demonstrate how the same problem could be solved with different strategies (Example 3.2) and how different problems could be solved with the same strategy (Example 3.3). These kinds of examples emphasize that strategies can be used flexibly; that is, students are not limited to using one strategy for a particular problem type, and similarly, problems that appear different on the surface may in some cases be solved using the same strategy. Label, compare, and provide mathematical justification for each solution step in these solved problems to illustrate how the strategies differ. Have students articulate the rationale behind a strategy to identify misconceptions and to ensure they understand that a strategy is more than a series of steps to memorize. Students also can discuss their ideas for alternative solution strategies with a partner. Through facilitated whole-class discussion, students can articulate how different strategies can be used for the same problem, and whether certain strategies are appropriate or effective in solving a problem.

Example 3.1. Alternative and conventional solution strategies^39

Example 3.2. The same problem solved using two different solution strategies^40

Example 3.3. Two different problems, each solved with the same strategy

To avoid overwhelming students, introduce one or two different solution strategies at a time. Show the class work from past or current students to demonstrate examples of students using multiple strategies. Students who are struggling can be overwhelmed by the rapid introduction of multiple strategies. Before introducing a new strategy, provide sufficient time to practice the strategies they already know and ensure students understand when to use each strategy.

After students are introduced to different strategies, help them develop skills for selecting which strategy to use. Present a problem during whole-class instruction, and ask students to write down questions they might ask themselves to determine appropriate solution strategies. Alternatively, provide a list of questions that students may ask themselves (Example 3.4). Students can then draw on this list of reflective questions as they work independently or in small groups. Teachers can also provide students with model answers to the questions in Example 3.4. First, teachers can present students with a problem and corresponding answers to each of the questions as they relate to the problem. Once students have examined these answers, teachers can present a new problem and ask students to go through the same exercise to answer the questions from Example 3.4 on their own.

After students find a solution to a problem, challenge them to solve the problem in another way and to generate additional strategies during group work and independent practice. Use tables similar to those in Examples 3.5 and 3.6 to demonstrate a few strategies that may be useful for solving quadratic equations and linear equations, respectively. Present alternative strategies to students after they become comfortable with a few standard strategies. Emphasize that not all strategies are appropriate for all problems; different strategies have advantages and disadvantages, depending on the problem. If students run into roadblocks or challenges when solving a problem using one strategy, encourage students to try a different strategy.

Example 3.4. Reflective questions for selecting and considering solution strategies

Example 3.5. Some possible strategies for solving quadratic equations

Example 3.6. Some possible strategies for solving linear systems^41

2. Encourage students to articulate the reasoning behind their choice of strategy and the mathematical validity of their strategy when solving problems.

Have students describe their reasoning while analyzing the problem structure, determining their solution strategy, solving a problem, and analyzing another student’s solution. Describing their reasoning helps students understand the choices they make and goals they set when selecting a strategy. Students should communicate their reasoning verbally and through written work. Ask students working independently to write out their strategic reasoning in addition to solving the problem. Ask students working in pairs or small groups to discuss their solution strategies and explain their mathematical reasoning for why a strategy works (i.e., its mathematical validity). Provide a list of prompts, such as those in Example 3.7, to accompany practice problems or initiate small-group discussion to encourage students to articulate their reasoning.

When initially introducing group activities, model how to work with a partner to discuss potential strategies, how to label the steps of each strategy, and how to explain the similarities and differences observed between strategies. (Example 3.8 presents a sample dialogue discussing Devon and Elena’s solution strategies from Example 3.2.) After students become more accustomed to these tasks, they can work to complete them independently.

Example 3.7. Prompts to encourage students to articulate their reasoning

Example 3.8. Sample student dialogue discussing two different solution strategies^42

3. Have students evaluate and compare different strategies for solving problems.

Encourage students to compare problem structures and solution strategies to discover the relationships among similar and different problems, strategies, and solutions. Begin comparison activities after students understand one strategy, so that students are able to identify similarities and differences between the familiar strategy and the newly learned strategy. Teachers can incorporate prompts such those presented in Examples 3.1, 3.3, and 3.7 into whole-class discussion or independent practice activities that encourage students to explain why they might choose one solution strategy over another. Comparison activities can be carried into small-group discussions, as illustrated in Example 3.9.

Use solved problems showing two strategies side by side to enable students to see the number, type, and sequence of solution steps. This will allow them to compare solution strategies and consider the accuracy, efficiency, and applicability of various combinations of solution steps. When presenting pairs of solved problems to communicate a particular instructional goal to students, use solved problems that are moderately similar to each other (as opposed to highly different from each other) to help students look past the surface features of the problems and instead focus on the underlying solution structure. By visually aligning the steps of different solution strategies, students can make connections among strategies and analyze their similarities and differences. Teachers can initially provide reasoning for the steps of solution strategies and later ask students to label the steps of the strategies they use.

Example 3.9. Small-group comparison and discussion activity

Potential roadblocks and suggested approaches

Roadblock 3.1. I’m worried about confusing my students by teaching them multiple strategies for solving a problem. They have a hard enough time learning one strategy! Isn’t it easier for them to master one strategy for solving algebra problems?

Suggested Approach. The goal in teaching alternative strategies is not to have students master all strategies, but rather to have students compare them in order to develop a deeper understanding of the strategy they choose to use. With this understanding, they can use their selected strategy flexibly to solve problems they have not seen before. Multiple strategies also do not need to be taught all at once. Teachers may introduce one strategy and give students time to become comfortable with it before introducing alternative strategies in a structured way to emphasize the comparison of strategies to develop student understanding.

Although one student may grasp a particular strategy very well, another student may find that strategy confusing or unclear. Seeing alternative strategies provides students with more than one way to think about solutions to algebra problems, moving beyond rote memorization and toward a deeper conceptual understanding of the problem, the thought process required to solve the problem, and the reasons why particular solution strategies work. Note that this recommendation is not advocating that students be fluent in all possible strategies for solving a given problem type. However, seeing alternative strategies provides students with different options that they can choose from when they encounter a familiar or unfamiliar problem, and the flexibility in choice can lead to greater success, confidence, and knowledge in algebra.

Students naturally encounter new strategies as their classmates offer ideas about how to solve problems. Including alternative strategies in the classroom conversation allows students to see ways to evaluate and use the alternatives they encounter. Seeing several solution strategies can help students who are struggling in algebra choose a strategy they feel comfortable with, understand, and may be able to better remember, as opposed to relying on a single strategy that has been memorized by rote and may be easily forgotten.

Roadblock 3.2. My special-education students need a very structured process for solving algebra problems. Introducing multiple strategies and asking students to choose among strategies might be hard on them.

Suggested Approach. Special-education students may require explicit instruction in how to solve problems, but it is important to distinguish between providing explicit instruction and teaching only a single solution strategy and asking students to memorize the steps of that strategy. Students are better served if they come to view mathematics not as a game like “Memory” in which they associate a problem with a specific method. This view of mathematics places even greater demands on students’ memory—given the number of problem types and methods to be memorized—and thus presents particular challenges for students in special education. Teachers can help special-education students understand alternative strategies by being explicit not only about the steps of a strategy but also about its underlying rationale, including how, when, and why it is applicable or useful for particular problems.

Roadblock 3.3. I can’t seem to teach my students multiple strategies without them becoming confused. I presented and compared five algebra strategies for solving quadratic equations during one class period, and my students didn’t understand. What should I do?

Suggested Approach. Comparing several strategies at once can overwhelm students who are trying to learn many new concepts at one time. Teachers can begin simply—by introducing one strategy to students. After students are comfortable with this strategy, teachers may compare that strategy with one or two other new strategies for the purpose of deepening student understanding of the underlying mathematics. Additional strategies may be introduced if they are useful as students are ready. Often just one alternative strategy will help students acquire a strategic approach to problem solving. Reviewing multiple strategies at once may be useful prior to assessments to remind students of the many solution strategies available to them.

Roadblock 3.4. Teaching students to use and compare multiple strategies requires knowledge of many strategies, and our textbook presents only one strategy.

Suggested Approach. Examples 3.5 and 3.6 list some of the multiple solution strategies available for solving quadratic equations and linear equations. Devoting time to discussing multiple strategies in professional learning communities enables and supports teachers in deepening their own understanding of strategies. Teachers may also learn new strategies from students, who frequently develop their own strategies. Teachers may choose to post different strategies in the classroom or create handouts for students to reference at their desks.

Roadblock 3.5. How can I stay on schedule teaching everything required to meet state standards and still have time to teach students to use multiple strategies?

Suggested Approach. Teaching multiple strategies is not about adding additional content to teachers’ lesson plans or making time for students to work on each strategy repeatedly. Instead, the focus is on teaching students how to think about an algebra problem and recognize when an alternative strategy may be appropriate. Encouraging these types of skills can happen within the existing framework of a teacher’s lesson plan, instead of placing any additional burden on teachers.

Teaching students alternative strategies creates a strong foundation of reasoning skills as students learn to select appropriate solution methods based on the problems they encounter. Teaching students multiple strategies equips them with an array of tools to use to master state standards.

Glossary

A

Abstract reasoning is processing and analyzing complex, non-concrete concepts.

An algebraic expression is a symbol or combination of symbols for variables, numbers, and arithmetic operations used to represent a quantity. Examples of algebraic expressions are

9 – and 3x – 4y + 7.

An algebraic representation is a way to display an algebraic relationship or quantity. Algebraic representations can have symbolic, numeric, verbal, or graphic forms.

An algorithm is a series of steps that, when executed in the prescribed order, lead to the desired or expected outcome.

E

Strategies can be evaluated on the basis of their efficiency, when a more efficient strategy is one that can be executed relatively easily, quickly, and/or without error, as compared to another strategy that may be more difficult or more tedious to implement and/or that may be more likely to lead to error. Determining which strategy is the most efficient for solving a problem may depend on the problem’s structure, the features of the strategy, and the knowledge and goals of the person implementing the strategy.

Evidence-based practices, policies, or recommendations are those that are supported by studies that meet WWC design standards with or without reservations.

F

Flexibility includes the knowledge of a range of strategies for solving a given problem and the ability to select the most appropriate strategy for that problem. An appropriate strategy may be the most efficient for the problem and/or the one that is the best match for the problem’s features and structure.

I

An incomplete solved problem for instructional purposes is a solved problem or worked example that shows both the problem and some of the steps taken to solve the problem. One or more steps are left blank for students to complete.

An incorrect solved problem for instructional purposes is a solved problem or worked example that shows both the problem and the steps taken to try to solve the problem, with at least one error in the steps, so that a correct outcome is not reached. An error could arise from using the wrong strategy to solve the problem, using improper mathematical reasoning, or making a computational or strategic error.

M

Mathematical validity is a characteristic of a solution or the mathematical reasoning that logically flows from valid assumptions; essentially, it is why a strategy works.

P

Precise mathematical language can convey the underlying logical, accurate meaning of algebraic structure, operations, solution steps, and strategies.

A procedural or computational error is an error in applying a procedure and can occur when a step of a procedure is not executed correctly, such as combining like terms incorrectly or incorrectly applying the order of operations.

R

Reflective questioning is a process in which students ask themselves relevant questions as they work to solve a problem.

S

The solution set to an equation is the set of values for which the equation is true. The solution steps illustrate the process used to reach the solution.

A solved problem for instructional purposes is an example that shows both the problem and the steps used to reach the problem’s solution. A solved problem is also referred to as a “worked example.”

A strategic error is a mistake that occurs because of incorrect reasoning, a misunderstanding of mathematical structure, or an incorrect choice of strategy.

A strategy comprises the overarching choices made and steps taken in attempting to solve a problem. A strategy may include sequences of steps to be executed, as well as the rationale behind the use and effectiveness of these steps.

Structure refers to the underlying mathematical features and relationships of an expression, representation, or equation. Structure includes quantities, variables, arithmetic operations, and relationships among these things (including equality and inequality). For example, “the difference of two squares” is one way to describe the underlying structure of – 9.

T

Think-aloud questions are questions that guide students to verbalize their internal thinking. They can be used to help students analyze solved problems.

Appendix A

Postscript from the Institute of Education Sciences

What is a practice guide?

The Institute of Education Sciences (IES) publishes practice guides to share evidence and expert guidance on addressing education-related challenges not readily solved with a single program, policy, or practice. Each practice guide’s panel of experts develops recommendations for a coherent approach to a multifaceted problem. Each recommendation is explicitly connected to supporting evidence. Using What Works Clearinghouse (WWC) design standards, the supporting evidence is rated to reflect how well the research demonstrates the effectiveness of the recommended practices. Strong evidence means positive findings are demonstrated in multiple well-designed, well-executed studies, leaving little or no doubt that the positive effects are caused by the recommended practice. Moderate evidence means well-designed studies show positive impacts, but there are questions about whether the findings can be generalized beyond the study samples or whether the studies definitively show evidence that the practice is effective. Minimal evidence means that there is not definitive evidence that the recommended practice is effective in improving the outcome of interest, although there may be data to suggest a correlation between the practice and the outcome of interest. (See Table A.1 for more details on levels of evidence.)

How are practice guides developed?

To produce a practice guide, IES first selects a topic. Topic selection is informed by inquiries and requests to the WWC Help Desk, a limited literature search, and evaluation of the topic’s evidence base. Next, IES recruits a panel chair who has a national reputation and expertise in the topic. The chair, working with IES and WWC staff, then selects panelists to help develop the guide. Panelists are selected based on their expertise in the topic area and the belief that they can work together to develop relevant, evidence-based recommendations. Panels include two practitioners with expertise in the topic.

Relevant studies are identified through panel recommendations and a systematic literature search. These studies are then reviewed using the WWC design standards by certified reviewers who rate each effectiveness study. The panel synthesizes the evidence into recommendations. WWC staff summarize the research and help draft the practice guide.

IES practice guides are then subjected to external peer review. This review is done independently of the IES staff that supported the development of the guide. A critical task of the peer reviewers of a practice guide is to determine whether the evidence cited in support of particular recommendations is up-to-date and that studies of similar or better quality that point in a different direction have not been overlooked. Peer reviewers also evaluate whether the level of evidence category assigned to each recommendation is appropriate. After the review, a practice guide is revised to meet any concerns of the reviewers and to gain the approval of the standards and review staff at IES.

Institute of Education Sciences levels of evidence for What Works Clearinghouse practice guides

This section provides information about the role of evidence in IES’s WWC practice guides. It describes how practice guide panels determine the level of evidence for each recommendation and explains the criteria for each of the three levels of evidence (strong evidence, moderate evidence, and minimal evidence).

The level of evidence assigned to each recommendation in this practice guide represents the panel’s judgment of the quality of the existing research to support a claim that, when these practices were implemented in past research, positive effects were observed on student outcomes. After careful review of the studies supporting each recommendation, panelists determine the level of evidence for each recommendation using the criteria in Table A.1. The panel first considers the relevance of individual studies to the recommendation and then discusses the entire evidence base, taking the following into consideration:

• the number of studies

• the study designs

• the internal validity of the studies

• whether the studies represent the range of participants and settings on which the recommendation is focused

• whether findings from the studies can be attributed to the recommended practice

• whether findings in the studies are consistently positive

A rating of strong evidence refers to consistent evidence that the recommended strategies, programs, or practices improve student outcomes for a diverse population of students.43 In other words, there is strong causal and generalizable evidence.

A rating of moderate evidence refers either to evidence from studies that allow strong causal conclusions but cannot be generalized with assurance to the population on which a recommendation is focused (perhaps because the findings have not been widely replicated) or to evidence from studies that are generalizable but have some causal ambiguity. It also might be that the studies that exist do not specifically examine the outcomes of interest in the practice guide, although the studies may be related to the recommendation.

A rating of minimal evidence suggests that the panel cannot point to a body of evidence that demonstrates the practice’s positive effect on student achievement. In some cases, this simply means that the recommended practices would be difficult to study in a rigorous, experimental fashion;44 in other cases, it means that researchers have not yet studied this practice, or that there is weak or conflicting evidence of effectiveness. A minimal evidence rating does not indicate that the recommendation is any less important than other recommendations with a strong or moderate evidence rating.

In developing the levels of evidence, the panel considers each of the criteria in Table A.1. The level of evidence rating is determined by the lowest rating achieved for any individual criterion. Thus, for a recommendation to get a strong rating, the research must be rated as strong on each criterion. If at least one criterion receives a rating of moderate and none receives a rating of minimal, then the level of evidence is determined to be moderate. If one or more criteria receive a rating of minimal, then the level of evidence is determined to be minimal.

The panel relied on WWC design standards to assess the quality of evidence supporting education programs and practices. The WWC evaluates evidence for the causal validity of instructional programs and practices according to WWC design standards. Information about these standards is available at http://whatworks.ed.gov. Eligible studies that meet WWC designs standards without reservations or meet WWC design standards with reservations are indicated by bold text in the endnotes and references pages.

A final note about IES practice guides

In policy and other arenas, expert panels typically try to build a consensus, forging statements that all its members endorse. Practice guides do more than find common ground; they create a list of actionable recommendations. Where research clearly shows which practices are effective, the panelists use this evidence to guide their recommendations. However, in some cases research does not provide a clear indication of what works. In these cases, the panelists’ interpretation of the existing (but incomplete) research plays an important role in guiding the recommendations. As a result, it is possible that two teams of recognized experts working independently to produce a practice guide on the same topic would come to very different conclusions. Those who use the guides should recognize that the recommendations represent, in effect, the advice of consultants. However, the advice might be better than what a school or district could obtain on its own. Practice guide authors are nationally recognized experts who collectively endorse the recommendations, justify their choices with supporting evidence, and face rigorous independent peer review of their conclusions. Schools and districts would likely not find such a comprehensive approach when seeking the advice of individual consultants.

Institute of Education Sciences

Table A.1. Institute of Education Sciences levels of evidence for What Works Clearinghouse practice guides

Appendix B

About the Authors

Panelists

Jon R. Star, Ph.D., (chair) is the Nancy Pforzheimer Aronson Associate Professor in Human Development and Education in the Graduate School of Education at Harvard University. A former middle school and high school mathematics teacher, his current research focuses on the learning and teaching of mathematics, particularly algebra. Dr. Star’s work has been supported by several grants from the Institute of Education Sciences (IES) and from the National Science Foundation. He is the author of more than 50 peer-reviewed articles, many of which have been published in leading journals in mathematics education and educational psychology. In addition, Dr. Star has served as an author on two prior IES practice guides (Encouraging Girls in Math and Science and Assisting Students Struggling with Mathematics: Response to Intervention (RtI) for Elementary and Middle Schools). He currently serves as an editor at the Elementary School Journal and Cognition and Instruction. Dr. Star is also an experienced teacher of in-service and pre-service mathematics teachers.

Anne Foegen, Ph.D., is a professor of special education and the Director of Graduate Education in the School of Education at Iowa State University. Dr. Foegen’s research has explored the development and implementation of progress-monitoring assessments in mathematics for students with, or at risk for, disabilities. Her most recent work has focused on measures to reflect student progress during initial algebra instruction at the middle school and high school levels. Dr. Foegen has also conducted research on the development and use of mathematics progress-monitoring measures for the primary grades and for middle school mathematics. She has been principal investigator on three U.S. Department of Education grants and sub-award principal investigator on a fourth, all of which have supported her work in mathematics progress monitoring.

Matthew R. Larson, Ph.D., is the K–12 mathematics curriculum specialist for the Lincoln Public Schools in Lincoln, NE. He has served on the board of directors for the National Council of Teachers of Mathematics (NCTM), as past chair of NCTM’s research committee, and as a member of NCTM’s task force on linking research and practice. He is the author or coauthor of numerous journal articles, professional books, and K–12 mathematics programs. He has taught mathematics at the secondary and college levels.

William G. McCallum, Ph.D., is a University Distinguished Professor of Mathematics at the University of Arizona. He received his Ph.D. in mathematics from Harvard University in 1984, under the supervision of Barry Mazur. After spending two years at the University of California, Berkeley, and one at the Mathematical Sciences Research Institute in Berkeley, he joined the faculty at the University of Arizona in 1987. In 1989, he joined the Harvard calculus consortium, and he is the lead author of the consortium’s multivariable calculus and college algebra texts. In 1993/94, he spent a year at the Institut des Hautes Études Scientifiques, and in 1995/96 he spent a year at the Institute for Advanced Study on a Centennial Fellowship from the American Mathematical Society. He received the Director’s Award for Distinguished Teaching Scholars from the National Science Foundation in 2005. In 2006, he founded the Institute for Mathematics and Education at the University of Arizona, and he is currently its director. He was one of the lead writers for the Common Core State Standards in Mathematics. His professional interests include arithmetical algebraic geometry and mathematics education. He has received grants and written articles, essays, and books in both areas.

Jane Porath (Peterson) has been teaching secondary mathematics for 18 years. She is currently an 8th-grade mathematics teacher at Traverse City East Middle School in Traverse City, Michigan. In 2013, Jane was selected as a state finalist for the Presidential Award of Excellence of Mathematics and Science Teaching. In 2012, Jane was named the Community Champion of Traverse City for bringing the Annual Conference of the Michigan Council of Teachers of Mathematics (MCTM) to the region. She served on the executive board for the MCTM and currently serves as its publications director. She has been involved professionally with the Connected Mathematics Project 3 (CMP3) for most of her career. She has piloted the curriculum, worked on the development of the assessments for CMP3, and provided professional development around the country to teachers implementing the curriculum. Through this work, she has focused extensively on pedagogy and helping students make sense of mathematics. Jane was also an adjunct faculty member for Central Michigan University, teaching physical science to pre-service elementary teachers in Traverse City. Jane is involved professionally with the National Council of Teachers of Mathematics (NCTM). She is currently serving on the board of directors and the program committee for the 2013 NCTM Louisville Kentucky Regional Conference. Jane also served on the editorial panel for Mathematics Teaching in the Middle School as a member and chair from 2008 to 2010. Jane authored a book chapter in A Decade of Middle School Mathematics Curriculum Implementation: Lessons Learned from the Show-Me Project in 2007.

Rose Mary Zbiek, Ph.D., is a professor of mathematics education at the Pennsylvania State University and is the Director of Graduate Studies in Curriculum and Instruction. Dr. Zbiek’s research has focused on the use of technology in mathematics learning and teaching, relationships between teacher knowledge and classroom practice, and the roles of mathematical modeling and technology in secondary school curricula in general and algebra curricula in particular. Her work includes the development and evaluation of mathematics curriculum materials that capitalize on the use of technology and engage students in mathematical modeling for the learning of curricular mathematics. Her current research investigates relationships among mathematical processes, technological expertise, and classroom practice of beginning and experienced secondary school mathematics teachers. Dr. Zbiek has served as principal investigator or co–principal investigator for grants from the National Science Foundation and corporate foundations and has received recognition for classroom-based research and the integration of research, teaching,
and service. In addition to having been a member of editorial teams for both research and practice publications, she is the series editor of the National Council of Teachers of Mathematics 16-volume Essential Understanding series and is a past associate editor of the Journal for Research in Mathematics Education.

Staff

Pia Caronongan, Ed.D., is a researcher at Mathematica Policy Research. She oversaw efforts to assess the rigor and relevance of studies related to the panel’s recommendations. Outside of the What Works Clearinghouse, Dr. Caronongan focuses on early childhood programs and policies and has played key roles on studies of child-care quality rating systems, home-visiting programs, universal preschool, and Early Head Start, among others.

Joshua Furgeson, Ph.D., is a senior researcher at Mathematica Policy Research and a former high school teacher. He is the lead methodologist for the special education topic areas in the What Works Clearinghouse, and he leads the practice guides under Mathematica’s contract. He oversaw efforts to assess the rigor and relevance of studies related to the panel’s recommendations for the Improving Mathematical Problem Solving in Grades 4 Through 8 practice guide.

Betsy Keating, M.P.P., is a research analyst at Mathematica Policy Research. She helped coordinate the review of studies related to the panel’s recommendations and helped draft the text of the practice guide. Ms. Keating is certified as a What Works Clearinghouse reviewer. She has worked on various other projects focused on education, including an implementation study of district-charter collaboration grants and a pilot study of teacher-effectiveness ratings in Pennsylvania.

Julia Lyskawa, M.P.P., is a research analyst at Mathematica Policy Research. She led the panel’s efforts to translate research findings into practitioner-friendly text. Ms. Lyskawa also contributed to the Teaching Math to Young Children practice guide and summary. She is certified as a What Works Clearinghouse reviewer. She has worked on many other education-focused projects, including an evaluation of child outcomes and coaching methods in the Los Angeles Universal Preschool Program, a feasibility study of extending learning time in elementary schools, and a redesign of the Head Start Family and Child Experiences Survey.

Appendix C

Disclosure of Potential Conflicts of Interest

Practice guide panels are composed of individuals who are nationally recognized experts on the topics about which they are making recommendations. IES expects the experts to be involved professionally in a variety of matters that relate to their work as a panel. Panel members are asked to disclose these professional activities and institute deliberative processes that encourage critical examination of their views as they relate to the content of the practice guide. The potential influence of the panel members’ professional activities is further muted by the requirement that they ground their recommendations in evidence that is documented in the practice guide. In addition, before all practice guides are published, they undergo an independent external peer review focusing on whether the evidence related to the recommendations in the guide has been presented appropriately.

The professional activities reported by each panel member that appear to be most closely associated with the panel recommendations are noted below.

Anne Foegen is the principal investigator on two current IES-funded research projects that involve the development and implementation of measures to monitor student progress in algebra. She provides in-person training for teachers to learn how to administer, score, and interpret the progress monitoring measures and data that are produced.

Matthew R. Larson is a K–12 textbook author with the Houghton Mifflin Harcourt Publishing Company based in Boston, MA. This includes coauthorship of a first- and second-year algebra textbook. He receives royalties and consulting compensation from this relationship.

Jane Porath provides professional development consulting services for Michigan State University related to the Connected Mathematics Project.

Jon R. Star is currently a paid consultant for both Scholastic and Pearson, two publishers who are developing algebra curricula that are intended to be aligned with the Common Core State Standards.

Appendix D

Rationale for Evidence Ratings^48

The level of evidence rating is based on the findings of studies that examined the effectiveness of recommended practices and meet What Works Clearinghouse (WWC) design standards.49 The studies were primarily identified through a keyword search of several databases. The search was limited to studies published between 1993 and 2013 that examined practices for teaching algebra to students in grades 6–12. The search was supplemented with studies recommended by the panel.

The search identified more than 2,800 studies, including 30 eligible group-design studies reviewed using WWC group design standards and four single-case design studies reviewed according to WWC pilot single-case design standards.50 No single-case design studies met WWC single-case design standards. Fifteen group-design studies met evidence standards with or without reservations and tested interventions related to one or more recommendations. Study effects were calculated and classified as having a positive or negative effect when the result was either

• statistically significant 51 or

• substantively important as defined by the WWC^52

Some studies met WWC group design standards but did not adjust statistical significance when there were multiple significance tests or when the unit of assignment was different from the unit of analysis (“clustering,” for example, when classrooms are assigned to conditions but individual children’s test scores are analyzed). In these cases, the WWC adjusted for clustering and multiple tests within a domain.53

The text and tables in this appendix focus on total or full-scale scores on the outcome closest to the end of the intervention; these are labeled posttests. All outcome measures administered after the posttest are described in table notes.

The review team for each study classified each outcome into one of three domains, consulting with the panel chair as needed to determine the proper domain. Table D.1 provides definitions and representative sample items from the reviewed studies for each outcome domain.

Table D.1. Outcome domain definitions and sample items

Conceptual knowledge, procedural knowledge, and procedural flexibility are distinct competencies.59 Mathematical proficiency results when children develop these and other competencies and form links between them. This guide groups outcomes into these three domains because, according to the National Mathematics Advisory Panel definition of proficiency, each domain represents an important ability or understanding.60

Though theoretically distinct, these competencies are often difficult to measure empirically. Even when outcomes in a study are classified into one domain, the outcomes might also measure broader knowledge or competencies beyond that domain. Some measures of conceptual and procedural knowledge adequately assess the specific competency, but studies have raised concerns about the validity and independence of several measures. For example, studies have found that conceptual and procedural knowledge measures may overlap, assess additional competencies, and can fail to adequately distinguish between the different types of knowledge.61

When studies have multiple posttest outcome measures administered within the same domain, effect sizes for each measure are averaged, and the overall average is reported in the tables. Findings for individual outcomes within each domain are described in table notes.

Recommendation 1. Use solved problems to engage students in analyzing algebraic reasoning and strategies.

Level of evidence: Minimal Evidence

WWC staff and the panel assigned a minimal level of evidence based on four studies that meet WWC group design standards without reservations and examined this practice (see Table D.2).62 Although the studies collectively demonstrated strong internal validity, the level of evidence was rated as minimal because of limited generalizability. All studies finding a positive impact compare solved problems to additional practice problems; in the only study comparing solved problems to another intervention (meta-cognition, a practice listed in Recommendation 2), the authors find negative impacts.

Three of the four studies found positive effects of using solved problems, providing a preponderance of evidence of positive effects on conceptual knowledge.63 To measure conceptual knowledge, these studies asked students to translate word problems into algebra equations or assessed whether participants could understand how the terms and operations in an expression can be written differently and still be equivalent (the latter measures understanding of the meaning of the expression features). One of these studies also examined effects on procedural knowledge (measured by whether students solved equations or algebra word problems correctly), and found neither positive nor negative effects.64 No studies examined outcomes in the procedural flexibility domain. Each of the three studies was a randomized controlled trial with low sample attrition, resulting in high internal validity. The three studies finding positive effects examined the effect of solved problems without other intervention components, providing a direct test of the recommendation.

However, each of the three studies compared providing solved problems to having students solve practice problems, limiting generalization of the findings. Moreover, none of the interventions involved regular or sustained instruction with solved problems: two of the studies involved interventions that lasted for only one day or used one worksheet,65 and the remaining study intervention involved eight solved problems.66 Two studies compared students who independently studied correct solved problems alongside practice problems to students who were not provided with any solved problems and were instead provided with more practice problems.67

One study found that providing students with correct and incorrect solved problems and prompting them to describe the solution steps and explain why they were carried out had positive effects compared to students who were prompted similarly as they solved practice problems.68 The interventions ranged in duration from one day to a month. In one of the studies, students completed the unit at their own pace, so the duration of the intervention varied by the individual (for example, most students completed the intervention within eight class sessions over a four-week period).69

The three studies included students of varying ability and were conducted in high schools across the United States (including the Midwest, West Coast and Mid-Atlantic regions). One of the studies included students in regular Algebra I classes;70 one study included students from basic, regular, and honors algebra classes;71 and one study included high school students enrolled in a remedial mathematics class with participants aged 14 to 17.72 One study took place in a computer lab setting, with the intervention delivered via computer software.73 The other two studies took place in regular classrooms, and involved worksheets.74 For all three studies, the intervention occurred during scheduled algebra classes within the regular school day. The studies as a whole provide moderate external and ecological validity.

One randomized controlled trial that meets WWC group design standards without reservations found that solved problems had negative effects on conceptual and procedural knowledge.75 The study compared students who studied and discussed solved problems in groups to students who used reflective questioning (a practice suggested in Recommendation 2) to solve practice problems in groups.76 This negative finding indicates that the effectiveness of this practice can depend on what students do instead of using solved problems.

Table D.2. Studies providing evidence for Recommendation 1

Table D.2. Studies providing evidence for Recommendation 1 (continued)

All studies in this table meet WWC group design standards without reservations. Studies are listed alphabetically by last name of the first author.

Each row in this table represents a study, defined by the WWC as an examination of the effect of an intervention on a distinct sample. In some cases, multiple studies were described in a single article.

For studies that included multiple outcomes in a domain, reported effect sizes and statistical significance are for the domain and calculated as described in the WWC Procedures and Standards Handbook 3.0 (pp. 28–29).

* = statistically significant at 0.05 level

a This row summarizes Booth et al. (2013) Experiment 1. Results from Experiment 2 are not reported because all of the participants in that study were exposed to solved problems.

b The study examined two outcomes in this domain: isomorphic procedural knowledge and transfer procedural knowledge. For isomorphic procedural knowledge, the WWC-calculated effect size is 0.03 and the effect is not statistically significant. For transfer procedural knowledge, the WWC-calculated effect size is 0.31 and the effect is not statistically significant.

c This row summarizes Carroll (1994) Experiment 1.

d This study grouped participants into higher and lower achievers based on scores on a placement test created by the researcher. The effects of the intervention did not vary by student achievement level.

e The study also reports finding for a delayed posttest, administered one day after the completion of the intervention. However, there was an unknown amount of attrition and the authors did not demonstrate baseline equivalence of the analytic samples.

f This row summarizes Carroll (1994) Experiment 2.

g The outcome measure in this study was based on the number of errors. The intervention group committed fewer errors and thus had lower scores. Because fewer errors indicate a more positive impact of the intervention, we have reversed the sign on the effect size.

h The intervention of interest in this study was centered on reflective questions. However, because this table summarizes evidence about the effects of solved problems, we present that as the intervention condition.

i A delayed posttest (n = 122) was also administered one year after the intervention. The WWC-calculated effect size is –0.15 and the effect is not statistically significant.

j The study examined two posttest outcomes in this domain: an overall test score and knowledge of algebraic solution. For the overall test score, the WWC-calculated effect size is –0.34 and the effect is not statistically significant. For knowledge of algebraic solution, the WWC-calculated effect size is –0.59 and the effect is not statistically significant. A delayed posttest (n = 122) was also administered to assess the two outcomes one year after the intervention. For the overall test score, the WWC-calculated effect size is –0.32 and the effect is not statistically significant. For knowledge of algebraic solution, the WWC-calculated effect size is –0.15 and the effect is not statistically significant.

Recommendation 2. Teach students to utilize the structure of algebraic representations.

Level of evidence: Minimal Evidence

WWC staff and the panel assigned a minimal level of evidence based on four studies that meet WWC group design standards without reservations and examined this practice (see Table D.3).77 In addition, two studies do not meet WWC group design standards but contributed to the level of evidence because the analyses adequately controlled for selection bias.78 The six studies examined different methods of instruction on the structure of algebraic representations, but none of the studies examined the use of language that reflects mathematical structure (a component of Recommendation 2), resulting in a minimal level of evidence rating.

Four studies showed positive effects of instruction on the structure of algebraic representations for outcomes in the procedural knowledge domain.79 In these studies, procedural knowledge was measured by asking students to solve equations or word problems. Three studies found a positive effect on conceptual knowledge.80 To measure conceptual knowledge, one study asked a series of open-ended questions asking students to justify their procedures for solving systems of equations.81 The other two studies asked students to formulate equations based on information provided in problem text.82

Four of the studies had high internal validity because they were randomized controlled trials with low sample attrition.83 The fifth study was a randomized controlled trial with high sample attrition, and the sixth study used a quasi-experimental design; although the analytic samples for these studies were not shown to be equivalent at baseline, the analyses included a pretest covariate as a statistical control for selection bias.84

Each study conducted a direct test of a component of the recommendation. Although none of the studies evaluated the effectiveness of instruction that intentionally taught students to utilize structure, the panel determined that the interventions studied are effective because they encourage students to notice, discuss, and analyze structure.

Two studies compared students who used reflective questioning to solve practice problems in small groups to students who studied solved problems in small groups or students who received regular instruction without any explicit instruction on reflective questioning.85 Students were taught to ask themselves questions about their comprehension of the problem, the similarities and differences between each problem and previous problems they had solved, and strategies for organizing the information provided and solving the problem.

The other four studies compared students who used graphical representations to solve problems to students who did not use graphical representations or were not encouraged to use any type of representation other than standard algebraic equations. One study used computer-based animation to transform elements of word problems into concrete objects and then mathematical symbols.86 For example, a problem asked about the rates of damage to two types of trees, and the animation

• displayed a set of trees with the damaged trees disappearing from view,

• morphed into an abstract representation of the rates of damage, with the initial set of trees represented by a large square and the damaged trees represented by small squares within the large square, and

• showed the problem as an equation and included all the steps needed to solve the problem.

Two studies taught students to use graphical representations to depict the elements of a word problem and break down solution steps.87 For example, in one study, students used a table with three columns and two rows to solve a system of equations with three variables.88 Students were taught to use the top row of the table to eliminate variables as they moved from one column to the next and to then use the bottom row of the table to substitute variables until they had solved the entire system. The final study taught students to utilize verbal, tabular, graphical, and symbolic representations to describe the same problem.89 For example, students were asked to find errors in a sales invoice; analyze the pattern of errors using a graph, chart, table, or equation; and write a description of the pattern.

The study samples and settings were diverse. Two of the studies were conducted in schools outside of the United States: one included 8th-grade students from Israel,90 and another included 9th-grade students from Germany.91 The other four studies were conducted in the United States: one study included 7th- and 8th-grade students and another examined high school students in Algebra I.92 Participants in the other two studies conducted in the United States were 6th- to 12th-grade students who had learning disabilities or emotional disorders or were at risk for failing mathematics based on teacher reports and test scores.93 The delivery of the intervention also varied. Four of the interventions were delivered by teachers, one was delivered by a combination of doctoral students and experienced teachers,94 and one was computer administered.95 The studies as a whole provide moderate to high external and ecological validity.

Table D.3. Studies providing evidence for Recommendation 2

Table D.3. Studies providing evidence for Recommendation 2 (continued)

Each row in this table represents a study, defined by the WWC as an examination of the effect of an intervention on a distinct sample. In some cases, multiple studies were described in a single article.

For studies that included multiple outcomes in a domain, the reported effect sizes and statistical significance are for the domain and calculated as described in the WWC Procedures and Standards Handbook 3.0 (pp. 28–29).

* = statistically significant at 0.05 level

a This row summarizes Ives (2007) Experiment 1. Experiment 2 does not meet WWC group design standards because only one classroom was assigned to each condition.

b A delayed posttest (n = 30) was also administered two to three weeks after the intervention. The WWC-calculated effect sizes are 0.00 for procedural knowledge and 0.88 for conceptual knowledge.

c This study grouped participants into higher and lower achievers based on pretest scores. The effects of the intervention did not vary by student achievement level.

d A delayed posttest (n = 122) was also administered one year after the intervention. The WWC-calculated effect size is 0.15 and the effect is not statistically significant.

e The study examined two posttest outcomes in this domain: an overall test score and knowledge of algebraic solution. For the overall test score, the WWC-calculated effect size is 0.34 and the effect is not statistically significant. For knowledge of algebraic solutions, the WWC-calculated effect size is 0.59 and the effect is statistically significant. A delayed posttest (n = 122) was also administered to assess the two outcomes one year after the intervention. For the overall test score, the WWC-calculated effect size is 0.32 and the effect is not statistically significant. For knowledge of algebraic solutions, the WWC-calculated effect size is 0.15 and the effect is not statistically significant.

f The study examined three outcomes in this domain: equivalent problems, similar problems, and unrelated problems. For equivalent problems, the WWC-calculated effect size is 0.37 and the effect is not statistically significant. For similar problems, the WWC-calculated effect size is 1.14 and the effect is statistically significant. For unrelated problems, the WWC-calculated effect size is 1.11 and the effect is statistically significant.

g The authors reported an eta-squared effect size statistic, and did not respond to requests for more information. To make this effect size more similar to the standard WWC effect size (Hedge’s g), we: (1) took the square root of eta-squared to obtain the correlation (r) between the treatment and the outcome; (2) used the following equation to estimate Cohen’s d —with the sample sizes in this study, g and d are similar; and (3) multiplied the resulting d by , a small sample correction, to obtain an estimate for the WWC effect size.

h The study examined two outcomes in this domain: functional word problem representation and word problem representation. For functional word problem representation, the WWC-calculated effect size is 0.91 and the effect is statistically significant. For word problem representation, the WWC-calculated effect size is 0.64 and the effect is statistically significant.

i The study examined three outcomes in this domain: functional word problem correct solution, a word problem-solving test, and an equation-solving test. For functional word problem correct solution, the WWC-calculated effect size is 0.41 and the effect is not statistically significant. For the word problem-solving test, the WWC-calculated effect size is 0.11 and the effect is not statistically significant. For the equation solving test, the WWC-calculated effect size is –0.29 and the effect is not statistically significant.

Recommendation 3. Teach students to intentionally choose from alternative algebraic strategies when solving problems.

Level of evidence: Moderate Evidence

WWC staff and the panel assigned a moderate level of evidence based on six studies that meet WWC group design standards without reservations (see Table D.4).96

Four studies showed positive effects of teaching alternative algebraic strategies, providing a preponderance of evidence of positive effects.97 Consistently positive effects were found in the procedural flexibility outcome domain. To measure procedural flexibility, these studies asked students to solve a problem in a different way, to identify appropriate first steps in solving a problem, and to describe whether a first step taken was appropriate and efficient. Findings pertaining to the other outcome domains were mixed. Only one of the four studies found a positive effect on conceptual knowledge,98 with the remaining studies finding neither significant nor substantively important effects on conceptual knowledge. These studies measured conceptual knowledge by asking students to identify equivalent equations—which measured understanding of the meaning of equation features—and to recognize equivalent terms that could be combined. A positive effect on procedural knowledge was also only found in one of the four studies,99 with the remaining studies finding neither significant nor substantively important effects. To measure procedural knowledge, these studies asked students to solve linear equations. The studies were randomized controlled trials with low sample attrition, resulting in high internal validity.100

Each of these four studies examined the effects of reviewing and practicing different solution methods for the same problem—there were no other intervention components—providing a direct test of the recommendation.101 Three of the studies examined the effects of having students study different solutions to the same solved problem, with the solutions presented side by side on the same page to facilitate comparison.102 In two of the three studies, students in the comparison condition also studied solved problems, but the solution methods to the problems were presented one at a time on different pages.103 In the third study, students in the comparison condition studied two different problems solved with the same solution method.104 The fourth study compared students who solved a problem one way and then were asked to solve it a second time using a different method or ordering of steps to students who were asked to use only one solution method to solve problems.105 The interventions in these three studies ranged from two to five classroom periods.

The four studies included students aged 11 to 15 in 6th, 7th, and 8th grade. The studies took place in rural, suburban, and urban settings within the United States. Three interventions were implemented in classrooms during regular class time,106 and one study was conducted during the summer.107 The interventions were implemented by researchers, or by researchers working alongside classroom teachers. The studies as a whole provide moderate external and ecological validity.

Two studies involving students with no knowledge of algebra—as measured by a pretest—found negative or mixed effects.108 Both studies were randomized trials with low attrition, and the interventions asked students to review and practice multiple strategies. The first study occurred at a low-performing middle school and compared the use of multiple strategies to asking students to use only one solution method to solve problems. The study reported results separately for students who did not use algebra to solve problems on a pretest and students who did, and the findings were significantly different for these two groups. For the students who had no knowledge of algebra at baseline, the study found negative effects across procedural knowledge, procedural flexibility, and conceptual knowledge; for the students who had attempted some algebra on the pretest, the study found effects that were neither significant nor substantively important in those domains.109 The other study was conducted in multiple schools in a single district in the Midwest and compared the use of multiple strategies to asking students to solve a new problem that was similar in underlying structure to a problem they had previously solved. This study found a negative effect on procedural knowledge but a positive effect on procedural flexibility.110 These findings indicate that teaching alternative algebraic strategies can improve achievement once students have developed some procedural knowledge of algebra.

Table D.4. Studies providing evidence for Recommendation 3

Table D.4. Studies providing evidence for Recommendation 3 (continued)

Table D.4. Studies providing evidence for Recommendation 3 (continued)

Each row in this table represents a study, defined by the WWC as an examination of the effect of an intervention on a distinct sample. In some cases, multiple studies were described in a single article.

For studies that included multiple outcomes in a domain, the reported effect sizes and statistical significance are for the domain and calculated as described in the WWC Procedures and Standards Handbook 3.0 (pp. 28–29).

* = statistically significant at 0.05 level

a This study included two intervention groups and one comparison group. The effect sizes presented for this study were calculated by comparing the outcomes for each intervention group to the single comparison group.

b The sample size for the conceptual knowledge and procedural flexibility outcomes was 105 students, while the sample size for the procedural knowledge outcome was 98 students.

c A delayed posttest was also administered two weeks after the intervention. The WWC-calculated effect sizes are 0.01 (n = 94) for procedural knowledge and 0.29 (n = 101) for conceptual knowledge.

d The study examined two outcomes in this domain: flexibility knowledge and flexibility use. For flexibility knowledge, the WWC-calculated effect size is 0.45 and the effect is statistically significant. For flexibility use, the WWC-calculated effect size is 0.25 and the effect is not statistically significant. A delayed posttest (n = 101) was also administered to assess the two outcomes two weeks after the intervention. For flexibility knowledge, the WWC-calculated effect size is 0.47 and the effect is statistically significant. For flexibility use, the WWC-calculated effect size is 0.53 and the effect is statistically significant. The WWC-calculated domain effect size for procedural flexibility on the delayed posttest is 0.50 and the effect is statistically significant.

e The sample size for the conceptual knowledge and procedural flexibilty outcomes was 106 students, while the sample size for the procedural knowledge outcome was 101 students.

f A delayed posttest was also administered two weeks after the intervention. The WWC-calculated effect sizes are –0.20 (n = 99) for procedural knowledge and 0.01 for conceptual knowledge (n = 104).

g The study examined two outcomes in this domain: flexibility knowledge and flexibility use. For flexibility knowledge, the WWC-calculated effect size is 0.30 and the effect is not statistically significant. For flexibility use, the WWC-calculated effect size is 0.41 and the effect is not statistically significant. A delayed posttest (n = 104) was also administered two weeks after the intervention. For flexibility knowledge, the WWC-calculated effect size is 0.32 and the effect is statistically significant. For flexibility use, the WWC-calculated effect size is 0.38 and the effect is not statistically significant. The WWC-calculated domain effect size for procedural flexibility on the delayed posttest is 0.35.

h The study examined two outcomes in this domain: flexibility measured via solution strategy use and flexibility via an independent measure. For flexibility measured via solution strategy use, the WWC-calculated effect size is 0.33 and the effect is statistically significant. For flexibility via an independent measure, the WWC-calculated effect size is 0.47 and the effect is statistically significant.

i The study included two intervention groups and one comparison group. The effect sizes presented for this study were calculated by comparing the outcomes for each intervention group to the single comparison group.

j A delayed posttest (n = 118) was also administered one month after the intervention. The WWC-calculated effect sizes are 0.15 for procedural knowledge and 0.14 for conceptual knowledge.

k The study examined two outcomes in this domain: flexibility knowledge and flexibility use. For flexibility knowledge, the WWC-calculated effect size is 0.30 and the effect is not statistically significant. For flexibility use, the WWC-calculated effect size is 0.44 and the effect is statistically significant. A delayed posttest (n = 118) was also administered one month after the intervention. For flexibility knowledge, the WWC-calculated effect size is 0.31 and the effect is not statistically significant. For flexibility use, the WWC-calculated effect size is 0.52 and the effect is statistically significant. The WWC-calculated domain effect size for procedural flexibility on the delayed posttest is 0.42 and the effect is statistically significant.

l This comparison meets WWC group design standards with reservations due to high sample attrition. Two outcomes were examined in the procedural flexibility domain: flexibility knowledge and flexibility use. For flexibility knowledge, the WWC-calculated effect size is –0.06 and the effect is not statistically significant. For flexibility use, the WWC-calculated effect size is –0.24 and the effect is not statistically significant.

m Effect sizes for this study are calculated using the standard deviations provided in the paper (calculated with imputed data) because standard deviations based on non-imputed data were not available.

n The study examined two outcomes in this domain: flexibility knowledge and flexibility use. For flexibility knowledge, the WWC-calculated effect size is 0.03 and the effect is not statistically significant. For flexibility use, the WWC-calculated effect size is 0.21 and the effect is not statistically significant.

o The study examined two outcomes in this domain: flexibility knowledge and flexibility use. For flexibility knowledge, the WWC-calculated effect size is –0.37 and the effect is not statistically significant. For flexibility use, the WWC-calculated effect size is –0.33 and the effect is not statistically significant.

p This study included two intervention groups and one comparison group. The effect sizes presented for this study were calculated by comparing the outcomes for each intervention group to the single comparison group.

q The study examined three outcomes in this domain: knowledge of multiple strategies, use of multiple strategies, and use of efficient strategies. For knowledge of multiple strategies, the WWC-calculated effect size is 0.68 and the effect is statistically significant. For use of multiple strategies, the WWC-calculated effect size is 0.53 and the effect is not statistically significant. For use of efficient strategies, the WWC-calculated effect size is 0.23 and the effect is not statistically significant.

r The study examined three outcomes in this domain: knowledge of multiple strategies, use of multiple strategies, and use of efficient strategies. For knowledge of multiple strategies, the WWC-calculated effect size is 0.52 and the effect is not statistically significant. For use of multiple strategies, the WWC-calculated effect size is 0.03 and the effect is not statistically significant. For use of efficient strategies, the WWC-calculated effect size is 0.93 and the effect is statistically significant.

s The study examined two outcomes in this domain: percent correct on the test, and knowledge of standard solutions. For percent correct on the test, the WWC-calculated effect size is –0.35 and the effect is not statistically significant. For knowledge of standard solutions, the WWC-calculated effect size is –0.34 and the effect is not statistically significant.

t The study examined two outcomes in this domain: use of multiple strategies, and use of inventions. For use of multiple strategies, the WWC-calculated effect size is 0.58 and the effect is not statistically significant. For use of inventions, the WWC-calculated effect size is 0.70 and the effect is not statistically significant.

Referencesa

Akkus, R., Seymour, J., & Hand, B. (2007). Improving dialogical interaction in classrooms through improving teacher pedagogical practices: Impact on students’ scores. Paper presented at the annual meeting of the North American Chapter of the International Group for the Psychology of Mathematics Education, University of Nevada, Reno, Nevada.

Although this study did not meet WWC group-design standards, it contributed to the level of evidence because the analysis adequately controlled for selection bias (see the description of the levels of evidence in Appendix A).

American Educational Research Association, American Psychological Association, & National Council on Measurement in Education. (1999). The standards for educational and psychological testing. Washington, DC: American Educational Research Association Publications.

Atkinson, R., Derry, S. J., Renkl, A., & Wortham, D. (2000). Learning from examples: Instructional principles from the worked examples research. Review of Educational Research, 70(2), 181–214.

Booth, J. L., Koedinger, K. R., & Pare-Blagoev, J. (2010). Does practice really make perfect? Improving upon the ubiquitous practice assignment. Unpublished manuscript.

Booth, J. L., Lange, K. E., Koedinger, K. R., & Newton, K. J. (2013). Using example problems to improve student learning in algebra: Differentiating between correct and incorrect examples. Learning and Instruction, 25, 24–34.

Brenner, M. E., Mayer, R. E., Moseley, B., Brar, T., Durán, R., Reed, B. S., & Webb, D. (1997). Learning by understanding: The role of multiple representations in learning algebra. American Educational Research Journal, 34(4), 663–689.

Although this study did not meet WWC group-design standards, it contributed to the level of evidence because the analysis adequately controlled for selection bias (see the description of the levels of evidence in Appendix A).

Carroll, W. M. (1991). Using worked examples in learning algebra: Classroom studies of high school algebra (Doctoral dissertation). Available from ProQuest Dissertations and Theses database. (UMI No. 9213423)

Carroll, W. M. (1994). Using worked examples as an instructional support in the algebra classroom. Journal of Educational Psychology, 86(3), 360–367.

Hutchinson, N. L. (1993). Effects of cognitive strategy instruction on algebra problem solving of adolescents with learning disabilities. Learning Disability Quarterly, 16(1), 34–63.

Ives, B. (2007). Graphic organizers applied to secondary algebra instruction for students with learning disorders. Learning Disabilities Research and Practice, 22(2), 110–118.

Kalyuga, S., Chandler, P., Tuovinen, J., & Sweller, J. (2001). When problem solving is superior to studying worked examples. Journal of Educational Psychology, 93(3), 579–588.

Katz, V. J. (Ed.). (2007). Algebra: Gateway to a technological future. Washington, DC: Mathematical Association of America.

Mevarech, Z. R., & Kramarski, B. (2003). The effects of metacognitive training versus worked-out examples on students’ mathematical reasoning. British Journal of Educational Psychology, 73(4), 449–471.

National Council of Teachers of Mathematics. (2010). Focus in high school mathematics: Reasoning and sense making in algebra. Reston, VA: National Council of Teachers of Mathematics.

National Research Council. (2001). Adding it up: Helping children learn mathematics. J. Kilpatrick, J. Swafford, and B. Findell (Eds.). Mathematics Learning Study Committee, Center for Education, Division of Behavioral and Social Sciences and Education. Washington, DC: National Academy Press.

Rittle-Johnson, B., & Star, J. R. (2007). Does comparing solution methods facilitate conceptual and procedural knowledge? An experimental study on learning to solve equations. Journal of Educational Psychology, 99(3), 561–574.

Rittle-Johnson, B., & Star, J. R. (2009). Compared to what? The effects of different comparisons on conceptual knowledge and procedural flexibility for equation solving. Journal of Educational Psychology, 101(3), 529–544.

Rittle-Johnson, B., Star, J. R., & Durkin, K. (2009). The importance of prior knowledge when comparing examples: Influences on conceptual and procedural knowledge of equation solving. Journal of Educational Psychology, 101(4), 836–852.

Rittle-Johnson, B., Star, J. R., & Durkin, K. (2012). Developing procedural flexibility: Are novices prepared to learn from comparing procedures? British Journal of Educational Psychology, 82(3), 436–455.

Scheiter, K., Gerjets, P., & Schuh, J. (2010, September). The acquisition of problem-solving skills in mathematics: How animations can aid understanding of structural problem features and solution procedures. Instructional Science, 38(5), 487–502.

Schneider, M., Rittle-Johnson, B., & Star, J. R. (2011). Relations among conceptual knowledge, procedural knowledge, and procedural flexibility in two samples differing in prior knowledge. Developmental Psychology, 47(6), 1525–1538.

Schneider, M., & Stern, E. (2010). The developmental relations between conceptual and procedural knowledge: A multimethod approach. Developmental Psychology, 46(1), 178–192.

Star, J. R., & Rittle-Johnson, B. (2008). Flexibility in problem solving: The case of equation solving. Learning and Instruction, 18(6), 565–579.

Star, J. R., & Seifert, C. (2006). The development of flexibility in equation solving. Contemporary Educational Psychology, 31(3), 280–300.

Sweller, J. (1988). Cognitive load during problem solving: Effects on learning. Cognitive Science, 12(2), 257–285.

Sweller, J., Ayres, P., & Kalyuga, S. (2011). The worked example and problem completion effects. In J. Sweller, P. Ayres, & S. Kalyuga (Eds.), Cognitive Load Theory (pp. 99–109). New York: Springer.

Sweller, J., & Cooper, G. A. (1985). The use of worked examples as a substitute for problem solving in learning algebra. Cognition and Instruction, 2(1), 59–89.

Sweller, J., van Merrienboer, J., & Paas, F. (1998). Cognitive architecture and instructional design. Educational Psychology Review, 10(3), 251–296.

Tarmizi, R., & Sweller, J. (1988). Guidance during mathematical problem solving. Journal of Educational Psychology, 80(4), 424–436.

U.S. Department of Education. (2008). Foundations for success: The final report of the Mathematics National Advisory Panel. National Mathematics Advisory Panel: Washington, DC.

Ward, M., & Sweller, J. (1990). Structuring effective worked examples. Cognition and Instruction, 7(1), 1–39.

Xin, Y. P., Jitendra, A. K., & Deatline-Buchman, A. (2005). Effects of mathematical word problem-solving instruction on middle school students with learning problems. The Journal of Special Education, 39(3), 181–192.

Zhu, X., & Simon, H. (1987). Learning mathematics from examples and by doing. Cognition and Instruction, 4(3), 137–166.

Endnotesa

1. Katz (2007).

2. Bolded terms in this practice guide are defined in the Glossary.

3. The review protocol is available via the WWC website at http://www.whatworks.ed.gov.

4. Atkinson et al. (2000); Kalyuga et al. (2001); Sweller (1988).

5. Sweller, Ayres, and Kalyuga (2011); Tarmizi and Sweller (1988); Ward and Sweller (1990).

6. Sweller and Cooper (1985); Sweller, van Merrienboer, and Paas (1998); Zhu and Simon (1987).

7. Booth et al. (2013) Experiment 1; Carroll (1994) Experiment 1; Carroll (1994) Experiment 2; Mevarech and Kramarski (2003).

8. Studies were classified as having a positive or negative effect when the result was either statistically significant (unlikely to occur by chance) or substantively important (producing considerable differences in outcomes).

9. Booth et al. (2013) Experiment 1; Carroll (1994) Experiment 1; Carroll (1994) Experiment 2.

10. Booth et al. (2013) Experiment 1.

11. Carroll (1994) Experiment 1; Carroll (1994) Experiment 2.

12. Mevarech and Kramarski (2003).

13. One randomized controlled trial that meets WWC group design standards without reservations found that solved problems had negative effects on procedural knowledge. Mevarech and Kramarski (2003) compared students who studied and discussed solved problems in groups to students who used reflective questioning (a practice suggested in Recommendation 2) to solve practice problems in groups. These negative findings indicate that the effects of this practice can depend on what students do instead of using solved problems.

14. In these and other similar examples, teachers can include a “verify my solution” section in the solved problem to model how students can verify their solutions by substituting their answers into the original equation.

15. The third factoring problem is adapted from an example in CCSS.MATH.CONTENT.HSA-SSE.A.2.

16. Adapted from Booth, Koedinger, and Pare-Blagoev (2010) Experiment 2.

17. Ives (2007) Experiment 1; Mevarech and Kramarski (2003); Scheiter, Gerjets, and Schuh (2010); Xin, Jitendra, and Deatline-Buchman (2005); Akkus, Seymour, and Hand (2007); Brenner et al. (1997). Brenner et al. (1997) was a randomized controlled trial with high sample attrition, and Akkus, Seymour, and Hand (2007) used a quasi-experimental design; although the analytic samples for these studies were not shown to be equivalent at baseline, the analyses included a pretest covariate as a statistical control for selection bias.

18. Studies were classified as having a positive or negative effect when the result was either statistically significant (unlikely to occur by chance) or substantively important (producing considerable differences in outcomes).

19. Mevarech and Kramarski (2003); Scheiter, Gerjets, and Schuh (2010); Xin, Jitendra, and Deatline-Buchman (2005); Akkus, Seymour, and Hand (2007).

20. Brenner et al. (1997); Ives (2007) Experiment 1; Scheiter, Gerjets, and Schuh (2010).

21. Akkus, Seymour, and Hand (2007); Mevarech and Kramarski (2003).

22. Mevarech and Kramarski (2003); Akkus, Seymour, and Hand (2007).

23. Brenner et al. (1997); Ives (2007) Experiment 1; Scheiter, Gerjets, and Schuh (2010); Xin, Jitendra, and Deatline- Buchman (2005).

24. Scheiter, Gerjets, and Schuh (2010); Xin, Jitendra, and Deatline-Buchman (2005).

25. Brenner et al. (1997); Ives (2007) Experiment 1.

26. Mevarech and Kramarski (2003); Scheiter, Gerjets, and Schuh (2010).

27. Ives (2007) Experiment 1; Xin, Jitendra, and Deatline-Buchman (2005).

28. Adapted from the “Throwing Horseshoes” exercise from the National Council of Teachers of Mathematics (2010).

29. Ives (2007) Experiment 1; Scheiter, Gerjets, and Schuh (2010); Xin, Jitendra, and Deatline-Buchman (2005).

30. Rittle-Johnson and Star (2009); Rittle-Johnson and Star (2007); Rittle-Johnson, Star, and Durkin (2012); Rittle-Johnson, Star, and Durkin (2009); Star and Rittle-Johnson (2008); Star and Seifert (2006).

31. Studies were classified as having a positive or negative effect when the result was either statistically significant (unlikely to occur by chance) or substantively important (producing considerable differences in outcomes).

32. Rittle-Johnson and Star (2009); Rittle-Johnson and Star (2007); Rittle-Johnson, Star, and Durkin (2012); Star and Rittle-Johnson (2008).

33. Rittle-Johnson, Star, and Durkin (2009); Star and Seifert (2006).

34. Rittle Johnson and Star (2009); Rittle Johnson and Star (2007); Rittle Johnson, Star, and Durkin (2012).

35. Star and Rittle-Johnson (2008).

36. Rittle-Johnson, Star, and Durkin (2009); Star and Seifert (2006).

37. Rittle-Johnson, Star, and Durkin (2009).

38. Star and Seifert (2006).

39. Adapted from Rittle-Johnson and Star (2007).

40. Ibid.

41. These are a subset of possible strategies for solving linear systems. Not all strategies will be applicable or useful for all types of linear systems.

42. Adapted from Rittle-Johnson and Star (2007).

43. Following WWC guidelines, improved outcomes are indicated by either a positive statistically significant effect or a positive, substantively important effect size. The WWC defines substantively important, or large, effects on outcomes to be those with effect sizes greater than or equal to 0.25 standard deviations. See the WWC guidelines at http://whatworks.ed.gov.

44. For more information, see the WWC Frequently Asked Questions page for practice guides at http://whatworks.ed.gov.

45. This includes randomized control trials (RCTs) and quasi-experimental design studies (QEDs). Studies not contributing to levels of evidence include single-case designs (SCDs) evaluated with WWC pilot SCD standards and regression discontinuity designs (RDDs) evaluated with pilot RDD standards.

46. The research may include studies generally meeting WWC design standards and supporting the effectiveness of a program, practice, or approach with small sample sizes and/or other conditions of implementation or analysis that limit generalizability. The research may include studies that support the generality of a relation but do not meet WWC design standards; however, they have no major flaws related to internal validity other than lack of demonstrated equivalence at pretest for QEDs. QEDs without equivalence must include a pretest covariate as a statistical control for selection bias. These studies must be accompanied by at least one relevant study meeting WWC design standards.

47. American Educational Research Association, American Psychological Association, and National Council on Measurement in Education (1999).

48. Eligible studies that meet WWC evidence standards without reservations or meet evidence standards with reservations are indicated by bold text in the endnotes and references pages.

49. As indicated in Table A.1, studies that do not meet WWC design standards but have no major flaws related to internal validity other than a lack of demonstrated equivalence can contribute to a moderate level of evidence if the study includes a pretest covariate as a statistical control for selection bias.

50. Hutchinson (1993) was reviewed as both a group design study and a single-case design study. It is included in both counts.

51. A statistically significant finding is unlikely to occur by chance.

52. Recognizing that some studies lack the statistical power to classify practically important effects as statistically significant, the panel also accepts substantively important effects as evidence of effectiveness. Substantively important effects are defined as an effect size greater than or equal to 0.25 or less than or equal to –0.25, as measured by Hedge’s g.

53. For multiple comparison adjustments and cluster corrections, see the WWC Procedures and Standards Handbook, Version 3.0, at http://www.whatworks.ed.gov.

54. Adapted from a problem in Booth et al. (2013).

55. Taken from Carroll (1994).

56. Adapted from a problem in Booth et al. (2013).

57. Taken from Rittle-Johnson, Star, and Durkin (2012).

58. Adapted from a problem in Rittle-Johnson and Star (2009).

59. National Research Council (2001); U.S. Department of Education (2008).

60. “By the term proficiency, the panel means that students should understand key concepts; achieve automaticity as appropriate (e.g., with addition and related subtraction facts); develop flexible, accurate, and automatic execution of the standard algorithms; and use these competencies to solve problems.” (U.S. Department of Education 2008; p. 22).

61. Schneider and Stern (2010); Schneider, Rittle-Johnson, and Star (2011).

62. Booth et al. (2013) Experiment 1; Carroll (1994) Experiment 1; Carroll (1994) Experiment 2; Mevarech and Kramarski (2003).

63. Booth et al. (2013) Experiment 1; Carroll (1994) Experiment 1; Carroll (1994) Experiment 2.

64. Booth et al. (2013) Experiment 1.

65. Carroll (1994) Experiment 1; Carroll (1994) Experiment 2.

66. Booth et al. (2013) Experiment 1.

67. Carroll (1994) Experiment 1; Carroll (1994) Experiment 2.

68. Booth et al. (2013) Experiment 1.

69. Ibid.

70. Ibid.

71. Carroll (1994) Experiment 1 grouped participants into higher and lower achievers based on scores on a placement test created by the researcher. The effects of the intervention did not vary by student achievement level.

72. Carroll (1994) Experiment 2.

73. Booth et al. (2013) Experiment 1.

74. Carroll (1994) Experiment 1; Carroll (1994) Experiment 2.

75. Mevarech and Kramarski (2003).

76. Ibid.

77. Ives (2007) Experiment 1; Mevarech and Kramarski (2003); Scheiter, Gerjets, and Schuh (2010); Xin, Jitendra, and Deatline-Buchman (2005).

78. Akkus, Seymour, and Hand (2007); Brenner et al. (1997).

79. Akkus, Seymour, and Hand (2007); Mevarech and Kramarski (2003); Scheiter, Gerjets, and Schuh (2010); Xin, Jitendra, and Deatline-Buchman (2005).

80. Brenner et al. (1997); Ives (2007) Experiment 1; Mevarech and Kramarski (2003).

81. Ives (2007) Experiment 1.

82. Brenner et al. (1997); Mevarech and Kramarski (2003).

83. Ives (2007) Experiment 1; Mevarech and Kramarski (2003); Scheiter, Gerjets, and Schuh (2010); Xin, Jitendra, and Deatline-Buchman (2005).

84. Akkus, Seymour, and Hand (2007); Brenner et al. (1997).

85. Akkus, Seymour, and Hand (2007); Mevarech and Kramarski (2003).

86. Scheiter, Gerjets, and Schuh (2010).

87. Ives (2007) Experiment 1; Xin, Jitendra, and Deatline-Buchman (2005).

88. Ives (2007) Experiment 1.

89. Brenner et al. (1997).

90. Mevarech and Kramarski (2003).

91. Scheiter, Gerjets, and Schuh (2010).

92. Akkus, Seymour, and Hand (2007); Brenner et al. (1997).

93. Ives (2007) Experiment 1; Xin, Jitendra, and Deatline-Buchman (2005).

94. Xin, Jitendra, and Deatline-Buchman (2005).

95. Scheiter, Gerjets, and Schuh (2010).

96. Rittle-Johnson and Star (2009); Rittle-Johnson and Star (2007); Rittle-Johnson, Star, and Durkin (2012); Rittle-Johnson, Star, and Durkin (2009); Star and Rittle-Johnson (2008); Star and Seifert (2006).

97. Rittle-Johnson and Star (2009); Rittle-Johnson and Star (2007); Rittle-Johnson, Star, and Durkin (2012); Star and Rittle-Johnson (2008).

98. Rittle-Johnson and Star (2009).

99. Rittle-Johnson and Star (2007).

100. One of the studies conducted a comparison for which there was high sample attrition. The analytic samples for this study were shown to be equivalent at baseline, resulting in moderate internal validity. The comparison found effects that were neither statistically significant nor substantively important.

101. Rittle-Johnson and Star (2009); Rittle-Johnson and Star (2007); Rittle-Johnson, Star, and Durkin (2012); Star and Rittle-Johnson (2008).

102. Rittle-Johnson and Star (2009); Rittle-Johnson and Star (2007); Rittle-Johnson, Star, and Durkin (2012).

103. Rittle-Johnson and Star (2007); Rittle-Johnson, Star, and Durkin (2012).

104. Rittle-Johnson and Star (2009).

105. Star and Rittle-Johnson (2008).

106. Rittle-Johnson and Star (2009); Rittle-Johnson and Star (2007); Rittle-Johnson, Star, and Durkin (2012).

107. Star and Rittle-Johnson (2008).

108. Rittle-Johnson, Star, and Durkin (2009); Star and Seifert (2006).

109. Rittle-Johnson, Star, and Durkin (2009).

110. Star and Seifert (2006).

a Eligible studies that meet WWC evidence standards or meet evidence standards with reservations are indicated by bold text in the endnotes and references pages. For more information about these studies, please see Appendix D.

Table of Contents

Teaching Strategies for Improving Algebra Knowledge in Middle and High School Students

Table of Contents

Introduction to the Teaching Strategies for Improving Algebra Knowledge in Middle and High School Students Practice Guide

Recommendation 1. Use solved problems to engage students in analyzing algebraic reasoning and strategies

Recommendation 2. Teach students to utilize the structure of algebraic representations

Recommendation 3. Teach students to intentionally choose from alternative algebraic strategies when solving problems

Glossary

Appendix A. Postscript from the Institute of Education Sciences

Appendix B. About the Authors

Appendix C. Disclosure of Potential Conflicts of Interest

Appendix D. Rationale for Evidence Ratings

References

Endnotes

List of Tables

Table 1. Recommendations and corresponding levels of evidence

Table A.1. Institute of Education Sciences levels of evidence for What Works Clearinghouse practice guides

Table D.1. Outcome domain definitions and sample items

Table D.2. Studies providing evidence for Recommendation 1

Table D.2. Studies providing evidence for Recommendation 1 (continued)

Table D.3. Studies providing evidence for Recommendation 2

Table D.3. Studies providing evidence for Recommendation 2 (continued)

Table D.4. Studies providing evidence for Recommendation 3

Table D.4. Studies providing evidence for Recommendation 3 (continued)

Table D.4. Studies providing evidence for Recommendation 3 (continued)

Recommendation 1

Example 1.1. Questions to facilitate discussion of solved problems

Example 1.2. Questions to facilitate discussion of the structure of problems

Example 1.3. Examples of solved problems for different learning objectives

Example 1.4. Presenting multiple solved problems with varying degrees of difficulty

Example 1.5. One way to introduce incorrect solved problems

Example 1.6. Parallel correct and incorrect solved problems, factoring

Example 1.7. Parallel correct and incorrect solved problems, completing the square

Example 1.8. Using solved problems in whole-class discussion

Example 1.9. Solved problems in small-group work

Example 1.10. Incomplete solved problems

Example 1.11. Independent practice that incorporates solved problems

Recommendation 2

Example 2.1. Seeing structure in algebraic representations

Example 2.2. Modeling precise mathematical language, the distributive property

Example 2.3. Imprecise vs. precise mathematical language

Example 2.4. Student using reflective questioning

Example 2.5. Reflective questions for noticing structure

Example 2.6. Equations of the same line in different forms

Example 2.7. Equations of the same parabola in different forms

Example 2.8. Multiple algebraic representations

Example 2.8. Multiple algebraic representations (continued)

Example 2.9. Using diagrams to compare algebraic representations

Example 2.10. Examples of cooperative learning strategies

Recommendation 3

Example 3.1. Alternative and conventional solution strategies

Example 3.2. The same problem solved using two different solution strategies

Example 3.3. Two different problems, each solved with the same strategy

Example 3.4. Reflective questions for selecting and considering solution strategies

Example 3.5. Some possible strategies for solving quadratic equations

Example 3.6. Some possible strategies for solving linear systems

Example 3.7. Prompts to encourage students to articulate their reasoning

Example 3.8. Sample student dialogue discussing two different solution strategies

Example 3.9. Small-group comparison and discussion activity