Program Evaluation Toolkit

In chapter 1, you will explore the basic components of a logic model and learn how it can be leveraged to plan and implement your evaluation.

In chapter 2, you will develop a problem statement that you would like to address. This chapter will guide you through the process of articulating the problem and why the problem exists.

In chapter 3, you will explore the resources, activities, and outputs that make up the essence of the program you intend to implement. These logic model components help to describe a program that will address the problem defined in the problem statement, discussed in chapter 2.

In chapter 4, you will explore the short-term, mid-term, and long-term outcomes that you expect your program will achieve to address the problem identified in chapter 2.

In chapter 1, you will learn about the different types of evaluation questions and how each type can be used to inform the implementation and outcomes of your program.

In chapter 2, you will learn how to write evaluation questions using a framework called PARSEC, which stands for pertinent, answerable, reasonable, specific, evaluative, and complete. You will then apply the framework to sample evaluation questions.

In chapter 3, you will learn about a process for prioritizing your evaluation questions. This process will help ensure your evaluation questions provide the information you need to assess whether your program is meeting intended outcomes.

In chapter 1, you will learn about different evaluation design categories that can be adapted to meet your evaluation needs. You will learn about how each category relates to the claims you can make about your program.

In chapter 2, you will learn about threats to validity in evaluation design. You should examine threats to validity regardless of what your evaluation design is or whether you ask process or outcome evaluation questions. Minimizing threats to validity is an essential part of any evaluation.

In chapter 3, you will learn about levels of evidence and design standards. The level of evidence you require may influence your selection of an appropriate evaluation design. If you want to provide higher levels of evidence to support a program, you must adhere to more rigorous standards of evaluation design.

In chapter 1, you will be introduced to the concept of sampling and learn how it can be leveraged to meet the needs of your evaluation efforts.

In chapter 2, you will learn about different sampling techniques. It is often not feasible to collect data from an entire population, so a smaller sample must be selected. Important considerations in sampling include identifying whom to include in the sample, deciding whether a random or nonrandom sampling technique is appropriate, and determining an appropriate sample size.

In chapter 3, you will explore methods for determining sample size and create a sampling plan. Before you begin collecting data, you should determine the adequate sample size you will need to feel confident that your results are not due merely to chance.

In chapter 1, you will be introduced to different forms of data. You will explore qualitative and quantitative data and how each can be used to answer your evaluation questions.

In chapter 2, you will review data quality considerations. As you begin to address your evaluation questions, it is important to consider the quality of the data you will use. If the quality of the data is poor, your findings may not accurately represent the program resources, activities, outputs, or outcomes.

In chapter 3, you will match data sources to your evaluation questions. In doing so, you will understand how completely you will be able to address your questions.

In chapter 1, you will learn about instruments that might be developed to collect evaluation data, such as surveys, focus groups, and interview protocols.

Chapter 2 covers developing and using observation protocols that include, for example, recording checklists and open field notes to collect data. This chapter includes guiding documents and an example observation protocol.

Chapter 3 focuses on survey development and implementation. This chapter includes guiding documents and an example survey instrument. At the conclusion of this chapter, you will apply what you have learned about data collection instruments to developing your own instrument to answer an evaluation question.

In chapter 1, you will be introduced to common approaches that can be used in the preparation of your data for analysis and interpretation.

In chapter 2, you will explore some examples of common data analysis methods. This chapter focuses on two relatively straightforward examples of data analysis, one quantitative and one qualitative, presented in relation to an example evaluation.

In chapter 3, you will use a framework for thinking about how to act on the results of your data analyses. You will explore the process for interpreting your results, generating implications, and making recommendations.

Chapter 1 outlines key considerations for developing a dissemination plan, such as the audience, the message, the best approach for communicating the message, and the best time to share the information with the audience. The chapter also includes important considerations for ensuring dissemination products are accessible to all members of the audience.

Chapter 2 reviews key considerations for visualizing data, including the audience, message, and approach. The chapter also presents example data visualizations, such as graphs, charts, and tables, that can help make the data more easily understandable.