Skip to Content

Design the Evaluation

What is an Evaluation Plan?

An evaluation plan flows from your logic model and helps you think through how to measure progress and achievements. It’s the blueprint for your evaluation. It lays out what you need to know to implement your evaluation and how to get that information.

  • The specific evaluation questions you want to answer to know if your program is effective
  • The measures or indicators you will use to know if you are moving toward your goals
  • The practical and realistic data sources and methods you will use to get the information to answer your questions
  • The people who will help you collect your data and the timeline and budget that will guide your work

An Evaluation Plan Ensures Your Evaluation Is:

  • Useful
  • Feasible
  • Ethical
  • Accurate

How Do You Design an Evaluation Plan?

We’ve broken the steps of designing an evaluation plan into two parts on the next few pages. Each part has a tool with more information, shows an example in practice, and provides a template to work on each step.

Part 1: Develop

Design the Evaluation, Part 1: Develop Evaluation Questions and Indicators

What Are Evaluation Questions?

Developing evaluation questions is the starting point for a strong evaluation. They help you to focus in on the program elements that you and your stakeholders think are the most important to understand if your program is effective.

Evaluation questions should flow easily from your logic model:

  • Process questions ask about implementation, or how the inputs, activities and participation sections of your logic model are working. These questions will help you understand whether the program is being implemented according to your plan and identify opportunities for improvement.
  • Outcome questions ask about whether you have accomplished your outcomes as described in your logic model. These questions will help you understand progress and changes that are made as a result of your program. For example, have there been changes in people’s knowledge or health behaviors as a result of your short term outcomes?

An evaluation can ask both process and outcome questions. Typically your evaluation will focus more heavily on process questions in early years, but it’s important to start collecting data for outcome measurement early on.

Over time, the accumulation of outcome data will allow you to answer your outcome questions.

Cottage Population Health - Case Study

Case Study

The Health Connect program team sat down with their logic model and brainstormed a list of evaluation questions. It was tough to prioritize them, but they focused on those that would be the most useful and feasible to answer.

They chose a few indicators for each evaluation question and tried to pick a mix of qualitative and quantitative indicators to more completely answer the question.

Download the Design the Evaluation, Part 1: Develop Evaluation Questions and Indicators Case Study


What Are Indicators?

The next step is determining how to answer your evaluation questions. To do this you need to identify indicators, which are the evidence that you are making progress. They answer the question "if the outcome is achieved, how will we know it?"

Indicators also can help you see where you are making progress if you cannot yet show your program’s impact on the longer-term outcomes.

If this step is skipped, it can be difficult to make sure you pick the right data collection method to actually answer your questions. Here are some examples:

Cottage Health Evaluation Toolkit - Walkability

Walkability

An indicator of a residential area's walkability is the proportion of streets that have sidewalks.

Cottage Health Evaluation Toolkit - Obesity

Children's Health

An indicator of children's health status is the percentage of those who are obese.

Cottage Health Evaluation Toolkit - Graduation

Academic Achievement

An indicator of academic achievement is the high school graduation rate.

Part 2: Plan

Design the Evaluation, Part 2: Plan for Data Collection

What is Data Collection?

The next step in creating an evaluation plan is picking the source that will provide the best information, while also thinking about how easy or difficult it will be to get that information. Often there is more than one source for data and more than one way to collect it from that source.

What Data Sources Should We Use?

The data you use will depend entirely on your evaluation questions. Evaluation data generally come from three sources:

Cottage Health Evaluation Toolkit - People

People

Clients, program staff, funders, staff of other organizations, community leaders, elected officials, general public, critics, etc.

Cottage Health Evaluation Toolkit - Documentation

Documents and electronic systems

Tracking sheets for service delivery, grant proposals, meeting minutes, publicity materials, existing data, previous evaluation reports, electronic health records, etc.

Cottage Health Evaluation Toolkit - Observation

First hand observation

Directly observing and documenting the program's setting or key features, or how things work in practice like personal interactions and program activities


What Methods Should You Use to Collect Your Data?

Once you have explored potential data sources, the next step is to choose which method will be most effective. You can:

  • Use existing data. This includes data already published or collected in the past by other parties, as well as data previously collected by your organization for other purposes. You might use existing program documentation, electronic health records, or local secondary data from sources like CottageHealth.Data2Go.
  • Collect new data. Data you collect yourself (or have someone collect on your behalf) are called primary data. This might include interviewing people, holding focus groups, using online or phone surveys, or observing the program in action. We will walk through all the typical methods used to collect evaluation data in the next step - Collect.

How Do You Plan Your Data Collection?

Consider these factors to select the best data sources and methods for your evaluation:

  • What are the advantages and disadvantages of different sources and methods?
  • How intrusive will it be to collect the data in this way?
  • Is there an existing data source you could use?
  • Is this a culturally appropriate data collection method?
  • Who will collect the data and when?
Cottage Population Health - Case Study

Case Study

The Health Connect program team brainstormed possible data sources and methods for each of their indicators and then prioritized which would work best for their evaluation.

For example, the program team wanted to know who the Community Health Worker (CHW) program serves. One indicator is the number of unduplicated clients. The team determined that these data were already available through program documents like intake forms and the CHW schedule.

As part of their evaluation, they would conduct a document review to establish the number of clients served.

Download the Design the Evaluation, Part 2 – Plan for Data Collection Case Study

Connect With Pop Health

We want to hear from you -- our community leaders! E-mail the Population Health team at: listenpophealth@sbch.org, or to be added to our mailing list please fill out this brief form.

Mailing List Form

Community Health Needs Assessment

Population Health - Community Health Needs Assessment

The CHNA describes the well-being of Santa Barbara County’s residents and selected social determinants of their health, with comparisons to California’s health profile as a whole.

Go to the CHNA Full Report

Population Health Listening Tour

Cottage Population Health Video - Santa Barbara

Learn about the challenges facing populations in Santa Barbara County and the implications that they have on public health.

Take the Tour

Listening Tour Findings

View a brief summary of key findings from the Cottage Population Health Listening Tour.

View Listening Tour Key Findings

Listening Tour Survey

The results from this survey helped Cottage Health prioritize the health indicators highlighted in the Community Health Needs Assessment (CHNA) report.

View Listening Tour Survey Results