Skip to Content

Collect Evaluation Data

How Will You Collect Your Evaluation Data?

You may not have to start from scratch—there are many questions and instruments that have been developed and tested by other organizations. Sometimes these will work without changes, and sometimes they need to be adapted to fit your evaluation. If existing resources will not work, you can design your own instrument.

Quality Check:

  • Accurate – Is your method measuring what you intend to measure? Is it doing so consistently?
    • Is your method reliable? Reliability is the degree to which your method produces consistent results over many uses. For example, a reliable survey question would be one where every person who took the survey thought the question was asking the same thing. Or a reliable observation tool would help any observers score or describe what they are seeing in the same way.
    • Is your method valid? Validity is the degree to which your method actually measures what you intend it to measure. A simple example is that a ruler is a valid measuring device for length, but would not be valid for measuring weight. Often using more than one method for data collection can help you be more confident that you are measuring what you intend to measure.
  • Ethical – Do you have a method to keep people’s information confidential? If the data you are collecting will be sensitive or personal, your process will need to build in safeguards to protect people's privacy and make sure the data are secure.

When Should You Collect Data?

When you collect data depends on what you want to learn and the data collection method you will be using. To answer some questions, you may only need to collect that data at one point during the program. To answer other questions, it may be useful to collect data before and after the program and then compare the results. Most of your data should probably be collected on a regular basis, with enough time between data points to show change over time.

How to Plan and Conduct a Focus Group

Why Conduct a Focus Group?

Cottage Population Health - Toolkit

Focus Group Tool Overview

PLANNING

  • Decide when
  • Figure out logistics
  • Identify the sample
  • Recruit participants
  • Identify and train a facilitator
  • Develop a discussion guide
  • Pilot test

COLLECTING DATA

  • Use the data collection tips
  • Track & document data
  • Key follow up steps

Download the How to Plan and Conduct a Focus Group Tool

Focus groups are structured small group discussions with a set of questions that guide the group through a conversation about your program. They are an excellent way to get rich stories and descriptions of how your program is working from a group of people.

Focus groups are most appropriate to use for group process and brainstorming because people can build off of each other’s comments. Because of this, focus groups are often used to test messages, get ideas for program improvement, etc. Good focus groups also leave space for people to share individual points of view, but they are not an appropriate way to discuss topics that are sensitive, controversial, or personal.

Focus Groups Can Help You Explore:

  • In-depth information about a particular issue or topic
  • The range of perceptions, views and opinions about a particular issue or topic (not just what people agree about)
  • Factors that influence opinions or behaviors

Typically, when they are used for evaluation, multiple focus groups are conducted. You could conduct focus groups with several groups of people that all share similar characteristics. You could also conduct multiple focus groups with diverse participants, as long as each focus group itself has similar characteristics or contexts. Then you can compare responses of different groups.

Planning and Developing Focus Groups

Focus groups can help you collect information from multiple people in a short period of time. However, they can be time consuming and difficult to arrange because they require coordinating the logistics of bringing together a group of people in-person and finding a strong facilitator. Be sure to allot time for planning and organizing.

How to Plan and Conduct Interviews

Why Use Interviews?

Cottage Population Health - Toolkit

Interview Tool Overview

PLANNING

  • Decide when
  • Identify the sample
  • Recruit participants
  • Identify and train interviewers
  • Develop interview protocol
  • Pilot test
  • Finalize logistics

COLLECTING DATA

  • Track & document data
  • Key follow up steps

Download the How to Plan and Conduct Interviews Tool

An interview is a one-on-one conversation with someone who has knowledge and experience about the specific topic you are interested in. Because the people who provide the data have perspectives that are important or “key”, interviews for data collection are often called “key informant” interviews.

Interviewing is Useful When:

  • The topic is sensitive or people are likely to be inhibited by speaking about the topic in front of others
  • People have a limited reading ability
  • Bringing a group of people together is difficult
  • You need in-depth understanding about complex topics (e.g., community participation, empowerment, or cohesiveness)
  • You have a lot of open-ended questions and want to be able to ask for clarification or probe for understanding

How to Plan and Conduct Direct Observation

Why Choose Direct Observation to Collect Data?

Cottage Population Health - Toolkit

Observation Tool Overview

PLANNING

  • Decide who, what & when
  • Focus
  • Select format & type
  • Develop a tool
  • Identify & train observers

COLLECTING DATA

  • Use the tips
  • Key follow up steps

Download the How to Plan and Conduct Direct Observation Tool

Direct observation is the collection of information using your senses. By observing, you can document activities, behavior, and physical aspects of a situation without having to depend on peoples’ willingness or ability to respond accurately to questions.

Observation is Useful When:

  • You are trying to understand an ongoing process or behavior, or an unfolding situation or event.
  • There is physical evidence, or products or outcomes that can be seen.
  • Written or other data collection methods seem inappropriate.

Observation can occur in public situations, such as observing peoples’ participation in a training or documenting how people use a community garden. Observation can also occur in more private settings like observing a patient visit in a clinic.

How to Plan and Conduct Surveys

Why Choose a Survey to Collect Data?

Cottage Population Health - Toolkit

Survey Tool Overview

PLANNING

  • Define the sample
  • Decide how & when to administer
  • Develop the questions
  • Pilot test

COLLECTING DATA

  • Distribute & collect
  • Monitor & record responses
  • Calculate response rate
  • Key follow up steps

Download the How to Plan and Conduct Surveys Tool

A survey is a questionnaire (written or verbal) that is administered verbally or in writing to your focus population. Surveys are a good method to use to learn about:

  • Knowledge—what people know or how well they understand something
  • Beliefs—people’s attitudes, opinions
  • Behaviors—what people do
  • Attributes/demographics—who people are

In comparison to many of the other qualitative methods (interviews, focus groups, and observations), surveys provide more standardized and consistent data that can be more easily combined and analyzed. Surveys can help you to gather both qualitative and quantitative information. Also, survey responses can be anonymous, which is very helpful if you are asking about sensitive or personal information.

Case Study

The Health Connect program’s evaluation plan included a variety of measures that required data collection before, during, and after the program.

Some data were collected to answer questions about how the program was working. Other data were collected to understand the program’s outcomes, such as improvement in health behaviors related to healthy eating and physical activity.

The program team was also able to use existing electronic health record data for their long-term outcomes.

Below are examples of their data collection at different time points.

Cottage Health Evaluation Toolkit - Collect Data - Baseline

Baseline

Before delivering new program services, pull data on health outcomes related to diabetes for new clients, using existing electronic health record data.

Cottage Health Evaluation Toolkit - Collect Data - During the Program

During the Program

Collect program participation data, (e.g., number of clients served) and interview clients to see if their goals and action plans are useful.

Cottage Health Evaluation Toolkit - Collect Data - After the Program

After the Program

Once the program is complete, leverage existing electronic health record data to measure improvement in health outcomes related to diabetes and compare to baseline.

Connect With Pop Health

We want to hear from you -- our community leaders! E-mail the Population Health team at: listenpophealth@sbch.org, or to be added to our mailing list please fill out this brief form.

Mailing List Form

Population Health Listening Tour

Cottage Population Health Video - Santa Barbara

Learn about the challenges facing populations in Santa Barbara County and the implications that they have on public health.

Take the Tour

Community Health Needs Assessment

Population Health - Community Health Needs Assessment

The CHNA describes the well-being of Santa Barbara County’s residents and selected social determinants of their health, with comparisons to California’s health profile as a whole.

Go to the CHNA Full Report

Listening Tour Findings

View a brief summary of key findings from the Cottage Population Health Listening Tour.

View Listening Tour Key Findings

Listening Tour Survey

The results from this survey helped Cottage Health prioritize the health indicators highlighted in the Community Health Needs Assessment (CHNA) report.

View Listening Tour Survey Results