Discussion

Purpose vs. Goals: You can think of your purpose as overarching reason that you are communicating or the type of change you are hoping to accomplish. Goals are more specific, they outline what your destination looks like in practical terms or how you define success.

Goals vs. Evaluative Criteria: If goals are how you define success, the evaluative criteria are objective measures that will help you to determine whether or not you have achieved your goals.  A goal without a measure of success is not a goal worth having.

Now that you have identified the general purpose for your communication campaign, it is time to set specific goals. A tried and true tool for goal setting is the SMART acronym. Specifically, good goals are:

• Specific (What will you do? What will you accomplish?)
• Measurable (How will you know you have achieved your goals?)
• Attainable (Are you reasonably certain of success?)
• Relevant (Why is the goal even important?)
• Time bound (What is the time frame for completion?)

But how can you know for sure your goals are SMART? The best way to go about goal setting is actually to start with the end — the evaluation. What?!? That seems a little backward. Are we really asking you to think about how you will evaluate your communication program before you have even designed it? The answer is…yes! This is because goals and evaluation criteria are best developed in conjunction with one another. Having a good idea in mind about how your program could and should be evaluated actually helps you to design a better program — just like a grading rubric helps to focus you and keep you on track for a class assignment!

Developing goals and evaluative criteria

You should have goals and evaluative criteria related to both the process and the outcome of your communication campaign. 

The process

Ultimately, the success of your communication plan can be measured by answering one simple question: Did you achieve your goals? This illustrates why it is important to spend time thinking about and verbalizing your goals and evaluation from the outset.Process goals and evaluations are focused on the mechanics of the plan and its implementation. For example, you might want to set goals and evaluative criteria around whether your message reached the target audience, the consistency and timeliness of messaging across different platforms, how information flowed within the communications team, etc. These areas of focus will be highly dependent on the scenario but could include components that you view as particularly critical to the success of the program and/or components where you anticipate having problems. As much as possible your process goals/evaluation criteria should be mapped out in advance, but you should also make sure you revisit them throughout the communication process, because chances are that you are going to encounter barriers (or opportunities!) that you did not anticipate, and you may even have to adjust or change your goals accordingly. One component of process evaluation that is often ignored is efficiency. Did you manage to implement your program and achieve your goal with the least amount of wasted time, energy, money, etc.?

The outcome

Outcome goals and evaluations are focused on the impact of your plan on its target audience. If your goal was to inform, did people understand more about the issue in question after versus before your communication effort? If your goal was to change behavior, did the behavior of your target audience actually change? ‘Unmonitored for outcome, risk communication consumes and wastes valuable resources, are ineffective, and create a false sense of achievement on those who are responsible.’ – Gaya Gamhewage, World Health Organization That being said, these are lofty goals and can be very difficult to measure. That does not mean that you should not strive for them, but it might be wise to come up with some creative but still impactful outcome aims. For example, did your communication effort increase your audience’s access to information about the issue in question? Did people feel empowered as a result of your communication? If you are involving stakeholders in the communication process, an aim could be whether those stakeholders felt that they were engaged in a positive and meaningful way.

Once you have decided on your goals and evaluative criteria, you will need to outline how you will conduct the evaluation. Again, this is important to do at the beginning because it will help you to assess the feasibility of your goals and evaluative criteria.

Communication program evaluation

Timing

There are three key times when an evaluation can take place:

Before

This is called pre-testing. This involves implementing all or part of your communication plan on representatives from your intended audience (ideally). For example, perhaps you might have a focus group with stakeholders to review your risk communication materials and make sure that everyone understands the message and finds it compelling.

During

It can be helpful to have a mid-point review. You often can’t know what works and what does not until you have started to implement. Additionally, there may be unforeseen barriers and opportunities. This is usually more of a process-focused check-up than a full evaluation — the purpose is to make sure that everything is on track. For programs that take a long time to bear fruit, you may need several mid-point check-ins.

After

Once there has been time for your communication effort to bear fruit, it is time for a more formal and in-depth evaluation. This is often focused on outcomes, but, don’t forget the process component because the two go hand in hand. Without a process review, it will be hard to identify the root causes of your success or failure.

Data sources

A process evaluation relies largely on internal data sources, for example risk communication plans, messages, and communication products (i.e., media interviews, written and visual materials, internet posts, etc.), and interviews with those involved in the communication process.

An outcome evaluation, on the other hand, relies largely on external data sources such as surveys, interviews, and focus groups with segments of your target audience, usage tracking for websites and social media, and data to identify a change in behavior.

Process

This is not rocket science! Program evaluations are often omitted from communication plans and we think it is because people make them more complicated than they need to be. For some reason you use the word ‘evaluation’ and the tendency is to think that is needs some complicated, energy-draining, and time-consuming process. In reality, if you need this sort of process, your goals and evaluative criteria are probably way too complex. The goal of your evaluation is not to completely, utterly, and undeniably prove that you have achieved your goal, but rather to provide reasonable evidence to support that you made steps in the right direction.

Step 1: Identify the goals for the evaluation. These will be derived from your process/outcome goals and evaluative criteria.

Step 2: Determine what data you will need to perform the evaluation.

Step 3: Collect those data.

Step 4: Analyze those data.

Step 5: Draw conclusions and act on those conclusions by modifying your plan.

Example

We developed a communication campaign for rural Sri Lankans in communities with a high incidence of dog and human rabies.

The purpose of this campaign was to educate adult community members about rabies. A process goal was that 75% of adults in the target communities had seen and read the poster. An outcome goal was that people who had read the poster demonstrated an increased understanding of the risks associated with rabies and how to mitigate those risks. The process evaluative criteria was self-reported rates of seeing reading the poster using one-on-one interviews conducted after the communication campaign. The outcome evaluative criteria was a demonstrated increase in understanding regarding rabies risk and risk mitigation between one-on-one interviews conducted before and after the communication campaign.

The evaluation process was as follows:

Step 1. The goal of the evaluation was to estimate the proportion of the target audience who had seen and read the posters and to assess whether or not those who had read the posters demonstrated an increased understanding of rabies.

Step 2. Data were collected through one-on-one interviews with a subset of adults identified by a community leader.

Step 3. Participants were interviewed before and after the educational campaign.

Step 4. Data were analyzed quantitatively and qualitatively.

Step 5. Data were used to determine whether goals had been met and to identify whether important gaps remained.

Takeaways