Evaluating external communications

The Communications Maturity Model described in section 1.2 can be used to evaluate the effectiveness of an external communications function. The model can be used to evaluate the effectiveness of the organization’s communications function and to identify areas for improvement. This chapter explicitly examines the importance of evaluating communications activities, and the ways in which evaluation can be conducted.

All communications activities should be measured and evaluated to confirm that they achieve communications objectives. As Peter Drucker famously said, “what gets measured gets done.” Measurement and evaluation should be a consideration throughout a communications activity and should not wait until an activity has concluded.

Done effectively, measurement leads to iterative improvements to communication. Evaluation can guide future activities and strategy development. Measurement also presents opportunities for continuous improvement and helps organizations develop an understanding of the impact of communications activities. In turn, an evaluation demonstrates the tremendous return on investment that good communication can provide.

The most common form of measuring communication activities is the use of Key Performance Indicators or KPIs.   According to the website KPI.org, “KPIs are the critical (key) indicators of progress toward an intended result.  KPIs provide a focus for strategic and operational improvement, create an analytical basis for decision making and help focus attention on what matters.

KPIs should be identified at the start of the activity, not when the final results are in. KPIs should demonstrate the impact of the activity on organizational goals and, if they are already in place, provide an effective benchmark that allows for easy tracking of improvements and progress.

When developing KPIs, it is important to recognize and distinguish between output measures and outcome measures. Output measures track the statistical organization’s activities. These are typically reasonably easy to collect. Outcome measures track the effect those activities had on the intended audience. These are often difficult, sometimes impossible, to collect. Nonetheless, outcome measures are the gold standard of evaluation.

Three types of common communication measures can become KPIs:

• Activity-based―if no other measures are available, report what you did (e.g., three newsletters, 10 seminars and seven webpages).
• Measures of communication channels―quantitative measures (e.g., number of webpage hits and views, dwell times, phone calls, attendees, Twitter retweets/likes, Facebook comments/followers) indicate the uptake of information.
• Analytical―these measures bring quantitative measures together and complement them with qualitative understanding. These are the most detailed measures, and they provide a deeper understanding of performance. They address awareness, understanding, behavioural change, sentiment, share of voice, ownership and the difference between creating and informing the news.

Using a combination of measures, particularly with an increased use of analytical measures, can help to evaluate the contribution that communication makes to the overall project outcome.

It is important to recognize the difference between communications objectives and business/program objectives. Communications objectives typically focus on what the audience is expected to think, feel or do differently because of the communications activities. Business/program objectives present the results that a manager hopes to achieve in a project or business enterprise. Recognizing the difference allows for a separate and objective evaluation of each set of objectives. For example, communications objectives could be met, but the project may not have met all its business/program objectives. This usually means that the initial business/program objectives were faulty.

Thinking about objectives can help identify issues or areas for improvement within the communications or program implementation. It also promotes the maturity and professionalism of the communications industry.

A number of questions can be asked to help assess the success of communications objectives separately from the success of the business or project. These include

• Did you reach the right audience?
• Did you use the right communications tools and/or channels?
• Did your audience understand your messages?
• Were decisions taken as a result of your messages?
• Did the target audience take action as a result of your messages? Was it the desired action?
• Did you comply with the budget? If not, why?
• What would you do differently next time?

In addition to these questions, consider the following PROOF principles when measuring or evaluating communications activities:

• Pragmatic: Use the best available information source. Do not seek to generate perfect numbers instantly. Metrics should be fit for purpose. It is important to use what you have and to improve it through iteration.
• Realistic: Always seek to prove the things you can or acknowledge those you cannot. Evaluations should stick to the facts and only state what you know. Do not extrapolate meaning or conflate correlation with causation.
• Open: Record and share as much as possible. Do not hide results. Remember that communication objectives and business/program objectives are different, albeit closely linked.
• Objective: Remain honest now to learn for the future. Recognize both successes and failures. Record lessons learned.
• Fully integrated: Make evaluations ever-present, not add-ons at the end of an activity. Monitoring and evaluation should be embedded into your communications strategy, which is why you should start monitoring and evaluation at the beginning of your project.

Figure 9 presents a method to align objectives with activities and lessons learned.

Back to top

Figure 9  Examples of communication evaluation

Back to top