Blog: How to evaluate PR campaigns – excerpt from Making Waves

In the latest excerpt from his forthcoming book ‘Making Waves: PR strategies to transform your maritime business’ Polaris MD Ben Pinnington examines the area of campaign evaluation.

Evaluating PR campaigns is one of the hottest topics in PR and what I provide here is an introduction to a very big debate. 

Evaluation has been seen as a weakness of the PR industry. Too often PRs who pride themselves at storytelling do not tell the story of the campaign from the data and intelligence they have, or in some cases not, collected.  Evaluation can be seen as dull and esoteric when it is actually one of the most important elements of any campaign. Evaluation is the business end of the campaign where you can track progress in line with comms plan objectives. We advise the CEO and the PR team work together to understand how the campaign will be evaluated – it is basic good management and may help in setting the comms plan objectives in the first place.

You want to understand what you are achieving, what is the return on investment and what you are doing with the findings.   This final point is critical because one of the prime benefits of evaluation is to adapt and improve the campaign. 

It is also worth recognising how nuanced PR has become – it is not just about media relations. The media landscape has changed so much with PR opportunities abundant on owned media (own websites, blogs, social media) as well as ‘paid for’ promotion on channels like LinkedIn (as discussed in the social media chapter) and mainstream media. This has led to a new measurement definition being created called paid, earned, shared and owned (PESO). The concept gained increased recognition when used by Gini Dietrich  in her 2014 book ‘Spin Sucks: Communication and Reputation Management in the Digital Age’ PESO is applied to integrated marketing campaigns and examines the strengths and weaknesses of media types. The below graphic and grid show the different forms of PESO and the benefits and weakness of each type of media in terms of trust, scale, cost and predictability.

Graphic from spinsucks.com

 

Grid by Ketchum Inc https://wadds.co.uk/blog/peso-for-marketing-and-pr 

As well as being sober about the impacts of different types media it is also worth being realistic about what can be achieved when you evaluate your campaign, hence why evaluation can inform the creation of objectives. Ensure you do not overestimate PR either by setting impractical coverage expectations or by expecting PR to change highly entrenched views. 

But to start the evaluation process we advise following the excellent work of the International Association for Measurement and Evaluation of Communication (AMEC) which is driving PR measurement techniques. It really is worth taking time to explore the AMEC website for the latest commentary, case studies and in depth reports on the field of evaluation.  AMEC advises to deliver effectiveness, you should see outputs, out-takes, outcomes as a chain – one leads to the next and together they help to deliver the desired impact. They are defined as follows:

Outputs: When media cuttings folders or social media reports are produced it is a measurement of output only.

Outgrowths:  What takes place after a PR campaign has been in place for some time. When people start to understand and remember your messaging.

Outcomes: The next stage when the understanding grows into trust and people change their attitudes or behaviour because of the PR activities.

Impact:  Is the ultimate follow-on results related to your objectives which your communication achieved or contributed to.


Source AMEC

Use these measurements to evaluate how you are meeting your comms plan objectives and generally if you are experiencing the benefits and outcomes/impacts of PR which can also include: winning of awards, willingness of stakeholders to engage with you personally and on social media and support you in difficult times, greater interest in your business from the media with increased take up of your press releases and more requests for interview and comment, increase in the value of your business and your price points, approaches to speak at industry events and join committees, proactive feedback on your publicity from stakeholders, new business leads.

Some tips to help this process:

Outputs – publicity

Maintain reporting structure. Here the PR consultancy or press office can submit data on how many press releases have been issued and what coverage has been generated.

Use a smart reporting tool. We upload all media coverage using the CoverageBook software. This crunches the readership of print and online coverage. In addition, it calculates shares on Twitter, Facebook but not LinkedIn. It can further be accessed in real time via a permanent link as it is updated.

Note print readership is calculated by the number of readers per copy. The industry standard is to multiple the circulation by between 2.5 and 4 readers -meaning for each copy sold it is estimated that up to four people will read it. 

Outgrowths/outcomes/impacts – publicity

Quality of coverage. Simply looking at the quantity of coverage and the Advertising Value Equivalent (AVE) of cuttings are a blunt instruments for measuring for the impact of the PR campaign. Coverage can afterall include negative stories, moreover in terms of readership while you may hit high numbers in the maritime press you may actually be reaching many of the same people so called ‘cross reading’. So, for example, some people who read Tradewinds are also reading Lloyds List. As a result audience figures, while a useful barometer, are an inexact science. 

The benefits of PR are not in the AVE – but in what the PR achieves in influencing attitudes, opinion and behaviour. PR is different to advertising, it has the impact of impartiality because a journalist has independently decided to run the story for free it is not paid for.  It is the quality of the coverage that counts most. And this is what you need to capture and report back.

Is the company appearing in titles read by your target audiences, is the messaging aligned to your comms plan objectives? 

Keep in mind that one article or feature interview in say the Financial Times, Ship Repair Journal or Tradewinds could be more valuable to your company than numerous articles in local press if it reaches more of your target audience.

A single article or particular story may have disproportionate impact on stakeholder attitudes and that is usually the type of ‘outcome and impact’ you are looking for. Think of Princess Diana and the Martin Bashir interview – Diana generated huge volumes of coverage but that one interview and story had more impact than countless other reports and stories at winning public sympathy and support for her. Conversely when the UK Prime Minister Boris Johnson’s then chief of staff Dominic Cummings broke the lockdown rules by visiting his family in the North of England coverage around this story had a big impact on the Government’s credibility and the authority of its stay at home message. The outcome and impact, although very powerful in influencing attitudes, was opposite to what the comms team was trying to achieve – and apparently the Government had spent £482m on the Covid 19 public safety information campaign! So it is very important to assess media coverage in the right way not solely volume and high readership rates – but impact.

In terms of quality analysis you could start a points based system for marking each piece of coverage starting with the importance of the publication then looking at each piece of coverage and giving marks for inclusion of a product, picture or senior person you want to position as a key person of influence. How big is the report, is it on the stronger right hand page in print or is it positioned prominently on a news website or enewsletter. Does it include your key messages and any independent commentary on your status in the sector positively or negatively. 

Survey attitudes. If a prime ‘outcome and impact’ benefit of PR is to change attitudes and behaviours and improve reputation it is worth finding out what people think. You can employ a third party to undertake surveys or focus groups for you and the costs of these have reduced considerably in recent times.

In politics polling is constant but much less common in business particularly in the maritime sector. For example, the former leader of the Scottish Conservatives Jackson Carlaw quit after only six months in the job in July 2020 after polls revealed that he was failing to halt the growing popularity of the SNP (https://www.scotsman.com/news/politics/analysis-why-are-scottish-tories-failing-make-inroads-snp-dominance-3100565). How many businesses would be bold enough to seek this kind of feedback on the impact of their messages and leaders? In another example of the influence of polling the UK Labour Party leader Sir Keir Starmer apparently strikes a more patriotic tone, by giving press statements in front of a Union Jack for example, because a section of Labour’s voter base disliked Jeremy Corbyn for his supposed lack of patriotism. Polling has created a change of message and strategy at the heart of the Labour leadership. Learning from this is it worth you surveying your customers to identify messages which score and miss with your customers and adapt your strategy and PR accordingly? (https://www.thetimes.co.uk/article/5eb7c90e-66c3-11eb-908c-00b0fcb974f6)   

You may feel it is a lot of effort to hold a stakeholder focus group so why not start with a short survey of your database or with a trade body. Then if it reveals useful information about the ‘outcomes’ of your PR campaign you could try a professional polling body. It may not always make comfortable reading but it may make you more profitable if you understand your customer better. 

It is worth considering surveys at the start of the PR campaign, or the beginning of the year so you understand your current position and then repeat it at six to 12 month intervals to gauge impact.

Survey at mega maritime trade fairs

A brilliant place to undertake surveys is at the big maritime trade fairs around the world. Organisers at SMM in Hamburg send out groups of young people armed with ipads to capture delegates views of the event. The trade fairs present a great opportunity to speak to the industry on mass face to face and gain invaluable insight into you and your particular niche in the maritime industry. But one piece of advice – key your surveys short and quick!

Outputs – social media 

There is a fantastical array of reporting tools for social media. Our advice is to keep it simple and avoid ‘data puking’ as the industry calls it. You do not want reams of reports which no-one reads you want clear data that tells you about the ‘outcomes’ and ‘impact’ you are having.

Start monthly social media reports detailing follower numbers, impressions and top posts. It is important to do this for LinkedIn and Twitter which supply good reporting data which can be exported into reports. Check follower numbers against targets.

Hit rates to the website. Is this going up with the PR campaign? What are the referral sites? Is your earned media coverage in key maritime titles for example taking readers to rich content on your website? Use Google analytics so you can identify which news websites drive traffic to your website so you can target them. 

Outgrowths/outcomes/impacts – social media

Report the conversation you are having on social media. It is no use simply reporting followers, impressions, likes and retweets in bald numbers. Which journalists and stakeholders are being engaged? What are they saying and is this aligning to your objectives gaining support for your messages? What type of themes are most popular, what posts are dud? How can you adapt your campaigns to gain more traction? Is there a time of day where you gain more interest? Is the social media campaign creating leads and opportunities in line with objectives? 

Outgrowths/outcomes/impacts – anecdotal feedback

Perhaps the one area most deficient in PR reporting, in our experience, is the collection of anecdotal feedback. Companies must document this critical feedback with their PR firm or comms team so it can be presented in reviews. A Google cloud shared between all relevant parties could be one vehicle with sections for opportunities created and feedback received.

L-R James Troop’s Robert Pollock, Bob Troop with Defence Minister Philip Dunne and Derek Bate also of James Troop

Reports from the frontline of the business – meetings, trade shows, networking, gatherings with friends and family, approaches from politicians.  Find a way to capture precious pieces of intelligence from these sources – even if it sounds a bit Stasi-like! We have seen campaigns, for example, encourage greater buy-in to clients from local and national politicians who have requested more access to the business as a result of a higher profile as well as a willingness to support clients by writing to ministers and by asking questions in Westminster. As an example we wrote a press release for well known Liverpool ship engine company James Troop detailing a £2million deal it had won to supply engines for Royal Navy minesweepers. As a result of sending the PR the MOD to include a quote the then minister for defence procurement Philip Dunne came to visit the company to personally thank the whole team in a company address! He also sat with management discussing future opportunities in the procurement process. What a result!

Business leads. Ensure the senior leadership team and sales team ask leads where they heard about the company and feed this back so the publications, awards, trade associations and social media platforms that yield leads can be documented. This is absolutely critical to helping show a financial return on investment.

Opportunities created. In our experience PR can open a lot of doors especially if your PR consultant is willing and able to open their contacts book. PR driven opportunities can come in many forms from being approached to join stakeholders on a committee or giving a talk to a relevant business group or simply being asked for a coffee by an influential stakeholder.

Leave room for flexibility in the objectives in your campaign to let PR work its magic and document it in this anecdotal evidence section. 

Another example is a campaign we undertook for a newly established ship repair business. We ran an 18 month publicity campaign for them and their smart objectives included engaging high level decision makers in the maritime industry, winning business and raising awareness of their new fabricating and welding facility. We opened our contacts book and made direct introductions to contacts in the shipbuilding industry which led to high level meetings and helped pave the way for major contracts and regular work.  The company also usefully expanded their profile and network by joining a trade body we recommended. We further generated regular publicity often with stakeholders quoted in the press releases. All demonstrable outcomes and impacts from the PR campaign. 

Another time we introduced a client to a trade body which led to them meeting a key target stakeholder in BAE Systems and as a result pitching for big long term contracts. A key part of the campaign objectives was to help them raise awareness in the naval sector and win business so a clear outcome and impact. 

PR moves in mysterious unpredictable ways and often it is about putting yourself out there in the media and finding what tradewinds start catching your sails to help meet your objectives. But it is important to record all the achievements so you know the sources and messages that are driving the ‘outcomes’ and ‘impacts’. 

Case study International Safety Products

Objective 

Polaris was appointed by the world’s biggest lifejacket maker Liverpool based International Safety Products (ISP) to support the sale of the business. The two owners were nearing retirement and prime objectives of the campaign included raising awareness of the business in the international marine industry to attract high quality buyers and maximise the sale price. A problem we needed to tackle was the low profile of the business and lack of recognition in the industry as to how well they were performing. 

What we did

For two and half years we promoted the ISP’s successes to the regional press and international marine media including key marine titles like Boating Business and International Boat Industry magazine read by their target audiences. The regional press coverage proved impactful at reaching international audiences as in-depth interviews were retained online on highly optimised news websites like the Liverpool Echo. A factor prospective buyers from around the world commented on when they Googled ISP as part of their due diligence ahead of meeting the owners. 

The media coverage consistently focused on key messages around the growth and profitability of the company, its position as a market leader, its trusted status as a business with a 30 year heritage, the calibre of its client base including the UK Ministry Of Defence, as well as the Australian, New Zealand, South African and Pakistan navies. Importantly the publicity emphasised the strength of its exports which accounted for 70pc of its sales built on existing clients and an established distributor network. 

We further deployed press materials on owned channels such as social media, its website and ran a regular enewsletter to its stakeholders. 

Other activity included inviting journalists for the first time to its popular equipment demonstration days at Liverpool’s Albert Dock and providing PR support at trade fairs like METs in Amsterdam and DSEI in London. This ensured its stands were professionally promoted in the media for the first time to stakeholders with press releases appearing on owned, shared and earned media around the shows. We further boosted awareness by organising press calls and journalist visits to its stands. 

Outcome and impacts what the client Geoff Billington ISP director said:

“In 2013, International Safety Products decided to use an external source to extensively promote and tell the industry and community about the successful business that was undertaken at our factories on Merseyside. We had a very low profile and felt we were doing a lot better than perceived by our stakeholders and customers. We interviewed several candidates and were immediately bowled over by the enthusiasm which came over at our initial meeting with Polaris. Polaris offered us a complete package and immediately set about improving our external profile. We constantly appeared in the marine press , our MD was featured on mainstream BBC news, we had coverage on Radio City and also German TV. When came to exit the business in 2015 , the shareholders recognised the contribution that Polaris had made in helping International Safety Products attract top class interest in acquiring our business.”  

Other evaluation tips

Benchmark competitors. To assess how you are performing you can quite easily benchmark yourself against competitors by setting up a media monitoring service through cuttings agencies like Gorkana or starting your own in-house monitoring using Google News alerts and harvest weekly or monthly reports on their social media activity.  Your PR consultancy can pull all this together say for a two or three month period to give you an idea of the volume of PR competitors are putting out, the amount and quality of media coverage they are generating and what themes they are focusing on.

Think long term

As we have discussed PR is a slow burn and the process of securing ‘outcomes’ and ‘impacts’ is different with every business. In our experience you need at least six months of consistent PR to make headway, you will then climb up the ladder of changing attitudes further after 12 months and 18 months. The key is then to maintain awareness and momentum, as without profile you are quickly forgotten.

 

  • Evaluation is changing it is no longer enough to ‘send stuff out’ and collect cuttings.
  • Evaluation is the business end of the campaign where you can track progress in line with comms plan objectives.
  • Be realistic about what can be achieved. Use evaluation to inform comms plan objectives.
  • Follow evaluation techniques pioneered by organisations like AMEC which advises to deliver effectiveness, you should see outputs, out-takes, outcomes as a chain – one leads to the next and together they deliver the desired impact.
  • A prime benefit of evaluation is to adapt and improve the campaign.
  • It is the quality not quantity coverage that counts. Is the company appearing in titles read by your target audiences, is the messaging aligned to your comms plan objectives? 
  • A single article or particular story may have disproportionate impact on stakeholder attitudes.
  • Avoid data puking. Report the conversation you are having on social media not simply followers, impressions, likes and retweets in bald numbers.
  • Collect anecdotal feedback from meetings, trade shows, networking, gatherings with friends and family and politicians.
Menu