Excerpt from Polaris MD Ben Pinnington’s forthcoming book Making Waves: PR strategies to transform your maritime business’ which will be published by Rethink Press in summer 2021.
Evaluating PR campaigns is one of the hottest topics in PR and what I provide here is an introduction to a very big debate.
Evaluation has been seen as a weakness of the PR industry. Too often PRs who pride themselves at storytelling do not tell the story of the impact of campaign from the data and intelligence they have collected. Evaluation can be seen as dull and esoteric when it is actually one of the most important elements of any campaign. Evaluation is the business end of the campaign where you can track progress in line with comms plan objectives. We advise the CEO and the PR team work together to understand how the campaign will be measured – it is basic good management and will help in setting the comms plan objectives in the first place.
You want to understand what you are achieving, what is the return on investment and what you are doing with the findings. This final point is critical because one of the prime benefits of evaluation is to adapt and improve the campaign.
It is also worth recognising how nuanced PR has become. The media landscape has changed so much with PR opportunities abundant on owned media (own websites, blogs, social media) as well as ‘paid for’ promotion on channels like LinkedIn (as discussed in the social media chapter) and mainstream media. This has led to a new measurement definition being created called paid, earned, shared and owned (PESO). The concept gained increased recognition when used by Gini Dietrich in her 2014 book ‘Spin Sucks: Communication and Reputation Management in the Digital Age’ PESO is applied to integrated marketing campaigns and examines the strengths and weaknesses of media types. The below graphic and grid show the different forms of PESO and the benefits and weakness of each type of media in terms of trust, scale, cost and predictability.
Graphic from spinsucks.com
|Earned||High||medium||Low to medium||No|
|Owned||Low to medium||Low||Low to medium||Yes|
Grid by Ketchum Inc https://wadds.co.uk/blog/peso-for-marketing-and-pr
As well as being sober about the impacts of different types media it is also worth being realistic about what can be achieved when you evaluate your campaign, hence why evaluation can inform the creation of objectives. Ensure you do not overestimate PR either by setting impractical coverage expectations or by expecting PR to change highly entrenched views.
But to start the evaluation process we advise following the excellent work of the International Association for Measurement and Evaluation of Communication (AMEC) which is driving PR measurement techniques. AMEC references Alex Aiken, Executive Director for Government Communications at the UK Government, who talks about SOS – sending out stuff (or sometimes a less polite ‘S’ word! L) and the perils of being a PR machine that churns out content and ticks boxes rather than focusing on outcomes. It really is worth taking time to explore the AMEC website for the latest commentary, case studies and in depth reports on the field of evaluation. AMEC advises to deliver effectiveness, you should see outputs, out-takes, outcomes as a chain – one leads to the next and together they help to deliver the desired impact. They are defined as follows:
Outputs: When media cuttings folders or social media reports are produced it is a measurement of output only.
Out-takes: What takes place after a PR campaign has been in place for some time. When people start to understand and remember your messaging.
Outcomes: The next stage when the understanding grows into trust and people change their attitudes or behaviour because of the PR activities.
Impact: Is the ultimate follow-on results related to your objectives which your communication achieved or contributed to.
|Outputs include||Out-takes are your target audiences initial responses and reactions to your communications. These may be interim Steps towards your objectives not necessarily achievement of your objectives. outtakes may include evidence of:||Outcomes of the effect that your communication has on your target audiences that align to your objectives. Examples of outcomes of communication include:||Impact Is the ultimate follow-on results related to your objectives which your communication achieved or contributed to. Impact can include:|
|Publicity information||audience attention for example: unique visitors, views, click throughs||Learning/knowledge E.g. through survey or interview data quizzes or tests||reputation improvement recognised as an intangible asset|
|Websites blogs partnerships or supplements||Awareness e.g. recall||Trust e.g. increased trust in surveys||Relationships established or improved also recognised as an intangible asset by the international integrated reporting council|
|Events held||Understanding e.g. comments||Preference e.g. in surveys or social media comment||Reaching targets e.g. sales revenue, fundraising or membership targets, health campaign targets such as reduced smoking|
|Sponsorships launched||Interest and liking e.g Likes, follows,
|intention e.g. through surveys,inquiries, trialing or registration||increased staff loyalty or retention E.g. reduce staff turnover and recruitment costs|
|Engagement e.g. return visits, positive comments, subscribing||Attitude change through surveys or interview data||Organisational change e.g. insights to inform future strategy, realignment of policies to stakeholders|
|Complying behaviour e.g. sales, donations driving safely, voting||Social change e.g. improved health and well being or increased access to information|
|Advocacy e.g. endorsements in online comments|
Use these measurements to evaluate how you are meeting your comms plan objectives and generally if you are experiencing the benefits and outcomes/impacts of PR which can also include: winning of awards, willingness of stakeholders to engage with you personally and support you in difficult times, greater interest in your business from the media with increased take up of your press releases and more requests for interview and comment, increase in the value of your business and your price points, approaches to speak at industry events and join committees, proactive feedback on your publicity from stakeholders, new business leads.
Some tips to help this process:
Outputs – publicity
Maintain reporting structure. Here the PR consultancy or press office can submit data on how many press releases have been issued and what coverage has been generated.
Use a smart reporting tool. We upload all media coverage using the CoverageBook software. This crunches the readership of print and online coverage. In addition, it calculates shares on Twitter, Facebook but not LinkedIn. It can further be accessed in real time via a permanent link as it is updated.
Note print readership is calculated by the number of readers per copy. The industry standard is to multiple the circulation by between 2.5 and 4 readers -meaning for each copy sold it is estimated that up to four people will read it.
Out-takes/outcomes/impact – publicity
Quality of coverage. Simply looking at the quantity of coverage and the Advertising Value Equivalent (AVE) of cuttings are a blunt instruments for measuring for the impact of the PR campaign. Coverage can afterall include negative stories, moreover in terms of readership while you may hit high numbers in the maritime press you may actually be reaching many of the same people so called ‘cross reading’. So, for example, some people who read Tradewinds are also reading Lloyds List. As a result audience figures, while a useful barometer, are an inexact science.
The benefits of PR are not in the AVE – but in what the PR achieves in influencing attitudes, opinion and behaviour. PR is different to advertising, it has the impact of impartiality because a journalist has independently decided to run the story for free it is not paid for. It is the quality of the coverage that counts most. And this is what you need to capture and report back.
Is the company appearing in titles read by your target audiences, is the messaging aligned to your comms plan objectives?
Keep in mind that one article or feature interview in say the Financial Times, Ship Repair Journal or Tradewinds could be more valuable to your company than numerous articles in local press if it reaches more of your target audience.
A single article or particular story may have disproportionate impact on stakeholder attitudes and that is usually the type of ‘outcome and impact’ you are looking for. Think of Princess Diana and the Martin Bashir interview – Diana generated huge volumes of coverage but that one interview and story had more impact than countless other reports and stories at winning public sympathy and support for her. The more recent interview with the Duke and Duchess of Sussex with Oprah Winfrey is a more contemporary example.
Conversely when the UK Prime Minister Boris Johnson’s then chief of staff Dominic Cummings broke the lockdown rules by visiting his family in the North of England coverage around this story had a big impact on the Government’s credibility and the authority of its stay at home message. The outcome and impact, although very powerful in influencing attitudes, was opposite to what the comms team was trying to achieve – and apparently the Government had spent £482m on the Covid 19 public safety information campaign! So it is very important to assess media coverage in the right way not solely volume and high readership rates – but impact.
In terms of quality analysis you could start a points based system for marking each piece of coverage starting with the importance of the publication then looking at each piece of coverage and giving marks for inclusion of a product, picture or senior person you want to position as a key person of influence. How big is the report, is it on the stronger right hand page in print or is it positioned prominently on a news website or enewsletter. Does it include your key messages and any independent commentary on your status in the sector positively or negatively.
Survey attitudes. If a prime ‘outcome and impact’ of PR is to change attitudes and behaviours and improve reputation it is worth finding out what people think. You can employ a third party to undertake surveys or focus groups for you and the costs of these have reduced considerably in recent times.
You may feel it is a lot of effort to hold a stakeholder focus group so why not start with a short survey with trade body. Alternatively you can use www.surveymonkey.com which provides free survey templates and guides to analysing results.
If these tools reveal useful information about the ‘out-takes, outcomes and impact’ of your PR campaign you could then try a professional polling body. It may not always make comfortable reading but it may make you more profitable, if you understand your customer better.
In politics polling is constant but much less common in business particularly in the maritime sector. For example, the former Scottish Conservatives leader Jackson Carlaw quit after only six months in July 2020 after polls revealed that he was failing to halt the growing popularity of the SNP . How many businesses would be bold enough to seek this kind of feedback on the impact of their messages and leaders? In another example of polling power the UK Labour Party leader Sir Keir Starmer apparently strikes a more patriotic tone, by giving press statements in front of a Union Jack for example, because a section of Labour’s voter base disliked the previous leader Jeremy Corbyn for his supposed lack of patriotism. Polling has created a change of message and strategy at the heart of the Labour leadership. Learning from this is it worth surveying your customers to identify messages which resonate with your customers and adapt your strategy and PR accordingly?
It is worth considering surveys at the start of the PR campaign, or the beginning of the year so you understand your current position and then repeat it at six to 12 month intervals to gauge impact.
Survey at mega maritime trade fairs
A brilliant place to undertake surveys is at the big maritime trade fairs around the world. Organisers at SMM in Hamburg send out groups of young people armed with ipads to capture delegates views of the event. The trade fairs present a great opportunity to speak to the industry on mass face to face and gain invaluable insight into you and your particular niche in the maritime industry. But – keep your surveys short and quick!
Outputs – social media
There is a fantastical array of reporting tools for social media. Our advice is to keep it simple and avoid ‘data puking’ as the industry calls it. You do not want reams of reports which no-one reads you want clear data that tells you about the ‘outcomes’ and ‘impact’ you are having.
Start monthly social media reports detailing follower numbers, impressions and top posts. It is important to do this for LinkedIn and Twitter which supply good reporting data which can be exported into reports. Check follower numbers against targets.
Hit rates to the website. Is this going up with the PR campaign? What are the referral sites? Is your earned media coverage in key maritime titles for example taking readers to rich content on your website? Use Google analytics so you can identify which social media and news websites drive traffic to your website so you can target them. Note most visits to websites are still driven by Google search.
Out-takes/outcomes/impact – social media
Report the conversation you are having on social media. It is no use simply reporting followers, impressions, likes and retweets in bald numbers. Which journalists and stakeholders are being engaged? What are they saying and is this aligning to your objectives gaining support for your messages? What type of themes are most popular, what posts are dud? How can you adapt your campaigns to gain more traction? Is there a time of day where you gain more interest (early morning is best we find)? Is the social media campaign creating leads and opportunities in line with objectives?
Out-takes/outcomes/impact – anecdotal feedback
Perhaps the one area most deficient in PR reporting, in our experience, is the collection of anecdotal feedback. Companies must document this critical feedback with their PR firm or comms team so it can be presented in reviews. Using the company CRM system or a Google cloud shared between relevant parties can provide vehicles for logging this information with sections for opportunities created and feedback received.
Reports from the frontline of the business – meetings, trade shows, networking, gatherings with friends and family, approaches from politicians. Find a way to capture precious pieces of intelligence from these sources – even if it sounds a bit Stasi-like! We have seen campaigns, for example, encourage greater buy-in to clients from local and national politicians who have requested more access to the business as a result of a higher profile as well as a willingness to support clients by writing to ministers and by asking questions in Westminster.
As an example we wrote a press release for well known Liverpool ship engine company James Troop detailing a £2million deal it had won to supply engines for Royal Navy minesweepers. As a result of sending the PR to the MOD for a quote the then minister for defence procurement Philip Dunne came to visit the company to personally thank the whole team in a company address! He also sat with management discussing future opportunities in the procurement process.
Business leads. Ensure the senior leadership team and sales team ask leads where they heard about the company and feed this back so the publications, awards, trade associations and social media platforms that yield leads can be documented. This is absolutely critical to helping show a financial return on investment.
Opportunities created. In our experience PR can open a lot of doors especially if your PR consultant is willing and able to open their contacts book. PR driven opportunities can come in many forms from being approached to join stakeholders on a committee or giving a talk to a relevant business group or simply being asked for a coffee by an influential stakeholder.
Leave room for flexibility in the comms objectives to let PR work its magic and document it in this anecdotal evidence section. For example we undertook a campaign for a newly established ship repair business. We ran an 18 month publicity campaign for the company and smart objectives included engaging high level decision makers in the maritime industry, winning business and raising awareness of its new engineering facility. We opened our contacts book and made direct introductions to contacts in the shipbuilding industry which led to high level meetings paving the way for major contracts and regular work. The company also usefully expanded its profile and network by joining a trade body we recommended. We further generated regular publicity often with key stakeholders quoted in the press releases. All demonstrable outcomes and impacts from the PR campaign.
Another time we introduced a client to a trade body which led to them meeting a key target stakeholder in BAE Systems and as a result pitching for big long term contracts. A key part of the campaign objectives was to help them raise awareness in the naval sector and win business so a clear outcome and impact.
PR moves in mysterious unpredictable ways and often it is about putting yourself out there in the media and finding what tradewinds start catching your sails to help meet your objectives. But it is important to record all the achievements so you know the sources and messages that are driving the ‘outcomes’ and ‘impacts’.
Case study International Safety Products
Polaris was appointed by the world’s biggest lifejacket maker Liverpool based International Safety Products (ISP) to support the sale of the business. The two owners were nearing retirement and prime objectives of the campaign included raising awareness of the business in the international marine industry to attract high quality buyers and maximise the sale price. A problem we needed to tackle was the low profile of the business and lack of recognition of its strong performance.
What we did
For two and half years we promoted the ISP’s successes to the regional press and international marine media including key titles like Boating Business and International Boat Industry read by their target audiences. The regional press coverage also proved impactful at reaching international audiences as in-depth interviews were retained online on highly optimised news websites like the Liverpool Echo. A factor prospective buyers from around the world commented on when they Googled ISP as part of their due diligence ahead of meeting the owners.
The media coverage consistently focused on key messages around the growth and profitability of the company, its position as a market leader, its trusted status as a business with a 30 year heritage, the calibre of its client base including the UK Ministry Of Defence, as well as the Australian, New Zealand, South African and Pakistan navies. Importantly the publicity emphasised the strength of its exports which accounted for 70pc of its sales built on long standing clients and an established distributor network.
We further deployed press materials on owned channels such as social media, its website and ran a regular enewsletter to its stakeholders.
Other activity included inviting journalists for the first time to its popular equipment demonstration days at Liverpool’s Albert Dock and providing PR support at trade fairs like METs in Amsterdam and DSEI in London. This ensured its stands were professionally promoted in the media for the first time to stakeholders with press releases appearing on owned, shared and earned media around the shows. We further boosted awareness by organising press calls and journalist visits to its stands.
Outcome and impacts what the client Geoff Billington ISP director said:
“In 2013, International Safety Products decided to use an external source to extensively promote and tell the industry and community about the successful business that was undertaken at our factories on Merseyside. We had a very low profile and felt we were doing a lot better than perceived by our stakeholders and customers. We interviewed several candidates and were immediately bowled over by the enthusiasm which came over at our initial meeting with Polaris. Polaris offered us a complete package and immediately set about improving our external profile. We constantly appeared in the marine press , our MD was featured on mainstream BBC news, we had coverage on Radio City and also German TV. When came to exit the business in 2015 , the shareholders recognised the contribution that Polaris had made in helping International Safety Products attract top class interest in acquiring our business.”
Case study Atlantic Container Line royal naming
- Boost local, national and international awareness of the royal naming of the new ACL vessel the Atlantic Sea by HRH Princess Anne the first royal naming of a ship on the River Mersey since 1961.
- Raise global awareness of ACL’s new fleet of five Container/Roll‐on Roll‐off (CONRO) ships coming into service operating on ACL’s North America to Europe route. Show the ships’ capabilities, the type of cargo they carry and which companies could benefit from using the ships.
- Generate media recognition for the spectacular nature of the event and encourage public to attend naming ceremony and evening fireworks display.
- Raise awareness of ACL’s commitment to Liverpool as christening coincided with the opening of a new purpose built European head office for ACL in the city with 170 staff – the first purpose-built shipping line headquarters to be built in Liverpool since 1920.
- Engage stakeholders in the PR around the event including Shipping Minister and port operator Peel Ports.
- Ensured event was reported in key title the Liverpool Echo with large page lead a week beforehand so public had time to plan to attend Christening and fireworks display.
- Communicated key messages around company’s commitment to the city across media including sit down interview with Liverpool Echo and managing director before the event.
- Ensured a range of media attended reflecting the different audiences ACL wanted to reach. This included international media in BBC World TV, regional press in BBC Radio Merseyside, BBC North West TV, ITV Granada TV and the Liverpool Echo.
- We ensured strong industry recognition of the event by inviting key reporters from the international maritime trade offering face to face interviews with senior management from ACL and its owners the famous Grimaldi Shipping Group. Publications attending included Lloyds List, IHS, International Bulk Journal and Freight Business Journal.
- ITV Granada used the fireworks display as the final closing pictures of its evening bulletin giving event and messages around investment in Liverpool and the transatlantic route a high a high degree of recognition.
- Ensured key stakeholders in the Shipping Minister and Peel Ports were quoted in the press release and interviewed by the media.
- Event encouraged positive engagement from reporters and stakeholders on social media.
- Large number of public engaged by attending the events on Liverpool Waterfront.
Further evaluation tips
Benchmark competitors. To assess how you are performing you can quite easily benchmark yourself against competitors by setting up a media monitoring service through cuttings agencies like Gorkana or starting your own in-house monitoring using Google News alerts and harvest weekly or monthly reports on their social media activity. Your PR consultancy can pull all this together say for a two or three month period to give you an idea of the volume of PR competitors are putting out, the amount and quality of media coverage they are generating and what themes they are focusing on.
Think long term
As we have discussed PR is a slow burn and the process of securing ‘outcomes’ and ‘impacts’ is different with every business. In our experience you need at least six months of consistent PR to make headway, you will then climb up the ladder of changing attitudes further after 12 months and 18 months. The key is then to maintain awareness and momentum, as without profile you are quickly forgotten.
- Evaluation is changing it is no longer enough to ‘send stuff out’ and collect cuttings.
- Evaluation is the business end of the campaign where you can track progress in line with comms plan objectives.
- Be realistic about what can be achieved. Use evaluation to inform comms plan objectives.
- Follow evaluation techniques pioneered by organisations like AMEC which advises to deliver effectiveness, you should see outputs, out-takes, outcomes as a chain – one leads to the next and together they deliver the desired impact.
- A prime benefit of evaluation is to adapt and improve the campaign.
- It is the quality not quantity coverage that counts.
- A single article or particular story may have disproportionate impact on stakeholder attitudes.
- Avoid data puking. Report the conversation you are having on social media not simply bald numbers.
- Collect precious anecdotal feedback from meetings, trade shows and networking.