Survey
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
Sigried Caspar European Commission, DG Employment & Social Affairs Moderator Céline Mas Occurrence communication research & consulting company, France Workshop A7: how to evaluate your communication activities 2014-2020 December 10, 2013 – Brussels Céline Mas Occurrence est certifiée ISO 9001 depuis 2004 "Half the money I spend on advertising is wasted; the trouble is I don’t know which half." John Wanamaker Inventor of mass retailing in the United States Occurrence - Workshop A7 - 10/12/2013 Key issue > Communications are under pressure: - How much does it cost? How much does it contribute? € What does it contribute? > View it as an investment and not as an expense > Provide the resources to prove your effectiveness: 5% to evaluation. > "How much does overlooked inefficiency cost?“ : if you can not assess it, you can not improve it! Occurrence - Workshop A7 - 10/12/2013 5 The missing link in a virtuous circle Communication plan Most of the time, evaluation is occasional or partial. A goals and resources « contract » with institution or company management It rarely shapes dialogue between the communications team and the other decision makers. Assessment Reporting on the achievement of the targets, or on the progress and effectiveness of the implementation Occurrence - Workshop A7 - 10/12/2013 Actions A goals and resources « contract » with company management Behaving like any other function > Communications must not exclude itself from Quality and Operating Excellence systems > Communications is a job and a skill; it must include ongoing improvement procedures The Deming Wheel Quality management system Occurrence - Workshop A7 - 10/12/2013 7 The 4 main benefits of an evaluation 1. PROMOTING 2. MANAGING • Circulating results and performances • Allocating resources in accordance to other departments/teams with performance indicators • Achieving investment choices that are • Identifying the most effective initiatives based on targets, and not on expenditures on resources for achieving your various objectives 4. SHARING 3. SAVING TIME • Prioritizing/Sorting initiatives by order • Gathering all the activity and of effectiveness • Concentrating your efforts and budget on the most effective initiatives • effectiveness data • Highlighting best practices AND SPECIFIC TO THE PUBLIC SECTOR Giving evidence of a sound use of public money • Reinforce citizen’s trust Occurrence - Workshop A7 - 10/12/2013 8 2 main categories of performance (KPI) indicators Activity including Resources (What?) Effectivenes s including the Audiences (For what purpose?) e.g. Number of initiatives/tools, type of initiatives/tools, assessment of the content issued (Press releases, and internal communications), Ressources: Who? How much? How long? e.g. Memorization, Understanding, Buy-in, Incentive, Transformation, Satisfaction, Improving the brand's image, and satisfying internal customers Audience: How many people attended? How many Likes ? How many readers? … > A third, highly practical approach is possible: assessing the satisfaction of (internal) customers > Defining performance thresholds for each indicator Occurrence - Workshop A7 - 10/12/2013 9 Communications Dashboard communications plan Process : global picture Target audience Communications objectives Segmentation 1. Define 2. Count 3. Contact details Opinions/perceptions to share Do you agree with the following assertion ? Surveys, research - "ABC respects environment” media analysis, - … Yes observations Indicators - x 1 [File qualification] - "ABC respects environment” 60% Current score: 60 % Occurrence - Workshop A7 - 10/12/2013 Targeted score : 90 % x N [Sample survey] Gap between results and objectives !! ! OK Yes : No : Don’t know : Management • • • • • Allocate Understand Adjust Maintain Etc… Advice > Efficiency means producing the desired effects on the desired target audiences > Therefore, you need to define the target audiences that you want to reach with which effect, prior to the initiative, and ideally to define the performance threshold > As a starting point: KISS (Keep It Simple, Stupid) - Keep things simple at the beginning > BUT assess them regularly > And don't change the assessment system for each evaluation > Do not restrict the evaluation to the activity, in order not to limit communications to initiatives and tools > Design SMART (Specific, Measurable, Achievable, Realistic, and Timed) tools and goals for each initiative > Share the results and the decisions they help to take in order to enhance the value added of evaluation Occurrence - Workshop A7 - 10/12/2013 11 Thank you > Contact : Céline Mas > Partner & Research Director Occurrence > [email protected] Occurrence - Workshop A7 - 10/12/2013 Brussels, 9-10 December Ben Ward Head of Evaluation and Research, Coffey International Development, UK Instructions for use • This template has been built with custom styles and formats aligned to the new brand identity. • All charts, org charts, tables and smart graphics have the custom colour palette and font styles embedded. Choose the style that best suits the presentation of your information. • Evaluating Inserting chartscommunication as images and including photography will increase the file size – include lower resolution files. • The Coffey orange should only be used in headers, sub headers and bullets – avoid using it in charts, tables and graphs. Ben Ward • December Secondary colours are only used in charts, tables and graphs 9th 2013 – avoid using as headers, sub headers and bullets. • Bolder tones are best used for charts – as they appear clearer on screen. • Please do not change the primary colour set ups. • Headlines on pages: if they are one line in length please ensure they are presented in black. If the headline/title splits across two lines – please retain the first line in black and put the second line in the Coffey Orange (select from custom colour palette). IDEV Executive Workshop 4 November 2013 16 Evaluating communication activities Contents • Who we are and what do we do? • Challenges faced by public sector communicators • Evaluation - hints and tips • Discussion Telling the story conference December 2013 17 Evaluating communication activities Coffey Evaluation and Research Example I&C assignments • Founded in 1990, acquired by Coffey International Development in 2009 • Evaluation on EU Regional Policy communication (DG REGIO) • Team of 30 international and multilingual M&E professionals • Work in EU, UK domestic, and international development evaluation markets Telling the story conference December 2013 • Ex ante evaluation of campaign on violence against women (DG JUSTICE) • Evaluation of ESF communication activities (DG EMPL) • Evaluation of DG TRADE’s communication strategy 18 Evaluating communication activities - Challenges • The communication challenge in the public sector is greater than for business communicators Greater resource constraints Extreme media scrutiny Public sector comms Legal constraints Political agendas (Adapted: Liu and Horsley, 2007) Telling the story conference December 2013 19 Evaluating communication activities - Challenges Telling the story conference December 2013 20 Evaluating communication activities – Hints and tips 1. Take an evidence based approach to communications design • Design of any communications campaign should be driven by good evidence • The evidence base: o Primary and secondary o Quantitative and qualitative o Research areas: - Build deep knowledge around what you are communicating – e.g. the problem itself - Defining target audience(s) - Appropriate communication tools and activities Telling the story conference December 2013 21 Evaluating communication activities – Hints and tips 2. Evaluation should start at the very beginning • Set the objective(s): What are you trying to achieve? • Develop an intervention logic and SMART indicators • Mapping how you will get to your desired outcomes (use an evidence base to inform this) • Workshop - communications team and other relevant stakeholders “He who fails to plan is planning to fail”. Telling the story conference December 2013 22 Evaluating communication activities – Hints and tips 3. Effective evaluation: Focus on measuring the outcomes • Measure as much as you can with the resources available • Outputs and outtakes both have their place (So what?) • However, achieving outcomes ultimately help to justify the use of communications • Producing a lot of material, such as leaflets and posters, and getting out the right messages is no good if you fail to get the final result!! Telling the story conference December 2013 23 Evaluating communication activities – Hints and tips 4. A few final tips…….. • Recognise what you can and you cant do with your evaluation (but also with your comms!) • • Take a proportionate approach to evaluation o Size of intervention o Repeatability / Scalability factors Use mixed methods (Quant and Qual) Telling the story conference December 2013 24 Evaluating communication activities Thank you [email protected] Telling the story conference December 2013 25 Brussels, 9-10 December