Scottish comms network paul njoku - 14 and 15 may 2014

704 views
777 views

Published on

Paul Njoku's presentation to evaluation workshop organised by Scottish Communicators Network

0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
704
On SlideShare
0
From Embeds
0
Number of Embeds
230
Actions
Shares
0
Downloads
3
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide
  • Outlined on this slide are a few findings from the review which promoted us to act –Hence the project I’m leadingCommunications overall was found to be tactically strong but strategically weak – they were good at what we refer to as S.O.SI’m sure a few of you recognises a few of this issue that we found, particularly 2, 3 and 4.I was brought in just over a year ago to help address some of these issues – and when I started I QUICKLY identified A FEW barriers that were stopping people from evaluating NEXT
  • BUILD SLIDEFor the evaluation of ANY activity to meaningful, relevant and/or effective there has to be what I refer to here as ‘strategic alignment’ …. This stems from having clear SMART business or policy objective which in-turn should inform your comms objectives that then FLOWS and feeds into your individual activity objectivesCLICK (BUILD PART 2)So setting the right objectives that flows from your policy aims is crucial and when doing so, Always make sure your objectives focus on outcomes not on outputs!NEXT – So what have we been up to in the UK
  • BUILD SLIDEFor the evaluation of ANY activity to meaningful, relevant and/or effective there has to be what I refer to here as ‘strategic alignment’ …. This stems from having clear SMART business or policy objective which in-turn should inform your comms objectives that then FLOWS and feeds into your individual activity objectivesCLICK (BUILD PART 2)So setting the right objectives that flows from your policy aims is crucial and when doing so, Always make sure your objectives focus on outcomes not on outputs!NEXT – So what have we been up to in the UK
  • Scottish comms network paul njoku - 14 and 15 may 2014

    1. 1. Scottish Communicators Network Evaluation Workshop Paul Njoku Cabinet Office, UK 14th & 15th May 2014
    2. 2. A • Welcome & Introductions • GCS Mandatory Evaluation Project (MEP) - Overview B • Why evaluate • Barriers to evaluation C • How to go about it • 4 stage evaluation process - The big IDIA • Main performance categories - KPI indicators sources • Take Away - Tips D • GCS Mandatory Evaluation Project (MEP) - Update & Next steps Agenda Group interactive session
    3. 3. Government Communications Review: Summary findings • Tactically strong, strategically weak  Half of departments had no communication strategy • Poor links between policy and communications • Major activity rarely evaluated • Inconsistent and variable standards and approaches • Little join up across departments • Digital skills falling behind private sector
    4. 4. GCS Mandatory Evaluation Project – Overview Mandatory Evaluation Project (MEP) is a key part of Government Communication reform programme. Objective: To create a ‘step change’ in approach to evaluation of communication activities across departments. Four Key deliverables: 1.Develop high-level performance frameworks and dashboards. 2.Introduce mandatory evaluation training across all comms disciplines. 3.Enhance evaluation capability across the GCS. 4.Develop an evaluation ‘centre of excellence’ & promote best-practice. A
    5. 5. Agenda 13.45 – 14.00 Welcome and introduction – Paul 14.00 – 14.10 Scene setting: Conference theme, aims and objective 14.10 -14.30 Guest Speaker: Selvin Brown - DWP Testimony 14.30 – 14.50 Vision, objectives and challenges for Government Communications Alex Aiken, Exec Director of Government Communications 15.00 -15.45 Group Exercise 15.45 – 16.00 TEA BREAK 16.00 – 16.30 Guest Speaker: Spenser Fox (CEO, Reputation Institute Topic: Significance of Reputation as a key outcome measure 16.30 – 16.50 Guest Speaker: Gareth Evans (Independent Consultant-MR2) Topic: Communication ROI case study for public sector 16.50 – 17.05 Guest speakers: Q & A Panel 17.05 -17.15 Evaluating communication activity – Never an optional extra Sean Larkin, Head of Government Communications Policy and Capability 17.15 -17.20 Next Steps & Close Departments & ALBs - Membership 40 Depts. /ALBs 80 ‘evaluation champions’ A1
    6. 6. STEPCHANGE No Evaluation happening at Campaign / Project level Evaluation @ Project / Campaign level but no overall Comms KPI’s (Outcome measures) Evaluation & Comms KPI’s exist but partially aligned with dept. strategic priorities Fully functional Comms performance centre (hub) aligned with dept. strategic priorities Stage 1 Stage 2 Stage 3 Stage 4 Dept. Outcomes measures aligned with Govt. priorities TIME Journey (start)… 39% 5% 56% A2
    7. 7. 7 Why evaluate? B
    8. 8. John Wanamaker Inventor of mass retailing in the United States "Half the money I spend on advertising is wasted; the trouble is I don’t know which half."
    9. 9. Group Exercise (15mins) • Why do we need to evaluate? • What are the barriers ?
    10. 10. Context • Austere times  Need to make every £ count  View as an investment not an expense • Media landscape & consumption patterns  Evidence of what works and what does not  Optimise use of scarce resources • The role of communications under scrutiny  How it supports achievement of policy outcomes  Business planning & activity prioritisation  Sharing best practice B1
    11. 11. 1111 Barriers – stopping it happening Insufficient time / resource / budget Lack of SMART or unrealistic policy objectives / targets Difficulty accessing the right data / tools Culture & entrenched behaviours Gaps in evaluation standards & capability Lack of integrated communication strategy linked to policy objectives B2
    12. 12. 12 Strategic Alignment  Start with the policy aim, then develop communications objectives that help deliver this.  Objectives should be measurable, focused on outcomes not outputs, and related to changing attitudes and/or behaviour. Setting the right objectives is crucial C1
    13. 13. 13 Strategic AlignmentC2 Strategic Objectives & KPIs Comms Objectives, & KPIs Activity objective
    14. 14. Evaluation stages –The Big IDIAC3 Identify The scope of your project Develop Your evaluation plan Implement Gather data to measure performance Analyse & report Performance against plan 1 2 3 4 Task 1: Define what you need to evaluate by asking: • What activity am I evaluating? • What do I know & what factors could affect the outcome? • What is my evaluation expected to achieve? Output: Summary of your proposed evaluation approach Task 2: Define how you’ll measure success: • Set SMART objectives & define your target audience • Map out how activity will work • Set performance metrics (KPIs) & agree baseline / targets Output: Draft evaluation plan Task 3: Identify and gather evaluation data: • Make most of existing data • Gather additional data (research, feedback & proxies) • Review data gaps (more budget ?) manage expectations Output: Completed evaluation plan Task 4: Assess the success of your activity: • Analyse effectiveness & provide insights for future • Demonstrate efficiency and value for money • Demonstrate role of communications in supporting policy objective delivery (outcome) Output: Final evaluation report
    15. 15. Key performance indicator basketC4 Activity Effectiveness Result
    16. 16. 1. Strategic alignment – Ensure activity objectives are SMART, aligned and supports policy delivery. 2. Business impact – Always measure true business impact (outcome) rather than outputs and out-takes 3. Big IDIA – Try to adopt the suggested frameworks and follow the big IDIA stages. 4. Continuous improvement – Ensure results drive appropriate actions and any learnings inform future activities. 5. Best practice – Be objective, share results and make evaluation an integral part of your communications approach. Top TipsC5
    17. 17. GCS Mandatory Evaluation Project Progress update & Next steps D
    18. 18. D1: Performance Frameworks & Dashboards 1. Performance frameworks /dashboards (Phase one: 37 departments /ALBs)  33 depts. /ALBs (89%) with complete or near complete frameworks.  27 depts./ALBs (73%) expected to have fully functional dashboards by April 14.
    19. 19. Headline measures Audience Team activity / Performance indicators Resource Risks / Concerns Stakeholders Partners Staff Claimants Staff People • FTE (actual vs. plan) • FTE (team and profiles) • Flexible resource stats Financials • Allocations (Staff vs. Non-staff costs) • Spend vs. Forecast • ERF submissions / spend (teams) 1 2 3 4 5 Directorate Targets • Sickness levels: AWDL (teams) • Directorate engagement score • HSE incident: • Compliance targets (e.g.)  mandatory training  performance discussions held  Skills manager Risk Impact Action Lead 1 H 2 M 3 L Risk register No Raised by date Status 1 2 3 3 C’s / Hot Topic register Press office ‘Forward Look’ – up coming events 12 months rolling activity communications plan Prioritisation activity list Planning Section New Existing 6 SRP1: Welfare Reform Business Objective: Introduce the Universal Credit and other reforms to simplify the welfare system; Ensure that the system always incentivises work and work always pays; Help to make the welfare system affordable longer term. Comms Objective / Outcome: W S R Team / Area Input / output / outcome indicators) Universal Credit • Milestones – performance up date • Nos. engaged / events held • No. of web visits / downloads. • No. of comms issued • Staff wavelength / People survey • % Net positive media coverage SRP2: Labour Market (GBW) Business Objective: Implement and manage the Work Programme; Deliver personalised package of support to get people into work. Comms Objective / Outcome: Stakeholders Employers Individuals Staff G B W Team / Area Input / output /Outcome (indicators) Labour Market • Milestones – performance up date • Nos. events held / attendance levels • No. of web visits / downloads. • No. of comms issued • Staff wavelength / People survey • % Net positive media coverage SRP3: Tackling Poverty Business Objective: Develop a welfare system that recognises work as the primary route out of poverty; Implement a the Social Justice and Child Poverty Strategies that focuses on eradicating child poverty by 2020. Comms Objective / Outcome: P O V E R T Y Team / Area Input / output metrics (indicators) Social Justice • Milestones – performance up date • Nos. events held / attendance levels • No. of web visits / downloads. • No. of comms issued • Staff wavelength / People survey • % Net positive media coverage Stakeholders OGDs General public Staff SRP4: Pensions Reform (PR) Business Objective: Provide decent State Pensions; Encourage employers to provide high quality pensions and make automatic enrolment and higher pension saving a reality; Phase out the default retirement age Comms Objective / Outcome: P R Team / Area Input / output metrics (indicators) Pensions • Milestones – performance up date • Camp tracker data (Ind / employers) • No. of web visits / downloads. • No. of comms issued • % Net positive media coverage • Auto enrolment data Stakeholders OGDs General public SRP5: Disability Equality Objective: Develop and implement a disability strategy to enable disable people fulfil their potential; Support more independent living for disable people through an effective strategy and evidence-based policies for disability benefits; Support disabled people take up employment opportunities through specialist programmes. Comms Objective / Outcome: D I S A B I L T Y Team / Area Input / output metrics (indicators) A2 work DLA PIP ESA • Milestones – performance update • No. of events (dialogues) carried out • No. of partnerships established • No. of web visits / downloads. • No. of comms issued • % Net positive media coverage • % take up rates (PIP) Stakeholders OGDs General public Disabled people Claimants Staff SRP6: Improving our services Objectives • Effective communications delivery • Staff engagement improvements • Public and Corporate info delivery • Secure and effective digital comms • Resource management / R&E Outcomes: People Survey – Headline scores People survey scores for: B31,41,47,50- 54, B06-08. S E R V I C E S Teams Input / output metrics (indicators) Internal Comms • Milestone – performance update • No of staff comms issued • No. of proactive briefings (SCS & Line mgrs) • No. of bright ideas (submitted vs. Implemented) • Intranet staff visits to:  Home page / DWP Story pages  Have your say / forum discussions • Wavelength scores (QB1, QB5,QC1,QE1, QE3) Corp Info • No. of leaflets / forms delivered or reduced • No. of evaluation reports requested / delivered • No. of jobs successfully delivered • L&D core training delivered / attendance levels • People survey scores for: B22, B25-26 R&E Business MI* Existing
    20. 20. D2: Mandatory Evaluation Training 2. Evaluation training - Free half-day courses:  34 training sessions delivered.  541GCS staff trained to date.  4,402 unique page views to our online evaluation guide. Evaluation Training (400 free places)* • Introduction to Evaluation for Marketing • Introduction to Evaluation for Press & Media • Guest speaker Events * NB: The nature of the programme this year 2014/15 under review (TBC)
    21. 21. D3: Evaluation Standards & Capability • New evaluation standards – Three levels • Capability audit to start soon - April / May 2014 • Tailored package of support & training for each level Advance Practitioner Foundation
    22. 22. D4: Evaluation ‘ centre of excellence’ • Established an Evaluation Council - Industry expert who meet regularly and act as a sounding board for new evaluation thinking as well as to review and validate government evaluation plans and outputs. • Creating a repository for:  Case studies  Evaluation Guide  Performance Framework Top tips  Standard Metrics
    23. 23. What Next • GCS Live - 1st April 2014 • Govt. Comms Plan launched – 12th May • Capability Audit – April /May 2014 • Developing a tier package of support for each level • Phase 2 roll-out across all departments / ALBs (330) • Regular updates to GCS Programme board and Council  Invite departments / ALB to share their experience & approach • Lots more events planned – Register on GCS now!
    24. 24. 24 Thank you! Contact: Paul Njoku Email : paul.njoku@cabinet-office.gsi.gov.uk Web link to guide: https://gcn.civilservice.gov.uk/guidance/evaluation/
    25. 25. Appendix
    26. 26. Products Customer Service Investments Employment Branding Public Relations Marketing Social Responsibility MEDIA (Traditional, Social) Topic Experts, Leaders, Friends/Family Perceptions & expectations Supportive Behaviour Results What your department Says/Does Direct Experience What Others Say How reputation is created
    27. 27. The attributes and dimensions have different meanings and importance for different stakeholders. Beneath the 7 dimensions, 32 attributes underpin the individual dimension themes. Different stakeholder groups typically have unique attributes that are found more important than others (reputation drivers). Reputation Attributes Reputation Dimensions The seven dimensions specify at a more operational level, which aspects are most important for stakeholders’ perceptions and expectations – i.e. what’s driving a company’s reputation Reputation Strength A measure of the emotional connection. Reputation has a positive/negative impact on support. An increase in reputation = an increase in support. Support (such as buying products and services, saying something positive, giving the benefit of doubt in times of crisis (etc.) leads to increased business results Supportive Behaviour Reputation drivers & dimensions
    28. 28. Measuring outputs 28 Content metrics Favourability Messages Themes / Issues Campaigns Article type Proactive / reactive Prominence Calls to action Delivery metrics Volume Opportunities to see Reach and Frequency Media type Key publications Journalists Spokespeople Regional breakdown Impact measures Output Metrics Are we saying the right thing... ....to the right audience? Activity Intermediary Effect Audience Effect ‘Earned’ media Model Amplification from social media
    29. 29. Government analysis audit 29 Are we saying the right thing... ....to the right audience Metric % of projects Favourability 100% Message delivery 91% Campaign tracking 48% Theme / issues tracking 57% Spokespeople / commentators 78% Prominence 57% Article type 43% Proactive / reactive 22% Calls to action 39% Volume (number of articles) 100% Opportunities to see (OTS) 74% Reach and frequency 78% Impact measures 26% Media type 83% Regional split 57% Publications 87% Journalists 87% Advertising value equivalent AVE) 57% Cost per thousand (CPM) 22% Linking ouput to outcomes 22% Content Metrics Delivery metrics Efficiency / effectiveness metrics

    ×