ULearning Skillnet

Overview of Evaluation Approach

       Rob Cosgrove & Kieran Swail
       Monday 8th September 2008
 ...
Role & Approach

• Role = monitor, evaluate and report

• Approach = Listen, observe, engage,
  advise, recommend and revi...
Scope of evaluation
• 2 reports Interim early 2009, Final early
  2010
• Key data & indicators set by Skillnet
  – Mandato...
Phillips ROI Model
          The Phillips ROI methodology
          is a 10-step process preferred
          by Skillnets
...
Proposed ROI Method

• Identify module(s) to target for measurement
• Criteria
  – Multiple students doing same module in ...
Progress to date

• Attended Launch in Shannon
• Met with Project Co-ordinator and
  members of Steering Group in Dublin
•...
Methodology / Outputs to Interim Report
• Attend Steering Group Meetings Sep, Oct – Dec 08
• f2f Consultations with select...
Added Value & Support

• Wiki
• Spot Evaluation of an event - pre & post
  survey – process and content review Nov/Dec
  0...
Risk                           Area             Severity Probability Mitigation
ROI may not be easily          Evaluation ...
Q How do you attract new                                   Apply W5
network members and trainees?                         ...
ULearning Skillnet

        Thankyou
Rob Cosgrove & Kieran Swail
Monday 8th September 2008
Ulearning evaluation approach
Upcoming SlideShare
Loading in …5
×

Ulearning evaluation approach

550 views

Published on

Outline of evaluation approach focus on networking

0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
550
On SlideShare
0
From Embeds
0
Number of Embeds
3
Actions
Shares
0
Downloads
3
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Ulearning evaluation approach

  1. 1. ULearning Skillnet Overview of Evaluation Approach Rob Cosgrove & Kieran Swail Monday 8th September 2008 Shannon Business Park
  2. 2. Role & Approach • Role = monitor, evaluate and report • Approach = Listen, observe, engage, advise, recommend and review
  3. 3. Scope of evaluation • 2 reports Interim early 2009, Final early 2010 • Key data & indicators set by Skillnet – Mandatory, Strongly recommended, Optional – Kirkpatrick scale of evaluating training Level 1-3 & Phillips Level 5 see Guidelines handout. – Innovation & Networking = Opportunity to stand out
  4. 4. Phillips ROI Model The Phillips ROI methodology is a 10-step process preferred by Skillnets We need to select suitable course/modules Consider Impact Measurement Skillnet Training for this initiative and next round.
  5. 5. Proposed ROI Method • Identify module(s) to target for measurement • Criteria – Multiple students doing same module in same company failing that – similar companies closely spaced – preference for larger companies – preference for modules with concrete skill deliverables – Interview module lecturer for modules chosen for ROI measurement. – In depth interviews with reps from each provider – In depth interviews/ liaison with trainers in modules selected for ROI assessment
  6. 6. Progress to date • Attended Launch in Shannon • Met with Project Co-ordinator and members of Steering Group in Dublin • Met with Skillnets Liaison Officer in Dublin • Evaluators met and agreed joint approach
  7. 7. Methodology / Outputs to Interim Report • Attend Steering Group Meetings Sep, Oct – Dec 08 • f2f Consultations with select no of network member companies Oct – Dec 08 • Review Lead Generation Process & General Marketing Activity • Identify course/module (s) for ROI assessment • Students/Trainees – Pre Module Start • Subset 10-15 minute telephone interviews • eSurvey or integrate with AUA pre- commencement data capture – Interview Student Cohort Nov 08 – Review End Module feedback sheets • Draft Report Dec 08 & Interim Report Feb 09
  8. 8. Added Value & Support • Wiki • Spot Evaluation of an event - pre & post survey – process and content review Nov/Dec 08? • Ideas – Calendar of Events Network Activities & promotional activities – Online Forum? – Analysis of sales funnel – lead generation to sales conversion
  9. 9. Risk Area Severity Probability Mitigation ROI may not be easily Evaluation Moderate High Select modules for measurable via Phillips. ROI assessment Degree level ROI less with care. tangible ROI slower to accrue Softer skills harder to measure credible ROI. Training days targets may Marketing High High not be hit. Network may seem Administrative ?? Moderate Ensure network supply led rather than members are the demand led – a sales key driver of channel for universities. course content – not providers. Regional Focus on the Marketing High Moderate Retarget marketing west misses key markets activity to Greater – Dublin - is less effective Dublin Area Marketing focus Marketing High Moderate Monitor & Evaluate unstructured – not leading marketing/sales to closed sales. process
  10. 10. Q How do you attract new Apply W5 network members and trainees? Who?, What?, When?, Where?, Why? Trainees • Telephone – cold v warm / Network Members • Text Mkting -Marketeers call the •Network - Regular themed events 3rd screen •Branding & Merchandising • Direct Mail – postcard mkting •Trade shows, Exhibitions • Competitions •Discussion Forums HR/CPD • Demos - “wow” factor • PR / Word of Mouth www.linkedin.com • Testimonials / Referrals •Conferences • Web Marketing – Web 2.0 •Market Research – email, blog, podcasts, social •Joint Venture e.g. IRCSET - the Irish networking, Flickr YouTube, Research Council for Science, Facebook, Bebo, MySpace RSS feeds.. Engineering and Technology • Advertising •Journalists Visits – education, science – Radio, newspapers, supplements, journals. Local & technology business circulars, Magazines, •Sponsorship Billboards, Google Adwords,
  11. 11. ULearning Skillnet Thankyou Rob Cosgrove & Kieran Swail Monday 8th September 2008

×