JISC BCE: Evaluation Phase 1

909 views

Published on

Presentation outlining the proposed evaluation frameowrk to be used by the BCE Programme.

Published in: Education, Technology, Business
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
909
On SlideShare
0
From Embeds
0
Number of Embeds
6
Actions
Shares
0
Downloads
0
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

JISC BCE: Evaluation Phase 1

  1. 1. JISC BCE Programme – ‘Evaluation Phase 1’ Evaluation Framework – Implications for BCE Projects A Presentation by: Dr. Steve Johnson WM Enterprise 05.03.2009
  2. 2. Introduction <ul><li>WM Enterprise (WME) is a specialist economic development & regeneration consultancy </li></ul><ul><li>JISC has commissioned WME to undertake ‘Evaluation Phase 1’ (work stream 6) of the Business and Community Engagement (BCE) programme, to establish an evaluation framework for the programme and its 6 constituent work streams </li></ul><ul><li>WME Project Team: </li></ul><ul><ul><li>Project Director – Dr. Steve Johnson </li></ul></ul><ul><ul><li>Project Manager – Ruth Winterton </li></ul></ul><ul><ul><li>Research Consultant – Helen Blakeley </li></ul></ul>
  3. 3. Elements of the Evaluation Framework Capital Evaluation Framework JISC Evaluation Handbook Attribution Sustainability Practical solutions Identification of good practice Quality Flexibility Short, medium & long-term measures Programme, work stream and project level evaluation Strategic & operational levels Formative & summative Elements to be included within the Framework
  4. 4. BCE Programme - Logic Chain Rationale “ In the ‘knowledge economy,’ BCE makes a key contribution to innovation, competitive advantage, skills enhancement workforce development and social cohesion.” <ul><li>High-level Aims </li></ul><ul><li>Aims: </li></ul><ul><li>To enhance institutional efficiency, effectiveness and opportunities in business and community activities. </li></ul><ul><li>To improve access to institutions’ knowledge assets for business and community orgainsations. </li></ul>Inputs & Activities Financial Resources: c.£2 million per annum up to 2011 Participation from public & private sectors and social, cultural and civic arenas. <ul><li>Short-term Outputs </li></ul><ul><li>Range of outputs across 6 work streams, including: </li></ul><ul><li>New IT tools and collaborations to support BCE; </li></ul><ul><li>Tailored workshops and training sessions; </li></ul><ul><li>Dissemination and networking events; and </li></ul><ul><li>Pilot projects and trials of various innovative IT solutions. </li></ul><ul><li>Medium-term Outcomes </li></ul><ul><li>(Benefits) </li></ul><ul><li>Improved ICT to support BCE </li></ul><ul><li>Wider engagement in BCE activities; </li></ul><ul><li>Productivity enhancement; </li></ul><ul><li>Innovation and enterprise; </li></ul><ul><li>Employer engagement; and </li></ul><ul><li>New and improved relationships between institutions and external BCE partners. </li></ul><ul><li>Long-term Impacts </li></ul><ul><li>More effective BCE activities; </li></ul><ul><li>Enhanced institutional efficiency, effectiveness and opportunities in BCE activities; </li></ul><ul><li>Improved access to institutions’ knowledge assets for BC organisations; and </li></ul><ul><li>Contributions to more; highly skilled workforce; competitive economy; and cohesive, knowledge-based society. </li></ul>
  5. 5. Key Outcomes and Impacts <ul><li>Our document review and consultations have identified that the impact of the BCE programme falls into 5 main categories : </li></ul><ul><ul><ul><li>High-level strategy and policy impact </li></ul></ul></ul><ul><ul><ul><li>Impact on JISC </li></ul></ul></ul><ul><ul><ul><li>(Impact on) institutional strategies </li></ul></ul></ul><ul><ul><ul><li>(Impact on) institutional BCE activities </li></ul></ul></ul><ul><ul><ul><li>Wider BCE impacts </li></ul></ul></ul><ul><li>All aspects of the BCE programme contribute to the higher-level aims and objectives of the programme (and to the wider knowledge transfer/innovation/BCE agenda). </li></ul><ul><li>However, each work stream/project addresses specific areas of the agenda. Therefore, on a work stream and project level , some of the above categories are more directly relevant to specific work stream benefits/project aims and objectives than others. </li></ul><ul><li>The framework recognises the above and sets out our initial view of the nature of impacts on a work stream and project level where: </li></ul><ul><li>Indirect impact </li></ul><ul><li>Direct and indirect impact </li></ul><ul><li>Direct impact </li></ul>
  6. 6. Evaluation Framework – Programme & Work Stream Level High-level Strategy and Policy Impact Type of Impact Reach and impact Spatial Type of institution Contribution to policy development Local level Regional level National level Work stream 6 Work stream 5 Work stream 4 Work stream 3 Work stream 2 Work stream 1 Outcome/ Impact
  7. 7. Evaluation Framework – Programme & Work Stream Level Impact on JISC Type of Impact Impact on JISC strategy/activity Awareness of JISC JISC-wide coherence Work stream 6 Work stream 5 Work stream 4 Work stream 3 Work stream 2 Work stream 1 Outcome/ Impact
  8. 8. Evaluation Framework – Programme & Work Stream Level (Impact on) Institutional Strategies Type of Impact Role of IT in institutional strategies Collaboration between institutions Institutional joined-up thinking Impact on institutional strategies Work stream 6 Work stream 5 Work stream 4 Work stream 3 Work stream 2 Work stream 1 Outcome/ Impact
  9. 9. Evaluation Framework – Programme & Work Stream Level (Impact on) Institutional BCE Activities Type of Impact Take-up of BCE products Increased BCE income Engagement in partnership and network activity Work stream 6 Work stream 5 Work stream 4 Work stream 3 Work stream 2 Work stream 1 Outcome/ Impact
  10. 10. Evaluation Framework – Programme & Work Stream Level Wider BCE Impacts Type of Impact Improved quality and quantity of work-based and community education Improving the quality of services offered to employers, communities, third sector groups and learners Work stream 6 Work stream 5 Work stream 4 Work stream 3 Work stream 2 Work stream 1 Outcome/ Impact
  11. 11. Evaluation Framework: Project Level TYPE OF IMPACT OUTCOME/IMPACT Project-level evaluation: focus on BENEFITS Measurement against specific project aims/objectives Improved quality and quantity of work-based and community education Improving the quality of services offered to employers, community, third sector groups and learners Wider BCE Impacts Take-up of BCE products Increased BCE income Engagement in partnership and network activity Institutional BCE Activities Collaboration between institutions Role of IT in institutional strategies Institutional joined-up thinking Impact on institutional strategies Institutional strategies Awareness of JISC Impact on JISC strategy/activity JISC-wide coherence Impact on JISC Reach and impact (spatial, type of institution) Contribution to policy development (local, regional, national level) To be assessed on project-by-project basis, using Evaluation Framework as guide To feed into overall strategic evaluation of JISC BCE High-level strategy and policy impact
  12. 12. BCE Evaluation When? <ul><li>Evaluation should be ongoing process </li></ul><ul><ul><li>… starting as early as possible </li></ul></ul><ul><ul><li>… and built into programme and project planning </li></ul></ul><ul><li>Early activity should be primarily formative </li></ul><ul><ul><li>… with focused activity from April/May until Oct/Nov 2009 </li></ul></ul><ul><ul><li>… informing project delivery on ongoing basis </li></ul></ul><ul><li>Regular feedback/review throughout timescale of programme </li></ul><ul><ul><li>… at least every 6 months through workshops or other appropriate mechanisms </li></ul></ul><ul><li>Final summative phase in 2011 </li></ul><ul><ul><li>… with focus on outcomes, impacts, attribution, value for money etc </li></ul></ul>
  13. 13. BCE Evaluation Who and How? <ul><li>Overall strategic evaluation should be undertaken by an independent, external body </li></ul><ul><ul><li>… acting as ‘ critical friend ’; providing ‘supportive challenge’ </li></ul></ul><ul><ul><li>… working within agreed Evaluation Framework </li></ul></ul><ul><li>Programme, workstream and projects should all have evaluation built into ongoing activities as key priority </li></ul><ul><ul><li>… using external resources if/when appropriate </li></ul></ul><ul><ul><li>… with active and ongoing involvement of key stakeholders e.g. government agencies, funding councils, BCE organisations, employer and community bodies, innovation bodies etc. </li></ul></ul>
  14. 14. WM Enterprise Bradford Office The Design Exchange 34 Peckover Street Little Germany Bradford BD1 5BD t: 01274 743600 e: [email_address] w: www.wm-enterprise.co.uk  Birmingham  London  Merseyside  Bradford  Surrey

×