Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

JISC BCE: Evaluation Phase 1

Presentation outlining the proposed evaluation frameowrk to be used by the BCE Programme.

Related Books

Free with a 30 day trial from Scribd

See all

Related Audiobooks

Free with a 30 day trial from Scribd

See all
  • Be the first to comment

  • Be the first to like this

JISC BCE: Evaluation Phase 1

  1. 1. JISC BCE Programme – ‘Evaluation Phase 1’ Evaluation Framework – Implications for BCE Projects A Presentation by: Dr. Steve Johnson WM Enterprise 05.03.2009
  2. 2. Introduction <ul><li>WM Enterprise (WME) is a specialist economic development & regeneration consultancy </li></ul><ul><li>JISC has commissioned WME to undertake ‘Evaluation Phase 1’ (work stream 6) of the Business and Community Engagement (BCE) programme, to establish an evaluation framework for the programme and its 6 constituent work streams </li></ul><ul><li>WME Project Team: </li></ul><ul><ul><li>Project Director – Dr. Steve Johnson </li></ul></ul><ul><ul><li>Project Manager – Ruth Winterton </li></ul></ul><ul><ul><li>Research Consultant – Helen Blakeley </li></ul></ul>
  3. 3. Elements of the Evaluation Framework Capital Evaluation Framework JISC Evaluation Handbook Attribution Sustainability Practical solutions Identification of good practice Quality Flexibility Short, medium & long-term measures Programme, work stream and project level evaluation Strategic & operational levels Formative & summative Elements to be included within the Framework
  4. 4. BCE Programme - Logic Chain Rationale “ In the ‘knowledge economy,’ BCE makes a key contribution to innovation, competitive advantage, skills enhancement workforce development and social cohesion.” <ul><li>High-level Aims </li></ul><ul><li>Aims: </li></ul><ul><li>To enhance institutional efficiency, effectiveness and opportunities in business and community activities. </li></ul><ul><li>To improve access to institutions’ knowledge assets for business and community orgainsations. </li></ul>Inputs & Activities Financial Resources: c.£2 million per annum up to 2011 Participation from public & private sectors and social, cultural and civic arenas. <ul><li>Short-term Outputs </li></ul><ul><li>Range of outputs across 6 work streams, including: </li></ul><ul><li>New IT tools and collaborations to support BCE; </li></ul><ul><li>Tailored workshops and training sessions; </li></ul><ul><li>Dissemination and networking events; and </li></ul><ul><li>Pilot projects and trials of various innovative IT solutions. </li></ul><ul><li>Medium-term Outcomes </li></ul><ul><li>(Benefits) </li></ul><ul><li>Improved ICT to support BCE </li></ul><ul><li>Wider engagement in BCE activities; </li></ul><ul><li>Productivity enhancement; </li></ul><ul><li>Innovation and enterprise; </li></ul><ul><li>Employer engagement; and </li></ul><ul><li>New and improved relationships between institutions and external BCE partners. </li></ul><ul><li>Long-term Impacts </li></ul><ul><li>More effective BCE activities; </li></ul><ul><li>Enhanced institutional efficiency, effectiveness and opportunities in BCE activities; </li></ul><ul><li>Improved access to institutions’ knowledge assets for BC organisations; and </li></ul><ul><li>Contributions to more; highly skilled workforce; competitive economy; and cohesive, knowledge-based society. </li></ul>
  5. 5. Key Outcomes and Impacts <ul><li>Our document review and consultations have identified that the impact of the BCE programme falls into 5 main categories : </li></ul><ul><ul><ul><li>High-level strategy and policy impact </li></ul></ul></ul><ul><ul><ul><li>Impact on JISC </li></ul></ul></ul><ul><ul><ul><li>(Impact on) institutional strategies </li></ul></ul></ul><ul><ul><ul><li>(Impact on) institutional BCE activities </li></ul></ul></ul><ul><ul><ul><li>Wider BCE impacts </li></ul></ul></ul><ul><li>All aspects of the BCE programme contribute to the higher-level aims and objectives of the programme (and to the wider knowledge transfer/innovation/BCE agenda). </li></ul><ul><li>However, each work stream/project addresses specific areas of the agenda. Therefore, on a work stream and project level , some of the above categories are more directly relevant to specific work stream benefits/project aims and objectives than others. </li></ul><ul><li>The framework recognises the above and sets out our initial view of the nature of impacts on a work stream and project level where: </li></ul><ul><li>Indirect impact </li></ul><ul><li>Direct and indirect impact </li></ul><ul><li>Direct impact </li></ul>
  6. 6. Evaluation Framework – Programme & Work Stream Level High-level Strategy and Policy Impact Type of Impact Reach and impact Spatial Type of institution Contribution to policy development Local level Regional level National level Work stream 6 Work stream 5 Work stream 4 Work stream 3 Work stream 2 Work stream 1 Outcome/ Impact
  7. 7. Evaluation Framework – Programme & Work Stream Level Impact on JISC Type of Impact Impact on JISC strategy/activity Awareness of JISC JISC-wide coherence Work stream 6 Work stream 5 Work stream 4 Work stream 3 Work stream 2 Work stream 1 Outcome/ Impact
  8. 8. Evaluation Framework – Programme & Work Stream Level (Impact on) Institutional Strategies Type of Impact Role of IT in institutional strategies Collaboration between institutions Institutional joined-up thinking Impact on institutional strategies Work stream 6 Work stream 5 Work stream 4 Work stream 3 Work stream 2 Work stream 1 Outcome/ Impact
  9. 9. Evaluation Framework – Programme & Work Stream Level (Impact on) Institutional BCE Activities Type of Impact Take-up of BCE products Increased BCE income Engagement in partnership and network activity Work stream 6 Work stream 5 Work stream 4 Work stream 3 Work stream 2 Work stream 1 Outcome/ Impact
  10. 10. Evaluation Framework – Programme & Work Stream Level Wider BCE Impacts Type of Impact Improved quality and quantity of work-based and community education Improving the quality of services offered to employers, communities, third sector groups and learners Work stream 6 Work stream 5 Work stream 4 Work stream 3 Work stream 2 Work stream 1 Outcome/ Impact
  11. 11. Evaluation Framework: Project Level TYPE OF IMPACT OUTCOME/IMPACT Project-level evaluation: focus on BENEFITS Measurement against specific project aims/objectives Improved quality and quantity of work-based and community education Improving the quality of services offered to employers, community, third sector groups and learners Wider BCE Impacts Take-up of BCE products Increased BCE income Engagement in partnership and network activity Institutional BCE Activities Collaboration between institutions Role of IT in institutional strategies Institutional joined-up thinking Impact on institutional strategies Institutional strategies Awareness of JISC Impact on JISC strategy/activity JISC-wide coherence Impact on JISC Reach and impact (spatial, type of institution) Contribution to policy development (local, regional, national level) To be assessed on project-by-project basis, using Evaluation Framework as guide To feed into overall strategic evaluation of JISC BCE High-level strategy and policy impact
  12. 12. BCE Evaluation When? <ul><li>Evaluation should be ongoing process </li></ul><ul><ul><li>… starting as early as possible </li></ul></ul><ul><ul><li>… and built into programme and project planning </li></ul></ul><ul><li>Early activity should be primarily formative </li></ul><ul><ul><li>… with focused activity from April/May until Oct/Nov 2009 </li></ul></ul><ul><ul><li>… informing project delivery on ongoing basis </li></ul></ul><ul><li>Regular feedback/review throughout timescale of programme </li></ul><ul><ul><li>… at least every 6 months through workshops or other appropriate mechanisms </li></ul></ul><ul><li>Final summative phase in 2011 </li></ul><ul><ul><li>… with focus on outcomes, impacts, attribution, value for money etc </li></ul></ul>
  13. 13. BCE Evaluation Who and How? <ul><li>Overall strategic evaluation should be undertaken by an independent, external body </li></ul><ul><ul><li>… acting as ‘ critical friend ’; providing ‘supportive challenge’ </li></ul></ul><ul><ul><li>… working within agreed Evaluation Framework </li></ul></ul><ul><li>Programme, workstream and projects should all have evaluation built into ongoing activities as key priority </li></ul><ul><ul><li>… using external resources if/when appropriate </li></ul></ul><ul><ul><li>… with active and ongoing involvement of key stakeholders e.g. government agencies, funding councils, BCE organisations, employer and community bodies, innovation bodies etc. </li></ul></ul>
  14. 14. WM Enterprise Bradford Office The Design Exchange 34 Peckover Street Little Germany Bradford BD1 5BD t: 01274 743600 e: [email_address] w: www.wm-enterprise.co.uk  Birmingham  London  Merseyside  Bradford  Surrey

×