Work package 7 Evaluation & Quality Assurance Mireille Pouget MYPS
Internal operation framework <ul><li>Across all WPs </li></ul><ul><li>Embedded monitoring, feedback, evaluation -QA mechan...
What evaluation & what quality assurance? <ul><li>QA & evaluation plan agreed by all partners and covering all participant...
Project evaluation <ul><li>Making judgements of “worth”  </li></ul><ul><li>How well the project’s objectives have been ach...
Outputs versus outcomes <ul><li>( Deliverables): tracking outputs </li></ul><ul><li>program logic model: inputs --> activi...
Monitoring by stakeholders: activities - products <ul><li>Trainers – tutors  </li></ul><ul><li>WP leaders </li></ul><ul><l...
Feedback from stakeholders & to stakeholders <ul><li>Learners (end users) </li></ul><ul><li>Trainers – tutors </li></ul><u...
Monitoring & evaluating processes <ul><li>During the project’s formative stage (i.e. planning, design and development)  </...
Products & processes <ul><li>Outputs & deliverables </li></ul><ul><li>Devices </li></ul><ul><li>Learning tools </li></ul><...
What processes <ul><li>Planning process  (now)   </li></ul><ul><li>Development process  (milestones, budgets, tasks, QA pr...
Stakeholder Analysis Worksheet <ul><li>Needs: learning, social,  personal  - motivation </li></ul><ul><li>Understanding of...
Sample plan for tracking outputs (monitoring) Evaluat Proj Lead On going Before inter rpt Tutors  WP L Use of L tools Lear...
Tools for monitoring & evaluation   <ul><li>(WP2) Surveys –questionnaires – sampling </li></ul><ul><li>Quantitative – qual...
Short & medium terms objectives <ul><li>Evaluation & QA plan to be elaborated </li></ul><ul><li>In consultation with partn...
Upcoming SlideShare
Loading in …5
×

D2 milan01-10-wp7- mp-pres-2

328 views

Published on

MilanPPT mp

Published in: Education
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
328
On SlideShare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
Downloads
5
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

D2 milan01-10-wp7- mp-pres-2

  1. 1. Work package 7 Evaluation & Quality Assurance Mireille Pouget MYPS
  2. 2. Internal operation framework <ul><li>Across all WPs </li></ul><ul><li>Embedded monitoring, feedback, evaluation -QA mechanisms/ tools in each activity for deliverables </li></ul><ul><li>Clear roles & responsibilities for all partners </li></ul><ul><li>Full consultation with all stakeholders </li></ul><ul><li>Clear & agreed methodology for QA & evaluation </li></ul><ul><li>Overall project plan – A sub plan for each WP? </li></ul><ul><ul><li>Project outputs and outcomes </li></ul></ul><ul><ul><li>Processes </li></ul></ul>
  3. 3. What evaluation & what quality assurance? <ul><li>QA & evaluation plan agreed by all partners and covering all participants </li></ul><ul><li>Definitions of SMART KPIs (M2) </li></ul><ul><li>Monitoring strategy </li></ul><ul><li>-> Feedback from stakeholders </li></ul><ul><li>Formative and summative evaluation </li></ul><ul><li>Internal reports at M12, M29, M36 </li></ul><ul><li>External evaluation committee </li></ul><ul><li>2 focus groups M9, M29 </li></ul><ul><li>Starts NOW </li></ul>
  4. 4. Project evaluation <ul><li>Making judgements of “worth” </li></ul><ul><li>How well the project’s objectives have been achieved (in terms of outputs and educational outcomes) </li></ul><ul><li>How well the project has run as an enterprise (processes) </li></ul>
  5. 5. Outputs versus outcomes <ul><li>( Deliverables): tracking outputs </li></ul><ul><li>program logic model: inputs --> activities or processes --> outputs --> outcomes </li></ul><ul><li>To provides indicators, in terms of output and outcome measures of performance </li></ul><ul><li>To specify the desired results </li></ul><ul><li>To consider how to monitor them over time </li></ul><ul><li>To focus on results or outcomes </li></ul>
  6. 6. Monitoring by stakeholders: activities - products <ul><li>Trainers – tutors </li></ul><ul><li>WP leaders </li></ul><ul><li>ICT developers </li></ul><ul><li>WP7 leader </li></ul><ul><li>Project management </li></ul><ul><ul><li>For feedback </li></ul></ul><ul><ul><li>For formative evaluation </li></ul></ul><ul><ul><li>For interim evaluation reports </li></ul></ul><ul><ul><li>For QA of testing, tools, and processes </li></ul></ul><ul><ul><li>For impact & outcomes evaluation </li></ul></ul><ul><ul><li>For summative evaluation </li></ul></ul>
  7. 7. Feedback from stakeholders & to stakeholders <ul><li>Learners (end users) </li></ul><ul><li>Trainers – tutors </li></ul><ul><li>ICT Developers </li></ul><ul><li>Partners and WP leaders & users </li></ul><ul><li>Project managers </li></ul><ul><li>WP7 leader </li></ul>
  8. 8. Monitoring & evaluating processes <ul><li>During the project’s formative stage (i.e. planning, design and development) </li></ul><ul><li>For problems diagnosis </li></ul><ul><li>Suitable corrective action </li></ul><ul><li>Critically reflecting on processes upon completion of the project for future use </li></ul>
  9. 9. Products & processes <ul><li>Outputs & deliverables </li></ul><ul><li>Devices </li></ul><ul><li>Learning tools </li></ul><ul><li>Evaluation & QA mechanisms & tools </li></ul><ul><li>Reports </li></ul><ul><li>Observation of change in </li></ul><ul><ul><li>Learning skills </li></ul></ul><ul><ul><li>Attitudes </li></ul></ul><ul><ul><li>Confidence </li></ul></ul><ul><ul><li>motivation </li></ul></ul><ul><li>Diversity-cultural issues </li></ul><ul><li>Experience of tutors </li></ul><ul><li>Flexibility of ICT developers </li></ul><ul><li>Project communication/ </li></ul><ul><li>Management /processes </li></ul>
  10. 10. What processes <ul><li>Planning process (now) </li></ul><ul><li>Development process (milestones, budgets, tasks, QA processes) </li></ul><ul><li>Communication/ teamwork process (effectiveness, functioning of teams) </li></ul><ul><li>Monitoring and control process </li></ul><ul><li>Turning process concerns into questions for evaluation </li></ul>
  11. 11. Stakeholder Analysis Worksheet <ul><li>Needs: learning, social, personal - motivation </li></ul><ul><li>Understanding of project’ aims and objectives, output and outcomes </li></ul><ul><li>Time to learn, to attend </li></ul><ul><li>Access/ convenience - Tools’ usability/ ease of use </li></ul><ul><li>Ease of communication with virtual community (other learners) </li></ul><ul><li>Evidence of change in: attitude towards new technology , Performance , Motivation </li></ul><ul><li>Perception of “pay-off” </li></ul>Learners <ul><li>Project management processes: structure, communications, time/costs … </li></ul><ul><li>Achievement of projects’ stated aims and objectives </li></ul><ul><li>Meeting milestones (outputs/ deliverables, outcomes) within the project </li></ul><ul><li>Adherence to document plans, methodology and QA standards </li></ul><ul><li>Overall lessons to be learnt (how to improve, process, poduct) </li></ul><ul><li>Others </li></ul>Project management group- leader <ul><li>Perceptions of the learners’ learning experience </li></ul><ul><li>Usability/ suitability of technological tools, surveys and other tools </li></ul><ul><li>Amount of learning support required within required time frame </li></ul><ul><li>Identification of barriers and drivers </li></ul><ul><li>Achievement of project outputs or deliverable within timeframe </li></ul><ul><li>Achievement of projects’ stated aims and objectives </li></ul><ul><li>Others </li></ul>Trainers Tutors <ul><li>Quality of participants’ learning experience: their perception, reaction to content/ technology, learning processes… </li></ul><ul><li>Learners’ time on tasks </li></ul><ul><li>Usability/ suitability of technological tools, surveys and other tools </li></ul><ul><li>Extent to which learners interact with others – create a social network </li></ul><ul><li>Level of support for project staff/ trainers from consortium- project management </li></ul><ul><li>Achievement of projects’ stated aims and objectives </li></ul><ul><li>Others….. </li></ul>Design/ development staff Timefr Priori Possible concerns Stakeholders
  12. 12. Sample plan for tracking outputs (monitoring) Evaluat Proj Lead On going Before inter rpt Tutors WP L Use of L tools Learners’ progress Output tracking Ongoing Progress- Assessing Proj L EC X Evaluator Proj L Summary of data collected Interim report Evaluat Proj Lead intake Tutors WP lead Quest Focus gp Survey1 info Baseline For whom? Due date Who is respons.? Description & use Activity /report- WP…
  13. 13. Tools for monitoring & evaluation <ul><li>(WP2) Surveys –questionnaires – sampling </li></ul><ul><li>Quantitative – qualitative data collection and evaluation; checklists </li></ul><ul><li>Other tools for evaluation of learners’ learning (change/ attitude..): </li></ul><ul><ul><li>i.e. confidence (b) logs: learning development, before and after </li></ul></ul><ul><ul><li>Recorded/ videoed focus group discussion </li></ul></ul><ul><ul><li>Electronic tracking of use of technology </li></ul></ul>
  14. 14. Short & medium terms objectives <ul><li>Evaluation & QA plan to be elaborated </li></ul><ul><li>In consultation with partners/ WP leaders </li></ul><ul><li>Identifying quantitative & qualitative KPIs </li></ul><ul><ul><li>Milestones/ feedbacks/ reports </li></ul></ul><ul><ul><li>Establishing monitoring ‘schedule’ or tracking outputs plan </li></ul></ul><ul><ul><li>Parallel QA activities through monitoring </li></ul></ul><ul><li>Interface with external experts and focus group </li></ul>

×