D2 milan01-10-wp7- mp-pres-2
Upcoming SlideShare
Loading in...5

Like this? Share it with your network


D2 milan01-10-wp7- mp-pres-2



MilanPPT mp

MilanPPT mp



Total Views
Views on SlideShare
Embed Views



0 Embeds 0

No embeds



Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
Post Comment
Edit your comment

D2 milan01-10-wp7- mp-pres-2 Presentation Transcript

  • 1. Work package 7 Evaluation & Quality Assurance Mireille Pouget MYPS
  • 2. Internal operation framework
    • Across all WPs
    • Embedded monitoring, feedback, evaluation -QA mechanisms/ tools in each activity for deliverables
    • Clear roles & responsibilities for all partners
    • Full consultation with all stakeholders
    • Clear & agreed methodology for QA & evaluation
    • Overall project plan – A sub plan for each WP?
      • Project outputs and outcomes
      • Processes
  • 3. What evaluation & what quality assurance?
    • QA & evaluation plan agreed by all partners and covering all participants
    • Definitions of SMART KPIs (M2)
    • Monitoring strategy
    • -> Feedback from stakeholders
    • Formative and summative evaluation
    • Internal reports at M12, M29, M36
    • External evaluation committee
    • 2 focus groups M9, M29
    • Starts NOW
  • 4. Project evaluation
    • Making judgements of “worth”
    • How well the project’s objectives have been achieved (in terms of outputs and educational outcomes)
    • How well the project has run as an enterprise (processes)
  • 5. Outputs versus outcomes
    • ( Deliverables): tracking outputs
    • program logic model: inputs --> activities or processes --> outputs --> outcomes
    • To provides indicators, in terms of output and outcome measures of performance
    • To specify the desired results
    • To consider how to monitor them over time
    • To focus on results or outcomes
  • 6. Monitoring by stakeholders: activities - products
    • Trainers – tutors
    • WP leaders
    • ICT developers
    • WP7 leader
    • Project management
      • For feedback
      • For formative evaluation
      • For interim evaluation reports
      • For QA of testing, tools, and processes
      • For impact & outcomes evaluation
      • For summative evaluation
  • 7. Feedback from stakeholders & to stakeholders
    • Learners (end users)
    • Trainers – tutors
    • ICT Developers
    • Partners and WP leaders & users
    • Project managers
    • WP7 leader
  • 8. Monitoring & evaluating processes
    • During the project’s formative stage (i.e. planning, design and development)
    • For problems diagnosis
    • Suitable corrective action
    • Critically reflecting on processes upon completion of the project for future use
  • 9. Products & processes
    • Outputs & deliverables
    • Devices
    • Learning tools
    • Evaluation & QA mechanisms & tools
    • Reports
    • Observation of change in
      • Learning skills
      • Attitudes
      • Confidence
      • motivation
    • Diversity-cultural issues
    • Experience of tutors
    • Flexibility of ICT developers
    • Project communication/
    • Management /processes
  • 10. What processes
    • Planning process (now)
    • Development process (milestones, budgets, tasks, QA processes)
    • Communication/ teamwork process (effectiveness, functioning of teams)
    • Monitoring and control process
    • Turning process concerns into questions for evaluation
  • 11. Stakeholder Analysis Worksheet
    • Needs: learning, social, personal - motivation
    • Understanding of project’ aims and objectives, output and outcomes
    • Time to learn, to attend
    • Access/ convenience - Tools’ usability/ ease of use
    • Ease of communication with virtual community (other learners)
    • Evidence of change in: attitude towards new technology , Performance , Motivation
    • Perception of “pay-off”
    • Project management processes: structure, communications, time/costs …
    • Achievement of projects’ stated aims and objectives
    • Meeting milestones (outputs/ deliverables, outcomes) within the project
    • Adherence to document plans, methodology and QA standards
    • Overall lessons to be learnt (how to improve, process, poduct)
    • Others
    Project management group- leader
    • Perceptions of the learners’ learning experience
    • Usability/ suitability of technological tools, surveys and other tools
    • Amount of learning support required within required time frame
    • Identification of barriers and drivers
    • Achievement of project outputs or deliverable within timeframe
    • Achievement of projects’ stated aims and objectives
    • Others
    Trainers Tutors
    • Quality of participants’ learning experience: their perception, reaction to content/ technology, learning processes…
    • Learners’ time on tasks
    • Usability/ suitability of technological tools, surveys and other tools
    • Extent to which learners interact with others – create a social network
    • Level of support for project staff/ trainers from consortium- project management
    • Achievement of projects’ stated aims and objectives
    • Others…..
    Design/ development staff Timefr Priori Possible concerns Stakeholders
  • 12. Sample plan for tracking outputs (monitoring) Evaluat Proj Lead On going Before inter rpt Tutors WP L Use of L tools Learners’ progress Output tracking Ongoing Progress- Assessing Proj L EC X Evaluator Proj L Summary of data collected Interim report Evaluat Proj Lead intake Tutors WP lead Quest Focus gp Survey1 info Baseline For whom? Due date Who is respons.? Description & use Activity /report- WP…
  • 13. Tools for monitoring & evaluation
    • (WP2) Surveys –questionnaires – sampling
    • Quantitative – qualitative data collection and evaluation; checklists
    • Other tools for evaluation of learners’ learning (change/ attitude..):
      • i.e. confidence (b) logs: learning development, before and after
      • Recorded/ videoed focus group discussion
      • Electronic tracking of use of technology
  • 14. Short & medium terms objectives
    • Evaluation & QA plan to be elaborated
    • In consultation with partners/ WP leaders
    • Identifying quantitative & qualitative KPIs
      • Milestones/ feedbacks/ reports
      • Establishing monitoring ‘schedule’ or tracking outputs plan
      • Parallel QA activities through monitoring
    • Interface with external experts and focus group