Pekka Silvén Director Professional Development Services Evaluation, Quality and Feedback
Pekka’s ”cv” <ul><li>Evaluation, quality and feedback </li></ul><ul><li>Use of Technology in Healthcare Services in  Techn...
We all agree on… that feedback (queries) has an essential role in improving the  teaching, learning, arrangements, managem...
1/2001 The common problems of  evaluation / feedback processes <ul><li>expensive… </li></ul><ul><li>lots of work and paper...
  “ Right Things Right” “ the Zef-method”
 
 
 
ZEF Solutions Oy
Zef-method vs. Likert Scale ZEF Solutions Oy <ul><li>Course feedback  - Company visits </li></ul><ul><li>Study material - ...
Averages and Gaussian curve OR  Priotising with Z-score
Excel/SPSS? Index? Ranking list? Average or the direction / priorities Reference material(X) Reference material(Y) Home  A...
Euros  Quality Key figures Present state Target Feedback   Action plan attractiveness of the course  ( the number of stud...
Benchmarking
<ul><li>Customer satisfaction </li></ul><ul><li>Response rate 83% </li></ul><ul><li>How have we succeeded in serving you? ...
PwC: Recession survey Blue: SMEs
Red: Large companies

1. International networks
2. Focusing on real customer needs
3. Mar...
Employee well-being 1. I am valued and trusted
2. Fairness in the workplace
3. Giving and receiving feedback
4. Communicat...
Hazard and risk assessments Response rate: 73%
Statutory hazard and risk assessments

To what extent do you see the follow...
Innovation evaluations Which innovations are put into practice?

Confidential ideas.
Organisation's self-evaluation Section 4: Information and knowledge management :
1. We monitor the fulfilment and developm...
<ul><li>Strategy analysis </li></ul><ul><li>Response rate 88% </li></ul><ul><li>Case: TEKES, $200M FinnWell program </li><...
Values of the Centre Party 15 largest towns and cities vs. other towns

1. Patriotism
2. Family and friends
3. Equality 
4...
 
 
<ul><li>All Answers </li></ul><ul><li>Explanations </li></ul><ul><li>Blogs </li></ul><ul><li>Wikis </li></ul><ul><li>Podca...
Greatest Potential <ul><li>All Answers </li></ul><ul><li>Explanations </li></ul><ul><li>Blogs </li></ul><ul><li>Wikis </li...
Revised Competence might help to recognize the Potential <ul><li>All Answers </li></ul><ul><li>Explanations </li></ul><ul>...
Revise your Competence <ul><li>All Answers </li></ul><ul><li>Explanations </li></ul><ul><li>Blogs </li></ul><ul><li>Wikis ...
<ul><li>IT Specialists </li></ul><ul><li>Teachers with eLearning Experience </li></ul>Mobile Technology Questionnaire:  Cl...
Evaluation Engine query excample:  http://www.zef.fi/service/user/?q=425   Shall we try “Zefing” EduZef example: http://ww...
Services / Solutions: ZEF® Evaluation Engine ZEF® Comparison Engine Other solutions Consultant services 1995 - Idea  1997 ...
 
 
 
 
MTV3's European Election Engine (EU-vaalikone) helps you find the best MEP candidate to vote for. By answering the questio...
MTV3: Presidential Election Engine More than half a million users. Should fertility treatments be allowed for female coupl...
Talouselämä newspaper: Economic Election Engine The Economic Election Engine (Talousvaalikone) focuses on the hard core of...
Pekka Silvén Director Professional Development Services Oulu University of Applied Sciences [email_address] +358 10 27 23195
Upcoming SlideShare
Loading in...5
×

Right Things Right: ZEF Evaluation

405

Published on

Pekka Silven presents the ZEF Tool Evaluation software at CCDE

Published in: Technology, Business
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
405
On Slideshare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
4
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide
  • But before that…let´s go back to basics… It all matters a good evaluation and feedback method… please don’t run away! I know that you might be thinking that this is waste of time and boring! That is the way I also used to think before. Today I am more than excited about this!
  • I know all about this too well and it makes me mad every time when I see people struggling with the feedback forms and senseless queries.
  • This is a simple example of the course feedback! Course feedback is typically made at the end of the course. I would like to ask why. Also the students might like to ask what is it good for them? Why not to ask already at the beginning of the course or after the few lessons. That is the only way to be able to affect also the on-going course members to benefit of the feedback. The results should be public (at least for the group).
  • In Finland each student will bring about 7000 dollars each year  max 4 years The maximum number of students for each year is defined by the goverment We can have more, but we won’t get the money of extra persons. So, you must understand that these are some of the key figures in our University. Without a good success in figures, we would be in trouble. What are the reasons for the students to apply for our school? Where and how did they received the information and made them to choose our University? What are they worried about at the starting phase of the studies  reason for the delays of the studies  reason for quitting the studies
  • Right Things Right: ZEF Evaluation

    1. 1. Pekka Silvén Director Professional Development Services Evaluation, Quality and Feedback
    2. 2. Pekka’s ”cv” <ul><li>Evaluation, quality and feedback </li></ul><ul><li>Use of Technology in Healthcare Services in Technology Health Centers </li></ul><ul><li>International training programs and projects </li></ul><ul><li>EU-projects </li></ul><ul><li>Developmental neuropsychology </li></ul><ul><li>Teacher </li></ul><ul><li>Freelance radio journalist </li></ul>
    3. 3. We all agree on… that feedback (queries) has an essential role in improving the teaching, learning, arrangements, management, well being and all the other important processes in any ( educational) organisation But….. … do we really succeed well with this task in real life? Feedback / Evaluation processes
    4. 4. 1/2001 The common problems of evaluation / feedback processes <ul><li>expensive… </li></ul><ul><li>lots of work and paper wasted… </li></ul><ul><li>the whole process takes too much time… </li></ul><ul><li>results are unclear and difficult to make conclusions… </li></ul><ul><li> lack of improvements based on the feedback… </li></ul><ul><li>goal and purpose of the process / study remains unclear.. </li></ul><ul><li>difficulties to see the effectiveness… </li></ul>
    5. 5. “ Right Things Right” “ the Zef-method”
    6. 9. ZEF Solutions Oy
    7. 10. Zef-method vs. Likert Scale ZEF Solutions Oy <ul><li>Course feedback - Company visits </li></ul><ul><li>Study material - On-line support </li></ul><ul><li>Interactivity with the other students </li></ul>1 2 3 4 5 X = 1,6
    8. 11. Averages and Gaussian curve OR Priotising with Z-score
    9. 12. Excel/SPSS? Index? Ranking list? Average or the direction / priorities Reference material(X) Reference material(Y) Home Assignments (X) Home Assignments (Y) Interactivity with the teacher(X) Interactivity with the teacher (Y) Equipments (X) Equipments (Y) 1,86 3,62 1,97 4,12 3,83 3,83 4,15 3,45 4,83 4,8 4,71 4,88 4,71 4,85 4,46 4,75 4,45 4,51 4,23 4,77 4,66 4,82 3,14 4,37 4,74 4,8 3,75 3,78 4,29 4,42 4,57 3,72 4,68 4,66 4,66 4,51 4,46 4,65 4,11 4,65 3,85 4,23 4,12 3,78 3,45 3,38 4,46 4,46 4,46 4,37 4,17 4,11 4,38 4,12 4,68 4,8 2,15 4,72 2,92 3,85 1,62 4,66 2,55 4,83 4,94 4,91 4,92 4,92 4,92 4,92 4,92 4,92 4,92 4,65 4,92 4,35 4,71 4,46 4,74 4,08 4,71 4,72 4,71 4,72 4,65 4,74 4,72 4,68 2,09 4,15 2,31 3,92 3,89 4,23 2,28 4,58 4,42 4,37 3,78 4,11 4,15 4,52 4,23 4,69 5 5 5 5 5 5 5 5 3,94 3,85 3,95 4,03 4,85 4,92 4,83 4,92 3,29 4,75 3,43 4,66 3,91 4,68 3,48 4,57 4,74 4,86 4,68 4,8 4,69 4,83 3,69 4,92 4,55 3,43 2,38 3,29 4,28 4,51 3,97 4,49 3,94 4,02 4,95 4,95 4,31 4,48 4,14 4,97 4,89 4,85 4,94 4,94 4,98 4,97 3,89 4,97 3,55 4,8 3,68 4,94 3,52 4,89 4,55 4,92 4,71 4,72 4,25 4,43 4,69 4,85 4,06 4,83 4,75 4,85 4,25 4,25 4,8 4,86 4,82 4,86 1,92 3,58 2,17 3,74 2,35 3,85 2,08 4,31
    10. 13. Euros Quality Key figures Present state Target Feedback  Action plan attractiveness of the course ( the number of students applying for the school / courses) 2,5 3,8 Report –case1  action plan how many students will complete the studies in scheduled time 67 % 85 % Report –case2  action plan How many students will interrupt the studies 25 % 5 % Report –case3  action plan Employment rate 74 % 92 % Report –case4  action plan
    11. 14. Benchmarking
    12. 15. <ul><li>Customer satisfaction </li></ul><ul><li>Response rate 83% </li></ul><ul><li>How have we succeeded in serving you? </li></ul><ul><li>Project Management </li></ul><ul><li>Project Planning </li></ul><ul><li>Systematic Execution </li></ul><ul><li>Communication </li></ul><ul><li>Keeping our promises </li></ul><ul><li>Experience and know-how </li></ul><ul><li>Cooperation </li></ul><ul><li>Dealing with errors and problems </li></ul><ul><li>Flexibility </li></ul><ul><li>Satisfactory results </li></ul>
    13. 16. PwC: Recession survey Blue: SMEs
Red: Large companies

1. International networks
2. Focusing on real customer needs
3. Marketing of innovative products and services
4. Role of suppliers in the value chain of your company
5. Public and university research in your innovation activity
    14. 17. Employee well-being 1. I am valued and trusted
2. Fairness in the workplace
3. Giving and receiving feedback
4. Communication in the workplace
5. Compliance with rules and regulations
6. Development opportunities
7. Communication on difficult topics
8. I support my manager in management
9. My line manager values feedback
10. Use of satisfaction surveys
11.My work produces satisfaction
    15. 18. Hazard and risk assessments Response rate: 73%
Statutory hazard and risk assessments

To what extent do you see the following items as hazards/risk factors:

1. Effect of the work environment on job satisfaction
2. Threat of violence and substance abuse
3. Cooperation between managers and employees
4. Communication in the workplace
5. Uncertainty regarding job security
6. Ambiguity in work delegation and responsibilities
7. Deficiencies in job orientation
8. Lack of support from manager
9. Lack of valuation of your work
10. Inappropriate treatment in the workplace
11. Substance abuse
    16. 19. Innovation evaluations Which innovations are put into practice?

Confidential ideas.
    17. 20. Organisation's self-evaluation Section 4: Information and knowledge management :
1. We monitor the fulfilment and development of our business objectives with appropriate indicators and set target values, in relation to the whole business and to individual processes.
2. We monitor the impact and efficiency of our business processes.
3. We can easily acquire analytical, up-to-date information on our company's performance and its development in relation to our strategic objectives and other points of comparison.
4. Our real-time internal accounting is predictive.
5. Our employees measure, analyse and evaluate the quality of their own work in order to make decisions and improvement measures related to their work in line with the whole company's development measures.
6. Our employees and the company's most important stakeholders receive accurate information as to how the company operates as a whole and in relation to themselves, and how its objectives are fulfilled.
7. We comprehensively manage the measurement data on our company and its functions, and the know-how of our people.
8. In knowledge management we utilise effective, creativity-oriented employer-employee negotiations.
    18. 21. <ul><li>Strategy analysis </li></ul><ul><li>Response rate 88% </li></ul><ul><li>Case: TEKES, $200M FinnWell program </li></ul><ul><li>Components of the Vision </li></ul><ul><li>Increase customer orientation in products and services </li></ul><ul><li>Innovate new diagnostic methods and products </li></ul><ul><li>Innovate new patient care methods and products </li></ul><ul><li>Innovate new biodiagnostics methods and products </li></ul><ul><li>Innovate new biotech patient care products </li></ul><ul><li>Increase user friendliness of the tech and products </li></ul><ul><li>Innovate new preventive care technologies </li></ul><ul><li>Digitalize patient transactions </li></ul><ul><li>Digitalize professional transactions </li></ul><ul><li>Create new services utilizing new technologies </li></ul><ul><li>Increase the use of new technologies in healthcare processes </li></ul><ul><li>Innovate intelligent decision support systems </li></ul><ul><li>Increase pilot cooperation between companies and hospitals </li></ul><ul><li>Facilitate the integration of health care systems </li></ul><ul><li>Facilitate more flexible hospital environments </li></ul><ul><li>Create new logistics systems and solutions </li></ul>
    19. 22. Values of the Centre Party 15 largest towns and cities vs. other towns

1. Patriotism
2. Family and friends
3. Equality 
4. Nature and environment
    20. 25. <ul><li>All Answers </li></ul><ul><li>Explanations </li></ul><ul><li>Blogs </li></ul><ul><li>Wikis </li></ul><ul><li>Podcasting </li></ul><ul><li>Forums </li></ul><ul><li>Instant messaging </li></ul><ul><li>Audio / video conferences </li></ul><ul><li>Emails </li></ul><ul><li>RSS feeds </li></ul><ul><li>Social bookmarking </li></ul><ul><li>Shared web based calendar services </li></ul><ul><li>Sharing of learning materials produced by teachers and students. </li></ul>
    21. 26. Greatest Potential <ul><li>All Answers </li></ul><ul><li>Explanations </li></ul><ul><li>Blogs </li></ul><ul><li>Wikis </li></ul><ul><li>Podcasting </li></ul><ul><li>Forums </li></ul><ul><li>Instant messaging </li></ul><ul><li>Audio / video conferences </li></ul><ul><li>Emails </li></ul><ul><li>RSS feeds </li></ul><ul><li>Social bookmarking </li></ul><ul><li>Shared web based calendar services </li></ul><ul><li>Sharing of learning materials produced by teachers and students . </li></ul>Start with These Services
    22. 27. Revised Competence might help to recognize the Potential <ul><li>All Answers </li></ul><ul><li>Explanations </li></ul><ul><li>Blogs </li></ul><ul><li>Wikis </li></ul><ul><li>Podcasting </li></ul><ul><li>Forums </li></ul><ul><li>Instant messaging </li></ul><ul><li>Audio / video conferences </li></ul><ul><li>Emails </li></ul><ul><li>RSS feeds </li></ul><ul><li>Social bookmarking </li></ul><ul><li>Shared web based calendar services </li></ul><ul><li>Sharing of learning materials produced by teachers and students. </li></ul>Revise your Competence
    23. 28. Revise your Competence <ul><li>All Answers </li></ul><ul><li>Explanations </li></ul><ul><li>Blogs </li></ul><ul><li>Wikis </li></ul><ul><li>Podcasting </li></ul><ul><li>Forums </li></ul><ul><li>Instant messaging </li></ul><ul><li>Audio / video conferences </li></ul><ul><li>Emails </li></ul><ul><li>RSS feeds </li></ul><ul><li>Social bookmarking </li></ul><ul><li>Shared web based calendar services </li></ul><ul><li>Sharing of learning materials produced by teachers and students. </li></ul>Re-evaluate the Potential for Learning
    24. 29. <ul><li>IT Specialists </li></ul><ul><li>Teachers with eLearning Experience </li></ul>Mobile Technology Questionnaire: Clear Need For Communication
    25. 30. Evaluation Engine query excample: http://www.zef.fi/service/user/?q=425 Shall we try “Zefing” EduZef example: http://www.zef.fi/arviointikone/player/?q=837-af98fk7e How to use Zef-tool
    26. 31. Services / Solutions: ZEF® Evaluation Engine ZEF® Comparison Engine Other solutions Consultant services 1995 - Idea 1997 - First evaluation solution 2001 - Founded 2005 - Innofinland award 2007 - Biggest survey ever 2008 - Office founded in silicon valley 18 employees 5 offices in Finland, 1 in California ZEF Solutions Ltd.
    27. 36. MTV3's European Election Engine (EU-vaalikone) helps you find the best MEP candidate to vote for. By answering the questions in the service, you can compare your opinions to those of candidates . MTV3: European Election Engine
    28. 37. MTV3: Presidential Election Engine More than half a million users. Should fertility treatments be allowed for female couples? Shown are the responses from the presidential candidates. Also shown are the averages for all users, and all male and female users
    29. 38. Talouselämä newspaper: Economic Election Engine The Economic Election Engine (Talousvaalikone) focuses on the hard core of parliamentary power: economic administration. In deciding on the State budget, members of parliament also decide on the taxation and benefits of private individuals and organisations. Question shown: Should corporate taxation be reduced? Shown are the responses of the parties and the voters.
    30. 39. Pekka Silvén Director Professional Development Services Oulu University of Applied Sciences [email_address] +358 10 27 23195
    1. A particular slide catching your eye?

      Clipping is a handy way to collect important slides you want to go back to later.

    ×