Presentation given on final project conference, November 2009, Reykjavik

1. Guidance for Educators, Mentors and Students
Final Conference
November 6th 2008
External Evaluation
- Method and Results
of the GEMS user survey
Jón Torfi Jónasson (jtj@hi.is) and
Anne-Christin Tannhäuser (annechr@hi.is)
Andrea G. Dofradóttir (annechr@hi.is)
Social Science Research Institute
University of Iceland
Jul 13, 2009 Jón Torfi Jónasson, Anne-Christin Tannhäuser, 1
Andrea G. Dofradóttir

2. Outline of the Presentation
 Short note on external evaluations
 Timeline of evaluation process
 GEMS material user survey
 Method
 Results
 Conclusions
Jul 13, 2009 Jón Torfi Jónasson, Anne-Christin Tannhäuser, 2
Andrea G. Dofradóttir

3. Short note: external evaluations
 Framework for Leonardo projects (Leonardo UK
National Agency)
 What should be evaluated?
 the output – what was done;
 and the process – how the output was achieved, including
management and transnational partnership
 2 steps
 Interim evaluation: formative
 Final evaluation: summative
Jul 13, 2009 Jón Torfi Jónasson, Anne-Christin Tannhäuser, 3
Andrea G. Dofradóttir

4. Timeline of GEMS evaluation activities
Dec 2007 Partner survey
Jan 2008 Formative evaluation report I
Feb 2008 Partner interviews
Apr 2008 GEMS material user survey
Sep 2008 Working paper on results
Jul 13, 2009 Jón Torfi Jónasson, Anne-Christin Tannhäuser, 4
Andrea G. Dofradóttir

5. GEMS material user survey
Participants
Sending/Coordinating
Receiving country Mentors Students Coordinators institution/organization
Ester Mosesson Gymnasium
Sweden 3 4 1
in Sweden
Nowy Sacz School of
Poland 4 2 1
Business in Poland
Stella Maris Hotel and
Belgium 8 10 1
Catering School in Belgium
Kópavogur Grammar School
Iceland 4 4 1
in Iceland
/ / 2 Iðan Fræðslusetur
Southwest College in
Ireland 0 0 0
Ireland
TOTAL 19 20 6
Hospitality/Tourism 6 4
Users
work Kitchen 11 14
sector
Restaurant service 2 2
Jul 13, 2009 Jón Torfi Jónasson, Anne-Christin Tannhäuser, 5
TOTAL 19 20
Andrea G. Dofradóttir

6. GEMS material user survey
Focus of the questionnaire
The POETiC framework
Pedagogical aspects
Organisational aspects
Economic aspects
Technological aspects
i
Cultural aspects
Jul 13, 2009 Jón Torfi Jónasson, Anne-Christin Tannhäuser, 6
Andrea G. Dofradóttir

7. Results
Pedagogical aspects
It helped a great deal
How well or how poorly did the students logbooks/GEMS It helped somewhat
material help you to communicate with... It helped a little
It did not help at all
mentors students
I do not know
... your mentor about your WBL 15 25 25 30 5
... your students about their WBL 5 63 26 5 5
0% 20% 40% 60% 80% 100%
Jul 13, 2009 Jón Torfi Jónasson, Anne-Christin Tannhäuser, 7
Andrea G. Dofradóttir

8. Results
Pedagogical aspects
I strongly agree
Do you agree or disagree with the following statement? The students
I agree
logbooks/GEMS material helped me...
I disagree
I strongly disagree
... to make the WBL more effective. 84 11 5
mentors
... to keep track with the learning of my
68 16 16
students.
... to organise my students' work placement
5 63 21 11
period.
students
... to organise my learning at my work
30 40 25 5
placement.
0% 20% 40% 60% 80% 100%
Jul 13, 2009 Jón Torfi Jónasson, Anne-Christin Tannhäuser, 8
Andrea G. Dofradóttir

9. Results
Pedagogical aspects
How effective or ineffective is the GEMS material in It is very effective
making learning outcomes more transparent? It is rather effective
It is slightly effective
coordinator 17 67 17
0% 20% 40% 60% 80% 100%
Jul 13, 2009 Jón Torfi Jónasson, Anne-Christin Tannhäuser, 9
Andrea G. Dofradóttir

10. Results
Organisational aspects
It was rather easy
How easy or how difficult was it for you to... It was rather difficult
It was very difficult
mentors students
... integrate the student logbooks into your
work placement?
25 55 10
... fit the GEMS material into your work
arrangements? 53 37 11
0
0,0 20,0 40,0 60,0 80,0 100,0
Jul 13, 2009 Jón Torfi Jónasson, Anne-Christin Tannhäuser, 10
Andrea G. Dofradóttir

11. Results
Economic aspects
Do you agree with the following statement? Working with the
GEMS material is too time-consuming. I strongly agree
I agree
10 55 30 5 I disagree
students
I strongly disagree
I do not know
mentors 5 26 53 11 5
0% 20% 40% 60% 80% 100%
Jul 13, 2009 Jón Torfi Jónasson, Anne-Christin Tannhäuser, 11
Andrea G. Dofradóttir

12. Results
Economic aspects
In your opinion, of how much or how little use is the GEMS
material for making the students work-based learning more
efficient in your training sector?
It is very efficient
mentors 5 84 5 5 It is somewhat efficient
It is somewhat unefficient
0% 20% 40% 60% 80% 100% I do not know
Jul 13, 2009 Jón Torfi Jónasson, Anne-Christin Tannhäuser, 12
Andrea G. Dofradóttir

13. Results
Cultural aspects
How well or how poorly did the students logbook/ the GEMS Very well
material match with ... Rather well
Rather poorly
Very poorly
mentors students
... what you were learning at your work
5 50 35 10
placement
... your working culture? 5 74 16 5
0% 20% 40% 60% 80% 100%
Jul 13, 2009 Jón Torfi Jónasson, Anne-Christin Tannhäuser, 13
Andrea G. Dofradóttir

14. Results
Future use of the GEMS material
Are you planning to use the GEMS material in I will use it to some extent
educating your future students?
I am willing to use the guide if
changes will be made
mentors 63 5 5 26 I will not use it
I do not know
0% 20% 40% 60% 80% 100%
Jul 13, 2009 Jón Torfi Jónasson, Anne-Christin Tannhäuser, 14
Andrea G. Dofradóttir

15. Conclusions: The GEMS logbooks
Pedagogical aspects – The material ...
 assures awareness of rights and obligations during
the WBL,
 creates a dialogue between mentors, students and
coordinators
 offers a new approach for mentors and students to
WBL in this sector
 enhances transparency of learning outcomes, and
can be enhanced further
Challenge
 Keep students motivated
 Clear indicators on choosing mentors and
companies
Jul 13, 2009 Jón Torfi Jónasson, Anne-Christin Tannhäuser,
Andrea G. Dofradóttir
15

16. Conclusions: The GEMS logbooks
Organisational aspects
 Mixed feedback
Challenge
 Adaptability into specific workplaces
 Flexibility concerning logbook contents, avoid
prescriptiveness, suitable to company profiles
“In my opinion GEMS materials can and will not define range of
knowledge needed in my trade. But still the GEMS materials help
to define how to work with the students.”
[mentor, tourism and hospitality]
Jul 13, 2009 Jón Torfi Jónasson, Anne-Christin Tannhäuser, 16
Andrea G. Dofradóttir

17. Conclusions: The GEMS logbooks
Economic aspects – A Challenge
 Recommendation: Reduce time expenditure for
usage
Technical and cultural aspects
 Translation important to users, comprehensible
 Preference for digital version
Jul 13, 2009 Jón Torfi Jónasson, Anne-Christin Tannhäuser, 17
Andrea G. Dofradóttir

18. General Conclusions
 Suggested improvements mainly concern
communication channels and operational
details
 Successful procedure to develope material
 Output (logbooks) and dissemination
 rather positive attitude of mentors concerning
future use of logbooks and collaboration for
development
 transnational partnership
Jul 13, 2009 Jón Torfi Jónasson, Anne-Christin Tannhäuser, 18
Andrea G. Dofradóttir

19. Thank you
Jón Torfi Jónasson (jtj@hi.is) and
Anne-Christin Tannhäuser (annechr@hi.is)
Andrea G. Dofradóttir (andread@hi.is)
Social Science Research Institute
University of Iceland
Jul 13, 2009 Jón Torfi Jónasson, Anne-Christin Tannhäuser, 19
Andrea G. Dofradóttir