Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Designing and evaluating incidental learning

558 views

Published on

Presented at the CALRG Annual Conference, June 2015. http://www.open.ac.uk/research-groups/calrg/conferences/annual-conference
Abstract
Over the past three years we have been developing a framework intended to facilitate both the occurrence and evaluation of incidental learning. This work has been part of the MASELTOV project, supported by the European Commission, eInclusion programme FP7-ICT-7. The project intends to exploit the potential of mobile services for promoting integration and cultural diversity in Europe, and is focusing on support for immigrants with particular needs e.g. those who have not learned foreign languages, and who have a cultural background that contrasts with that of their host country.
 
The goal of the Incidental Learning Framework is to facilitate the creation of technology rich learning opportunities which emanate from incidental learning i.e. learning that is spontaneous and unplanned. As a design tool its use should encourage links and triggers to structured and reflective learning to back up and deepen learning that happens incidentally. The framework is a descriptive mechanism that enhances analysis and evaluation of incidental learning, and a generative tool to support discussions around software system design. It facilitates the communication of learning design ideas both visually and textually.
 
Initial work was presented at the CALRG conference in 2012, and the framework has been under development since that time (Brasher et al., 2012, Gaved et al., 2013, Kukulska-Hulme et al., in press). Development of the framework has included expert evaluation, and changes in response to data acquired from field trials of a mobile app intended to support language and cultural learning. In this paper we describe the development, use and evaluation of the framework to date, and reflect on lessons learnt.

Published in: Education
  • Be the first to comment

  • Be the first to like this

Designing and evaluating incidental learning

  1. 1. www.maseltov.eu Designing and evaluating incidental learning Andrew Brasher, Ann Jones, Agnes Kukulska- Hulme, Mark Gaved, Eileen Scanlon, Lucy Norris IET, The Open University
  2. 2. www.maseltov.eu Maseltov project “Mobile Assistance for Social Inclusion and Empowerment of Immigrants with Persuasive Learning Technologies and Social Network Services” www.maseltov.eu Joanneum Research, Austria CURE - Centre for Usability Research & Engineering, Austria University of Applied Sciences, FH Joanneum, Austria Athens Information Technology, Greece busuu.com -, Spain Universitat Oberta de Catalunya, Spain Open University, UK Coventry University, UK Czech Technical University, Czech Republic Telecom Italia SpA, Italy Fluidtime Data Service GmbH, Austria Fundacian Desarrollo Sostenido, Spain Verein Danaida, Austria Migrants Resource Centre, UK Pearson Publishing, UK
  3. 3. www.maseltov.eu Contents • Overview of the Incidental Learning Framework • Evaluation & use – Workshop at plenary meeting – expert researchers – Iliinsky’s work • Challenges –Use in evaluation: analysis of MK trial data • Conclusions
  4. 4. www.maseltov.eu Research questions Research question driving the development of the Incidental Learning Framework
  5. 5. www.maseltov.eu Goals of the Incidental Learning Framework • to facilitate the creation of technology rich learning opportunities which emanate from incidental learning i.e. learning that is spontaneous and unplanned; • to encourage links and triggers to structured and reflective learning to back up and deepen learning that happens incidentally; • to enhance analysis and evaluation of incidental learning; • to support discussions around software system design; • to facilitate the communication of learning design ideas both visually and textually.
  6. 6. www.maseltov.eu Review of frameworks • Design frameworks –to support the planning and instantiation of new learning events and situations • Analytical frameworks –for analysing, understanding and evaluating learning events • Learner models –to describe the state of the learner so that a system can react appropriately
  7. 7. www.maseltov.eu Review of frameworks • Design frameworks • Analytical frameworks • Learner models Design framework Analytical framework
  8. 8. www.maseltov.eu Examples of frameworks Analytical • Theory of Learning for the Mobile Age (Sharples et al., 2007) • A Pedagogical Framework for Mobile Learning (Park, 2011) Design • Ecology of Resources design framework (Luckin, 2010) • Four dimensional framework (de Freitas et al., 2010) • Language learning defined by time and place (Kukulska-Hulme, 2012) Learner model • Advances in learner & skill modeling in intelligent learning environments (Desmarais and Baker, 2011)
  9. 9. www.maseltov.eu Kolb’s learning cycle
  10. 10. www.maseltov.eu ILF version 1 Outcomes Social Tasks Tools Plac e Learner’s journey Time Incident: understandi ng notice board Language tool Start TextLens Task: finding station Task: asking directions Reflect/plan Structured learning Language Tool/Social Task: navigating Navigation tool Language Tool Language Tool/Social
  11. 11. www.maseltov.eu Example, stage 1
  12. 12. www.maseltov.eu Example, stage 2
  13. 13. www.maseltov.eu Evaluation • Workshop: adding detail to scenarios • Expert focus group
  14. 14. www.maseltov.eu Evaluation The framework • “generates helpful discussions about the learner journeys which immigrants might make” • Exposes “misunderstandings and different interpretations of the same problem space through different domain experts views” • The current representation was sufficient to engage participants and encourage discussion. (Gaved et al., 2013)
  15. 15. www.maseltov.eu Evaluation • Ordering of dimensions was questioned • Outer dimension of “learner’s journey” was debated: - too general? - how to show e.g. social inclusion in to a new community (Gaved et al., 2013)
  16. 16. www.maseltov.eu Extension ideas (1) Visualisation of specific learner journeys, showing opportunities for planned and incidental language learning Gaved et al. (2012)
  17. 17. www.maseltov.eu Extension ideas (2) Visualisation of the range of affordances that may be associated with particular places that a MASELTOV service user is likely to encounter during their daily activities
  18. 18. www.maseltov.eu Extension ideas (3) Visualisation of the range of dimensions associated with motivation and concerns that might be felt by the service user while engaging with mobile situated language learning
  19. 19. www.maseltov.eu Choosing visual properties Iliinsky, N. (2013)
  20. 20. www.maseltov.eu Place • Not just a location - also contextual information. • ‘A Place is a geographical and historical context for Names and Locations’ (Gillies, 2011). • a ‘Place’ has a name and a location, and each may be applicable for a particular time period. “Andrew’s workplace” currently has - Name: “Jennie Lee Building” - Location: map reference SP 88626 37058
  21. 21. www.maseltov.eu Learning outcomes Categorical Related
  22. 22. www.maseltov.eu Linear view Place Task Tools Outcomes Social Time Evening Language learning Practise work related vocabulary Understand work related vocabulary Sleep/Break 19.00 23.00 07.00 Language learning Practise past tense Communicate successfully in work related situation Breakfast 08.00
  23. 23. www.maseltov.eu Data gathering • Pre trial questionnaire: paper • Post trial interviews: face-to-face E.g. Lesson access duration: 2’39s Synchronous & located with MApp use • Event data from the MApp • Usage of other apps • Geo contextual data Asynchronous & not located with MApp useWeekly survey: online
  24. 24. www.maseltov.eu Participant’s use of Mapp services I have really liked the English course, it is very practical for the times when you have to use English for different occasions..it has helped me a lot The second is the translation tool because now I take a photo of everything I don't understand and at that moment I know what it says ... It has helped me a lot. I have learned a lot more English language I have learned to develop myself in different aspects of speaking English with the help of the course
  25. 25. www.maseltov.eu Use in analysis • Example language lesson data key value timestamp lesson transport 22/01/2015 13:22 publication mas_trav 22/01/2015 13:22 score 90.90908813 22/01/2015 13:22 lesson plan_journey 24/01/2015 18:41 publication mas_trav 24/01/2015 18:41 score 0 24/01/2015 18:41 lesson plan_journey 24/01/2015 18:41 publication mas_trav 24/01/2015 18:41 score 0 24/01/2015 18:41 lesson transport_chaos 24/01/2015 20:07 publication mas_trav 24/01/2015 20:07 score 100 24/01/2015 20:07 lesson body_and_health 24/01/2015 20:59 publication mas_health 24/01/2015 20:59 score 85.7142868 24/01/2015 20:59
  26. 26. www.maseltov.eu Conclusions • Development of ILF as a design tool –Work to do is guided by literature and experimental findings • Development of ILF as an analytical tool –Data available complex and multi-faceted; –Initial experiments started; –Transfer of design view not straightforward? –Learner models?; –Suggestions welcome! Levels of abstraction? Level of detail?
  27. 27. www.maseltov.eu To do • Include stuff from conclusions slide from 2012 Calrg presentation • …feedback from eval D7.12. • Other stuff…. • Issues – learning outcomes can be shown at gps locations, but need to show learning journey • To point to new developments – show new version
  28. 28. www.maseltov.eu Extensions • D7.1.2 i.e. - SCAMP - Learning journey
  29. 29. www.maseltov.eu References • de Freitas, S., Rebolledo-Mendez, G., Liarokapis, F., Magoulas, G. & Poulovassilis, A. 2010. Learning as immersive experiences: Using the four-dimensional framework for designing and evaluating immersive learning experiences in a virtual world. British Journal of Educational Technology, 41, 69-85. • Desmarais, M. C. & Baker, R. S. J. D. 2011. A review of recent advances in learner and skill modeling in intelligent learning environments. User Modeling and User-Adapted Interaction, 22, 9-38. • Kukulska-Hulme, A. 2012. Language learning defined by time and place: A framework for next generation designs. In: D´ıAZ-VERA, J. E. (ed.) Left to My Own Devices: Learner Autonomy and Mobile Assisted Language Learning. • Iliinsky, N. (2013). Choosing visual properties for successful visualizations. Retrieved 24/4/2014, from http://bit.ly/successfulvis • Luckin, R. 2010. Re-Designing Learning Contexts: Technology-Rich, Learner-Centred Ecologies (Foundations and Futures of Education). 208. • Park, Y. 2011. A Pedagogical Framework for Mobile Learning: Categorizing Educational Applications of Mobile Technologies into Four Types. International Review of Research in Open and Distance Learning, 12, 78-102 • Sharples, M., Taylor, J. & Vavoula, G. 2007. A Theory of Learning for the Mobile Age. In: Andrew, R. & Haythornwaite, C. (eds.) The Sage Handbook of Elearning Research. London: Sage.

×