MyPlan - similarity metrics for matching lifelong learner timelines

319 views

Published on

Published in: Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
319
On SlideShare
0
From Embeds
0
Number of Embeds
7
Actions
Shares
0
Downloads
5
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

MyPlan - similarity metrics for matching lifelong learner timelines

  1. 1. 02 December 2008 MyPlan - Similarity Metrics for Matching Lifelong Learner Timelines Nicolas Van Labeke
  2. 2. Using Similarity Metrics for Matching Lifelong Learners 2 The Context • Lifelong Learners? – Learning opportunities – All ages, all contexts • Role of Technology? – Ubiquitous access to resources and facilities – Learner-centred models of organising and delivering educational resources • Better support for planning?
  3. 3. Using Similarity Metrics for Matching Lifelong Learners 3 The MyPlan project • funded by the JISC e-Learning Capital programme, 1/9/2006 – 30/11/2008 (RA 1/4/2007 – 30/7/2008) • developing, deploying and evaluating new techniques and tools that allow personalised planning of lifelong learning • building on and extending the earlier L4All project and software prototype, funded by the JISC Distributed e-Learning Pilots programme 1/2/2005 – 31/10/2006
  4. 4. Using Similarity Metrics for Matching Lifelong Learners 4 Partners (MyPlan) • Birkbeck College – 80% of students are part-time • Institute of Education • Community College Hackney – A Level, GCSE, adult learning courses, teacher training and vocational qualifications • UCAS – UK central organisation through which applications are processed for entry to HE, providing information and services to prospective students and HE professionals. • Linking London Lifelong Learning Network (L4N) – support lifelong learners in the London region, providing them with access to information and resources that facilitates their progression from Secondary Education, through to Further Education (FE) and on into Higher Education
  5. 5. Using Similarity Metrics for Matching Lifelong Learners 5 L4ALL – Approach • Taking a holistic view of lifelong learners’ work and learning experience • Based on the notion of learning pathways • Sharing learning pathways with others: – identifying learning opportunities that may not otherwise have been considered – positioning successful learners “like me” as role models
  6. 6. Using Similarity Metrics for Matching Lifelong Learners 6 L4ALL – Methodology • User requirements elicitation, via interviews with HE and FE students, focus groups (educators, recruitment & careers specialists), workshop events, consultation with advisors: – use cases – examples of learning pathways – identification of critical decision points • Technical requirements elicitation – development of tools and standards – use of existing e-services where possible • User-centred design – Iterative & incremental prototyping – Usability
  7. 7. Using Similarity Metrics for Matching Lifelong Learners 8 L4ALL – Supporting Engagement & Participation 1. Lifelong learners require support not only at the level of the individual user but also at the level of a group or team, and of the learning community as a whole 2. There are critical decision points or periods where lifelong learners need increased support 3. A partnership between the different stakeholders (e.g. lifelong learners themselves but also learning providers, career advisors, adult learning organisations) is an important element in offering a holistic approach to personal development.
  8. 8. Using Similarity Metrics for Matching Lifelong Learners 9 L4ALL – Personalising the pathway through lifelong learning • Breaking the “one size fit all” mould • Recognition of diversity • Different interaction at different stage of the journey – Motivation – Curriculum – Logistic – Pedagogy – Assessment – Opportunity Why should I learn? What can I learn? How could I study? How will I learn? How do I know I've learned? Personalised needs-benefits analysis Access to advice, guidance, learners’ case studies Curriculum choice through HE partnerships Closer links to work and community Adaptive, interactive learning Communication, collaboration Assessment when ready Progress files, e-portfolios Access to information & guidance Qualifications - career options planner Flexible modes, locations etc. Mix of home, campus, overseas Where will it take me?
  9. 9. Using Similarity Metrics for Matching Lifelong Learners 10 L4ALL – Lifelong Learning for All The System • Timeline: record of a user’s learning trail – Educational, professional and personal • A web-based portal for lifelong learners – Access information about courses – Manage personal development plan – Annotate, Reflect & Share • Pilot System – Incremental design – Simple Service-Oriented Architecture – Ontology-based Learner Model (RDF - JENA) • Skeleton of a Social Network Platform?
  10. 10. Using Similarity Metrics for Matching Lifelong Learners 11 L4ALL – System Architecture
  11. 11. Using Similarity Metrics for Matching Lifelong Learners 12
  12. 12. Using Similarity Metrics for Matching Lifelong Learners 13 MyPlan - Introducing Personalised Functionalities • To develop and evaluate user models that reflect the needs of the diverse population of lifelong learners. – Lifelong learner ontology, interoperability (H. Baajour) • To allow learners to role-play different learning and career progressions, by integrating game-based applications into the system – Second Life sessions (S. De Freitas) • To enhance individual learners’ engagement with the lifelong learning process by developing, deploying and evaluating personalised functionalities for searching and recommendation of learning opportunities – Personalised search of timelines – Recommendations  Redesigning the GUI
  13. 13. Using Similarity Metrics for Matching Lifelong Learners 14 SIMILE Javascript Timeline – http://simile.mit.edu/timeline/
  14. 14. Using Similarity Metrics for Matching Lifelong Learners 15 Searching the L4ALL User Model • A three-part model – User Profile: identification, personal information, … – Learning Profile: learning goals, skills, qualification, … – Timeline, as set of episodes: description, title, classification, start date, duration, … • Search by keywords  Personalised search for “people like me” – Reflect structure and semantic of timelines – Detect “similarities” between learners’ pathway
  15. 15. Using Similarity Metrics for Matching Lifelong Learners 16 Similarity Metrics • Textual-based metrics with algorithm-specific indication of similarity between 2 strings – “SAM” / “SAMUEL” • Levenshtein Distance (Edit Distance) – number of transpositions, substitutions and deletions needed to transform one string into another • Information integration & applied CS – bioinformatics, musicology, phonetic, etc – ITS: sequence of instructional activities (
  16. 16. Using Similarity Metrics for Matching Lifelong Learners 17 Our approach • Black-box – Reusing existing metrics – Identifying behaviour in the context of timeline • Different interpretations of “people like me” • Focus on usability, not accuracy Tokenisation of Timelines
  17. 17. Using Similarity Metrics for Matching Lifelong Learners 18 Hypothesis 1 & 2 : Time • Timelines are (obviously) time-dependent – Essential for user’s own pathways – No evidence for relevance in “people like me” • Similar episode two years apart? • Similar episode twice as long (part-time)?  Start dates and duration ignored  Gap between episodes ignored  Relative position used to sort episodes
  18. 18. Using Similarity Metrics for Matching Lifelong Learners 19 Hypothesis 3 : Category of episode • Different categories of episodes – Educational – Occupation – Personal • Importance for own pathways – critical turning point • Irrelevant for “people like me”?  Categories to be filtered out by user Description SC Attended school CL Attended college UN Attended University DG Obtained a degree CS Attended a particular course WK Employed VL Voluntary work in charity/voluntary organisation BS Started a business ML Attended military service RE Retired UE Unemployed CR Home carer MV Moved to a different location TV Spent some time abroad CH Birth in the family AD Adopted a child DE Death in the family MA Got married SE Divorced DS Developed a (permanent) disability IL Developed a (temporary) illness OT Any user-defined episode not covered previously
  19. 19. Using Similarity Metrics for Matching Lifelong Learners 20 Hypothesis 4 : Classification of episodes 0.0.0.0 Unknown 1.0.0.0 Managers and Senior Officials 2.0.0.0 Professional Occupations 2.3.0.0 Teaching and Research Professionals 2.3.2.0 Research Professionals 2.3.2.1 Scientific Researchers 2.3.2.2 Social Science Researchers 2.3.2.9 Researchers N.E.C. - -2.3.2.1 6.4.0.0WK Secondary classification (e.g. discipline, activity sector) Primary classification (e.g. qualification, occupation) Episode Category (e.g. work, college, military service, …) 0.0.0.0 Unknow 1.0.0.0 Medicine and Dentistry 6.0.0.0 Mathematical and Computer Sciences 6.4.0.0 Computer Science • Category of episode alone not sufficient • Most important episodes have extra classifications • But fine-grained description may not be useful  User to vary depth of classification
  20. 20. Using Similarity Metrics for Matching Lifelong Learners 21 Tokenisation of TimelinesExpressivity
  21. 21. Using Similarity Metrics for Matching Lifelong Learners 22 Similarity Metrics SimMetrics JAVA package – http://www.dcs.shef.ac.uk/~sam/simmetrics.html Levenshtein Needleman – Wunsch Jaro Matching Coefficient Euclidean Distance Block Distance Jaccard Similarity Cosine Similarity Dice Similarity Overlap Coefficient
  22. 22. Using Similarity Metrics for Matching Lifelong Learners 23 Encoding of some timelines ID Description Encoding Source The original timeline used as the source for the similarity measure Cl-00 Un-00 Mv-00 Wk-00 Id A timeline similar to the source. Cl-00 Un-00 Mv-00 Wk-00 Re A timeline containing the same episodes as the source but in a totally different order (i.e. no episode is at the same position in the string). Un-00 Wk-00 Cl-00 Mv-00 ADe A new work episode (similar to an existing one) is added to the timeline. Cl-00 Un-00 Mv-00 Wk-00 Wk-00 ADn A new episode (different from all existing ones) is added to the timeline. Cl-00 Un-00 Mv-00 Wk-00 Bs-00 RMw The last episode is removed from the source timeline. Cl-00 Un-00 Mv-00 RMu One of the episodes of the source timeline is removed. Cl-00 Mv-00 Wk-00 SBn One of the episodes of the source timeline is substituted by a new one (different from all existing ones). Cl-00 Un-00 Mv-00 Bs-00 SBe One of the episodes of the source timeline is substituted by an existing episode. Cl-00 Un-00 Mv-00 Un-00 SBv One of the episodes of the source timeline is substituted by a variant of an existing episode. Cl-00 Un-00 Mv-00 Wk-10
  23. 23. Using Similarity Metrics for Matching Lifelong Learners 24 Comparison of Metrics ID RE ADe ADn RMw RMu SBn SBe SBv Levenshtein 1 0 0.8 0.8 0.75 0.75 0.75 0.75 0.75 Needleman - Wunsch 1 0 0.8 0.8 0.75 0.75 0.75 0.75 0.88 Jaro 1 0.72 0.93 0.93 0.92 0.92 0.83 0.83 0.83 Matching Coefficient 1 1 0.8 0.8 0.75 0.75 0.75 0.75 0.75 Euclidean Distance 1 1 0.84 0.84 0.8 0.8 0.75 0.75 0.75 Block Distance 1 1 0.89 0.89 0.86 0.86 0.75 0.75 0.75 Jaccard Similarity 1 1 1 0.8 0.75 0.75 0.6 0.75 0.6 Cosine Similarity 1 1 1 0.89 0.87 0.87 0.75 0.87 0.75 Dice Similarity 1 1 1 0.89 0.86 0.86 0.75 0.86 0.75 Overlap Coefficient 1 1 1 1 1 1 0.75 1 0.75 User-defined cost functionsUser-defined cost functions
  24. 24. Using Similarity Metrics for Matching Lifelong Learners 25 Search for “People like me” • “Existential” search • Filtering by – User profile – Episode categories • Tuning by – Classification depth – Similarity Metrics • Ranking by timeline similarity
  25. 25. Using Similarity Metrics for Matching Lifelong Learners 26
  26. 26. Using Similarity Metrics for Matching Lifelong Learners 27 Explaining Similarity Measures • Needleman – Wunsch • Computing alignment of strings – Copy/substituting tokens – Insertion/deletion • Optimal score for alignment of the first i characters in T1 and the first j characters in T2 • Score indicates minimal edit distance • Backtracking for alignment(s) 0 0 0 10 321234D 32123C 432112B 543211A 654321 CBECBA 1 ___ CBE D _ C C B B A A G G d
  27. 27. Using Similarity Metrics for Matching Lifelong Learners 28
  28. 28. Using Similarity Metrics for Matching Lifelong Learners 29 “What should I do next?” • “Recommendation” too strong term – Suggesting reliability & objectivity; difficulty of obtaining expert pathways • Role Model – source of inspiration – This is what people have done after following a pathway similar to yours; why not consider a similar future ?  Exploiting String alignments • Identifying common patterns & possible future pathways • Naïve “Rule of Thumb” approach • Lack of semantic BETWEEN episodes
  29. 29. Using Similarity Metrics for Matching Lifelong Learners 30
  30. 30. Using Similarity Metrics for Matching Lifelong Learners 31
  31. 31. Using Similarity Metrics for Matching Lifelong Learners 32
  32. 32. Using Similarity Metrics for Matching Lifelong Learners 33 Conclusions • Different metrics, different aspects of string comparison – Not one particularly adequate or “better” – Context of use important: what does “people like me” mean? • What are they good for? – Separation between encoding and matching – Encoding does not depend on context, embeds some – not all – of the timeline’s semantic • Persistent storage, indexing, RSS feed, alerts • What are they no so good for? – Discrepancy between string similarity and timeline similarity – Lack of explanation on the reasons for similarity • The way forward? – Identifying contexts of usage and deploying tailored mechanism – User-defined mechanism
  33. 33. Using Similarity Metrics for Matching Lifelong Learners 34 Which Measure of (Dis)similarity? • Needleman – Wunsch – Distance between tokens? – Cost functions • G: gap (insert/delete) • d: distance (substitute) • Normalised Similarity? – algorithm-specific ___ DCBA CBE _CBA E _CBA 66% (4/6) 50% (2/4) Similarity Dissimilarity - -2.3.2.1 6.4.0.0WK - -1.0.0.0 4.2.0.0WK
  34. 34. Using Similarity Metrics for Matching Lifelong Learners 35 An Holistic Approach of Timeline Matching
  35. 35. Using Similarity Metrics for Matching Lifelong Learners 36 Multiple String Alignments
  36. 36. Using Similarity Metrics for Matching Lifelong Learners 37 Future Work (?) • (Multiple) External Representations of timelines AND similarities • Full-fledged Social Network functionalities – Reflection – Help & advice seeking, interventions (peers, institutions, …) • “Recommendation” – Dependencies BETWEEN episodes – Domain knowledge (e.g. course entry profile, alternatives to top-down taxonomies)

×