FROM FINDABILITY TO              AWARENESS: METADATA IN              MUSIC AND TECHNOLOGY               ENHANCED LEARNING ...
http://www.britannica.com/bps/media-view/96031/1/0/0   2Wednesday 12 September 12
3Wednesday 12 September 12
http://www.flickr.com/photos/markkelley/1022720488/   4Wednesday 12 September 12
http://www.flickr.com/photos/markkelley/1022720488/   4Wednesday 12 September 12
METADATA    • title: Maiden Voyage    • main artist: Herbie Hancock    • artists: Freddie Hubbard, George      Coleman, Ro...
CONTEXT.                               6Wednesday 12 September 12
OVERVIEW.                                7Wednesday 12 September 12
OVERVIEW.                                7Wednesday 12 September 12
OVERVIEW.                                7Wednesday 12 September 12
OVERVIEW.                                7Wednesday 12 September 12
OVERVIEW.                                7Wednesday 12 September 12
OVERVIEW.                                7Wednesday 12 September 12
OVERVIEW.                                7Wednesday 12 September 12
OVERVIEW.                                7Wednesday 12 September 12
OVERVIEW.                                7Wednesday 12 September 12
OVERVIEW.                                7Wednesday 12 September 12
OVERVIEW.                                7Wednesday 12 September 12
OVERVIEW.                                     RQ1: How can metadata be                                  leveraged to provi...
OVERVIEW.    RQ2: How can metadata creation       be made more scalable?                                 7Wednesday 12 Sep...
OVERVIEW.        RQ3: How can metadata    standards be selected for specific      functionalities of applications?         ...
OVERVIEW. can attention                                  RQ4: How                                  metadata be leveraged t...
DOMAINS.                            Music          Technology-enhanced                                              Learni...
STEP 1 – DEFINE.                                   9Wednesday 12 September 12
STEP 1 – DEFINE.            RQ3: How can metadata        standards be selected for specific          functionalities of app...
METADATA SCHEMA                            <rdf:RDF                              xmlns:rdf="http://www.w3.org/1999/02/22-r...
METADATA SCHEMA                             @prefix mo: <http://purl.org/ontology/mo/> .                             @prefix...
METADATA SCHEMA                             :track1                               	 a mo:Track ;                          ...
FINDING AN APPROPRIATE               METADATA SCHEMA...                                 Corthaut, Nik; Govaerts, Sten; Ver...
...FOR THE RIGHT                               APPLICATION.                                        Corthaut, Nik; Govaerts...
...FOR THE RIGHT                               APPLICATION.                                        Corthaut, Nik; Govaerts...
STEP 2 – CREATE.                                   14Wednesday 12 September 12
STEP 2 – CREATE.                      RQ2: How can metadata creation                         be made more scalable?       ...
country of origin of an artist. http://www.flickr.com/photos/twicepix/4437851857/                                          ...
COUNTRY & CONTINENT      • useful          for:           • subgenres           • popularity           • recommendations  ...
COUNTRY & CONTINENT      • useful          for:           • subgenres           • popularity           • recommendations  ...
COUNTRY & CONTINENT      • useful          for:           • subgenres           • popularity           • recommendations  ...
MASHUP.                                                                   google maps                                     ...
Three data sources                                    last.fm                           91%        • last.fm              ...
Three data sources                                last.fm                        91%                Ennio Morricone (born ...
Three data sources                                    last.fm                           91%        • last.fm              ...
Three data sources                                    last.fm                           91%        • last.fm              ...
Three data sources                                    last.fm                           91%        • last.fm              ...
Three data sources        • last.fm        • freebase        • biographies                                                ...
Three data sources        • last.fm        • freebase        • biographies                                                ...
LAST.FM MASHUP                     http://www.cs.kuleuven.be/~sten/lastonamfm  step 2 - create.                       19We...
musical genre.http://www.flickr.com/photos/7593077@N03/5076100406/in/photostream/   20Wednesday 12 September 12
USING SEARCH ENGINES  step 2 - create.          21Wednesday 12 September 12
USING SEARCH ENGINES  step 2 - create.          21Wednesday 12 September 12
USING SEARCH ENGINES  step 2 - create.          21Wednesday 12 September 12
USING SEARCH ENGINES  step 2 - create.          21Wednesday 12 September 12
USING SEARCH ENGINES  step 2 - create.          21Wednesday 12 September 12
CLASSIFICATION WITH                        SEARCH ENGINES                            using co-occurrence                  ...
CLASSIFICATION WITH                        SEARCH ENGINES                            using co-occurrence                  ...
step 2 - create.          23Wednesday 12 September 12
Rock:                     Jazz:       Blues:                     Pop:        Country:                 Metal:  step 2 - cre...
Rock:                             Jazz:                            0,013                 0,013       Blues:               ...
RESULTS   • 1st       results were much worse   • what           happened?   • re-run            the original experiment  ...
step 2 - create.          25Wednesday 12 September 12
step 2 - create.          25Wednesday 12 September 12
step 2 - create.          26Wednesday 12 September 12
WHAT TO USE?   • use        Google when it’s stable else rely on Yahoo!   • when            is it stable? test with a smal...
WHAT TO USE?   • use        Google when it’s stable else rely on Yahoo!   • when            is it stable? test with a smal...
STEP 4 – UTILISE.                                    28Wednesday 12 September 12
STEP 4 – UTILISE.                                    28Wednesday 12 September 12
STEP 4 – UTILISE.                                    Ch 5 & 6                                    RQ1: How can metadata be ...
FEDERATED SEARCH.                                    Govaerts, Sten; El Helou, Sandy; Duval, Erik; Gillet, Denis. A federa...
FEDERATED SEARCH.                                    Govaerts, Sten; El Helou, Sandy; Duval, Erik; Gillet, Denis. A federa...
FEDERATED SEARCH.                                    Govaerts, Sten; El Helou, Sandy; Duval, Erik; Gillet, Denis. A federa...
FEDERATED SEARCH.                                    Govaerts, Sten; El Helou, Sandy; Duval, Erik; Gillet, Denis. A federa...
FEDERATED SEARCH.                                    Govaerts, Sten; El Helou, Sandy; Duval, Erik; Gillet, Denis. A federa...
FEDERATED SEARCH.                                    Govaerts, Sten; El Helou, Sandy; Duval, Erik; Gillet, Denis. A federa...
FEDERATED SEARCH.                                    Govaerts, Sten; El Helou, Sandy; Duval, Erik; Gillet, Denis. A federa...
CONTEXT   • Personal               Learning Environment:        • customizable        • re-use, creation          & mashup...
CONTEXT   • Personal               Learning Environment:        • customizable        • re-use, creation          & mashup...
CONTEXT   • Personal               Learning Environment:        • customizable        • re-use, creation          & mashup...
WHAT’S A WIDGET ?!?  step 4 - utilise.                  31Wednesday 12 September 12
WHAT’S A WIDGET ?!?  step 4 - utilise.                  31Wednesday 12 September 12
STEP 5 & 6 – CAPTURE &                       PERSONALISE.                            32Wednesday 12 September 12
STEP 5 & 6 – CAPTURE &                       PERSONALISE.                                      Ch 5 & 6                   ...
ATTENTION METADATA                            Attention metadata does not describe                            the content ...
ATTENTION METADATA                                 Attention metadata does not describe                                 th...
ATTENTION METADATA                                 Attention metadata does not describe                                 th...
UI.  step 4 - utilise.          34Wednesday 12 September 12
UI.  step 4 - utilise.          34Wednesday 12 September 12
SOCIAL RECOMMENDATIONS.                                         Govaerts, Sten; El Helou, Sandy; Duval, Erik; Gillet, Deni...
SOCIAL RECOMMENDATIONS.                                         Govaerts, Sten; El Helou, Sandy; Duval, Erik; Gillet, Deni...
SOCIAL RECOMMENDATIONS.                                         Govaerts, Sten; El Helou, Sandy; Duval, Erik; Gillet, Deni...
SOCIAL RECOMMENDATIONS.                                         Govaerts, Sten; El Helou, Sandy; Duval, Erik; Gillet, Deni...
PAGE RANK                            HYPERLINK              WEB              SITE                                         ...
OUR CASE                                                                                          hare                    ...
THREE CASE STUDIES                       1               2        3   step 5 & 6 - capture/personalise.       38Wednesday ...
KU LEUVEN & EPFL   • 15        PhD students.   step 5 & 6 - capture/personalise.   39Wednesday 12 September 12
KU LEUVEN & EPFL   • 15        PhD students.   • usability, satisfaction,       usefulness   step 5 & 6 - capture/personal...
KU LEUVEN & EPFL   • 15        PhD students.   • usability, satisfaction,       usefulness   • usefulness             of r...
KU LEUVEN & EPFL   • 15        PhD students.   • usability, satisfaction,       usefulness   • usefulness             of r...
KU LEUVEN & EPFL   • 15        PhD students.   • usability, satisfaction,       usefulness   • usefulness             of r...
KU LEUVEN & EPFL   • 15        PhD students.   • usability, satisfaction,       usefulness   • usefulness             of r...
WHY THE DIFFERENT SUS?   • 1st       phase by 2 interviewers   step 5 & 6 - capture/personalise.     40Wednesday 12 Septem...
WHY THE DIFFERENT SUS?   • 1st       phase by 2 interviewers   step 5 & 6 - capture/personalise.     40Wednesday 12 Septem...
WHY THE DIFFERENT SUS?   • 1st       phase by 2 interviewers   step 5 & 6 - capture/personalise.     40Wednesday 12 Septem...
WHY THE DIFFERENT SUS?   •   issues:        • distraction           from unrelated widgets        • layout            too ...
DESIRABILITY...   step 5 & 6 - capture/personalise.   42Wednesday 12 September 12
DESIRABILITY...   step 5 & 6 - capture/personalise.   42Wednesday 12 September 12
DESIRABILITY...   step 5 & 6 - capture/personalise.   42Wednesday 12 September 12
FESTO   • LMS           (CLIX) extended with widgets   • 25        heterogeneous participants   •   evaluation setup:     ...
FESTO   • LMS           (CLIX) extended with widgets   • 25        heterogeneous participants   •   evaluation setup:     ...
FESTO RESULTS   step 5 & 6 - capture/personalise.   44Wednesday 12 September 12
FESTO RESULTS                    88% would recommend it to their colleagues            81% agreed that it helps them to be...
JIAO TONG UNIVERSITY   • offers           higher education for adults with a full-time job   • early          prototype wi...
JIAO TONG UNIVERSITY   • offers           higher education for adults with a full-time job   • early          prototype wi...
RESULTS   • ran        over 3 weeks, 13 respondents   • results           for federated search widget:   •   69% used the ...
WHY?   • an       established LMS augmented with PLE features                  brand new system   • cultural              ...
WHY?   • an       established LMS augmented with PLE features                  brand new system   • cultural              ...
WHY?   • an       established LMS augmented with PLE features                  brand new system   • cultural differences? ...
STEP 7 – CREATE AWARENESS.                            48Wednesday 12 September 12
STEP 7 – CREATE AWARENESS.                                 Ch 7                            48Wednesday 12 September 12
STEP 7 – CREATE AWARENESS.                                                     Ch 7                                RQ4: Ho...
PROBLEM                            http://www.ypfp.org/classroom   step 5 & 7 - capture/create awareness.                 ...
PROBLEM                            http://www.ypfp.org/classroom   step 5 & 7 - capture/create awareness.                 ...
STUDENT ACTIVITY METER   •   visualisations of attention metadata                                                         ...
STUDENT ACTIVITY METER   •   visualisations of attention metadata                                                         ...
STUDENT ACTIVITY METER   •   visualisations of attention metadata                                                         ...
STUDENT ACTIVITY METER   •   visualisations of attention metadata                                                         ...
STUDENT ACTIVITY METER   •   visualisations of attention metadata                                                         ...
Govaerts, Sten; Verbert, Katrien, Duval, Erik; Pardo, Abelardo. The                             Student Activity Meter for...
demogra        evaluation         design                                                              negative            ...
ITERATION TWO &                                  THREE   • CGIAR: 19              teachers - LAK: 12   • An    online surv...
CGIAR              LAK                                              1   2   3   4   issue for2 add- issue for add-        ...
CGIAR              LAK                                                      issue for add- issue for add-                 ...
CGIAR              LAK                                                      issue for add- issue for add-                 ...
CGIAR CASE STUDY   step 5 & 7 - capture/create awareness.      55Wednesday 12 September 12
CGIAR CASE STUDY        want to search           detect outliers         for students                                more ...
CGIAR CASE STUDY        want to search           detect outliers         for students                                more ...
LAK CASE STUDY   step 5 & 7 - capture/create awareness.                                                                   ...
LAK CASE STUDY                                                      find students experiencing                             ...
LAK CASE STUDY                                                      find students experiencing                             ...
demogra        evaluation        design                                                             negative              ...
ITERATION FOUR   •a      CS course on C++ programming   • 11        people: 7 teachers, 2 TA’s & 1 course planner   • rich...
ITERATION FOUR   •a      CS course on C++ programming   • 11        people: 7 teachers, 2 TA’s & 1 course planner   • rich...
ACHIEVABLE TASKS    conflicting visions on get a time                                       identify students doing     stu...
ACHIEVABLE TASKS    conflicting visions on get a time                                       identify students doing     stu...
ACHIEVABLE TASKS    conflicting visions on get a time                                          identify students doing     ...
demo-          evaluation           design                                                                  negative      ...
CONCLUSION   •   SAM is perceived useful and contributes to better       awareness.   step 5 & 7 - capture/create awarenes...
CONCLUSION   •   SAM is perceived useful and contributes to better       awareness.   •   SAM enables:   step 5 & 7 - capt...
CONCLUSION   •   SAM is perceived useful and contributes to better       awareness.   •   SAM enables:        • to      de...
CONCLUSION   •   SAM is perceived useful and contributes to better       awareness.   •   SAM enables:        • to detect ...
CONCLUSION   •   SAM is perceived useful and contributes to better       awareness.   •   SAM enables:        • to detect ...
CONCLUSION   •   SAM is perceived useful and contributes to better       awareness.   •   SAM enables:        • to detect ...
CONCLUSION   •   SAM is perceived useful and contributes to better       awareness.   •   SAM enables:        • to detect ...
CONTRIBUTIONS.    1. A structured method for selecting an appropriate metadata schema    2. The design & evaluation of a m...
Ch 8                            CONTRIBUTIONS.    1. A structured method for selecting an appropriate metadata schema    2...
Ch 8                            CONTRIBUTIONS.                                                                        RQ 3...
Ch 8                            CONTRIBUTIONS.                                                                        RQ 3...
Ch 8                            CONTRIBUTIONS.                                                                        RQ 3...
Ch 8                            CONTRIBUTIONS.                                                                        RQ 3...
Ch 8                            CONTRIBUTIONS.                                                                        RQ 3...
Ch 8                            CONTRIBUTIONS.                                                                        RQ 3...
Ch 8                            CONTRIBUTIONS.                                                                        RQ 3...
Ch 8                            CONTRIBUTIONS.                                                                        RQ 3...
publications. http://dailypracticeblog.com/how-paper-claims-will-be-impacted-by-5010/   63Wednesday 12 September 12
domain         steps         RQ          conf.                               publication               define,             ...
period             project   domain RQ           steps          pubs.         2005 - Aristo IWT       RQ2,                ...
some                metrics...                             66Wednesday 12 September 12
CITATIONS PER YEAR.    total citations: 72    external citations: 49                          30                          ...
CITATIONS PER YEAR.    total citations: 72    external citations: 49                          30                          ...
number of citations                                                   0                                                   ...
number of citations                                                   0                                                   ...
TIME SPENT.          1: schemas         2: origin    2: search     4,5&6: search & recomm.   5&7: awareness               ...
TIME SPENT.          1: schemas          2: origin    2: search      4,5&6: search & recomm.    5&7: awareness            ...
70Wednesday 12 September 12
Thank you for your attention!                          Questions?                                70Wednesday 12 September 12
Upcoming SlideShare
Loading in …5
×

From Findability to Awareness: Metadata in Music and Technology Enhanced Learning.

885
-1

Published on

Slides of my PhD defense on September 11, 2012.

Published in: Technology, Education, Business
0 Comments
4 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
885
On Slideshare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
9
Comments
0
Likes
4
Embeds 0
No embeds

No notes for slide

From Findability to Awareness: Metadata in Music and Technology Enhanced Learning.

  1. 1. FROM FINDABILITY TO AWARENESS: METADATA IN MUSIC AND TECHNOLOGY ENHANCED LEARNING Vinden en beseffen: Metadata in muziek en E-learning Sten Govaerts Promotor: Prof. dr. ir. Erik Duval Co-promotor: Dr. Katrien Verbert 1Wednesday 12 September 12
  2. 2. http://www.britannica.com/bps/media-view/96031/1/0/0 2Wednesday 12 September 12
  3. 3. 3Wednesday 12 September 12
  4. 4. http://www.flickr.com/photos/markkelley/1022720488/ 4Wednesday 12 September 12
  5. 5. http://www.flickr.com/photos/markkelley/1022720488/ 4Wednesday 12 September 12
  6. 6. METADATA • title: Maiden Voyage • main artist: Herbie Hancock • artists: Freddie Hubbard, George Coleman, Ron Carter, Anthony Williams • record label: Blue Note, Capitol Records • instruments: trumpet, tenor saxophone, piano, bass, drums. • recording date: March 17, 1965 • studio: Van Gelder Studio, New Jersey • version: remastered by Rudy Van Gelder 5Wednesday 12 September 12
  7. 7. CONTEXT. 6Wednesday 12 September 12
  8. 8. OVERVIEW. 7Wednesday 12 September 12
  9. 9. OVERVIEW. 7Wednesday 12 September 12
  10. 10. OVERVIEW. 7Wednesday 12 September 12
  11. 11. OVERVIEW. 7Wednesday 12 September 12
  12. 12. OVERVIEW. 7Wednesday 12 September 12
  13. 13. OVERVIEW. 7Wednesday 12 September 12
  14. 14. OVERVIEW. 7Wednesday 12 September 12
  15. 15. OVERVIEW. 7Wednesday 12 September 12
  16. 16. OVERVIEW. 7Wednesday 12 September 12
  17. 17. OVERVIEW. 7Wednesday 12 September 12
  18. 18. OVERVIEW. 7Wednesday 12 September 12
  19. 19. OVERVIEW. RQ1: How can metadata be leveraged to provide efficient and 7 effective access to content?Wednesday 12 September 12
  20. 20. OVERVIEW. RQ2: How can metadata creation be made more scalable? 7Wednesday 12 September 12
  21. 21. OVERVIEW. RQ3: How can metadata standards be selected for specific functionalities of applications? 7Wednesday 12 September 12
  22. 22. OVERVIEW. can attention RQ4: How metadata be leveraged to improve awareness and self-reflection? 7Wednesday 12 September 12
  23. 23. DOMAINS. Music Technology-enhanced Learning (TEL) 8Wednesday 12 September 12
  24. 24. STEP 1 – DEFINE. 9Wednesday 12 September 12
  25. 25. STEP 1 – DEFINE. RQ3: How can metadata standards be selected for specific functionalities of applications? 9Wednesday 12 September 12
  26. 26. METADATA SCHEMA <rdf:RDF xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#" xmlns:dc="http://purl.org/dc/elements/1.1/"> <rdf:Description rdf:about="http://media.example.com/audio/guide.ra"> <dc:creator>Herbie Hancock</dc:creator> <dc:title>Maiden Voyage</dc:title> Dublin Core <dc:contributor>Freddie Hubbard</dc:contributor> <dc:contributor>George Coleman</dc:contributor> <dc:contributor>Ron Carter</dc:contributor> <dc:contributor>Anthony Williams</dc:contributor> <dc:description>Maiden Voyage is the fifth album led by jazz musician Herbie Hancock, and was recorded by Rudy Van Gelder on May 17, 1965 for Blue Note Records.</dc:description> <dc:date>1965-03-17</dc:date> <dc:publisher>Blue Note</dc:publisher> <dc:type>Music album</dc:type> <dc:format>audio/mp3</dc:format> <dc:rights>copyrighted by Capitol Records, Inc.</dc:rights> </rdf:Description> </rdf:RDF> step 1 - define. 10Wednesday 12 September 12
  27. 27. METADATA SCHEMA @prefix mo: <http://purl.org/ontology/mo/> . @prefix : <#> . @prefix dc: <http://purl.org/dc/elements/1.1/> . @prefix xsd: <http://www.w3.org/2001/XMLSchema#> . @prefix event: <http://purl.org/NET/c4dm/event.owl#> . :work Music Ontology a mo:MusicalWork ; dc:title "Herbie Hancock - Maiden Voyage" . :performance a mo:Performance ; dc:title "Maiden Voyage" ; mo:performance_of :work ; mo:performer <http://dbpedia.org/page/Herbie_Hancock> ; dc:date "1965-03-17"^^xsd:date . event:place <http://dbpedia.org/page/Van_Gelder_Studio> ; mo:performance_of <http://dbpedia.org/page/ Maiden_Voyage_(Herbie_Hancock_album)> . mo:recorded_as [ a mo:Signal ; mo:published_as :track1 ; dc:title "Original recording at the Van Gelder Studio" ; ]. step 1 - define. 11Wednesday 12 September 12
  28. 28. METADATA SCHEMA :track1 a mo:Track ; mo:track_number 1 ; dc:title "Maiden Voyage". :signal mo:time [ tl:duration "PT2M14S"^^xsd:duration; Music Ontology tl:timeline :tl; ]. :chorus1 a ps:Chorus; rdfs:label "First chorus"; event:time [ tl:timeline :tl; tl:start "PT0S"^^xsd:duration; tl:duration "PT9S"^^xsd:duration; ]. :verse1 a ps:Verse; rdfs:label "First verse"; event:time [ tl:timeline :tl; tl:start "PT9S"^^xsd:duration; tl:duration "PT33S"^^xsd:duration; step 1 - define. ]. 11Wednesday 12 September 12
  29. 29. FINDING AN APPROPRIATE METADATA SCHEMA... Corthaut, Nik; Govaerts, Sten; Verbert, Katrien; Duval, step 1 - define. 12 Erik. Connecting the dots: music metadata generation, schemas and applications. ISMIR 2008.Wednesday 12 September 12
  30. 30. ...FOR THE RIGHT APPLICATION. Corthaut, Nik; Govaerts, Sten; Verbert, Katrien; Duval, step 1 - define. 13 Erik. Connecting the dots: music metadata generation, schemas and applications. ISMIR 2008.Wednesday 12 September 12
  31. 31. ...FOR THE RIGHT APPLICATION. Corthaut, Nik; Govaerts, Sten; Verbert, Katrien; Duval, step 1 - define. 13 Erik. Connecting the dots: music metadata generation, schemas and applications. ISMIR 2008.Wednesday 12 September 12
  32. 32. STEP 2 – CREATE. 14Wednesday 12 September 12
  33. 33. STEP 2 – CREATE. RQ2: How can metadata creation be made more scalable? 14Wednesday 12 September 12
  34. 34. country of origin of an artist. http://www.flickr.com/photos/twicepix/4437851857/ 15Wednesday 12 September 12
  35. 35. COUNTRY & CONTINENT • useful for: • subgenres • popularity • recommendations step 2 - create. 16Wednesday 12 September 12
  36. 36. COUNTRY & CONTINENT • useful for: • subgenres • popularity • recommendations • expensive to annotate step 2 - create. 16Wednesday 12 September 12
  37. 37. COUNTRY & CONTINENT • useful for: • subgenres • popularity • recommendations • expensive to annotate • very little existing research step 2 - create. 16Wednesday 12 September 12
  38. 38. MASHUP. google maps freebase last.fm google app engine twitter last.fm last.fm yahoo! pipes last on am/fm website dapper youtube step 2 - create. 17Wednesday 12 September 12
  39. 39. Three data sources last.fm 91% • last.fm freebase 90% • freebase biography 71% • biographies combination 77% 0% 25% 50% 75% 100% Accuracy last.fm 7% Evaluation: freebase 26% ground truth data biography 55% set of 11275 artists combination 59% Govaerts, Sten; Duval, Erik. A Web-based approach to determine the origin of an artist. ISMIR 2009. 0% 15% 30% 45% 60% Coverage step 2 - create. 18Wednesday 12 September 12
  40. 40. Three data sources last.fm 91% Ennio Morricone (born 10 November 1928 in Rome) is an Italian • last.fm freebase composer especially noted for his film scores. He has composed the 90% scores of more than 500 films and TV series. Although only 30 of • freebase biography 71% these are for Western films, it is for this work which he is best known. Morricone’s sparse style of composition for the genre is particularly • biographies combination 77% exemplified by the soundtracks of the classic spaghetti westerns The Good, the Bad and the Ugly (Sergio Leone, 1966) and Once Upon A Time In The West (Sergio Leone, 1968). 0% 25% 50% 75% 100% [...] Accuracy In 2003 he scored the Japanese Taiga drama about Miyamoto Musashi, Japan’s legendary warrior. In 2005 he was on a world tour last.fm 7% with the Roma Sinfonietta Orchestra, touring from cities like New Evaluation:York, Bilbao, Bratislava and Tokyo. In 2006, Morricone collaborated with Morrissey by composing the string part for 26% freebase his song “Dear God ground truth data Please Help Me”, recorded in Rome’s Forum Music Village Studios. In recognition of his vast body of work and dedication, Morricone 55% biography set of 11275 artists received an honorary Academy Award in 2007; Morricone had received nominations for Best Original Score in the past, but had 59% combination failed Erik. A Web-based approach to Govaerts, Sten; Duval, to win. 0% 15% 30% 45% 60% determine the origin of an artist. ISMIR 2009. Coverage step 2 - create. 18Wednesday 12 September 12
  41. 41. Three data sources last.fm 91% • last.fm freebase 90% • freebase biography 71% • biographies combination 77% 0% 25% 50% 75% 100% Accuracy last.fm 7% Evaluation: freebase 26% ground truth data biography 55% set of 11275 artists combination 59% Govaerts, Sten; Duval, Erik. A Web-based approach to determine the origin of an artist. ISMIR 2009. 0% 15% 30% 45% 60% Coverage step 2 - create. 18Wednesday 12 September 12
  42. 42. Three data sources last.fm 91% • last.fm freebase 90% • freebase biography 71% • biographies combination 77% 0% 25% 50% 75% 100% Accuracy last.fm 7% Evaluation: freebase 26% ground truth data biography 55% set of 11275 artists combination 59% Govaerts, Sten; Duval, Erik. A Web-based approach to determine the origin of an artist. ISMIR 2009. 0% 15% 30% 45% 60% Coverage step 2 - create. 18Wednesday 12 September 12
  43. 43. Three data sources last.fm 91% • last.fm freebase 90% • freebase biography 71% • biographies combination 77% 0% 25% 50% 75% 100% Accuracy last.fm 7% Evaluation: freebase 26% ground truth data biography 55% set of 11275 artists combination 59% Govaerts, Sten; Duval, Erik. A Web-based approach to determine the origin of an artist. ISMIR 2009. 0% 15% 30% 45% 60% Coverage step 2 - create. 18Wednesday 12 September 12
  44. 44. Three data sources • last.fm • freebase • biographies last.fm freebase biography combination 71% 77% 91% 90% } 0% 25% 50% 75% 100% Accuracy last.fm 7% Evaluation: freebase 26% ground truth data biography 55% set of 11275 artists combination 59% Govaerts, Sten; Duval, Erik. A Web-based approach to determine the origin of an artist. ISMIR 2009. 0% 15% 30% 45% 60% Coverage step 2 - create. 18Wednesday 12 September 12
  45. 45. Three data sources • last.fm • freebase • biographies last.fm freebase biography combination 71% 77% 91% 90% } 0% 25% 50% 75% 100% Accuracy Evaluation: ground truth data set of 11275 artists last.fm freebase biography combination 7% 26% 55% } 59% Govaerts, Sten; Duval, Erik. A Web-based approach to determine the origin of an artist. ISMIR 2009. 0% 15% 30% 45% 60% Coverage step 2 - create. 18Wednesday 12 September 12
  46. 46. LAST.FM MASHUP http://www.cs.kuleuven.be/~sten/lastonamfm step 2 - create. 19Wednesday 12 September 12
  47. 47. musical genre.http://www.flickr.com/photos/7593077@N03/5076100406/in/photostream/ 20Wednesday 12 September 12
  48. 48. USING SEARCH ENGINES step 2 - create. 21Wednesday 12 September 12
  49. 49. USING SEARCH ENGINES step 2 - create. 21Wednesday 12 September 12
  50. 50. USING SEARCH ENGINES step 2 - create. 21Wednesday 12 September 12
  51. 51. USING SEARCH ENGINES step 2 - create. 21Wednesday 12 September 12
  52. 52. USING SEARCH ENGINES step 2 - create. 21Wednesday 12 September 12
  53. 53. CLASSIFICATION WITH SEARCH ENGINES using co-occurrence M. Schedl, T. Pohle, P. Knees, G. Widmer, “Assigning and Visualizing Music Genres by Web-based Co-occurrence Analysis”, Proceedings of the 7th International Conference on Music Information Retrieval, 2006, pp. 260-265. step 2 - create. 22Wednesday 12 September 12
  54. 54. CLASSIFICATION WITH SEARCH ENGINES using co-occurrence M. Schedl, T. Pohle, P. Knees, G. Widmer, “Assigning and Visualizing Music Genres by Web-based Co-occurrence Analysis”, Proceedings of the 7th International Conference on Music Information Retrieval, 2006, pp. 260-265. step 2 - create. 22Wednesday 12 September 12
  55. 55. step 2 - create. 23Wednesday 12 September 12
  56. 56. Rock: Jazz: Blues: Pop: Country: Metal: step 2 - create. 23Wednesday 12 September 12
  57. 57. Rock: Jazz: 0,013 0,013 Blues: Pop: 0,009 0,015 Country: Metal: 0,009 0,005 step 2 - create. 23Wednesday 12 September 12
  58. 58. RESULTS • 1st results were much worse • what happened? • re-run the original experiment • evaluate on the same data set: 1995 artists and 9 genres. • different search engines: Google,Yahoo! and Microsoft. • over time: 8 times over a period of 36 days. step 2 - create. 24Wednesday 12 September 12
  59. 59. step 2 - create. 25Wednesday 12 September 12
  60. 60. step 2 - create. 25Wednesday 12 September 12
  61. 61. step 2 - create. 26Wednesday 12 September 12
  62. 62. WHAT TO USE? • use Google when it’s stable else rely on Yahoo! • when is it stable? test with a small set • some artists get classified incorrectly on bad days • compare the accuracy achieved with the test set to the average. step 2 - create. 27Wednesday 12 September 12
  63. 63. WHAT TO USE? • use Google when it’s stable else rely on Yahoo! • when is it stable? test with a small set • some artists get classified incorrectly on bad days • compare the accuracy achieved with the test set to the average. Govaerts, Sten; Corthaut, Nik; Duval, Erik. Using search engine for classification: does it still work? adMIRe WS, ISM2009. step 2 - create. 27Wednesday 12 September 12
  64. 64. STEP 4 – UTILISE. 28Wednesday 12 September 12
  65. 65. STEP 4 – UTILISE. 28Wednesday 12 September 12
  66. 66. STEP 4 – UTILISE. Ch 5 & 6 RQ1: How can metadata be leveraged to provide efficient and effective access to content? 28Wednesday 12 September 12
  67. 67. FEDERATED SEARCH. Govaerts, Sten; El Helou, Sandy; Duval, Erik; Gillet, Denis. A federated search and social recommendation widget. SRS WS, CSCW 2011. Govaerts, Sten; Verbert, Katrien; et al. Towards Responsive Open Learning Environments: the ROLE Interoperability framework. EC-TEL 2011. step 4 - utilise. 29Wednesday 12 September 12
  68. 68. FEDERATED SEARCH. Govaerts, Sten; El Helou, Sandy; Duval, Erik; Gillet, Denis. A federated search and social recommendation widget. SRS WS, CSCW 2011. Govaerts, Sten; Verbert, Katrien; et al. Towards Responsive Open Learning Environments: the ROLE Interoperability framework. EC-TEL 2011. step 4 - utilise. 29Wednesday 12 September 12
  69. 69. FEDERATED SEARCH. Govaerts, Sten; El Helou, Sandy; Duval, Erik; Gillet, Denis. A federated search and social recommendation widget. SRS WS, CSCW 2011. Govaerts, Sten; Verbert, Katrien; et al. Towards Responsive Open Learning Environments: the ROLE Interoperability framework. EC-TEL 2011. step 4 - utilise. 29Wednesday 12 September 12
  70. 70. FEDERATED SEARCH. Govaerts, Sten; El Helou, Sandy; Duval, Erik; Gillet, Denis. A federated search and social recommendation widget. SRS WS, CSCW 2011. Govaerts, Sten; Verbert, Katrien; et al. Towards Responsive Open Learning Environments: the ROLE Interoperability framework. EC-TEL 2011. step 4 - utilise. 29Wednesday 12 September 12
  71. 71. FEDERATED SEARCH. Govaerts, Sten; El Helou, Sandy; Duval, Erik; Gillet, Denis. A federated search and social recommendation widget. SRS WS, CSCW 2011. Govaerts, Sten; Verbert, Katrien; et al. Towards Responsive Open Learning Environments: the ROLE Interoperability framework. EC-TEL 2011. step 4 - utilise. 29Wednesday 12 September 12
  72. 72. FEDERATED SEARCH. Govaerts, Sten; El Helou, Sandy; Duval, Erik; Gillet, Denis. A federated search and social recommendation widget. SRS WS, CSCW 2011. Govaerts, Sten; Verbert, Katrien; et al. Towards Responsive Open Learning Environments: the ROLE Interoperability framework. EC-TEL 2011. step 4 - utilise. 29Wednesday 12 September 12
  73. 73. FEDERATED SEARCH. Govaerts, Sten; El Helou, Sandy; Duval, Erik; Gillet, Denis. A federated search and social recommendation widget. SRS WS, CSCW 2011. Govaerts, Sten; Verbert, Katrien; et al. Towards Responsive Open Learning Environments: the ROLE Interoperability framework. EC-TEL 2011. step 4 - utilise. 29Wednesday 12 September 12
  74. 74. CONTEXT • Personal Learning Environment: • customizable • re-use, creation & mashup of tools, resources • enable users to access content • in different contexts step 4 - utilise. 30Wednesday 12 September 12
  75. 75. CONTEXT • Personal Learning Environment: • customizable • re-use, creation & mashup of tools, resources • enable users to access content • in different contexts step 4 - utilise. 30Wednesday 12 September 12
  76. 76. CONTEXT • Personal Learning Environment: • customizable • re-use, creation & mashup of tools, resources • enable users to access content • in different contexts step 4 - utilise. 30Wednesday 12 September 12
  77. 77. WHAT’S A WIDGET ?!? step 4 - utilise. 31Wednesday 12 September 12
  78. 78. WHAT’S A WIDGET ?!? step 4 - utilise. 31Wednesday 12 September 12
  79. 79. STEP 5 & 6 – CAPTURE & PERSONALISE. 32Wednesday 12 September 12
  80. 80. STEP 5 & 6 – CAPTURE & PERSONALISE. Ch 5 & 6 RQ1: How can metadata be leveraged to provide efficient and effective access to content? 32Wednesday 12 September 12
  81. 81. ATTENTION METADATA Attention metadata does not describe the content itself, but what people do with the content. step 5 & 6 - capture/personalise. 33Wednesday 12 September 12
  82. 82. ATTENTION METADATA Attention metadata does not describe the content itself, but what people do with the content. copyright by Warner Bros. Pictures step 5 & 6 - capture/personalise. 33Wednesday 12 September 12
  83. 83. ATTENTION METADATA Attention metadata does not describe the content itself, but what people do with the content. copyright by Warner Bros. Pictures step 5 & 6 - capture/personalise. 33Wednesday 12 September 12
  84. 84. UI. step 4 - utilise. 34Wednesday 12 September 12
  85. 85. UI. step 4 - utilise. 34Wednesday 12 September 12
  86. 86. SOCIAL RECOMMENDATIONS. Govaerts, Sten; El Helou, Sandy; Duval, Erik; Gillet, Denis. A federated search and social recommendation widget. SRS WS, CSCW 2011. Govaerts, Sten; Verbert, Katrien; et al. Towards Responsive Open Learning Environments: the ROLE Interoperability framework. EC-TEL 2011. step 5 & 6 - capture/personalise. 35Wednesday 12 September 12
  87. 87. SOCIAL RECOMMENDATIONS. Govaerts, Sten; El Helou, Sandy; Duval, Erik; Gillet, Denis. A federated search and social recommendation widget. SRS WS, CSCW 2011. Govaerts, Sten; Verbert, Katrien; et al. Towards Responsive Open Learning Environments: the ROLE Interoperability framework. EC-TEL 2011. step 5 & 6 - capture/personalise. 35Wednesday 12 September 12
  88. 88. SOCIAL RECOMMENDATIONS. Govaerts, Sten; El Helou, Sandy; Duval, Erik; Gillet, Denis. A federated search and social recommendation widget. SRS WS, CSCW 2011. Govaerts, Sten; Verbert, Katrien; et al. Towards Responsive Open Learning Environments: the ROLE Interoperability framework. EC-TEL 2011. step 5 & 6 - capture/personalise. 35Wednesday 12 September 12
  89. 89. SOCIAL RECOMMENDATIONS. Govaerts, Sten; El Helou, Sandy; Duval, Erik; Gillet, Denis. A federated search and social recommendation widget. SRS WS, CSCW 2011. Govaerts, Sten; Verbert, Katrien; et al. Towards Responsive Open Learning Environments: the ROLE Interoperability framework. EC-TEL 2011. step 5 & 6 - capture/personalise. 35Wednesday 12 September 12
  90. 90. PAGE RANK HYPERLINK WEB SITE Rank of node i: A node is important if and only if many other important nodes point to it step 5 & 6 - capture/personalise. 36Wednesday 12 September 12
  91. 91. OUR CASE hare d R1 d /s save saved/shared d en tion Erik R2 fr i e c dis n like c on d lik ed R3 d n Sandy ien ctio fr e c on n lik ed R4 Sten R5 step 5 & 6 - capture/personalise. 37Wednesday 12 September 12
  92. 92. THREE CASE STUDIES 1 2 3 step 5 & 6 - capture/personalise. 38Wednesday 12 September 12
  93. 93. KU LEUVEN & EPFL • 15 PhD students. step 5 & 6 - capture/personalise. 39Wednesday 12 September 12
  94. 94. KU LEUVEN & EPFL • 15 PhD students. • usability, satisfaction, usefulness step 5 & 6 - capture/personalise. 39Wednesday 12 September 12
  95. 95. KU LEUVEN & EPFL • 15 PhD students. • usability, satisfaction, usefulness • usefulness of recommendations: 11/14 pro. step 5 & 6 - capture/personalise. 39Wednesday 12 September 12
  96. 96. KU LEUVEN & EPFL • 15 PhD students. • usability, satisfaction, usefulness • usefulness of recommendations: 11/14 pro. • user satisfaction: System Usability Scale (SUS) & MS Desirability Toolkit step 5 & 6 - capture/personalise. 39Wednesday 12 September 12
  97. 97. KU LEUVEN & EPFL • 15 PhD students. • usability, satisfaction, usefulness • usefulness of recommendations: 11/14 pro. • user satisfaction: System Usability Scale (SUS) & MS Desirability Toolkit • SUS score: 66,25% step 5 & 6 - capture/personalise. 39Wednesday 12 September 12
  98. 98. KU LEUVEN & EPFL • 15 PhD students. • usability, satisfaction, usefulness • usefulness of recommendations: 11/14 pro. • user satisfaction: System Usability Scale (SUS) & MS Desirability Toolkit KU Leuven: high (75%) • SUS score: 66,25% EPFL + one KU Leuven: low (50%) step 5 & 6 - capture/personalise. 39Wednesday 12 September 12
  99. 99. WHY THE DIFFERENT SUS? • 1st phase by 2 interviewers step 5 & 6 - capture/personalise. 40Wednesday 12 September 12
  100. 100. WHY THE DIFFERENT SUS? • 1st phase by 2 interviewers step 5 & 6 - capture/personalise. 40Wednesday 12 September 12
  101. 101. WHY THE DIFFERENT SUS? • 1st phase by 2 interviewers step 5 & 6 - capture/personalise. 40Wednesday 12 September 12
  102. 102. WHY THE DIFFERENT SUS? • issues: • distraction from unrelated widgets • layout too dense • height of widgets too small • not evaluating the widget but the whole experience... step 5 & 6 - capture/personalise. 41Wednesday 12 September 12
  103. 103. DESIRABILITY... step 5 & 6 - capture/personalise. 42Wednesday 12 September 12
  104. 104. DESIRABILITY... step 5 & 6 - capture/personalise. 42Wednesday 12 September 12
  105. 105. DESIRABILITY... step 5 & 6 - capture/personalise. 42Wednesday 12 September 12
  106. 106. FESTO • LMS (CLIX) extended with widgets • 25 heterogeneous participants • evaluation setup: 1. video introduction to the new system 2. period of experimentation 3. questionnaire step 5 & 6 - capture/personalise. 43Wednesday 12 September 12
  107. 107. FESTO • LMS (CLIX) extended with widgets • 25 heterogeneous participants • evaluation setup: 1. video introduction to the new system 2. period of experimentation 3. questionnaire step 5 & 6 - capture/personalise. 43Wednesday 12 September 12
  108. 108. FESTO RESULTS step 5 & 6 - capture/personalise. 44Wednesday 12 September 12
  109. 109. FESTO RESULTS 88% would recommend it to their colleagues 81% agreed that it helps them to be more effective step 5 & 6 - capture/personalise. 44Wednesday 12 September 12
  110. 110. JIAO TONG UNIVERSITY • offers higher education for adults with a full-time job • early prototype with • evaluation setup: 1. task list (ETA 15 mins) 2. questionnaire (in Chinese) (partially from ResQUE) step 5 & 6 - capture/personalise. 45Wednesday 12 September 12
  111. 111. JIAO TONG UNIVERSITY • offers higher education for adults with a full-time job • early prototype with • evaluation setup: 1. task list (ETA 15 mins) 2. questionnaire (in Chinese) (partially from ResQUE) step 5 & 6 - capture/personalise. 45Wednesday 12 September 12
  112. 112. RESULTS • ran over 3 weeks, 13 respondents • results for federated search widget: • 69% used the federated search widget • 56% found the recommendations relevant to the course • 38% wanted to continue using the federated search widget • Personal learning environment results • 23% found it useful for the course • 39% found it easy to use, 15% found it navigationally consistent • 46% were satisfied with the personal learning environment step 5 & 6 - capture/personalise. 46Wednesday 12 September 12
  113. 113. WHY? • an established LMS augmented with PLE features brand new system • cultural differences? • slow internet connection (long loading and querying times) • lesson learned: the choice of evaluation environment is very important step 5 & 6 - capture/personalise. 47Wednesday 12 September 12
  114. 114. WHY? • an established LMS augmented with PLE features brand new system • cultural differences? users perceive federated search as useful and • slow internet connection effective IF the content in the repositories relates (long loading and querying times) to their tasks. [Belliston et al., 2007], [Korah et al., 2010], [Gore, 2008] • lesson learned: the choice of evaluation environment is very important step 5 & 6 - capture/personalise. 47Wednesday 12 September 12
  115. 115. WHY? • an established LMS augmented with PLE features brand new system • cultural differences? our evaluation indicates that users benefit from users perceive federated search as useful and • slow internet connection recommendations. the social effective IF the content in the repositories relates recommendations from friends are preferred over (long loading and querying times) to their tasks. system recommendations. [Shinha et al., 2001] [Belliston et al., 2007], [Korah et al., 2010], [Gore, 2008] • lesson learned: the choice of evaluation environment is very important step 5 & 6 - capture/personalise. 47Wednesday 12 September 12
  116. 116. STEP 7 – CREATE AWARENESS. 48Wednesday 12 September 12
  117. 117. STEP 7 – CREATE AWARENESS. Ch 7 48Wednesday 12 September 12
  118. 118. STEP 7 – CREATE AWARENESS. Ch 7 RQ4: How can attention metadata be leveraged to improve awareness and self-reflection? 48Wednesday 12 September 12
  119. 119. PROBLEM http://www.ypfp.org/classroom step 5 & 7 - capture/create awareness. 49Wednesday 12 September 12
  120. 120. PROBLEM http://www.ypfp.org/classroom step 5 & 7 - capture/create awareness. 49Wednesday 12 September 12
  121. 121. STUDENT ACTIVITY METER • visualisations of attention metadata evaluation • self-monitoring for learners design implementation step 5 & 7 - capture/create awareness. 50Wednesday 12 September 12
  122. 122. STUDENT ACTIVITY METER • visualisations of attention metadata evaluation • self-monitoring for learners • awareness for teachers design implementation step 5 & 7 - capture/create awareness. 50Wednesday 12 September 12
  123. 123. STUDENT ACTIVITY METER • visualisations of attention metadata evaluation • self-monitoring for learners • awareness for teachers design • time tracking implementation step 5 & 7 - capture/create awareness. 50Wednesday 12 September 12
  124. 124. STUDENT ACTIVITY METER • visualisations of attention metadata evaluation • self-monitoring for learners • awareness for teachers design • time tracking implementation • learning resource use and recommendations step 5 & 7 - capture/create awareness. 50Wednesday 12 September 12
  125. 125. STUDENT ACTIVITY METER • visualisations of attention metadata evaluation • self-monitoring for learners • awareness for teachers design • time tracking implementation • learning resource use and recommendations • part of Learning Analytics research [ACM LAK conf., Siemens 2011, Duval 2011] step 5 & 7 - capture/create awareness. 50Wednesday 12 September 12
  126. 126. Govaerts, Sten; Verbert, Katrien, Duval, Erik; Pardo, Abelardo. The Student Activity Meter for Awareness and Self-reflection. CHI 2012. demo... 51 http://www.leveraas.com/Wednesday 12 September 12
  127. 127. demogra evaluation design negative positive phics goal changes •↑learnability usability, •↓errors 12 CS satisfaction, small usability I. 1st iteration •good satisfaction students preliminary issues •usefulness usefulness positive step 5 & 7 - capture/create awareness. 52Wednesday 12 September 12
  128. 128. ITERATION TWO & THREE • CGIAR: 19 teachers - LAK: 12 • An online survey about usefulness, teacher issues and how the tool can resolve these. step 5 & 7 - capture/create awareness. 53Wednesday 12 September 12
  129. 129. CGIAR LAK 1 2 3 4 issue for2 add- issue for add- 5 1 3 4 5 teacher ressed teacher ressed Provide feedback to the Provide feedback to the students students ✔ ? ✔ ✔ Being aware of what Being aware of whatdoing students are students are doing ✔ ✔ ✔ ? ✗ ✗ Knowing about collaboration Knowing and communication * and communication about collaboration ✔ ✔ ? Knowing which documents Knowing are used and how much used and how much which documents are ✔ ✔ ✔ ? ✗ ? Knowing how and when Knowing online and whenbeen used have been used how tools have online tools ✔ Finding the students who are not doing well Finding the students who are not doing well ✔ ? ✔ ? Finding the best students Finding the best students Knowing how much time ? ? ? ✗ ? ✔ ? students spent Knowing Knowing if externalstudents how much time learning spent ✔ ? ? ? resources are used Knowing if external learning resources areIssues used ✔Addressed by SAM step 5 & 7 - capture/create awareness. 54Wednesday 12 September 12
  130. 130. CGIAR LAK issue for add- issue for add- teacher ressed teacher ressed Provide feedback to the students ✔ ? ✔ ✔ Being aware of what students are doing ✔ ✔ ✔ ? Knowing about collaboration and communication ✔ ✗ ✔ ✗ Knowing which documents are used and how much ✔ ✔ ✔ ? Knowing how and when online tools have been used ? ✔ ✗ ? Finding the students who are not doing well ✔ ? ✔ ? Finding the best students ? ? ? ✗ Knowing how much time students spent ? ✔ ? ✔ Knowing if external learning resources are used ✔ ? ? ? step 5 & 7 - capture/create awareness. 54Wednesday 12 September 12
  131. 131. CGIAR LAK issue for add- issue for add- teacher ressed teacher ressed Provide feedback to the students ✔ ? ✔ ✔ Being aware of what students are doing ✔ ✔ ✔ ? Knowing about collaboration and communication ✔ ✗ ✔ ✗ Knowing which documents are used and how much ✔ ✔ ✔ ? Knowing how and when online tools have been used ? ✔ ✗ ? Finding the students who are not doing well ✔ ? ✔ ? Finding the best students ? ? ? ✗ Knowing how much time students spent ? ✔ ? ✔ Knowing if external learning resources are used ✔ ? ? ? step 5 & 7 - capture/create awareness. 54Wednesday 12 September 12
  132. 132. CGIAR CASE STUDY step 5 & 7 - capture/create awareness. 55Wednesday 12 September 12
  133. 133. CGIAR CASE STUDY want to search detect outliers for students more details on good indicator for effort student understand the workload progress evolution compare students obtain course overview increase awareness want better 1 to 1 use for course comparison tool design optimization more metrics wanted step 5 & 7 - capture/create awareness. 55Wednesday 12 September 12
  134. 134. CGIAR CASE STUDY want to search detect outliers for students more details on good indicator for effort student understand the workload progress evolution compare students obtain course overview increase awareness want better 1 to 1 use for course comparison tool design optimization more metrics wanted step 5 & 7 - capture/create awareness. 55Wednesday 12 September 12
  135. 135. LAK CASE STUDY step 5 & 7 - capture/create awareness. 56 http://maranash.blogspot.com/2011/01/conference-shout-out.htmlWednesday 12 September 12
  136. 136. LAK CASE STUDY find students experiencing problems and low verify the status of the engagement classroom activity more chronological course dwell time metrics self-reflection to measure progress and increase motivation more data step 5 & 7 - capture/create awareness. 56 http://maranash.blogspot.com/2011/01/conference-shout-out.htmlWednesday 12 September 12
  137. 137. LAK CASE STUDY find students experiencing problems and low verify the status of the engagement classroom activity more chronological course dwell time metrics self-reflection to measure progress and increase motivation more data step 5 & 7 - capture/create awareness. 56 http://maranash.blogspot.com/2011/01/conference-shout-out.htmlWednesday 12 September 12
  138. 138. demogra evaluation design negative positive phics goal changes usability, •↑learnability 12 CS satisfaction, small usability •↓errors I. 1st iteration students preliminary issues •good satisfaction usefulness •usefulness positive assessing •provides awareness 19 resource teacher •all vis. useful II. teachers help function recomm. not needs, use & •many uses & TA’s useful usefulness •90% want it •provides awareness assessing re-orderable most and feedback 12 teacher parallel addressed •many uses III. participan needs, expert coordinates needs are •66% want it ts feedback, use with indecisive •recomm. can be & usefulness histograms useful 57Wednesday 12 September 12
  139. 139. ITERATION FOUR •a CS course on C++ programming • 11 people: 7 teachers, 2 TA’s & 1 course planner • richer data set: tracking from programming environment • qualitative study using a structured face-2-face interview with 25 open questions. step 5 & 7 - capture/create awareness. 58Wednesday 12 September 12
  140. 140. ITERATION FOUR •a CS course on C++ programming • 11 people: 7 teachers, 2 TA’s & 1 course planner • richer data set: tracking from programming environment • qualitative study using a structured face-2-face interview with 25 open questions. step 5 & 7 - capture/create awareness. 58Wednesday 12 September 12
  141. 141. ACHIEVABLE TASKS conflicting visions on get a time identify students doing students doing well summary well and at risk and at risk better course monitor tool and overview resource use understand time spent better detect students course planning for assistance step 5 & 7 - capture/create awareness. http://www.tdpri.com/forum/telecaster-discussion- 59 forum/48431-ic503z-cap-52-ri-blackguard-book.htmlWednesday 12 September 12
  142. 142. ACHIEVABLE TASKS conflicting visions on get a time identify students doing students doing well summary well and at risk and at risk better course monitor tool and overview resource use understand time spent better detect students course planning for assistance step 5 & 7 - capture/create awareness. http://www.tdpri.com/forum/telecaster-discussion- 59 forum/48431-ic503z-cap-52-ri-blackguard-book.htmlWednesday 12 September 12
  143. 143. ACHIEVABLE TASKS conflicting visions on get a time identify students doing students doing well summary well and at risk and at risk all: want tocourse using it better continue monitor tool and 9/11: give it to students overview resource use understand time spent better detect students course planning for assistance step 5 & 7 - capture/create awareness. http://www.tdpri.com/forum/telecaster-discussion- 59 forum/48431-ic503z-cap-52-ri-blackguard-book.htmlWednesday 12 September 12
  144. 144. demo- evaluation design negative positive graphics goal changes usability, •↑learnability 12 CS satisfaction, small usability •↓errors I. 1st iteration students preliminary issues •good satisfaction usefulness •usefulness positive •provides awareness 19 assessing teacher resource •all vis. useful II. teachers needs, use & help function recomm. not •many uses & TA’s usefulness useful •90% want it •provides awareness and assessing teacher most 12 re-orderable feedback needs, expert addressed III. participan PC with •many uses feedback, use & needs are ts histograms •66% want it usefulness indecisive •recomm. can be useful filter & •provides time overview search, conflicting •provides course 11 icons, visions of overview use, usefulness IV. teacher zooming in students •PC assist with detecting & satisfaction s & TA’s line chart, doing well problems editing PC or at risk •many uses & insights axes •100% want it 60Wednesday 12 September 12
  145. 145. CONCLUSION • SAM is perceived useful and contributes to better awareness. step 5 & 7 - capture/create awareness. 61Wednesday 12 September 12
  146. 146. CONCLUSION • SAM is perceived useful and contributes to better awareness. • SAM enables: step 5 & 7 - capture/create awareness. 61Wednesday 12 September 12
  147. 147. CONCLUSION • SAM is perceived useful and contributes to better awareness. • SAM enables: • to detect potential students doing well and at risk step 5 & 7 - capture/create awareness. 61Wednesday 12 September 12
  148. 148. CONCLUSION • SAM is perceived useful and contributes to better awareness. • SAM enables: • to detect potential students doing well and at risk • a better course overview step 5 & 7 - capture/create awareness. 61Wednesday 12 September 12
  149. 149. CONCLUSION • SAM is perceived useful and contributes to better awareness. • SAM enables: • to detect potential students doing well and at risk • a better course overview • understanding student time spending step 5 & 7 - capture/create awareness. 61Wednesday 12 September 12
  150. 150. CONCLUSION • SAM is perceived useful and contributes to better awareness. • SAM enables: • to detect potential students doing well and at risk • a better course overview • understanding student time spending • to find a wide variety of new insights step 5 & 7 - capture/create awareness. 61Wednesday 12 September 12
  151. 151. CONCLUSION • SAM is perceived useful and contributes to better awareness. • SAM enables: • to detect potential students doing well and at risk • a better course overview • understanding student time spending • to find a wide variety of new insights • almost all want to continue using SAM step 5 & 7 - capture/create awareness. 61Wednesday 12 September 12
  152. 152. CONTRIBUTIONS. 1. A structured method for selecting an appropriate metadata schema 2. The design & evaluation of a metadata generator for the origin of an artist 3. The re-evaluation of a general metadata generation approach 4. The design & evaluation of federated search tool with social recommendations 5. The design & evaluation of visualisations of attention metadata to support awareness and self-reflection. 62Wednesday 12 September 12
  153. 153. Ch 8 CONTRIBUTIONS. 1. A structured method for selecting an appropriate metadata schema 2. The design & evaluation of a metadata generator for the origin of an artist 3. The re-evaluation of a general metadata generation approach 4. The design & evaluation of federated search tool with social recommendations 5. The design & evaluation of visualisations of attention metadata to support awareness and self-reflection. 62Wednesday 12 September 12
  154. 154. Ch 8 CONTRIBUTIONS. RQ 3 1. A structured method for selecting an appropriate metadata schema 2. The design & evaluation of a metadata generator for the origin of an artist 3. The re-evaluation of a general metadata generation approach 4. The design & evaluation of federated search tool with social recommendations 5. The design & evaluation of visualisations of attention metadata to support awareness and self-reflection. 62Wednesday 12 September 12
  155. 155. Ch 8 CONTRIBUTIONS. RQ 3 1. A structured method for selecting an appropriate metadata schema 2. The design & evaluation of a metadata generator for the origin of an artist 62Wednesday 12 September 12
  156. 156. Ch 8 CONTRIBUTIONS. RQ 3 1. A structured method for selecting an appropriate metadata schema 2. The design & evaluation of a metadata generator for the origin of an artist 3. The re-evaluation of a general metadata generation approach 62Wednesday 12 September 12
  157. 157. Ch 8 CONTRIBUTIONS. RQ 3 1. A structured method for selecting an appropriate metadata schema 2. The design & evaluation of a metadata generator for the origin of an artist 3. The re-evaluation of a general metadata generation approach RQ 2 62Wednesday 12 September 12
  158. 158. Ch 8 CONTRIBUTIONS. RQ 3 1. A structured method for selecting an appropriate metadata schema 2. The design & evaluation of a metadata generator for the origin of an artist 3. The re-evaluation of a general metadata generation approach RQ 2 4. The design & evaluation of federated search tool with social recommendations 62Wednesday 12 September 12
  159. 159. Ch 8 CONTRIBUTIONS. RQ 3 1. A structured method for selecting an appropriate metadata schema 2. The design & evaluation of a metadata generator for the origin of an artist 3. The re-evaluation of a general metadata generation approach RQ 2 4. The design & evaluation of federated search tool with social recommendations RQ 1 62Wednesday 12 September 12
  160. 160. Ch 8 CONTRIBUTIONS. RQ 3 1. A structured method for selecting an appropriate metadata schema 2. The design & evaluation of a metadata generator for the origin of an artist 3. The re-evaluation of a general metadata generation approach RQ 2 4. The design & evaluation of federated search tool with social recommendations RQ 1 5. The design & evaluation of visualisations of attention metadata to support awareness and self-reflection. 62Wednesday 12 September 12
  161. 161. Ch 8 CONTRIBUTIONS. RQ 3 1. A structured method for selecting an appropriate metadata schema 2. The design & evaluation of a metadata generator for the origin of an artist 3. The re-evaluation of a general metadata generation approach RQ 2 4. The design & evaluation of federated search tool with social recommendations RQ 1 5. The design & evaluation of visualisations of attention metadata to support awareness and self-reflection. RQ 4 62Wednesday 12 September 12
  162. 162. publications. http://dailypracticeblog.com/how-paper-claims-will-be-impacted-by-5010/ 63Wednesday 12 September 12
  163. 163. domain steps RQ conf. publication define, Govaerts, Sten; Corthaut, Nik; Duval, Erik. Moody tunes: the (RQ1, RQ3) ISMIR’06 rockanango project create, utilise create (RQ2) ISMIR’07 Govaerts, Sten; Corthaut, Nik; Duval, Erik. Mood-ex-machina: towards automation of moody tunes Corthaut, Nik; Govaerts, Sten; Verbert, Katrien; Duval, Erik. music RQ3 ISMIR’08 Connecting the dots: music metadata generation, define schemas and applications RQ2 ISMIR’09 Govaerts, Sten; Duval, Erik. A Web-based approach to determine the origin of an artist create ISM’09- Govaerts, Sten; Corthaut, Nik; Duval, Erik. Using search engine RQ2 for classification: does it still work? WS Govaerts, Sten; Verbert, Katrien; Klerkx, Joris; Duval, Erik. RQ4 ICWL’10 Visualizing activities for self-reflection and awareness capture, Evaluating the create RQ4 ICWL’11 Govaerts, Sten; Verbert, Katrien; Duval, Erik.studies student activity meter: two case awareness Govaerts, Sten; Verbert, Katrien, Duval, Erik; Pardo, Abelardo. The TEL RQ4 CHI’12 Student Activity Meter for Awareness and Self- reflection CSCW’11 Govaerts, Sten; El Helou, Sandy; Duval, Erik; Gillet, Denis. A utilise, RQ1 federated search and social recommendation widget -WS personalise, Govaerts, Sten; Verbert, Katrien; et al. Towards Responsive capture RQ1 EC-TEL’11 Open Learning Environments: the ROLE Interoperability framework 64Wednesday 12 September 12
  164. 164. period project domain RQ steps pubs. 2005 - Aristo IWT RQ2, define, create, ISMIR’06, music ISMIR’07, 2007 project 1 RQ3 (utilise) ISMIR’08, ISMIR’09, 2007 - Aristo IWT RQ2, ISM’09- music define, create 2009 project 2 RQ3 WS ICWL’10, utilise, personalise, 2009 - RQ1, ICWL’11, ROLE TEL capture, create ECTEL’11, now RQ4 awareness CHI’12 10/2010- Research RQ1, utilise, personalise, CSCW’11- TEL WS 12/2010 stay at EPFL RQ4 capture 65Wednesday 12 September 12
  165. 165. some metrics... 66Wednesday 12 September 12
  166. 166. CITATIONS PER YEAR. total citations: 72 external citations: 49 30 self citations h-index: 6 external citations 23 number of citations 15 8 0 2007 2008 2009 2010 2011 2012 year 67Wednesday 12 September 12
  167. 167. CITATIONS PER YEAR. total citations: 72 external citations: 49 30 self citations h-index: 6 6 external citations 23 number of citations 15 8 23 8 11 6 2 7 1 2 3 3 0 2007 2008 2009 2010 2011 2012 year 67Wednesday 12 September 12
  168. 168. number of citations 0 4 8 11 15Wednesday 12 September 12 ISMIR 2006 ISMIR 2007 ISMIR 2008 ISMIR 2009 (1) ISMIR 2009 (2) adMIRe 2009 68 IJETEL 2010 papers ICWL 2010 SRS 2011 ECTEL 2011 EFEPLE 2011 CITATIONS PER PAPER. ICWL 2011 self citations CHI 2012 external citations
  169. 169. number of citations 0 4 8 11 15Wednesday 12 September 12 9 5 ISMIR 2006 2 4 ISMIR 2007 9 ISMIR 2008 6 2 ISMIR 2009 (1) 1 ISMIR 2009 (2) 1 adMIRe 2009 68 6 1 IJETEL 2010 papers 7 4 ICWL 2010 3 2 SRS 2011 2 2 ECTEL 2011 2 1 EFEPLE 2011 CITATIONS PER PAPER. 2 ICWL 2011 self citations 1 CHI 2012 external citations
  170. 170. TIME SPENT. 1: schemas 2: origin 2: search 4,5&6: search & recomm. 5&7: awareness time time per RQ 0% 25% 50% 75% 100% RQ1 RQ2 RQ3 RQ4 69Wednesday 12 September 12
  171. 171. TIME SPENT. 1: schemas 2: origin 2: search 4,5&6: search & recomm. 5&7: awareness time 15% 20% 10% 18% 37% time per RQ 18% 30% 15% 37% 0% 25% 50% 75% 100% RQ1 RQ2 RQ3 RQ4 69Wednesday 12 September 12
  172. 172. 70Wednesday 12 September 12
  173. 173. Thank you for your attention! Questions? 70Wednesday 12 September 12
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×