ESWC2011 Summer School: Front-end to the Semantic Web

“interface is the message” on the path to a usable & personal Semantic Web Lora Aroyo VU University Amsterdam @laroyoWednesday, June 1, 2011 1

o utline front-end to semantics: how do we interact with SemWeb Apps? personalization: what do we need to adapt to users? example applications: what good & bad is out there? evaluation: why is continuous evaluation so important?Wednesday, June 1, 2011 2

why interfaces? invisible computers multitude of interaction modes context-sensitive apps networked devices: bridges between virtual & physical worlds GUI become central constantly increasing competitionWednesday, June 1, 2011 3

take ho me message combine content semantics with user context integrate seamlessly physical & web worlds identify relevance to user to rank & select information to present continuous feedback cycle: to and from user you need to deal with GUI on configuration level perform continuous user testing use real world dataWednesday, June 1, 2011 4

“interface is the message” Aaron Koblin: Artfully visualizing our humanity, TED Talk, 2011Wednesday, June 1, 2011 5

FRONT-END TO SEMANTICS how do we interact with the SemWeb Apps?Wednesday, June 1, 2011 6

do SemWeb apps really d iffer?Wednesday, June 1, 2011 7

semantics: what’s special? explicit semantics (often from open sources, e.g. LOD) used for system decisions and results use facetted presentation, searching and browsing of information use typically classifications, typologies or other structures of concepts integrate data from different sources aggregate dataWednesday, June 1, 2011 8

credits: Dan BrickleyWednesday, June 1, 2011 9

RDF dataWednesday, June 1, 2011 10

interaction w ith semanticsWednesday, June 1, 2011 11

http://twitpic.com/il1w/full © BBC MMVIIIWednesday, June 1, 2011 12

http://www.bbc.co.uk/programmes/b00c06n2.rdfWednesday, June 1, 2011 13

converting vocabulariesWednesday, June 1, 2011 14

PERSONALIZATION what do we need to adapt to us?Wednesday, June 1, 2011 15

the user matters when we consider interaction & interfaces, then the user plays a key role for good interface design, a good characterization of the user is needed first, some concept from theory and literatureWednesday, June 1, 2011 16

user proﬁle Definition: A ‘user profile’ is a data structure that represents a characterization of a user (u) at a particular moment of time (t) So, a user profile represents what (from a given (system) perspective) there is to know about a user. The data in a user profile can be explicitly given by the user or have been derived.Wednesday, June 1, 2011 17

user characteris tics Personal data Friend and relations Experience System access Browsing history Knowledge (learning) Device data Location data PreferencesWednesday, June 1, 2011 18

user mo del Definition: The ‘user model’ contains the definitions and rules for the interpretation of observations about the user and about the translation of that interpretation into the characteristics in a user profile. So, a user model is the recipe for obtaining and interpreting user profiles.Wednesday, June 1, 2011 19

user mo deling Definition: ‘user modeling’ is the process of creating user profiles following the definitions and rules of the user model. This includes the derivation of new user profile characteristics from observations about the user and the old user profile based on the user model. So, user modeling is the process of representing the user.Wednesday, June 1, 2011 20

stereotyping Stereotyping is one example of user modeling. A user is considered to be part of a group of similar people, the stereotype. Question: What could be stereotypes for conference participants (when we design the conference website)?Wednesday, June 1, 2011 21

user-adaptive system Definition: A ‘user-adaptive system’ is a system that adapts itself to a specific user. Often, a user-adaptive system (or adaptive system, in short) uses user profiles to base its adaptation on. So, designing an adaptive system implies designing the user modeling.Wednesday, June 1, 2011 22

user adaptation User-adaptation is often used for personalization, i.e. making a system appear to function in a personalized way. Question: What user profile characteristics would be useful in personalizing the conference’s registration site? Question: How would you obtain those characteristics?Wednesday, June 1, 2011 23

examples: user adaptation Device-dependence Accessibility (disabilities) Location-dependence Adaptive workflow Question: Can you give concrete examples for interface adaptation, both the adaptation effect as the prior user modeling necessary?Wednesday, June 1, 2011 24

adaptive hyperme d ia Well-studied example of adaptation is ‘adaptive hypermedia’: a hypertext’s content and navigation are then adapted to the user’s browsing of the hypertext.Wednesday, June 1, 2011 25

DESIGNING INTERFACESWednesday, June 1, 2011 26

d ialog principles [Grice] Be cooperative Be informative Be truthful Be relevant Be perspicuous (be clear)Wednesday, June 1, 2011 27

UI principles [Shnei der mann] Strive for consistency Enable frequent users to use shortcuts Offer informative feedback Design dialog to yield closure Offer simple error handling Permit easy reversal of actions Support internal locus of control Reduce short-term memory loadWednesday, June 1, 2011 28

usability heuristics [Nielsen] Visibility of system status Match between system and real world User control and freedom Consistency and standards Error prevention Recognition rather than recall Flexibility and efficiency of use Aesthetic and minimalist design Help users recognize, diagnose and recover from errors Help and documentationWednesday, June 1, 2011 29

all abo ut the user’s perspective modeling the user: what are user’s preferences, interests, history, activities, etc. modeling the user’s context: e.g. location, time, device which of all the data available is relevant for this user in this context also called context-awareWednesday, June 1, 2011 30

user’s context d is tribute d switching between one context and another doing things not only for him/herself, e.g. buying present for a girlfriendWednesday, June 1, 2011 31

PERSONALIZED INTERACTION sWednesday, June 1, 2011 32

interaction mo des search, e.g. keyword, faceted browse, story lines, narratives through collections annotations of multimedia, e.g. (collaborative) tagging, professional annotation of text, images and video, tagging games explanations, hints, user feedback, e.g. explanation of recommendation results, explanation of autocompletion suggestionsWednesday, June 1, 2011 33

typical examples recommendation systems, e.g. movies, music, art user statistics and analysis, e.g. user usage data, profile, group profiles, etc. social networkingWednesday, June 1, 2011 34

reco m mender systems Definition: A ‘recommender system’ is a system that recommends to a user, based on her individual interests, items that the user could find interesting. Examples: music, movies, people, restaurants Types: collaborative (reason about similar users), content-based (reason about similar items) Problems: new users, new items, sparsity, gray sheepWednesday, June 1, 2011 35

reco m mender systems movies & TV programs, e.g. Netflix, MovieLens, TiVo, personalized TV guides music, e.g. LastFM, Pandora, iTunes Genius food & tourism, e.g. guides adapted to location, current time, preferences news, e.g. Google reader, news filters e-shopping, e.g. Amazon’s recommendations advertisement, e.g. Facebook personalized ads art, museums, e.g. personalized search, personalized museum guidesWednesday, June 1, 2011 36

consi derations Collection of activities/context/attention data Derive interests from this data Recommender-specific problems, e.g. cold start, over-specialization Surface items of interest in the ‘long tail’ Cross-domain recommendations Multi-person recommending Granular control for usersWednesday, June 1, 2011 37

user proﬁles & stats overview of user preferences, e.g. settings, privacy overview of user interests, e.g. ranking of interests, links to content overview of user/group activities, e.g. per topics, per activity, per date, over a period, overall comparative views between users, e.g. LastFM, livingSocial movies user similarity, Twitter similar users to you different views/visualization over the same set of user dataWednesday, June 1, 2011 38

Wednesday, June 1, 2011 39

social networking professional networks & events, e.g. LinkedIn, Mendeley people, organizations, e.g. Facebook, MySpace Twitter social bookmarking, e.g. Delicious, StumbleUpon, Diggit GetGlue Books, e.g. LibabryThingWednesday, June 1, 2011 41

EXAMPLE APPLICATIONS Interfaces & Personalization on SemWebWednesday, June 1, 2011 42

the big guysWednesday, June 1, 2011 43

The Recommendation and Like plugins let users share any content they like back to their profile.Wednesday, June 1, 2011 48

The Activity Feed plugin shows users what their friends are doing on your site through likes and comments.Wednesday, June 1, 2011 49

activity streams http://xmlns.notu.be/aair/Wednesday, June 1, 2011 51

weig hte d interest http://xmlns.notu.be/wiWednesday, June 1, 2011 52

EXAMPLE 1 what do Gerrit Dou and Rembrandt have in common? http://www.chip-project.orgWednesday, June 1, 2011 54

enriched Rijksmuseum collectionWednesday, June 1, 2011 55

mili<a teacher of: Ferdinand Bol teacher of: Nicolaes Maes self-‐portrait teacher of: Gerrit Dou style: Baroque place: Amsterdam, 1625 to 1650Wednesday, June 1, 2011 56

goal & central role of UMWednesday, June 1, 2011 57

personalized experience Personalized Web Access Online Tour Wizard Personalized Mobile Tour Interactive tours Semantic Search Interactive user modeling On-the-ﬂy adaptation Museum tour maps Recommendations of artworks & art topics Synchronized user Historic timeline proﬁleWednesday, June 1, 2011 58

semantic recommendationsWednesday, June 1, 2011 59

personalized toursWednesday, June 1, 2011 62

Interactive Museum Guide h"p://chip-‐project.org Wednesday, June 1, 2011 63

Interactive Museum GuideWednesday, June 1, 2011 64

event-based browsingWednesday, June 1, 2011 65

dynamic adaptation For each artwork in the museum: Related works Include in the tour ( & recalculate the map/tour) Indicate relevance in terms of e.g. personal interest, position, recommended by friends, by Rijks, on view Rate to indicate interest At any point of the tour: Include/exclude artworks Adjust tour length Change navigation in and outside of the tour Save for other toursWednesday, June 1, 2011 66

EXAMPLE 2 professionals vs. lay users on Web 2.0 semantic annotation of Rijksmuseum prints http://e-culture.multimedian.nl/pk/annotate? semantic tagging: http://waisda.nlWednesday, June 1, 2011 67

Autocompletion with multiple vocabularies http://slashfacet.semanticweb.org/wordnet/search http://slashfacet.semanticweb.org/autocomplete/demos/Wednesday, June 1, 2011 68

EXAMPLE 3 semantic television http://notube.tvWednesday, June 1, 2011 73

watching TV in a group for more details check out our blog at http://notube.tvWednesday, June 1, 2011 78

watching TV in a group for more details check out our blog at http://notube.tvWednesday, June 1, 2011 79

watching TV in a groupWednesday, June 1, 2011 80

watching TV in a group Environment Age Interact with the second 15 - 35 years old screen as a group Friend interaction at home Type of Activities Watching as a group quiz and betting games change camera view Synchronization information regarding the TV & Second Screen content of the program between second screens textual captions between second screens & TV show content provider Type of Program SportsWednesday, June 1, 2011 81

observations for more details check out our blog at http://notube.tvWednesday, June 1, 2011 82

observations for more details check out our blog at http://notube.tvWednesday, June 1, 2011 83

second screen & TV functionalities shared virtual space synchronization with second voice dubbing screen subtitles “overlay” on top of the main related information TV-picture quizzes censoring voting & betting different camera views scene-grab & share group alerts social interaction live-chat parental advisory uncensored version different camera viewsWednesday, June 1, 2011 84

CONTINUOUS EVALUATIONWednesday, June 1, 2011 85

CHIP users Target users’ characteristics small groups with 2-4 persons and a male taking the leading role (67%) middle-aged people in 30-60 years old (75%) higher-educated (62%) no prior knowledge about the Rijksmuseum collection (62%) visit the museum for education (98%)Wednesday, June 1, 2011 86

contextual analysis Context ual obse rvations Define familiarity with the domain s Define familiarity with iew collections/vocabularies ter v r in Use Va Identify use cases lid ate Identify navigation patterns sks Model user’s ta Identify requirements for user groupsWednesday, June 1, 2011 88

do main explorationWednesday, June 1, 2011 89

usability testingWednesday, June 1, 2011 90

resultsWednesday, June 1, 2011 94

http://www.cs.vu.nl/intertain/Wednesday, June 1, 2011 96

take ho me message combine content semantics with user context integrate seamlessly physical & web worlds identify relevance to user to rank & select information to present continuous feedback cycle: to and from user you need to deal with GUI on configuration level perform continuous user testing use real world dataWednesday, June 1, 2011 97

ESWC2011 Summer School: Front-end to the Semantic Web

by Lora Aroyo

Accessibility

Categories

Upload Details

Usage Rights

Statistics