UMAP – 08 July 2014 University of Leeds
Using DBpedia as a Knowledge Source for Culture-
Related User Modelling Questionna...
UMAP – 08 July 2014 University of Leeds
Outline
• Definitions
• Problem: Modelling of learner's intercultural
awareness
• ...
UMAP – 08 July 2014 University of Leeds
t
Kashima, Y. Conceptions of Culture and Person for Psychology. Journal of Cross-C...
University of LeedsUMAP – 8 July 2014
Problem
• Cultural Awareness –
key 21st Century skills
• User-adaptive
situational s...
University of LeedsUMAP – 8 July 2014
Main Challenge
How to obtain a profile of a user’s knowledge of
cultural aspects for...
UMAP – 8 July 2014 University of Leeds
Our Approach
Perico Dialogue Agent
Knowledge
Probing
Quiz Generator
User Profile
Ge...
UMAP – 8 July 2014 University of Leeds
Extract Knowledge Pool
from DBpedia
Karanasios, S., Thakker, D., Lau, L., Allen, D....
UMAP – 08 July 2014 University of Leeds
Distilling Intercultural Facts
from DBpedia
Identify DBpedia categories
for seed t...
UMAP – 8 July 2014 University of Leeds
Extracted Knowledge Pool
• 40K facts (OWL logical axioms)
• 565 items of clothing
•...
University of LeedsUMAP – 8 July 2014
Probing and Modelling
User’s Knowledge
• Goal: Assess learner's (socio-political and...
University of LeedsUMAP – 8 July 2014
Interaction with Perico
http://imash.leeds.ac.uk:8080/perico/
https://www.youtube.co...
University of LeedsUMAP – 8 July 2014
Evaluation Study
Q1: Usability and interaction
Is Perico usable and intuitive for th...
University of LeedsUMAP – 8 July 2014
Evaluation Studies
Participants
- 22 participants (age
mean=28),
- Using GLOBE:
- Gr...
University of LeedsUMAP – 8 July 2014
Usability
three questions were tailored to Perico’s interaction:
(SUS11) “The questi...
University of LeedsUMAP – 8 July 2014
Qualitative Feedback
Deficiencies of DBpedia Pool
- ‘People in Cyprus use a garment ...
University of LeedsUMAP – 8 July 2014
Interaction Feedback
Limited content:
- Some countries are prominent – e.g. gestures...
University of LeedsUMAP – 8 July 2014
UM Accuracy
None or Low
Exposure
Medium or High
Exposure
Overestimation 17% 3%
Under...
University of LeedsUMAP – 8 July 2014
UM Feedback
Answer indicated in the question
-‘Indian Rupee’, ‘Japanese Yen’, ‘Bulga...
University of LeedsUMAP – 8 July 2014
Lessons Learnt
Using Linked data for
domain specific tasks
Extensibility
Missing or ...
Upcoming SlideShare
Loading in …5
×

UMAP 2014 - Using DBpedia as a Knowledge Source for Culture-related User Modelling Questionnaires

817 views

Published on

In the culture domain, questionnaires are often used to obtain profiles of users for adaptation. Creating questionnaires requires subject matter experts and diverse content, and often does not scale to a variety of cultures and situations. This paper presents a novel approach that is inspired by crowdwisdom and takes advantage of freely available structured linked data. It presents a mechanism for extracting culturally-related facts from DBpedia, utilised as a knowledge source in an interactive user modelling system. A user study, which examines the system usability and the accuracy of the resulting user model, demonstrates the potential of using DBpedia for generating culture-related user modelling questionnaires and points at issues for further investigation.

0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
817
On SlideShare
0
From Embeds
0
Number of Embeds
157
Actions
Shares
0
Downloads
9
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

UMAP 2014 - Using DBpedia as a Knowledge Source for Culture-related User Modelling Questionnaires

  1. 1. UMAP – 08 July 2014 University of Leeds Using DBpedia as a Knowledge Source for Culture- Related User Modelling Questionnaires Dhaval Thakker (Uni. Of Leeds/UoL) Lydia Lau(UoL), Ronald Denaux(iSOCO, Spain), Vania Dimitrova(UoL), Paul Brna(UoL) and Christina Steiner(TU Graz, Austria)
  2. 2. UMAP – 08 July 2014 University of Leeds Outline • Definitions • Problem: Modelling of learner's intercultural awareness • Approach: interactive dialogue exploiting Linked data • Evaluation • Summary
  3. 3. UMAP – 08 July 2014 University of Leeds t Kashima, Y. Conceptions of Culture and Person for Psychology. Journal of Cross-Cultural Psychology. 31, 1 (2000), 14-32. Gupta, V., Hanges, P.J., Dorfman, P. Cultural clusters: methodology and findings. Journal of World Business, 37,2 (2002) 11-15. Culture can be defined as a set of beliefs, values, behaviours and practices that characterise a given group of people [Kashima, 2000] Broad notion of culture Nationality and countries have been used as fairly reliable indicators for tackling cultural diversity [Gupta et al, 2002; Globe clusters; Hofstede dimensions] Narrowed scope: National Culture Definitions
  4. 4. University of LeedsUMAP – 8 July 2014 Problem • Cultural Awareness – key 21st Century skills • User-adaptive situational simulators are effective for training - first encounter • BUT suffer from COLD START Situational simulators - Helps users in developing intercultural competencies - Provide User- Adaptive Learning Experience - Take into account learner’s knowledge of cultures
  5. 5. University of LeedsUMAP – 8 July 2014 Main Challenge How to obtain a profile of a user’s knowledge of cultural aspects for different countries? Questionnaires and situational tests • Do not scale • Are not engaging • Do not handle diversity Use the crowdwisdom from Wikipedia DBPedia – semantic version of Wikipedia • Large, multi-domain knowledge base
  6. 6. UMAP – 8 July 2014 University of Leeds Our Approach Perico Dialogue Agent Knowledge Probing Quiz Generator User Profile GeneratorDialogue Planner User Model • Cultural Exposure Score Knowledge Pool Challenge 2: Extract Knowledge Pool from DBpedia How to generate relevant knowledge pool from huge knowledge such as DBpedia? Challenge 1: Knowledge probing to assess learner's knowledge of a country
  7. 7. UMAP – 8 July 2014 University of Leeds Extract Knowledge Pool from DBpedia Karanasios, S., Thakker, D., Lau, L., Allen, D., Dimitrova, V., & Norman, A. (2013). Making sense of digital traces: An activity theory driven ontological approach. Journal of the American Society for Information Science and Technology,(JASIST) 64(12), 2452-2467.
  8. 8. UMAP – 08 July 2014 University of Leeds Distilling Intercultural Facts from DBpedia Identify DBpedia categories for seed terms (Clothing) Find specific categories and sub-categories (Loden_cape) Find broader categories (German_Culture; Germany) Generate facts “Loden_cape is a Clothing” “Loden_cape occursIn Germany”
  9. 9. UMAP – 8 July 2014 University of Leeds Extracted Knowledge Pool • 40K facts (OWL logical axioms) • 565 items of clothing • >4K items of food • 88 gestures • 159 currencies • 288 languages • 20K annotations (labels and depictions) Available at : http://imash.leeds.ac.uk/ontologies/amon/
  10. 10. University of LeedsUMAP – 8 July 2014 Probing and Modelling User’s Knowledge • Goal: Assess learner's (socio-political and intercultural) knowledge of country • Ask facts (derived) from DBpedia • Ask “trick” questions: close world around country • Mark answers based on expected truth value and add to Country-awareness Profile Ronald Denaux, Vania Dimitrova, Lydia Lau, Paul Brna, Dhavalkumar Thakker, Christina M. Steiner: Employing linked data and dialogue for modelling cultural awareness of a user. IUI 2014: 241-246
  11. 11. University of LeedsUMAP – 8 July 2014 Interaction with Perico http://imash.leeds.ac.uk:8080/perico/ https://www.youtube.com/watch?v=GTUtKSFjdLo
  12. 12. University of LeedsUMAP – 8 July 2014 Evaluation Study Q1: Usability and interaction Is Perico usable and intuitive for the intended users; and what are the possible limitations of the interaction with Perico? Q2: UM accuracy Is the user model produced by Perico accurate against the user’s perception of his/her knowledge in the selected cultural aspects?
  13. 13. University of LeedsUMAP – 8 July 2014 Evaluation Studies Participants - 22 participants (age mean=28), - Using GLOBE: - Group1 – Narrow Cultural Exposure - Group 2 – Broad Cultural Exposure Method - Online - Interaction with Perico – four countries(none, low, med, high) familiarity - Total 92 questions in total User Model - Aggregated User model for each country and each topic - not-good, need- improvement, ok, very good - Participant rating -> accurate, under/over estimated Post-study - SUS usability questionaries' Q1: Usability and interaction Q2: UM accuracy
  14. 14. University of LeedsUMAP – 8 July 2014 Usability three questions were tailored to Perico’s interaction: (SUS11) “The questions asked during the dialogue were easy to understand”; (SUS12) “The instructions provided during the dialogue were clear”; and (SUS13) “The assessment made by the dialogue was correct”. Answering 92 questions in 30-45 minutes, the mean dialogue-score indicates good quality. System was easy to learn and did not require Additional support Integration & Consistency
  15. 15. University of LeedsUMAP – 8 July 2014 Qualitative Feedback Deficiencies of DBpedia Pool - ‘People in Cyprus use a garment called Icknield High School’. - ‘Frank is currency used in Germany Did not take globalisation into account - food, clothing, gestures have become common in countries which they did not originate from. - Inadequate assertions are hard to detect automatically. Possible solution - Allowing users to indicate that something is wrong - filtering or extending of the extracted DBpedia fact pool.
  16. 16. University of LeedsUMAP – 8 July 2014 Interaction Feedback Limited content: - Some countries are prominent – e.g. gestures coming largely from USA; - or that for some countries, e.g. Jordan, the dialogue presented mainly facts related to other countries. Possible solution - Expansion of topics: such as capital, population, climate, religion, festivals, popular sports, points of interest, were suggested. Lacking coherence: - interaction was jumping from question to question and lacked structure - due to the random selection from the pool of possible axioms Possible solution - to follow the GLOBE clusters - probing the knowledge on countries in the same cultural cluster, broadening, i.e. exploring countries from different clusters, - Or comparing, ‘How does Italy’s income inequality compare to the UK – higher/lower?’).
  17. 17. University of LeedsUMAP – 8 July 2014 UM Accuracy None or Low Exposure Medium or High Exposure Overestimation 17% 3% Underestimation 7% 12% Accurate 76% 86%
  18. 18. University of LeedsUMAP – 8 July 2014 UM Feedback Answer indicated in the question -‘Indian Rupee’, ‘Japanese Yen’, ‘Bulgarian Lev’, ‘Polish Zloty’ Possible solution - use rdfs:label (name without the country) - use dbpprop:nickname (e.g. ‘kint’ instead of ‘Bulgarian lev’). Answer given via knowledge elimination Able to answer the questions by knowing the facts about other countries they knew (canada -> china) (both cases) Possible solution (i) Explicitly ask if the user knew or guessed the answer; and (ii) ask for additional justification or explanation. Answer given using a clue in the question
  19. 19. University of LeedsUMAP – 8 July 2014 Lessons Learnt Using Linked data for domain specific tasks Extensibility Missing or Dated Combine: give users opportunity to enter knowledge Gamifying questionnaires Engaging (>30 mis) Variety Examples Media Conversation-like Structure (GLOBE) Thanks! Dr.Dhaval Thakker Research Fellow Artificial Intelligence Group University of Leeds, UK http://dhavalthakker.wordpress.com/ Twitter: @dr_thakker

×