User Satisfaction of a Hybrid Ontology-Engineering Tool

287 views

Published on

User Satisfaction of a Hybrid Ontology-Engineering Tool
C. Debruyne and I. Ciuciu
LNCS 8186, p. 414 ff.

Published in: Technology, Education
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
287
On SlideShare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
6
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

User Satisfaction of a Hybrid Ontology-Engineering Tool

  1. 1. User Satisfaction of a Hybrid Ontology-engineering Tool Christophe Debruyne and Ioana Ciuciu 13-09-2013 @ META4eS, OTM 2013
  2. 2. Overview ‣ Motivation ‣ Background ‣ Usability Test Design ‣ User Satisfaction ‣ Results and Recommendations ‣ Conclusions and Future Work 2
  3. 3. Motivation ‣ Semantic interoperability and vocabularies for Linked Data initiatives ‣ Ontology engineering is far from trivial ‣ Requires appropriate methods and tools ‣ Hybrid Ontology Engineering platform in the context of the Open Semantic Cloud for Brussels (OSCB) project ‣ Evaluate usability and user satisfaction of the hybrid ontology platform 3
  4. 4. Background ‣ IBM Post-study System Usability Questionnaire (PSSUQ) 4 Information Quality System Use Interface Quality Overall
  5. 5. Background ‣ Hybrid Ontology Engineering ‣ Community promoted to first-class citizen ‣ Social interactions are ‣ formalized ‣ lead to ontology evolution ‣ supported by a glossary ‣ Concepts are described ‣ formally --> with binary fact-types called lexons e.g., <Research Domain, Call, with, of, Deadline> ‣ informally --> with glosses 5
  6. 6. GOSPL Method and Tool 6
  7. 7. GOSPL Method and Tool 7
  8. 8. GOSPL Method and Tool ‣ With respect to the previous study ‣ System usefulness scored best ‣ Information quality needed most improvement ‣ Changes made to the system to tackle problems ‣ More (online) documentation ‣ Changes to stylesheets ‣ RSS to keep track of discussions ‣ Additional functionalities ‣ Tool support for managing annotations ‣ A reputation framework 8
  9. 9. Usability Test Design ‣ 23 MSc student volunteers part of an ontology engineering course ‣ Objective: ‣ prototype ontology related to research domain ‣ use ontology to annotate ‣ “reverse engineered” existing application ‣ existing research information system ‣ Evaluate formative and summative user satisfaction of the ontology engineering platform ‣ Compare with results from a previous study with similar experiment 9
  10. 10. Usability Test Design ‣ Tasks on the platform ‣ Propose changes to the hybrid ontology ‣ Discuss and vote ‣ Conclude discussions --> ontology evolution ‣ Create and manage communities ‣ Annotate information systems ‣ Participants ‣ 36 students, 9 groups for the ontology engineering experiment ‣ Background in computer science or similar ‣ 23 students participated in the survey (of which 4 complete groups) 10
  11. 11. Summative User Satisfaction ‣ Summative evaluation ‣ System usefulness and interface quality remained the same ‣ Information quality and overall satisfaction improved ‣ Formative evaluation ‣ Tracking “hottest” discussions ‣ Correcting mistakes ‣ Voting mechanism (!) and verbalization of constraints (!) ‣ Positive outcome given the problems encountered with application management and the reputation framework 11 [2] Ioana Ciuciu, Christophe Debruyne:Assessing the User Satisfaction with an Ontology Engineering Tool Based on Social Processes. OTM Workshops 2012: 242-251
  12. 12. Recommendations ‣ Overview of discussions need improvement ‣ (Error-) management of application commitments ‣ Verbalization ‣ Removing voting mechanism 12
  13. 13. Conclusions and Future Work ‣ Conclusions ‣ System usefulness performs best ‣ Information quality improved a lot ‣ Overall satisfaction remained stable ‣ Given the fact that new functionalities posed problems, this can be interpreted as a considerable improvement of the tool ‣ Future work ‣ Third iteration planned in May 2014 ‣ Discussion overview + lightweight user participation 13
  14. 14. Questions?

×