Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

User Satisfaction of a Hybrid Ontology-Engineering Tool

314 views

Published on

User Satisfaction of a Hybrid Ontology-Engineering Tool
C. Debruyne and I. Ciuciu
LNCS 8186, p. 414 ff.

Published in: Technology, Education
  • Be the first to comment

  • Be the first to like this

User Satisfaction of a Hybrid Ontology-Engineering Tool

  1. 1. User Satisfaction of a Hybrid Ontology-engineering Tool Christophe Debruyne and Ioana Ciuciu 13-09-2013 @ META4eS, OTM 2013
  2. 2. Overview ‣ Motivation ‣ Background ‣ Usability Test Design ‣ User Satisfaction ‣ Results and Recommendations ‣ Conclusions and Future Work 2
  3. 3. Motivation ‣ Semantic interoperability and vocabularies for Linked Data initiatives ‣ Ontology engineering is far from trivial ‣ Requires appropriate methods and tools ‣ Hybrid Ontology Engineering platform in the context of the Open Semantic Cloud for Brussels (OSCB) project ‣ Evaluate usability and user satisfaction of the hybrid ontology platform 3
  4. 4. Background ‣ IBM Post-study System Usability Questionnaire (PSSUQ) 4 Information Quality System Use Interface Quality Overall
  5. 5. Background ‣ Hybrid Ontology Engineering ‣ Community promoted to first-class citizen ‣ Social interactions are ‣ formalized ‣ lead to ontology evolution ‣ supported by a glossary ‣ Concepts are described ‣ formally --> with binary fact-types called lexons e.g., <Research Domain, Call, with, of, Deadline> ‣ informally --> with glosses 5
  6. 6. GOSPL Method and Tool 6
  7. 7. GOSPL Method and Tool 7
  8. 8. GOSPL Method and Tool ‣ With respect to the previous study ‣ System usefulness scored best ‣ Information quality needed most improvement ‣ Changes made to the system to tackle problems ‣ More (online) documentation ‣ Changes to stylesheets ‣ RSS to keep track of discussions ‣ Additional functionalities ‣ Tool support for managing annotations ‣ A reputation framework 8
  9. 9. Usability Test Design ‣ 23 MSc student volunteers part of an ontology engineering course ‣ Objective: ‣ prototype ontology related to research domain ‣ use ontology to annotate ‣ “reverse engineered” existing application ‣ existing research information system ‣ Evaluate formative and summative user satisfaction of the ontology engineering platform ‣ Compare with results from a previous study with similar experiment 9
  10. 10. Usability Test Design ‣ Tasks on the platform ‣ Propose changes to the hybrid ontology ‣ Discuss and vote ‣ Conclude discussions --> ontology evolution ‣ Create and manage communities ‣ Annotate information systems ‣ Participants ‣ 36 students, 9 groups for the ontology engineering experiment ‣ Background in computer science or similar ‣ 23 students participated in the survey (of which 4 complete groups) 10
  11. 11. Summative User Satisfaction ‣ Summative evaluation ‣ System usefulness and interface quality remained the same ‣ Information quality and overall satisfaction improved ‣ Formative evaluation ‣ Tracking “hottest” discussions ‣ Correcting mistakes ‣ Voting mechanism (!) and verbalization of constraints (!) ‣ Positive outcome given the problems encountered with application management and the reputation framework 11 [2] Ioana Ciuciu, Christophe Debruyne:Assessing the User Satisfaction with an Ontology Engineering Tool Based on Social Processes. OTM Workshops 2012: 242-251
  12. 12. Recommendations ‣ Overview of discussions need improvement ‣ (Error-) management of application commitments ‣ Verbalization ‣ Removing voting mechanism 12
  13. 13. Conclusions and Future Work ‣ Conclusions ‣ System usefulness performs best ‣ Information quality improved a lot ‣ Overall satisfaction remained stable ‣ Given the fact that new functionalities posed problems, this can be interpreted as a considerable improvement of the tool ‣ Future work ‣ Third iteration planned in May 2014 ‣ Discussion overview + lightweight user participation 13
  14. 14. Questions?

×