This document discusses interaction patterns for semantic content management systems. It defines interaction patterns as describing recurring user actions when interacting with a computer to achieve tasks. An interaction pattern consists of the problem, the pattern/solution, use cases, and how the pattern applies. The document provides examples of interaction patterns and discusses standards for semantic annotation, presentation, and interaction to enable semantic interaction.

1. Knowledge
Interaction
and
Semantic CMS Community
Presentation
Lecturer
Organization
Date of presentation
Co-funded by the
1 Copyright IKS Consortium
European Union

2. Page:
Part I: Foundations
(1) Introduction of Content Foundations of Semantic
(2)
Management Web Technologies
Part II: Semantic Content Part III: Methodologies
Management
Knowledge Interaction Requirements Engineering
(3) (7)
and Presentation for Semantic CMS
(4) Knowledge Representation
and Reasoning
(8)
Designing
Semantic CMS
Semantifying
(5) Semantic Lifting (9) your CMS
Storing and Accessing Designing Interactive
(6) Semantic Data
(10) Ubiquitous IS
www.iks-project.eu Copyright IKS Consortium

3. Page: 3
What is this Lecture about?
 How can semantic web Part II: Semantic Content
technologies by used to improve Management
the state-of-play in content Knowledge Interaction
(3)
management? and Presentation
 We aim... (4) Knowledge Representation
and Reasoning
 ... for an appropriate presentation
of knowledge. (5) Semantic Lifting
 ... to provide the user the ability to
interact with the knowledge. (6)
Storing and Accessing
Semantic Data
www.iks-project.eu Copyright IKS Consortium

4. Page: 4
Overview
 Introduction
 Semantic Web: Interaction with Content
 Standards (RDF, RDFa, JSON-LD, …)
 Semantic Interaction Patterns
 What are patterns?
 List of patterns
 IKS Interaction with Knowledge
 The Stack: Interaction & Presentation
 VIE
 Interaction Patterns & Widgets
www.iks-project.eu Copyright IKS Consortium

5. Page: 5
Overview
 Introduction
 Semantic Web: Interaction with Content
 Standards (RDF, RDFa, JSON-LD, …)
 Semantic Interaction Patterns
 What are patterns?
 List of patterns
 IKS Interaction with Knowledge
 The Stack: Interaction & Presentation
 VIE
 Interaction Patterns & Widgets
www.iks-project.eu Copyright IKS Consortium

6. Page: 6
Semantic Interaction: What is it?
 Imagine you want want to plan a vacation in NY
www.iks-project.eu Copyright IKS Consortium

7. Page: 7
Semantic Interaction: What is it?
 Imagine you want to plan a vacation in NY
 You found the right package on a touristic portal
www.iks-project.eu Copyright IKS Consortium

8. Page: 8
Semantic Interaction: What is it?
 Imagine you want to plan a vacation in NY
 You found the right package on a touristic portal…
 …and want to access additional information
www.iks-project.eu Copyright IKS Consortium

9. Page: 9
Semantic Interaction: What is it?
 Imagine you want to plan a vacation in NY
 You found the right package on a touristic portal…
 …and want to access additional information
 Usually you have to grab the information from many
sources manually (i.e. looking in many web pages)
www.iks-project.eu Copyright IKS Consortium

10. Page: 10
Semantic Interaction: What is it?
 Imagine you want to plan a vacation in NY
 You found the right package on a touristic portal…
 …and want to access additional information
 Usually you have to grab the information manually (i.e. looking in many web pages)
 With semantics you can access this information
automatically
(e.g., Terkait)
https://github.com/neogermi/terkait
www.iks-project.eu Copyright IKS Consortium

11. Page: 11
Semantic Web: In general
 Motivation
 Definition
 Web of Data vs WWW
www.iks-project.eu Copyright IKS Consortium

12. Page: 12
Semantic Web: the potential.
 Web of Data as a CMS  Web Communities as CMS
Editors
www.iks-project.eu Copyright IKS Consortium

13. Page: 13
Integrate Data with the Web
 WWW
www.iks-project.eu Copyright IKS Consortium

14. Page: 14
Integrate Data with the Web
 WWW  DATA Silos
www.iks-project.eu Copyright IKS Consortium

15. Page: 15
Integrate Data with the Web
 WWW  DATA Silos
Web of Data
www.iks-project.eu Copyright IKS Consortium

16. Page: 16
The Entire Process
 Exporting data (more datasets)
 Grab information from your content (i.e.,
recognize the „entities“)
 Merging your data
 Merge it from different data
 Conbine with different datasets/content
 Use data to interact with (e.g., configure)
web services
 Publishing Semantics/Content/interaction
 Enrich your content with dinamically
generated, interactive information
www.iks-project.eu Copyright IKS Consortium

17. Page: 17
The Entire Process
 Exporting data (more datasets)
 Grab information from your content (i.e.,
recognize the „entities“)
 Merging your data
 Merge it from different data
 Conbine with different datasets/content
 Use data to interact with (e.g., configure)
web services
 Publishing Semantics/Content/interaction
 Enrich your content with dinamically
generated, interactive information
www.iks-project.eu Copyright IKS Consortium

18. Page: 18
The Entire Process
 Exporting data (more datasets)
 Grab information from your content (i.e.,
recognize the „entities“)
 Merging your data
 Merge it from different data
 Conbine with different datasets/content
 Use data to interact with (e.g., configure)
web services
 Publishing Semantics/Content/interaction
 Enrich your content with dinamically
generated, interactive information
www.iks-project.eu Copyright IKS Consortium

19. Page: 19
The Entire Process
 Exporting data (more datasets)
 Grab information from your content (i.e.,
recognize the „entities“)
 Merging your data
 Merge it from different data
 Conbine with different datasets/content
 Use data to interact with (e.g., configure)
web services
 Publishing Semantics/Content/interaction
 Enrich your content with dinamically
generated, interactive information
www.iks-project.eu Copyright IKS Consortium

20. Page: 20
Standard Resources
 Web standards and accessibility resources
 World Wide Web Consortium (W3C)
 Web Standards Project
 Web Accessibility in Mind
 Tutorials, references, statistics, and forums
 W3Schools
 JavaScript Kit
 The jQuery Project
 HTML5 Demos
 HTML5 Reset
 Web Safe Font Tester
 Web Style Guide
 960 Grid System
 HTML/XHTML code validators
 W3C Markup Validator
 W3C Link Checker
 W3C Log Validator
 WDG HTML Validator
 CSS code validators
 W3C CSS Validator
 Browser compatibility verification
 Browershots
 Browser Cam
www.iks-project.eu Copyright IKS Consortium

21. Page: 21
Standard Resources: Presentation
 HTML5
 CSS3
 XAML
 SMIL
 XUL
www.iks-project.eu Copyright IKS Consortium

22. Page: 22
Standard Resources: Interaction
 Java
 Jscript libraries
 Jquery
 Backbone js
 Popcorn js…
 Php/perl/
www.iks-project.eu Copyright IKS Consortium

23. Page: 23
Standard Resources: Annotation
 RDF/OWL
 RDFa
 Microdata
 Microformats
 XML
 JSON
www.iks-project.eu Copyright IKS Consortium

24. Page: 24
Technologies Perspective
XAML
OWL2 SMIL
Microdata
Microformats XUL
CSS3
RDFa
RDFa API HTML5
APML
JavaScript
SKOS
MPEG7 FOAF JQuery
iCal AJAX
RESTful SCXML
VoiceXML
WSDL
EMMA
SOAP SSML
CCXML
www.iks-project.eu Copyright IKS Consortium

25. Page: 25
Publish RDF: RDF + HTML
 Plugins for
Subject Pred Object
 HTML + RDFa
 HTML + Microformats
 HTML + Microdata
 …
<http://dbpedia.org/resource/Steve_Jobs>
<http://www.w3.org/1999/02/22-rdf-syntax-ns#type>
<http://dbpedia.org/ontology/Person>
www.iks-project.eu Copyright IKS Consortium

26. Page: 26
Overview
 Introduction
 Semantic Web: Interaction with Content
 Standards (RDF, RDFa, JSON-LD, …)
 Semantic Interaction Patterns
 What are patterns?
 List of patterns
 IKS Interaction with Knowledge
 The Stack: Interaction & Presentation
 VIE
 Interaction Patterns & Widgets
www.iks-project.eu Copyright IKS Consortium

27. Page: 27
AQCI Lifecycle
www.iks-project.eu Copyright IKS Consortium

28. Page: 28
AQCI Lifecycle
 Author: “A user of the system can author (create) semantically
enhanced content.”
 Query: “A user can query for semantically enhanced content.”
 Consume: “A user can consume semantically enhanced content.”
 Interact: “A user can interact with the content at the level of his
domain knowledge.”
www.iks-project.eu Copyright IKS Consortium

29. Page: 29
Interaction Patterns: Definition
 Design pattern provide proven solutions to frequent design problems in a generative
and human-readable format.
 Interaction patterns (special case of design patterns) describes
 recurring actions
 a user performs
 when interacting with a computer
 to achieve a certain goal of a task.
 These actions are
 implicit if they arise from the discourse context - the previous actions of the user -
and are called
 explicit if they are directly triggered by the user (e.g., pushing a button). In the
context of IKS, we focus the interaction patterns on interaction with content and
moreover the interaction with the (ontological) knowledge that corresponds to this
content element.
 An interaction pattern consists of four parts that describe:
 the problem
 the pattern (i.e., the solution of the problem)
 use cases for the pattern
 how the pattern applies for the use cases
www.iks-project.eu Copyright IKS Consortium

30. Page: 30
Interaction Patterns
www.iks-project.eu Copyright IKS Consortium

31. Page: 31
Interaction Patterns
 Interaction Pattern:
 describes recurring actions
 a user performs when interacting with a
computer
 to achieve a certain goal of a task.
 Actions are
 Implicit: arise from the discourse context (e.g.,
the previous actions of the user)
 Explicit: triggered by the user (e.g., pushing a
button).
 An interaction pattern consists of four parts:
 the problem
 the pattern (i.e., the solution of the problem)
 use cases for the pattern
 how the pattern applies for the use cases
www.iks-project.eu Copyright IKS Consortium

32. Page: 32
Interaction Patterns
 Interaction Pattern:
 describes recurring actions
 a user performs when interacting with a
computer
 to achieve a certain goal of a task.
 Actions are
 Implicit: arise from the discourse context (e.g.,
the previous actions of the user)
 Explicit: triggered by the user (e.g., pushing a
button).
 An interaction pattern consists of four parts:
 the problem
 the pattern (i.e., the solution of the problem)
 use cases for the pattern
 how the pattern applies for the use cases
www.iks-project.eu Copyright IKS Consortium

33. Page: 33
Interaction Patterns
 Interaction Pattern:
 describes recurring actions
 a user performs when interacting with a
computer
 to achieve a certain goal of a task.
 Actions are
 Implicit: arise from the discourse context (e.g.,
the previous actions of the user)
 Explicit: triggered by the user (e.g., pushing a
button).
 An interaction pattern consists of four parts:
 the problem
 the pattern (i.e., the solution of the problem)
 use cases for the pattern
 how the pattern applies for the use cases
www.iks-project.eu Copyright IKS Consortium

34. Page: 34
Interaction Patterns
 Interaction Pattern:
 describes recurring actions
 a user performs when interacting with a
computer
 to achieve a certain goal of a task.
 Actions are
 Implicit: arise from the discourse context (e.g.,
the previous actions of the user)
 Explicit: triggered by the user (e.g., pushing a
button).
 An interaction pattern consists of four parts:
 the problem
 the pattern (i.e., the solution of the problem)
 use cases for the pattern
 how the pattern applies for the use cases
www.iks-project.eu Copyright IKS Consortium

35. Page: 35
Interaction Patterns
 Interaction Pattern:
 describes recurring actions
 a user performs when interacting with a
computer
 to achieve a certain goal of a task.
 Actions are
 Implicit: arise from the discourse context (e.g.,
the previous actions of the user)
 Explicit: triggered by the user (e.g., pushing a
button).
 An interaction pattern consists of four parts:
 the problem
 the pattern (i.e., the solution of the problem)
 use cases for the pattern
 how the pattern applies for the use cases
www.iks-project.eu Copyright IKS Consortium

36. Page: 36
Interaction Patterns
 Interaction Pattern:
 describes recurring actions
 a user performs when interacting with a
computer
 to achieve a certain goal of a task.
 Actions are
 Implicit: arise from the discourse context (e.g.,
the previous actions of the user)
 Explicit: triggered by the user (e.g., pushing a
button).
 An interaction pattern consists of four parts:
 the problem
 the pattern (i.e., the solution of the problem)
 use cases for the pattern
 how the pattern applies for the use cases
www.iks-project.eu Copyright IKS Consortium

37. Page: 37
Interaction Patterns: Basics
Following the AQCI Lifecycle:
DIMENTIONS
 “Create”: An author/user of the
system can create semantically Knowledge
enhanced content.
 “Query”: An author/user can
query for semantically enhanced
content.
 “Consume”: An author/user can
consume semantically enhanced
content.
 “Interact”: An author/user can
interact with the content at the Content
level of his/her domain
knowledge.
www.iks-project.eu Copyright IKS Consortium

38. Page: 38
Interaction Patterns: IKS List
Knowledge Content
 Manual annotation of content with knowledge  Automatic generation of templates
 Semi-Automatic annotation of content with  Autocompletion support for content
knowledge
Create  Fully-Automatic annotation of content with  Recommendation of content links (Duplicate: Query)
knowledge
 Rectifying knowledge annotation
 Linking of entities
 Semi-automatic Extraction of Knowledge from
Content repository
 Automatic Extraction of Knowledge from
Content repository
 Search and Suggest Service: Annotations  Faceted search
for textual content  Disambiguation of query
 Using annotated content items for searching
Query  Providing personalized/contextual search result
filters
 Recommendation of content links (Duplicate:
Create)
www.iks-project.eu Copyright IKS Consortium

39. Page: 39
Interaction Patterns: IKS List
 Contextual rendering of content
 Highlighting of content of interest
 Receive a stream of information/content of interest
Consume  Browsing through entity hierarchy
 Zoom in/out different data hierarchy levels along a tree
 Visalizing content dependencies
 Comparing content, based on the knowledge
 Recommendation of content links (Duplicate: Query)
 Contextual activity pane
 Contextual export interaction
 Semantic Drag&Drop interaction
 Registering and receiving updates of content changes
Query  Managing audits according to business rules
 Assure a level of consistency of content according to business rules
 Filtering content based on knowledge rules
 Automatic clustering of content/knowledge
 Automatic summarization of content/knowledge
 Exporting knowledge
www.iks-project.eu Copyright IKS Consortium

40. Page: 40
Overview
 Introduction
 Semantic Web: Interaction with Content
 Standards (RDF, RDFa, JSON-LD, …)
 Semantic Interaction Patterns
 What are patterns?
 List of patterns
 IKS Interaction with Knowledge
 The Stack: Interaction & Presentation
 VIE
 Interaction Patterns & Widgets
www.iks-project.eu Copyright IKS Consortium

41. Page: 41
The Stack: Interaction & Presentation
Functionality categories of the IKS stack’s
layer of interaction and presentation:
 Author: “A user of the system can
author (create) semantically
enhanced content.”
 Query: “A user can query for
semantically enhanced content.”
 Consume: “A user can consume
semantically enhanced content.”
 Interact: “A user can interact with
the content at the level of his
domain knowledge.”
www.iks-project.eu Copyright IKS Consortium

42. Page: 42
Interaction with Content in IKS
Common representation of content on JavaScript level
 If Content expressed with RDFa the content model can be easily
extracted into JavaScript. Using Backbone.js:
 supplies structure to JavaScript-heavy applications by
 providing models with key-value binding and custom events,
 collections with a rich API of enumerable functions,
 views with declarative event handling,
 connects it all to existing applications over a RESTful JSON
interface.
 With Backbone, the content extracted from the RDFa-annotated
HTML page is easily manageable via JavaScript.
www.iks-project.eu Copyright IKS Consortium

43. Page: 43
VIE
 Pedigree:
 Name: Vienna IKS Editable
 Functionality: makes the content of web pages editable through
annotations. Supports semantic-web developers in
 Retrieval of semantic data
 Storing semantic data
 Accessing semantic web services (e.g., Apache Stanbol
Enhancer)
 Semantic markup (e.g., RDFa or Microdata)
 Coordinates:
 Basic concepts: http://wiki.iks-project.eu/index.php/VIE
 Development: http://github.com/IKS/VIE
www.iks-project.eu Copyright IKS Consortium

44. Page: 44
It‘s about abstraction
VIE - UI Widgets
„VIE-W“
VIE
VIE-2
„Edit your
„Edit your
content w.
Semantics“
Semantics“
(Semantic) Services
(e.g., Stanbol Enhancer, -
EntityHub, Zemanta, ...)
(Semantic) Databases
(e.g., DBPedia, Geonames, ...)
www.iks-project.eu Copyright IKS Consortium

45. Page: 45
VIE: UI Widgets
On top of VIE we gathered a bunch
of UI widgets in a library that help to
simplifying embedding VIEs power
into a webpage more directly.
UI Widgets
www.iks-project.eu Copyright IKS Consortium

46. Page: 46
VIE Widgets
Widgets Widgets
 VIE-Widgets are a sort of jQuery UI Widgets
in order to:
 achive maximum portability
 accelerating lerning curve
www.iks-project.eu Copyright IKS Consortium

47. Page: 47
UI Widget Example
Description:
The VIE Image Search widget search
for images using semantic annotated
content as parameter for the search.
Once included in an HTML page, the
developer can easily query and retrieve
images from the photo service Flickr,
based on the type-specific properties
of the current entity in focus, e.g., if the
current entity is a city it makes sense to
start a geographic query, whereas, for
persons, it would make sense to query
for the name of that person.
(http://neogermi.github.com/VIEwidgets/widgets/image_search/index.html)
www.iks-project.eu Copyright IKS Consortium

48. Page: 48 July 5, 2011
W3C: Interaction and Semantics
 Interaction  Semantic Web
 HTML5  OWL
 JavaScript  RDF
 JQuery  RDFa
 JSON
www.iks-project.eu Copyright IKSIKS Consortium
Copyright Consortium

49. Page: 49 July 5, 2011
W3C: Interaction and Semantics
 Interaction  Semantic Web
 HTML5  OWL
 JavaScript  RDF
 JQuery  RDFa
 JSON
VIE
www.iks-project.eu Copyright IKSIKS Consortium
Copyright Consortium

50. Page: 50 July 5, 2011
IKS: Coordinates
 Stanbol  Interaction Patterns
 VIE  WIDGETS
 VIE2
W3C
|
Technology and Society Domain
|
Semantic Web Activity
|
RDF Web Applications Working Group (RDFWA)
www.iks-project.eu Copyright IKS Consortium

51. Page: 51
References
 http://linkeddata-specs.info/
 http://www.w3.org/wiki/SweoIG/TaskForces/CommunityProjects/LinkingOpenData
 http://www.bioontology.org/wiki/images/6/6a/Triple_Stores.pdf
 https://github.com/IKS
 https://github.com/IKS/VIE
 https://github.com/neogermi/VIEwidgets
 http://schema.org
 http://wiki.iks-project.eu/index.php/Semantic_Editor
 http://www.w3.org/TR/xhtml-rdfa-primer/
 http://www.w3.org/2006/07/SWD/
 http://www.w3.org/TR/rdfa-syntax/
 http://www.w3.org/TR/xhtml-rdfa-primer/
 http://www.w3.org/2010/02/rdfa/
 http://www.w3.org/TR/2011/WD-rdfa-api-20110419/
 http://documentcloud.github.com/backbone/
 http://www.w3.org/2001/11/13-RDF-Query-Rules/
 http://json-ld.org
 http://www.interactive-knowledge.org/sites/www.interactive-
www.iks-project.eu Copyright IKS Consortium
knowledge.org/files/Tutorial_Knowledge_Interaction_Presentation.pptx