1.
OpenAnnotationandDomainOntologies–PaoloCiccarese
Integrating OPEN ANNOTATION
with any DOMAIN ONTOLOGY
Paolo Ciccarese, PhD
Assistant Professor of Neurology at Harvard Medical School
Knowledge Architect of the MGH Biomedical Informatics Core
Co-chair of the W3C Open Annotation Community Group
Ontology and Imaging Informatics, Amherst, NY, June 24-25 2014
Thanks to Tim Clark, MIND Informatics Director

2.
OpenAnnotationandDomainOntologies–PaoloCiccarese
My background
• Evidence-based decision support (integration
of guidelines based decision support with
patient records and clinical workflows)
• Structured Electronic Patient Records (using
ontologies and archetypes)
• Representation of scientific discourse (claims,
hypothesis, evidence)
• Annotation of digital content
• I build and deliver systems

3.
OpenAnnotationandDomainOntologies–PaoloCiccarese
Annotation is emerging as…
• A uniform, pervasive method for describing…
bookmarking, integrating, connecting, sharing
… a variety of (digital) resources
• A starting point and not only an end point
• Valuable content.. sometimes even more
valuable than the annotated content
• Important search/discovery mechanism

4.
OpenAnnotationandDomainOntologies–PaoloCiccarese
… we commonly use annotation
• We annotate prints,
HTML and PDFs
• We bookmark/tag web
pages…
• … and publications
(citations/references)
• We comment on web
pages, blogs, forums and
emails
• youtube, vimeo, flickr
slideshare, twitter…
http://www.plosone.org/article/info%3Adoi%2F10.1371%2Fjournal.pone.0059717

5.
OpenAnnotationandDomainOntologies–PaoloCiccarese
Problem is…
• Most of applications make very hard to export
and aggregate our own annotation
• Applications have a life span and they can
disappear along with your annotation
• No application will cover all the annotation
related use cases
• Domains are really challenging to regulate
• Need for enabling applications developers and
researchers to experiment

6.
OpenAnnotationandDomainOntologies–PaoloCiccarese
Goal: annotation interoperability
… and integration with ontologies
(semantic annotation)

7.
OpenAnnotationandDomainOntologies–PaoloCiccarese
Annotation Ontology (AO) - 2009
• OWL vocabulary for representing and sharing
annotation of digital resources (text, images,
audio, video, …) and their fragments in RDF
format
• It is orthogonal to the domain of interest
• A little under-defined and with extension
points
– pros: easy adoption, extensible, customizable
– cons: not always applied correctly, users not
always sharing the custom solutions

8.
OpenAnnotationandDomainOntologies–PaoloCiccarese
Annotation Ontology adopters
The Living Document
Project
Biotea

9.
OpenAnnotationandDomainOntologies–PaoloCiccarese
W3C Open Annotation Community Group
• Started with the reconciliation of Annotation Ontology
and Open Annotation Collaboration (interoperability
starts from people)
• 127 participants from around the world: 4th of 181
groups
• Adopted by a draft specification for EPUB*
• On track to become a W3C Working Group (impacting
browsers?)
http://www.w3.org/community/openannotation/
*EPUB® is a registered trademark of the International Digital Publishing Forum.

10.
OpenAnnotationandDomainOntologies–PaoloCiccarese
W3C Open Annotation Community Group
http://www.w3.org/community/openannotation/

11.
OpenAnnotationandDomainOntologies–PaoloCiccarese
Mailing List

12.
OpenAnnotationandDomainOntologies–PaoloCiccarese
Open Annotation Data Model (Feb 2013)
http://www.openannotation.org/spec/core/

13.
OpenAnnotationandDomainOntologies–PaoloCiccarese
Open Annotation
Not only for annotating the web!

14.
OpenAnnotationandDomainOntologies–PaoloCiccarese
Open Annotation Model
• An annotation is considered to be a set of connected
resources, it includes body(ies) and target(s), and
conveys that the body is related to the target.
• Ideally all the resources are identified by URIs

15.
OpenAnnotationandDomainOntologies–PaoloCiccarese
Bodies and Targets
• The Body and Target may be of any media
type, and contain any type of content.
– Annotate an Image with a video and a comment
– Annotate multiple images with a webpage
– Annotate an Image and an article with a comment
– Annotate an image and a database record with a
comment
– Annotate annotation(s)

16.
OpenAnnotationandDomainOntologies–PaoloCiccarese
Annotation
Body Target
http://coastalnhneurosurgeons.com/http://annonc.oxfordjournals.org/content/21/suppl_5/v190.full
MRI scan of a glioblastoma multiforme annotated with
ESMO Clinical Practice Guidelines

17.
OpenAnnotationandDomainOntologies–PaoloCiccarese
Annotation
Body
https://www.youtube.com/watch?v=s5RM81_6vC8
http://coastalnhneurosurgeons.com/
Target
MRI scan of a glioblastoma multiforme annotated with
surgical footage

18.
OpenAnnotationandDomainOntologies–PaoloCiccarese
Body
Annotation
Body
Target
MRI scan of a glioblastoma multiforme annotated with
surgical footage

19.
OpenAnnotationandDomainOntologies–PaoloCiccarese
Motivations
• Reasons why the Annotation was created
(describing, tagging, commenting, identifying…)

20.
OpenAnnotationandDomainOntologies–PaoloCiccarese
Motivation modeled as skos:Concepts
You can create a SKOS Schema for your own community

21.
OpenAnnotationandDomainOntologies–PaoloCiccarese
anno1
body1 target1
http://coastalnhneurosurgeons.com/http://en.wikipedia.org/wiki/Glioblastoma_multiforme
oa:Annotation
rdf:type
oa:hasTargetoa:hasBody
oa:describing
oa:motivatedBy
dctypes:Image
dctypes:Text

22.
OpenAnnotationandDomainOntologies–PaoloCiccarese
http://annonc.oxfordjournals.org/content/21/suppl_5/v190.full
anno1
body1 target1
http://coastalnhneurosurgeons.com/
oa:Annotation
rdf:type
oa:hasTargetoa:hasBody
oa:describing
oa:motivatedBy
dctypes:Image
dctypes:Text
?
?

23.
OpenAnnotationandDomainOntologies–PaoloCiccarese
How to identify
fragments?

24.
OpenAnnotationandDomainOntologies–PaoloCiccarese
Specific Resources
• Identify and describe resource fragments (for
both bodies and targets)

25.
OpenAnnotationandDomainOntologies–PaoloCiccarese
anno1
body1 sptarget1
http://coastalnhneurosurgeons.com/…jpghttp://annonc.oxfordjournals.org/content/21/suppl_5/v190.full
oa:Annotation
rdf:type
oa:hasTargetoa:hasBody
oa:describing
oa:motivatedBy
SELECTOR
oa:hasSource

26.
OpenAnnotationandDomainOntologies–PaoloCiccarese
Example of Selectors for images
Source: http://maphub.github.io/api/

27.
OpenAnnotationandDomainOntologies–PaoloCiccarese
You can create more selectors!
Please contribute them back!

28.
OpenAnnotationandDomainOntologies–PaoloCiccarese
Tagging

29.
OpenAnnotationandDomainOntologies–PaoloCiccarese
anno1
body1
oa:Annotation
rdf:type
oa:hasTarget
oa:hasBody
oa:tagging
oa:motivatedBy
oa:Tag
cnt:ContentAsText
http://coastalnhneurosurgeons.com/…jpg
SELECTOR
sptarget1
“glioblastoma
multiforme”
cnt:chars

30.
OpenAnnotationandDomainOntologies–PaoloCiccarese
Adding semantics…

31.
OpenAnnotationandDomainOntologies–PaoloCiccarese
Simplest example: Semantic Tagging

32.
OpenAnnotationandDomainOntologies–PaoloCiccarese
anno1
http://purl.obolibrary.org/obo/DOID_3068 sptarget1
http://coastalnhneurosurgeons.com/
oa:Annotation
rdf:type
oa:hasTargetoa:hasBody
oa:describing
oa:motivatedBy
oa:SemanticTag
SELECTOR
Semantic Tagging 1
rdf:type

33.
OpenAnnotationandDomainOntologies–PaoloCiccarese
anno1
http://purl.obolibrary.org/obo/GO_0005634
sptarget1
http://nanobiotechnews.com/
oa:Annotation
rdf:type
oa:hasTarget
oa:hasBody
oa:describing
oa:motivatedBy
oa:SemanticTag
SELECTOR
Semantic Tagging 2
rdf:type
sptarget2
oa:hasTarget

34.
OpenAnnotationandDomainOntologies–PaoloCiccarese
Web Annotation Tool
• Domeo is a web application for producing and
sharing stand-off annotation
• Science and semantics linked in a few clicks
• Domeo is open source and designed as an
open system…
– http://annotationframework.org
– https://twitter.com/DomeoTool

35.
OpenAnnotationandDomainOntologies–PaoloCiccarese
Manual and automatic annotation
URLIamannotating
Manualannotationtools
Automaticannotationtools
Exploration panels

36.
OpenAnnotationandDomainOntologies–PaoloCiccarese
Semantic tagging
NCBO BioPortal
NIF Registry
Domeo can query external services and use as qualifiers anything that
has a unique identifier.

37.
OpenAnnotationandDomainOntologies–PaoloCiccarese
Semantic tagging
We could refer to diseases, historic figures, galaxies, places, events…
APPintheProteinOntology

38.
OpenAnnotationandDomainOntologies–PaoloCiccarese
Text Mining
Service based:
NCBO Annotator
NIF Annotator
UIMA
GATE

39.
OpenAnnotationandDomainOntologies–PaoloCiccarese
Text Mining Results

40.
OpenAnnotationandDomainOntologies–PaoloCiccarese
Crowd sourcing results curation (v.1)
Customizable
SKOS-like… mappings

41.
OpenAnnotationandDomainOntologies–PaoloCiccarese
Images in Domeo

42.
OpenAnnotationandDomainOntologies–PaoloCiccarese
Image annotation
By semantically tagging figures in a paper, I make them discoverable…
And we can integrate inference capabilities
We have a plugin that integrates to the Yale Image Finder

43.
OpenAnnotationandDomainOntologies–PaoloCiccarese

44.
OpenAnnotationandDomainOntologies–PaoloCiccarese
Integration with Drupal7 (Semtech 2012)

45.
OpenAnnotationandDomainOntologies–PaoloCiccarese
And more semantics…

46.
OpenAnnotationandDomainOntologies–PaoloCiccarese
Annotating with Named Graphs

47.
OpenAnnotationandDomainOntologies–PaoloCiccarese
Intracellular Processing of APP Generates a
Variety of Cleavage Products
SpecificTarget1
SpecificTarget2
alpha-sAPP
is derived from
APP

48.
OpenAnnotationandDomainOntologies–PaoloCiccarese
anno1
sptarget1
oa:Annotation
rdf:type
oa:describing
oa:motivatedBy
sptarget2
oa:hasTarget
oa:hasTarget
Graph1
oa:hasBody
alpha-sAPP
(depicted bysptarget2)
is derived from
APP
(depicted by sptarget1)

49.
OpenAnnotationandDomainOntologies–PaoloCiccarese
Annotation body: Graph1
<http://purl.bioontology.org/ontology/MSH/C096899> depiction <sptarget2>
<http://purl.bioontology.org/ontology/MSH/C096899> preferredName “alpha-sAPP
protein, human”
<http://purl.obolibrary.org/obo/HGNC_620> depiction <sptarget1>
<http://purl.obolibrary.org/obo/HGNC_620> preferredName “APP (human)”
<http://purl.bioontology.org/ontology/MSH/C096899> derivedFrom
<http://purl.obolibrary.org/obo/HGNC_620>

50.
OpenAnnotationandDomainOntologies–PaoloCiccarese
Some examples using graphs
• Antibodies annotation (NIF Antibody registry)
• SWAN Ontology for scientific discourse and
Micro-publications
• Nano-publications

51.
OpenAnnotationandDomainOntologies–PaoloCiccarese
NIF Antibody Registry
http://antibodyregistry.org/

52.
OpenAnnotationandDomainOntologies–PaoloCiccarese
Antibody Registry Integration
• The NIF (https://www.neuinfo.org/) team
made available a web service for searching
against the registry
• A plug-in for DOMEO has been developed to
create antibodies-related annotation:
– search the registry and display of the results
– selection of the results
– specification of the protocol and the subject of the
study

53.
OpenAnnotationandDomainOntologies–PaoloCiccarese
Domeo: antibodies
Contributed to PubMed LinkOut through NIF (http://neuinfo.org)
Translates into a formal OWL/RDF representation
Antibodyregistry.org
Paolo Ciccarese, PhD DILS 2013

54.
OpenAnnotationandDomainOntologies–PaoloCiccarese
Export of the annotation in RDF
• DOMEO can serialize the annotation in several
serialization formats.

55.
OpenAnnotationandDomainOntologies–PaoloCiccarese
Integration with NIF

56.
OpenAnnotationandDomainOntologies–PaoloCiccarese
Domeo: Hypotheses management (v1)
Translates into a formal OWL/RDF representation (SWAN Ontology)
Possibility for integrating
Nanopublications and BEL
Data as evidence
Paolo Ciccarese, PhD DILS 2013

57.
OpenAnnotationandDomainOntologies–PaoloCiccarese

58.
OpenAnnotationandDomainOntologies–PaoloCiccarese
ANNOTOPIA
UNIVERSAL ANNOTATION HUB
Full ‘live’ version of the slides on:
http://www.slideshare.net/paolociccarese/
This work is licensed under the Creative Commons Attribution-ShareAlike 4.0 International License. To
view a copy of this license, visit http://creativecommons.org/licenses/by-sa/4.0/

59.
OpenAnnotationandDomainOntologies–PaoloCiccarese
What is it Annotopia?
• It is a Universal Annotation Hub
• It provides you with back-end technology so
that you can focus on the user interface
(client) and the knowledge creation process
• It consists of a collections of modules that
provide useful services for creating, persisting,
searching, sharing and enriching annotation
• It is Open Annotation* compliant
*Open Annotation Data Model (http://www.openannotation.org/spec/core/)
https://github.com/Annotopia

60.
OpenAnnotationandDomainOntologies–PaoloCiccarese
Annotopia
Instance 1
Annotopia: De-Siloing Annotation
Aggregated
querying and
browsing
Applications
Text Mining
Mashups
APIs
Annotopia
Instance 2
APIs
JSON-LD
JSON-LD

61.
OpenAnnotationandDomainOntologies–PaoloCiccarese
Thank you!
paolo.ciccarese@gmail.com