(Live) Annotopia Overview by Paolo Ciccarese (Architect and principal developer)

  • 463 views
Uploaded on

Annotopia is a Universal Annotation Hub that provides you with back-end technology so that you can focus on the user interface and the knowledge creation process. Annotopia talk at 'I Annotate 2014': …

Annotopia is a Universal Annotation Hub that provides you with back-end technology so that you can focus on the user interface and the knowledge creation process. Annotopia talk at 'I Annotate 2014': https://www.youtube.com/watch?v=UGvUbFv0Zl8

  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
No Downloads

Views

Total Views
463
On Slideshare
0
From Embeds
0
Number of Embeds
1

Actions

Shares
Downloads
0
Comments
0
Likes
2

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. Annotopia Architecture by Paolo Ciccarese Paolo Ciccarese, PhD Assistant Professor of Neurology at Harvard Medical School Co-chair of the W3C Open Annotation Community Group Architect of the Annotopia Open Annotation Server ANNOTOPIA UNIVERSAL ANNOTATION HUB Overview (Draft 11 – 2014-11-19) This work is licensed under the Creative Commons Attribution-ShareAlike 4.0 International License. To view a copy of this license, visit http://creativecommons.org/licenses/by-sa/4.0/
  • 2. Annotopia Architecture by Paolo Ciccarese What is it Annotopia? • It is a Universal Annotation Hub • It provides you with back-end technology so that you can focus on the user interface and the knowledge creation process • It consists of a collections of modules that provide useful services for creating, persisting, searching, sharing and enriching annotation • It is Open Annotation* compliant *Open Annotation Data Model (http://www.openannotation.org/spec/core/)
  • 3. Annotopia Architecture by Paolo Ciccarese Imagine annotating… • Documents (HTML and PDF) with multiple tools (Domeo, Utopia, annotator.js) • (Bibliographic) records with bibliographic management platforms • (Biological) Entities in databases and knowledge bases • Hi-res imaging (Brain imaging) and 3D models (proteins, brain anatomy…) • Data in various shapes and domains (genomic data, clinical data…)
  • 4. Annotopia Architecture by Paolo Ciccarese Annotopia: De-Siloing Annotation Custom Annotators Annotopia Instance 1 Applications Text Mining Aggregated querying and browsing Mashups APIs APIs Annotopia Instance 2 JSON-LD JSON-LD
  • 5. Annotopia Architecture by Paolo Ciccarese Current Architecture (see github*) Spring security supporting OpenIDs and OAuth Admin Dashboard Annotopia Storage APIs Data Cruncher Agents/Users/Groups Smart Storage Virtuoso Triple Store MySQL Relational DBMS Connectors Text Mining Terms Search Entity Recognition Vocabulary search Linked Data Mashups Bibliographic Metadata Manager Ext 1 Ext 2 Ext n … https://github.com/Annotopia
  • 6. Annotopia Architecture by Paolo Ciccarese Connectors (Annotation services) • They are extension points, new connectors can be added for integrating new resources • Provide one or more of the following features – Terms search – Text mining – Vocabulary listing • They normalize all the content so that clients get always the same format, no matter what service they asked for
  • 7. Annotopia Architecture by Paolo Ciccarese Connectors (2) • They are dynamically loaded by Annotopia • They can be access to a façade that provides for an API • They can be ran standalone without having to run the full Annotopia infrastructure • They normalize the text mining results to the Open Annotation Model
  • 8. Annotopia Architecture by Paolo Ciccarese Implemented Connectors • BioPortal: allows for term searching, vocabulary listing and text mining through the NCBO Annotator – https://github.com/Annotopia/CnBioPortalConnector • NIF: allows for term searching and text mining through the NIF Annotator – https://github.com/Annotopia/CnNifConnector
  • 9. Annotopia Architecture by Paolo Ciccarese Connectors in progress • EBI (European Bioinformatics Institute) • DBpedia
  • 10. Annotopia Architecture by Paolo Ciccarese Current Architecture (see github*) Annotopia Spring security supporting OpenIDs and OAuth Admin Dashboard Storage APIs Data Cruncher * https://github.com/Annotopia Connectors NCBO Annotator NIF Annotator NCBO Terms Search NIF Resources Search …. Agents/Users/Groups Smart Storage Virtuoso Triple Store MySQL Relational DBMS Bibliographic Metadata Manager Ext 1 Ext 2 Ext n …
  • 11. Annotopia Architecture by Paolo Ciccarese Annotopia: Smart Storage • It is the module in charge of persisting annotation • It uses Virtuoso triple store through a Jena API • Uses JSON-LD as serialization method https://github.com/Annotopia/AtSmartStorage/
  • 12. Annotopia Architecture by Paolo Ciccarese Currently Available APIs • Open Annotation: for storing/retrieving Open Annotation content • Open Annotation Sets: for storing/retrieving Open Annotation content organized in annotation sets • Open Annotation Validation: provides services for validating Open Annotation content • Reporting: provides some basic counters of the storage status
  • 13. Annotopia Architecture by Paolo Ciccarese Annotopia Storage APIs openAnnotationSet openAnnotation openAnnotationSetStorage openAnnotationVirtuoso jenaVirtuosoStore Virtuoso Triple Store openAnnotationStorage jena APIs Services openAnnotationValidation Elastic Search
  • 14. Annotopia Architecture by Paolo Ciccarese Open Annotation API
  • 15. Annotopia Architecture by Paolo Ciccarese Posting Open Annotation Content • The Open Annotation API is a RESTful API that supports GET, POST, PUT and DELETE • Payload is represented in JSON-LD, which through appropriate context/framing can be stored/retrieved from the triple store • POST, PUT and DELETE now support a single item at a time
  • 16. Annotopia Architecture by Paolo Ciccarese Open Annotation API: POST (1) • Creates a new Annotation. Param Description Default apiKey The application assigned apikey. The API calls are available only for registered and authorized applications. POST list of parameters REQUIRED item The Annotation content REQUIRED outCmd It determines the kind of JSON-LD post processing. Possible values are 'none', 'context', 'frame'. In the case of the POST the framing is applied to the response which will send the updated annotation back to the client. none incGph If true, this will include the metadata (provenance and more) of the graph that is wrapping the Annotation false validate If 'ON' the software will go through the appropriate validation algorithm before being stored OFF
  • 17. Annotopia Architecture by Paolo Ciccarese Open Annotation API: POST (2) • The Current JSON-LD specification do not support framing for Named Graphs. Therefore the option ”outCmd":”frame” cannot be used when: – The annotation consists of multiple graphs – The option ”incGph":”true” as the metadata graph will be returned, as a separate Named Graph, together with the annotation. This result in a result of at least two graphs.
  • 18. Annotopia Architecture by Paolo Ciccarese An example of POST (with OAuth) curl -i -X POST http://serverip:port/s/annotation -H "Content-Type: application/json" -H "Authorization: Bearer 521b611a-efc6-4487-bb09-f8c8556b2cb0" -d'{"apiKey":"testkey”, ”outCmd":”frame", "item":{ OA CONTENT/PAYLOAD }} • The content or payload is formatted according to the Open Annotation model (Eds. Sanderson, Ciccarese, Van de Sompel; 03 Feb 2013): http://www.openannotation.org/spec/core/
  • 19. Annotopia Architecture by Paolo Ciccarese Ex of Open Annotation content - POST { "apiKey": "testkey", ”outCmd":”frame”, "item": { "@context": "https://raw2.github.com/Annotopia/AtSmartStorage/master/web-app/data/OAContext.json", "@id": "urn:temp:001", "@type": "http://www.w3.org/ns/oa#Annotation", "motivatedBy": "oa:commenting", "annotatedBy": { "@id": "http://orcid.org/0000-0002-5156-2703", "@type": "foaf:Person", "foaf:name": "Paolo Ciccarese" }, "annotatedAt": "2014-02-17T09:46:11EST", "serializedBy": "urn:application:domeo", "serializedAt": "2014-02-17T09:46:51EST", "hasBody": { "@type": [ "cnt:ContentAsText", "dctypes:Text" ], "cnt:chars": "What does this mean for...", "dc:format": "text/plain" }, "hasTarget": { "@id": "urn:temp:2", "@type": "oa:SpecificResource", "hasSelector": { "@type": "oa:TextQuoteSelector", "exact": "a sequence of progressive...", "prefix": "The data suggest that ", "suffix": "" }, "hasSource": { "@id": "http://www.ncbi.nlm.nih.gov/pmc/articles/PMC3292759/", "@type": "dctypes:Text" } } } } body target
  • 20. Annotopia Architecture by Paolo Ciccarese Smart Storage behavior • Smart storage performs some basic checks on the payload • It mints stable URIs for the components of the payload (Annotation, Body, Target) • It stores each Open Annotation in a Named Graph and it also generates additional metadata that are stored in a separate global Provenance Graph • It returns the payload with the above updates and a short summary wrapper
  • 21. Annotopia Architecture by Paolo Ciccarese Open Annotation POST response (a) {"status":"saved", "result": {"duration": "89ms","graphs":"1","item":[{ "@graph" : [ { "@id" : "http://localhost:8090/s/annotation/C59DBC51-D81E-4B4D-B71B-822BB8FA2C43", "@type" : "oa:Annotation", "http://purl.org/pav/previousVersion" : "urn:temp:001", "annotatedAt" : "2014-02-17T09:46:11EST", "annotatedBy" : { "@id" : "http://orcid.org/0000-0002-5156-2703", "@type" : "foaf:Person", "name" : "Paolo Ciccarese" }, "hasBody" : { "@id" : "http://localhost:8090/s/content/EC3E3D9C-18E9-4820-9A35-5B0E979A8F6E", "@type" : [ "dctypes:Text", "cnt:ContentAsText" ], "format" : "text/plain", "http://purl.org/pav/previousVersion" : "blank", "chars" : "What does this mean for..." }, "hasTarget" : { "@id" : "http://localhost:8090/s/resource/701FDC2F-59AE-4AC1-BF5E-20B57B5146ED", "@type" : "oa:SpecificResource", "http://purl.org/pav/previousVersion" : "urn:temp:8", "hasSelector" : { "@id" : "_:b0", "@type" : "oa:TextQuoteSelector", "exact" : "a sequence of progressive...", "prefix" : "The data suggest that ", "suffix" : "" }, "hasSource" : { "@id" : "http://www.ncbi.nlm.nih.gov/pmc/articles/PMC3292759/", "@type" : "dctypes:Text" } }, "motivatedBy" : "oa:commenting", "serializedAt" : "2014-02-17T09:46:51EST", "serializedBy" : "urn:application:domeo" } ] }]}} Summary body target
  • 22. Annotopia Architecture by Paolo Ciccarese Open Annotation POST response (b) {"status":"saved", "result": {"duration": "89ms","graphs":"1","item":[{ "@graph" : [ { "@id" : "http://localhost:8090/s/annotation/C59DBC51-D81E-4B4D-B71B-822BB8FA2C43", "@type" : "oa:Annotation", "http://purl.org/pav/previousVersion" : "urn:temp:001", "annotatedAt" : "2014-02-17T09:46:11EST", "annotatedBy" : { "@id" : "http://orcid.org/0000-0002-5156-2703", "@type" : "foaf:Person", "name" : "Paolo Ciccarese" }, "hasBody" : { "@id" : "http://localhost:8090/s/content/EC3E3D9C-18E9-4820-9A35-5B0E979A8F6E", "@type" : [ "dctypes:Text", "cnt:ContentAsText" ], "format" : "text/plain", "http://purl.org/pav/previousVersion" : "blank", "chars" : "What does this mean for..." }, "hasTarget" : { "@id" : "http://localhost:8090/s/resource/701FDC2F-59AE-4AC1-BF5E-20B57B5146ED", "@type" : "oa:SpecificResource", "http://purl.org/pav/previousVersion" : "urn:temp:8", "hasSelector" : { "@id" : "_:b0", "@type" : "oa:TextQuoteSelector", "exact" : "a sequence of progressive...", "prefix" : "The data suggest that ", "suffix" : "" }, "hasSource" : { "@id" : "http://www.ncbi.nlm.nih.gov/pmc/articles/PMC3292759/", "@type" : "dctypes:Text" } }, "motivatedBy" : "oa:commenting", "serializedAt" : "2014-02-17T09:46:51EST", "serializedBy" : "urn:application:domeo" } ] }]}} Minted URIs old URI Minted URIs Minted URIs old URI
  • 23. Annotopia Architecture by Paolo Ciccarese Open Annotation API: PUT (1) • Updates an existing Annotation. Param Description Default apiKey The application assigned apikey. The API calls are available only for registered and authorized applications. PUT list of parameters REQUIRED item The Annotation content REQUIRED outCmd It determines the kind of JSON-LD post processing. Possible values are 'none', 'context', 'frame'. In the case of the POST the framing is applied to the response which will send the updated annotation back to the client. none incGph If true, this will include the metadata (provenance and more) of the graph that is wrapping the Annotation false validate If 'ON' the software will go through the appropriate validation algorithm before being stored OFF
  • 24. Annotopia Architecture by Paolo Ciccarese An example of PUT (with OAuth) curl -i -X PUT http://serverip:port/s/annotation -H "Content-Type: application/json" -H "Authorization: Bearer 521b611a-efc6-4487-bb09-f8c8556b2cb0" -d'{"apiKey":"testkey”, ”outCmd":”frame", "item":{ CONTENT/PAYLOAD }} • The content or payload is formatted according to the Open Annotation model (Eds. Sanderson, Ciccarese, Van de Sompel; 03 Feb 2013): http://www.openannotation.org/spec/core/
  • 25. Annotopia Architecture by Paolo Ciccarese Ex of Open Annotation content - PUT { "apiKey": "testkey", ”outCmd":”frame”, "item": { "@context": "https://raw2.github.com/Annotopia/AtSmartStorage/master/web-app/data/OAContext.json", "@id" : "http://localhost:8090/s/annotation/830ED7EE-BF7B-4A18-8AE1-A9AF96AC135B", "@type" : "oa:Annotation", "http://purl.org/pav/previousVersion" : "urn:temp:001", "annotatedAt" : "2014-02-17T09:46:11EST", "annotatedBy" : { "@id" : "http://orcid.org/0000-0002-5156-2703", "@type" : "foaf:Person", "name" : "Paolo Ciccarese" }, "hasBody" : { "@id" : "http://localhost:8090/s/content/79317F9E-57FC-4938-AF52-C66962371451", "@type" : [ "dctypes:Text", "cnt:ContentAsText" ], "format" : "text/plain", "http://purl.org/pav/previousVersion" : "blank", "chars" : ”New updated body" }, "hasTarget" : { "@id" : "http://localhost:8090/s/resource/73AC02CA-B719-478E-B6A3-EE8D18D57ED2", "@type" : "oa:SpecificResource", "http://purl.org/pav/previousVersion" : "urn:temp:2", "hasSelector" : { "@id" : "_:b0", "@type" : "oa:TextQuoteSelector", "exact" : "a sequence of progressive...", "prefix" : "The data suggest that ", "suffix" : "" }, "hasSource" : { "@id" : "http://www.ncbi.nlm.nih.gov/pmc/articles/PMC3292759/", "@type" : "dctypes:Text" } }, "motivatedBy" : "oa:commenting", "serializedAt" : "2014-02-17T09:46:51EST", "serializedBy" : "urn:application:domeo" }} body target
  • 26. Annotopia Architecture by Paolo Ciccarese Open Annotation POST response (a) {"status":"saved", "result": {"duration": "89ms","graphs":"1","item":[{ "@graph" : [ { "@id" : "http://localhost:8090/s/annotation/C59DBC51-D81E-4B4D-B71B-822BB8FA2C43", "@type" : "oa:Annotation", "http://purl.org/pav/previousVersion" : "urn:temp:001", "annotatedAt" : "2014-02-17T09:46:11EST", "annotatedBy" : { "@id" : "http://orcid.org/0000-0002-5156-2703", "@type" : "foaf:Person", "name" : "Paolo Ciccarese" }, "hasBody" : { "@id" : "http://localhost:8090/s/content/EC3E3D9C-18E9-4820-9A35-5B0E979A8F6E", "@type" : [ "dctypes:Text", "cnt:ContentAsText" ], "format" : "text/plain", "http://purl.org/pav/previousVersion" : "blank", "chars" : "What does this mean for..." }, "hasTarget" : { "@id" : "http://localhost:8090/s/resource/701FDC2F-59AE-4AC1-BF5E-20B57B5146ED", "@type" : "oa:SpecificResource", "http://purl.org/pav/previousVersion" : "urn:temp:8", "hasSelector" : { "@id" : "_:b0", "@type" : "oa:TextQuoteSelector", "exact" : "a sequence of progressive...", "prefix" : "The data suggest that ", "suffix" : "" }, "hasSource" : { "@id" : "http://www.ncbi.nlm.nih.gov/pmc/articles/PMC3292759/", "@type" : "dctypes:Text" } }, "motivatedBy" : "oa:commenting", "serializedAt" : "2014-02-17T09:46:51EST", "serializedBy" : "urn:application:domeo" } ] }]}} Summary body target
  • 27. Annotopia Architecture by Paolo Ciccarese Param Description Default apiKey The application assigned apikey. The API calls are available only for registered and authorized applications. REQUIRED max The number of records to display per page 10 offset The page to be returned 0 tgtUrl If specified, this will limit the results to the Annotations that are targeting the specified URL none tgtFgt If true, this will include in the results the Annotations that are targeting a fragment of the specified URL true outCmd It determines the kind of JSON-LD post processing. Possible values are 'none', 'context', 'frame'. none incGph If true, this will include the metadata (provenance and more) of the graph that is wrapping the Annotation false Open Annotation API: GET (1) • Get existing Annotation(s). GET list of parameters
  • 28. Annotopia Architecture by Paolo Ciccarese Some examples of GET (with OAuth) curl -i -X GET http://localhost:8080/s/annotation/ -H "Content-Type: application/json" -H "Authorization: Bearer 521b611a-efc6-4487-bb09-f8c8556b2cb0" --data '{"apiKey":"testkey", “max”:”5”,”offset”:”0”,”outCmd":”frame"}' curl -i -X GET http://localhost:8080/s/annotation/ -H "Content-Type: application/json" -H "Authorization: Bearer 521b611a-efc6-4487-bb09-f8c8556b2cb0" --data '{"apiKey":"testkey", “max”:”5”,”offset”:”0”,”outCmd":”frame”,”tgtUrl”:” http://www.ncbi.nlm.nih.gov/pmc/articles/PMC3292759”}'
  • 29. Annotopia Architecture by Paolo Ciccarese Ex of Open Annotation content - GET { {"status":"results", "result": {"total":”1", "pages":"0", "duration": ”46ms", "offset": "0", "max": ”5", "items":[{ "@graph" : [ { "@id" : "http://localhost:8090/s/annotation/C59DBC51-D81E-4B4D-B71B-822BB8FA2C43", "@type" : "oa:Annotation", "http://purl.org/pav/previousVersion" : "urn:temp:001", "annotatedAt" : "2014-02-17T09:46:11EST", "annotatedBy" : { "@id" : "http://orcid.org/0000-0002-5156-2703", "@type" : "foaf:Person", "name" : "Paolo Ciccarese" }, "hasBody" : { "@id" : "http://localhost:8090/s/content/EC3E3D9C-18E9-4820-9A35-5B0E979A8F6E", "@type" : [ "dctypes:Text", "cnt:ContentAsText" ], "format" : "text/plain", "http://purl.org/pav/previousVersion" : "blank", "chars" : "What does this mean for..." }, "hasTarget" : { "@id" : "http://localhost:8090/s/resource/701FDC2F-59AE-4AC1-BF5E-20B57B5146ED", "@type" : "oa:SpecificResource", "http://purl.org/pav/previousVersion" : "urn:temp:8", "hasSelector" : { "@id" : "_:b0", "@type" : "oa:TextQuoteSelector", "exact" : "a sequence of progressive...", "prefix" : "The data suggest that ", "suffix" : "" }, "hasSource" : { "@id" : "http://www.ncbi.nlm.nih.gov/pmc/articles/PMC3292759/", "@type" : "dctypes:Text" } }, "motivatedBy" : "oa:commenting", "serializedAt" : "2014-02-17T09:46:51EST", "serializedBy" : "urn:application:domeo" } ] }]}} Summary body target
  • 30. Annotopia Architecture by Paolo Ciccarese Annotation Sets API
  • 31. Annotopia Architecture by Paolo Ciccarese Annotation Sets • They are not part of the current specification • They allow to manage one or more Open Annotations as a bundle • Annotation Sets are used for instance by the Domeo Annotation Tool* • The current draft of Open Annotation in EPUB is including the notion of set** *http://annotationframework.org ** http://www.idpf.org/epub/oa/
  • 32. Annotopia Architecture by Paolo Ciccarese Annotation Sets API: POST (1) • Creates a new set of Annotations. Param Description Default apiKey The application assigned apikey. The API calls are available only for registered and authorized applications. POST list of parameters REQUIRED set The set of Annotation items REQUIRED outCmd It determines the kind of JSON-LD post processing. Possible values are 'none', 'context', 'frame'. In the case of the POST the framing is applied to the response which will send the updated annotation back to the client. none incGph If true, this will include the metadata (provenance and more) of the graph that is wrapping the Annotation false validate If 'ON' the software will go through the appropriate validation algorithm before being stored OFF
  • 33. Annotopia Architecture by Paolo Ciccarese In progress…
  • 34. Annotopia Architecture by Paolo Ciccarese Annotation Faceted Search