Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
A PRACTICAL INTRODUCTION TO
SADI SEMANTIC WEB SERVICES
AND HYDRA QUERY TOOL
Alexandre Riazanov, CTO
IPSNP Computing Inc
Os...
PLAN OF THE TALK
• A brief reminder of the previous episode: data
federation with SADI and HYDRA.
• RDF and OWL as syntact...
DATA FEDERATION: QUERYING MULTIPLE
HETEROGENEOUS SOURCES AS A SINGLE DB
QUERY EXAMPLES
• Find the names of drugs that contain chemical category Y as
active ingredients.
• Find documents mentioni...
HOW WE DO IT WITH HYDRA AND SADI
SEMANTIC WEB SERVICES
A HIGH LEVEL VIEW OF THE HYDRA APPROACH
● Given a SPARQL query, HYDRA analyses it
by using an intelligent logic-based algo...
HOW IS THIS ALL POSSIBLE?
• Key ingredient: the SADI framework for
Semantic Web services (Semantic Automated
Discovery and...
DIGRESSION: RDF
• W3C RDF = Resource Description
Framework
• Standartised graph-based data model and a
few standard render...
RDF EXAMPLES
@prefix mt: <http://localhost:8080/medical_terminology.owl#> .
<http://example.com/patient#1234> rdf:type mt:...
DIGRESSION: OWL
• W3C OWL = Web Ontology Language
• Essentially, extends RDF with definitions and other axioms
for classes...
SADI SERVICE I/O
• Input: RDF description of an input object.
• Output: another RDF graph providing more
(computed or retr...
COMPLETE SEMANTIC DESCRIPTIONS
OF SERVICE FUNCTIONALITY
SADI services publish semantic descriptions of their
I/O that comp...
Example: computeBMI service I/O
SEMANTIC FUNCTIONALITY DESCRIPTION
• OWL syntax is repurposed to define what RDF graphs are
acceptable as input, and what ...
SERVICE INPUT CLASS
• Specifies what kind of objects (RDF descriptions) the service
expects in the input. OWL syntax is co...
SERVICE OUTPUT CLASS
• A SADI service advertises itself by publishing its
output class specifying what the service promise...
DIGRESSION: SPARQL
• W3C SPARQL - standard query language for
the RDF data model.
• SPARQL clients are programs that execu...
AUTOMATIC SERVICE DISCOVERY
• With the I/O descriptions, a sufficiently intelligent
client can figure out that it can call...
QUERY, EXECUTION, ANSWERS
Query:
FROM <.......rdf> # seed data
SELECT ?bmi_value
{
patient1234 a Person;
has_BMI ?bmi_valu...
MULTIPLE SERVICES
• Suppose, we don’t know patient’s height/mass, but can
retrieve them from a DB by patient’s medical rec...
AUTOMATIC SERVICE COMPOSITION
• HYDRA can figure out automatically that the output of
patientInfo can be submitted to comp...
INTELLIGENT (REASONING-ENABLED) QUERY
EXECUTION
● Some queries are too complex unless generality
can be exploited:
➢ For e...
(TINY) REASONING EXAMPLE
Query defines ?patient as a Patient instead of Person:
?patient a Patient ; has_MRN “1234” ; ...
...
RESOURCE PUBLISHING WITH SADI (1)
• Specify the source of data / software you
want to publish with SADI.
• Model data sema...
RESOURCE PUBLISHING WITH SADI (2)
• Define your services I/O semantically: decide how to describe
the operation of your se...
PRACTICAL EXAMPLE (1)
● Specify the source of data / software you want to
publish with SADI.
➢Database (CSV file) containi...
PRACTICAL EXAMPLE (2)
● Model data semantically: find ontologies describing your
domains and decide how your data will be ...
PRACTICAL EXAMPLE (3)
Background ontology medical_terminology.owl
Deploy:
cp medical_terminology.owl
/var/lib/tomcat7/weba...
PRACTICAL EXAMPLE (4)
● Define your services I/O semantically: decide how to describe the
operation of your services in th...
PRACTICAL EXAMPLE (5)
● Code the business logic of your services in Java, Perl or Python.
➢ There is a good open-source Ja...
PRACTICAL EXAMPLE (6)
Edit pom.xml and run service skeleton creation plug-in:
PRACTICAL EXAMPLE (7)
Just add your business logic code in processInput():
PRACTICAL EXAMPLE (8)
Source database patientsDB.csv :
PRACTICAL EXAMPLE (9)
Finished processInput()
for service
patientInfo :
PRACTICAL EXAMPLE (10)
Finished processInput() for service
computeBMI :
PRACTICAL EXAMPLE (11)
• Deploy the services:
COPY target/my-sadi-services.war TO /var/lib/tomcat7/webapps/
• Test service...
PRACTICAL EXAMPLE (12)
Test RDF for the services:
PRACTICAL EXAMPLE (13)
Service test runs with HTTP POST:
PRACTICAL EXAMPLE (14)
Running HYDRA command line application:
HYDRA PACKAGING
• Java API - can be embedded in something else.
• Command line application - convenient for small
experime...
REMEMBER OUR BIG VISION?
BIGGER VISION:
SELF-SERVICE AD HOC QUERYING OF FEDERATED DATA
THERE ARE NO PRINCIPLE OBSTACLES
TO SELF-SERVICE QUERYING BECAUSE ..
● HYDRA implements semantic querying:
○ users need no...
HYDRA QUERY COMPOSITION GUI PRINCIPLES
● Queries are rendered as highly readable graphs.
● A lot of query composition is d...
HYDRA GUI SCREENSHOTS
READABLE QUERY DESCRIPTION
EMPTY CANVAS
SERVICE REGISTRY
Note that we added allPatients that enumerates all patients with their MRN.
KEYPHRASE INPUT
HYDRA GUI PROPOSES QUERY GRAPHS
THE USER CAN CONFIRM THE WHOLE GRAPH
OR SOME PARTS OF IT
ADDING MNEMONIC VARIABLE NAME
MNEMONIC VARIABLE NAME ADDED
MORE KEYPHRASE INPUT
HYDRA GUI PROPOSES GRAPH AUGMENTATIONS
VARIABLE NAME
VARIABLE NAME ADDED
MANUALLY ADDING RELATIONS
Numeric
comparison <
here, but could
be any kinds of
relations.
EXTENDED GRAPH
SPECIFYING A DATA VALUE
EXTENDED GRAPH
The query is ready.
It finds all patients
with 20 < BMI < 30
and outputs their
BMI values and
MRNs.
HYDRA GUI GENERATES SPARQL
FROM QUERY GRAPHS
EXECUTING THE QUERY
ANSWERS
SAVING THE ANSWERS AS AN EXCEL
SPREADSHEET
THANK YOU!
Further materials/services are available on request:
• Live and recorded demos.
• Publications on previous (aca...
Upcoming SlideShare
Loading in …5
×

A practical introduction to SADI semantic Web services and HYDRA query tool

446 views

Published on

This sequel to the talk "Comrehensive Self-Service Life Science Data Federation with SADI semantic Web services and HYDRA" is more technical, although relatively self-contained, and tailored to people interested in practical aspects of SADI: I go step-by-step through an example of SADI service creation, deployment and use.

Published in: Science
  • Be the first to comment

A practical introduction to SADI semantic Web services and HYDRA query tool

  1. 1. A PRACTICAL INTRODUCTION TO SADI SEMANTIC WEB SERVICES AND HYDRA QUERY TOOL Alexandre Riazanov, CTO IPSNP Computing Inc Oslo University, Sep 23, 2015
  2. 2. PLAN OF THE TALK • A brief reminder of the previous episode: data federation with SADI and HYDRA. • RDF and OWL as syntactic foundations of service I/O and functionality descriptions. • Query execution with automatic service discovery and reasoning. • Resource publishing process with SADI, with a detailed practical example (time permitting).
  3. 3. DATA FEDERATION: QUERYING MULTIPLE HETEROGENEOUS SOURCES AS A SINGLE DB
  4. 4. QUERY EXAMPLES • Find the names of drugs that contain chemical category Y as active ingredients. • Find documents mentioning enzyme activity X, extract info on protein mutations and visualize mutations on 3D structure. • Annotate a DNA sequence X with molecular functions of proteins produced by the corresponding gene. • Find patients with precondition X diagnosed with infections Y resulting from procedure Z. • Find patients diagnosed with X while taking drug C.
  5. 5. HOW WE DO IT WITH HYDRA AND SADI SEMANTIC WEB SERVICES
  6. 6. A HIGH LEVEL VIEW OF THE HYDRA APPROACH ● Given a SPARQL query, HYDRA analyses it by using an intelligent logic-based algorithm (proprietary, unlike SADI itself). ● HYDRA requests descriptions of potentially useful services from available SADI service registries. ● HYDRA processes the descriptions and figures out which services have to be invoked, on what data and in what order. SPARQL is a W3C standard semantic query language -- much more intuitive than SQL.
  7. 7. HOW IS THIS ALL POSSIBLE? • Key ingredient: the SADI framework for Semantic Web services (Semantic Automated Discovery and Integration). • SADI services are: • RESTful services • consuming and producing one format -- RDF, • with semantic descriptions (in OWL) fully defining their functionality.
  8. 8. DIGRESSION: RDF • W3C RDF = Resource Description Framework • Standartised graph-based data model and a few standard rendering formats. • Nodes = objects (URIs) and data values like “abc”^^xsd:string or “123”^^xsd:integer. • Edges: binary relations.
  9. 9. RDF EXAMPLES @prefix mt: <http://localhost:8080/medical_terminology.owl#> . <http://example.com/patient#1234> rdf:type mt:Patient . <http://example.com/patient#1234> mt:has_mass _:hm . _:hm rdf:type mt:Measurement . _:hm mt:has_value "92.0"^^xsd:float . _:hm mt:has_units mt:kg . @prefix mt: <http://localhost:8080/medical_terminology.owl#> . <http://example.com/patient#1234> a mt:Person ; mt:has_mass [a mt:Measurement; mt:has_value "92.0"^^xsd:float; mt:has_units mt:kg] . The original XML- based rendering format is also popular.
  10. 10. DIGRESSION: OWL • W3C OWL = Web Ontology Language • Essentially, extends RDF with definitions and other axioms for classes (types of objects) and properties (binary relations). • Most useful axiom types -- class and property chierarchies: Patient subClassOf Person loves subPropertyOf knows • SADI reuses property restriction syntax: has_MRN exactly 1 string
  11. 11. SADI SERVICE I/O • Input: RDF description of an input object. • Output: another RDF graph providing more (computed or retrieved) info about the input object or linking it to other objects. • Since all SADI services “talk the same language” (RDF), they are 100% syntactically interoperable: – output of one SADI service can be directly consumed by any other SADI services.
  12. 12. COMPLETE SEMANTIC DESCRIPTIONS OF SERVICE FUNCTIONALITY SADI services publish semantic descriptions of their I/O that completely define what the service expects and can accept as input, and what RDF assertions the service can output. • Unique and extremely powerful property: it facilitates completely automatic discovery and orchestration of services.
  13. 13. Example: computeBMI service I/O
  14. 14. SEMANTIC FUNCTIONALITY DESCRIPTION • OWL syntax is repurposed to define what RDF graphs are acceptable as input, and what RDF graphs may be produced in the output. • Input(computeBMI) = Person and (has_height exactly 1 (Measurement and (has_value exactly 1 float))) • Output(computeBMI) = has_BMI exactly 1 float
  15. 15. SERVICE INPUT CLASS • Specifies what kind of objects (RDF descriptions) the service expects in the input. OWL syntax is convenient for such definitions. • Almost always just an enumeration of attributes of the input objects the SADI service expects. ● If the input class is defined as Person and (has_height exactly 1 (Measurement and (has_value exactly 1 float) and (has_units exactly 1 {m})) and (has_mass exactly 1 (Measurement and (has_value exactly 1 float) and (has_units exactly 1 {kg})) … the service expects something like this in the input: patient1234 a Person; has_height [a Measurement; has_value “1.7"^^xsd:float; has_units m]; has_mass [a Measurement; has_value “92.0"^^xsd:float; has_units kg]
  16. 16. SERVICE OUTPUT CLASS • A SADI service advertises itself by publishing its output class specifying what the service promises to produce as the output. • The class must enumerate attributes that the service will add to the input object. This fully semantically defines what the service does! ● If the output class is defined as has_BMI exactly 1 float … service clients can expect something like this in the output: patient1234 has_BMI “31.83”^^xsd:float
  17. 17. DIGRESSION: SPARQL • W3C SPARQL - standard query language for the RDF data model. • SPARQL clients are programs that execute SPARQL queries, typically on RDF triplestores. PREFIX mt: <http://localhost:8080/medical_terminology.owl#> SELECT ?mass { <http://example.com/patient#1234> a mt:Person ; mt:has_mass [a mt:Measurement; mt:has_value ?mass; mt:has_units mt:kg] . } • HYDRA is also a SPARQL client, but for virtual RDF DBs.
  18. 18. AUTOMATIC SERVICE DISCOVERY • With the I/O descriptions, a sufficiently intelligent client can figure out that it can call the service if the client has to satisfy a query condition like this: patient1234 has_BMI ?bmi_value • The query condition suggests that a service with has_BMI in the output may be useful if called on the object patient1234 • To make the call, the client must have enough information about patient1234 : according to the input class, has_height and has_mass must be attached to it and sent to the service.
  19. 19. QUERY, EXECUTION, ANSWERS Query: FROM <.......rdf> # seed data SELECT ?bmi_value { patient1234 a Person; has_BMI ?bmi_value } Execution: HYDRA ● seed data in FROM clause describes the heights and weights of some people, including patient1234, using has_height and has_mass; ● since has_BMI is there, HYDRA looks for all services in the available registries that can attach has_BMI and finds computeBMI; ● patient1234 satisfies the input condition of computeBMI, so HYDRA calls it; ● computeBMI returns patient1234 has_BMI “32.3” so HYDRA can return an an answer: ?bmi_value = “32.3”
  20. 20. MULTIPLE SERVICES • Suppose, we don’t know patient’s height/mass, but can retrieve them from a DB by patient’s medical record number (MRN). • We write another SADI service, patientInfo : Output(patientInfo) = (has_height exactly 1 (Measurement and (has_value exactly 1 float) and (has_units exactly 1 {m})) and (has_mass exactly 1 (Measurement and (has_value exactly 1 float) and (has_units exactly 1 {kg})) Input(patientInfo) = Person and (has_MRN exactly 1 string)
  21. 21. AUTOMATIC SERVICE COMPOSITION • HYDRA can figure out automatically that the output of patientInfo can be submitted to computeBMI, and the composition of the services can solve the query SELECT ?bmi_value { ?patient a Person ; has_MRN “1234” ; has_BMI ?bmi_value } (no has_height or has_mass anywhere !)
  22. 22. INTELLIGENT (REASONING-ENABLED) QUERY EXECUTION ● Some queries are too complex unless generality can be exploited: ➢ For example, query concerning all antibiotics requires generalisation, otherwise all types of antibiotics would have to be enumerated in the query. ● Much better way to do this is to import a classification of drugs and use it in query execution. ● HYDRA facilitates such reasoning and even more complex reasoning with rules.
  23. 23. (TINY) REASONING EXAMPLE Query defines ?patient as a Patient instead of Person: ?patient a Patient ; has_MRN “1234” ; ... ● HYDRA is still able to call patientInfo on the Patient instance, say patient1234, if there is an axiom Patient subClassOf Person. It infers patient1234 a Person, which can be used as input to patientInfo. ● The axiom can be included in the definition of Output (patientInfo), or specified separately.
  24. 24. RESOURCE PUBLISHING WITH SADI (1) • Specify the source of data / software you want to publish with SADI. • Model data semantically: find ontologies describing your domains and decide how your data will be expressed in the terms of these ontologies. For example, a patient database and a BMI computation algorithm.
  25. 25. RESOURCE PUBLISHING WITH SADI (2) • Define your services I/O semantically: decide how to describe the operation of your services in the terms of the domain ontologies, i.e., what will be written in the input and output classes. • Code the business logic of your services in Java, Perl or Python. If a service wraps a DB, convert the input RDF into a query and the query results back to RDF. The coding effort is usually tiny compared to the modelling. • Overall development costs may be considerable, but this cost is well amortized because SADI services are highly reusable, due to their unprecedented degree of interoperability and discoverability.
  26. 26. PRACTICAL EXAMPLE (1) ● Specify the source of data / software you want to publish with SADI. ➢Database (CSV file) containing patient MRN, name, height, weight, etc. We will use it to implement patientInfo. ➢BMI computation algorithm: BMI = mass, kg / height, m ^2.
  27. 27. PRACTICAL EXAMPLE (2) ● Model data semantically: find ontologies describing your domains and decide how your data will be expressed in the terms of these ontologies. ➢ Create ontology clinical_terms.owl in Protégé: ➢ Classes: Person, Patient, Measurement, Units ➢ Properties: has_BMI, has_MRN, has_height, has_mass, has_value, has_units. ➢ Individuals: m, kg. ➢ RDF data sample: patient1234 a Patient; has_MRN “1234”^^xsd:string; has_height [a Measurement; has_value “1.7"^^xsd:float; has_units m]; . . .
  28. 28. PRACTICAL EXAMPLE (3) Background ontology medical_terminology.owl Deploy: cp medical_terminology.owl /var/lib/tomcat7/webapps/ROOT/ URL: http://localhost:8080/medical_terminology.owl
  29. 29. PRACTICAL EXAMPLE (4) ● Define your services I/O semantically: decide how to describe the operation of your services in the terms of the domain ontologies, i.e., what will be written in the input and output classes. ➢ I/O ontologies: patientInfo.owl and computeBMI.owl, importing medical_terminology.owl
  30. 30. PRACTICAL EXAMPLE (5) ● Code the business logic of your services in Java, Perl or Python. ➢ There is a good open-source Java library for creating SADI services as Java Servlets. ➢ A skeleton code for a service is generated automatically; we just have to fill the body of one method. ➢ The library takes care of all the HTTP connectivity issues, parses the input RDF to a simple abstract representation (Jena), and renders the output RDF. ➢ The compiled WAR file can be immediately deployed on a servlet container (Tomcat, Jetty, etc). ➢ SADI services take only 10-15 min to code (if the business logic is simple or already programmed).
  31. 31. PRACTICAL EXAMPLE (6) Edit pom.xml and run service skeleton creation plug-in:
  32. 32. PRACTICAL EXAMPLE (7) Just add your business logic code in processInput():
  33. 33. PRACTICAL EXAMPLE (8) Source database patientsDB.csv :
  34. 34. PRACTICAL EXAMPLE (9) Finished processInput() for service patientInfo :
  35. 35. PRACTICAL EXAMPLE (10) Finished processInput() for service computeBMI :
  36. 36. PRACTICAL EXAMPLE (11) • Deploy the services: COPY target/my-sadi-services.war TO /var/lib/tomcat7/webapps/ • Test service description availability (HTTP GET):
  37. 37. PRACTICAL EXAMPLE (12) Test RDF for the services:
  38. 38. PRACTICAL EXAMPLE (13) Service test runs with HTTP POST:
  39. 39. PRACTICAL EXAMPLE (14) Running HYDRA command line application:
  40. 40. HYDRA PACKAGING • Java API - can be embedded in something else. • Command line application - convenient for small experiments. • Web service (Java servlet) with – JSON-based protocol – Java client-side API.
  41. 41. REMEMBER OUR BIG VISION?
  42. 42. BIGGER VISION: SELF-SERVICE AD HOC QUERYING OF FEDERATED DATA
  43. 43. THERE ARE NO PRINCIPLE OBSTACLES TO SELF-SERVICE QUERYING BECAUSE .. ● HYDRA implements semantic querying: ○ users need not know how the source data is organised or accessed. ● HYDRA can apply concept hierarchies and rules: ○ syntactically simple queries for complex questions. We just need an adequate user interface for building queries.
  44. 44. HYDRA QUERY COMPOSITION GUI PRINCIPLES ● Queries are rendered as highly readable graphs. ● A lot of query composition is done by entering keyphrases in English; ○ HYDRA GUI suggests (sub)graphs implementing a given keyphrase. ● Nodes can be delete/added manually; ○ the system suggests possibilities (navigation).
  45. 45. HYDRA GUI SCREENSHOTS
  46. 46. READABLE QUERY DESCRIPTION
  47. 47. EMPTY CANVAS
  48. 48. SERVICE REGISTRY Note that we added allPatients that enumerates all patients with their MRN.
  49. 49. KEYPHRASE INPUT
  50. 50. HYDRA GUI PROPOSES QUERY GRAPHS
  51. 51. THE USER CAN CONFIRM THE WHOLE GRAPH OR SOME PARTS OF IT
  52. 52. ADDING MNEMONIC VARIABLE NAME
  53. 53. MNEMONIC VARIABLE NAME ADDED
  54. 54. MORE KEYPHRASE INPUT
  55. 55. HYDRA GUI PROPOSES GRAPH AUGMENTATIONS
  56. 56. VARIABLE NAME
  57. 57. VARIABLE NAME ADDED
  58. 58. MANUALLY ADDING RELATIONS Numeric comparison < here, but could be any kinds of relations.
  59. 59. EXTENDED GRAPH
  60. 60. SPECIFYING A DATA VALUE
  61. 61. EXTENDED GRAPH The query is ready. It finds all patients with 20 < BMI < 30 and outputs their BMI values and MRNs.
  62. 62. HYDRA GUI GENERATES SPARQL FROM QUERY GRAPHS
  63. 63. EXECUTING THE QUERY
  64. 64. ANSWERS
  65. 65. SAVING THE ANSWERS AS AN EXCEL SPREADSHEET
  66. 66. THANK YOU! Further materials/services are available on request: • Live and recorded demos. • Publications on previous (academic) case studies. • Training/consulting. • http://ipsnp.com/ (Canada) and http://ipsnp.co/ (UK)

×