This sequel to the talk "Comrehensive Self-Service Life Science Data Federation with SADI semantic Web services and HYDRA" is more technical, although relatively self-contained, and tailored to people interested in practical aspects of SADI: I go step-by-step through an example of SADI service creation, deployment and use.

1. A PRACTICAL INTRODUCTION TO
SADI SEMANTIC WEB SERVICES
AND HYDRA QUERY TOOL
Alexandre Riazanov, CTO
IPSNP Computing Inc
Oslo University, Sep 23, 2015

2. PLAN OF THE TALK
• A brief reminder of the previous episode: data
federation with SADI and HYDRA.
• RDF and OWL as syntactic foundations of service I/O
and functionality descriptions.
• Query execution with automatic service discovery and
reasoning.
• Resource publishing process with SADI, with a detailed
practical example (time permitting).

3. DATA FEDERATION: QUERYING MULTIPLE
HETEROGENEOUS SOURCES AS A SINGLE DB

4. QUERY EXAMPLES
• Find the names of drugs that contain chemical category Y as
active ingredients.
• Find documents mentioning enzyme activity X, extract info on
protein mutations and visualize mutations on 3D structure.
• Annotate a DNA sequence X with molecular functions of
proteins produced by the corresponding gene.
• Find patients with precondition X diagnosed with infections Y
resulting from procedure Z.
• Find patients diagnosed with X while taking drug C.

5. HOW WE DO IT WITH HYDRA AND SADI
SEMANTIC WEB SERVICES

6. A HIGH LEVEL VIEW OF THE HYDRA APPROACH
● Given a SPARQL query, HYDRA analyses it
by using an intelligent logic-based algorithm
(proprietary, unlike SADI itself).
● HYDRA requests descriptions of potentially
useful services from available SADI service
registries.
● HYDRA processes the descriptions and
figures out which services have to be
invoked, on what data and in what order.
SPARQL is a W3C
standard semantic
query language --
much more intuitive
than SQL.

7. HOW IS THIS ALL POSSIBLE?
• Key ingredient: the SADI framework for
Semantic Web services (Semantic Automated
Discovery and Integration).
• SADI services are:
• RESTful services
• consuming and producing one format -- RDF,
• with semantic descriptions (in OWL) fully defining
their functionality.

8. DIGRESSION: RDF
• W3C RDF = Resource Description
Framework
• Standartised graph-based data model and a
few standard rendering formats.
• Nodes = objects (URIs) and data values like
“abc”^^xsd:string or “123”^^xsd:integer.
• Edges: binary relations.

9. RDF EXAMPLES
@prefix mt: <http://localhost:8080/medical_terminology.owl#> .
<http://example.com/patient#1234> rdf:type mt:Patient .
<http://example.com/patient#1234> mt:has_mass _:hm .
_:hm rdf:type mt:Measurement .
_:hm mt:has_value "92.0"^^xsd:float .
_:hm mt:has_units mt:kg .
@prefix mt: <http://localhost:8080/medical_terminology.owl#> .
<http://example.com/patient#1234> a mt:Person ;
mt:has_mass [a mt:Measurement;
mt:has_value "92.0"^^xsd:float;
mt:has_units mt:kg] .
The original XML-
based rendering
format is also
popular.

10. DIGRESSION: OWL
• W3C OWL = Web Ontology Language
• Essentially, extends RDF with definitions and other axioms
for classes (types of objects) and properties (binary
relations).
• Most useful axiom types -- class and property chierarchies:
Patient subClassOf Person
loves subPropertyOf knows
• SADI reuses property restriction syntax:
has_MRN exactly 1 string

11. SADI SERVICE I/O
• Input: RDF description of an input object.
• Output: another RDF graph providing more
(computed or retrieved) info about the input
object or linking it to other objects.
• Since all SADI services “talk the same
language” (RDF), they are 100% syntactically
interoperable:
– output of one SADI service can be directly
consumed by any other SADI services.

12. COMPLETE SEMANTIC DESCRIPTIONS
OF SERVICE FUNCTIONALITY
SADI services publish semantic descriptions of their
I/O that completely define what the service expects and
can accept as input, and what RDF assertions the service
can output.
• Unique and extremely powerful property: it facilitates
completely automatic discovery
and
orchestration of services.

13. Example: computeBMI service I/O

14. SEMANTIC FUNCTIONALITY DESCRIPTION
• OWL syntax is repurposed to define what RDF graphs are
acceptable as input, and what RDF graphs may be produced
in the output.
• Input(computeBMI) =
Person and (has_height exactly 1
(Measurement and (has_value exactly 1
float)))
• Output(computeBMI) = has_BMI exactly 1 float

15. SERVICE INPUT CLASS
• Specifies what kind of objects (RDF descriptions) the service
expects in the input. OWL syntax is convenient for such definitions.
• Almost always just an enumeration of attributes of the input objects
the SADI service expects.
● If the input class is defined as
Person and
(has_height exactly 1
(Measurement and
(has_value exactly 1 float) and
(has_units exactly 1 {m})) and
(has_mass exactly 1
(Measurement and
(has_value exactly 1 float) and
(has_units exactly 1 {kg}))
… the service expects something
like this in the input:
patient1234 a Person;
has_height [a Measurement;
has_value “1.7"^^xsd:float;
has_units m];
has_mass [a Measurement;
has_value “92.0"^^xsd:float;
has_units kg]

16. SERVICE OUTPUT CLASS
• A SADI service advertises itself by publishing its
output class specifying what the service promises
to produce as the output.
• The class must enumerate attributes that the
service will add to the input object. This fully
semantically defines what the service does!
● If the output class is defined as
has_BMI exactly 1 float
… service clients can expect
something like this in the output:
patient1234 has_BMI “31.83”^^xsd:float

17. DIGRESSION: SPARQL
• W3C SPARQL - standard query language for
the RDF data model.
• SPARQL clients are programs that execute
SPARQL queries, typically on RDF triplestores.
PREFIX mt: <http://localhost:8080/medical_terminology.owl#>
SELECT ?mass
{
<http://example.com/patient#1234> a mt:Person ;
mt:has_mass [a mt:Measurement;
mt:has_value ?mass;
mt:has_units mt:kg] . }
• HYDRA is also a
SPARQL client,
but for virtual
RDF DBs.

18. AUTOMATIC SERVICE DISCOVERY
• With the I/O descriptions, a sufficiently intelligent
client can figure out that it can call the service if the
client has to satisfy a query condition like this:
patient1234 has_BMI ?bmi_value
• The query condition suggests that a service with
has_BMI in the output may be useful if called on the
object patient1234
• To make the call, the client must have enough
information about patient1234 : according to the input
class, has_height and has_mass must be attached to it
and sent to the service.

19. QUERY, EXECUTION, ANSWERS
Query:
FROM <.......rdf> # seed data
SELECT ?bmi_value
{
patient1234 a Person;
has_BMI ?bmi_value
}
Execution: HYDRA
● seed data in FROM clause describes the
heights and weights of some people, including
patient1234, using has_height and has_mass;
● since has_BMI is there, HYDRA looks for all
services in the available registries that can
attach has_BMI and finds computeBMI;
● patient1234 satisfies the input condition of
computeBMI, so HYDRA calls it;
● computeBMI returns
patient1234 has_BMI “32.3”
so HYDRA can return an an answer:
?bmi_value = “32.3”

20. MULTIPLE SERVICES
• Suppose, we don’t know patient’s height/mass, but can
retrieve them from a DB by patient’s medical record
number (MRN).
• We write another SADI service, patientInfo :
Output(patientInfo) =
(has_height exactly 1
(Measurement and
(has_value exactly 1 float) and
(has_units exactly 1 {m})) and
(has_mass exactly 1
(Measurement and
(has_value exactly 1 float) and
(has_units exactly 1 {kg}))
Input(patientInfo) =
Person and (has_MRN exactly 1 string)

21. AUTOMATIC SERVICE COMPOSITION
• HYDRA can figure out automatically that the output of
patientInfo can be submitted to computeBMI, and
the composition of the services can solve the query
SELECT ?bmi_value
{
?patient a Person ;
has_MRN “1234” ;
has_BMI ?bmi_value
}
(no has_height or has_mass anywhere !)

22. INTELLIGENT (REASONING-ENABLED) QUERY
EXECUTION
● Some queries are too complex unless generality
can be exploited:
➢ For example, query concerning all antibiotics
requires generalisation, otherwise all types of
antibiotics would have to be enumerated in the
query.
● Much better way to do this is to import a
classification of drugs and use it in query execution.
● HYDRA facilitates such reasoning and even more
complex reasoning with rules.

23. (TINY) REASONING EXAMPLE
Query defines ?patient as a Patient instead of Person:
?patient a Patient ; has_MRN “1234” ; ...
● HYDRA is still able to call patientInfo on the Patient
instance, say patient1234, if there is an axiom Patient
subClassOf Person. It infers patient1234 a Person, which
can be used as input to patientInfo.
● The axiom can be included in the definition of Output
(patientInfo), or specified separately.

24. RESOURCE PUBLISHING WITH SADI (1)
• Specify the source of data / software you
want to publish with SADI.
• Model data semantically: find ontologies
describing your domains and decide how
your data will be expressed in the terms of
these ontologies.
For example, a
patient database
and a BMI
computation
algorithm.

25. RESOURCE PUBLISHING WITH SADI (2)
• Define your services I/O semantically: decide how to describe
the operation of your services in the terms of the domain
ontologies, i.e., what will be written in the input and output classes.
• Code the business logic of your services in Java, Perl or
Python. If a service wraps a DB, convert the input RDF into a
query and the query results back to RDF. The coding effort is usually
tiny compared to the modelling.
• Overall development costs may be considerable, but this cost is
well amortized because SADI services are highly reusable, due
to their unprecedented degree of interoperability and
discoverability.

26. PRACTICAL EXAMPLE (1)
● Specify the source of data / software you want to
publish with SADI.
➢Database (CSV file) containing patient MRN, name,
height, weight, etc. We will use it to implement
patientInfo.
➢BMI computation algorithm:
BMI = mass, kg / height, m ^2.

27. PRACTICAL EXAMPLE (2)
● Model data semantically: find ontologies describing your
domains and decide how your data will be expressed in the
terms of these ontologies.
➢ Create ontology clinical_terms.owl in Protégé:
➢ Classes: Person, Patient, Measurement, Units
➢ Properties: has_BMI, has_MRN, has_height, has_mass,
has_value, has_units.
➢ Individuals: m, kg.
➢ RDF data sample:
patient1234 a Patient;
has_MRN “1234”^^xsd:string;
has_height [a Measurement;
has_value “1.7"^^xsd:float;
has_units m];
. . .

28. PRACTICAL EXAMPLE (3)
Background ontology medical_terminology.owl
Deploy:
cp medical_terminology.owl
/var/lib/tomcat7/webapps/ROOT/
URL: http://localhost:8080/medical_terminology.owl

29. PRACTICAL EXAMPLE (4)
● Define your services I/O semantically: decide how to describe the
operation of your services in the terms of the domain ontologies, i.e.,
what will be written in the input and output classes.
➢ I/O ontologies: patientInfo.owl and computeBMI.owl, importing
medical_terminology.owl

30. PRACTICAL EXAMPLE (5)
● Code the business logic of your services in Java, Perl or Python.
➢ There is a good open-source Java library for creating SADI
services as Java Servlets.
➢ A skeleton code for a service is generated automatically; we just
have to fill the body of one method.
➢ The library takes care of all the HTTP connectivity issues, parses
the input RDF to a simple abstract representation (Jena), and
renders the output RDF.
➢ The compiled WAR file can be immediately deployed on a servlet
container (Tomcat, Jetty, etc).
➢ SADI services take only 10-15 min to code (if the business logic is
simple or already programmed).

31. PRACTICAL EXAMPLE (6)
Edit pom.xml and run service skeleton creation plug-in:

32. PRACTICAL EXAMPLE (7)
Just add your business logic code in processInput():

33. PRACTICAL EXAMPLE (8)
Source database patientsDB.csv :

34. PRACTICAL EXAMPLE (9)
Finished processInput()
for service
patientInfo :

35. PRACTICAL EXAMPLE (10)
Finished processInput() for service
computeBMI :

36. PRACTICAL EXAMPLE (11)
• Deploy the services:
COPY target/my-sadi-services.war TO /var/lib/tomcat7/webapps/
• Test service description availability (HTTP GET):

37. PRACTICAL EXAMPLE (12)
Test RDF for the services:

38. PRACTICAL EXAMPLE (13)
Service test runs with HTTP POST:

39. PRACTICAL EXAMPLE (14)
Running HYDRA command line application:

40. HYDRA PACKAGING
• Java API - can be embedded in something else.
• Command line application - convenient for small
experiments.
• Web service (Java servlet) with
– JSON-based protocol
– Java client-side API.

41. REMEMBER OUR BIG VISION?

42. BIGGER VISION:
SELF-SERVICE AD HOC QUERYING OF FEDERATED DATA

43. THERE ARE NO PRINCIPLE OBSTACLES
TO SELF-SERVICE QUERYING BECAUSE ..
● HYDRA implements semantic querying:
○ users need not know how the source data is organised or
accessed.
● HYDRA can apply concept hierarchies and rules:
○ syntactically simple queries for complex questions.
We just need an adequate user interface
for building queries.

44. HYDRA QUERY COMPOSITION GUI PRINCIPLES
● Queries are rendered as highly readable graphs.
● A lot of query composition is done by entering
keyphrases in English;
○ HYDRA GUI suggests (sub)graphs
implementing a given keyphrase.
● Nodes can be delete/added manually;
○ the system suggests possibilities (navigation).

45. HYDRA GUI SCREENSHOTS

46. READABLE QUERY DESCRIPTION

47. EMPTY CANVAS

48. SERVICE REGISTRY
Note that we added allPatients that enumerates all patients with their MRN.

49. KEYPHRASE INPUT

50. HYDRA GUI PROPOSES QUERY GRAPHS

51. THE USER CAN CONFIRM THE WHOLE GRAPH
OR SOME PARTS OF IT

52. ADDING MNEMONIC VARIABLE NAME

53. MNEMONIC VARIABLE NAME ADDED

54. MORE KEYPHRASE INPUT

55. HYDRA GUI PROPOSES GRAPH AUGMENTATIONS

56. VARIABLE NAME

57. VARIABLE NAME ADDED

58. MANUALLY ADDING RELATIONS
Numeric
comparison <
here, but could
be any kinds of
relations.

59. EXTENDED GRAPH

60. SPECIFYING A DATA VALUE

61. EXTENDED GRAPH
The query is ready.
It finds all patients
with 20 < BMI < 30
and outputs their
BMI values and
MRNs.

62. HYDRA GUI GENERATES SPARQL
FROM QUERY GRAPHS

63. EXECUTING THE QUERY

64. ANSWERS

65. SAVING THE ANSWERS AS AN EXCEL
SPREADSHEET

66. THANK YOU!
Further materials/services are available on request:
• Live and recorded demos.
• Publications on previous (academic) case studies.
• Training/consulting.
• http://ipsnp.com/ (Canada) and http://ipsnp.co/ (UK)