Voxxed Athens 2018 - Graph databases & data integration by Dimitris Kontokostas

1. GraphDatabases&
dataintegration
VoxxedDaysAthens2018
DimitrisKontokostas
SeniorKnowledgeEngineer@GeoPhy

2. Aboutme
●Datageek,softwareengineer&opensourceenthusiast
●InvolvedinmanyR&Dprojectssince2003
●Participate(d)ingraph-relatedstandardizationactivities
●PhDinknowledgeextractionandqualityassessment
●WorkingontheGeoPhyRealEstateKnowledgeGraph

3. Agenda
●Graphs
●RDFGraphs(*)
●Semantics&whytheymatter(*)
●LinkedData
●WhousesRDF
●HowGoogleusesRDF
●Howwe(GeoPhy)usesRDF
(*)
Someconceptsaresimplifiedorskippedtomakethistalkeasiertodigestintheallocatedtime

5. HeatmapforGraphDatabases
(*)Seealsothis
Gartnerstudyin2013found:
●manyorganizationsfindthe
varietydimensionagreater
challengethanvolumeor
velocity.
GraphDBstotherescue:
●Combinemultiplesourceswith
differentstructures
●Retaintheflexibilitytoadd
newoneswithoutadapting
schemas
●Querycombineddata,or
multiplesourcesatonce
●Detectpatternsinthedata

6. ©ImagebyMaxDeMargi

7. ●Agraphisawayofspecifyingrelationshipsamongacollectionofitems
●Itemscanbe:
○Nodes:Alice,Bob,…
○Edges
■undirected:knows,…
■directed:follows,…
○Attributes:name,age,type,since,...
○Values:18,2001/10/13,...
Graphs
Imagesourcefromwikimediacommons

8. GraphDataModels
Propertygraphs
●Industrystandards
○CyphermainlyNeo4j
○GremlintraversalAPI
(ApacheTinkerPop)
=>Mostcommon
○GraphQL
●Dataimport/exportusingCypher,
gremlinorvendor-specific
●Usuallyoptimizedforspecific
operations/usecases
RDFGraphs
●W3Cstandards
○LikeXML,HTML,defineonce
runeverywhere™
●Standardisedwayforquerying
(SPARQL),exporting&importing
(RDF)
SlideinputfromAndySeaborn@VoxxedDaysBristol

9. GraphDatabasesLandscape
PropertyGraphs
GremlintraversalAPI
RDFGraphs
SPARQL
Hybrid
GremlinAPI+SPARQL
+Cypher

10. ●Eachnodehas
○uniqueidentifier
○outgoingedges
○incomingedges
○key-valuepropertiescollection
●Eachedgehas
○uniqueidentifier
○direction
○labelfortherelationship
○key-valuepropertiescollection
●Extremeflexibility
PropertyGraphs

11. RDF-ResourceDescriptionFramework
●AnRDFGraphisasetofRDFTriples
●AnRDFtripleconsistsofonlythreecomponents(simplified):
○thesubjectwhichisaThing
○thepredicatewhichisa(special)Thing
○theobjectthatcanbeeitheraThingoraLiteral(Value)
●ThingsarerepresentedwithURIs
●Literalshaveavalueandavaluetypeoralanguagetag(defaultstostring)
SubjectPredicateObject

12. RDF-ResourceDescriptionFramework
●AnRDFGraphisasetofRDFTriples
●AnRDFtripleconsistsofonlythreecomponents(simplified):
○thesubjectwhichisaThing
○thepredicatewhichisa(special)Thing
○theobjectthatcanbeeitheraThingoraLiteral(Value)
●ThingsarerepresentedwithURIs
●Literalshaveavalueandavaluetypeoralanguagetag(defaultstostring)
SubjectPredicateObject

13. RDF-ResourceDescriptionFramework
Dependingontheserializationformat,URIscanbeabbreviatedwithnamespaces
>justlikeXML
>Improvesreadability,e.g.
@prefixdbpedia:<http://dbpedia.org/resource/>.
@prefixschema:<http://schema.org/>.
SubjectPredicateObject

14. RDFisanabstractdatamodel
Manydifferentserializationformats…
Turtle,NTriples,JSON-LD,XML,RDFa,Microdata*

15. RDFisanabstractdatamodel
Manydifferentserializationformats…
Turtle,NTriples,JSON-LD,XML,RDFa,Microdata*
@prefixdbpedia:<http://dbpedia.org/resource/>.
@prefixschema:<http://schema.org/>.
@prefixxsd:<http://www.w3.org/2001/XMLSchema#>.
dbpedia:Friends
schema:name"Friends"@en;
schema:datePublished"1994-09-22"^^xsd:date;
schema:numberOfSeasons10;
schema:genredbpedia:Sitcom.
dbpedia:The_Office
schema:name"TheOffice"@en;
schema:genredbpedia:Sitcom.

16. RDFisanabstractdatamodel
Manydifferentserializationformats…
Turtle,NTriples,JSON-LD,XML,RDFa,Microdata*

17. RDFisanabstractdatamodel
Manydifferentserializationformats…
Turtle,NTriples,JSON-LD,XML,RDFa,Microdata*

18. RDFisanabstractdatamodel
Manydifferentserializationformats…
Turtle,NTriples,JSON-LD,XML,RDFa,Microdata*

19. [Funfact]
WhatdoesRSSstandfor?
RichSiteSummarybut...
Originalnamewas:RDFSiteSummary
BasedonfirstversionsofRDF/XML
Seehttps://en.wikipedia.org/wiki/RSS

20. RDFisanabstractdatamodel
Manydifferentserializationformats…
Turtle,NTriples,JSON-LD,XML,RDFa,Microdata*

21. RDFisanabstractdatamodel
Manydifferentserializationformats…
Turtle,NTriples,JSON-LD,XML,RDFa,Microdata*

22. YoucanstoreRDF...
Insimple(text)files,
locally,remote,HDFS,...
Embeddedwebdocuments
Ingraphdatabases

23. RDF&Graphs(Separate)
@prefixdbpedia:<http://dbpedia.org/resource/>.
@prefixschema:<http://schema.org/>.
@prefixxsd:<http://www.w3.org/2001/XMLSchema#>.
dbpedia:Friends
schema:numberOfSeasons10;
schema:datePublished"1994-09-22"^^xsd:date;
schema:genredbpedia:Sitcom.
@prefixdbpedia:<http://dbpedia.org/resource/>.
@prefixschema:<http://schema.org/>.
@prefixxsd:<http://www.w3.org/2001/XMLSchema#>.
dbpedia:Friendsschema:name"Friends"@en.
dbpedia:The_Officeschema:name"TheOffice"@en.
/data/tvseries/labels.ttl
/data/tvseries/metadata.ttl

24. RDF&Graphs(merge)
File_all.ttl
Canyounameofany
otherformatwherefiles
canbemergedwithout
losingdataintegrity?
CSV,SQL,XML,JSON,...
@prefixdbpedia:<http://dbpedia.org/resource/>.
@prefixschema:<http://schema.org/>.
@prefixxsd:<http://www.w3.org/2001/XMLSchema#>.
dbpedia:Friends
schema:name"Friends"@en;
schema:numberOfSeasons10;
schema:datePublished"1994-09-22"^^xsd:date;
schema:genredbpedia:Sitcom.
dbpedia:The_Office
schema:name"TheOffice"@en;
schema:genredbpedia:Sitcom.
/data/tvseries.ttl

25. Datasets/multi-graphTriGfiles
@prefixdbpedia:<http://dbpedia.org/resource/>.
@prefixschema:<http://schema.org/>.
@prefixxsd:<http://www.w3.org/2001/XMLSchema#>.
<http://example.com/labels>{
dbpedia:Friendsschema:name"Friends"@en;
dbpedia:The_Officeschema:name"TheOffice"@en;
}
<http://example.com/metadata>{
dbpedia:Friends
schema:datePublished"1994-09-22"^^xsd:date;
schema:numberOfSeasons10.
}
<http://example.com/genre>{
dbpedia:Friendsschema:genredbpedia:Sitcom.
dbpedia:The_Officeschema:genredbpedia:Sitcom.
}
/data/tvseries.trig

26. RDFispersistent,whereverit’sstored
RDFDB
Input
Files
Output
Files
Import
Export
Exactly
same(*)
(*)
Thepropertermisisomorphicgraphs,tocoversomespecialcasescalledblanknodes
Query

27. Bigecosystem
SPARQL:RDFquerylanguage
RDFS,OWL:RDFschemalanguages
SHACL,ShEx:RDFconstraintlanguages
Seehttp://book.validatingrdf.com(freeonline)
R2RML:VirtualRDFviewsontopofRDBMS(i.e.MySQL)
Andmanymorespecification&tools...

28. Takeawaypoints,sofar...
RDFisagraphdatamodel
>canbeserializedinmanyformats
>identifiersarepersistentbydesign
Nativelystore&integratesdiversedata
RDFiskindofthenewXML
>butitismuchcooler...
>andyoudon’tneedtowriteXML;)

29. Semantics&RDF
Whytheymatter

30. Semantics&RDF
●RDFisacorepartoftheSemanticWebvision
●Semanticsisdefinedas:
○themeaningofsomething(word,phrase,text,etc)
○thebranchoflinguisticsandlogicconcernedwithmeaning
●Tooacademic?
“ALittleSemanticsGoesaLongWay”
byprof.J.Hendler
BuzzwordAlert!!!

31. RDF&Semantics
Ontologiesaretheresultsofmodellingaspecificdomain
Somepeopleprefertheterms:model,vocabulary,taxonomy,schema
(doesn’tmakemuchdifference)
OntologiesinRDFdealwithclasses&properties
>Somepartismachinereadable
>Somepartishumanreadable
Canyoutellwhichpartismoreimportant?
(...amorepragmaticview)

32. @prefixex:<http://example.com/>
ex:TVSeries
rdf:typerdfs:Class;
rdfs:comment“SeriesdedicatedtoTVbroadcast”;
rdfs:subClassOfex:CreativeWork.
ex:CreativeWork
rdf:typerdfs:Class;
rdfs:comment“Agenerickindofcreativework,i.e.books,movies,etc.”.
RDFSchema-Classes
ClassesofThings
Machine-Readable
Semantics
Human-Readable
Semantics
…andwecanassigntypestoThings
(i.e.“Friends”isaninstanceof“TVSeries”)
dbpedia:Friendsrdf:typeex:TVSeries.

33. @prefixex:<http://example.com/>
ex:actor
rdf:typerdf:Property;
rdfs:comment“ThepersonthatistheactorofaTVSeries.”;
rdfs:domainex:TVSeries;
rdfs:rangeex:Person.
RDFSchema-Properties
Relationshipsbetweensubjectsandobjects
Machine-Readable
Semantics
Human-Readable
Semantics
dbpedia:Friendsex:actordbpedia:Jennifer_Aniston.
…andwecanusethisinRDFstatements

34. toInferortoValidate?
Givenonlythefollowing,whatcanwesayabout
dbpedia:Jennifer_Anistonanddbpedia:Friends?
dbpedia:Jennifer_Anistonrdf:typeex:Person.
dbpedia:Friendsrdf:typeex:TVSeries.
ex:actor
rdf:typerdf:Property;
rdfs:domainex:TVSeries;
rdfs:rangeex:Person.
dbpedia:Friendsex:actordbpedia:Jennifer_Aniston.

35. toInferortoValidate?
Givenonlythefollowing,whatcanwesay?
ex:actor
rdf:typerdf:Property;
rdfs:domainex:TVSeries;
rdfs:rangeex:Person.
ex:Dimitrisrdf:typeex:Person.
ex:VoxxedDaysAthensrdf:typeex:Conference.
ex:VoxxedDaysAthensex:actorex:Dimitris.
Somethingis
notright…
ex:VoxxedDaysAthens
isnotaex:TVSeries

36. toInferortoValidate?
Givenonlythefollowing,whatcanwesay?
ex:actorrdf:typerdf:Property;
rdfs:domainex:TVSeries;
rdfs:rangeex:Person.
ex:Dimitrisrdf:typeex:Person.
dbpedia:Friendsrdf:typeex:TVSeries.
dbpedia:Friendsex:actorex:Dimitris.
Appearslegit

37. Schemastored&queriedasData
ex:TVSeries
rdf:typerdfs:Class;
rdfs:subClassOfex:CreativeWork.
ex:BookSeries
rdf:typerdfs:Class;
rdfs:subClassOfex:CreativeWork.
ex:CreativeWork
rdf:typerdfs:Class.
dbpedia:Friendsrdf:typeex:TVSeries.
dbpedia:The_Officerdf:typeex:TVSeries.
dbpedia:Narniardf:typeex:BookSeries.
SELECT?sWHERE{
?srdfs:subClassOfex:CreativeWork.
}
ex:TVSeries,ex:BookSeries
SELECT?sWHERE{
?srdf:typeex:TVSeries.
}
dbpedia:Friends,dbpedia:The_Office

38. Schemastored&queriedasData
Navigatesthe
classhierarchy
SELECT?sWHERE{
?srdf:type/rdfs:subClassOf*
ex:CreativeWork}
dbpedia:Friends,
dbpedia:The_Office,
dbpedia:Narnia
Hierarchycanbe
extendedwithout
breakingthequery
ex:TVSeries
rdf:typerdfs:Class;
rdfs:subClassOfex:CreativeWork.
ex:BookSeries
rdf:typerdfs:Class;
rdfs:subClassOfex:CreativeWork.
ex:CreativeWork
rdf:typerdfs:Class.
dbpedia:Friendsrdf:typeex:TVSeries.
dbpedia:The_Officerdf:typeex:TVSeries.
dbpedia:Narniardf:typeex:BookSeries.

39. ManyAvailablefreeSchemas
Manyexistingfree(asinbeer)ontologies(orschemas)
modeldifferentdomains
>Generalpurpose(DBpedia,schema.org)
>Geographical(geo)
>Provenance(prov-o)
>Taxonomies/Classification(SKOSfamily)
>Organizations(org)
>Find~600entriesathttp://lov.okfn.org

40. ReusingAvailable(Free)schemas
Getpartofyourdatamodelingforfree
>Groupsofpeoplealreadyworkedonmodelingthedomain
>Spenttimedefininghumanandmachine-readablesemantics
Facilitatesdataintegrationeasier
>Datapublishedwithcommonschemas
>Dataeasiertobeconsumed

41. MappingtoAvailable(Free)schemas
Mapwhennotreusing
>integratedatainalooselycoupledway
ex:TVSeriesowl:equivalentClassschema:TVSeries.
ex:actorowl:equivalentPropertyschema:actor.

42. RDF&Semantics-takeawaypoints
It’sallaboutClasses&Properties
Human-readablesemantics
>Commonlyacceptedmodellingconventions
Machine-readablesemantics
>Canbeusedforinferenceand/orvalidation
>Canbequeriedtogetherwithdata
Reusing[orlinkingto]commonontologies/schemas
>Integratingdatawithlessvariety
>Networkeffect(themorepeople/datauseitthebetter)
>Developingreusableapplicationsagainstschemas

43. LinkedData&RDF

44. Givenonlythis,cancanwedo/say?
<https://voxxeddays.com/athens><https://schema.org/attendee><http://kontokostas.com>.
schema:Event(domain)schema:Person(range)Apersonattendingtheevent.
HTTP GET
<https://voxxeddays.com/athens>
rdf:typeschema:Event;
schema:name“VoxxedAthens”;
schema:startDate“2018-06-01”;
schema:endDate“2018-06-02”;
schema:inLanguage“English”
schema:description“...”
HTTPGET
<http://kontokostas.com>
rdf:typeschema:Person;
schema:givenName“Dimitris”;
schema:familyName“Kontokostas”;
schema:birthPlacedbpedia:Greece;
schema:jobTitle“DataEngineer”;
schema:worksFor<https://geophy.com>.
HTTPGET

45. Followyournosepattern
<http://kontokostas.com><https://schema.org/birthPlace><http://dbpedia.org/resource/Greece>.
schema:Person(domain)schema:Place(range)Theplacewherethepersonwasborn.
HTTP GET
<http://kontokostas.com>
rdf:typeschema:Person;
schema:givenName“Dimitris”;
schema:familyName“Kontokostas”;
schema:birthPlacedbpedia:Greece;
schema:jobTitle“DataEngineer”;
schema:worksFor<https://geophy.com>.
HTTPGET
<http://dbpedia.org/resource/Greece>
rdf:typeschema:Place,dbpedia:Country;
dbo:capitaldbpedia:Athens;
dbo:currencydbpedia:Euro;
geo:lat“39.0”^^xsd:float;
geo:long“22.0”^^xsd:float.
HTTPGET

46. RDF&LinkedData
Thingsrepresentedwithhttp(s)-basedURIs
canbeself-published
HTTPGETrequestsonThingsreturnRDFTriples
whereitisasubject(oranobject)
Decentralizedstorage/access/semantics
(*)a.k.a.theWebofData,seeTEDtalkfromTimBernersLee(CreatorofWWW)

47. RDF&LinkedData(ontheweb)
kontokostas.com
example.com
voxxeddays.com/At
hens
DBpedia
WebofDataDBpedia
DBpedia
DBpedia
Wikipedia
AsRDF

48. RDF&LinkedData(ontheenterprise)
WebofData
RDF
DBx
LDx
RDF
DBy
LDy
RDF
DBz
LDz
LDw

49. LinkedOpenDataCloud
Diagramfrom2014
v2018istoobig
1.184datasets
15.993links
https://lod-cloud.net/

50. Reusingavailabledatasets/identifiers
Justlikereusingschemas,referencing/reusingexternal
identifiers,facilitates:
Dataintegration
e.g.dbpedia:FriendsrepresentstheFriendsTVseries,notsomefriends
>usedbpedia:Friendsdirectly
>linkit:ex:tv_series_123owl:sameAsdbpedia:Friends
Dataenrichment
e.g.dbpedia:Friendsmayhaveadditionalinformationabouttheseriesthanour
database,andwecaneasily(http)getit

51. RDF&LinkedData-takeawaypoints
DecentralisationofDataManagement
Self-documentedschemas&data
Scaleyour[local]graphstothe[Enterprise]Web
Bigpoolofstableidentifiers(i.e.DBpedia)

52. Payasyougodataintegration
Youcangetbenefitwithloweffort
>RDFviewsontopofRDBMSwithR2RML(mappings,SPARQL2SQLtranslation)
>ConvertXML/JSON/CSV/…toRDFwithRML
Themoretimeyouinvestthebettertheresults
>Schemadevelopement,mapping&linking
>Semi-automaticallylinkdiscoverywithtoolslikeLimes&Silk
e.g.:ex:tv_series_123owl:sameAsdbpedia:Friends
RDFdoesnotneedtobeyourmasterdataset

53. WhousesRDF
https://github.com/json-ld/json-ld.org/wiki/Users-of-JSON-LD

54. 28%ofTLD(or39%ofHTMLpages)
>3.7MMicrodata
>2.7MJSON-LD
>1.2MRDFa
Intotal9billionThings&38billionRDFtriples
Fullreportathttp://webdatacommons.org/structureddata/#results-2017-1
Structureddataontheweb(Nov2017)

55. RDF@Google

56. RDFOntology
>Lessstrict/formal
>PromotesJSON-LD
Funded&maintained
byallSearchengines
drivesmanygoogle
products...

58. Schema.org&&Google&&Search
https://developers.google.com/search/docs/guides/search-features

59. Googleis...
UsingtheRDFgraphmodeltointegratediverse
datafromwebpages&emails
ByusingtheconceptofLinkedData
Andthisisallempoweredbya
commonontology(orschema)

60. RDF@GeoPhy

61. GeoPhyprovidesvalue,
risk,&qualitymetrics
foreverybuildingintheworld

62. RDF@GeoPhy
Wecollect&integratealotofdata
>onproperties,onitssurroundings,andonthemarketconditions
MasterdatasetonRealEstate(akaKnowledgeGraph)
>drivingourMachineLearning/DeepLearningmodels
Challenges...
>Wehavethousandsofsources,
>Sourcesareupdatedatarbitraryintervals
>WegetourdatainCSV,inthegooddays
And,ofcourse…
wearenotGoogle
tomakepeople
writeRDFforus:-)

63. GeophyDataManagementPlatform
CSVPDF
GeoPhy
Ontologies
Transform
ToRDF
Validate
Identify&
Deduplicate
Conflict
resolution
DataFusion
Data
Wrangling&
Extraction
Annotation&
Provenance
Modeling
Mapping
CoreDB
Provenance
(value-level)
DataIndexing
DataIngestion
DataEnrichment
Dependency
DetectionGeo
EnrichmentTriggerML/DL
API

64. Andtheclosingslide...
PeoplethinkRDFisapainbecauseitiscomplicated.
Thetruthisevenworse.
RDFispainfullysimplistic,butitallowsyoutoworkwith
real-worlddataandproblemsthatarehorriblycomplicated.
WhileyoucanavoidRDF,itishardertoavoidcomplicated
dataandcomplicatedcomputerproblems.
DanBrickley,Schema.organdGoogle
LibbyMiller,BBC

65. Thankyouforyourattention
Questions?
ManythankstoSander,MattandthewholeGeoPhyEng.Teamfortheirfeedback