Programming with LOD
Upcoming SlideShare
Loading in...5
×

Like this? Share it with your network

Share

Programming with LOD

  • 1,886 views
Uploaded on

Tutorial at IASLOD2012

Tutorial at IASLOD2012

More in: Technology , Travel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
No Downloads

Views

Total Views
1,886
On Slideshare
1,886
From Embeds
0
Number of Embeds
0

Actions

Shares
Downloads
40
Comments
0
Likes
4

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. Programming with LOD Fumihiro Kato Research Organization of Information and Systems Transdisciplinary Research Integration Center National Institute of Informatics 1
  • 2. Linked Data Principles1. Use URIs as names for things2. Use HTTP URIs so that people can look up those names.3. When someone looks up a URI, provide useful information, using the standards (RDF, SPARQL)4. Include links to other URIs, so that they can discover more things. 2
  • 3. http://dbpedia.org/resource/South_Korea3
  • 4. 1. request /resource/South_Korea 2. return an HTML document for /resource/South_KoreaWeb Browser http://dbpedia.org 4
  • 5. 1. request /resource/South_Korea 2. return an RDF document for /resource/South_KoreaLOD Agent http://dbpedia.org @prefix dbpedia-owl: <http://dbpedia.org/ontology/> . @prefix dbpedia: <http://dbpedia.org/resource/> . dbpedia:Seoul dbpedia-owl:country dbpedia:South_Korea . dbpedia:JnC dbpedia-owl:hometown dbpedia:South_Korea . @prefix dbpprop: <http://dbpedia.org/property/> . dbpedia:JnC dbpprop:origin dbpedia:South_Korea . dbpedia:Lee_Jae_Jin dbpedia-owl:hometown dbpedia:South_Korea ; dbpprop:origin dbpedia:South_Korea . dbpedia:Kang_Sung_Hoon dbpedia-owl:hometown dbpedia:South_Korea ; dbpprop:origin dbpedia:South_Korea . dbpedia:Northern_Asia-Pacific_Division_of_Seventh-day_Adventists dbpedia-owl:location dbpedia:South_Korea ; dbpedia-owl:regionServed dbpedia:South_Korea ; dbpprop:location dbpedia:South_Korea . dbpedia:Korean_Amateur_Radio_League dbpedia-owl:regionServed dbpedia:South_Korea ; dbpprop:regionServed dbpedia:South_Korea . dbpedia:Asia-Pacific_Association_for_International_Education dbpprop:headquarters dbpedia:South_Korea ; dbpedia-owl:headquarter dbpedia:South_Korea . dbpedia:The_Korean_Chemical_Society dbpedia-owl:location dbpedia:South_Korea ; dbpprop:location dbpedia:South_Korea . dbpedia:Korean_e-Sports_Association dbpedia-owl:regionServed dbpedia:South_Korea . dbpedia:Miss_Asia_Pacific_World dbpprop:headquarters dbpedia:South_Korea ; dbpedia-owl:headquarter dbpedia:South_Korea . dbpedia:Daegu_Stadium dbpedia-owl:location dbpedia:South_Korea . dbpedia:Chuncheon_Civic_Stadium dbpedia-owl:location dbpedia:South_Korea . 5
  • 6. RDF Libraries• Java: Jena, Sesame • Python: rdflib• C: Redland • Perl: PerlRDF• C#: dotNetRDF • JavaScript: rdfQuery,• Ruby: RDF.rb rdfstore-js• PHP: ARC2, • .... GraphiteSee http://www.w3.org/2001/sw/wiki/Tools 6
  • 7. RDF.rb• http://rdf.rubyforge.org• http://github.com/ruby-rdf• $ gem install linkeddata equivalent-xml 7
  • 8. Reading RDF 1 #!/usr/bin/env ruby 2 # -*- coding: utf-8 -*- 3 4 require linkeddata 5 6 RDF::Reader.open("http://dbpedia.org/resource/South_Korea") do |reader| 7 reader.each do |statement| 8 puts statement.inspect 9 end10 end 8
  • 9. Reading RDF 1 #!/usr/bin/env ruby 2 # -*- coding: utf-8 -*- 3 4 require linkeddata 5 6 RDF::Reader.open("http://dbpedia.org/resource/South_Korea") do |reader| 7 reader.each do |statement| 8 puts statement.inspect 9 end 10 end.......#<RDF::Statement:0x3fd362ca7338(<http://dbpedia.org/resource/South_Korea> <http://dbpedia.org/ontology/PopulatedPlace/areaTotal> 100209.22997701018 .)>#<RDF::Statement:0x3fd362ca7338(<http://dbpedia.org/resource/South_Korea> <http://dbpedia.org/ontology/PopulatedPlace/areaTotal> 100209.22997701018 .)>#<RDF::Statement:0x3fd362cabd70(<http://dbpedia.org/resource/South_Korea> <http://dbpedia.org/ontology/PopulatedPlace/areaTotal> 100210.0 .)>#<RDF::Statement:0x3fd362cabd70(<http://dbpedia.org/resource/South_Korea> <http://dbpedia.org/ontology/PopulatedPlace/areaTotal> 100210.0 .)>#<RDF::Statement:0x3fd3623772cc(<http://dbpedia.org/resource/South_Korea> <http://dbpedia.org/ontology/PopulatedPlace/populationDensity> 491.0 .)>#<RDF::Statement:0x3fd3623772cc(<http://dbpedia.org/resource/South_Korea> <http://dbpedia.org/ontology/PopulatedPlace/populationDensity> 491.0 .)>#<RDF::Statement:0x3fd362c797bc(<http://dbpedia.org/resource/South_Korea> <http://dbpedia.org/ontology/PopulatedPlace/populationDensity> 490.73584350744875 .)>#<RDF::Statement:0x3fd362c797bc(<http://dbpedia.org/resource/South_Korea> <http://dbpedia.org/ontology/PopulatedPlace/populationDensity> 490.73584350744875 .)>#<RDF::Statement:0x3fd362c59c28(<http://dbpedia.org/resource/South_Korea> <http://dbpedia.org/ontology/abstract> Corea del Sud, of icialment la República de Corea) és un país de lÀsia que ocupa laregió meridional de la península de Corea. Al nord, limita amb Corea del Nord, al qual estava unit fins el 1945. A loest, creuant la mar Groga, es troba la República Popular de Xina, i a lest, creuant lEstret de Corea, es troba el Japó. La capital i la ciutat més gran nés Seül, làrea metropolitana de la qual és una de les tres més poblades del món. La llengua oficial de Corea del Sud és el coreà, i és,actualment, la desena llengua més parlada del món. El nom del país, en coreà, significa "La Gran Nació del Poble Coreà", hom es refereix al país sovint com la "Terra de la Calma del Matí". llegendari DangunKanggeom. Levidència arqueològica suggereix que els coreans han habitat la península des del Paleolític Baix. Després de la unificació dels Tres Regnes Coreans sota el comandament de Silla el 668 dC, Coreava romandre com a nació unificada sota les dinasties de Goryeao i Joseaon fins la caiguda de lImperi Coreà el 1910. Després de la divisió i de lestabliment de la república moderna de Corea del Sud, el país hatingut que reconstruir-se després del període dexplotació japonesa i de la Guerra de Corea. No obstant això, lestàndard de vida, així com leconomia, van créixer de manera sorprenent des de la dècada de1980. El 1987, després de diverses manifestacions nacionals, el govern autoritari es va col·lapsar, i des de llavors, shi han celebrat eleccions democràtiques cada cinc anys. Avui dia, Corea del Sur és un delsEstats pioners de la democràcia a Àsia; els ciutadans coreans gaudeixen dun dels nivells més alts de...... 9
  • 10. Accessing statements 1 #!/usr/bin/env ruby 2 # -*- coding: utf-8 -*- 3 4 require linkeddata 5 6 RDF::Reader.open("http://dbpedia.org/resource/Korea") do |reader| 7 reader.each do |statement| 8 puts "s: #{statement.subject}, p: #{statement.predicate}, o: #{statement.object}" 9 end10 end 10
  • 11. RDF Serialization• N-Triples • RDFa• RDF/XML • JSON-LD• Turtle (N3) • RDF/JSON• TriX • ... 11
  • 12. Reading text/n3 1 #!/usr/bin/env ruby 2 # -*- coding: utf-8 -*- 3 4 require linkeddata 5 6 RDF::Reader.open("http://dbpedia.org/data/South_Korea.n3", :format => "n3") do |reader| 7 reader.each do |statement| 8 puts statement.inspect! 9 end10 end 12
  • 13. Writing RDF 1 #!/usr/bin/env ruby 2 # -*- coding: utf-8 -*- 3 4 require linkeddata 5 6 RDF::Writer.open("writing.ttl") do |writer| 7 subject = RDF::URI("http://dbpedia.org/resource/South_Korea") 8 predicate = RDF::RDFS.label 9 object = RDF::Literal("韓国", :language => :ja)10 writer << RDF::Steatement.new(subject, predicate, object)11 end 13
  • 14. Loading and Merging RDF 1 #!/usr/bin/env ruby 2 # -*- coding: utf-8 -*- 3 4 require linkeddata 5 6 graph = RDF::Graph.new 7 graph.load("http://dbpedia.org/resource/South_Korea") 8 puts graph.size 9 graph.load("http://dbpedia.org/resource/Daejeon")10 puts graph.size 14
  • 15. Querying RDF 1 #!/usr/bin/env ruby 2 # -*- coding: utf-8 -*- 3 4 require linkeddata 5 6 graph = RDF::Graph.new 7 graph.load("http://dbpedia.org/resource/Daejeon") 8 graph.query([RDF::URI("http://dbpedia.org/resource/Daejeon"), 9 RDF::URI("http://dbpedia.org/ontology/country"),10 nil]).each do |statement|11 puts statement.object12 end dbpedia:South_Korea dbpedia-owl:country rdf:type dbpedia-owl:City dbpedia:Daejeon geo:lat rdfs:label 36.350101 Daejeon 15
  • 16. Defining a vocabulary 1 #!/usr/bin/env ruby 2 # -*- coding: utf-8 -*- 3 4 require linkeddata 5 6 DBP = RDF::Vocabulary.new("http://dbpedia.org/resource/") 7 DBP_ONT = RDF::Vocabulary.new("http://dbpedia.org/ontology/") 8 9 graph = RDF::Graph.new10 graph.load(DBP.Daejeon)11 graph.query([DBP.Daejeon, DBP_ONT.country, nil]).each do |statement|12 puts statement.object13 end dbpedia-owl:South_Korea dbpedia-owl:country rdf:type dbpedia-owl:City dbpedia:Daejeon geo:lat rdfs:label 36.350101 Daejeon 16
  • 17. Query SQL SPARQL DB RDB RDF StoreModel Relational Model RDF Model 17
  • 18. SPARQL• SPARQL Protocol and RDF Query Language • W3C Standards • SQL-like syntax • Graph Patterns • Result Formats: XML, JSON, (CSV, TSV)• $ gem install sparql 18
  • 19. Using SPARQL in RDF.rb 1 #!/usr/bin/env ruby 2 # -*- coding: utf-8 -*- 3 4 require linkeddata 5 6 graph = RDF::Graph.new 7 graph.load("http://dbpedia.org/resource/Daejeon") 8 9 sse = SPARQL.parse("""SELECT ?o10 WHERE {11 <http://dbpedia.org/resource/Daejeon> <http://dbpedia.org/ontology/country> ?o .12 }""")1314 sse.execute(graph).each do |solution|15 puts solution[:o]16 end 19
  • 20. Querying a SPARQL Endpoint 1 #!/usr/bin/env ruby 2 # -*- coding: utf-8 -*- 3 4 require linkeddata 5 6 client = SPARQL::Client.new("http://dbpedia.org/sparql") 7 8 query = """SELECT ?o 9 WHERE {10 <http://dbpedia.org/resource/Daejeon> <http://dbpedia.org/ontology/country> ?o .11 }"""1213 client.query(query).each do |solution|14 puts solution[:o]15 end 20
  • 21. dbpedia-owl:Country rdf:type rdfs:labeldbpedia-owl:City dbpedia-owl:capital dbpedia:South_Korea South Korea rdf:type dbpedia-owl:coutnry dbpedia-owl:Company dbpedia:Seoul dbpedia-owl:location rdf:type dbpedia-owl:location rdfs:label dbpedia:Samsung_Electronics dbpedia-owl:owner owl:sameAs dbpprop:foundingYear dbpedia-owl:Lee_Kun-hee 1969-01-01 Seoul rdfs:label freebase:Seoul foaf:homepage 삼성전자 rdfs:label http://global.samsungtomorrow.com/ Samsung Electronics 21
  • 22. PREFIX dbpedia: <http://dbpedia.org/resource/>PREFIX dbpedia-owl: <http://dbpedia.org/ontology/>PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#>SELECT ?location ?labelWHERE { dbpedia:Samsung_Electronics dbpedia-owl:location ?location . ?location rdfs:label ?label .} dbpedia-owl:Country rdf:type rdfs:labeldbpedia-owl:City dbpedia-owl:capital dbpedia:South_Korea South Korea rdf:type dbpedia-owl:coutnry dbpedia-owl:Company dbpedia:Seoul dbpedia-owl:location rdf:type dbpedia-owl:location rdfs:label dbpedia:Samsung_Electronics dbpedia-owl:owner owl:sameAs dbpprop:foundingYear dbpedia-owl:Lee_Kun-hee 1969-01-01 Seoul rdfs:label freebase:Seoul foaf:homepage 삼성전자 rdfs:label http://global.samsungtomorrow.com/ Samsung Electronics 22
  • 23. 23
  • 24. 24
  • 25. Querying a SPARQL Endpoint 1 #!/usr/bin/env ruby 2 # -*- coding: utf-8 -*- 3 4 require linkeddata 5 6 client = SPARQL::Client.new("http://dbpedia.org/sparql") 7 8 query = """PREFIX dbpedia: <http://dbpedia.org/resource/> 9 PREFIX dbpedia-owl: <http://dbpedia.org/ontology/>10 PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#>1112 SELECT ?location ?label13 WHERE {14 dbpedia:Samsung_Electronics dbpedia-owl:location ?location .15 ?location rdfs:label ?label .16 }"""1718 client.query(query).each do |solution|19 puts solution.inspect20 end 25
  • 26. PREFIX dbpedia: <http://dbpedia.org/resource/>PREFIX dbpedia-owl: <http://dbpedia.org/ontology/>PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#>SELECT ?location ?labelWHERE { dbpedia:Samsung_Electronics dbpedia-owl:location ?location . ?location a dbpedia-owl:City ; rdfs:label ?label .} dbpedia-owl:Country rdf:type rdfs:labeldbpedia-owl:City dbpedia-owl:capital dbpedia:South_Korea South Korea rdf:type dbpedia-owl:coutnry dbpedia-owl:Company dbpedia:Seoul dbpedia-owl:location rdf:type dbpedia-owl:location rdfs:label dbpedia:Samsung_Electronics dbpedia-owl:owner owl:sameAs dbpprop:foundingYear dbpedia-owl:Lee_Kun-hee 1969-01-01 Seoul rdfs:label freebase:Seoul foaf:homepage 삼성전자 rdfs:label http://global.samsungtomorrow.com/ Samsung Electronics 26
  • 27. 27
  • 28. PREFIX dbpedia: <http://dbpedia.org/resource/>PREFIX dbpedia-owl: <http://dbpedia.org/ontology/>PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#>SELECT ?location ?labelWHERE { dbpedia:Samsung_Electronics dbpedia-owl:location ?location . ?location a dbpedia-owl:City ; rdfs:label ?label . FILTER(lang(?label) = ‘ko’) dbpedia-owl:Country} rdf:type rdfs:labeldbpedia-owl:City dbpedia-owl:capital dbpedia:South_Korea South Korea rdf:type dbpedia-owl:coutnry dbpedia-owl:Company dbpedia:Seoul dbpedia-owl:location rdf:type dbpedia-owl:location rdfs:label dbpedia:Samsung_Electronics dbpedia-owl:owner owl:sameAs dbpprop:foundingYear dbpedia-owl:Lee_Kun-hee 1969-01-01 Seoul rdfs:label freebase:Seoul foaf:homepage 삼성전자 rdfs:label http://global.samsungtomorrow.com/ Samsung Electronics 28
  • 29. 29
  • 30. SPARQL 1.0• CRUD • SELECT • ASK • CONSTRUCT • DESCRIBE 30
  • 31. SELECTクエリパターン変数の値を取得SELECT *WHERE { ?s ?p ?o .}LIMIT 100OFFSET 100 31
  • 32. ASKPREFIX dbpedia: <http://dbpedia.org/resource/>PREFIX dbpedia-owl: <http://dbpedia.org/ontology/>ASK WHERE { dbpedia:Daejeon ?p ?o .} 32
  • 33. CONSTRUCTPREFIX foaf: <http://xmlns.com/foaf/0.1/>PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#>CONSTRUCT { ?s rdfs:label ?name .}WHERE { ?s foaf:name ?name .} 33
  • 34. DESCRIBEDESCRIBE <http://dbpedia.org/resource/Tokyo> 34
  • 35. • http://www.w3.org/wiki/SparqlEndpoints http://www.w3.org/wiki/SparqlEndpoints http://www.w3.org/wiki/SparqlEndpoints 35
  • 36. • http://www.w3.org/wiki/SparqlEndpoints http://www.w3.org/wiki/SparqlEndpoints 36 http://labs.mondeca.com/sparqlEndpointsStatus/
  • 37. http://thedatahub.org/group/lodcloud http://ckan.net/package?res_format=api%2Fsparql37
  • 38. 38 http://lod.ac/apps/lodac_location/
  • 39. PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#>PREFIX geo: <http://www.w3.org/2003/01/geo/wgs84_pos#>SELECT DISTINCT ?link ?title ?lat ?longWHERE { ?link rdfs:label ?title; geo:lat ?lat; geo:long ?long . FILTER ( ?lat > #{sw_lat} && ?lat < #{ne_lat} && ?long > #{sw_long} && ?long < #{ne_long} && lang(?title) = "ja" )} 39
  • 40. Query SQL SPARQL DB RDB RDF StoreModel Relational Model RDF Model 40
  • 41. RDF Store• Triplestore / Quadstore / Graph DB• SPARQL Endpoint• Options Bigdata® • RDFS/OWL Inference • Full text indexing • Clustering• See http://en.wikipedia.org/wiki/ Triplestore 41
  • 42. http://virtuoso.openlinksw.com/dataspace/dav/wiki/Main/42
  • 43. Virtuoso Open Source• http://virtuoso.openlinksw.com/dataspace/dav/wiki/Main/• install binary• install from source $ tar xvzf virtuoso-opensource-6.1.6.tar.gz $ cd virtuoso-opensource-6.1.6 $ ./configure $ make $ make install $ /usr/local/virtuoso-opensource/bin/virtuoso-t +configfile /usr/local/virtuoso- opensource/var/lib/virtuoso/db/virtuoso.ini 43
  • 44. http://localhost:8890/conductor/44
  • 45. http://localhost:8890/conductor/45
  • 46. 46
  • 47. 47
  • 48. http://localhost:8890/sparql48
  • 49. Storage Adapter• Memory• Sesame• Virtuoso• DataObjects (SQLite3, PostgreSQL)• RedStore• 4Store• MongoDB• Cassandra• Talis• SPARQL 49
  • 50. Virtuoso Adapter $ gem install rdf-virtuoso 1 #!/usr/bin/env ruby 2 # -*- coding: utf-8 -*- 3 4 require linkeddata 5 require rdf/virtuoso 6 7 REPO_URI = "http://localhost:8890" 8 USER = "dba" 9 PASS = "dba" # this should be changed10 DBP = RDF::Vocabulary.new("http://dbpedia.org/resource/")1112 repo = RDF::Virtuoso::Repository.new(REPO_URI, :username => USER,13 :password => PASS, :auth_method => digest)14 query = RDF::Virtuoso::Query.select.where([:s, :p, DBP.South_Korea])15 result = repo.select(query)1617 result.each do |solution|18 puts solution.inspect19 end 50
  • 51. Summary• how to consume LOD • RDF • SPARQL 51
  • 52. twitter: fumi1 fumi@nii.ac.jp52