NOSQL also means RDF stores: an Android case study
Upcoming SlideShare
Loading in...5
×

Like this? Share it with your network

Share
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
No Downloads

Views

Total Views
2,353
On Slideshare
2,352
From Embeds
1
Number of Embeds
1

Actions

Shares
Downloads
96
Comments
0
Likes
11

Embeds 1

http://linkeddata.uriburner.com 1

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

























































































































































Transcript

  • 1. NoSQL also means RDF stores: an Android case study Fabrizio Giudici, Senior Java Architect Tidalwave s.a.s - fabrizio.giudici@tidalwave.it
  • 2. Agenda NoSQL also means RDF stores: an Android case study 2
  • 3. Agenda • Meaning and reasons for NoSQL NoSQL also means RDF stores: an Android case study 2
  • 4. Agenda • Meaning and reasons for NoSQL • Introducing the blueBill (Mobile) project NoSQL also means RDF stores: an Android case study 2
  • 5. Agenda • Meaning and reasons for NoSQL • Introducing the blueBill (Mobile) project • What’s a RDF store? Why a RDF store? NoSQL also means RDF stores: an Android case study 2
  • 6. Agenda • Meaning and reasons for NoSQL • Introducing the blueBill (Mobile) project • What’s a RDF store? Why a RDF store? • RDF open solutions in Java: OpenSesame, Elmo NoSQL also means RDF stores: an Android case study 2
  • 7. Agenda • Meaning and reasons for NoSQL • Introducing the blueBill (Mobile) project • What’s a RDF store? Why a RDF store? • RDF open solutions in Java: OpenSesame, Elmo • RDF design issues NoSQL also means RDF stores: an Android case study 2
  • 8. Agenda • Meaning and reasons for NoSQL • Introducing the blueBill (Mobile) project • What’s a RDF store? Why a RDF store? • RDF open solutions in Java: OpenSesame, Elmo • RDF design issues • Running an RDF store on Android NoSQL also means RDF stores: an Android case study 2
  • 9. Agenda • Meaning and reasons for NoSQL • Introducing the blueBill (Mobile) project • What’s a RDF store? Why a RDF store? • RDF open solutions in Java: OpenSesame, Elmo • RDF design issues • Running an RDF store on Android • Conclusions NoSQL also means RDF stores: an Android case study 2
  • 10. About the speaker NoSQL also means RDF stores: an Android case study 3
  • 11. About the speaker • Senior Software Architect, Mentor, Technical Writer • Fourteen years of Java Experience (JSE, JEE, JME, etc...) • Sun Consultant since 1998, Oracle consultant since 2010 • Author of a number of open source projects • Speaker at JavaOne, Devoxx, Jazoon, JAX Italia and other events • Member of the NetBeans Dream Team • Co-leader of JUG Milano • Java.Net blogger at http://www.java.net/blogs/fabriziogiudici NoSQL also means RDF stores: an Android case study 3
  • 12. NoSQL? What? NoSQL also means RDF stores: an Android case study 4
  • 13. NoSQL? What? • SQL: query language, also refers to “relational model” (RDBM) NoSQL also means RDF stores: an Android case study 4
  • 14. NoSQL? What? • SQL: query language, also refers to “relational model” (RDBM) • SQL/RDBM are the de-facto standards for ACID persistence NoSQL also means RDF stores: an Android case study 4
  • 15. NoSQL? What? • SQL: query language, also refers to “relational model” (RDBM) • SQL/RDBM are the de-facto standards for ACID persistence • Tired of that? NoSQL also means RDF stores: an Android case study 4
  • 16. NoSQL? What? • SQL: query language, also refers to “relational model” (RDBM) • SQL/RDBM are the de-facto standards for ACID persistence • Tired of that? • OO impedance NoSQL also means RDF stores: an Android case study 4
  • 17. NoSQL? What? • SQL: query language, also refers to “relational model” (RDBM) • SQL/RDBM are the de-facto standards for ACID persistence • Tired of that? • OO impedance • Scalability, elastic scalability (performance) NoSQL also means RDF stores: an Android case study 4
  • 18. NoSQL? What? • SQL: query language, also refers to “relational model” (RDBM) • SQL/RDBM are the de-facto standards for ACID persistence • Tired of that? • OO impedance • Scalability, elastic scalability (performance) • Flexibility (data schema) NoSQL also means RDF stores: an Android case study 4
  • 19. NoSQL? What? • SQL: query language, also refers to “relational model” (RDBM) • SQL/RDBM are the de-facto standards for ACID persistence • Tired of that? • OO impedance • Scalability, elastic scalability (performance) • Flexibility (data schema) • Distributed data model, distributed computation NoSQL also means RDF stores: an Android case study 4
  • 20. NoSQL? What? • SQL: query language, also refers to “relational model” (RDBM) • SQL/RDBM are the de-facto standards for ACID persistence • Tired of that? • OO impedance • Scalability, elastic scalability (performance) • Flexibility (data schema) • Distributed data model, distributed computation • etc NoSQL also means RDF stores: an Android case study 4
  • 21. NoSQL NoSQL also means RDF stores: an Android case study 5
  • 22. NoSQL • Interpretations NoSQL also means RDF stores: an Android case study 5
  • 23. NoSQL • Interpretations • #1: No SQL (at all) NoSQL also means RDF stores: an Android case study 5
  • 24. NoSQL • Interpretations • #1: No SQL (at all) • #2: Not Only SQL (maybe on top of it) NoSQL also means RDF stores: an Android case study 5
  • 25. NoSQL • Interpretations • #1: No SQL (at all) • #2: Not Only SQL (maybe on top of it) • ACID or not? NoSQL also means RDF stores: an Android case study 5
  • 26. NoSQL Five advantages of NoSQL • Interpretations scaling 1. Elastic 2. Big data • #1: No SQL Goodbye DBAs (see you later?) 3. (at all) 4. Economics • #2: Not Only SQL data models top of 5. Flexible (maybe on it) • ACID orFive challenges of NoSQL not? 1. Maturity 2. Support 3. Analytics and business intelligence 4. Administration 5. Expertise Guy Harrison NoSQL also means RDF stores: an Android case study 5
  • 27. NoSQL • Interpretations • #1: No SQL (at all) • #2: Not Only SQL (maybe on top of it) • ACID or not? NoSQL also means RDF stores: an Android case study 5
  • 28. NoSQL • Interpretations • #1: No SQL (at all) • #2: Not Only SQL (maybe on top of it) • ACID or not? • Examples NoSQL also means RDF stores: an Android case study 5
  • 29. NoSQL • Interpretations • #1: No SQL (at all) • #2: Not Only SQL (maybe on top of it) • ACID or not? • Examples • BigTable (Google), Cassandra (Facebook), Dynamo (Amazon), ... NoSQL also means RDF stores: an Android case study 5
  • 30. NoSQL • Interpretations • #1: No SQL (at all) • #2: Not Only SQL (maybe on top of it) • ACID or not? • Examples • BigTable (Google), Cassandra (Facebook), Dynamo (Amazon), ... • RDF stores NoSQL also means RDF stores: an Android case study 5
  • 31. NoSQL • Interpretations • #1: No SQL (at all) • #2: Not Only SQL (maybe on top of it) • ACID or not? • Examples • BigTable (Google), Cassandra (Facebook), Dynamo (Amazon), ... • RDF stores • This talk is only about the latter topic NoSQL also means RDF stores: an Android case study 5
  • 32. Introducing blueBill (Mobile) NoSQL also means RDF stores: an Android case study 6
  • 33. Introducing blueBill (Mobile) • Take field notes of bird observations NoSQL also means RDF stores: an Android case study 6
  • 34. Introducing blueBill (Mobile) • Take field notes of bird observations • Learn about birds NoSQL also means RDF stores: an Android case study 6
  • 35. Introducing blueBill (Mobile) • Take field notes of bird observations • Learn about birds • Aggregating many different knowledge sources NoSQL also means RDF stores: an Android case study 6
  • 36. Introducing blueBill (Mobile) • Take field notes of bird observations • Learn about birds • Aggregating many different knowledge sources • Collaborative, open, community-oriented NoSQL also means RDF stores: an Android case study 6
  • 37. Introducing blueBill (Mobile) • Take field notes of bird observations • Learn about birds • Aggregating many different knowledge sources • Collaborative, open, community-oriented • Subtle requirements NoSQL also means RDF stores: an Android case study 6
  • 38. Introducing blueBill (Mobile) • Take field notes of bird observations • Learn about birds • Aggregating many different knowledge sources • Collaborative, open, community-oriented • Subtle requirements • Complex data structure NoSQL also means RDF stores: an Android case study 6
  • 39. Introducing blueBill (Mobile) • Take field notes of bird observations • Learn about birds • Aggregating many different knowledge sources • Collaborative, open, community-oriented • Subtle requirements • Complex data structure • Most advanced mobile version currently running on Android NoSQL also means RDF stores: an Android case study 6
  • 40. Introducing blueBill (Mobile) • Take field notes of bird observations • Learn about birds • Aggregating many different knowledge sources • Collaborative, open, community-oriented • Subtle requirements • Complex data structure • Most advanced mobile version currently running on Android NoSQL also means RDF stores: an Android case study 6
  • 41. Complex data structure NoSQL also means RDF stores: an Android case study 7
  • 42. Complex data structure • Checklists / Taxonomies NoSQL also means RDF stores: an Android case study 7
  • 43. Complex data structure • Checklists / Taxonomies • List of species in a given geographic area NoSQL also means RDF stores: an Android case study 7
  • 44. Complex data structure • Checklists / Taxonomies • List of species in a given geographic area • Compiled and vetted by ornithologist and birdwatcher associations NoSQL also means RDF stores: an Android case study 7
  • 45. Complex data structure • Checklists / Taxonomies • List of species in a given geographic area • Compiled and vetted by ornithologist and birdwatcher associations • Define official common names (with variants) NoSQL also means RDF stores: an Android case study 7
  • 46. Complex data structure • Checklists / Taxonomies • List of species in a given geographic area • Compiled and vetted by ornithologist and birdwatcher associations • Define official common names (with variants) • Continuously evolving NoSQL also means RDF stores: an Android case study 7
  • 47. Complex data structure • Checklists / Taxonomies • List of species in a given geographic area • Compiled and vetted by ornithologist and birdwatcher associations • Define official common names (with variants) • Continuously evolving • Taxonomies NoSQL also means RDF stores: an Android case study 7
  • 48. Complex data structure • Checklists / Taxonomies • List of species in a given geographic area • Compiled and vetted by ornithologist and birdwatcher associations • Define official common names (with variants) • Continuously evolving • Taxonomies • Scientific names, hierarchical NoSQL also means RDF stores: an Android case study 7
  • 49. Complex data structure • Checklists / Taxonomies • List of species in a given geographic area • Compiled and vetted by ornithologist and birdwatcher associations • Define official common names (with variants) • Continuously evolving • Taxonomies • Scientific names, hierarchical • Not universal consensus NoSQL also means RDF stores: an Android case study 7
  • 50. Complex data structure • Checklists / Taxonomies • List of species in a given geographic area • Compiled and vetted by ornithologist and birdwatcher associations • Define official common names (with variants) • Continuously evolving • Taxonomies • Scientific names, hierarchical • Not universal consensus • Split & lump NoSQL also means RDF stores: an Android case study 7
  • 51. What’s RDF? NoSQL also means RDF stores: an Android case study 8
  • 52. What’s RDF? • RDF is the core of the Semantic Web family of technologies NoSQL also means RDF stores: an Android case study 8
  • 53. What’s RDF? • RDF is the core of the Semantic Web family of technologies • Semantic Web: give semantic features to the web NoSQL also means RDF stores: an Android case study 8
  • 54. What’s RDF? • RDF is the core of the Semantic Web family of technologies • Semantic Web: give semantic features to the web • Semantic technologies != Semantic Web NoSQL also means RDF stores: an Android case study 8
  • 55. What’s RDF? • RDF is the core of the Semantic Web family of technologies • Semantic Web: give semantic features to the web • Semantic technologies != Semantic Web • They can be used even when the web doesn’t matter NoSQL also means RDF stores: an Android case study 8
  • 56. What’s RDF? • RDF is the core of the Semantic Web family of technologies • Semantic Web: give semantic features to the web • Semantic technologies != Semantic Web • They can be used even when the web doesn’t matter • Just a better way to model data NoSQL also means RDF stores: an Android case study 8
  • 57. What’s RDF? • RDF is the core of the Semantic Web family of technologies • Semantic Web: give semantic features to the web • Semantic technologies != Semantic Web • They can be used even when the web doesn’t matter • Just a better way to model data • RDF can be useful to you even though you don’t endorse the Semantic Web model NoSQL also means RDF stores: an Android case study 8
  • 58. RDF principles NoSQL also means RDF stores: an Android case study 9
  • 59. RDF principles • Data are modeled with a single “schema” NoSQL also means RDF stores: an Android case study 9
  • 60. RDF principles • Data are modeled with a single “schema” • subject - predicate - object (statement, triple) NoSQL also means RDF stores: an Android case study 9
  • 61. RDF principles • Data are modeled with a single “schema” • subject - predicate - object (statement, triple) • Formal ways to represent them NoSQL also means RDF stores: an Android case study 9
  • 62. RDF principles • Data are modeled with a single “schema” • subject - predicate - object (statement, triple) • Formal ways to represent them • URNs, URIs, URLs NoSQL also means RDF stores: an Android case study 9
  • 63. RDF principles • Data are modeled with a single “schema” • subject - predicate - object (statement, triple) • Formal ways to represent them • URNs, URIs, URLs • namespaces NoSQL also means RDF stores: an Android case study 9
  • 64. RDF principles • Data are modeled with a single “schema” • subject - predicate - object (statement, triple) • Formal ways to represent them • URNs, URIs, URLs • namespaces • RDF != XML NoSQL also means RDF stores: an Android case study 9
  • 65. RDF principles • Data are modeled with a single “schema” • subject - predicate - object (statement, triple) • Formal ways to represent them • URNs, URIs, URLs • namespaces • RDF != XML • RDF is a model NoSQL also means RDF stores: an Android case study 9
  • 66. RDF principles • Data are modeled with a single “schema” • subject - predicate - object (statement, triple) • Formal ways to represent them • URNs, URIs, URLs • namespaces • RDF != XML • RDF is a model • RDF-XML is just one of the many ways to serialize RDF data NoSQL also means RDF stores: an Android case study 9
  • 67. Ontologies • Standard “vocabularies” of terms with agreed-on semantics • Identified by their own namespaces • Can be mixed together • Semantic equivalence • Declaring that two concepts from two different ontologies are the same thing • Can be enforced locally NoSQL also means RDF stores: an Android case study 10
  • 68. RDF examples • Show some examples NoSQL also means RDF stores: an Android case study 11
  • 69. What’s a RDF store? NoSQL also means RDF stores: an Android case study 12
  • 70. What’s a RDF store? • A software facility for managing data modeled in RDF NoSQL also means RDF stores: an Android case study 12
  • 71. What’s a RDF store? • A software facility for managing data modeled in RDF • Persistent storage NoSQL also means RDF stores: an Android case study 12
  • 72. What’s a RDF store? • A software facility for managing data modeled in RDF • Persistent storage • CRUD operations NoSQL also means RDF stores: an Android case study 12
  • 73. What’s a RDF store? • A software facility for managing data modeled in RDF • Persistent storage • CRUD operations • SPARQL and other query languages NoSQL also means RDF stores: an Android case study 12
  • 74. What’s a RDF store? • A software facility for managing data modeled in RDF • Persistent storage • CRUD operations • SPARQL and other query languages • Indipendent implementation, or sitting upon a SQL database NoSQL also means RDF stores: an Android case study 12
  • 75. What’s a RDF store? • A software facility for managing data modeled in RDF • Persistent storage • CRUD operations • SPARQL and other query languages • Indipendent implementation, or sitting upon a SQL database • Transactions, in the standard way NoSQL also means RDF stores: an Android case study 12
  • 76. What’s a RDF store? • A software facility for managing data modeled in RDF • Persistent storage • CRUD operations • SPARQL and other query languages • Indipendent implementation, or sitting upon a SQL database • Transactions, in the standard way • Typically suitable for distributed, web-oriented queries NoSQL also means RDF stores: an Android case study 12
  • 77. What’s a RDF store? • A software facility for managing data modeled in RDF • Persistent storage • CRUD operations • SPARQL and other query languages • Indipendent implementation, or sitting upon a SQL database • Transactions, in the standard way • Typically suitable for distributed, web-oriented queries • Not necessarily relevant for one’s purposes NoSQL also means RDF stores: an Android case study 12
  • 78. Why a RDF store? NoSQL also means RDF stores: an Android case study 13
  • 79. Why a RDF store? • Most important point: no schema constraints NoSQL also means RDF stores: an Android case study 13
  • 80. Why a RDF store? • Most important point: no schema constraints • “It’s easier to add a row in a table than changing a schema” NoSQL also means RDF stores: an Android case study 13
  • 81. Why a RDF store? • Most important point: no schema constraints • “It’s easier to add a row in a table than changing a schema” • Perfectly suitable for data models that: NoSQL also means RDF stores: an Android case study 13
  • 82. Why a RDF store? • Most important point: no schema constraints • “It’s easier to add a row in a table than changing a schema” • Perfectly suitable for data models that: • have got a “fuzzy” schema, initially or forever NoSQL also means RDF stores: an Android case study 13
  • 83. Why a RDF store? • Most important point: no schema constraints • “It’s easier to add a row in a table than changing a schema” • Perfectly suitable for data models that: • have got a “fuzzy” schema, initially or forever • are subject to a (strong) iterative development NoSQL also means RDF stores: an Android case study 13
  • 84. Why a RDF store? • Most important point: no schema constraints • “It’s easier to add a row in a table than changing a schema” • Perfectly suitable for data models that: • have got a “fuzzy” schema, initially or forever • are subject to a (strong) iterative development • need to be integrated with other data models NoSQL also means RDF stores: an Android case study 13
  • 85. Why a RDF store? • Most important point: no schema constraints • “It’s easier to add a row in a table than changing a schema” • Perfectly suitable for data models that: • have got a “fuzzy” schema, initially or forever • are subject to a (strong) iterative development • need to be integrated with other data models • The storage model is the same thing as the serialization model NoSQL also means RDF stores: an Android case study 13
  • 86. Designing RDF NoSQL also means RDF stores: an Android case study 14
  • 87. Designing RDF • RDF has got its own design rules NoSQL also means RDF stores: an Android case study 14
  • 88. Designing RDF • RDF has got its own design rules • Validation rules, best practices, etc... NoSQL also means RDF stores: an Android case study 14
  • 89. Designing RDF • RDF has got its own design rules • Validation rules, best practices, etc... • Not necessarily easy, but may be incremental NoSQL also means RDF stores: an Android case study 14
  • 90. Designing RDF • RDF has got its own design rules • Validation rules, best practices, etc... • Not necessarily easy, but may be incremental • Not relevant if used internally, apart your own maintenance burden NoSQL also means RDF stores: an Android case study 14
  • 91. Designing RDF • RDF has got its own design rules • Validation rules, best practices, etc... • Not necessarily easy, but may be incremental • Not relevant if used internally, apart your own maintenance burden • Very relevant if used in a distributed data scenario NoSQL also means RDF stores: an Android case study 14
  • 92. Designing RDF • RDF has got its own design rules • Validation rules, best practices, etc... • Not necessarily easy, but may be incremental • Not relevant if used internally, apart your own maintenance burden • Very relevant if used in a distributed data scenario • The big challenge is to properly reuse others’ ontologies NoSQL also means RDF stores: an Android case study 14
  • 93. Designing RDF • RDF has got its own design rules • Validation rules, best practices, etc... • Not necessarily easy, but may be incremental • Not relevant if used internally, apart your own maintenance burden • Very relevant if used in a distributed data scenario • The big challenge is to properly reuse others’ ontologies • RDFs, OWL, Dublin Core, Darwin Core NoSQL also means RDF stores: an Android case study 14
  • 94. Designing RDF • RDF has got its own design rules • Validation rules, best practices, etc... • Not necessarily easy, but may be incremental • Not relevant if used internally, apart your own maintenance burden • Very relevant if used in a distributed data scenario • The big challenge is to properly reuse others’ ontologies • RDFs, OWL, Dublin Core, Darwin Core • Which ones are established and which ones not? NoSQL also means RDF stores: an Android case study 14
  • 95. blueBill Specs NoSQL also means RDF stores: an Android case study 15
  • 96. blueBill Specs • Integrate data from different sources NoSQL also means RDF stores: an Android case study 15
  • 97. blueBill Specs • Integrate data from different sources • Wikipedia, media data bases, other information sources NoSQL also means RDF stores: an Android case study 15
  • 98. blueBill Specs • Integrate data from different sources • Wikipedia, media data bases, other information sources • A common model core can be defined NoSQL also means RDF stores: an Android case study 15
  • 99. blueBill Specs • Integrate data from different sources • Wikipedia, media data bases, other information sources • A common model core can be defined • But many things are “discovered” by working on them NoSQL also means RDF stores: an Android case study 15
  • 100. blueBill Specs • Integrate data from different sources • Wikipedia, media data bases, other information sources • A common model core can be defined • But many things are “discovered” by working on them • Iterative, incremental approach NoSQL also means RDF stores: an Android case study 15
  • 101. blueBill Specs • Integrate data from different sources • Wikipedia, media data bases, other information sources • A common model core can be defined • But many things are “discovered” by working on them • Iterative, incremental approach • Manipulating a SQL schema is cumbersome NoSQL also means RDF stores: an Android case study 15
  • 102. OpenSesame, Elmo NoSQL also means RDF stores: an Android case study 16
  • 103. OpenSesame, Elmo • Two major players in the Java ecosystem NoSQL also means RDF stores: an Android case study 16
  • 104. OpenSesame, Elmo • Two major players in the Java ecosystem • Jena (donated by HP) NoSQL also means RDF stores: an Android case study 16
  • 105. OpenSesame, Elmo • Two major players in the Java ecosystem • Jena (donated by HP) • OpenSesame (by Aduna) NoSQL also means RDF stores: an Android case study 16
  • 106. OpenSesame, Elmo • Two major players in the Java ecosystem • Jena (donated by HP) • OpenSesame (by Aduna) • Both FLOSS NoSQL also means RDF stores: an Android case study 16
  • 107. OpenSesame, Elmo • Two major players in the Java ecosystem • Jena (donated by HP) • OpenSesame (by Aduna) • Both FLOSS • My preference for OpenSesame, seems more modular NoSQL also means RDF stores: an Android case study 16
  • 108. OpenSesame, Elmo • Two major players in the Java ecosystem • Jena (donated by HP) • OpenSesame (by Aduna) • Both FLOSS • My preference for OpenSesame, seems more modular • Fundamental feature for Android development... NoSQL also means RDF stores: an Android case study 16
  • 109. OpenSesame • RDF store implementation in 100% Java • Provides store persistence, query support • Small store can be manipulated in memory (Graph) • Regular stores can be memory-based or persistent (Repository) • Can be backed by flat file or SQL database NoSQL also means RDF stores: an Android case study 17
  • 110. Elmo NoSQL also means RDF stores: an Android case study 18
  • 111. Elmo • Annotation-based persistence for OpenSesame NoSQL also means RDF stores: an Android case study 18
  • 112. Elmo • Annotation-based persistence for OpenSesame • Similar (and partially compatible) with JPA NoSQL also means RDF stores: an Android case study 18
  • 113. Repository repository = new SailRepository(new MemoryStore()); ElmoModule elmoModule = new ElmoModule(); Elmo SesameManagerFactory smf = new SesameManagerFactory(elmoModule, repository); ElmoManager em = smf.createElmoManager(); // similar to EntityManager, thread-bound //////////////////////////////// RepositoryConnection conn = repository.getConnection(); ValueFactory valueFactory = conn.getValueFactory(); RepositoryResult<Statement> statements = conn.getStatements(null, valueFactory.createURI(...), null, false); while (statements.hasNext()) { Statement statement = statements.next(); Object subject = statement.getSubject(); Object object = statement.getObject(); • // ... } Annotation-based persistence for OpenSesame conn.close(); • Similar (and partially compatible) with JPA //////////////////////////////// Elmo annotations @rdf("http://http://rs.tdwg.org/dwc/2009-12-07/terms/Taxon) public class Taxon { @rdf("http://http://rs.tdwg.org/dwc/2009-12-07/terms/scientificNameID") private String scientificNameID; @rdf("http://http://rs.tdwg.org/dwc/2009-12-07/terms/taxonRank") private String rank; // getters and setters } NoSQL also means RDF stores: an Android case study 18
  • 114. Running OpenSesame on Android NoSQL also means RDF stores: an Android case study 19
  • 115. Running OpenSesame on Android • Q: Is Android Java? Java™? JSE? A JDK? Something else? NoSQL also means RDF stores: an Android case study 19
  • 116. Running OpenSesame on Android • Q: Is Android Java? Java™? JSE? A JDK? Something else? • Not interested (here) in legal aspects: developer’s point of view NoSQL also means RDF stores: an Android case study 19
  • 117. Running OpenSesame on Android • Q: Is Android Java? Java™? JSE? A JDK? Something else? • Not interested (here) in legal aspects: developer’s point of view • A: It can run most of Java bytecode, with differences NoSQL also means RDF stores: an Android case study 19
  • 118. Running OpenSesame on Android • Q: Is Android Java? Java™? JSE? A JDK? Something else? • Not interested (here) in legal aspects: developer’s point of view • A: It can run most of Java bytecode, with differences • Obvious missing parts: UI (not a problem for RDF) NoSQL also means RDF stores: an Android case study 19
  • 119. Running OpenSesame on Android • Q: Is Android Java? Java™? JSE? A JDK? Something else? • Not interested (here) in legal aspects: developer’s point of view • A: It can run most of Java bytecode, with differences • Obvious missing parts: UI (not a problem for RDF) • Not-so-obvious differences: XML, JDBC (depending also on version) NoSQL also means RDF stores: an Android case study 19
  • 120. Running OpenSesame on Android • Q: Is Android Java? Java™? JSE? A JDK? Something else? • Not interested (here) in legal aspects: developer’s point of view • A: It can run most of Java bytecode, with differences • Obvious missing parts: UI (not a problem for RDF) • Not-so-obvious differences: XML, JDBC (depending also on version) • Much, much better than JME, but not plain JSE NoSQL also means RDF stores: an Android case study 19
  • 121. blueBill Android issues NoSQL also means RDF stores: an Android case study 20
  • 122. blueBill Android issues • Targeting Android 1.5 (still 10% of appliances) NoSQL also means RDF stores: an Android case study 20
  • 123. blueBill Android issues • Targeting Android 1.5 (still 10% of appliances) • Some missing XML APIs NoSQL also means RDF stores: an Android case study 20
  • 124. blueBill Android issues • Targeting Android 1.5 (still 10% of appliances) • Some missing XML APIs • e.g. Duration, XMLGregorianCalendar, QName NoSQL also means RDF stores: an Android case study 20
  • 125. blueBill Android issues • Targeting Android 1.5 (still 10% of appliances) • Some missing XML APIs • e.g. Duration, XMLGregorianCalendar, QName • Reflection works, but Google warns it’s slow NoSQL also means RDF stores: an Android case study 20
  • 126. blueBill Android issues • Targeting Android 1.5 (still 10% of appliances) • Some missing XML APIs • e.g. Duration, XMLGregorianCalendar, QName • Reflection works, but Google warns it’s slow • Things might have changed with 2.2 NoSQL also means RDF stores: an Android case study 20
  • 127. blueBill Android issues • Targeting Android 1.5 (still 10% of appliances) • Some missing XML APIs • e.g. Duration, XMLGregorianCalendar, QName • Reflection works, but Google warns it’s slow • Things might have changed with 2.2 • OpenSesame doesn’t support sqlLite out-of-the-box NoSQL also means RDF stores: an Android case study 20
  • 128. blueBill Android issues • Targeting Android 1.5 (still 10% of appliances) • Some missing XML APIs • e.g. Duration, XMLGregorianCalendar, QName • Reflection works, but Google warns it’s slow • Things might have changed with 2.2 • OpenSesame doesn’t support sqlLite out-of-the-box • Performance (?) NoSQL also means RDF stores: an Android case study 20
  • 129. Fixing XML NoSQL also means RDF stores: an Android case study 21
  • 130. Fixing XML • Android apps can’t bundle java.*, javax.* packages NoSQL also means RDF stores: an Android case study 21
  • 131. Fixing XML • Android apps can’t bundle java.*, javax.* packages • Obvious security reasons NoSQL also means RDF stores: an Android case study 21
  • 132. Fixing XML • Android apps can’t bundle java.*, javax.* packages • Obvious security reasons • Solution: Maven Shade plugin NoSQL also means RDF stores: an Android case study 21
  • 133. Fixing XML • Android apps can’t bundle java.*, javax.* packages • Obvious security reasons • Solution: Maven Shade plugin • Missing classes from Harmony into it.tidalwave.android.xml.* NoSQL also means RDF stores: an Android case study 21
  • 134. Fixing XML • Android apps can’t bundle java.*, javax.* packages • Obvious security reasons • Solution: Maven Shade plugin • Missing classes from Harmony into it.tidalwave.android.xml.* • Shade plugin renames references in existing bytecode NoSQL also means RDF stores: an Android case study 21
  • 135. Fixing XML • Android apps can’t bundle java.*, javax.* packages • Obvious security reasons • Solution: Maven Shade plugin • Missing classes from Harmony into it.tidalwave.android.xml.* • Shade plugin renames references in existing bytecode • Similar solution would work with Ant NoSQL also means RDF stores: an Android case study 21
  • 136. Fixing XML <groupId>it.tidalwave.bluebill</groupId> <artifactId>it-tidalwave-android-org-openrdf-sesame-model</artifactId> <version>2.2.3.8-SNAPSHOT</version> <packaging>jar</packaging> <name>Android Patch: OpenSesame Model</name> <dependencies> <dependency> <groupId>org.openrdf.sesame</groupId> <artifactId>sesame-model</artifactId> </dependency> <dependency> <groupId>it.tidalwave.bluebill</groupId> <artifactId>it-tidalwave-android-javax-xml-datatype</artifactId> </dependency> • Android apps can’t bundle java.*, javax.* packages </dependencies> <build> <plugins> • Obvious security reasons <plugin> <groupId>org.apache.maven.plugins</groupId> <artifactId>maven-shade-plugin</artifactId> <executions> • Solution: Maven Shade plugin <execution> <phase>package</phase> <goals> <goal>shade</goal> • </goals> Missing classes from Harmony into it.tidalwave.android.xml.* <configuration> .... <relocations> • <relocation> Shade plugin renames references in existing bytecode <pattern>javax.xml.namespace.QName</pattern> <shadedPattern>it.tidalwave.android.javax.xml.namespace.QName</shadedPattern> </relocation> • <relocation> Similar solution would work with Ant <pattern>javax.xml.datatype</pattern> <shadedPattern>it.tidalwave.android.javax.xml.datatype</shadedPattern> </relocation> </relocations> </configuration> </execution> </executions> </plugin> </plugins> </build> NoSQL also means RDF stores: an Android case study 21
  • 137. Hand-written marshalling NoSQL also means RDF stores: an Android case study 22
  • 138. Hand-written marshalling • Replaces Elmo reflected-based support NoSQL also means RDF stores: an Android case study 22
  • 139. Hand-written marshalling • Replaces Elmo reflected-based support • Write a custom *Marshaller and *Unmarshaller for each entity NoSQL also means RDF stores: an Android case study 22
  • 140. Hand-written marshalling • Replaces Elmo reflected-based support • Write a custom *Marshaller and *Unmarshaller for each entity • Might go away NoSQL also means RDF stores: an Android case study 22
  • 141. Hand-written marshalling • Replaces Elmo reflected-based support • Write a custom *Marshaller and *Unmarshaller for each entity • Might go away • If Android 2.2 proves to support fast reflection NoSQL also means RDF stores: an Android case study 22
  • 142. Hand-written marshalling • Replaces Elmo reflected-based support • Write a custom *Marshaller and *Unmarshaller for each entity • Might go away • If Android 2.2 proves to support fast reflection • When 2.2 becomes the target Android version NoSQL also means RDF stores: an Android case study 22
  • 143. Hand-written marshalling public class SimpleObservationMarshaller extends AsStatementMarshaller { @Override @Nonnull public void marshal (final @Nonnull As entity, final @Nonnull Context context) { super.marshal(entity, context); final Observation observation = (Observation)entity; context.addStatement(observation, Vocabulary.RDF_TYPE, Vocabulary.OBSERVATION); context.addStatement(observation, Vocabulary.DC_DATE, observation.getDate()); context.addStatement(observation, Vocabulary.MADE_AT, observation.getLocation()); • Replaces Elmo reflected-based support context.addStatement(observation, Vocabulary.DC_PUBLISHER, observation.getSource()); for (final Observer observer : observation.findObservers().results()) • { Write a custom *Marshaller and *Unmarshaller for } context.addStatement(observation, Vocabulary.DC_CREATOR, observer); each entityObservationItem observationItem : observation.findObservationItems().results()) for (final { • context.addStatement(observation, Vocabulary.SKOS_NARROWER, observationItem); Might go away } try • If Android 2.2 proves to support fast reflection { context.addStatement(observation.as(Media.class), Vocabulary.FOAF_TOPIC, observation); } • When 2.2 becomes the target Android version catch (AsException e) { // ok, no attachment } } } NoSQL also means RDF stores: an Android case study 22
  • 144. Hand-written marshalling public class SimpleObservationUnmarshaller extends StatementUnmarshallerSupport public class SimpleObservationMarshaller extends AsStatementMarshaller { {@Override @Nonnull public As unmarshal (final @Nonnull List<Statement> statements, final @Nonnull Context context) @Override @Nonnull { public SimpleObservationSet observationSet = (SimpleObservationSet)context.peek(); final void marshal (final @Nonnull As entity, final @Nonnull Context context) { final Date date = (Date)getLiteral(statements, Vocabulary.DC_DATE); super.marshal(entity, context); final Location location = (Location)findEntity(context, statements, Vocabulary.MADE_AT); final Source source = (Source)findEntity(context, statements, Vocabulary.DC_PUBLISHER); final Observation observation = (Observation)entity; final Set<Observer> observers = new HashSet<Observer>(); context.addStatement(observation, Vocabulary.RDF_TYPE, Vocabulary.OBSERVATION); final Set<ObservationItem> observationItems = new HashSet<ObservationItem>(); context.addStatement(observation, Vocabulary.DC_DATE, observation.getDate()); final List<Object> extras = new ArrayList<Object>(); extras.add(new DefaultIdentifiable(Converter.valueToId(statements.get(0).getSubject()))); context.addStatement(observation, Vocabulary.MADE_AT, observation.getLocation()); • Replaces Elmo reflected-based support observation.getSource()); context.addStatement(observation, Vocabulary.DC_PUBLISHER, final String displayName = (String)getLiteral(statements, Vocabulary.RDFS_LABEL); if for (final!= null) (displayName Observer observer : observation.findObservers().results()) • { { Write a custom *Marshaller and *Unmarshaller for } } extras.add(new DefaultDisplayable(displayName, "SimpleObservation")); context.addStatement(observation, Vocabulary.DC_CREATOR, observer); each entityObservationItem observationItem : observation.findObservationItems().results()) for (final Statement statement : context.findStatements(Vocabulary.FOAF_TOPIC, for (final { Converter.valueToId(statements.get(0).getSubject()))) { extras.add(context.find(statement.getSubject())); • context.addStatement(observation, Vocabulary.SKOS_NARROWER, observationItem); Might go away } } final SimpleObservation observation = new SimpleObservation(observationSet, date, location, observers, observationItems, source, extras); try context.push(observation); • If Android 2.2 proves to support fast reflection { for (final Statement statement : findStatementsWithPredicate(statements, Vocabulary.SKOS_NARROWER)) { context.addStatement(observation.as(Media.class), Vocabulary.FOAF_TOPIC, observation); } observationItems.add((SimpleObservationItem)context.find(statement.getObject())); • When 2.2 becomes the target Android version { }catch (AsException e) for (final Statement statement : findStatementsWithPredicate(statements, Vocabulary.DC_CREATOR)) // ok, no attachment { } observers.add((SimpleObserver)context.find(statement.getObject())); }} } context.pop(); return observation; } } NoSQL also means RDF stores: an Android case study 22
  • 145. Iterations NoSQL also means RDF stores: an Android case study 23
  • 146. Iterations • Iteration #1: no RDF, just ad hoc code (and a few data) NoSQL also means RDF stores: an Android case study 23
  • 147. Iterations • Iteration #1: no RDF, just ad hoc code (and a few data) • Current iteration #2: RDF, OpenSesame, custom marshalling NoSQL also means RDF stores: an Android case study 23
  • 148. Iterations • Iteration #1: no RDF, just ad hoc code (and a few data) • Current iteration #2: RDF, OpenSesame, custom marshalling • 4400+ birds, about 500.000 statements NoSQL also means RDF stores: an Android case study 23
  • 149. Iterations • Iteration #1: no RDF, just ad hoc code (and a few data) • Current iteration #2: RDF, OpenSesame, custom marshalling • 4400+ birds, about 500.000 statements • Statements grouped per-bird in N3 syntax, zipped NoSQL also means RDF stores: an Android case study 23
  • 150. Iterations • Iteration #1: no RDF, just ad hoc code (and a few data) • Current iteration #2: RDF, OpenSesame, custom marshalling • 4400+ birds, about 500.000 statements • Statements grouped per-bird in N3 syntax, zipped • Zip provides random access NoSQL also means RDF stores: an Android case study 23
  • 151. Iterations • Iteration #1: no RDF, just ad hoc code (and a few data) • Current iteration #2: RDF, OpenSesame, custom marshalling • 4400+ birds, about 500.000 statements • Statements grouped per-bird in N3 syntax, zipped • Zip provides random access • Future iteration #3: Using OpenSesame on a single flat file NoSQL also means RDF stores: an Android case study 23
  • 152. Iterations • Iteration #1: no RDF, just ad hoc code (and a few data) • Current iteration #2: RDF, OpenSesame, custom marshalling • 4400+ birds, about 500.000 statements • Statements grouped per-bird in N3 syntax, zipped • Zip provides random access • Future iteration #3: Using OpenSesame on a single flat file • Future iteration #4: Using OpenSesame on a single sqlLite NoSQL also means RDF stores: an Android case study 23
  • 153. Iterations • Iteration #1: no RDF, just ad hoc code (and a few data) • Current iteration #2: RDF, OpenSesame, custom marshalling • 4400+ birds, about 500.000 statements • Statements grouped per-bird in N3 syntax, zipped • Zip provides random access • Future iteration #3: Using OpenSesame on a single flat file • Future iteration #4: Using OpenSesame on a single sqlLite • Future iteration #5: Eventually use Elmo NoSQL also means RDF stores: an Android case study 23
  • 154. Challenges NoSQL also means RDF stores: an Android case study 24
  • 155. Challenges • How to import (once) 500.000 statements NoSQL also means RDF stores: an Android case study 24
  • 156. Challenges • How to import (once) 500.000 statements • Given that 50” are an acceptable only-once initialization time... NoSQL also means RDF stores: an Android case study 24
  • 157. Challenges • How to import (once) 500.000 statements • Given that 50” are an acceptable only-once initialization time... • ... it would require 0.1ms per statement (probably impossible!) NoSQL also means RDF stores: an Android case study 24
  • 158. Challenges • How to import (once) 500.000 statements • Given that 50” are an acceptable only-once initialization time... • ... it would require 0.1ms per statement (probably impossible!) • Writing sqlLite integration code for OpenSesame NoSQL also means RDF stores: an Android case study 24
  • 159. Challenges • How to import (once) 500.000 statements • Given that 50” are an acceptable only-once initialization time... • ... it would require 0.1ms per statement (probably impossible!) • Writing sqlLite integration code for OpenSesame • Query performance of a single database is unknown NoSQL also means RDF stores: an Android case study 24
  • 160. Conclusion NoSQL also means RDF stores: an Android case study 25
  • 161. Conclusion • RDF stores are a NoSQL approach for flexible data schemata NoSQL also means RDF stores: an Android case study 25
  • 162. Conclusion • RDF stores are a NoSQL approach for flexible data schemata • Not necessarily related to the full Semantic Web NoSQL also means RDF stores: an Android case study 25
  • 163. Conclusion • RDF stores are a NoSQL approach for flexible data schemata • Not necessarily related to the full Semantic Web • 100% Java, FLOSS solutions exist NoSQL also means RDF stores: an Android case study 25
  • 164. Conclusion • RDF stores are a NoSQL approach for flexible data schemata • Not necessarily related to the full Semantic Web • 100% Java, FLOSS solutions exist • Can be at least partially deployed with Android smartphones NoSQL also means RDF stores: an Android case study 25
  • 165. Conclusion • RDF stores are a NoSQL approach for flexible data schemata • Not necessarily related to the full Semantic Web • 100% Java, FLOSS solutions exist • Can be at least partially deployed with Android smartphones • Full deployment being researched NoSQL also means RDF stores: an Android case study 25
  • 166. Conclusion • RDF stores are a NoSQL approach for flexible data schemata • Not necessarily related to the full Semantic Web • 100% Java, FLOSS solutions exist • Can be at least partially deployed with Android smartphones • Full deployment being researched • Proved to be effective in production NoSQL also means RDF stores: an Android case study 25
  • 167. Conclusion • RDF stores are a NoSQL approach for flexible data schemata • Not necessarily related to the full Semantic Web • 100% Java, FLOSS solutions exist • Can be at least partially deployed with Android smartphones • Full deployment being researched • Proved to be effective in production • blueBill Mobile works NoSQL also means RDF stores: an Android case study 25
  • 168. Conclusion • RDF stores are a NoSQL approach for flexible data schemata • Not necessarily related to the full Semantic Web • 100% Java, FLOSS solutions exist • Can be at least partially deployed with Android smartphones • Full deployment being researched • Proved to be effective in production • blueBill Mobile works • The Semantic Web has actually got good points NoSQL also means RDF stores: an Android case study 25
  • 169. Conclusion • RDF stores are a NoSQL approach for flexible data schemata • Not necessarily related to the full Semantic Web • 100% Java, FLOSS solutions exist • Can be at least partially deployed with Android smartphones • Full deployment being researched • Proved to be effective in production • blueBill Mobile works • The Semantic Web has actually got good points • blueBill effort cut in half if people provided data in (good) RDF NoSQL also means RDF stores: an Android case study 25
  • 170. Question time NoSQL also means RDF stores: an Android case study 26
  • 171. Question time • Follow me • at my Java.Net blog • at DZone • Useful links • http://bluebill.tidalwave.it/mobile • http://www.openrdf.org NoSQL also means RDF stores: an Android case study 26