Design your application using Persistent Graphs and OrientDB

19,037 views
17,651 views

Published on

This

Published in: Technology
2 Comments
23 Likes
Statistics
Notes
No Downloads
Views
Total views
19,037
On SlideShare
0
From Embeds
0
Number of Embeds
358
Actions
Shares
0
Downloads
396
Comments
2
Likes
23
Embeds 0
No embeds

No notes for slide

Design your application using Persistent Graphs and OrientDB

  1. Design your application using Persistent Graphs and OrientDB Luca Garulli – Founder and CEO NuvolaBase Ltd May 2012 29 - 30 in Cologne, Germany(c) Luca Garulli Licensed under a Creative Commons Attribution-NoDerivs 3.0 Unported License Page 1 www.orientechnologies.com
  2. Usually NoSQL products are selectedbecause are fast and super scalable, but at what price?(c) Luca Garulli Licensed under a Creative Commons Attribution-NoDerivs 3.0 Unported License Page 2
  3. Can you really renounce to Transactions, an expressive Query language and all the features available for years by RDBMS?(c) Luca Garulli Licensed under a Creative Commons Attribution-NoDerivs 3.0 Unported License Page 3
  4. Can we have a fast and scalable NoSQL product with flexible schema, transactions, SQL, security and the support for complex types ?(c) Luca Garulli Licensed under a Creative Commons Attribution-NoDerivs 3.0 Unported License Page 4
  5. The answer is OrientDB, the document-graph NoSQL dbms(c) Luca Garulli Licensed under a Creative Commons Attribution-NoDerivs 3.0 Unported License Page 5
  6. The answer is OrientDB, the document-graph NoSQL dbms I never will change my RDBMS with anything!(c) Luca Garulli Licensed under a Creative Commons Attribution-NoDerivs 3.0 Unported License Page 6
  7. Mission?Reduce to the minimum the compromises on fitting the application domain to a persistent database supporting multiple models(c) Luca Garulli Licensed under a Creative Commons Attribution-NoDerivs 3.0 Unported License Page 7
  8. OrientDB = { flexibility of Document databases + complexity of the Graph model + Object Oriented concepts + fast Index + powerful SQL dialect }(c) Luca Garulli Licensed under a Creative Commons Attribution-NoDerivs 3.0 Unported License Page 8
  9. +13 years of research(c) Luca Garulli Licensed under a Creative Commons Attribution-NoDerivs 3.0 Unported License Page 9
  10. +3 years of design and development(c) Luca Garulli Licensed under a Creative Commons Attribution-NoDerivs 3.0 Unported License Page 10
  11. Relationships are direct links no Relational JOINS to connect multiple tables Load trees and graphs in few ms!(c) Luca Garulli Licensed under a Creative Commons Attribution-NoDerivs 3.0 Unported License Page 11
  12. Ø config download, unzip, run! cut & paste the db(c) Luca Garulli Licensed under a Creative Commons Attribution-NoDerivs 3.0 Unported License Page 12
  13. 150,000 records per second (flat records, no index, on commodity hw)(c) Luca Garulli Licensed under a Creative Commons Attribution-NoDerivs 3.0 Unported License Page 13
  14. Schema-less schema is not mandatory, relaxed model,collect heterogeneous documents all together(c) Luca Garulli Licensed under a Creative Commons Attribution-NoDerivs 3.0 Unported License Page 14
  15. Schema-fullschema with constraints on fields and validation rules Customer.age > 17 Customer.address not null Customer.surname is mandatory Customer.email matches b[A-Z0-9._%+-]+@[A-Z0-9.-]+.[A-Z]{2,4}b(c) Luca Garulli Licensed under a Creative Commons Attribution-NoDerivs 3.0 Unported License Page 15
  16. Schema-mixedschema with mandatory and optional fields + constraints the best of schema-less and schema-full modes(c) Luca Garulli Licensed under a Creative Commons Attribution-NoDerivs 3.0 Unported License Page 16
  17. ACID Transactions db.begin(); try{ // your code ... db.commit(); } catch( Exception e ) { db.rollback(); }(c) Luca Garulli Licensed under a Creative Commons Attribution-NoDerivs 3.0 Unported License Page 17
  18. Complex typesnative support for collections, maps (key/value) and embedded documents no more additional tables to handle them(c) Luca Garulli Licensed under a Creative Commons Attribution-NoDerivs 3.0 Unported License Page 18
  19. SQLselect * from employee where name like %Jay% and status=0(c) Luca Garulli Licensed under a Creative Commons Attribution-NoDerivs 3.0 Unported License Page 19
  20. Why reinvent yet another language when the 100% of developers already know SQL? OrientDB begins from SQL but improves it with new operators for graph manipulation(c) Luca Garulli Licensed under a Creative Commons Attribution-NoDerivs 3.0 Unported License Page 20
  21. For the most of the queries everyday a programmer needs SQL is simpler, more readable and compact then Scripting (Map/Reduce)(c) Luca Garulli Licensed under a Creative Commons Attribution-NoDerivs 3.0 Unported License Page 21
  22. SELECT SUM(price) as prices, SUM(cost) as costs, prices-costs, margin/price FROM BalanceVSfunction (key, values) { var price = 0.0, cost = 0.0, margin = 0.0, marginPercent = 0.0; for (var i = 0; i < values.length; i++) { price += values[i].price; cost += values[i].cost; } margin = price - cost; marginPercent = margin / price; return { price: price, cost: cost, margin: margin, marginPercent: marginPercent };} Luca Garulli(c) Licensed under a Creative Commons Attribution-NoDerivs 3.0 Unported License Page 22
  23. Asynchronous Query invoke callback when a record matches the condition doesnt collect the result set perfect for immediate results useful to compute aggregates(c) Luca Garulli Licensed under a Creative Commons Attribution-NoDerivs 3.0 Unported License Page 23
  24. Enhanced SQL SQL is not enough for collections, maps, trees and graphs need to enhance SQL syntax Easy syntax derived from JDO/JPA standards(c) Luca Garulli Licensed under a Creative Commons Attribution-NoDerivs 3.0 Unported License Page 24
  25. SQL & relationships select from Account where address.city.country.name = Italy select from Account where addresses contains (city.country.name = Italy)(c) Luca Garulli Licensed under a Creative Commons Attribution-NoDerivs 3.0 Unported License Page 25
  26. SQL & trees/graphsselect out[label=friend].in from V where name = Luca and surname = Garulliselect out[@class=knows].in from V where name = Jay and surname = Miner traverse friends from Profile where $depth < 7(c) Luca Garulli Licensed under a Creative Commons Attribution-NoDerivs 3.0 Unported License Page 26
  27. SQL sub queries select from ( traverse friends from Profile where $depth < 7 ) where home.city.name = ‘Cologne’(c) Luca Garulli Licensed under a Creative Commons Attribution-NoDerivs 3.0 Unported License Page 27
  28. SQL & strings select from Profile where name.toUpperCase() = LUCA select from City where country.name.substring(1,3).toUpperCase() = TAL select from Agenda where phones contains ( number.indexOf( +39 ) > -1 ) select from Agenda where email matches bA-Z0-9._%+-?+@A-Z0-9.-?+.A-Z?{2,4}b(c) Luca Garulli Licensed under a Creative Commons Attribution-NoDerivs 3.0 Unported License Page 28
  29. SQL & schema-less select from Profile where any() like %Jay% select from Stock where all() is not null(c) Luca Garulli Licensed under a Creative Commons Attribution-NoDerivs 3.0 Unported License Page 29
  30. SQL & collections select from Tree where children contains ( married = true ) select from Tree where children containsAll ( married = true ) select from User where roles containsKey shutdown select from Graph where edges.size() > 0(c) Luca Garulli Licensed under a Creative Commons Attribution-NoDerivs 3.0 Unported License Page 30
  31. runs Java everywhere is available JRE1.5+ ® robust engine(c) Luca Garulli Licensed under a Creative Commons Attribution-NoDerivs 3.0 Unported License Page 31
  32. Language bindings Java as native JRuby, PHP, C, Scala, .NET, Ruby, Clojure, Node.js, Python and Javascript(c) Luca Garulli Licensed under a Creative Commons Attribution-NoDerivs 3.0 Unported License Page 32
  33. Is your languagenot supported (yet)? Write an adapter using the C, Java or HTTP binding(c) Luca Garulli Licensed under a Creative Commons Attribution-NoDerivs 3.0 Unported License Page 33
  34. Binary protocol Fast compressed JSON over tcp/ip available for Java and soon C, C++ and Ruby(c) Luca Garulli Licensed under a Creative Commons Attribution-NoDerivs 3.0 Unported License Page 34
  35. HTTP RESTful firewall friendly use it from the web browser use it from the ESB (SOA)(c) Luca Garulli Licensed under a Creative Commons Attribution-NoDerivs 3.0 Unported License Page 35
  36. Native JSON ODocument = new ODocument().fromJSON( " { @rid = 26:10, @class = Developer, name : Luca, surname : Garulli, out : [ #10:33, #10:232 ] }“ );(c) Luca Garulli Licensed under a Creative Commons Attribution-NoDerivs 3.0 Unported License Page 36
  37. Import/Export uses JSON format online operations (dont stop the database) > export database demo(c) Luca Garulli Licensed under a Creative Commons Attribution-NoDerivs 3.0 Unported License Page 37
  38. MVRB-Tree index the best of B+Tree and RB-Tree fast on browsing, low insertion cost its a new algorithm(c) Luca Garulli Licensed under a Creative Commons Attribution-NoDerivs 3.0 Unported License Page 38
  39. Hooks similar to triggers catch events against records, database and transactions implement custom cascade deletion algorithm enforce constraints(c) Luca Garulli Licensed under a Creative Commons Attribution-NoDerivs 3.0 Unported License Page 39
  40. Security users and roles, encrypted passwords fine grain privileges (similar to what RDBMS offer)(c) Luca Garulli Licensed under a Creative Commons Attribution-NoDerivs 3.0 Unported License Page 40
  41. Cache You can avoid using 3°party caches like Memcached 2 Level caches: Level1: Database level, 1 per thread Level2: Storage level, 1 per JVM(c) Luca Garulli Licensed under a Creative Commons Attribution-NoDerivs 3.0 Unported License Page 41
  42. Inheritance OGraphVertex (V) OGraphEdge (E) Person Vehicle Works Resides Knows Address : Address brand : BRANDS since : Date since : Date till : Date Level : LEVELS Customer Provider totSold : float totBuyed : float(c) Luca Garulli Licensed under a Creative Commons Attribution-NoDerivs 3.0 Unported License Page 42
  43. Polymorphic SQL Query OgraphVertex (V) Person Vehicle Address : Address brand : BRANDS select from Person where city.name = Rome‘ Queries are polymorphics Customer Provider and subclasses of Person can be totSold : float totBuyed : float part of result set(c) Luca Garulli Licensed under a Creative Commons Attribution-NoDerivs 3.0 Unported License Page 43
  44. Fetch plans Choose what to fetch on query and vertexes/edges loadingVertexes/Edges not fetched will be lazy-loaded on request Optimizes network latency(c) Luca Garulli Licensed under a Creative Commons Attribution-NoDerivs 3.0 Unported License Page 44
  45. Fetch plans Load only the root vertex Vertex = *:1 Luca | | lives city +---------> Vertex ------------> Vertex | 10th street Italy | knows +--------->* [Vertex Vertex Vertex ] [ Marko John Nicholas](c) Luca Garulli Licensed under a Creative Commons Attribution-NoDerivs 3.0 Unported License Page 45
  46. Fetch plans Load root + address Vertex = *:1 lives:2 Luca | | lives city +---------> Vertex ------------> Vertex | 10th street Italy | knows +--------->* [Vertex Vertex Vertex ] [ Marko John Nicholas](c) Luca Garulli Licensed under a Creative Commons Attribution-NoDerivs 3.0 Unported License Page 46
  47. Fetch plans Load root + all known Vertex = *:1 knows:1 Luca | | lives city +---------> Vertex ------------> Vertex | 10th street Italy | knows +--------->* [Vertex Vertex Vertex ] [ Marko John Nicholas](c) Luca Garulli Licensed under a Creative Commons Attribution-NoDerivs 3.0 Unported License Page 47
  48. Fetch plans Load up 3rd level of depth Vertex = *:3 Luca | | lives city +---------> Vertex ------------> Vertex | 10th street Italy | knows +--------->* [Vertex Vertex Vertex ] [ Marko John Nicholas](c) Luca Garulli Licensed under a Creative Commons Attribution-NoDerivs 3.0 Unported License Page 48
  49. Graph Database model Built as wrapper on top of The Document DatabaseFew simple concepts: Vertex, Edge, Property and Index(c) Luca Garulli Licensed under a Creative Commons Attribution-NoDerivs 3.0 Unported License Page 49
  50. Why OrientDB is so special on handling Graphs? Can I model a graph on top of a RDBMS?(c) Luca Garulli Licensed under a Creative Commons Attribution-NoDerivs 3.0 Unported License Page 50
  51. A GraphDB has an “index-free adjacency” mechanism to cross the graph without any index lookup(c) Luca Garulli Licensed under a Creative Commons Attribution-NoDerivs 3.0 Unported License Page 51
  52. OrientDB doesn’t use JOIN but the “link” to traverse millions of elements per second In Blueprints benchmark, with a hot cache,traverses 29,6M of records in less than 5 seconds = 5,92M of nodes traversed per sec! (c) Luca Garulli Licensed under a Creative Commons Attribution-NoDerivs 3.0 Unported License Page 52
  53. 2 different API OGraphDatabase Native, damn fast, not the most beautiful API All APIs are compatible among them! So use the right one for the right case OrientGraph TinkerPop Blueprints, slowest but: common to other impls, Gremlin, SPARQL (via Sail)(c) Luca Garulli Licensed under a Creative Commons Attribution-NoDerivs 3.0 Unported License Page 53
  54. What to choose? OGraphDatabase Native, damn fast, not the most beautiful API OGraphDatabase if you need performance at any cost. Use it for massive insertion or low resources(c) Luca Garulli Licensed under a Creative Commons Attribution-NoDerivs 3.0 Unported License Page 54
  55. What to choose? OrientGraph TinkerPop Blueprints, slowest but: common to other impls, Gremlin, SPARQL (Sail) OrientGraph if you want to stay Portable at the cost of less speed and more memory used or to use Gremlin language, or as RDF store + SPARQL(c) Luca Garulli Licensed under a Creative Commons Attribution-NoDerivs 3.0 Unported License Page 55
  56. TinkerPop technologies sort of “standard” for GraphDB a lot of free open-source projects http://tinkerpop.com(c) Luca Garulli Licensed under a Creative Commons Attribution-NoDerivs 3.0 Unported License Page 56
  57. TinkerPop Blueprints basic API to interact with GraphDB implements transactional and indexable property graph model bidirectional edges(c) Luca Garulli Licensed under a Creative Commons Attribution-NoDerivs 3.0 Unported License Page 57
  58. GraphDB & Blueprints API OrientGraph graph = new OrientGraph("local:/tmp/db/graph”); Vertex actor = graph.addVertex(null); actor.setProperty("name", "Leonardo"); actor.setProperty("surname", "Di Caprio"); Vertex movie = graph.addVertex(null); movie.setProperty("name", "Inception"); Edge edge = graph.addEdge(null, actor, movie, "StarredIn"); graph.shutdown();(c) Luca Garulli Licensed under a Creative Commons Attribution-NoDerivs 3.0 Unported License Page 58
  59. TinkerPop scripting language easy to learn and understand Used for operations against graphs(c) Luca Garulli Licensed under a Creative Commons Attribution-NoDerivs 3.0 Unported License Page 59
  60. graph-example-1.xml(c) Luca Garulli Licensed under a Creative Commons Attribution-NoDerivs 3.0 Unported License Page 60 www.orientechnologies.com
  61. Load graph Run the console, open the database and load a graph in xml format marko:~/software/gremlin$ ./gremlin.sh ,,,/ (o o) -----oOOo-(_)-oOOo----- gremlin> $_g := orientdb:open(/tmp/graph/test) ==>orientgraph[/tmp/graph/test] gremlin> g:load(data/graph-example-1.xml) ==>true gremlin> $_g ==>orientgraph[/tmp/graph/test](c) Luca Garulli Licensed under a Creative Commons Attribution-NoDerivs 3.0 Unported License Page 61
  62. Search Displays outgoing edges of vertices with name equals to marko, then the name of inbound vertices gremlin> g:key-v(name,marko)/outE ==>e[6:0][5:2-knows->5:1] ==>e[6:1][5:2-knows->5:4] ==>e[6:4][5:2-created->5:0] gremlin> g:key-v(name,marko)/outE/inV/@name ==>vadas ==>josh ==>lop gremlin> g:close() ==>true(c) Luca Garulli Licensed under a Creative Commons Attribution-NoDerivs 3.0 Unported License Page 62
  63. High-Availability Multi-Master replication Servers can be heterogeneous with different replicated databases(c) Luca Garulli Licensed under a Creative Commons Attribution-NoDerivs 3.0 Unported License Page 63
  64. Where is the previous OrientDB Master/Slave architecture?(c) Luca Garulli Licensed under a Creative Commons Attribution-NoDerivs 3.0 Unported License Page 64
  65. Mas only ter/sla recy . Do no ve cle! t(c) Luca Garulli Licensed under a Creative Commons Attribution-NoDerivs 3.0 Unported License Page 65
  66. After first tests we decided to throw away the old Master-Slave architecture because it was against the OrientDB philosophy: (1) It didnt scale and (2) It was hard to configure(c) Luca Garulli Licensed under a Creative Commons Attribution-NoDerivs 3.0 Unported License Page 66
  67. Console ORIENT database v.0.9.23 www.orientechnologies.com Type help to display all the commands supported. > connect remote:localhost/demo admin admin Connecting to database [remote:localhost/demo] with user admin...OK > select from profile where nick.startsWith(L) ---+--------+--------------------+--------------------+--------------------+ #| REC ID |NICK |SEX |AGE | ---+--------+--------------------+--------------------+--------------------+ 0| 10:0|Lvca |male |34 1| 10:3|Leo |male |22 2| 10:7|Luisa |female |27 3 item(s) found. Query executed in 0.013 sec(s). > close Disconnecting from the database [demo]...OK > quit(c) Luca Garulli Licensed under a Creative Commons Attribution-NoDerivs 3.0 Unported License Page 67
  68. OrientDB Studio/View graph(c) Luca Garulli Licensed under a Creative Commons Attribution-NoDerivs 3.0 Unported License Page 68
  69. Multi-Model use case(c) Luca Garulli Licensed under a Creative Commons Attribution-NoDerivs 3.0 Unported License Page 69
  70. (c) Luca Garulli Licensed under a Creative Commons Attribution-NoDerivs 3.0 Unported License Page 70
  71. Always Free Open Source Apache 2 license free for any purposes, even commercials(c) Luca Garulli Licensed under a Creative Commons Attribution-NoDerivs 3.0 Unported License Page 71
  72. Do y o Conta u want to ct info be a @nuv partn olaba er? se.co Prof€$$ional m $€rvic€$ directly by NuvolaBase Ltd or partners support, training, consulting, mentoring(c) Luca Garulli Licensed under a Creative Commons Attribution-NoDerivs 3.0 Unported License Page 72
  73. OrientDB OrientDB for Java developers Master Development 8 hours 14 hours OrientDB OrientDB for SOA and the power of graphs 6 hours 6 hours OrientDB OrientPlanet for DBA for Web Developers 6 hours 6 hours(c) Luca Garulli Licensed under a Creative Commons Attribution-NoDerivs 3.0 Unported License Page 73
  74. Certification Program to be part of the network do courses share revenues for support work as consultant info@ Contact nuvo labas e.co m(c) Luca Garulli Licensed under a Creative Commons Attribution-NoDerivs 3.0 Unported License Page 74
  75. “OrientDB in Action” book by Manning Publications is coming: January 2013(c) Luca Garulli Licensed under a Creative Commons Attribution-NoDerivs 3.0 Unported License Page 75
  76. NuvolaBase.com The first Graph Database on the Cloud always available few seconds to setup it use it from app & mobile(c) Luca Garulli Licensed under a Creative Commons Attribution-NoDerivs 3.0 Unported License Page 76
  77. Luca Garulli Author of CEO at Document-Graph NoSQL Open Source project Ltd, London UK www.twitter.com/lgarulli(c) Luca Garulli Licensed under a Creative Commons Attribution-NoDerivs 3.0 Unported License Page 77

×