1. A new collaborative way to develop
Internet Apps powered by NoSQL and
Javascript
Luca Garulli –
Founder and CEO @NuvolaBase Ltd
Author of OrientDB Doc/Graph DB
Nov 17, 2012 – Paris, France
(c) Luca Garulli Licensed under a Creative Commons Attribution-NoDerivs 3.0 Unported License Page 1
www.orientechnologies.com
2. Agenda
(1) Developing Web App today
(2) Goal: reduce complexity
and improve performance
(3) The Next Generation Web Apps
(c) Luca Garulli Licensed under a Creative Commons Attribution-NoDerivs 3.0 Unported License Page 2
3. Can you remember when developing
Applications was simpler?
Client Client Client
WebApp
Few clients,
usually running
DB
inside a Intranet
(c) Luca Garulli Licensed under a Creative Commons Attribution-NoDerivs 3.0 Unported License Page 3
4. Can you remember when developing
Applications was simpler?
Client Client Client
WebApp
Pages were always
rendered
at the server side
DB
(c) Luca Garulli Licensed under a Creative Commons Attribution-NoDerivs 3.0 Unported License Page 4
5. Why today are more similar to this?
Client Client Client Client
Client Client Client Client
Client Client Client Client
Client Client Client Client
Balancer
WebApp 1 WebApp 2 WebApp 3 Batch tasks
Batch tasks
Batch tasks
Batch tasks
Batch tasks
Cache DB DB
(c) Luca Garulli Licensed under a Creative Commons Attribution-NoDerivs 3.0 Unported License Page 5
6. Mainly because the Internet
100x clients!
From few tens of
concurrent clients to
hundreds or more
(c) Luca Garulli Licensed under a Creative Commons Attribution-NoDerivs 3.0 Unported License Page 6
7. Mainly because the Internet
Response Times:
Network connection is not anymore
the bottleneck,
users now expect response
times less than 1 second per operation
(c) Luca Garulli Licensed under a Creative Commons Attribution-NoDerivs 3.0 Unported License Page 7
8. Mainly because the Internet
Big Data:
databases from
thousands of records
to millions or even billions
(c) Luca Garulli Licensed under a Creative Commons Attribution-NoDerivs 3.0 Unported License Page 8
9. How many languages are you using?
Client Client Client Client
Client Client Client Client
Client Client Client Client
Client Client Client Client
Balancer
WebApp 1 WebApp 2 WebApp 3 Batch tasks
Batch tasks
Batch tasks
Batch tasks
Batch tasks
Cache DB DB
(c) Luca Garulli Licensed under a Creative Commons Attribution-NoDerivs 3.0 Unported License Page 9
10. How many languages are you using?
Client Client Client Client
Client Client Client Client
Client Client Client Client
Client Client Client Client
Java? PHP?
.Net? Ruby? Javascript! (Coffescript?)
Python? Balancer
WebApp 1 WebApp 2 WebApp 3 Batch tasks
Batch tasks
Batch tasks
Batch tasks
Batch tasks
Cache DB DB Bash? Perl?
Oracle PL/SQL? ANT?
(c) Luca Garulli Licensed under a Creative Commons Attribution-NoDerivs 3.0 Unported License Page 10
11. Managing complex architecture is
very costly because:
-(1) requires very skilled people,
-specially if multiple technologies
-are involved
-(2) huge time for development
-and testing
(c) Luca Garulli Licensed under a Creative Commons Attribution-NoDerivs 3.0 Unported License Page 11
13. Sometimes the tuning takes
Developing WebApps
more time than developing*!
*Blood Bath
(c) Luca Garulli Licensed under a Creative Commons Attribution-NoDerivs 3.0 Unported License Page 13
14. Do we really need this level
of complexity to scale up?
(c) Luca Garulli Licensed under a Creative Commons Attribution-NoDerivs 3.0 Unported License Page 14
15. Main features
(c) Luca Garulli Licensed under a Creative Commons Attribution-NoDerivs 3.0 Unported License Page 15
16. OrientDB, the multi-model
NoSQL = {
flexibility of Document databases
+ complexity of the Graph model
+ Object Oriented concepts }
(c) Luca Garulli Licensed under a Creative Commons Attribution-NoDerivs 3.0 Unported License Page 16
17. +14 years
of research
(c) Luca Garulli Licensed under a Creative Commons Attribution-NoDerivs 3.0 Unported License Page 17
18. +4 years
of design and development
(c) Luca Garulli Licensed under a Creative Commons Attribution-NoDerivs 3.0 Unported License Page 18
19. Ø config
download, unzip, run!
cut & paste the db directory
(c) Luca Garulli Licensed under a Creative Commons Attribution-NoDerivs 3.0 Unported License Page 19
20. 150,000 records per second
(flat records, no index, on commodity hw)
(c) Luca Garulli Licensed under a Creative Commons Attribution-NoDerivs 3.0 Unported License Page 20
21. Schema-less
schema is not mandatory, relaxed model,
collect heterogeneous documents all together
(c) Luca Garulli Licensed under a Creative Commons Attribution-NoDerivs 3.0 Unported License Page 21
22. Schema-full
schema with constraints on fields and validation rules
Customer.age > 17
Customer.address not null
Customer.surname is mandatory
Customer.email matches 'b[A-Z0-9._%+-]+@[A-Z0-9.-]+.[A-Z]{2,4}b'
(c) Luca Garulli Licensed under a Creative Commons Attribution-NoDerivs 3.0 Unported License Page 22
23. Schema-mixed
schema with mandatory and optional fields + constraints
the best of schema-less and schema-full modes
(c) Luca Garulli Licensed under a Creative Commons Attribution-NoDerivs 3.0 Unported License Page 23
24. ACID Transactions
db.begin();
try{
// your code
...
db.commit();
} catch( Exception e ) {
db.rollback();
}
(c) Luca Garulli Licensed under a Creative Commons Attribution-NoDerivs 3.0 Unported License Page 24
25. Complex types
native support for collections, maps (key/value)
and embedded documents
no more additional tables to handle them
(c) Luca Garulli Licensed under a Creative Commons Attribution-NoDerivs 3.0 Unported License Page 25
26. SQL
select * from employee where name like '%Jay%' and status=0
(c) Luca Garulli Licensed under a Creative Commons Attribution-NoDerivs 3.0 Unported License Page 26
27. SQL & relationships
select from Account where address.city.country.name = 'Italy'
select from Account where addresses contains (city.country.name = 'Italy')
(c) Luca Garulli Licensed under a Creative Commons Attribution-NoDerivs 3.0 Unported License Page 27
28. SQL & trees/graphs
select out[label='friend'].in from V where name = 'Luca' and surname = 'Garulli'
select out[@class='knows'].in from V where name = 'Jay' and surname = 'Miner'
traverse friends from Profile while $depth < 7
(c) Luca Garulli Licensed under a Creative Commons Attribution-NoDerivs 3.0 Unported License Page 28
29. SQL sub queries
select from (
traverse friends from Profile where $depth <= 7
) where home.city.name = ‘Paris’
(c) Luca Garulli Licensed under a Creative Commons Attribution-NoDerivs 3.0 Unported License Page 29
30. SQL & strings
select from Profile where name.toUpperCase() = 'LUCA'
select from City where country.name.substring(1,3).toUpperCase() = 'TAL'
select from Agenda where phones contains ( number.indexOf( '+39' ) > -1 )
select from Agenda where email matches 'bA-Z0-9._%+-?+@A-Z0-9.-?+.A-Z?{2,4}b'
(c) Luca Garulli Licensed under a Creative Commons Attribution-NoDerivs 3.0 Unported License Page 30
31. SQL & schema-less
select from Profile where any() like '%Jay%'
select from Stock where all() is not null
(c) Luca Garulli Licensed under a Creative Commons Attribution-NoDerivs 3.0 Unported License Page 31
32. SQL & collections
select from Tree where children contains ( married = true )
select from Tree where children containsAll ( married = true )
select from User where roles containsKey 'shutdown'
select from Graph where edges.size() > 0
(c) Luca Garulli Licensed under a Creative Commons Attribution-NoDerivs 3.0 Unported License Page 32
33. runs
Java
everywhere is available JRE1.6+
®
robust engine
(c) Luca Garulli Licensed under a Creative Commons Attribution-NoDerivs 3.0 Unported License Page 33
34. Language bindings
Java as native
JRuby, PHP, C, C++, Scala, .NET,
Ruby, Clojure, Node.js,
Python, Javascript and more!
(c) Luca Garulli Licensed under a Creative Commons Attribution-NoDerivs 3.0 Unported License Page 34
35. Born for the Internet
Supports natively HTTP/RESTful protocol
Documents are transferred in JSON
(c) Luca Garulli Licensed under a Creative Commons Attribution-NoDerivs 3.0 Unported License Page 35
36. MVRB-Tree index
the best of B+Tree and RB-Tree
fast on browsing, low insertion cost
it's a new algorithm!
(c) Luca Garulli Licensed under a Creative Commons Attribution-NoDerivs 3.0 Unported License Page 36
37. Security
users and roles, encrypted passwords
fine grain privileges
(similar to what RDBMSs offer)
(c) Luca Garulli Licensed under a Creative Commons Attribution-NoDerivs 3.0 Unported License Page 37
38. Cache
You can avoid using 3°party caches
like Memcached
2 Levels of cache:
Level1: Database level, 1 per thread
Level2: Storage level, 1 per JVM
(c) Luca Garulli Licensed under a Creative Commons Attribution-NoDerivs 3.0 Unported License Page 38
40. Polymorphic SQL Query
OGraphVertex (V)
Person Vehicle
Address : Address brand : BRANDS
select * from Person
where city.name = 'Rome‘
Queries are polymorphics
Customer Provider and subclasses of Person can be
totSold : float totBuyed : float part of result set
(c) Luca Garulli Licensed under a Creative Commons Attribution-NoDerivs 3.0 Unported License Page 40
41. “A graph database is any
storage system
that provides
index-free adjacency”
- Marko Rodriguez
(author of TinkerPop Blueprints and Gremlin language)
(c) Luca Garulli Licensed under a Creative Commons Attribution-NoDerivs 3.0 Unported License Page 41
42. GraphDB handles relationships as a
physical LINK to the record
assigned when the relationship is created
on the other side
RDBMS computes the
relationship every time you query a
database!
(c) Luca Garulli Licensed under a Creative Commons Attribution-NoDerivs 3.0 Unported License Page 42
43. This means jumping from a
O(log N) algorithm to a near O(1)
traversing cost is not more affected
by database size!
This is huge in the BigData age
(c) Luca Garulli Licensed under a Creative Commons Attribution-NoDerivs 3.0 Unported License Page 43
44. OrientDB in the Blueprints micro-benchmark,
on common hw, with a hot cache,
traverses 29,6 Millions
of records in less than 5 seconds
about 6 Millions of nodes traversed per sec!
Do not try this at home
with any RDBMS*!
*unless you live in the Google’s server farm
(c) Luca Garulli Licensed under a Creative Commons Attribution-NoDerivs 3.0 Unported License Page 44
45. Now let’s start modelling
A Web App using
OrientDB
(c) Luca Garulli Licensed under a Creative Commons Attribution-NoDerivs 3.0 Unported License Page 45
46. Our simple domain
Address City
name : String
street : String
country : String
Customer
name: String
surname: String
Order
total : decimal
(c) Luca Garulli Licensed under a Creative Commons Attribution-NoDerivs 3.0 Unported License Page 46
47. Our simple domain
Address City
name : String
street : String
country : String
Customer
name: String Relationships in
surname: String
OrientDB are „links“
Order The link between two or
total : decimal more documents is
created at the
document save time
(c) Luca Garulli Licensed under a Creative Commons Attribution-NoDerivs 3.0 Unported License Page 47
49. { Document‘s unique
'@rid' = ‘#8:0’, RID (Record ID)
Assigned at creation
'@class' = ‘Customer',
time and never
'name' : 'Luca', changes!
'surname' : 'Garulli',
‘orders’ : [ ‘#10:33’, ‘#10:232’ ],
‘address’ : {
‘@class’ : ‘Address’,
‘street’ : ‘Via del Corso 15’,
‘city’ : ‘#9:0’
}
}
(c) Luca Garulli Licensed under a Creative Commons Attribution-NoDerivs 3.0 Unported License Page 49
50. {
'@rid' = ‘#8:0’,
'@class' = ‘Customer',
'name' : 'Luca',
'surname' : 'Garulli',
One-to-Many
‘orders’ : [ ‘#10:33’, ‘#10:232’ ], relationship as a array
‘address’ : { of links
‘@class’ : ‘Address’, The link is the RID of the
‘street’ : ‘Via del Corso 15’, target document(s)
‘city’ : ‘#9:0’
}
}
(c) Luca Garulli Licensed under a Creative Commons Attribution-NoDerivs 3.0 Unported License Page 50
51. {
'@rid' = ‘#8:0’,
'@class' = ‘Customer',
'name' : 'Luca',
'surname' : 'Garulli',
‘orders’ : [ ‘#10:33’, ‘#10:232’ ],
‘address’ : {
‘@class’ : ‘Address’, Embedded document
is saved inside the
‘street’ : ‘Via del Corso 15’,
owner document
‘city’ : ‘#9:0’ Has no own identity
}
}
(c) Luca Garulli Licensed under a Creative Commons Attribution-NoDerivs 3.0 Unported License Page 51
52. Create a new document of class «Customer»
>curl --basic -u admin:admin localhost:2480/document/jsdemo
-d '{"@class":"Customer","name":“Luca","surname":“Garulli"}'
#8:0
Retrieve the document just created
>curl --basic -u admin:admin localhost:2480/document/jsdemo/8:0
{
"@type": "d", "@rid": "#8:0", "@version": 0, "@class": "Customer",
"name": "Luca",
"surname": "Garulli"
}
(c) Luca Garulli Licensed under a Creative Commons Attribution-NoDerivs 3.0 Unported License Page 52
53. Since OrientDB talks natively
HTTP and JSON
Is it too insane using it directly from the
Web Pages bypassing any servers-side
layer?
(c) Luca Garulli Licensed under a Creative Commons Attribution-NoDerivs 3.0 Unported License Page 53
54. Pros
+ concurrency and response time:
It’s super fast, all happens at the
client side
+ development time having just
one layer to develop and maintain
(c) Luca Garulli Licensed under a Creative Commons Attribution-NoDerivs 3.0 Unported License Page 54
55. Javascript driver example I
<script src="js/orientdb-api.js" type="text/javascript"></script>
<script>
var db = new ODatabase(‘http://localhost:2480/jsdemo’);
var info = db.open(‘admin’, ‘admin’);
var customer = { “@class” : “Customer”,
“name” : “Luca”,
“surname“ : “Garulli” };
var rid = db.save( customer );
// reload the customer back
var customer = db.load(‘8:0’);
</script>
(c) Luca Garulli Licensed under a Creative Commons Attribution-NoDerivs 3.0 Unported License Page 55
56. Javascript driver example II
Display in a input field the total of orders
var date = $(‘#date’).val(); // jQuery
var resultset = db.query( ‘select sum(total) as total from Order
where date > ?’, [ date ] );
$(‘#total’).val( resultset.result[0].total ); // jQuery
(c) Luca Garulli Licensed under a Creative Commons Attribution-NoDerivs 3.0 Unported License Page 56
57. Cons
- Client-side JS can be easily hacked
- Sometimes it’s not a good idea
writing queries in the web pages
revealing your database schema
to the users
(c) Luca Garulli Licensed under a Creative Commons Attribution-NoDerivs 3.0 Unported License Page 57
58. Security
To improve security we supported
Record-Level access control
Even if the JSON is hacked at the client
side, the user can
read/write own records only
Can be enabled only for some classes of documents
(c) Luca Garulli Licensed under a Creative Commons Attribution-NoDerivs 3.0 Unported License Page 58
59. Record Level Security
Business Documents
Class «Doc»
A B
> insert into Doc (name) values (‘A’) > insert into Doc (name) values (‘B’)
> select * from Doc > select * from Doc
result: A result: B
(c) Luca Garulli Licensed under a Creative Commons Attribution-NoDerivs 3.0 Unported License Page 59
60. Server-Side functions
Let to OrientDB executing Javascript
at the server side. The code of the
function is stored in the database
Similar to the «old» Stored-Procedures,
but written using the Internet Language:
Javascript
(c) Luca Garulli Licensed under a Creative Commons Attribution-NoDerivs 3.0 Unported License Page 60
61. Server-Side Functions can access to special
objects in context:
db, as the current database instance. This
object is like the JS driver API to make easy
moving code between client and server
request, as the HTTP request
response, as the HTTP response
util, as utility helper class
(c) Luca Garulli Licensed under a Creative Commons Attribution-NoDerivs 3.0 Unported License Page 61
62. function user_create( name, roleName ) {
var role = db.query("select from ORole where name = ?", roleName );
if( role == null ){
response.send(404, "Role name not found", "text/plain",
"Error: role name not found" );
} else {
db.begin();
try{
var result = db.save({ "@class" : "OUser", name : "Luca",
password : "Luc4", roles : role});
db.commit();
return result;
} catch ( err ) {
db.rollback();
response.send(500, "Error on creating new user", "text/plain",
err.toString() );
}
}
(c) Luca Garulli Licensed under a Creative Commons Attribution-NoDerivs 3.0 Unported License Page 62
63. Server-Side Functions are automatically
published and reachable via HTTP REST.
Example:
http://localhost:2480/function/jsdemo/sum/3/20
Server-Side Functions must declare if are
Idempotent. Non-idempotent functions
can be only be called using the
POST HTTP method
(c) Luca Garulli Licensed under a Creative Commons Attribution-NoDerivs 3.0 Unported License Page 63
64. Demo, please
(c) Luca Garulli Licensed under a Creative Commons Attribution-NoDerivs 3.0 Unported License Page 64
65. OrientDB uses Mozilla Rhino as JS engine
Why Rhino?
+ Rhino allows execution of Java code,
very useful to access to tons of libraries
+ Financial and BI apps allows using
the java.math.BigDecimal class
+ Mature technology
+ Fast, (even if probably not the fastest?)
(c) Luca Garulli Licensed under a Creative Commons Attribution-NoDerivs 3.0 Unported License Page 65
66. Fetch plans
Choose what to fetch on query and document loading
Optimizes network latency
Load many records in one shot!
(c) Luca Garulli Licensed under a Creative Commons Attribution-NoDerivs 3.0 Unported License Page 66
67. Fetch plans
Load root + all the orders
Customer fetchPlan = “*:1 orders:1”
Luca
|
| address city
+-------> Address -----> City
| 10 street
th
Italy
| orders
+--------->* [Order Order ]
[Order1 Order2]
(c) Luca Garulli Licensed under a Creative Commons Attribution-NoDerivs 3.0 Unported License Page 67
69. Future plans
Support debugging via Studio
«Live Objects» by providing a true Object
Oriented interface to persistent
documents using JS to implement methods
Try Google V8 in place of Rhino
(c) Luca Garulli Licensed under a Creative Commons Attribution-NoDerivs 3.0 Unported License Page 69
70. Always Free
Open Source Apache 2 license
free for any purposes,
even commercials
(c) Luca Garulli Licensed under a Creative Commons Attribution-NoDerivs 3.0 Unported License Page 70
71. www.NuvolaBase.com
The OrientDB company
DB as service Professional support Training
on the cloud for production Courses and
always available systems certification
(c) Luca Garulli Licensed under a Creative Commons Attribution-NoDerivs 3.0 Unported License Page 71
72. Network of partners
er ?
partn
or k as m
he netw ase.co
ar t of t nuvolab
o be p t info@
W ant t ontac
c
To provide Professional Services through NuvolaBase Ltd:
support, training, consulting, mentoring
(c) Luca Garulli Licensed under a Creative Commons Attribution-NoDerivs 3.0 Unported License Page 72
73. The OrientDB Academy
Course + Certification in
all the world
as soon as we reach
20 attendees!
Want it in your city?
Send an email to:
info@nuvolabase.com
(c) Luca Garulli Licensed under a Creative Commons Attribution-NoDerivs 3.0 Unported License Page 73
74. Courses
OrientDB OrientDB
for Java developers Master Development
8 hours 14 hours
OrientDB OrientDB
for SOA and the power of graphs
6 hours 6 hours
OrientDB OrientBOX
for DBA for Web Developers
6 hours 6 hours
(c) Luca Garulli Licensed under a Creative Commons Attribution-NoDerivs 3.0 Unported License Page 74
75. “OrientDB in Action”
book
by Manning Publications
is coming...
(begin of 2013)
(c) Luca Garulli Licensed under a Creative Commons Attribution-NoDerivs 3.0 Unported License Page 75
76. The Vision
The next generation Web Applications are
going to be written in JS that talk
in HTTP/RESTful protocol against
Browser and multi-devices Smart Phones and
Tablets where the Business Logic
and Data are totally integrated
to achieve the maximum of the speed,
scalability and easy of deployment
(c) Luca Garulli Licensed under a Creative Commons Attribution-NoDerivs 3.0 Unported License Page 76
77. The Vision
WWW Mobile Tablet
HTTP RESTful
WebApp
Business Logic (JS)
Database (documents)
(c) Luca Garulli Licensed under a Creative Commons Attribution-NoDerivs 3.0 Unported License Page 77
78. Summary
(1) Reduce complexity
(2) Improve performance removing
layers
(3) Use 1 (one) language for all: JS
(c) Luca Garulli Licensed under a Creative Commons Attribution-NoDerivs 3.0 Unported License Page 78
79. Q&A
Luca Garulli
Author of CEO at
Document-Graph NoSQL
Open Source project Ltd, London UK
www.twitter.com/lgarulli
(c) Luca Garulli Licensed under a Creative Commons Attribution-NoDerivs 3.0 Unported License Page 79