1. Containers get Rich
NoSQL Day, Udine – Italy 15-11-2013

2. STEFANO VALLE
2

3. http://www.mvlabs.it
3

4. THE BIG DATA WE USE EVERY DAY

5. NOSQL IN RESCUE!

6. OMG! How to choose?

7. DATA MODEL

8. key
value

9. nosqlday
http://nosqlday.it
// Store value
> SET nosqlday http://nosqlday.it
// Retrieve value
> GET nosqlday
http://nosqlday.it

10. the_godfather

11. KEY - VALUE

12. the_godfather

13. DOCUMENT

14. COLUMN

15. Year
Cast
Rating
1994
Tom Hanks,
Robin Wright,
Gary Sinise
8,7
Braveheart
1995
Mel Gibson,
Sophie Marceau,
Patrick McGoohan
Fast &
Furious 7
2014
Forrest Gump

16. GRAPH

17. product
1
home
product
2
product
3
17

18. product
1
landing
page 1
home
landing
page 2
18
product
2
product
3

19. product
1
landing
page 1
home
landing
page 2
19
product
2
product
3

20. product
1
landing
page 1
home
landing
page 2
20
product
2
product
3

21. Classification by data model
Key - value
Column
21
Document
Graph

22. Is all about
data model?

23. Let’s suppose exists a RDBMS «category»…
23

24. Let’s suppose exists a RDBMS «category»…
24

25. NOSQL ARE SIMPLY
DATA CONTAINERS?

26. Many more
to say about
NoSQL datastores

27. OFFLINE WEB APPLICATIONS

28. OFFLINE WEB APPLICATIONS

29. GEOSPATIAL SEARCH

30. GEOSPATIAL SEARCH

31. KEY-VALUE + LISTS

32. KEY-VALUE + SETS

33. VALUE EXPIRATION

34. NoSQL Mobile Databases
[Lite]

35. ALL useful things!

36. ALL useful things!
but don’t limit to
features-first
comparison

37. 37
_______
_______
_______
_______
_______
Durability vs Performance

38. Durability vs Performance
_______
_______
_______
_______
_______
38
safe mode?
off = data loss risk

39. Durability vs Performance
safe mode?
off = data loss risk
_______
_______
_______
_______
_______
file system
39

40. Durability vs Performance
safe mode?
off = data loss risk
_______
_______
_______
_______
_______
Consider use of
Journaling
file system
40

41. Durability vs Performance
safe mode?
off = data loss risk
_______
_______
_______
_______
_______
Disk / RAID cache?
file system
41

42. And about scaling?

43. Goldfish, not thoroughbreds
Scale up

44. Goldfish, not thoroughbreds
Scale out
Scale up

45. Goldfish, not thoroughbreds
Allow for
fast, cost-effective,
on-demand growth
(or shrink)
Scale out
Scale up

46. "I know two companies that collapsed
due to inability to reduce operating
costs when the utilization of their sites
diminished"
Theo Schlossnagle
From Theo’s book "Scalable Internet Architectures"

47. Ease of
scalability
KeyValue
stores
ColumnFamily
stores
Document
databases
Graph
databases
> 90% of use cases
Complexity
Adapted from http://www.slideshare.net/emileifrem/an-overview-of-nosql-jfokus-2011

48. Query capability
48

49. Query capability
function(doc) {
if (doc.city == ‘London’) {
emit(doc._id, null)
}
}
49

50. Query capability
We couldn’t use user input here
function(doc) {
if (doc.city == ???) {
emit(doc._id, null)
}
}
50

51. Query capability
db.events.find(
{ city: ‘Rome’ }
)
Here we could use user input!
function(doc) {
if (doc.city == ???) {
emit(doc._id, null)
}
}
51

52. Distribution model
52

53. FILTERED MULTI-MASTER  S
Distribution model: MREPLICATION
MASTER
all data
SLAVE

54. FILTERED MULTI-MASTER  M
Distribution model: MREPLICATION
MASTER
MASTER
MASTER

55. FILTERED MULTI-MASTER
Filtered multi-master REPLICATION
Product list
MASTER
MASTER
(eg. head quarter)
(eg. customer plant)
Purchases

56. FILTERED MULTI-MASTER REPLICATION
Scaling reads
MASTER
MASTER
MASTER

57. FILTERED MULTI-MASTER REPLICATION
Scaling reads
client
client
MASTER
MASTER
MASTER
client

58. FILTERED MULTI-MASTER REPLICATION
Scaling writes?
MASTER
MASTER
MASTER

59. FILTERED MULTI-MASTER REPLICATION
Scaling writes?
client
MASTER
MASTER
MASTER

60. FILTERED
ShardingMULTI-MASTER REPLICATION
Shard 1
[A to F]
Shard 2
[G to N]
Shard 3
[O to T]
Shard 4
[U to Z]

61. FILTERED MULTI-MASTER REPLICATION
Scaling writes
client
Shard 1
[A to F]
Shard 2
[G to N]
Shard 3
[O to T]
Shard 4
[U to Z]

62. FILTERED MULTI-MASTER REPLICATION
Scaling writes
client
Shard 1
[A to F]
Shard 2
[G to N]
Shard 3
[O to T]
Shard 4
[U to Z]

63. R / W data from 2 nodes
63

64. R / W data from 2 nodes
T1
Node 1
C
X=0
Node 2
C
X=0
64

65. R / W data from 2 nodes
T1
T2
Node 1
C
X=0
C
X=1
Node 2
C
X=0
C
X=0
65

66. R / W data from 2 nodes
T1
T2
Node 1
C
X=0
C
X=1
Node 2
C
X=0
C
X=0
66

67. CAP Theorem
Consistency
Partition
Tolerance
67
Availability

68. Choose
2
CAP Theorem
Consistency
Partition
Tolerance
68
Availability

69. (Some of) available solutions
CP:
BigTable
Hbase
MongoDB
Redis
MemcacheDB
etc.
PA:
Dynamo
CouchDB
Cassandra
SimpleDB
Tokyo Cabinet
Voldemort
etc.
69
CA:
RDBMS
etc.
Consistency
Partition
Tolerance
Availability

70. from
CONSISTENCY

71. to
EVENTUAL CONSISTENCY

72. Basic
Availability
Soft state
Eventual consistency

73. Atomicity
Consistency
Isolation
Durability

74. Aggregates
Source: AggregateOrientedDatabase - http://martinfowler.com/bliki/AggregateOrientedDatabase.html
74

75. Aggregates
Atomicity and Isolation
are guaranteed inside
an aggregate
Source: AggregateOrientedDatabase - http://martinfowler.com/bliki/AggregateOrientedDatabase.html
75

76. ARE YOU SURE WE NEED ACID?

77. ARE YOU SURE WE NEED ACID?

78. Safety
vs
Liveness

79. Safety
vs
Liveness
Availability is revenue!

80. BACK TO CONTAINERS

81. Data Model
STARTING FROM DATA MODEL…

82. Scalability
model
Data durability
Query model
Position
on CAP
Some needful
feature
Performance
Data Model
MANY OTHER THINGS TO CONSIDER

83. RELATIONAL DBMS

84. RELATIONAL DBMS

85. "the relational model
is pretty magical"
Laurie Voss
http://seldo.com/weblog/2010/07/12/in_defence_of_sql

86. NOSQL DATASTORES

87. "Big data is like teenage sex: everyone
talks about it, nobody really knows
how to do it, everyone thinks everyone
else is doing it, so everyone claims
they are doing it..."
Dan Ariely
https://www.facebook.com/dan.ariely/posts/904383595868

88. Schemaless

89. Schemaless
$doc = $myDb->getDoc('the_godfather');
$year = $doc['year'];
$castCount = count($doc['cast']);
if ($castCount > 0) {
$firstCastName = $doc['cast'][0]['name'];
}

90. Schemaless
…are you sure?
$doc = $myDb->getDoc('the_godfather');
$year = $doc['year'];
$castCount = count($doc['cast']);
if ($castCount > 0) {
$firstCastName = $doc['cast'][0]['name'];
}
The application is aware of
document schema!

91. Polyglot persistence
Farm
Node 1
Provisioning
LAPP stack
Node n
91
Devices
status
Data
aggregation

92. Polyglot persistence made safe
Other
component
Provisioning
layer
92

93. Polyglot persistence made safe
Other
component
Data Store as a Service
Provisioning
layer
93

94. Polyglot persistence made safe
Other
component
Provisioning
Anti
Corruption
Layer
94
layer

95. GOOD APPLICATION DESIGN

96. GOOD APPLICATION DESIGN
THINK ABOUT DATA LIFECYCLE

97. GOOD APPLICATION DESIGN
THINK ABOUT DATA LIFECYCLE
NOT ONLY DATA MODEL

98. That’s all, folks!
Stefano Valle
@stefanovalle
s.valle@mvassociati.it

99. Photo credits
http://www.flickr.com/photos/aloha75/4571410233
http://www.flickr.com/photos/djnordic/167433120
http://www.flickr.com/photos/jpstanley/69523927
http://www.flickr.com/photos/lodigs/2833648828
http://www.flickr.com/photos/ppym1/387781444
http://www.flickr.com/photos/freefoto/3844247553
http://www.flickr.com/photos/jamesgood/1708602693
http://www.flickr.com/photos/ms_cwang/133084413
http://www.flickr.com/photos/birminghammag/7979485144
http://www.flickr.com/photos/capcase/4970062870