Apache Cassandra
Ruby Driver explained
Introduction
Cassandra Overview
© 2015 DataStax, All Rights Reserved.
Datacenter Datacenter
Cassandra Topology
3
Node
NodeNode
Node
Client Client
Node
NodeNode
Node
Client Client
Cluster
© 2015 DataStax, All Rights Reserved.
Datacenter Datacenter
Request Coordinator
4
Node
NodeNode
Node
Client Client
Node
NodeCoordinator
Node
Client Client
Coordinator node:
Forwards requests
to corresponding replicas
© 2015 DataStax, All Rights Reserved.
Datacenter
Row Replica
5
Replica
NodeNode
Replica
Client Client
Datacenter
Node
Node
Replica
Client Client
Coordinator
Replica node:
Stores a slice of total rows
of each keyspace
Quick Start
Installation and Usage
© 2015 DataStax, All Rights Reserved.
Installation
7
gem 'cassandra-driver', '~> 1.0.0'
gem install cassandra-driver
© 2015 DataStax, All Rights Reserved.
Usage
8
require 'cassandra'
cluster = Cassandra.cluster
cluster.each_host do |h|
puts "Host #{h.ip}: datacenter=#{h.datacenter} rack=#{h.rack}"
end
keyspace = 'system'
session = cluster.connect(keyspace)
future = session.execute_async('SELECT * FROM schema_columnfamilies')
future.on_success do |rows|
rows.each do |row|
puts “Table: #{row[‘keyspace_name']}.#{row['columnfamily_name']}"
end
end
future.join
Asynchronous Execution
IO Reactor, Request Pipelining and Future Composition
© 2015 DataStax, All Rights Reserved.
Asynchronous Core
10
Application Thread
Business Logic
Driver
Background Thread
IO Reactor
© 2015 DataStax, All Rights Reserved.
Request Pipelining
11
Client
Without
Request Pipelining
Server
Client Server
With
Request Pipelining
1
2
2
3
1
3
1
2
3
1
2
3
© 2015 DataStax, All Rights Reserved.
Future Composition
12
select_user = session.prepare("SELECT * FROM users WHERE id = ?")
select_page = session.prepare("SELECT * FROM pages WHERE slug = ?")
user_ids = [1, 2, 3, 4]
futures = user_ids.map do |id|
future = session.execute_async(select_user, id)
future.then do |users|
user = users.first
future = session.execute_async(select_page, user[‘username'])
future.then do |pages|
page = pages.first
User.new(user, Page.new(page))
end
end
end
Cassandra::Future.all(futures).get
© 2015 DataStax, All Rights Reserved.
Future Composition
13
select_user = session.prepare("SELECT * FROM users WHERE id = ?")
select_page = session.prepare("SELECT * FROM pages WHERE slug = ?")
user_ids = [1, 2, 3, 4]
futures = user_ids.map do |id|
future = session.execute_async(select_user, id)
future.then do |users|
user = users.first
future = session.execute_async(select_page, user[‘username'])
future.then do |pages|
page = pages.first
User.new(user, Page.new(page))
end
end
end
Cassandra::Future.all(futures).get
© 2015 DataStax, All Rights Reserved.
Future Composition
14
select_user = session.prepare("SELECT * FROM users WHERE id = ?")
select_page = session.prepare("SELECT * FROM pages WHERE slug = ?")
user_ids = [1, 2, 3, 4]
futures = user_ids.map do |id|
future = session.execute_async(select_user, id)
future.then do |users|
user = users.first
future = session.execute_async(select_page, user[‘username'])
future.then do |pages|
page = pages.first
User.new(user, Page.new(page))
end
end
end
Cassandra::Future.all(futures).get
© 2015 DataStax, All Rights Reserved.
Future Composition
15
select_user = session.prepare("SELECT * FROM users WHERE id = ?")
select_page = session.prepare("SELECT * FROM pages WHERE slug = ?")
user_ids = [1, 2, 3, 4]
futures = user_ids.map do |id|
future = session.execute_async(select_user, id)
future.then do |users|
user = users.first
future = session.execute_async(select_page, user[‘username'])
future.then do |pages|
page = pages.first
User.new(user, Page.new(page))
end
end
end
Cassandra::Future.all(futures).get
© 2015 DataStax, All Rights Reserved.
Future Composition
16
select_user = session.prepare("SELECT * FROM users WHERE id = ?")
select_page = session.prepare("SELECT * FROM pages WHERE slug = ?")
user_ids = [1, 2, 3, 4]
futures = user_ids.map do |id|
future = session.execute_async(select_user, id)
future.then do |users|
user = users.first
future = session.execute_async(select_page, user[‘username'])
future.then do |pages|
page = pages.first
User.new(user, Page.new(page))
end
end
end
Cassandra::Future.all(futures).get
© 2015 DataStax, All Rights Reserved.
Future Composition
17
select_user = session.prepare("SELECT * FROM users WHERE id = ?")
select_page = session.prepare("SELECT * FROM pages WHERE slug = ?")
user_ids = [1, 2, 3, 4]
futures = user_ids.map do |id|
future = session.execute_async(select_user, id)
future.then do |users|
user = users.first
future = session.execute_async(select_page, user[‘username'])
future.then do |pages|
page = pages.first
User.new(user, Page.new(page))
end
end
end
Cassandra::Future.all(futures).get
© 2015 DataStax, All Rights Reserved.
Future Composition
18
[#<User @id=1 @username="avalanche123"; @page=#<Page @slug="avalanche123" ... > ... >, ... ]
Load Balancing
Principles and Implementations
© 2015 DataStax, All Rights Reserved.
Application Driver
Load Balancing
20
Application
Thread
Node
Pool
Session
Pool
Pool
Pool
Application
Thread
Application
Thread
Client Cluster
Node
Node
Node
Load Balancing
Policy
© 2015 DataStax, All Rights Reserved.
Application Driver
Load Balancing
20
Application
Thread
Node
Pool
Session
Pool
Pool
Pool
Application
Thread
Application
Thread
Client Cluster
Node
Node
Node
Load Balancing
Policy
© 2015 DataStax, All Rights Reserved.
Application Driver
Load Balancing
20
Application
Thread
Node
Pool
Session
Pool
Pool
Pool
Application
Thread
Application
Thread
Client Cluster
Node
Node
Node
Load Balancing
Policy
© 2015 DataStax, All Rights Reserved.
Datacenter
Datacenter
DataCenter Aware Balancing
21
Node
Node
NodeClient
Node
Node
Node
Client
Client
Client
Client
Client
Local nodes are queried
first, if non are available,
the request could be
sent to a remote node.
© 2015 DataStax, All Rights Reserved.
Token Aware Balancing
22
Route request
directly to Replicas
Node
Node
Replica
Node
Client
Replica
Replica
Uses prepared statement
metadata to get the token
Fault Tolerance
Sources of Failure and Error Handling
© 2015 DataStax, All Rights Reserved.
Fault Tolerance
24
Coordinator
Node Replica
Replica
Replica
Node
Business Logic
Driver
Application
© 2015 DataStax, All Rights Reserved. 25
Coordinator
Node Replica
Replica
Replica
Node
Business Logic
Driver
Application
Invalid Requests
Network Timeouts
Server Errors
Possible Failures
© 2015 DataStax, All Rights Reserved.
Application Driver
Automatic Retry of Server Errors
26
Application
Thread
Node
Pool
Session
Pool
Pool
Pool
Application
Thread
Application
Thread
Client Cluster
Node
Node
Node
Load Balancing
Policy
© 2015 DataStax, All Rights Reserved.
Application Driver
Automatic Retry of Server Errors
26
Application
Thread
Node
Pool
Session
Pool
Pool
Pool
Application
Thread
Application
Thread
Client Cluster
Node
Node
Node
Load Balancing
Policy
© 2015 DataStax, All Rights Reserved.
Application Driver
Automatic Retry of Server Errors
26
Application
Thread
Node
Pool
Session
Pool
Pool
Pool
Application
Thread
Application
Thread
Client Cluster
Node
Node
Node
Load Balancing
Policy
© 2015 DataStax, All Rights Reserved. 27
Coordinator
Node Replica
Replica
Replica
Node
Business Logic
Driver
Application
Unreachable Consistency
© 2015 DataStax, All Rights Reserved.
Coordinator
Node Replica
Replica
Node
28
Replica
Business Logic
Driver
Application
Read / Write Timeout Error
© 2015 DataStax, All Rights Reserved.
Coordinator
Node Replica
Replica
Node
28
Replica
Business Logic
Driver
Application
Read / Write Timeout Error
© 2015 DataStax, All Rights Reserved.
Coordinator
Node Replica
Replica
Node
28
Replica
Business Logic
Driver
Application
Read / Write Timeout Error
read / write timeout
© 2015 DataStax, All Rights Reserved. 29
Coordinator
Node Replica
Replica
Replica
Node
Business Logic
Driver
Application
Unavailable Error
© 2015 DataStax, All Rights Reserved. 29
Coordinator
Node Replica
Replica
Replica
Node
Business Logic
Driver
Application
Unavailable Error
unavailable
© 2015 DataStax, All Rights Reserved. 30
Error Handling
Address Resolution
Topology Aware Client
© 2015 DataStax, All Rights Reserved.
Datacenter Datacenter
Multiple Addresses
32
Node
NodeNode
Node
Client Client
Node
NodeNode
Node
Client Client
Within Datacenter:

Private IPs
Across Datacenters:

Public IPs
© 2015 DataStax, All Rights Reserved.
Application Driver
Address Resolution
33
Application
Thread
Application
Thread
Application
Thread
Client Cluster
© 2015 DataStax, All Rights Reserved.
Application Driver
Address Resolution
33
Application
Thread Node
Cluster
Application
Thread
Application
Thread
Client Cluster
Address
Resolution Policy
© 2015 DataStax, All Rights Reserved.
Application Driver
Address Resolution
33
Application
Thread Node
Cluster
Application
Thread
Application
Thread
Client Cluster
Node
Node
Node
Address
Resolution Policy
Control Connection
© 2015 DataStax, All Rights Reserved.
Application Driver
Address Resolution
33
Application
Thread Node
Cluster
Application
Thread
Application
Thread
Client Cluster
Node
Node
Node
Address
Resolution Policy
Control Connection
© 2015 DataStax, All Rights Reserved.
Application Driver
Address Resolution
33
Application
Thread Node
Pool
Cluster
Pool
Pool
Pool
Application
Thread
Application
Thread
Client Cluster
Node
Node
Node
Address
Resolution Policy
Control Connection
Session
© 2015 DataStax, All Rights Reserved.
EC2 Multi-Region Address Resolution
34
Questions and Links
• Documentation
• Code
• Example Web-app

Ruby Driver Explained: DataStax Webinar May 5th 2015

  • 1.
  • 2.
  • 3.
    © 2015 DataStax,All Rights Reserved. Datacenter Datacenter Cassandra Topology 3 Node NodeNode Node Client Client Node NodeNode Node Client Client Cluster
  • 4.
    © 2015 DataStax,All Rights Reserved. Datacenter Datacenter Request Coordinator 4 Node NodeNode Node Client Client Node NodeCoordinator Node Client Client Coordinator node: Forwards requests to corresponding replicas
  • 5.
    © 2015 DataStax,All Rights Reserved. Datacenter Row Replica 5 Replica NodeNode Replica Client Client Datacenter Node Node Replica Client Client Coordinator Replica node: Stores a slice of total rows of each keyspace
  • 6.
  • 7.
    © 2015 DataStax,All Rights Reserved. Installation 7 gem 'cassandra-driver', '~> 1.0.0' gem install cassandra-driver
  • 8.
    © 2015 DataStax,All Rights Reserved. Usage 8 require 'cassandra' cluster = Cassandra.cluster cluster.each_host do |h| puts "Host #{h.ip}: datacenter=#{h.datacenter} rack=#{h.rack}" end keyspace = 'system' session = cluster.connect(keyspace) future = session.execute_async('SELECT * FROM schema_columnfamilies') future.on_success do |rows| rows.each do |row| puts “Table: #{row[‘keyspace_name']}.#{row['columnfamily_name']}" end end future.join
  • 9.
    Asynchronous Execution IO Reactor,Request Pipelining and Future Composition
  • 10.
    © 2015 DataStax,All Rights Reserved. Asynchronous Core 10 Application Thread Business Logic Driver Background Thread IO Reactor
  • 11.
    © 2015 DataStax,All Rights Reserved. Request Pipelining 11 Client Without Request Pipelining Server Client Server With Request Pipelining 1 2 2 3 1 3 1 2 3 1 2 3
  • 12.
    © 2015 DataStax,All Rights Reserved. Future Composition 12 select_user = session.prepare("SELECT * FROM users WHERE id = ?") select_page = session.prepare("SELECT * FROM pages WHERE slug = ?") user_ids = [1, 2, 3, 4] futures = user_ids.map do |id| future = session.execute_async(select_user, id) future.then do |users| user = users.first future = session.execute_async(select_page, user[‘username']) future.then do |pages| page = pages.first User.new(user, Page.new(page)) end end end Cassandra::Future.all(futures).get
  • 13.
    © 2015 DataStax,All Rights Reserved. Future Composition 13 select_user = session.prepare("SELECT * FROM users WHERE id = ?") select_page = session.prepare("SELECT * FROM pages WHERE slug = ?") user_ids = [1, 2, 3, 4] futures = user_ids.map do |id| future = session.execute_async(select_user, id) future.then do |users| user = users.first future = session.execute_async(select_page, user[‘username']) future.then do |pages| page = pages.first User.new(user, Page.new(page)) end end end Cassandra::Future.all(futures).get
  • 14.
    © 2015 DataStax,All Rights Reserved. Future Composition 14 select_user = session.prepare("SELECT * FROM users WHERE id = ?") select_page = session.prepare("SELECT * FROM pages WHERE slug = ?") user_ids = [1, 2, 3, 4] futures = user_ids.map do |id| future = session.execute_async(select_user, id) future.then do |users| user = users.first future = session.execute_async(select_page, user[‘username']) future.then do |pages| page = pages.first User.new(user, Page.new(page)) end end end Cassandra::Future.all(futures).get
  • 15.
    © 2015 DataStax,All Rights Reserved. Future Composition 15 select_user = session.prepare("SELECT * FROM users WHERE id = ?") select_page = session.prepare("SELECT * FROM pages WHERE slug = ?") user_ids = [1, 2, 3, 4] futures = user_ids.map do |id| future = session.execute_async(select_user, id) future.then do |users| user = users.first future = session.execute_async(select_page, user[‘username']) future.then do |pages| page = pages.first User.new(user, Page.new(page)) end end end Cassandra::Future.all(futures).get
  • 16.
    © 2015 DataStax,All Rights Reserved. Future Composition 16 select_user = session.prepare("SELECT * FROM users WHERE id = ?") select_page = session.prepare("SELECT * FROM pages WHERE slug = ?") user_ids = [1, 2, 3, 4] futures = user_ids.map do |id| future = session.execute_async(select_user, id) future.then do |users| user = users.first future = session.execute_async(select_page, user[‘username']) future.then do |pages| page = pages.first User.new(user, Page.new(page)) end end end Cassandra::Future.all(futures).get
  • 17.
    © 2015 DataStax,All Rights Reserved. Future Composition 17 select_user = session.prepare("SELECT * FROM users WHERE id = ?") select_page = session.prepare("SELECT * FROM pages WHERE slug = ?") user_ids = [1, 2, 3, 4] futures = user_ids.map do |id| future = session.execute_async(select_user, id) future.then do |users| user = users.first future = session.execute_async(select_page, user[‘username']) future.then do |pages| page = pages.first User.new(user, Page.new(page)) end end end Cassandra::Future.all(futures).get
  • 18.
    © 2015 DataStax,All Rights Reserved. Future Composition 18 [#<User @id=1 @username="avalanche123"; @page=#<Page @slug="avalanche123" ... > ... >, ... ]
  • 19.
  • 20.
    © 2015 DataStax,All Rights Reserved. Application Driver Load Balancing 20 Application Thread Node Pool Session Pool Pool Pool Application Thread Application Thread Client Cluster Node Node Node Load Balancing Policy
  • 21.
    © 2015 DataStax,All Rights Reserved. Application Driver Load Balancing 20 Application Thread Node Pool Session Pool Pool Pool Application Thread Application Thread Client Cluster Node Node Node Load Balancing Policy
  • 22.
    © 2015 DataStax,All Rights Reserved. Application Driver Load Balancing 20 Application Thread Node Pool Session Pool Pool Pool Application Thread Application Thread Client Cluster Node Node Node Load Balancing Policy
  • 23.
    © 2015 DataStax,All Rights Reserved. Datacenter Datacenter DataCenter Aware Balancing 21 Node Node NodeClient Node Node Node Client Client Client Client Client Local nodes are queried first, if non are available, the request could be sent to a remote node.
  • 24.
    © 2015 DataStax,All Rights Reserved. Token Aware Balancing 22 Route request directly to Replicas Node Node Replica Node Client Replica Replica Uses prepared statement metadata to get the token
  • 25.
    Fault Tolerance Sources ofFailure and Error Handling
  • 26.
    © 2015 DataStax,All Rights Reserved. Fault Tolerance 24 Coordinator Node Replica Replica Replica Node Business Logic Driver Application
  • 27.
    © 2015 DataStax,All Rights Reserved. 25 Coordinator Node Replica Replica Replica Node Business Logic Driver Application Invalid Requests Network Timeouts Server Errors Possible Failures
  • 28.
    © 2015 DataStax,All Rights Reserved. Application Driver Automatic Retry of Server Errors 26 Application Thread Node Pool Session Pool Pool Pool Application Thread Application Thread Client Cluster Node Node Node Load Balancing Policy
  • 29.
    © 2015 DataStax,All Rights Reserved. Application Driver Automatic Retry of Server Errors 26 Application Thread Node Pool Session Pool Pool Pool Application Thread Application Thread Client Cluster Node Node Node Load Balancing Policy
  • 30.
    © 2015 DataStax,All Rights Reserved. Application Driver Automatic Retry of Server Errors 26 Application Thread Node Pool Session Pool Pool Pool Application Thread Application Thread Client Cluster Node Node Node Load Balancing Policy
  • 31.
    © 2015 DataStax,All Rights Reserved. 27 Coordinator Node Replica Replica Replica Node Business Logic Driver Application Unreachable Consistency
  • 32.
    © 2015 DataStax,All Rights Reserved. Coordinator Node Replica Replica Node 28 Replica Business Logic Driver Application Read / Write Timeout Error
  • 33.
    © 2015 DataStax,All Rights Reserved. Coordinator Node Replica Replica Node 28 Replica Business Logic Driver Application Read / Write Timeout Error
  • 34.
    © 2015 DataStax,All Rights Reserved. Coordinator Node Replica Replica Node 28 Replica Business Logic Driver Application Read / Write Timeout Error read / write timeout
  • 35.
    © 2015 DataStax,All Rights Reserved. 29 Coordinator Node Replica Replica Replica Node Business Logic Driver Application Unavailable Error
  • 36.
    © 2015 DataStax,All Rights Reserved. 29 Coordinator Node Replica Replica Replica Node Business Logic Driver Application Unavailable Error unavailable
  • 37.
    © 2015 DataStax,All Rights Reserved. 30 Error Handling
  • 38.
  • 39.
    © 2015 DataStax,All Rights Reserved. Datacenter Datacenter Multiple Addresses 32 Node NodeNode Node Client Client Node NodeNode Node Client Client Within Datacenter:
 Private IPs Across Datacenters:
 Public IPs
  • 40.
    © 2015 DataStax,All Rights Reserved. Application Driver Address Resolution 33 Application Thread Application Thread Application Thread Client Cluster
  • 41.
    © 2015 DataStax,All Rights Reserved. Application Driver Address Resolution 33 Application Thread Node Cluster Application Thread Application Thread Client Cluster Address Resolution Policy
  • 42.
    © 2015 DataStax,All Rights Reserved. Application Driver Address Resolution 33 Application Thread Node Cluster Application Thread Application Thread Client Cluster Node Node Node Address Resolution Policy Control Connection
  • 43.
    © 2015 DataStax,All Rights Reserved. Application Driver Address Resolution 33 Application Thread Node Cluster Application Thread Application Thread Client Cluster Node Node Node Address Resolution Policy Control Connection
  • 44.
    © 2015 DataStax,All Rights Reserved. Application Driver Address Resolution 33 Application Thread Node Pool Cluster Pool Pool Pool Application Thread Application Thread Client Cluster Node Node Node Address Resolution Policy Control Connection Session
  • 45.
    © 2015 DataStax,All Rights Reserved. EC2 Multi-Region Address Resolution 34
  • 46.
    Questions and Links •Documentation • Code • Example Web-app