Michal Matczuk, Software Engineer
Tzach Livyatan, ScyllaDB VP of Product
2
+ The Real-Time Big Data Database
+ Drop-in replacement for Apache Cassandra
and Amazon DynamoDB
+ 10X the performance & low tail latency
+ Open Source, Enterprise and Cloud options
+ Founded by the creators of KVM hypervisor
+ HQs: Palo Alto, CA, USA; Herzelia, Israel;
Warsaw, Poland
Tzach Livyatan, VP of Product
Tzach has a 15 year career in development, system engineering and product
management. He has worked in the Telecom domain, focusing on carrier grade
systems, signalling, policy and charging applications for Oracle and others.
3
Michal Matczuk, Software Engineer
Michal is a software engineer working on Scylla management. He’s a Go enthusiast
and contributor to many open source projects. He has a background in network
programming. Prior joining ScyllaDB he worked with StratoScale and NTT.
4
+ Why and What is Alternator?
+ Why and What is Scylla Operator?
+ Live Demo with an interactive game of Tic Tac Toe
+ Status and Roadmap
+ Q&A
+ Great database
+ Requires operational
expertise
+ Great workload
management platform
Can we leverage Kubernetes to write a great management layer for Scylla ?
5
What is Alternator?
+ Scylla is an efficient NoSQL data store, announced September 2015.
+ It is open source, with enterprise support and cloud (SaaS) options.
+ Compatible with Cassandra and its APIs (CQL, Thrift).
+ Alternator: Adding a DynamoDB API to Scylla.
+ Efficient implementation for modern hardware
+ Throughput 10x higher than Cassandra
+ Linear scalability to many-core machines
+ Focused on modern fast SSDs
+ Low tail latency
+ Reliability
+ Autonomous database (minimal configuration)
Scylla now apply these advantages to AWS DynamoDB API
Better price/performance https://www.scylladb.com/product/benchmarks/dynamodb-benchmark/
+ Managed Scylla (“Scylla Cloud”) is
5 times cheaper than DynamoDB.
➔
➔
➔
➔
➔
➔
10
+ Users want to move their DynamoDB application to
+ a different cloud provider,
+ a private datacenter,
+ or a hybrid of several clouds or data centers.
+ Scylla can be run on any cloud or datacenter.
Fully Supported Mostly Supported No (yet) Supported
Table Operations ✓
Item Operations ✓
Batch Operations ✓
Scans ✓
Tags ✓
Secondary Indexes ✓
TTL ✓
Streams ✓
Transactions ✓
* Alternator: DynamoDB API in Scylla | Scylla Docs
Supported
Authentication ✓
Backup / Restore NC
Tracing NC
Monitoring NC
Alerts NC
* Alternator: DynamoDB API in Scylla | Scylla Docs
docker run --name some-scylla -d -p 8000:8000 scylladb/scylla:4.1.0 --alternator-port=8000
--alternator-write-isolation=always
aws --endpoint-url 'http://127.0.0.1:8000' dynamodb create-table --table-name MusicCollection
--attribute-definitions AttributeName=Artist,AttributeType=S
AttributeName=SongTitle,AttributeType=S --key-schema AttributeName=Artist,KeyType=HASH
AttributeName=SongTitle,KeyType=RANGE --provisioned-throughput
ReadCapacityUnits=5,WriteCapacityUnits=5
16
Scylla On K8s
StatefulSet
Pod
Rack N, Datacenter M
...
Cluster
Custom
Resource
Member
Services
(Static IP)
Controller
Member
Services
(Static IP)
Member
Services
(Static IP)
writewatch
Sidecar
HTTP
StatefulSet
Pod
Rack 1, Datacenter 1
Sidecar
HTTP
StatefulSet
Pod
Rack 1, Datacenter 2
Sidecar
HTTP
18
CRD + Controller + Sidecar
Sidecar,
Manager Agent
HTTP(S)
Sidecar needed to:
+ Setup config files
+ Install plugins at startup
+ Backup and Restore functionality
+ Future extensibility
Member
19
Operator = Controller(s) + CRD(s)
Controller
Observe
Calculate
Reconcile
write
Member/Node Pod
Cluster
Rack
Datacenter
StatefulSet
StatefulSets
Cluster
Custom Resource
(CRD)
21
Deployment
Deploy Scylla Operator
Deploy Scylla cluster
Deploy Scylla Monitoring Stack
Stress the Deployment
Common Procedures
Upgrade from 4.0.0 -> 4.0.1
Scale Down22
+ Deployment - GKE
+ Deploy Scylla Operator
+ Deploy Scylla cluster
+ Deploy Scylla Monitoring Stack
+ Stress the Deployment (Tic Tac Toe!)
+ Common Procedures
+ Scale Down
+ Upgrade from 4.0.0 -> 4.0.1
+ Demo Code Source: AWS docs
23
+ Production ready from Scylla Open Source 4.0
+ Docker, AMI, Scylla Cloud
+ Coming soon to Scylla Enterprise 2020.1.0
+ Scylla Alternator | Scylla Docs
24
Easy to get started: https://github.com/scylladb/scylla-operator
25
+ Topology Changes
+ Proper token ring initialization, with only one node bootstrapping at a time
+ Down scale
+ Out Scale
+ Cluster Management
+ Managed Configuration, including rolling update
+ Rolling Upgrades / Downgrades
+ Integrations
+ Prometheus / Grafana monitoring Stack
+ Secured
26
+ Multiple Data Centers Support
+ Auto Scaling
+ CI/CD
+ Support CSR
+ Scylla manager integration (repairs, backups)
+ Performance benchmarks on EKS, GKS
+ Scylla as K8s Service
+ Helm packaging
+ Your Feature Here!
27
Tzach Livyatan Michal Matczuk
28
henrik@scylladb.com
tzach@scylladb.com
29
United States Israel www.scylladb.com
@scylladb
+ forbid_rmw – Forbids write requests which require a read before a write.
Will return an error on read-modify-write operations, e.g., conditional updates (UpdateItem with a
ConditionExpression).
+ only_rmw_uses_lwt – Isolates only updates that require read-modify-write.
Use this setting only if you do not mix read-modify-write and write-only updates to the same item,
concurrently.
+ always – Isolate every write operation, even those that do not need a read before the write (default
in 4.0)
--tags Key=system:write_isolation,Value=always

Running a DynamoDB-compatible Database on Managed Kubernetes Services

  • 1.
    Michal Matczuk, SoftwareEngineer Tzach Livyatan, ScyllaDB VP of Product
  • 2.
    2 + The Real-TimeBig Data Database + Drop-in replacement for Apache Cassandra and Amazon DynamoDB + 10X the performance & low tail latency + Open Source, Enterprise and Cloud options + Founded by the creators of KVM hypervisor + HQs: Palo Alto, CA, USA; Herzelia, Israel; Warsaw, Poland
  • 3.
    Tzach Livyatan, VPof Product Tzach has a 15 year career in development, system engineering and product management. He has worked in the Telecom domain, focusing on carrier grade systems, signalling, policy and charging applications for Oracle and others. 3 Michal Matczuk, Software Engineer Michal is a software engineer working on Scylla management. He’s a Go enthusiast and contributor to many open source projects. He has a background in network programming. Prior joining ScyllaDB he worked with StratoScale and NTT.
  • 4.
    4 + Why andWhat is Alternator? + Why and What is Scylla Operator? + Live Demo with an interactive game of Tic Tac Toe + Status and Roadmap + Q&A
  • 5.
    + Great database +Requires operational expertise + Great workload management platform Can we leverage Kubernetes to write a great management layer for Scylla ? 5
  • 6.
  • 7.
    + Scylla isan efficient NoSQL data store, announced September 2015. + It is open source, with enterprise support and cloud (SaaS) options. + Compatible with Cassandra and its APIs (CQL, Thrift). + Alternator: Adding a DynamoDB API to Scylla.
  • 8.
    + Efficient implementationfor modern hardware + Throughput 10x higher than Cassandra + Linear scalability to many-core machines + Focused on modern fast SSDs + Low tail latency + Reliability + Autonomous database (minimal configuration) Scylla now apply these advantages to AWS DynamoDB API
  • 9.
    Better price/performance https://www.scylladb.com/product/benchmarks/dynamodb-benchmark/ +Managed Scylla (“Scylla Cloud”) is 5 times cheaper than DynamoDB.
  • 10.
  • 11.
    + Users wantto move their DynamoDB application to + a different cloud provider, + a private datacenter, + or a hybrid of several clouds or data centers. + Scylla can be run on any cloud or datacenter.
  • 12.
    Fully Supported MostlySupported No (yet) Supported Table Operations ✓ Item Operations ✓ Batch Operations ✓ Scans ✓ Tags ✓ Secondary Indexes ✓ TTL ✓ Streams ✓ Transactions ✓ * Alternator: DynamoDB API in Scylla | Scylla Docs
  • 13.
    Supported Authentication ✓ Backup /Restore NC Tracing NC Monitoring NC Alerts NC * Alternator: DynamoDB API in Scylla | Scylla Docs
  • 14.
    docker run --namesome-scylla -d -p 8000:8000 scylladb/scylla:4.1.0 --alternator-port=8000 --alternator-write-isolation=always aws --endpoint-url 'http://127.0.0.1:8000' dynamodb create-table --table-name MusicCollection --attribute-definitions AttributeName=Artist,AttributeType=S AttributeName=SongTitle,AttributeType=S --key-schema AttributeName=Artist,KeyType=HASH AttributeName=SongTitle,KeyType=RANGE --provisioned-throughput ReadCapacityUnits=5,WriteCapacityUnits=5
  • 16.
  • 17.
  • 18.
    StatefulSet Pod Rack N, DatacenterM ... Cluster Custom Resource Member Services (Static IP) Controller Member Services (Static IP) Member Services (Static IP) writewatch Sidecar HTTP StatefulSet Pod Rack 1, Datacenter 1 Sidecar HTTP StatefulSet Pod Rack 1, Datacenter 2 Sidecar HTTP 18
  • 19.
    CRD + Controller+ Sidecar Sidecar, Manager Agent HTTP(S) Sidecar needed to: + Setup config files + Install plugins at startup + Backup and Restore functionality + Future extensibility Member 19
  • 20.
    Operator = Controller(s)+ CRD(s) Controller Observe Calculate Reconcile write
  • 21.
  • 22.
    Deployment Deploy Scylla Operator DeployScylla cluster Deploy Scylla Monitoring Stack Stress the Deployment Common Procedures Upgrade from 4.0.0 -> 4.0.1 Scale Down22 + Deployment - GKE + Deploy Scylla Operator + Deploy Scylla cluster + Deploy Scylla Monitoring Stack + Stress the Deployment (Tic Tac Toe!) + Common Procedures + Scale Down + Upgrade from 4.0.0 -> 4.0.1 + Demo Code Source: AWS docs
  • 23.
  • 24.
    + Production readyfrom Scylla Open Source 4.0 + Docker, AMI, Scylla Cloud + Coming soon to Scylla Enterprise 2020.1.0 + Scylla Alternator | Scylla Docs 24
  • 25.
    Easy to getstarted: https://github.com/scylladb/scylla-operator 25
  • 26.
    + Topology Changes +Proper token ring initialization, with only one node bootstrapping at a time + Down scale + Out Scale + Cluster Management + Managed Configuration, including rolling update + Rolling Upgrades / Downgrades + Integrations + Prometheus / Grafana monitoring Stack + Secured 26
  • 27.
    + Multiple DataCenters Support + Auto Scaling + CI/CD + Support CSR + Scylla manager integration (repairs, backups) + Performance benchmarks on EKS, GKS + Scylla as K8s Service + Helm packaging + Your Feature Here! 27
  • 28.
  • 29.
  • 30.
    United States Israelwww.scylladb.com @scylladb
  • 31.
    + forbid_rmw –Forbids write requests which require a read before a write. Will return an error on read-modify-write operations, e.g., conditional updates (UpdateItem with a ConditionExpression). + only_rmw_uses_lwt – Isolates only updates that require read-modify-write. Use this setting only if you do not mix read-modify-write and write-only updates to the same item, concurrently. + always – Isolate every write operation, even those that do not need a read before the write (default in 4.0) --tags Key=system:write_isolation,Value=always