YugaByte + PKS CloudFoundry Meetup 10/15/2018

1© 2018 All rights reserved.
Introducing YugaByte DB +
PKS
Alan Caldera, Sr. Solutions Architect
October 15, 2018

About Us
Kannan Muthukkaruppan, CEO
Nutanix ♦ Facebook ♦ Oracle
IIT-Madras, University of California-Berkeley
Karthik Ranganathan, CTO
Nutanix ♦ Facebook ♦ Microsoft
IIT-Madras, University of Texas-Austin
Mikhail Bautin, Software Architect
Clear Story Data ♦ Facebook ♦ D.E.Shaw
Nizhny Novgorod State University, Stony Brook
 Founded Feb 2016
 Scaled the platform to serve many mission-critical use cases
• Facebook Messages - Distributed OLTP
• Facebook’s Operational Data Store - Fast Data
• Fraud Detection
• Site Integrity
 Early contributors to Cassandra even before it was open-sourced
 Created HBase - Facebook’s NoSQL platform
Founders
$24M Funding From Leading VCs in Cloud Infrastructure
Ravi Mhatre
Founding Investor at Nutanix ♦ AppDynamics ♦ Mulesoft
Lightspeed Venture Partners
.. and prominent angels
Deepak Jeevankumar
Managing Director,
Dell Technologies Capital

YugaByte Story

YugaByte story starts with….

Facebook in 2007

Facebook in 2008-2009
How to scale to a billion users?
Also: how to survive the week?

What happens at 1 Billion users?
Dozens of Petabytes
Billions of IOPS
Scale out frequently
Rolling upgrades – zero downtime!

WHAT IS YUGABYTE?

A transactional, high-performance database
for building planet-scale cloud services.

Bring SQL + NoSQL into ONE DB
SQL
Strong consistency
Secondary indexes
ACID transactions
Expressive query language
NoSQL
Tunable read latency
Write optimized for large data sets
Data expiry with TTL
Scale out and fault tolerant

Multi-API, Multi-Model
BETA
Distributed SQL DB
Linear Write & Read Scaling
Auto-Sharded/Auto-Rebalanced
Fault-Tolerant, Self-Healing
RELATIONAL
Low Latency & High Throughput
Distributed & Single Row ACID Txns
Consistent Secondary Indexes
Native JSON Data Type
FLEXIBLE SCHEMA
Cassandra++
Cassandra-compatible and more
Linear Scaling – Not Memory Bound
Auto-Sharded/Auto-Rebalanced
Fault-Tolerant, Self-Healing
Native TimeSeries Data Type
KEY-VALUE
Redis++
Redis-compatible and more

WHY ANOTHER DB?

Typical Stack Today
Fragile infrastructure with several moving parts
Datacenter 1
SQL Master SQL Slave
Application Tier (Stateless Microservices)
Datacenter 2
SQL for OLTP data
Manual sharding
Cost: dev team
Manual replication
Manual failover
Cost: ops team
NoSQL for other data
App aware of data silo
Cost: dev team
Cache for low latency
App does caching
Cost: dev team
Data inconsistency/loss
Fragile infrastructure
Hours of debugging
Cost: dev + ops team

Does AWS change this?
Datacenter 1
SQL Master SQL Slave
Datacenter 2
Elasticache
Aurora
DynamoDB
Still Complex
it’s the same architecture

How did the tech leaders simplify this?
Custom Data Platform
Transactional, Performant, Global
But there’s no general platform for the enterprise

tablet 1’
tablet 1’
YugaByte DB
Developer Agility, Operational Simplicity, Multi-Cloud
tablet 1’
YCQL
Cassandra-compatible
YEDIS
Redis-compatible BETA
Self-Healing, Fault-Tolerant
Auto Sharding & Rebalancing
Distributed ACID Transactions
Global Data Distribution
High Throughput

YugaByte DB Overview
tablet 1’
tablet3-leader
tablet2-leader
tablet1-leader
…
…
…
tablet2-follower
tablet2-follower
tablet3-follower tablet3-follower
tablet1-follower
tablet1-follower
SMACK
Apps … Mature ecosystems
tablet 1’
tablet 1’
tablet 1’
DocDB Storage
Transactional key-document store, based on a
heavily customized version of RocksDB
Raft-Based Replication
For data replication & leader election
n1 n2
n3
Common underlying DB engine
Transaction Manager
Tracks ACID txns across multi-row ops, incl. clock skew mgmt.
tablet1-leader
tablet2-leader
tablet3-leader
tablet2-follower
tablet3-follower
tablet2-follower
tablet3-follower
tablet1-follower
tablet1-follower
…
……
Automated Sharding & Load Balancing
YCQL
Cassandra-compatible
YEDIS
Redis-compatible BETA

Not Portable
Not Portable
Open Source
Not Portable
Open Source
Open Source
High Performance, Transactional, Planet-Scale High Performance, Transactional, Planet-Scale
High Performance, Transactional, Planet-Scale High Performance, Transactional, Planet-Scale
System-of-Record DBs for Global Apps

ACID Transactions
Globally Consistent
SQL API only
Not Transactional
Multi-Model
High Performance
Best of Cloud-Native Meets Open Source
Not Globally Consistent
Lower Performance

YUGABYTE + PKS

Basic PKS Integration Done

PKS + YugaByte Status
- Current state - basic integration is done
- Single zone deployments
- Capable of creating multiple clusters
- User controlled service accounts
- Enforcing resource limits
- Next steps
- Day 2 operations (backups, alerts, etc)
- Multi-zone and multi-DC deployments

YugaByte Broker Status – GA
- Current State
- Currently functional for most common flows
- Supports basic authentication
- Next steps
- SSL support
- Support for Kubernetes-specific functionality

DEMO

SIMPLIFIED APP DEV

Distributed ACID Transactions
Multi-Row/Multi-Shard Operations At Any Scale
YCQL

Secondary Indexes
Consistent & Low Latency
YCQL

Native JSON Data Type
Modeling document & flexible schema use-cases
YCQL

Auto Data Expiry with TTL
Database tracks and expires older data
YCQL YEDIS
Query the key right away
Query the key after 10 seconds
Write a key with a 10 second expiry

Native TimeSeries Data Type
Fine grained control on expiry of each record
YEDIS
Insert time-value data
Fine-grained expiry of each time-value pairQuery data in time windows
Delete time-value pairs

Spark Integration for AI/ML
Realtime analytics on top of transactional data without ETL
YCQL
1
2
3

Tunable Read Latency
Primary Cluster
(Region1 in Cloud1)
Read Replica Cluster
(Region2 in Cloud2)
tablet1-leader
tablet1-follower
tablet1-observer
App clients reading a key in tablet1
3. Strong Read
(Default)
1. Local Region Read
2. Follower Read
Async Replication

REAL-WORLD CASE STUDIES

1. MySQL master-slave replication
2. Cassandra cross-DC queue for cache updates
3. Per-DC Couchbase for caching
Current State
Case Study #1 – Global User Identity
Login, change password, view profile

With YugaByte DB
Case Study #1 – Global User Identity
Login, change password, view profile
Unified platform
Zero data loss
even on region failures
Add new regions with ease
1-click Deployment of Primary Cluster and Read Replicas
Read Replicas

Redis cluster for low latency reads
Fragile (manually sharded & load balanced)
Expensive (entire dataset in memory)
On-premises only, need hybrid/public cloud scaling and distribution
D
B
Current State
Case Study #2 - Financial Data Service

With YugaByte DB
Case Study #2 - Financial Data Service
Higher release velocity
Cost-efficient storage
Faster cloud migration
1-click Deploy of Redis as a Primary Database

Case Study #3 - Fast Data – The SKY Stack
Devices
Sensors
Apps
Event
Bus
Fast
Analytics
Event
Processing
Streaming Events to Business Insights .. in Real-Time
Reliable, Elastic
DB
Read the data
Write the model
Read the model
Streaming/Time Series
Events
Alerts/Notifications
Real-Time
Dashboards
Data
Modeling

Based on current customers and prospects
Other Interesting Use-Cases
Real-time analytics with Spark/Presto on YugaByte
API Rate-limiting using Redis TTL
User notifications based on events and preferences
Add a unique constraint on a column
• Massive data set and write operations
• Migration from RDBMS (AWS RDS)
Fraud detection
• High write throughput
• Weak secondary indexes were ok

Winning Customer Trust

Current State (Sept 2018)

Source: https://blog.yugabyte.com/building-a-strongly-consistent-cassandra-with-better-performance-aa96b1ab51d6
• Better than the most
performant NoSQL DBs
• 2x-5x better performance vs
Cassandra
• 5x-10x better data density vs
Cassandra
Performance - YugaByte DB 0.9 vs Existing NoSQL
Comparing against a high performance DB like Cassandra

• Stress tested up to 50 node clusters (blog link)
• Scales linearly for reads and writes
• Auto sharding and rebalancing
Performance – Linear Scale

• Tested up to 4.5TB/node test (read more)
• On cheaper, gp2 SSD on AWS!
• 18TB data set on 4 nodes (c4.4xlarge type)
• Scaled from 4-nodes to 5-nodes with ease
• New node operational as soon as first tablets moved
• Entire rebalance took only about 7 hrs
• Expected: Because gp2 SSD supports about 150MB/s max
• With Apache/DataStax Cassandra’s eventual consistency
• same operation can take days
• Above 1TB node is not recommended
Performance – Data Density

• Support cheaper tiers of storage
(HDD, Object Stores like AWS S3, etc.)
• Page-in hot data in a fine-grained manner
Tiered Storage (Roadmap)
Automatic support for data tiering

DEVELOPMENT FOCUSED
• Apache Spark
• Presto
• SpringBoot Ecosystem
• JanusGraph (Graph API)
• KairosDB (Timeseries)
• Kafka connector coming soon
Ecosystem Integrations
Supports the following integrations
OPERATOR FOCUSED
• Backups to AWS S3, SAN/NAS
• Route53/DNS based discovery
• Managed Kubernetes Support coming
soon (GKE, PKS)
• TLS Encryption
• Encryption at rest being worked on.

HIGHLY EFFICIENT
CLOUD OPS

Multi-Region Deployments in Minutes

Multi-Cloud

ARCHITECTURE
Overview

Process overview
• Universe = cluster of nodes
• Two sets of processes: YB-Master & YB-TServer
• Example universe
4 nodes
rf=3

Sharding data
• User table split into tablets

One tablet for every key

Tablets and replication
• Tablet = set of tablet-peers in a RAFT group
• Num tablet-peers in tablet = replication factor (RF)
Tolerate 1 failure : RF=3
Tolerate 2 failures: RF=5

YugaByte Query Layer (YQL)
• Stateless, runs in each YB-TServer process

YB-TServer
• Process that does IO
• Hosts tablet for tables
• Hosts transaction manager
• Auto memory sizing
Block cache
Memstores

YB-Master
• Not in critical path
• System metadata store
Keyspaces, tables, tablets
Users/roles, permissions
• Admin operations
Create/alter/drop of tables
Backups
Load balancing (leader and data balancing)
Enforces data placement policy

ARCHITECTURE
Data Persistence

Data Persistence in DocDB
• DocDB is YugaByte DB’s LSM storage engine
• Persistent key to document store
• Extends and enhances RocksDB
• Designed to support high data-densities per node

DocDB: Key-to-Document Store
• Document key
CQL/SQL/Redis primary key
• Document value
a CQL or SQL row
Redis data structure
• Fine-grained reads and writes

DocDB Data Format
Example Insert
Encoding

Some of the RocksDB enhancements
• WAL and MVCC enhancements
o Removed RocksDB WAL, re-uses Raft log
o MVCC at a higher layer
o Coordinate RocksDB memstore flushing and Raft log garbage collection
• File format changes
o Sharded (multi-level) indexes and Bloom filters
• Splitting data blocks & metadata into separate files for tiering support
• Separate queues for large and small compactions

More Enhancements to RocksDB
• Data model aware Bloom filters
• Per-SSTable key range metadata to optimize range queries
• Server-global block caches & memstore limits
• Scan-resistant block cache (single-touch and multi-touch)

ARCHITECTURE
Data Replication

Raft Related Enhancements
• Leader Leases
• Multiple Raft groups (1 per tablet)
• Leader Balancing
• Group Commits
• Observer Nodes / Read Replicas

Raft Extension: Leader Leases
Tablet Peer
(old leader)
Tablet Peer
(new leader)
Tablet Peer
(follower)
x=10 x=10
x=10
Network partition
Client writes x=20, and the new
leader replicates it
Client
Without leader leases: the client can still reach the old leader, read x=10
1
2
4
3x=20
x=20

Raft Extension: Leader Leases
TimeTablet Server 1 is the leader of a tablet
Leader lease
Tablet server 2 becomes leader,
cannot take load until the old
leader’s lease expires
Tablet Server 2 is a follower
Tablet Server 1
Tablet Server 2

Winning Customer Trust

ARCHITECTURE
SQL ON YUGABYTE

Distributed SQL Support – PostgreSQL API
KEY POINTS
• Fully wire-compatible with PG
• Re-using open-source codebase
(replace table store in PG)
• Q4 2018: Work to feature coverage
(build meaningful apps)
• Q1 2019: SQL in production
(staging to production)
WHAT’S SUPPORTED?
• Most data types
• Common for most queries
(joins, views, indexes)
• Next steps:
• Stored procedures
• Triggers

Changes to PostgreSQL
CLIENT Postman
(Authentication, authorization)
Rewriter Planner
OptimizerExecutor
WAL Writer BG Writer…
DISK
Reuse
Stateless
PostgreSQ
L

CLIENT Postman
Rewriter Planner
OptimizerExecutor
YugaByte Node
Reuse
Stateless
PostgreSQL
YugaByte Node ……
Replace table
storage with
YugaByte DB
YugaByte Node

CLIENT Postman
Rewriter Planner
OptimizerExecutor
YugaByte Node
Reuse
Stateless
PostgreSQL
YugaByte Node ……
Replace table
storage with
YugaByte DB
YugaByte Node
Enhance
optimizer and
executor for
distributed DB

ARCHITECTURE
Transactions

Single Shard Transactions
Raft Consensus Protocol
. . .
INSERT INTO T (k, v) VALUE (‘x’, 10) IF NOT EXISTS Lock Manager
(in memory, on leader only)
Acquire a lock on x
DocDB / RocksDB
Read current value of x
Submit a Raft operation for replication:
set x=10 at hybrid_time 100
Raft log
Tablet
follower
Tablet
follower
Replicate to
majority of
tablet peers
Apply to RocksDB and
release lock
x=10
@ht=100
1
2
5
3
4

MVCC for Lockless Reads
• Achieved through HybridTime (HT)
Monotonically increasing timestamp
• Allows reads at a particular HT without locking
• Multiple versions may exist temporarily
Reclaim older values during compactions

Single Shard Transactions
• Each tablet maintains a “safe time” for reads
o Highest timestamp such that the view as of that timestamp is fixed
o In the common case it is just before the hybrid time of the next
uncommitted record in the tablet

Distributed Transactions
• Fully decentralized architecture
• Every tablet server can act as a Transaction Manager
• A distributed Transaction Status table
Tracks state of active transactions
• Transactions can have 3 states:
pending, committed, aborted

Distributed Transactions – Write Path

Distributed Transactions – Write Path Step 1: Client request

Distributed Transactions – Write Path Step 2: Create status record

Distributed Transactions – Write Path Step 3: Write provisional records

Distributed Transactions – Write Path Step 4: Atomic commit

Distributed Transactions – Write Path Step 5: Respond to client

Distributed Transactions – Write Path Step 6: Apply provisional records

Isolation Levels
• Currently Snapshot Isolation is supported
o Write-write conflicts detected when writing provisional records
• Serializable isolation (roadmap)
o Reads in RW txns also need provisional records
• Read-only transactions are always lock-free

Clock Skew and Read Restarts
• Need to ensure the read timestamp is high enough
o Committed records the client might have seen must be visible
• Optimistically use current Hybrid Time, re-read if necessary
o Reads are restarted if a record with a higher timestamp that the client
could have seen is encountered
o Read restart happens at most once per tablet
o Relying on bounded clock skew (NTP, AWS Time Sync)
• Only affects multi-row reads of frequently updated records

Distributed Transactions – Read Path

Distributed Transactions – Read Path Step 1: Client request; pick ht_read

Distributed Transactions – Read Path Step 2: Read from tablet servers

Distributed Transactions – Read Path Step 3: Resolve txn status

Distributed Transactions – Read Path Step 4: Respond to YQL Engine

Distributed Transactions – Read Path Step 5: Respond to client

Distributed Transactions – Conflicts & Retries
• Every transaction is assigned a random priority
• In a conflict, the higher-priority transaction wins
o The restarted transaction gets a new random priority
o Probability of success quickly increases with retries
• Restarting a transaction is the same as starting a new one
• A read-write transaction can be subject to read-restart

ROADMAP

CORE DB PLANET-SCALESECURITY
SQL support
Jepsen testing
Authentication, authorization
Encryption (at rest, transit) Tiered storage
Multiple read-replica clusters
YugaByte DB Roadmap
CLOUD-NATIVE
Managed Kubernetes integration
(PKS, GKE)
Continuous backups
OPEN-SOURCE
Ecosystem Integration
(Improved Kafka, Spring Data integration)
Client Driver Support for new types & perf
(Optimizations for Go, C++, C#, drivers)

Questions?
Try it at docs.yugabyte.com/quick-start

YugaByte + PKS CloudFoundry Meetup 10/15/2018

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Similar to YugaByte + PKS CloudFoundry Meetup 10/15/2018

Similar to YugaByte + PKS CloudFoundry Meetup 10/15/2018 (20)

Recently uploaded

Recently uploaded (20)

YugaByte + PKS CloudFoundry Meetup 10/15/2018

Editor's Notes