The Next Chapter in the Sordid Love/Hate Relationship Between DBs a...

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

A ScyllaDB Community
The Next Chapter in the Sordid
Love/Hate Relationship Between
DBs and OSes
Andy Pavlo
Associate Profe...

The Next Chapter in the Sordid
Love/Hate Relationship
Between Databases and
Operating Systems
andy_pavlo
@andy_pavlo

DATABASE PEOPLE HATE THE OS
The bottom line is that operating
system services in many existing
systems are either too slow...

OS PEOPLE HATE DATABASES
[Async I/O] is a horrible ad-hoc
design… and we are implementing
it for compatibility because
dat...

DATABASE PEOPLE HATE THE OS
Linux tends to kill the postmaster in
out-of-memory situations, because it
blames the postmast...

CONFLICT AVOIDANCE
Databases are the most important software system
in the world that just want to help people store and
q...

Kernel-Bypass
OS→DBMS

KERNEL-BYPASS
8

KERNEL-BYPASS
8

KERNEL-BYPASS
8
SPDK
DPDK

KERNEL-BYPASS
Challenge #1: Must reimplement
protocols in user-space.
Challenge #2: Difficult to debug,
deploy, and mainta...

KERNEL-BYPASS
Challenge #1: Must reimplement
protocols in user-space.
Challenge #2: Difficult to debug,
deploy, and mainta...

User-Bypass
DBMS→OS
9

USER-BYPASS

USER-BYPASS

USER-BYPASS
Instead of pulling DBMS data to user-
space, push DBMS logic to kernel-
space.
Avoid copying buffers, scheduli...

EXTENDED BERKELEY PACKET FILTER
Framework to support safe, event-
driven programs in kernel-space
→ Available in Linux ker...

EXTENDED BERKELEY PACKET FILTER
Framework to support safe, event-
driven programs in kernel-space
→ Available in Linux ker...

How can Databases
Leverage eBPF?

TALK OUTLINE
Stateless Systems
→ Database Proxies / Middleware
Stateful Systems
→ Transactional Key-Value DBMS
15

DBMS PROXY CONNECTION POOLING
16

DBMS PROXY CONNECTION POOLING
16

USER-SPACE PROXY: BOTTLENECKS
Research shows ~50% of CPU cycles
in Linux networking stack is spent in
memcpy() calls!
Push...

TIGGER: USER-BYPASS PROXY
Fork of PgBouncer that leverages
eBPF's zero-copy kernel APIs to avoid
syscalls for most proxy c...

CONNECTION POOLING THROUGHPUT
22
33.1 31.0
36.6 36.9 38.9
32.1
38.6
43.6 44.0
46.9
45.4
48.1
50.8 49.7 51.8
0
20
40
60
lar...

CONNECTION POOLING THROUGHPUT
22
33.1 31.0
36.6 36.9 38.9
32.1
38.6
43.6 44.0
46.9
45.4
48.1
50.8 49.7 51.8
0
20
40
60
lar...

ROADMAP
Stateless Systems
→ Database Proxies / Middleware
Stateful Systems
→ Transactional Key-Value DBMS
23

BPF-DB
In-kernel data store with serializable
transactions (ACID) for operational
workloads.
→ Allow developers to build f...

BPF-DB: ARCHITECTURE
25

BPF-DB: ARCHITECTURE
25

THE VERIFIER STRIKES BACK
eBPF's Verifier limits programs to 1M instructions.
→ It explores all a program's branches and l...

27
GET TAIL-CALLS
eBPF Programs
→ Protocol Handlers
→ DBMS Processors

STORAGE MANAGEMENT
Store database contents in kernel-
resident thread-safe data structures
Challenges:
→ No heap ⇒ no mall...

STORAGE MANAGEMENT
Store database contents in kernel-
resident thread-safe data structures
Challenges:
→ No heap ⇒ no mall...

STORAGE MANAGEMENT
Store database contents in kernel-
resident thread-safe data structures
Challenges:
→ No heap ⇒ no mall...

TRANSACTION MANAGEMENT
Support multi-statement transactions
that ensure ACID properties.
Challenges:
→ Restrictive atomic ...

TRANSACTION MANAGEMENT
Support multi-statement transactions
that ensure ACID properties.
Challenges:
→ Restrictive atomic ...

TRANSACTION MANAGEMENT
Support multi-statement transactions
that ensure ACID properties.
Challenges:
→ Restrictive atomic ...

TRANSACTION MANAGEMENT
Support multi-statement transactions
that ensure ACID properties.
Challenges:
→ Restrictive atomic ...

DURABILITY
Persist database contents to disk via
write-ahead logging.
Challenges:
→ eBPF programs cannot initiate disk acc...

EXPERIMENTAL SETUP
Compare against other in-memory databases:
Workload:
→ YCSB-like (1KB values) via Redis' memtier_benchm...

KVS SERVER THROUGHPUT
32
0
0.5
1
1.5
2
0% 20% 40% 60% 80% 100%
Throughput
(op/sec)
SET Percentage
×1,000,000
Higher is Bet...

KVS SERVER THROUGHPUT
32
0
0.5
1
1.5
2
0% 20% 40% 60% 80% 100%
Throughput
(op/sec)
SET Percentage
×1,000,000
Higher is Bet...

KVS SERVER THROUGHPUT
32
Redis KeyDB Dragonfly BPF-DB
Percentage
Latency (ms)
10% SET + 90% GET
Zipfian Distribution

CONCLUSION
https://doi.org/10.14778/3659437.36594
eBPF allows a DBMS to live a better life.
User-bypass is a better approa...

END
pavlo@cs.cmu.edu

Share Slideshare

LinkedIn
Facebook
Twitter

Embed

Size (px)

Show related Slideshows at end

WordPress Shortcode

Link

Share
Email

2024 Trend Updates: What Really Wor... by Search Engine Jou... 1063677 views
Storytelling For The Web: Integrate... by Chiara Aliotta 982505 views
Artificial Intelligence, Data and C... by OECD Directorate ... 924689 views
How to Leverage AI to Boost Employe... by SocialHRCamp 392956 views
2024 State of Marketing Report – by... by Marius Sescu 222263 views
Everything You Need To Know About C... by Expeed Software 237949 views

View on Slideshare

1 of 47 Ad

View on Slideshare

1 of 47 Ad