Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Tailored database
solutions with Voron
Oren Eini
ayende@ayende.com ayende.com/blog
Hibernating Rhinos
Let’s implement a database
 Time series database, but what are those?
 Heart rate monitor watch.
 Every 30 second sends...
Traffic
 Per watch:
 2,880
events per day
 89,280
events per month
 1,051,200
events per year
 Globally (100K watches...
100 billion events / year
 How do we approach this?
Design…
 Each watch has an id (string, up to 40 chars)
 We model each watch as a tree in Voron
 The tree uses:
 Big En...
Let us build this
Great, now about those reads
 What kind of reads we need?
 Always on a specific watch. For a given date range.
Let us build this
What is left?
 Real performance testing…
 Concurrent reads & writes
 Connecting to a server…
 Dropping old data / cons...
Questions?
Upcoming SlideShare
Loading in …5
×

Tailored database solutions: Building a 100 billion records db in 20 minutes

29,429 views

Published on

In this talk, we can see how we can use Voron to build a time series database to store 100 billion records

Published in: Data & Analytics
  • Be the first to comment

Tailored database solutions: Building a 100 billion records db in 20 minutes

  1. 1. Tailored database solutions with Voron Oren Eini ayende@ayende.com ayende.com/blog Hibernating Rhinos
  2. 2. Let’s implement a database  Time series database, but what are those?  Heart rate monitor watch.  Every 30 second sends:  Watch Id, Time, Heart Rate  You sold 100,000 of those.  Users want to log in and see their stats.
  3. 3. Traffic  Per watch:  2,880 events per day  89,280 events per month  1,051,200 events per year  Globally (100K watches):  288,000,000 events per day  8,928,000,000 events per month  105,120,000,000 events per year Just under 3,500 events per second, sustained
  4. 4. 100 billion events / year  How do we approach this?
  5. 5. Design…  Each watch has an id (string, up to 40 chars)  We model each watch as a tree in Voron  The tree uses:  Big Endian date for key  Heart rate number as value  Let us go and build this…
  6. 6. Let us build this
  7. 7. Great, now about those reads  What kind of reads we need?  Always on a specific watch. For a given date range.
  8. 8. Let us build this
  9. 9. What is left?  Real performance testing…  Concurrent reads & writes  Connecting to a server…  Dropping old data / consolidating it.
  10. 10. Questions?

×