Your SlideShare is downloading. ×
Building a custom time series db - Colin Hemmings at #DOXLON
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×

Saving this for later?

Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime - even offline.

Text the download link to your phone

Standard text messaging rates apply

Building a custom time series db - Colin Hemmings at #DOXLON

684
views

Published on

Colin talks about how he architected and built a high performance time series database from the ground up at Dataloop.io. Handling hundreds of thousands of metrics per second. One of the objectives …

Colin talks about how he architected and built a high performance time series database from the ground up at Dataloop.io. Handling hundreds of thousands of metrics per second. One of the objectives was to provide real time graphing and alerting. If you're 'rolling your own' metrics, are interested in Node.JS, highly scalable architectures and like listening to plenty of war stories you should enjoy this talk.

Video: http://youtu.be/vx6Ms5TNtqo

DevOps Exchange Meetup Group: http://bit.ly/doxlonmeetup


0 Comments
2 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
684
On Slideshare
0
From Embeds
0
Number of Embeds
7
Actions
Shares
0
Downloads
5
Comments
0
Likes
2
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide

Transcript

  • 1. www.dataloop.io | @dataloopio | info@dataloop.io Colin Hemmings | Architect Time-series Datastore on Riak
  • 2. www.dataloop.io | @dataloopio | info@dataloop.io •Collection •Storage •Analytics Architecture
  • 3. www.dataloop.io | @dataloopio | info@dataloop.io Just stick it in a database, right? The Storage Problem
  • 4. www.dataloop.io | @dataloopio | info@dataloop.io Past Solutions TempoDB - the phantom menace
  • 5. www.dataloop.io | @dataloopio | info@dataloop.io Past Solutions MongoDB - return of the Jedi
  • 6. www.dataloop.io | @dataloopio | info@dataloop.io Riak - Our New Hope • Scales • Ops Friendly • Actually works • No random JVM crashes here
  • 7. www.dataloop.io | @dataloopio | info@dataloop.io Objectives • Handle the load • Semi-arbitrary queries • Data retention windows • Low latency
  • 8. www.dataloop.io | @dataloopio | info@dataloop.io Data structure • Resolution/rollup based queries • Minimum 24 hours at 1 second resolution • Second, minute and hour resolution
  • 9. www.dataloop.io | @dataloopio | info@dataloop.io Data structure • 86,400 data points per resolution • 1 second -> 24 hour retention • 1 minute -> 60 day retention • 1 hour -> 10 year retention
  • 10. www.dataloop.io | @dataloopio | info@dataloop.io Data structure • per metric -> 250k data points • 1000 metric per host -> 2.5M data points • 300 hosts per user -> 750M data points • 1000 customers -> 750B data points!!!!!
  • 11. www.dataloop.io | @dataloopio | info@dataloop.io Simple Riak Storage • Timestamp keyed object per metric value • 2i and MapReduce are too slow • Especially across millions of keys • Writes would soon cripple our Riak cluster
  • 12. www.dataloop.io | @dataloopio | info@dataloop.io Intelligent Riak Storage • Units of storage: time based data blocks • Compute keys • Mutable data windows
  • 13. www.dataloop.io | @dataloopio | info@dataloop.io Query Get cpu metrics for host A for period t1-t4 at 1 second resolution • Pull the correct blocks from riak, based on block boundaries • GET /buckets/host_a/keys/cpu_second_t1b • GET /buckets/host_a/keys/cpu_second_t2b • GET /buckets/host_a/keys/cpu_second_t3b • GET /buckets/host_a/keys/cpu_second_t4b
  • 14. www.dataloop.io | @dataloopio | info@dataloop.io Query • Filter points outside of our query range • Aggregate all the data points • Perform other operation if more complex query
  • 15. www.dataloop.io | @dataloopio | info@dataloop.io Expiring • Cleanup worker • Removes keys out of retention window • Host keyed, easier to clear all hosts or account data
  • 16. www.dataloop.io | @dataloopio | info@dataloop.io Our cluster • Riak 2.0 • 5 nodes on LevelDB • Each 2 x 500GB striped SSDs • Average 1ms GET and PUT latencies
  • 17. www.dataloop.io | @dataloopio | info@dataloop.io
  • 18. www.dataloop.io | @dataloopio | info@dataloop.io Comments • Awesome, especially for ops • A bit more work in application tier • Always compute keys avoid 2i and MapReduce • Looking forward to using the new data types