Wednesday, March 13, 13
#WHOIS                  Adron Hall | @adron | Coder, Messenger, ReconWednesday, March 13, 13
芭蕉Wednesday, March 13, 13
Distributed, masterless, highly-available key/value storeWednesday, March 13, 13
DESIGN GOALS              Fault-Tolerance          Ops Friendliness              Horizontal Scalability   Predictability  ...
Use, when & what to use Riak for...                Users/Profiles             Logging Systems               Metadata      ...
IN PRODUCTION AT                             And 1000s more...Wednesday, March 13, 13
DATA MODELWednesday, March 13, 13
{“Key”:“Value”}        •   Values are stored against keys        •   Key/Value + Metadata = Object        •   Fundamental ...
<<BUCKET>>/<<KEY>>        •   Virtual Namespace        •   Bucket + Keys = Object Address        •   Buckets have properti...
DATA ACCESSWednesday, March 13, 13
INTERFACES             HTTP API - Via a little piece of magic called Webmachine             Largely-faithful REST implemen...
CLIENT LIBS              Python      Java    Clojure              Erlang      PHP     Haskell                             ...
RIAK GIVES YOU                    [FOUR] WAYS TO STORE,                     RETRIEVE, AND QUERY                           ...
CRUD                  // GET          1          2          3          4        GET  /buckets/bucket/keys/key             ...
MapReduce              Distributed processing system using Riak Pipe              Efficient for targeted queries over known...
Secondary Indexing (2i)                 riak_object                 X-Riak-Index-email_bin                                ...
Riak Search             Store and index documents (JSON, text, XML, etc)             Current Riak Search supports subset o...
ARCHITECTURE                                      Consistent Hashing         The scaleability and                         ...
Consistent Hashing             Location of data in the Riak ring is determined based on hash of             bucket + key. ...
Consistent HashingWednesday, March 13, 13
Virtual Nodes             Unit of addressing and concurrency in Riak             Each physical host manage many vnodes    ...
Handoff/Rebalancing             Mechanisms for data rebalancing             When nodes join/leave cluster, handoff and reb...
Vector Clocks             VCs used to rectify object consistency at READ time.             Lots of knobs to turn; well-doc...
Append-Only Storage             Riak provides a pluggable backend interface. (Write your own;             we’ll probably h...
RIAK 1.3               (AKA “new hotness”)              Active Anti Entropy                             Riaknostic include...
FUTURE WORK*                   (1.4 and beyond)                  Dynamic Ring Size          Consistency                  Y...
Multi-tenant cloud storage software for                          public and private clouds.                          Desig...
Extends Riaks capabilities with:                          - multi-datacenter replication                          - SNMP C...
RIAK COMMUNITY              Mailing List - 1300 developers              IRC - 200+ people every day yelling about software...
ricon.io/east.html                                 May 13-14th in New York City                                Talks, hack...
GETTING STARTED        Downloads         - http://docs.basho.com/riak/latest/downloads/        Docs       - http://docs.ba...
Let’s Talk UI & CLI - Demo ThingsWednesday, March 13, 13
#WHOIS                  Adron Hall | @adron | Coder, Messenger, ReconWednesday, March 13, 13
Upcoming SlideShare
Loading in...5
×

Riak intro to..

681

Published on

Published in: Technology
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
681
On Slideshare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
5
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

Riak intro to..

  1. 1. Wednesday, March 13, 13
  2. 2. #WHOIS Adron Hall | @adron | Coder, Messenger, ReconWednesday, March 13, 13
  3. 3. 芭蕉Wednesday, March 13, 13
  4. 4. Distributed, masterless, highly-available key/value storeWednesday, March 13, 13
  5. 5. DESIGN GOALS Fault-Tolerance Ops Friendliness Horizontal Scalability Predictability Low-latency High-AvailabilityWednesday, March 13, 13
  6. 6. Use, when & what to use Riak for... Users/Profiles Logging Systems Metadata Sensor Data Session Storage Notification Systems Object Storage Record SystemsWednesday, March 13, 13
  7. 7. IN PRODUCTION AT And 1000s more...Wednesday, March 13, 13
  8. 8. DATA MODELWednesday, March 13, 13
  9. 9. {“Key”:“Value”} • Values are stored against keys • Key/Value + Metadata = Object • Fundamental Unit of Replication • Any Datatype will work • Record to disk in binary formatWednesday, March 13, 13
  10. 10. <<BUCKET>>/<<KEY>> • Virtual Namespace • Bucket + Keys = Object Address • Buckets have properties • Objects in bucket inherit properties • No relationships between bucketsWednesday, March 13, 13
  11. 11. DATA ACCESSWednesday, March 13, 13
  12. 12. INTERFACES HTTP API - Via a little piece of magic called Webmachine Largely-faithful REST implementation Protocol Buffers API - Thanks, Google! Compact, binary protocolWednesday, March 13, 13
  13. 13. CLIENT LIBS Python Java Clojure Erlang PHP Haskell ...and more via Basho Ruby C/C++ OCaml or our community. Perl .NET Scala Dart Go Node.jsWednesday, March 13, 13
  14. 14. RIAK GIVES YOU [FOUR] WAYS TO STORE, RETRIEVE, AND QUERY DATAWednesday, March 13, 13
  15. 15. CRUD // GET 1 2 3 4 GET  /buckets/bucket/keys/key         5 6 // PUT 7 8 9 10 POST  /buckets/bucket/keys/key        //  Riak-­‐defined  key 11 PUT  /buckets/bucket/keys/key            //  User-­‐defined  key 12 13 // DELETE 14 15 16 17 DELETE  /buckets/bucket/keys/key       18Wednesday, March 13, 13
  16. 16. MapReduce Distributed processing system using Riak Pipe Efficient for targeted queries over known key range Write jobs in Erlang or JS. (Erlang more performant)Wednesday, March 13, 13
  17. 17. Secondary Indexing (2i) riak_object X-Riak-Index-email_bin “mark@basho.com” riak_object X-Riak-Index-value_int “42” Tag objects with custom metadata on PUT... Exact match and range queries... No multi-index queries yet... Pagination is on its way...Wednesday, March 13, 13
  18. 18. Riak Search Store and index documents (JSON, text, XML, etc) Current Riak Search supports subset of Solr API Next iteration (Yokozuna; in beta)will implement distributed Solr on Riak. It will be sexy. Looking for beta testers to help harden YokozunaWednesday, March 13, 13
  19. 19. ARCHITECTURE Consistent Hashing The scaleability and Virtual Nodes ease of operation goals inform Handoff/Rebalancing architectural decisions. Vector Clocks These come with tradeoffs. Append-only storage Active Anti-Entropy*Wednesday, March 13, 13
  20. 20. Consistent Hashing Location of data in the Riak ring is determined based on hash of bucket + key. Provides even distribution of storage and query load Trades off advantages gained from locality - e.g. Range queries and aggregatesWednesday, March 13, 13
  21. 21. Consistent HashingWednesday, March 13, 13
  22. 22. Virtual Nodes Unit of addressing and concurrency in Riak Each physical host manage many vnodes Partition count / physical machines = vnodes/machine* Decouples physical assets from data distribution. This provides: - simplicity in cluster sizing - failure isolationWednesday, March 13, 13
  23. 23. Handoff/Rebalancing Mechanisms for data rebalancing When nodes join/leave cluster, handoff and rebalancing manage the date shuffling dynamically Trades off speed of convergence vs. effects on cluster performance - causes disk & network loadWednesday, March 13, 13
  24. 24. Vector Clocks VCs used to rectify object consistency at READ time. Lots of knobs to turn; well-documented Trades off space, speed, and complexity for safety - will store all sibling objects until resolved - can lead to object size issuesWednesday, March 13, 13
  25. 25. Append-Only Storage Riak provides a pluggable backend interface. (Write your own; we’ll probably hire you...) Bitcask, LevelDB are most-heavily used. Both are append - only Provides crash safety and speed. Trade off: periodic compaction/merge opsWednesday, March 13, 13
  26. 26. RIAK 1.3 (AKA “new hotness”) Active Anti Entropy Riaknostic included by default MapReduce Improvements Riak Control improvements IPv6 Support Much more Full release notes: https://github.com/basho/riak/blob/1.3/RELEASE-NOTES.mdWednesday, March 13, 13
  27. 27. FUTURE WORK* (1.4 and beyond) Dynamic Ring Size Consistency Yokozuna 2i Improvements CRDTs/Data Types Riak Pipe work Riak Object Much more (* all code subject to ship early, late, or not at all)Wednesday, March 13, 13
  28. 28. Multi-tenant cloud storage software for public and private clouds. Designed to provide simple, available, distributed cloud storage at any scale. S3-API compatible and supports per-tenant reporting for billing and metering use cases. Additional APIs on the way. Stores files of arbitrary size. Under the hood stores 1MB chunks along side a manifest. Stateless proxy (CS) does chunking. Riak does distribution, storage, etc.Wednesday, March 13, 13
  29. 29. Extends Riaks capabilities with: - multi-datacenter replication - SNMP Configuration - JMX-Monitoring - 24x7 support from Basho Engineers One cluster acts as a "source cluster". The source cluster replicates its data to one or more "sink clusters" using either real-time or full sync. Data transfer is unidirectional (source -> sink). Bidirectional synchronization can be achieved by configuring a pair of connections between clusters.Wednesday, March 13, 13
  30. 30. RIAK COMMUNITY Mailing List - 1300 developers IRC - 200+ people every day yelling about software GitHub - 1000s of watchers; 200+ contributors to all projects Meetups - 10 Countries, 23 Cities, 3700+ Members & growing fast! Deployments - 1000s in production.Wednesday, March 13, 13
  31. 31. ricon.io/east.html May 13-14th in New York City Talks, hacking, parties Dedicated to the future of Riak and distributed systems in production REGISTER https://ricon-east-2013.eventbrite.com/?discount=lovevnodes NOW!Wednesday, March 13, 13
  32. 32. GETTING STARTED Downloads - http://docs.basho.com/riak/latest/downloads/ Docs - http://docs.basho.com Riak Source Code - github.com/basho/riak All Basho source Code - github.com/basho/ Riak Mailing List - http://bit.ly/riak-list Email or Tweet me @adron or adron@basho.comWednesday, March 13, 13
  33. 33. Let’s Talk UI & CLI - Demo ThingsWednesday, March 13, 13
  34. 34. #WHOIS Adron Hall | @adron | Coder, Messenger, ReconWednesday, March 13, 13
  1. Gostou de algum slide específico?

    Recortar slides é uma maneira fácil de colecionar informações para acessar mais tarde.

×