Counters with Riak on Amazon EC2 at Hackover

1,848 views

Published on

0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
1,848
On SlideShare
0
From Embeds
0
Number of Embeds
3
Actions
Shares
0
Downloads
9
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • Counters with Riak on Amazon EC2 at Hackover

    1. 1. Distributed Counters with Riak on EC2 Hackover 2012 - hackathon.ro Andrei Savu - asavu@axemblr.com Twitter: @andreisavu
    2. 2. Me• Co-Founder of Axemblr.com• Co-Organiser of Bucharest JUG (bjug.ro)• OSS contributor (Apache Whirr, jclouds)• Connect with me on LinkedIn
    3. 3. ... with Riak on AWS (for fun and profit - real-time analytics)
    4. 4. Riak• Distributed Database (Key-Value)• Focused on: Availability, Fault-Tolerance, Operation Simplicity, Scalability• Each node is the same (no master)• Erlang (OTP)
    5. 5. Consistent Hashing
    6. 6. Riak - More Features• Consistent Hashing• Replication (including cross DC)• Automatic load-balancing• Hinted Handoff• Client-side conflict resolution
    7. 7. Riak Control
    8. 8. Amazon Cloud• Infrastructure as a Service (IaaS) provider• Services I use: • EC2 (compute) • ELB (load balancing) • Route53 (smart dns) • CloudWatch (monitoring)
    9. 9. Service API• GET /counters/<id>• POST /counters/<id> <value>
    10. 10. Eventual Consistency• ... and counters• Client side conflict resolution• counter => {last: 0, step: 0}
    11. 11. Basic Algorithm• {last: 1, step: 0}• +2 => {last: 1+0, step: 2} • Partition 1 • Partition 2• +3 => {last: 1+2, step: 3} • +4 => {last: 1+2, step: 4} • Fixed • Fixed• 1+(2 + 3 + 4) = 6 • read: {last: 3, step: 3 + 4}
    12. 12. Architecture US API ELB Riak w/ cross DC + replicationRoute53 UE API
    13. 13. Demo TimeLocal Machine and on Amazon EC2 (two regions)
    14. 14. Demo Resources• riak-ec2-launcher https://github.com/andreisavu/riak-ec2- cluster-launcher• riak-dt https://github.com/basho/riak_dt
    15. 15. Resources #1• PoC for Convergent Data Types https://github.com/basho/riak_dt• Study on Convergent Data Types http://hal.inria.fr/docs/00/55/55/88/PDF/ techreport.pdf• Gist: https://gist.github.com/882055
    16. 16. Resources #2• statebox - state "monad" for automated conflict resolution https://github.com/mochi/statebox• Convergent Replicated Data Types https://github.com/aphyr/meangirls• Strong Eventual Consistency and Conflict-free Replicated Data Types http://research.microsoft.com/apps/video/ default.aspx?id=153540
    17. 17. Resources #3• Cassandra Counters http://wiki.apache.org/cassandra/Counters• Convergent Replicated Data Types - SRC Fringe https://speakerdeck.com/bryce/convergent- replicated-data-types-src-fringe• http://amattn.com/2012/09/07/ riaks_two_contentions_and_crdts.html
    18. 18. Thanks! Questions? Andrei Savu - asavu@axemblr.com Twitter: @andreisavu

    ×