WebWorkersCamp 2010

1,668 views

Published on

Presentation from my talk at WebWorkersCamp 2010, probably a bit hard to follow with the slides only.

Published in: Technology, News & Politics
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
1,668
On SlideShare
0
From Embeds
0
Number of Embeds
4
Actions
Shares
0
Downloads
10
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide






















































  • WebWorkersCamp 2010

    1. 1. Click & Tune What’s new in our architectures ? Olivier Gutknecht olg@no-distance.net / twitter.com/olg
    2. 2. Me Apple alumnus Fotopedia co-founder & alumnus Author, french ebook - introduction to REST http://izibook.eyrolles.com/
    3. 3. Me Apple alumnus Fotopedia co-founder & alumnus Author, french ebook - introduction to REST http://izibook.eyrolles.com/
    4. 4. Click & Tune ?
    5. 5. START
    6. 6. Getting in tune
    7. 7. Getting in tune We have a lot of sliders, faders and buttons to tweak. And we often ignore most of them. Especially at architecture level.
    8. 8. What changed ?
    9. 9. What changed ? Concepts | Context | Implementations
    10. 10. Concepts | Context | Implementations “The Google File System” / Google 2003 “MapReduce: Simplified Data Processing on Large  Clusters” / Google 2004 “Life beyond Distributed Transactions: an Apostate’s Opinion” / Pat Helland, Amazon 2007 [ZFS, Dynamo, PNUTS, CAP, REST,...] papers
    11. 11. Concepts | Context | Implementations Client and apps evolution: html5, native apps, COMET, web of [docs,apps], ... Modeling problems: tags, social graphs, walls, feeds, ... “Like the big boys” - brought legitimacy
    12. 12. Concepts | Context | Implementations Nutch - 2003 Memcached - 2003 Rails - 2004 CouchDB - 2005 S3 & EC2 - 2006 Redis, RabbitMQ, MongoDB, Node.JS...
    13. 13. Sliders ? # of backend servers cache size fsync / checkpointing invalidation delay
    14. 14. interesting, but...
    15. 15. Sliders Consistency level Sync/Async mix R/W actions balance Model heterogeneity
    16. 16. Trade-offs
    17. 17. It’s the app, really. And use cases. And business cases.
    18. 18. Latency “Disk is the new tape” Cache ? invalidation & cold starts ...
    19. 19. Read Reads are a memory & locality problem. Caches. Partitions. Dupes. Smart shards.
    20. 20. Write Write now. Work later. (maybe not on reads) Divide and Conquer. And cheat. Our frameworks are not ready yet.
    21. 21. Trade-offs
    22. 22. Acceptable Latency Mandatory consistency (and transactions) Synchronous actions Maintainability (Cost)
    23. 23. NoSQL is not about No SQL. It’s about revisiting assumptions. CAP ? ACID ? Choose your poisons. (and enjoy your sliders, yay for Cassandra)
    24. 24. Consistency Poison #1
    25. 25. Partition Tolerance Poison #2
    26. 26. Availability
    27. 27. Ditto for async.
    28. 28. The state of art is not good enough. (we’ve been stuck in the same local minima for several years) Ilya Grigorik
    29. 29. Yay for node.js - and Grand Central - and EM - and Goliath - and JRuby - and ...
    30. 30. Ditto for batches.
    31. 31. Hadoop. Pig. Hive. Sqoop. Flume. HBase.
    32. 32. So ?
    33. 33. Full Stack
    34. 34. HTTP everywhere ? & AMQP ? & Thrift ? ProtoBufs ? Avro ?
    35. 35. AUT OMA
    36. 36. Because our stacks are state of the art (or bleeding edge) Beyond Packages Global config Change Do you really want to ssh to your servers ?
    37. 37. “But we have very specific needs”
    38. 38. You’re 10% unique Jesse Robbins, Opscode CEO
    39. 39. You’re 10% unique (and it’s probably the things you did wrong) Jesse Robbins, Opscode CEO
    40. 40. The Cloud What’s great about it ? Scalability ? On-demand capacity ? Reduced CapEx ?
    41. 41. “Everything fails all the time” Werner Vogels (Amazon CTO)
    42. 42. The Cloud The best way to discover new problems. (and it’s great)
    43. 43. The Cloud Think stateless. Think automation. Think latency. Think crappy machines. Think lost servers. Thinks lost disks. Think offline datacenters. Think ACL. Think security. Think autoconf.
    44. 44. “MyPAAS”
    45. 45. Think Holistic
    46. 46. Speed. Availability. Maintainability.
    47. 47. Know the limits of your tools
    48. 48. Episode IV: a New Hope C h e f , P u p p e t , D r i z z l e , C a s s a n d ra , MongoDB, Redis, HTTP, Rails, Node.js, EventMachine, Twisted, Heroku, SimpleDB, EC2, GAE, REST, Rack, CouchDB, Redis, Bitcask, Voldemort, Varnish, Pig, Hadoop, HBase, Thrift, Protobufs, Avro, ESI, Mahout, SolR, AMQP, Grand Central Dispatch, WebSockets, LINQ, Reconnoiter, Nginx, ...
    49. 49. Thanks. Creative Contents Images by Big Red Button by thirtyfootscrew - http://www.flickr.com/photos/thirtyfootscrew/3324101122/ The Steam Pipeline by stuck in customs - http://www.flickr.com/photos/stuckincustoms/3780481716 Digidesign Icon Faders by CJ Sorg - http://www.flickr.com/photos/cjsorg/3692124539/ Pro Control 24 by aud1073ch - qhttp://www.flickr.com/photos/aud1073ch/34438156/ Pipe Dream by Rishi Menon - http://www.flickr.com/photos/rxmflickr/4102530508/

    ×