The Where and When of NoSQL Platforms

1,481 views

Published on

Logicworks presents Just Say No to SQL as part of the Digital Media Series, moderated by Mashable's own Christina Warren.

Published in: Technology, Business
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
1,481
On SlideShare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
Downloads
10
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide
  • Mirror modeShort timeoutsLog errors DB2 is still database of record
  • The Where and When of NoSQL Platforms

    1. 1. PRESENTS <br />
    2. 2. Moderator <br /> Christina Warren, <br />Panelists<br />Harry Heymann, Head of Server Engineering at <br />Michael Bryzek, CTO & Founder at<br />Frank Weigel, Director of Products at<br />Matt Pfeil, VP of Customer Solutions and Co-Founder at<br />Dave Connors, VP of Operations at  <br />Dwight Merriman, CEO at 10Gen at <br />
    3. 3. The Old World<br />Pros<br />Transactional integrity<br />Sequential nature <br />Cons<br /><ul><li>Expensive
    4. 4. Scale up typically not out</li></li></ul><li>The New World<br /> <br />
    5. 5. Why ??<br />Huge volume of content<br />Distributed Infrastructure<br />Relaxed and agile<br />Throw the RD rules out the window<br />Speed of development<br />No DBA needed<br />Elastic scaling (out not up)<br />Major cost savings<br />
    6. 6. Platforms<br />
    7. 7. Companies<br />
    8. 8. Michael Bryzek CTO & Founder<br />
    9. 9. GILT GROUPEis an innovative e-commerce company offering highly coveted products and experiences at insider prices.<br />
    10. 10. One of the Most Innovating Companies<br />“Gilt Groupe, the online shopping phenomenon”<br />--Alexandra Kotur, Vogue<br />“The t-shirt arrived the very next day in a clean box with nicely folded black tissue paper and a friendly note from the founders thanking me for my purchase. They're thanking me? No, Gilt Groupe, thank you.”<br />--Damien Nunes, GQ (on his first purchase)<br />“The French invented the online sample sale for designer merchandise, spawning numerous American competitors, but none boasts the pedigree of Gilt Groupe.”<br />--Fortune, (from their story “Dotcom Divas”)<br />“People with a taste for high-end fashion items have turned Gilt into a guiltless addiction.”<br />--Roy Furchgott, The New York Times<br />
    11. 11. 10,000 foot view<br />
    12. 12. The Shopping Cart<br />
    13. 13. Harry Heymann<br />Head of Server Engineering<br />
    14. 14.
    15. 15. mongodb: some numbers<br /><ul><li>8 clusters
    16. 16. some sharded, some not
    17. 17. some master/slave, some replica sets
    18. 18. ~40 machines (68GB, m2.4xl on EC2)
    19. 19. 2.3 billion records
    20. 20. ~15k QPS</li></li></ul><li>Why MongoDB?<br />Biggest reason (by far): auto sharding:<br /><ul><li>Started on a single SQL database.  Eventually split to two nodes: 1 for check-ins (our biggest dataset), 1 for everything else.
    21. 21. It was clear that check-ins would grow beyond what a single machine could handle.
    22. 22. Major efficiency gains by outsourcing the development of the sharding layer to consultants at 10gen.</li></li></ul><li>Dave Connors, VP of Operations<br />
    23. 23. Constant Contact<br /><ul><li>Customers are Small Businesses
    24. 24. Email, Event, Survey & now Social Media
    25. 25. Over 450k paying customers
    26. 26. Business model
    27. 27. Many customers pay as little as $15 a month
    28. 28. ~2 million database transactions per minute
    29. 29. Business problem
    30. 30. Social Media  up to 100 times more data
    31. 31. Challenge with our business model</li></li></ul><li>Implementation<br />Cultural Challenges: NoSQL<br /><ul><li>Monitoring
    32. 32. Dev took lead with Munin
    33. 33. Logging
    34. 34. Ops took lead with Scribed
    35. 35. Roles & Responsibilities
    36. 36. DBA?
    37. 37. Still in progress…</li></ul>Traditional Roles<br />
    38. 38. Managing Risk: Phased Rollout<br /><ul><li>Switchable modes
    39. 39. Mirroring
    40. 40. Dial-able traffic </li></li></ul><li>Conclusion<br />Business Value Comparison<br />
    41. 41. Technologists<br />
    42. 42. Dwight Merriman <br />CEO<br />
    43. 43. Past:RDBMS<br />one size fits all<br />
    44. 44. other segments<br />e.g. search<br />RDBMS<br />BI / Data warehousing<br />
    45. 45. RDBMS<br />other segments<br />e.g. search<br />RDBMS<br />BI / Data warehousing<br />aster, greenplum, neteeza, teradata, vertica, hadoop<br />NoSQL<br />
    46. 46. <ul><li>very complex transactional semantics
    47. 47. legacy projects
    48. 48. projects requiring SQL</li></ul>RDBMS<br />other segments<br />e.g. search<br />RDBMS<br />BI / Data warehousing<br />aster, greenplum, neteeza, teradata, vertica, hadoop<br />NoSQL<br /><ul><li>a few gigantic queries
    49. 49. visual client tools important -> a need for SQL
    50. 50. scales horizontally (better ones)
    51. 51. agile
    52. 52. programmer friendly data modle
    53. 53. horizontall scalable
    54. 54. works for operational data (lots of small reads and writes)</li></li></ul><li>Frank Weigel, Director of Products <br />
    55. 55. Couchbase Server Architecture<br />11211<br />11210<br />memcapable 1.0<br />memcapable 2.0<br />moxi<br />Memcached Managed Cache<br />protocol listener/sender<br />REST management API/Web UI<br />vBucket state and replication manager<br />Rebalance orchestrator<br />Node health monitor<br />Heartbeat<br />Process monitor<br />Global singleton supervisor<br />Configuration manager<br />Data Manager <br />Cluster Manager <br />engine interface<br />Couchbase Storage Engine<br />http<br />on each node<br />one per cluster<br />Erlang/OTP<br />HTTP<br />distributed erlang<br />erlang port mapper<br />21100 – 21199<br />4369<br />8091<br />
    56. 56. MySQL<br />: Migrating to Couchbase<br />Example of online migration of existing application.<br />First migrated large or slow performing tables and frequently updated fields from MySQL to Couchbase<br />Web Server<br />Web Server<br />Apache + PHP<br />Apache + PHP<br />Client-side Moxi<br />Client-side Moxi<br />memcached<br />protocol listener/sender<br />memcached<br />protocol listener/sender<br />memcached<br />protocol listener/sender<br />TAP<br />TAP<br />TAP<br />engine interface<br />engine interface<br />engine interface<br />Couchbase Storage Engine<br />Couchbase Storage Engine<br />Couchbase Storage Engine<br />30<br />
    57. 57. Matt Pfeil, VP of Customer Solutions<br />
    58. 58. Architecture<br />
    59. 59. Better technology<br />Multi-master, multi-DC<br />Linearly scalable<br />Larger-than-memory datasets<br />Best-in-class performance (not just writes!)<br />Fully durable<br />Integrated caching<br />Tuneable consistency<br />
    60. 60. A performance retrospective<br />
    61. 61. Credit: Nathan Hurst, 2010<br />
    62. 62. QUESTIONS FROM THE AUDIENCE <br />
    63. 63. THANK YOU FOR JOINING US!<br />We look forward to seeing you at our upcoming events<br />

    ×