• Share
  • Email
  • Embed
  • Like
  • Save
  • Private Content
C* Summit 2013: Stepping Through the Lifecycle of a Service Offering with Cassandra by Chris McEniry and Igor von Nyssen
 

C* Summit 2013: Stepping Through the Lifecycle of a Service Offering with Cassandra by Chris McEniry and Igor von Nyssen

on

  • 1,050 views

It's a fine line to walk for incorporating new technologies in an organization with 15+ years of legacy software. In this presentation, we'll look at the lifecycle and adoption of Cassandra from a ...

It's a fine line to walk for incorporating new technologies in an organization with 15+ years of legacy software. In this presentation, we'll look at the lifecycle and adoption of Cassandra from a skunkworks project to a full fledged service in a legacy organization.

Statistics

Views

Total Views
1,050
Views on SlideShare
1,049
Embed Views
1

Actions

Likes
1
Downloads
12
Comments
0

1 Embed 1

http://localhost 1

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment
  • We has a lot of momentum/inertia Means it takes a while to get something changed Have to work within some limitations and around some limitations Separation of Duties Be ready to move - in a slow chnaging org, anytime you have a window, you have to move it push for the window
  • Can we beat the speed of light?

C* Summit 2013: Stepping Through the Lifecycle of a Service Offering with Cassandra by Chris McEniry and Igor von Nyssen C* Summit 2013: Stepping Through the Lifecycle of a Service Offering with Cassandra by Chris McEniry and Igor von Nyssen Presentation Transcript

  • Stepping Through the Lifecycleof a Service Offering with CassandraChris McEniry, Igor von Nyssen | Sr. Systems Architect, Sr. Software Architect | Sony Network Entertainment
  • *  aka Ops guy*  Bs EECS MIT*  15 years ops experience*  Python, Perl, C, LISP,Ruby, JS, some English*  C* 0.6*  cmceniry@mit.eduChris “Mac” McEniry Igor von Nyssen*  aka Dev guy*  MS CS TU Chemnitz*  MBA USC*  15 years dev experience*  Java, Ruby, English,German*  C* 1.0*  igor@vonnyssen.com
  • *  Preconception – Solution looking for a problem*  In utero – Connect and talk about it*  Infant – Initial Development*  Kid – Production rollout for an insignificant item*  Teenager – Start real maturing*  Adult – Take over the worldWhat we’re talking about: Lifecycle
  • *  Large entertainment ecommerce application*  100s of Millions of customers*  Billions of transactions per year*  Worldwide operations: 59 countries, 21 languages and dialects*  Peaks above 10,000 tpsWhat does SNEI look like?
  • *  Global start-up that has tocomply with any three letterrequirement know to man.*  Pressure to innovate*  Pressure to make moneyWhat does SNEI feel like?
  • Pre-Conception
  • Recognize Concerns/Motivations
  • Concerns
  • In-Utero
  • What’s the Real Problem?
  • Our Problem
  • Speed of Light Problem
  • Need to provide a service that−  Provides  user  authen0ca0on  −  Is  Always-­‐On  −  Has  low  latency  for  every  customer  Our Problem: Latency and Availability
  • Can I beat the speed of Light?
  • Infant
  • Find Allies
  • Be willing to be uncomfortable
  • Strategically decide who you’re not going to involveThe New RACI ChartRole   Responsible   Avoided   Circumvented   Ignored  You   X  Friend   X  Frenemy   X   X   X  
  • Child
  • Learn on a Small Scale
  • What’s your Minimally Viable Product?
  • Teenager
  • Spread the Word
  • Embody the Mantra
  • Adult
  • Adult
  • Where are we now?
  • THANK YOU
  • Slideshare
  • Stepping Through the Lifecycleof a Service Offering with CassandraChris McEniry, Igor von Nyssen | Sr. Systems Architect, Sr. Software Architect | Sony Network Entertainment
  • *  aka Ops guy*  Bs EECS MIT*  15 years ops experience*  Python, Perl, C, LISP,Ruby, JS, some English*  C* 0.6*  cmceniry@mit.eduChris “Mac” McEniry Igor von Nyssen*  aka Dev guy*  MS CS TU Chemnitz*  MBA USC*  15 years dev experience*  Java, Ruby, English,German*  C* 1.0*  igor@vonnyssen.com
  • *  Preconception – Solution looking for a problem*  In utero – Connect and talk about it*  Infant – Initial Development*  Kid – Production rollout for an insignificant item*  Teenager – Start real maturing*  Adult – Take over the worldWhat we’re talking about now: Lifecycle
  • *  Large entertainment ecommerce application*  100s of Millions of customers*  9 figure revenue per year*  Billions of transactions per year*  Worldwide operations: 59 countries, 21 languages and dialects*  Peaks above 10,000 tpsWhat does SNEI look like?
  • *  Global start-up that has tocomply with any three letterrequirement know to man.*  Pressure to innovate*  Pressure to make moneyWhat does SNEI feel like?
  • Pre-Conception
  • Recognize Concerns/Motivations
  • *  Curiosity*  Personal Growth*  Less pain*  The Hype – Exploit itPersonal
  • Ops Concerns
  • *  How do we keep it from blowing up?*  How do we keep it cared for for its lifetime?*  How do we keep from being a bottleneck?*  How do we keep from being woken up at 4AM?*  How do we keep up with “gimme!gimme!gimme?"Ops Concerns
  • *  Marketing features vs. real features−  What is on the outside of the box?−  What does actually work?*  C* is young−  our understanding of it is young*  Does it solve our problem adequately?−  POCPOCPOC−  We analyzed lots of in-memory data grids-­‐  Some had marketing feature problems-­‐  Some were too young-­‐  Some were too expensive-­‐  None of them addressed partitioning very well-­‐  If for nothing else, the cold startup time is a killer*  Adoption−  Can we implement C* in our organization?Dev Concerns
  • *  Dev – youre not running production.*  Ops – you’re not writing the code.*  Work with someone−  Who  sees  the  same  problem  −  Is  excited  about  the  technology  −  Agrees  on  the  direc0on  −  Does  not  have  an  inherent  conflict  of  interest  Can’t do it by yourself
  • In-Utero
  • What’s the Real Problem?
  • *  Be ready to move once you find a suitable problem−  anytime you have a window, you have to move*  Create the window*  But, be sure that you want it*  Be sure it’s the right problemYou Have a Solution Looking for a Problem
  • Our Problem
  • SF viewed from the World
  • Need to provide a service that−  Provides  user  authen0ca0on  −  Is  Always-­‐On  −  Has  low  latency  for  every  customer  Our Problem: Latency and Availability
  • Can I beat the speed of Light?
  • *  Not a problem that can be solved by slapping a cache in front ofexisting technology*  Need distributed storage*  Global distribution helps with both latency and disaster recovery−  Data  replica0on  and  synchroniza0on  are  built-­‐in  −  Par00on  events  are  handled  out  of  the  box  *  Continuous delivery is part of the C* philosophyWhy Does C* Fit Here?
  • Infant
  • Find Allies
  • *  Good Ally – Someone who−  Sees the same problem−  Is excited about the same technology−  Agrees on the direction*  Bad Ally – Someone who−  Will have an inherent conflict of interestGood Allies, Bad Allies
  • *  Develop your elevator pitch for the problem*  Proof it with your allies*  Keep it simpleLatency, Latency, LatencyCoin the Mantra
  • Be willing to be uncomfortable
  • Strategically decide who you’re not going to involveThe New RACI ChartRole   Responsible   Avoided   Circumvented   Ignored  You   X  Friend   X  Frenemy   X   X   X  
  • *  Its new technology, so youre outside your comfort zone*  Youre evangelizing, so the proof is on you*  Account for different adoption speeds by different people*  Stay positiveRemember: Shiny, But Unknown and Unproven
  • *  Ops−  Service-­‐  Central-­‐  Embedded−  Support-­‐  Formal-­‐  Community-­‐  On-your-own−  Schema definition ownershipFigure Out Your Models*  Dev−  How many nodes?−  How many copies?−  How many data centers?
  • *  No budget approvals*  Easy to get startedOnce you have an idea of what youre talking about, then start to askthe better questions.Cassandra Is Open Source – Use That Fact
  • Child
  • Learn on a Small Scale
  • What’s your Minimally Viable Product?
  • *  operational tasks−  Setup-­‐  roll everything in rpms-­‐  theres a bit of a manual setup - cluster join/remove−  Backup−  infrastructure changes (node additions, etc.)-­‐  theres a bit of a manual setup - cluster join/remove−  Monitoring-­‐  jolokia to tie into existing systems, but have to re-invent all of the metric metadata-­‐  Availability−  incident response-­‐  training and see monitoring/visibility−  handling regular tasks-­‐  nodetool repair-­‐  add keyspaces-­‐  updating keyspaces for infrastructure changes*  have a clear service description*  end user support−  self-help community−  email list (subscription), chat roomMinimum Viable (Go Live) Product
  • *  Store some data−  start with transient data−  TTLs help here−  If it’s lost, it’s an acceptable loss*  Select which client you will use−  Hector, Astyanax, or Datastax*  Figure out how clients connect to the server*  Understand how C* stores data−  Data model−  Serializers−  Comparators*  Free your mind−  Learn to love primary-key-only indexing−  Start de-normalizing−  Give up on transactions – they are evil to begin withMinimum Viable (Go Live) Product
  • Teenager
  • *  Take daddy’s credit card*  Get serious about other data classes*  Start putting something that you need to preserve in there*  Data life cycle management*  Security concerns*  Acknowledge gaps – where either the technology or you or yourimplementation of the technology are just not ready*  Revisit your assumptionsTeenager
  • Spread the Word
  • *  Allies−  Security−  RDMS zeal^Wusers*  Problems−  Adjacent problems – things that are close to what you’ve alreadysolved, but bigger and more complicated*  Key to success−  It is still early enough to be their idea−  But it is established enough that people take it seriouslyGather More
  • *  More visibility à More battles*  Continue to choose wisely−  Decide where not to focus?−  What not to try?−  What to be compliant with?*  Remember: Proof is on you−  Other groups will closely watch your success or failure−  Auditors will ask you the questions. There is no vendor certification (yet).Teenage Angst
  • *  If you really must do large multipart transactions, choose a differentproduct.*  Or just don’t. Change your mental model−  From: transactions and conflict prevention−  To: conflict detection and remediation*  Read this: http://www.eaipatterns.com/ramblings/18_starbucks.html*  Avoid sequences that attempt to centralize countingTeenage Angst
  • *  Search (DSE)*  Solidify your backup*  Solidify your deployment model*  Solidify your support*  Solidify your security−  Audit−  Authentication and Authorization−  EncryptionGrowth Spurts
  • Embody the MantraLatency, Latency, Latency
  • *  You can ask better questions.*  You have to ask the better questions.Now you have experience
  • Adult
  • Adult
  • Where are we now?
  • *  In terms of−  Data  −  Applicable  context  −  Ability  to  adopt  and  adapt  *  Less inertia*  Moving faster*  Less energy for the same distance : “More efficient”Underlying Problem: Critical Mass
  • Summarize*  Preconception – Explore it*  In utero – Start small*  Infant – Be comfortably uncomfortable*  Kid – Grow*  Teenager – Get help*  Adult – Profit
  • THANK YOU