• Share
  • Email
  • Embed
  • Like
  • Save
  • Private Content
Iterate Like a Whirling Dervish
 

Iterate Like a Whirling Dervish

on

  • 8,804 views

Continuous deployment is a a process that allows companies to release software in minutes instead of days, weeks, or months....

Continuous deployment is a a process that allows companies to release software in minutes instead of days, weeks, or months.

Pascal-Louis Perez will describe how to use continuous deployment to iterate so fast that you run circles around the competition. He will cover the high level concepts as well as the nitty gritty details including examples from the continuous deployment system that he and his team developed at KaChing.

Pascal-Louis is the VP of Engineering and CTO at KaChing, where he practices continuous deployment continuously.

He previously worked at Google and holds a Master's degree in Computer Science from Stanford University.

Statistics

Views

Total Views
8,804
Views on SlideShare
7,792
Embed Views
1,012

Actions

Likes
32
Downloads
0
Comments
2

16 Embeds 1,012

http://eng.wealthfront.com 701
http://torgronsund.wordpress.com 141
http://eng.kaching.com 79
http://torgronsund.com 48
http://www.techgig.com 13
http://www.encob.net 12
http://www.foundergems.com 5
http://www.investmentor.com 4
http://www.slideshare.net 2
http://www.techgig.timesjobs.com 1
http://www.leanmovement.com 1
http://leanstartups.wpengine.com 1
http://www.hooeeywebprint.com 1
http://www.hanrss.com 1
http://www.onlydoo.com 1
https://torgronsund.wordpress.com 1
More...

Accessibility

Categories

Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel

12 of 2 previous next

  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
  • Supermarket Analogy is inappropriate. Most of the companies develop a product on the shelf or few products on different shelves so supermarket opening times and closing isles are beyond their control.
    Are you sure you want to
    Your message goes here
    Processing…
  • Very nice presentation.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

    Iterate Like a Whirling Dervish Iterate Like a Whirling Dervish Presentation Transcript

    • Continuous Deployment at kaChing
      #leanstartup
    • “Connect investors with outstandinginvestment managers”
      Pascal-Louis Perez, kaChing Group Inc.
    • Pascal-Louis Perez, kaChing Group Inc.
      Assets Managed by Mutual Funds in the US
      $11,000,000,000,000
    • Pivot
      Our vision of bringing the quality of Ivy League Endowment management to retail is unaltered
      Powering a marketplace where customers entrust high end managers with their money in separately managed accounts. We are SEC & FINRA regulated.
      Three years ago, kaChing was a fantasy stock trading app for young adults on facebook. Our strategy was to create an eco system where avid amateurs could rise through the ranks and make it to pro league.
      Pascal-Louis Perez, kaChing Group Inc.
    • Constant tension between product and development.
      Right architecture only in the context of how it is used. Initially, you don’t know for sure how something will be used!
      We’ve adopted proportional investing
      Build cheap
      When you spend X hours of time fixing problems
      Spend a corresponding X hours improving the feature
      Pascal-Louis Perez, kaChing Group Inc.
      Build “Right”
      Features
    • Strive for 80% of the value for 20% of the cost
      Realize that 80% of what you build will go to waste. Validate before investing more time!
      Pascal-Louis Perez, kaChing Group Inc.
      80/20
    • Errors Are Costly!
      Continuous Deployment is a mission critical way of running our business which provides tremendous value
      Errors are costly
      Brand impact
      SLA penalties
      And when you’re regulated, there’s also liabilities
      Turning the wheels on the software with high confidence provides tremendous business value
      Pascal-Louis Perez, kaChing Group Inc.
    • Deploy Fast and Often
      Pascal-Louis Perez, kaChing Group Inc.
    • What is Continuous Deployment?
      Continuous, successful and repeatable methodology to deploying code
      Automates every steps of taking checked in code and making it run on production servers, used by customers
      Pascal-Louis Perez, kaChing Group Inc.
    • e.g. Twenty Deployments a Day
      Pascal-Louis Perez, kaChing Group Inc.
    • Release is a Marketing ConcernReducing Code Inventory is Engineering’s Job
      Pascal-Louis Perez, kaChing Group Inc.
    • Homage to Gursky: 'Kirksky‘copyright 2010 by Kirk Crippens
      Supermarket Analogy
      (Think Safeway)
      You don’t close the store to fill the shelves!
      Continuously get fresh merchandise out
      For product releases, simply close or hide the aisle
    • Traditional Release Organization
      Software organized as a tree
      Trunk is what goes on production, it is The Truth
      Engineers work on features in branches
      Features get integrated into trunk
      Release train on regular cyclic period
      QA ensures correctness of each train
      P1 bugs get patched back into the train
      Flip the switch release
      Pascal-Louis Perez, kaChing Group Inc.
    • Timeline
      Pascal-Louis Perez, kaChing Group Inc.
      QA
      Stage
      Release
      Fix P1 Bugs
      Development
      Cut a Release
      Integrate Patches
      1-4 Weeks
    • Timeline, Revisited
      Pascal-Louis Perez, kaChing Group Inc.
      Deployment
      Automated
      QA
      Stage
      Fix P1 Bugs
      Release
      Development
      or Testing
      Cut a Release
      Integrate Patches
      Monitoring
      Experiments
      5-10 Minutes
    • Experiments
      Split features which are not finished“if press person, show the new home page”if (isInExperiment(NEW_HOME_PAGE)) { …} else { …}
      Pascal-Louis Perez, kaChing Group Inc.
      (Yes, this is just like A/B Testing)
    • True Story
      Investment managers calls, comments about unintuitive trading flow.
      Improvements are made, and deployed within the next 20 minutes.
      We call him back “What do you think of the improvement?”
      Pascal-Louis Perez, kaChing Group Inc.
    • Benefits of Continuous Deployment
      It allows quick iterations
      Obsoletes processes, e.g. “cutting a release”
      Reduces risk
      Everyone is aware of production
      No one throws code over the wall
      Exposes 24x7 operational requirements
      Trunk stable
      Pascal-Louis Perez, kaChing Group Inc.
    • Continuous Deployment
      ImmuneSystem
      Culture
      ContinuousIntegration
      Pascal-Louis Perez, kaChing Group Inc.
      Test DrivenDevelopment
    • kaChing’s Architecture
      Service oriented system
      Vertical sharding
      Everything uses the same platform kawala
      Coordination using ZooKeeper
      Data interchange using JSON and Protobufs
      Pascal-Louis Perez, kaChing Group Inc.
    • Typical Stack
      Clustered services, multiple instances
      Replicated databases (e.g. MySQL, Redis, …)
      Caching (e.g. memcached)
      Denormalized data (e.g. Voldemort)
      Pascal-Louis Perez, kaChing Group Inc.
    • Timeline, Revisited
      Pascal-Louis Perez, kaChing Group Inc.
      Development
      Deployment
      Testing
      Monitoring
    • Development
      Trunk stable
      Small, frequent commits
      Unreleased features live behind experiments
      Forward & Backward compatibility
      Responsive code review
      Pair difficult problems
      Trivial rollbacks
    • Testing Philosophy
      Only automated testing matters
      If it isn’t tested, it isn’t finished or correct
      Write testable code
      Embrace abstractions
      Testing is cross functional, we all own quality
    • Benefits of TDD
      It allows quick iterations
      It empowers engineers to change anything, and as such helps in scaling the team
      It is more cost effective than debugging
      It obsoletes the need for functional QA
      It facilitates continuous refactoring, allowing the code to get better with age
      It attracts the right kind of engineers
      Pascal-Louis Perez, kaChing Group Inc.
    • Types of Testing
      Unit Testing - does the code work?
      Integration Testing - does the code work together?
      Regression Testing - learn from your mistakes
      Frontend Testing - a whole different ballgame
    • Defensive Testing
      Capture invariants in tests
      Nightmare scenarios
      A common conversation at lunch:
      Alice: What would happen if X blew up?
      Bob: uh... the site would go down.
      Fix it, test for it
    • Implementation
      Frameworks: junit, dbunit
      CI: Hudson
      Frontend: Selenium
    • Life of a Deployment
      Pascal-Louis Perez, kaChing Group Inc.
      DM
      PM
      PM
      PM
      ZK
    • Life of a Deployment
      Pascal-Louis Perez, kaChing Group Inc.
      DM
      PM
      PM
      PM
      ZK
    • Life of a Deployment
      Pascal-Louis Perez, kaChing Group Inc.
      DM
      PM
      PM
      ZK
    • Life of a Deployment
      Pascal-Louis Perez, kaChing Group Inc.
      DM
      PM
      PM
      PM
      ZK
    • Life of a Deployment
      Pascal-Louis Perez, kaChing Group Inc.
      DM
      PM
      PM
      PM
      ZK
    • Deployment
      Exponentially increasing group sizes
      Increased monitoring during deployment
      Self Test
      Automated rollbacks
    • Stability
      Cluster everything
      As little global state as possible
      Maintain global state through ZooKeeper
      Monitor everything
    • ZooKeeper
      Reliable Distributed Synchronization
      Service Discovery
      Service Status
      Coarse locking
    • Implementation
      Building: ant
      Deploying: rpm, yum
      Monitoring: nagios, jcollectd, RRDtool, jmx
      Custom automation
    • Monitoring Philosophy
      Prefer business metrics
      Monitor statistical deviations not absolute values
      Automatically annotate graphs
    • Monitoring Errors
      False negatives - errors of omission
      False positives - errors of implementation
    • End to End Monitoring
      Run Selenium on production
      Accessibility, Speed
      Ad-hoc Keynote - customized for our flows
      Must control data creation, analytics impact
    • #future-developments
      Pascal-Louis Perez, kaChing Group Inc.
    • Quarantining
      Isolate new releases
      Flexible partitioning of requests
      Gradually shift load to fresh services
      Pascal-Louis Perez, kaChing Group Inc.
    • Describing Infrastructure
      Many moving parts
      nagios, collectd, backups, services, databases, …
      Consistency is key
      Adding new tools should be easy and thorough
      Standardize best practices
      Pascal-Louis Perez, kaChing Group Inc.
    • TGIF?
      Pascal-Louis Perez, kaChing Group Inc.
      We will be hosting a TGIF in August in our office@pascallouis / @kachingeng
    • References
      We’re Recruiting jobs@kaching.com
      kaChing’s blog http://eng.kaching.com
      kawalahttp://bit.ly/kawala
      Deployment Infrastructure http://eng.kaching.com/2010/05/deployment-infrastructure-for.html
      Extreme Testing http://bit.ly/9bOFaA
      Writing Testable Codehttp://googletesting.blogspot.com/2008/08/by-miko-hevery-so-you-decided-to.html
      The Testability Explorer Bloghttp://misko.hevery.com
      ZooKeeperhttp://bit.ly/kc-zookeeper
      Lessons Learned http://startuplessonslearned.com
      Pascal-Louis Perez, kaChing Group Inc.