Iterate Like a Whirling Dervish

  • 7,770 views
Uploaded on

Continuous deployment is a a process that allows companies to release software in minutes instead of days, weeks, or months. …

Continuous deployment is a a process that allows companies to release software in minutes instead of days, weeks, or months.

Pascal-Louis Perez will describe how to use continuous deployment to iterate so fast that you run circles around the competition. He will cover the high level concepts as well as the nitty gritty details including examples from the continuous deployment system that he and his team developed at KaChing.

Pascal-Louis is the VP of Engineering and CTO at KaChing, where he practices continuous deployment continuously.

He previously worked at Google and holds a Master's degree in Computer Science from Stanford University.

More in: Technology
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
  • Supermarket Analogy is inappropriate. Most of the companies develop a product on the shelf or few products on different shelves so supermarket opening times and closing isles are beyond their control.
    Are you sure you want to
    Your message goes here
  • Very nice presentation.
    Are you sure you want to
    Your message goes here
No Downloads

Views

Total Views
7,770
On Slideshare
0
From Embeds
0
Number of Embeds
7

Actions

Shares
Downloads
0
Comments
2
Likes
33

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. Continuous Deployment at kaChing
    #leanstartup
  • 2. “Connect investors with outstandinginvestment managers”
    Pascal-Louis Perez, kaChing Group Inc.
  • 3. Pascal-Louis Perez, kaChing Group Inc.
    Assets Managed by Mutual Funds in the US
    $11,000,000,000,000
  • 4. Pivot
    Our vision of bringing the quality of Ivy League Endowment management to retail is unaltered
    Powering a marketplace where customers entrust high end managers with their money in separately managed accounts. We are SEC & FINRA regulated.
    Three years ago, kaChing was a fantasy stock trading app for young adults on facebook. Our strategy was to create an eco system where avid amateurs could rise through the ranks and make it to pro league.
    Pascal-Louis Perez, kaChing Group Inc.
  • 5. Constant tension between product and development.
    Right architecture only in the context of how it is used. Initially, you don’t know for sure how something will be used!
    We’ve adopted proportional investing
    Build cheap
    When you spend X hours of time fixing problems
    Spend a corresponding X hours improving the feature
    Pascal-Louis Perez, kaChing Group Inc.
    Build “Right”
    Features
  • 6. Strive for 80% of the value for 20% of the cost
    Realize that 80% of what you build will go to waste. Validate before investing more time!
    Pascal-Louis Perez, kaChing Group Inc.
    80/20
  • 7. Errors Are Costly!
    Continuous Deployment is a mission critical way of running our business which provides tremendous value
    Errors are costly
    Brand impact
    SLA penalties
    And when you’re regulated, there’s also liabilities
    Turning the wheels on the software with high confidence provides tremendous business value
    Pascal-Louis Perez, kaChing Group Inc.
  • 8. Deploy Fast and Often
    Pascal-Louis Perez, kaChing Group Inc.
  • 9. What is Continuous Deployment?
    Continuous, successful and repeatable methodology to deploying code
    Automates every steps of taking checked in code and making it run on production servers, used by customers
    Pascal-Louis Perez, kaChing Group Inc.
  • 10. e.g. Twenty Deployments a Day
    Pascal-Louis Perez, kaChing Group Inc.
  • 11. Release is a Marketing ConcernReducing Code Inventory is Engineering’s Job
    Pascal-Louis Perez, kaChing Group Inc.
  • 12. Homage to Gursky: 'Kirksky‘copyright 2010 by Kirk Crippens
    Supermarket Analogy
    (Think Safeway)
    You don’t close the store to fill the shelves!
    Continuously get fresh merchandise out
    For product releases, simply close or hide the aisle
  • 13. Traditional Release Organization
    Software organized as a tree
    Trunk is what goes on production, it is The Truth
    Engineers work on features in branches
    Features get integrated into trunk
    Release train on regular cyclic period
    QA ensures correctness of each train
    P1 bugs get patched back into the train
    Flip the switch release
    Pascal-Louis Perez, kaChing Group Inc.
  • 14. Timeline
    Pascal-Louis Perez, kaChing Group Inc.
    QA
    Stage
    Release
    Fix P1 Bugs
    Development
    Cut a Release
    Integrate Patches
    1-4 Weeks
  • 15. Timeline, Revisited
    Pascal-Louis Perez, kaChing Group Inc.
    Deployment
    Automated
    QA
    Stage
    Fix P1 Bugs
    Release
    Development
    or Testing
    Cut a Release
    Integrate Patches
    Monitoring
    Experiments
    5-10 Minutes
  • 16. Experiments
    Split features which are not finished“if press person, show the new home page”if (isInExperiment(NEW_HOME_PAGE)) { …} else { …}
    Pascal-Louis Perez, kaChing Group Inc.
    (Yes, this is just like A/B Testing)
  • 17. True Story
    Investment managers calls, comments about unintuitive trading flow.
    Improvements are made, and deployed within the next 20 minutes.
    We call him back “What do you think of the improvement?”
    Pascal-Louis Perez, kaChing Group Inc.
  • 18. Benefits of Continuous Deployment
    It allows quick iterations
    Obsoletes processes, e.g. “cutting a release”
    Reduces risk
    Everyone is aware of production
    No one throws code over the wall
    Exposes 24x7 operational requirements
    Trunk stable
    Pascal-Louis Perez, kaChing Group Inc.
  • 19. Continuous Deployment
    ImmuneSystem
    Culture
    ContinuousIntegration
    Pascal-Louis Perez, kaChing Group Inc.
    Test DrivenDevelopment
  • 20. kaChing’s Architecture
    Service oriented system
    Vertical sharding
    Everything uses the same platform kawala
    Coordination using ZooKeeper
    Data interchange using JSON and Protobufs
    Pascal-Louis Perez, kaChing Group Inc.
  • 21. Typical Stack
    Clustered services, multiple instances
    Replicated databases (e.g. MySQL, Redis, …)
    Caching (e.g. memcached)
    Denormalized data (e.g. Voldemort)
    Pascal-Louis Perez, kaChing Group Inc.
  • 22. Timeline, Revisited
    Pascal-Louis Perez, kaChing Group Inc.
    Development
    Deployment
    Testing
    Monitoring
  • 23. Development
    Trunk stable
    Small, frequent commits
    Unreleased features live behind experiments
    Forward & Backward compatibility
    Responsive code review
    Pair difficult problems
    Trivial rollbacks
  • 24. Testing Philosophy
    Only automated testing matters
    If it isn’t tested, it isn’t finished or correct
    Write testable code
    Embrace abstractions
    Testing is cross functional, we all own quality
  • 25. Benefits of TDD
    It allows quick iterations
    It empowers engineers to change anything, and as such helps in scaling the team
    It is more cost effective than debugging
    It obsoletes the need for functional QA
    It facilitates continuous refactoring, allowing the code to get better with age
    It attracts the right kind of engineers
    Pascal-Louis Perez, kaChing Group Inc.
  • 26. Types of Testing
    Unit Testing - does the code work?
    Integration Testing - does the code work together?
    Regression Testing - learn from your mistakes
    Frontend Testing - a whole different ballgame
  • 27. Defensive Testing
    Capture invariants in tests
    Nightmare scenarios
    A common conversation at lunch:
    Alice: What would happen if X blew up?
    Bob: uh... the site would go down.
    Fix it, test for it
  • 28. Implementation
    Frameworks: junit, dbunit
    CI: Hudson
    Frontend: Selenium
  • 29. Life of a Deployment
    Pascal-Louis Perez, kaChing Group Inc.
    DM
    PM
    PM
    PM
    ZK
  • 30. Life of a Deployment
    Pascal-Louis Perez, kaChing Group Inc.
    DM
    PM
    PM
    PM
    ZK
  • 31. Life of a Deployment
    Pascal-Louis Perez, kaChing Group Inc.
    DM
    PM
    PM
    ZK
  • 32. Life of a Deployment
    Pascal-Louis Perez, kaChing Group Inc.
    DM
    PM
    PM
    PM
    ZK
  • 33. Life of a Deployment
    Pascal-Louis Perez, kaChing Group Inc.
    DM
    PM
    PM
    PM
    ZK
  • 34. Deployment
    Exponentially increasing group sizes
    Increased monitoring during deployment
    Self Test
    Automated rollbacks
  • 35. Stability
    Cluster everything
    As little global state as possible
    Maintain global state through ZooKeeper
    Monitor everything
  • 36. ZooKeeper
    Reliable Distributed Synchronization
    Service Discovery
    Service Status
    Coarse locking
  • 37. Implementation
    Building: ant
    Deploying: rpm, yum
    Monitoring: nagios, jcollectd, RRDtool, jmx
    Custom automation
  • 38.
  • 39. Monitoring Philosophy
    Prefer business metrics
    Monitor statistical deviations not absolute values
    Automatically annotate graphs
  • 40. Monitoring Errors
    False negatives - errors of omission
    False positives - errors of implementation
  • 41. End to End Monitoring
    Run Selenium on production
    Accessibility, Speed
    Ad-hoc Keynote - customized for our flows
    Must control data creation, analytics impact
  • 42. #future-developments
    Pascal-Louis Perez, kaChing Group Inc.
  • 43. Quarantining
    Isolate new releases
    Flexible partitioning of requests
    Gradually shift load to fresh services
    Pascal-Louis Perez, kaChing Group Inc.
  • 44. Describing Infrastructure
    Many moving parts
    nagios, collectd, backups, services, databases, …
    Consistency is key
    Adding new tools should be easy and thorough
    Standardize best practices
    Pascal-Louis Perez, kaChing Group Inc.
  • 45. TGIF?
    Pascal-Louis Perez, kaChing Group Inc.
    We will be hosting a TGIF in August in our office@pascallouis / @kachingeng
  • 46. References
    We’re Recruiting jobs@kaching.com
    kaChing’s blog http://eng.kaching.com
    kawalahttp://bit.ly/kawala
    Deployment Infrastructure http://eng.kaching.com/2010/05/deployment-infrastructure-for.html
    Extreme Testing http://bit.ly/9bOFaA
    Writing Testable Codehttp://googletesting.blogspot.com/2008/08/by-miko-hevery-so-you-decided-to.html
    The Testability Explorer Bloghttp://misko.hevery.com
    ZooKeeperhttp://bit.ly/kc-zookeeper
    Lessons Learned http://startuplessonslearned.com
    Pascal-Louis Perez, kaChing Group Inc.