Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Continuous Delivery at Wix, Yaniv Even Haim


Published on

We will go over the motivations for R&D to move to a CI/CD/TDD model, how the model was implemented and the impact on Wix R&D. We will cover the tools used (developed in-house and 3rd party), change in methodologies, what we have learned during the transformation and the unexpected change in working with product and the rest of the company.

Presented in the Continuous Delivery track at DevOps Con Israel 2013

  • Dating direct: ❶❶❶ ❶❶❶
    Are you sure you want to  Yes  No
    Your message goes here
  • Sex in your area is here: ❤❤❤ ❤❤❤
    Are you sure you want to  Yes  No
    Your message goes here

Continuous Delivery at Wix, Yaniv Even Haim

  1. 1. Continuous Delivery @ Wix Yaniv Even-Haim VP R&D DevOpsCon 2013
  2. 2. Wix Intro
  3. 3. Wix in numbers• 29,000,000 registered users from 233 countries• ~1,200,000 new users every month• ~ 30,000 new websites every day• 26,000,000 web sites• 800,000 mobile sites• 1Tbyte of media uploaded every day• More than 1 Billion users media files• 250+ servers in 3 data centers• 350 employees• 100 R&D people
  4. 4. A story on Wix time machine
  5. 5. CD is business concern Revenue per month 2010-… 2010-… 2010-… 2011-… 2011-… 2011-…2012-22010-82010-92011-12011-22011-32011-42011-52011-62011-72011-82011-92012-12012-32012-4Traditional “Economies of scale” threaten by cloud technology Be faster than small startups CD “Economies of Agility “ = Velocity X Size
  6. 6. CD is culture & mindset• Trust the developers – Empower developers to change production – Developer knows his system best
  7. 7. CD is culture & mindset• Automation as a default choice – no more “is it worth to automate ? ” – Everything should be automated
  8. 8. Welcome to the twilight zone• Product/Dev/QA/Ops boundaries are going down – Everyone responsible for everything
  9. 9. Welcome to the twilight zone• Move fast - Less formality – Corridor - IN , Meeting Room - Out
  10. 10. Get out of thought land make sure you building the right it before build it right• The Law of Failure – Most new “its” will fail even if they are flawlessly executed• Invest less, in-touch less , better ability to admit it fail – Data beats opinions - Let the market decide
  11. 11. Some Best Practices• Everyone develops on the Trunk/Master – Release frequent – small pieces of functionality – It’s the developer responsibility not to break anything• Code can get to production at anytime – TDD & integration test• Use Feature Toggle – Unbaked new code can go to production – no harm done – New code goes with a guard – use new or old code
  12. 12. Some Best Practices• Backwards and Forwards compatible – Each component has to function with latest, next or prior version of other components (including DBs)• Gradual Deployment & Self Test – Deploy new version to one server and perform self test. If it passes, continue deployment to other servers• A/B Testing – Open a new feature to a percent of your users. Is it better? – Deployment is an engineering decision
  13. 13. CD – prepare to invest…..• Dev infrastructure - Refactor , Refactor, Refactor• Testing infrastructure & know how• Deployment infrastructure & tools• Automation , Automation , Automation• Monitoring (business and technical) – hundreds of aspects – thresholds use is a Must – Monitor business KPIs – Internal & external – Endless Tuning & learning
  14. 14. Scaling challenges• Deployment complexity – More services – More dependencies – Multi data centers• Development complexity – Very hard to work on master/trunk with 30+ dev on same code base – backward and forward comptability is not easy• Wider set of technologies ,tools & languages – Higher level of infrastructure investments – Harder to have all around players – More communication/sync required
  15. 15. Scaling challenges – Quality– Testing coverage challenge • Use cases increases • Browsers wild west • Tablets and mobile variety– Feature toggle combinations • Dozens of parallel feature toggles– Production frequent changes • Services are updated daily • Toggles switch on/off on daily basis
  16. 16. Scaling challenges - Availability• Service Availability • Cost of fault/downtime increasing as business grow • Service downtime during deployment is unacceptable • Human mistakes are inventible when changing production daily
  17. 17. Scaling challenges – Monitoring• Monitoring in multi technology env• Unified dashboard require major effort• Reduce alert noises resulted by deployments• Identify root cause is not simple – Some faults does not have immediate impacts – Cascading effects• Reliable business KPI without false positive – Marketing changes – Special events (holidays, sport event etc)
  18. 18. Scaling challenges – Product• Product MVP practices – Does MVP meet your product standards? • What about tooltip,help,first time ux, etc.. ? – How to define a product that can be developed in a day ? – And that can win in a/b test … To Be Implemented
  19. 19. Scaling challenges – Org• Complex Matrix Management - People moving between teams frequently - One can have several managers on different tasks - Hard both to managers and employees - Everyone should master the matrix management skills
  20. 20. Wix Developer Lifecycle
  21. 21. Wix Developer Lifecycle
  22. 22. New Relic - Monitoring
  23. 23. Application dashboard
  24. 24. Chix – Staging Deployment
  25. 25. It is not all or nothing game - it is continuous journey