Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

WinOps Conf 2016 - Matteo Emili - Development and QA Dilemmas in DevOps


Published on

The quick rise of Continuous Delivery in the enterprise means that common problems are often approached the other way round. Concepts like Feature Flags and Testing In Production caused several headaches to developers and QA engineers, especially where they have a wealth of experience about traditional development.

There are some challenges and approaches which are very common, and they still scare newcomers. Let's have a look at a few of these, with the most common solutions.

Published in: Technology
  • Be the first to comment

WinOps Conf 2016 - Matteo Emili - Development and QA Dilemmas in DevOps

  1. 1. Development andQA dilemmas in DevOps Matteo Emili ||
  2. 2. Who am I?  Visual Studio and DevelopmentTechnologies MVP (ALM)  Professional Scrum Master I  Systems EngineeringAdvisor @ Dell Software  Taunton Developers Meetup  London Microsoft DevOps Meetup  Other communities overseas
  3. 3. CDCycle Monitor Operate Workingsoftware inproduction Valuerealization
  4. 4. Are we all ready for it?
  5. 5. Sometimes things are the other way round…  The first thing to do should be creating a MinimumViable Product, including the infrastructure  The Pipeline must work from the very first moment  Dev andTest environments should be disposable, infrastructure must be idempotent  Testing can happen in Production
  6. 6. It is a shocking change for some  “I was frightened by not feeling a safety net”  “Does it actually work?”  “The feeling is not good at first”  “We don’t want to be accountable for all of this”  “It can’t work!”  “How do we know what to build?”  “It is hard to get the grip with these tools”
  7. 7. Prerequisite The team must buy into DevOps!
  8. 8. Scenario #0 Use your infrastructure!  Leverage on Infrastructure As Code (DSC,Azure RM, Amazon CloudFormations) to remove friction when factoring the infrastructure in  Spend time on defining what you need to have, and the provider (Azure, AWS, local deployment service) will deploy that for you  Idempotent and resilient – if there is something wrong, redeploy  Less friction here means no barrier between Dev and Ops, a fundamental (and time saving) change!
  9. 9. Scenario #1 Database schema change  You must bring a database schema change in production  Cloud or mission critical web service  Transparent for your users, regardless of public-facing or internal change  No downtime for migration – it must work!
  10. 10. The typical approach  Develop everything in a separate environment  Countless hours of tests  Big-bang migration then new feature for all the users  Big release date, people freak out  …lots of fingers crossed!
  11. 11. Shhh! Don’t make noise!  There is no such approach in DevOps  Each big change must be split in smaller changes  They are often invisible to the user  Silent deployments can happen across multiple sprints  If something breaks, rollback  Start using your MVP straight away!
  12. 12. Demo Infrastructure as Code Database schema change
  13. 13. Scenario #2 Feature Flags  Championed by Martin Fowler, Feature Flags are around since at least 2009  Pretty much all services nowadays use them  Flickr,Amazon, Microsoft, Facebook, Google, Netflix …  They are used to enable or disable certain areas of your application to your users
  14. 14. “…wait. Does it mean Microservices and all the new stuff around?”  No, it doesn’t!  You can use Feature Flags with all the architectures  It is a practice, not a product  It ties in with the previous scenario and makes testing easier
  15. 15. How?  Practically speaking, it is about a Configuration Database coupled with Identifiers  The Configuration Database can be a csv file, XML file, key-value model, a database (!)  Identifiers are areas you will easily split your applications into  They often go well together with Feature/Epics or even User Stories  Countless OSS libraries
  16. 16. It doesn’t come for free  It is expensive (in terms of time, and sometimes refactoring) to build it up from scratch in an existing application  Not just enable/disable – logging, kill switch, selective adoption, rollouts are all things to implement that takes time  OSS libraries help, as there is a choice for each platform, but there is still an implementation cost to keep in mind
  17. 17. Demo Feature Flags with LaunchDarkly
  18. 18. Scenario #3 A/BTesting, MVT Testing…is it about testing?  Little changes for QA in DevOps, except for automation  What is really new are practices ofTesting in Production - extremely common especially withWeb Applications  You deploy version B of your A application (or a permutation of possibilities), direct a percentage of the traffic towards it and compare the adoption results  Implementing them is extremely easy, getting value out of them is hard!
  19. 19. A bit of Lean…  "There is no such thing as a free lunch“ – technology alone is not going to deliver a successful product  My take is to adopt the Build-Measure-Learn loop  Start with a MinimumViable Product and measure its usage with appropriate metrics  Build on top of it, discard what doesn’t work
  20. 20. Getting visibility on what is actually happening  Is my application performing as expected?  What about availability?  Are my users having a good experience?  Above all, you need better, faster and more direct feedback
  21. 21. Learn from your users and improve your product  In the Build-Measure-Learn loop the Measure stage is the most important one  Metrics will tell if you are going in the right direction, so it is critical to ‘get it right’  Extend your product to include what really matters  Get insights on your users before they react  Proactively understand potential problems  You can analyse your data and understand if your application is behaving as expected
  22. 22. It is different from logging  We are not talking about plain application logs  What user actions are meaningful to your business goals?  Is the telemetry data answering this question?  Is there any bottlenecks which is penalising the business?  You need to have visibility on user events  Actionable insights are mapped into behaviours (what content is used, not how)  E.g.: when a user enters in a certain page/area, log on/off, Favourites, social stats, errors…
  23. 23. You won’t do that from scratch  It is too expensive to build something from scratch  You need to use an analytics provider  The development experience must be cross-platform, unobtrusive at first and then customisable with an SDK  Easily add custom telemetry wherever needed
  24. 24. Demo Learn from your users
  25. 25. So?  These scenarios are very common when approaching DevOps  Remember: DevOps is not the silver bullet, teams must buy-in into the process  Collaboration is key to success  Use data to take decisions  Don’t be afraid!