Successfully reported this slideshow.
Your SlideShare is downloading. ×

Development and QA dilemmas in DevOps

Loading in …3

Check these out next

1 of 35 Ad

More Related Content

Slideshows for you (20)

Similar to Development and QA dilemmas in DevOps (20)


More from Matteo Emili (20)

Recently uploaded (20)


Development and QA dilemmas in DevOps

  1. 1. Development and QA dilemmas in DevOps Matteo Emili Microsoft MVP
  2. 2. Who am I?  Visual Studio and DevelopmentTechnologies MVP (ALM)  Professional Scrum Master I  Systems EngineeringAdvisor @ One Identity  Taunton Developers Meetup | London Microsoft DevOps Meetup  | DomusDotNet || @MattVSTS ||
  3. 3. CDCycle Monitor Operate Workingsoftware inproduction Valuerealization
  4. 4. Are we all ready for it?
  5. 5. The typical (legacy) approach  Develop everything in a separate environment  Countless hours of tests  Big-bang migration then new feature for all the users  Big release date, people freaking out  …lots of fingers crossed!
  6. 6. Sometimes things work the other way round…  The first thing to do should be creating a MinimumViable Product, including the infrastructure  The Pipeline must work from the very first moment  Dev andTest environments should be disposable, infrastructure must be idempotent  Testing can happen in Production
  7. 7. It is a shocking change for some  Excerpts from interviews with developers, testers, managers…  “I was frightened by not feeling a safety net”  “Does it actually work?”  “The feeling is not good at first”  “We don’t want to be accountable for all of this”  “It can’t work!”  “How do we know what to build?”  “It is hard to get the grip with these tools”
  8. 8. Prerequisite The organisation must buy into DevOps!
  9. 9. “Should I go all-in with ‘the Cloud’ then?”
  10. 10. Of course not…  It is all about culture and practices, there is little technology you really need to know about  This change puts the customer at the centre of the process, where everybody across all the company work towards fulfilling the customers’ requests and demands  These practices can easily be reused with a totally on-premise product, there is no need of targeting a technology you do not need
  11. 11. Use your infrastructure!  Leverage on Infrastructure As Code (Azure RM,Amazon CloudFormations, PowerShell DSC) to remove friction when factoring the infrastructure in  Spend time on defining what you need to have, and the provider (Azure, AWS, local deployment service) will deploy that for you  Idempotent and resilient – if there is something wrong, redeploy
  12. 12. Package your application for distribution  Leverage on packaging and deployment technologies to create a flexible deployment and provisioning system  These technologies are usually proven and tested  MSDeploy  Web Archive format  NuGet  They take centre stage with a Release Pipeline, together with Infrastructure as Code
  13. 13. Demo Build once, deploy everywhere
  14. 14. “How should I handle changes?”
  15. 15. Shhh! Don’t make noise!  Each big change must be split in smaller changes  They are often invisible to the user  Silent deployments can happen across multiple sprints  If something breaks, rollback  Build incrementally, migrate data before full rollout  Start using your MVP straight away!
  16. 16. An example…  You must bring a database schema change in production (a new column)  Cloud or mission critical web service  Transparent for your users, regardless of public-facing or internal change  No downtime for migration – it must work!
  17. 17. Hang on. How about breaking changes?  It is essential to factor in some sort of mitigative solution that keeps the service online while the change is deployed and tested  A real world example?TFS 2015 introduced a massive change that resulted in large databases being unable to be upgraded in less than 48 hours  Microsoft released a tool that performed an online transformation of the affected tables so that the upgrade proper could happen in a reasonable time  Pro: feasible upgrade  Con: resource cost  More on that at handle-disruptive-changes.html
  18. 18. Demo Handle a database schema change in production
  19. 19. “Testing in Production?”
  20. 20. A/BTesting, MVT Testing…is it actually about testing?  Little changes for QA in DevOps, except for automation and the actual target of the system under test  What is really new are practices ofTesting in Production - extremely common especially withWeb Applications  You deploy version B of your A application (or a permutation of possibilities), direct a percentage of the traffic towards it and compare the adoption results  Implementing them is extremely easy, getting value out of them is hard!
  21. 21. Feature Flags to the rescue  Championed by Martin Fowler, Feature Flags are around since at least 2009  Pretty much all the major services nowadays use them  Flickr,Amazon, Microsoft, Facebook, Google, Netflix …  They are used to enable or disable certain areas of your application to your users  Also extremely valuable as a kill switch in case of issues
  22. 22. “…wait. Does it mean Microservices and all the new stuff around?”  No, it doesn’t!  You can use Feature Flags with any architecture  It is a practice, not necessarily a product  It ties in with the previous scenario and makes testing easier
  23. 23. How?  Practically speaking, it is about a Configuration Database coupled with Identifiers  The Configuration Database can be a csv file, XML file, key-value model, a database (!)  Identifiers are areas you will easily split your applications into  They often go well together with Feature/Epics or even User Stories  Countless OSS libraries
  24. 24. It doesn’t come for free  It is expensive (in terms of time, and sometimes refactoring) to build it up from scratch in an existing application  Not just enable/disable – logging, kill switch, selective adoption, rollouts are all things to implement that take time  OSS libraries help, as there is a choice for each platform, but there is still an implementation cost to keep in mind
  25. 25. Demo Feature Flags with LaunchDarkly
  26. 26. “How can I get better ROI from my R&D investment?”
  27. 27. A bit of Lean…  "There is no such thing as a free lunch“ – technology alone is not going to deliver a successful product  My take is to adopt the Build-Measure-Learn loop  Start with a MinimumViable Product and measure its usage with appropriate metrics  Build on top of it, discard what doesn’t work
  28. 28. Getting visibility on what is actually happening  Is my application performing as expected?  What about availability?  Are my users having a good experience?  Above all, you need better, faster and more direct feedback
  29. 29. Learn from your users and improve your product  In the Build-Measure-Learn loop the Measure stage is the most important one  Metrics will tell if you are going in the right direction, so it is critical to ‘get it right’  Extend your product to include what really matters  Get insights on your users before they react  Proactively understand potential problems  You can analyse your data and understand if your application is behaving as expected
  30. 30. It is different from logging  We are not talking about plain application logs  What user actions are meaningful to your business goals?  Is the telemetry data answering this question?  Is there any bottlenecks which is penalising the business?  You need to have visibility on user events  Actionable insights are mapped into behaviours (what content is used, not how)  E.g.: when a user enters in a certain page/area, log on/off, Favourites, social stats, errors…
  31. 31. You won’t do that from scratch  It is too expensive to build something from scratch  You need to use an analytics provider  The development experience must be cross-platform, unobtrusive at first and then customisable with an SDK  Easily add custom telemetry wherever needed
  32. 32. “Will I get a meaningful insight from data alone?”  Telemetry is not just reactive, it can also be proactive  You will need to report on top of your data, with your metrics  The alternative  Easily add custom telemetry wherever needed
  33. 33. Demo Learn from your users
  34. 34. So?  These scenarios are very common when approaching DevOps  Remember: DevOps is not the silver bullet, teams must buy-in into the process  Collaboration is key to success  Use data to take decisions  Don’t be afraid!
  35. 35. Dank!