Continuous Quality: What DevOps Means for QA

22,079 views
20,034 views

Published on

Published in: Technology
0 Comments
23 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
22,079
On SlideShare
0
From Embeds
0
Number of Embeds
258
Actions
Shares
0
Downloads
0
Comments
0
Likes
23
Embeds 0
No embeds

No notes for slide
  • Advantage of being around since rocks were young is you do lots of stuffMy career has spanned dev qa opsHad oppt to do a lot of each of them
  • Now that I’m a consultant I spend my time help dev qa ops better understand and work more closely with each otherI do it from the perspective of my personal obsession with interdisciplinarity and synthesis
  • When I look back to the early days of my career I remember applying for a QA Eng jobMechanicalNo input into goodness of softwareOnly input into spec if we couldn’t test
  • Key to talk: we’re in the business of service not software
  • Service helps customer accomplish something they’re trying to do
  • A defining characteristic of service is value cocreationNot just exchange of money for value-filled thingsAmazon gets this
  • Another defining characteristic of service is the customer journey
  • Customer does much more than just eatInteracts with much more than just food
  • Customer does much more than just spin up serversInteracts with much more than just API
  • Tracking cash flow requires integration is part of larger JTBD
  • Outage info points to JTBD
  • Outage management engages me as ops too - JTBD
  • Not only does service change what software is, but also how it’s deliveredTesting built inNew reliance on automationNew view of approach to failures
  • Reimagine QA as something concerned with 4 dimensions and inseparability as characteristic of qualityThat plays profound role enabling devops altogether
  • Adopt new mindsetLife gaze from traditional mechanical activities of execution and reportingNew job is quality advocacy
  • Do game days test service or just softwareOps: performance scalability security resilienceAlso monitoring and support
  • Deeply embed them Just because you have unit tests doesn’t mean their goodWhy would you want crabby testers telling you what’s wrong with everything?Think about good testers you’ve worked with
  • Modeling requirements allows us to talk about them and uncover misunderstanding confusion gaps
  • Opportunity for software engineering practice
  • Most important is planning
  • Do monitoring tools work properlyDoes deployment automation work properly
  • When bugs get into the wild QA gets blamed
  • Goal/challenge is to move from old to new
  • Some good at css, some at chef. Diff langs need to contribute to larger wholeQuality includes optimizing at the right local-global level
  • What is this whole thing that we’re delivering? Not func but op/dev of funcWhen you think about it, aws is part of your ops team
  • Ultimate def of what QA needs to validate
  • Continuous Quality: What DevOps Means for QA

    1. +Continuous Quality: WhatDevOps Means for QAJeff Sussna, Ingineering.IT@jeffsussnaCopyright © 2013 Ingineering.IT, LLC.All Rights Reserved.
    2. My CareerJobs: Dev: from compilers to content management systems QA: automation through the ages Ops: enterprise, managed hosting, ITIL, ops automation, cloudDesign thinker with a penchant for software systemsInveterate boundary-spanner and synthesistCopyright © 2013 Ingineering.IT, LLC. All Rights Reserved.
    3. My PracticeIT Service Innovation consultant Clients: Small development shops and large enterprise ITHelp Dev/QA/Ops work more closely with each otherHelp them better understand each other and their customersCopyright © 2013 Ingineering.IT, LLC. All Rights Reserved.
    4. The Old Definition of QualityThe software meets the specCopyright © 2013 Ingineering.IT, LLC. All Rights Reserved.
    5. …and then…Agile… put the focus on delivering value changed the development/QA relationshipCloud… turned software into serviceCopyright © 2013 Ingineering.IT, LLC. All Rights Reserved.
    6. The New Definition of QualityThe service helps customers accomplish their jobs-to-be-doneCopyright © 2013 Ingineering.IT, LLC. All Rights Reserved.
    7. Service Co-creates ValueService provider helps customer create value-in-useRestaurant: Not just about acquiring food Also about a having a relaxing, romantic night outCloud: Not just about acquiring Linux instances Also about cost-effectively operating elastic, resilient applicationsCopyright © 2013 Ingineering.IT, LLC. All Rights Reserved.
    8. Value Co-Creation is a JourneyCustomers engage via multiple touchpoints over timeCustomers judge service by the entirety of their experience"The food was great but the service was terrible"Copyright © 2013 Ingineering.IT, LLC. All Rights Reserved.
    9. RestaurantJourney: Find a restaurant Make a reservation Park Wait for a table, drink at the bar Order Eat Pay Recommend (or not)Touchpoints: Yellow pages, phone, valet, bar, waiter, food, cashier, social media 

    10. CloudJourney: Understand a new service or feature Provision instances Import data Get help/provide feedback Incorporate updates Deal with outages DeprovisionTouchpoints: Website, console, API, documentation, blogs, email, phone, twitter 

    11. Software as ServiceCo-creation has four dimensions: Functionality Operability Deliverability CoherencyTheir quality is inseparable in the customer’s mindCopyright © 2013 Ingineering.IT, LLC. All Rights Reserved.
    12. FunctionalityDoes the service help me do what I want to do (e.g., get paid ontime)? Track my time Generate and submit invoices Pester clients who havent paid on time Track my overall cash flowCopyright © 2013 Ingineering.IT, LLC. All Rights Reserved.
    13. OperabilityIs the service available when and how I need it? Is it up when I need to bill my clients? Does it perform whether I’m the only user or one of a million? Does it let my data get lost or stolen? Can I get help when I need it? Can I get information when it’s down?Copyright © 2013 Ingineering.IT, LLC. All Rights Reserved.
    14. DeliverabilityDoes the service respond as my needs change? 70% functionality OK on day 1, not in year 2 Today I want to manage invoices, tomorrow expenses I want bugs fixed in hours, not days/weeks/months I want changes delivered in an absorbable mannerCopyright © 2013 Ingineering.IT, LLC. All Rights Reserved.
    15. CoherencyDoes the service engage me throughout my journey? Discovery/understanding Adoption/onboarding Administration Usage Help/feedback New functionality delivery Outage management DepartureCopyright © 2013 Ingineering.IT, LLC. All Rights Reserved.
    16. Delivering Software as ServiceService changes how software is built and operated User-centered design Test-driven development Continuous delivery MTTR over MTBF21st-century IT builds in testing and learning-from-failureCopyright © 2013 Ingineering.IT, LLC. All Rights Reserved.
    17. Old QA Role DisintegratesBoundaries between QA and Design/Dev/Ops blurAutomation takes over mechanical activitiesCopyright © 2013 Ingineering.IT, LLC. All Rights Reserved.
    18. Opportunity for New QA RoleRepresentative of “service not software” perspective: Four dimensions of software as service Their inseparability from perspective of quality Co-creation as fundamental unit of valueBoundary-spanning mirrorCopyright © 2013 Ingineering.IT, LLC. All Rights Reserved.
    19. New QA MindsetLift one’s gaze above mechanics of testingTreat tests as code Adopt development tools and practicesFocus on building quality into the system Tester => Quality Advocate #seewhatIdidthereCopyright © 2013 Ingineering.IT, LLC. All Rights Reserved.
    20. New QA SkillsUnderstand and think about service Outages involve communications, not just fixing serversUnderstand and think about opsUnderstand and think about process/automationCopyright © 2013 Ingineering.IT, LLC. All Rights Reserved.
    21. New QA PracticesDeep participation in cross-functional teamsFacilitation of service requirements understandingOutside-in testingService delivery machine validationCopyright © 2013 Ingineering.IT, LLC. All Rights Reserved.
    22. ParticipationGood testers see systems and their partsGood testers ask probing questions Did you think about…? You forgot about…Good testers design good testsEngage that proficiency in design and test plan critiquesCopyright © 2013 Ingineering.IT, LLC. All Rights Reserved.
    23. Requirements UnderstandingBugs found earlier in the lifecycle are cheaper to fixMost important quality question: Do design/dev/qa/ops understand each other and the customer?Techniques for modeling service requirements: Customer Journey Mapping Behavior-Driven DevelopmentCopyright © 2013 Ingineering.IT, LLC. All Rights Reserved.
    24. Outside-in TestingUser testing validates usabilityUnit testing validates codeStill need to test whether the thing does what it’s supposed to: Functional tests System tests Integration testsCopyright © 2013 Ingineering.IT, LLC. All Rights Reserved.
    25. Continuous Outside-in TestingContinuous delivery challenges testers’ ability to keep upSolution: Part software tools and design skills Part agile process improvementCopyright © 2013 Ingineering.IT, LLC. All Rights Reserved.
    26. Delivery Machine ValidationOperability and deliverability need outside-in testing too BDD (not just for functional requirements)Co-creation necessitates designing for internal users Service BlueprintingCopyright © 2013 Ingineering.IT, LLC. All Rights Reserved.
    27. New QA ValueUnique holder of service understanding across all dimensionsAntidote to confirmation bias Everyone else is invested in what they designed and built QA is invested in customer satisfactionCopyright © 2013 Ingineering.IT, LLC. All Rights Reserved.
    28. Continuous QualityOld: tail wagging the dog Testing happens at the end QA acts as bottleneck Everyone hates QANew: continuous quality Testing happens all the time Entire process and organization infused with quality QA acts as mirrorCopyright © 2013 Ingineering.IT, LLC. All Rights Reserved.
    29. QA As MirrorDevOps is about spanning boundariesService is about inseparable valueNeed to translate between common and specialist viewpointsQA can provide a mirror that helps everyone see: The whole Their relationship to it and to each otherCopyright © 2013 Ingineering.IT, LLC. All Rights Reserved.
    30. Delivery and Co-CreationThe product is not the functionalityThe product is the delivery and operation of functionalityTrue co-creation dissolves customer/provider boundariesCopyright © 2013 Ingineering.IT, LLC. All Rights Reserved.
    31. The Ultimate Definition of QualityThe ability to continuously deliver and operate functionality thathelps customers accomplish their jobs-to-be-doneCopyright © 2013 Ingineering.IT, LLC. All Rights Reserved.
    32. Props to @guidostompffBoundary-Spanning Mirroring PhD thesishttp://designinteams.com/
    33. More InfoContact me about Continuous Quality consulting services: info@ingineering.it @jeffsussna http://www.linkedin.com/in/jeffsussnaRead my thoughts on DevOps, Cloud, and Design Thinking: http://blog.ingineering.itCopyright © 2013 Ingineering.IT, LLC. All Rights Reserved.

    ×