SlideShare a Scribd company logo
1 of 33
+
Continuous Quality: What
DevOps Means for QA
Jeff Sussna, Ingineering.IT
@jeffsussna
Copyright © 2013 Ingineering.IT, LLC.
All Rights Reserved.
My Career
Jobs:
 Dev: from compilers to content management systems
 QA: automation through the ages
 Ops: enterprise, managed hosting, ITIL, ops automation, cloud
Design thinker with a penchant for software systems
Inveterate boundary-spanner and synthesist
Copyright © 2013 Ingineering.IT, LLC. All Rights Reserved.
My Practice
IT Service Innovation consultant
 Clients: Small development shops and large enterprise IT
Help Dev/QA/Ops work more closely with each other
Help them better understand each other and their customers
Copyright © 2013 Ingineering.IT, LLC. All Rights Reserved.
The Old Definition of Quality
The software meets the spec
Copyright © 2013 Ingineering.IT, LLC. All Rights Reserved.
…and then…
Agile…
 put the focus on delivering value
 changed the development/QA relationship
Cloud…
 turned software into service
Copyright © 2013 Ingineering.IT, LLC. All Rights Reserved.
The New Definition of Quality
The service helps customers accomplish their jobs-to-be-done
Copyright © 2013 Ingineering.IT, LLC. All Rights Reserved.
Service Co-creates Value
Service provider helps customer create value-in-use
Restaurant:
 Not just about acquiring food
 Also about a having a relaxing, romantic night out
Cloud:
 Not just about acquiring Linux instances
 Also about cost-effectively operating elastic, resilient applications
Copyright © 2013 Ingineering.IT, LLC. All Rights Reserved.
Value Co-Creation is a Journey
Customers engage via multiple touchpoints over time
Customers judge service by the entirety of their experience
"The food was great but the service was terrible"
Copyright © 2013 Ingineering.IT, LLC. All Rights Reserved.
Restaurant
Journey:
 Find a restaurant
 Make a reservation
 Park
 Wait for a table, drink at the bar
 Order
 Eat
 Pay
 Recommend (or not)
Touchpoints:
 Yellow pages, phone, valet, bar, waiter, food, cashier, social media
 

Cloud
Journey:
 Understand a new service or feature
 Provision instances
 Import data
 Get help/provide feedback
 Incorporate updates
 Deal with outages
 Deprovision
Touchpoints:
 Website, console, API, documentation, blogs, email, phone, twitter
 

Software as Service
Co-creation has four dimensions:
 Functionality
 Operability
 Deliverability
 Coherency
Their quality is inseparable in the customer’s mind
Copyright © 2013 Ingineering.IT, LLC. All Rights Reserved.
Functionality
Does the service help me do what I want to do (e.g., get paid on
time)?
 Track my time
 Generate and submit invoices
 Pester clients who haven't paid on time
 Track my overall cash flow
Copyright © 2013 Ingineering.IT, LLC. All Rights Reserved.
Operability
Is the service available when and how I need it?
 Is it up when I need to bill my clients?
 Does it perform whether I’m the only user or one of a million?
 Does it let my data get lost or stolen?
 Can I get help when I need it?
 Can I get information when it’s down?
Copyright © 2013 Ingineering.IT, LLC. All Rights Reserved.
Deliverability
Does the service respond as my needs change?
 70% functionality OK on day 1, not in year 2
 Today I want to manage invoices, tomorrow expenses
 I want bugs fixed in hours, not days/weeks/months
 I want changes delivered in an absorbable manner
Copyright © 2013 Ingineering.IT, LLC. All Rights Reserved.
Coherency
Does the service engage me throughout my journey?
 Discovery/understanding
 Adoption/onboarding
 Administration
 Usage
 Help/feedback
 New functionality delivery
 Outage management
 Departure
Copyright © 2013 Ingineering.IT, LLC. All Rights Reserved.
Delivering Software as Service
Service changes how software is built and operated
 User-centered design
 Test-driven development
 Continuous delivery
 MTTR over MTBF
21st-century IT builds in testing and learning-from-failure
Copyright © 2013 Ingineering.IT, LLC. All Rights Reserved.
Old QA Role Disintegrates
Boundaries between QA and Design/Dev/Ops blur
Automation takes over mechanical activities
Copyright © 2013 Ingineering.IT, LLC. All Rights Reserved.
Opportunity for New QA Role
Representative of “service not software” perspective:
 Four dimensions of software as service
 Their inseparability from perspective of quality
 Co-creation as fundamental unit of value
Boundary-spanning mirror
Copyright © 2013 Ingineering.IT, LLC. All Rights Reserved.
New QA Mindset
Lift one’s gaze above mechanics of testing
Treat tests as code
 Adopt development tools and practices
Focus on building quality into the system
 Tester => Quality Advocate #seewhatIdidthere
Copyright © 2013 Ingineering.IT, LLC. All Rights Reserved.
New QA Skills
Understand and think about service
 Outages involve communications, not just fixing servers
Understand and think about ops
Understand and think about process/automation
Copyright © 2013 Ingineering.IT, LLC. All Rights Reserved.
New QA Practices
Deep participation in cross-functional teams
Facilitation of service requirements understanding
Outside-in testing
Service delivery machine validation
Copyright © 2013 Ingineering.IT, LLC. All Rights Reserved.
Participation
Good testers see systems and their parts
Good testers ask probing questions
 Did you think about…?
 You forgot about…
Good testers design good tests
Engage that proficiency in design and test plan critiques
Copyright © 2013 Ingineering.IT, LLC. All Rights Reserved.
Requirements Understanding
Bugs found earlier in the lifecycle are cheaper to fix
Most important quality question:
 Do design/dev/qa/ops understand each other and the customer?
Techniques for modeling service requirements:
 Customer Journey Mapping
 Behavior-Driven Development
Copyright © 2013 Ingineering.IT, LLC. All Rights Reserved.
Outside-in Testing
User testing validates usability
Unit testing validates code
Still need to test whether the thing does what it’s supposed to:
 Functional tests
 System tests
 Integration tests
Copyright © 2013 Ingineering.IT, LLC. All Rights Reserved.
Continuous Outside-in Testing
Continuous delivery challenges testers’ ability to keep up
Solution:
 Part software tools and design skills
 Part agile process improvement
Copyright © 2013 Ingineering.IT, LLC. All Rights Reserved.
Delivery Machine Validation
Operability and deliverability need outside-in testing too
 BDD (not just for functional requirements)
Co-creation necessitates designing for internal users
 Service Blueprinting
Copyright © 2013 Ingineering.IT, LLC. All Rights Reserved.
New QA Value
Unique holder of service understanding across all dimensions
Antidote to confirmation bias
 Everyone else is invested in what they designed and built
 QA is invested in customer satisfaction
Copyright © 2013 Ingineering.IT, LLC. All Rights Reserved.
Continuous Quality
Old: tail wagging the dog
 Testing happens at the end
 QA acts as bottleneck
 Everyone hates QA
New: continuous quality
 Testing happens all the time
 Entire process and organization infused with quality
 QA acts as mirror
Copyright © 2013 Ingineering.IT, LLC. All Rights Reserved.
QA As Mirror
DevOps is about spanning boundaries
Service is about inseparable value
Need to translate between common and specialist viewpoints
QA can provide a mirror that helps everyone see:
 The whole
 Their relationship to it and to each other
Copyright © 2013 Ingineering.IT, LLC. All Rights Reserved.
Delivery and Co-Creation
The product is not the functionality
The product is the delivery and operation of functionality
True co-creation dissolves customer/provider boundaries
Copyright © 2013 Ingineering.IT, LLC. All Rights Reserved.
The Ultimate Definition of Quality
The ability to continuously deliver and operate functionality that
helps customers accomplish their jobs-to-be-done
Copyright © 2013 Ingineering.IT, LLC. All Rights Reserved.
Props to @guidostompff
'Boundary-Spanning Mirroring' PhD thesis
http://designinteams.com/
More Info
Contact me about Continuous Quality consulting services:
 info@ingineering.it
 @jeffsussna
 http://www.linkedin.com/in/jeffsussna
Read my thoughts on DevOps, Cloud, and Design Thinking:
 http://blog.ingineering.it
Copyright © 2013 Ingineering.IT, LLC. All Rights Reserved.

More Related Content

What's hot

DOES15 - Sherry Chang - Intel’s Journey to Large Scale DevOps Transformation
DOES15 - Sherry Chang - Intel’s Journey to Large Scale DevOps Transformation DOES15 - Sherry Chang - Intel’s Journey to Large Scale DevOps Transformation
DOES15 - Sherry Chang - Intel’s Journey to Large Scale DevOps Transformation Gene Kim
 
Continuous Testing - What QA means for DevOps
Continuous Testing - What QA means for DevOpsContinuous Testing - What QA means for DevOps
Continuous Testing - What QA means for DevOpsSeaLights
 
DevOps vs Agile | DevOps Tutorial For Beginners | DevOps Training | Edureka
DevOps vs Agile | DevOps Tutorial For Beginners | DevOps Training | EdurekaDevOps vs Agile | DevOps Tutorial For Beginners | DevOps Training | Edureka
DevOps vs Agile | DevOps Tutorial For Beginners | DevOps Training | EdurekaEdureka!
 
Demystifying DevOps for Ops - Including Findings from the 2015 State of DevOp...
Demystifying DevOps for Ops - Including Findings from the 2015 State of DevOp...Demystifying DevOps for Ops - Including Findings from the 2015 State of DevOp...
Demystifying DevOps for Ops - Including Findings from the 2015 State of DevOp...Puppet
 
Webinar: DevOps challenges facing QA
Webinar: DevOps challenges facing QAWebinar: DevOps challenges facing QA
Webinar: DevOps challenges facing QAQualitest
 
DevOps Summit 2015 Presentation: Continuous Testing At the Speed of DevOps
DevOps Summit 2015 Presentation: Continuous Testing At the Speed of DevOpsDevOps Summit 2015 Presentation: Continuous Testing At the Speed of DevOps
DevOps Summit 2015 Presentation: Continuous Testing At the Speed of DevOpsSailaja Tennati
 
Drive Continuous Delivery With Continuous Testing
Drive Continuous Delivery With Continuous TestingDrive Continuous Delivery With Continuous Testing
Drive Continuous Delivery With Continuous TestingCA Technologies
 
Introduction To DevOps | Devops Tutorial For Beginners | DevOps Training For ...
Introduction To DevOps | Devops Tutorial For Beginners | DevOps Training For ...Introduction To DevOps | Devops Tutorial For Beginners | DevOps Training For ...
Introduction To DevOps | Devops Tutorial For Beginners | DevOps Training For ...Simplilearn
 
Developing a Testing Strategy for DevOps Success
Developing a Testing Strategy for DevOps SuccessDeveloping a Testing Strategy for DevOps Success
Developing a Testing Strategy for DevOps SuccessDevOps.com
 
DevOps Powerpoint Presentation Slides
DevOps Powerpoint Presentation SlidesDevOps Powerpoint Presentation Slides
DevOps Powerpoint Presentation SlidesSlideTeam
 
DevOps Maturity Curve v5
DevOps Maturity Curve v5DevOps Maturity Curve v5
DevOps Maturity Curve v5Paul Peissner
 
Agile Testing Transformation is as Easy as 1, 2, 3 by Michael Buening
Agile Testing Transformation is as Easy as 1, 2, 3 by Michael BueningAgile Testing Transformation is as Easy as 1, 2, 3 by Michael Buening
Agile Testing Transformation is as Easy as 1, 2, 3 by Michael BueningQA or the Highway
 
Four pillars of DevOps - John Shaw - Agile Cambridge 2014
Four pillars of DevOps - John Shaw - Agile Cambridge 2014Four pillars of DevOps - John Shaw - Agile Cambridge 2014
Four pillars of DevOps - John Shaw - Agile Cambridge 2014johnfcshaw
 
DevOps – The Evolution of Agile
DevOps – The Evolution of AgileDevOps – The Evolution of Agile
DevOps – The Evolution of AgileQualitest
 
¿Qué es DevOps y por qué es importante en el Ciclo de Software? por michelada.io
¿Qué es DevOps y por qué es importante en el Ciclo de Software? por michelada.io¿Qué es DevOps y por qué es importante en el Ciclo de Software? por michelada.io
¿Qué es DevOps y por qué es importante en el Ciclo de Software? por michelada.ioSoftware Guru
 
Agile, DevOps & Test
Agile, DevOps & TestAgile, DevOps & Test
Agile, DevOps & TestQualitest
 

What's hot (20)

DOES15 - Sherry Chang - Intel’s Journey to Large Scale DevOps Transformation
DOES15 - Sherry Chang - Intel’s Journey to Large Scale DevOps Transformation DOES15 - Sherry Chang - Intel’s Journey to Large Scale DevOps Transformation
DOES15 - Sherry Chang - Intel’s Journey to Large Scale DevOps Transformation
 
DevOps Foundation
DevOps FoundationDevOps Foundation
DevOps Foundation
 
Continuous Testing - What QA means for DevOps
Continuous Testing - What QA means for DevOpsContinuous Testing - What QA means for DevOps
Continuous Testing - What QA means for DevOps
 
DevOps vs Agile | DevOps Tutorial For Beginners | DevOps Training | Edureka
DevOps vs Agile | DevOps Tutorial For Beginners | DevOps Training | EdurekaDevOps vs Agile | DevOps Tutorial For Beginners | DevOps Training | Edureka
DevOps vs Agile | DevOps Tutorial For Beginners | DevOps Training | Edureka
 
Demystifying DevOps for Ops - Including Findings from the 2015 State of DevOp...
Demystifying DevOps for Ops - Including Findings from the 2015 State of DevOp...Demystifying DevOps for Ops - Including Findings from the 2015 State of DevOp...
Demystifying DevOps for Ops - Including Findings from the 2015 State of DevOp...
 
Webinar: DevOps challenges facing QA
Webinar: DevOps challenges facing QAWebinar: DevOps challenges facing QA
Webinar: DevOps challenges facing QA
 
DevOps Summit 2015 Presentation: Continuous Testing At the Speed of DevOps
DevOps Summit 2015 Presentation: Continuous Testing At the Speed of DevOpsDevOps Summit 2015 Presentation: Continuous Testing At the Speed of DevOps
DevOps Summit 2015 Presentation: Continuous Testing At the Speed of DevOps
 
Drive Continuous Delivery With Continuous Testing
Drive Continuous Delivery With Continuous TestingDrive Continuous Delivery With Continuous Testing
Drive Continuous Delivery With Continuous Testing
 
Introduction To DevOps | Devops Tutorial For Beginners | DevOps Training For ...
Introduction To DevOps | Devops Tutorial For Beginners | DevOps Training For ...Introduction To DevOps | Devops Tutorial For Beginners | DevOps Training For ...
Introduction To DevOps | Devops Tutorial For Beginners | DevOps Training For ...
 
Developing a Testing Strategy for DevOps Success
Developing a Testing Strategy for DevOps SuccessDeveloping a Testing Strategy for DevOps Success
Developing a Testing Strategy for DevOps Success
 
DevOps Powerpoint Presentation Slides
DevOps Powerpoint Presentation SlidesDevOps Powerpoint Presentation Slides
DevOps Powerpoint Presentation Slides
 
DevOps Maturity Curve v5
DevOps Maturity Curve v5DevOps Maturity Curve v5
DevOps Maturity Curve v5
 
A True Story of Why QA Loves DevOps
A True Story of Why QA Loves DevOpsA True Story of Why QA Loves DevOps
A True Story of Why QA Loves DevOps
 
Agile Testing Transformation is as Easy as 1, 2, 3 by Michael Buening
Agile Testing Transformation is as Easy as 1, 2, 3 by Michael BueningAgile Testing Transformation is as Easy as 1, 2, 3 by Michael Buening
Agile Testing Transformation is as Easy as 1, 2, 3 by Michael Buening
 
Building DevOps Toolchain
Building DevOps ToolchainBuilding DevOps Toolchain
Building DevOps Toolchain
 
Four pillars of DevOps - John Shaw - Agile Cambridge 2014
Four pillars of DevOps - John Shaw - Agile Cambridge 2014Four pillars of DevOps - John Shaw - Agile Cambridge 2014
Four pillars of DevOps - John Shaw - Agile Cambridge 2014
 
Bn1006 demo ppt devops
Bn1006 demo ppt devopsBn1006 demo ppt devops
Bn1006 demo ppt devops
 
DevOps – The Evolution of Agile
DevOps – The Evolution of AgileDevOps – The Evolution of Agile
DevOps – The Evolution of Agile
 
¿Qué es DevOps y por qué es importante en el Ciclo de Software? por michelada.io
¿Qué es DevOps y por qué es importante en el Ciclo de Software? por michelada.io¿Qué es DevOps y por qué es importante en el Ciclo de Software? por michelada.io
¿Qué es DevOps y por qué es importante en el Ciclo de Software? por michelada.io
 
Agile, DevOps & Test
Agile, DevOps & TestAgile, DevOps & Test
Agile, DevOps & Test
 

Viewers also liked

Quality Assurance in a DevOps World
Quality Assurance in a DevOps WorldQuality Assurance in a DevOps World
Quality Assurance in a DevOps WorldKit Plummer
 
Evolution of the DevOps Quality Management Office
Evolution of the DevOps Quality Management OfficeEvolution of the DevOps Quality Management Office
Evolution of the DevOps Quality Management OfficeCapgemini
 
Beyond DevOps: User-Centered IT
Beyond DevOps: User-Centered ITBeyond DevOps: User-Centered IT
Beyond DevOps: User-Centered ITJeff Sussna
 
Software MTTR: The Path from Continuous Integration to Continuous Delivery
Software MTTR: The Path from Continuous Integration to Continuous DeliverySoftware MTTR: The Path from Continuous Integration to Continuous Delivery
Software MTTR: The Path from Continuous Integration to Continuous DeliveryJeff Sussna
 
Promising Digital Service Quality
Promising Digital Service QualityPromising Digital Service Quality
Promising Digital Service QualityJeff Sussna
 
The Essence of DevOps
The Essence of DevOpsThe Essence of DevOps
The Essence of DevOpsJeff Sussna
 
Better Service Through Promise Theory
Better Service Through Promise TheoryBetter Service Through Promise Theory
Better Service Through Promise TheoryJeff Sussna
 
A Manifesto for 21st-Century IT
A Manifesto for 21st-Century ITA Manifesto for 21st-Century IT
A Manifesto for 21st-Century ITJeff Sussna
 
Continuous Delivery in a Legacy Shop—One Step at a Time
Continuous Delivery in a Legacy Shop—One Step at a TimeContinuous Delivery in a Legacy Shop—One Step at a Time
Continuous Delivery in a Legacy Shop—One Step at a TimeTechWell
 
Testers and developers think differently
Testers and developers think differentlyTesters and developers think differently
Testers and developers think differentlyNuthan Kumar
 
From Cybernetics to DevOps and Beyond
From Cybernetics to DevOps and BeyondFrom Cybernetics to DevOps and Beyond
From Cybernetics to DevOps and BeyondJeff Sussna
 
Software Quality in the Devops World: The Impact of Continuous Delivery on Te...
Software Quality in the Devops World: The Impact of Continuous Delivery on Te...Software Quality in the Devops World: The Impact of Continuous Delivery on Te...
Software Quality in the Devops World: The Impact of Continuous Delivery on Te...cornelia davis
 
SDLC Transformation: How Paychex Began Adopting Rational CLM (+ Agile) & Adap...
SDLC Transformation: How Paychex Began Adopting Rational CLM (+ Agile) & Adap...SDLC Transformation: How Paychex Began Adopting Rational CLM (+ Agile) & Adap...
SDLC Transformation: How Paychex Began Adopting Rational CLM (+ Agile) & Adap...Tom Sylvester
 
Create Your Tester Portfolio
Create Your Tester PortfolioCreate Your Tester Portfolio
Create Your Tester PortfolioShmuel Gershon
 
Tektronix keithley Product and Application update Q2 2016
Tektronix keithley Product and Application update Q2 2016Tektronix keithley Product and Application update Q2 2016
Tektronix keithley Product and Application update Q2 2016Jeff Sable
 
Poka yoke and Devops
Poka yoke and DevopsPoka yoke and Devops
Poka yoke and Devopsulfmansson
 

Viewers also liked (20)

Quality Assurance in a DevOps World
Quality Assurance in a DevOps WorldQuality Assurance in a DevOps World
Quality Assurance in a DevOps World
 
Evolution of the DevOps Quality Management Office
Evolution of the DevOps Quality Management OfficeEvolution of the DevOps Quality Management Office
Evolution of the DevOps Quality Management Office
 
Dev IS Ops
Dev IS OpsDev IS Ops
Dev IS Ops
 
Beyond DevOps: User-Centered IT
Beyond DevOps: User-Centered ITBeyond DevOps: User-Centered IT
Beyond DevOps: User-Centered IT
 
Beyond Products
Beyond ProductsBeyond Products
Beyond Products
 
Software MTTR: The Path from Continuous Integration to Continuous Delivery
Software MTTR: The Path from Continuous Integration to Continuous DeliverySoftware MTTR: The Path from Continuous Integration to Continuous Delivery
Software MTTR: The Path from Continuous Integration to Continuous Delivery
 
Promising Digital Service Quality
Promising Digital Service QualityPromising Digital Service Quality
Promising Digital Service Quality
 
The Essence of DevOps
The Essence of DevOpsThe Essence of DevOps
The Essence of DevOps
 
Better Service Through Promise Theory
Better Service Through Promise TheoryBetter Service Through Promise Theory
Better Service Through Promise Theory
 
A Manifesto for 21st-Century IT
A Manifesto for 21st-Century ITA Manifesto for 21st-Century IT
A Manifesto for 21st-Century IT
 
DevOps guide for awesome quality assurance
DevOps guide for awesome quality assuranceDevOps guide for awesome quality assurance
DevOps guide for awesome quality assurance
 
Continuous Delivery in a Legacy Shop—One Step at a Time
Continuous Delivery in a Legacy Shop—One Step at a TimeContinuous Delivery in a Legacy Shop—One Step at a Time
Continuous Delivery in a Legacy Shop—One Step at a Time
 
Testers and developers think differently
Testers and developers think differentlyTesters and developers think differently
Testers and developers think differently
 
From Cybernetics to DevOps and Beyond
From Cybernetics to DevOps and BeyondFrom Cybernetics to DevOps and Beyond
From Cybernetics to DevOps and Beyond
 
Software Quality in the Devops World: The Impact of Continuous Delivery on Te...
Software Quality in the Devops World: The Impact of Continuous Delivery on Te...Software Quality in the Devops World: The Impact of Continuous Delivery on Te...
Software Quality in the Devops World: The Impact of Continuous Delivery on Te...
 
SDLC Transformation: How Paychex Began Adopting Rational CLM (+ Agile) & Adap...
SDLC Transformation: How Paychex Began Adopting Rational CLM (+ Agile) & Adap...SDLC Transformation: How Paychex Began Adopting Rational CLM (+ Agile) & Adap...
SDLC Transformation: How Paychex Began Adopting Rational CLM (+ Agile) & Adap...
 
Create Your Tester Portfolio
Create Your Tester PortfolioCreate Your Tester Portfolio
Create Your Tester Portfolio
 
Quality assurance road map
Quality assurance road mapQuality assurance road map
Quality assurance road map
 
Tektronix keithley Product and Application update Q2 2016
Tektronix keithley Product and Application update Q2 2016Tektronix keithley Product and Application update Q2 2016
Tektronix keithley Product and Application update Q2 2016
 
Poka yoke and Devops
Poka yoke and DevopsPoka yoke and Devops
Poka yoke and Devops
 

Similar to Continuous Quality: What DevOps Means for QA

Full Service Ownership
Full Service OwnershipFull Service Ownership
Full Service OwnershipMandi Walls
 
Ambasoft Presentation Master 2010 Final 2
Ambasoft Presentation Master 2010 Final  2Ambasoft Presentation Master 2010 Final  2
Ambasoft Presentation Master 2010 Final 2Ed Stevenson
 
Ambasoft Presentation Master 2010 Final 2
Ambasoft Presentation Master 2010 Final  2Ambasoft Presentation Master 2010 Final  2
Ambasoft Presentation Master 2010 Final 2Ed Stevenson
 
Imaginea Overview
Imaginea OverviewImaginea Overview
Imaginea OverviewJimit Shah
 
Successful Agile Transformation - The NCS Story
Successful Agile Transformation - The NCS StorySuccessful Agile Transformation - The NCS Story
Successful Agile Transformation - The NCS StoryNUS-ISS
 
Agile driven development main principles
Agile driven development main principlesAgile driven development main principles
Agile driven development main principlesTanya Epstein
 
Agile, TOGAF and Enterprise Architecture: Will They Blend?
Agile, TOGAF and Enterprise Architecture:  Will They Blend?Agile, TOGAF and Enterprise Architecture:  Will They Blend?
Agile, TOGAF and Enterprise Architecture: Will They Blend?Danny Greefhorst
 
The Evolution of the Enterprise Operating Model - Ryan Lockard
The Evolution of the Enterprise Operating Model - Ryan LockardThe Evolution of the Enterprise Operating Model - Ryan Lockard
The Evolution of the Enterprise Operating Model - Ryan Lockardagilemaine
 
Quality at the speed of digital
Quality   at the speed of digitalQuality   at the speed of digital
Quality at the speed of digitalrajni singh
 
Software_Development_Master_Document
Software_Development_Master_DocumentSoftware_Development_Master_Document
Software_Development_Master_DocumentAKSHAY ASSOCIATE
 
Will They Blend? - Agile, TOGAF and Enterprise Architecture
Will They Blend? - Agile, TOGAF and Enterprise ArchitectureWill They Blend? - Agile, TOGAF and Enterprise Architecture
Will They Blend? - Agile, TOGAF and Enterprise ArchitectureITpreneurs
 
Service Analysis And Design
Service Analysis And DesignService Analysis And Design
Service Analysis And DesignRody Middelkoop
 
Scaling Agile Transformation in a Waterfall Enterprise
Scaling Agile Transformation in a Waterfall EnterpriseScaling Agile Transformation in a Waterfall Enterprise
Scaling Agile Transformation in a Waterfall EnterpriseVMware Tanzu
 
The Power and Promise of SaaS: CA Cloud Service Management Case Study
The Power and Promise of SaaS: CA Cloud Service Management Case StudyThe Power and Promise of SaaS: CA Cloud Service Management Case Study
The Power and Promise of SaaS: CA Cloud Service Management Case StudyCA Technologies
 
DevOps: Retooling the End-to-End IT Model
DevOps: Retooling the End-to-End IT ModelDevOps: Retooling the End-to-End IT Model
DevOps: Retooling the End-to-End IT ModelCA Technologies
 
Accenture SaaS Financial Applications
Accenture SaaS Financial ApplicationsAccenture SaaS Financial Applications
Accenture SaaS Financial ApplicationsDave Kellogg
 
Achieving DevOps using Open Source Tools in the Enterprise
Achieving DevOps using Open Source Tools in the EnterpriseAchieving DevOps using Open Source Tools in the Enterprise
Achieving DevOps using Open Source Tools in the EnterpriseCollabNet
 
Akhil Bhan- Resume
Akhil Bhan- ResumeAkhil Bhan- Resume
Akhil Bhan- ResumeAKHIL BHAN
 
ITPlusPoint | Delivering top-notch application development and business proce...
ITPlusPoint | Delivering top-notch application development and business proce...ITPlusPoint | Delivering top-notch application development and business proce...
ITPlusPoint | Delivering top-notch application development and business proce...Saurav Mohanty
 

Similar to Continuous Quality: What DevOps Means for QA (20)

Full Service Ownership
Full Service OwnershipFull Service Ownership
Full Service Ownership
 
Ambasoft Presentation Master 2010 Final 2
Ambasoft Presentation Master 2010 Final  2Ambasoft Presentation Master 2010 Final  2
Ambasoft Presentation Master 2010 Final 2
 
Ambasoft Presentation Master 2010 Final 2
Ambasoft Presentation Master 2010 Final  2Ambasoft Presentation Master 2010 Final  2
Ambasoft Presentation Master 2010 Final 2
 
TapoResume2015
TapoResume2015TapoResume2015
TapoResume2015
 
Imaginea Overview
Imaginea OverviewImaginea Overview
Imaginea Overview
 
Successful Agile Transformation - The NCS Story
Successful Agile Transformation - The NCS StorySuccessful Agile Transformation - The NCS Story
Successful Agile Transformation - The NCS Story
 
Agile driven development main principles
Agile driven development main principlesAgile driven development main principles
Agile driven development main principles
 
Agile, TOGAF and Enterprise Architecture: Will They Blend?
Agile, TOGAF and Enterprise Architecture:  Will They Blend?Agile, TOGAF and Enterprise Architecture:  Will They Blend?
Agile, TOGAF and Enterprise Architecture: Will They Blend?
 
The Evolution of the Enterprise Operating Model - Ryan Lockard
The Evolution of the Enterprise Operating Model - Ryan LockardThe Evolution of the Enterprise Operating Model - Ryan Lockard
The Evolution of the Enterprise Operating Model - Ryan Lockard
 
Quality at the speed of digital
Quality   at the speed of digitalQuality   at the speed of digital
Quality at the speed of digital
 
Software_Development_Master_Document
Software_Development_Master_DocumentSoftware_Development_Master_Document
Software_Development_Master_Document
 
Will They Blend? - Agile, TOGAF and Enterprise Architecture
Will They Blend? - Agile, TOGAF and Enterprise ArchitectureWill They Blend? - Agile, TOGAF and Enterprise Architecture
Will They Blend? - Agile, TOGAF and Enterprise Architecture
 
Service Analysis And Design
Service Analysis And DesignService Analysis And Design
Service Analysis And Design
 
Scaling Agile Transformation in a Waterfall Enterprise
Scaling Agile Transformation in a Waterfall EnterpriseScaling Agile Transformation in a Waterfall Enterprise
Scaling Agile Transformation in a Waterfall Enterprise
 
The Power and Promise of SaaS: CA Cloud Service Management Case Study
The Power and Promise of SaaS: CA Cloud Service Management Case StudyThe Power and Promise of SaaS: CA Cloud Service Management Case Study
The Power and Promise of SaaS: CA Cloud Service Management Case Study
 
DevOps: Retooling the End-to-End IT Model
DevOps: Retooling the End-to-End IT ModelDevOps: Retooling the End-to-End IT Model
DevOps: Retooling the End-to-End IT Model
 
Accenture SaaS Financial Applications
Accenture SaaS Financial ApplicationsAccenture SaaS Financial Applications
Accenture SaaS Financial Applications
 
Achieving DevOps using Open Source Tools in the Enterprise
Achieving DevOps using Open Source Tools in the EnterpriseAchieving DevOps using Open Source Tools in the Enterprise
Achieving DevOps using Open Source Tools in the Enterprise
 
Akhil Bhan- Resume
Akhil Bhan- ResumeAkhil Bhan- Resume
Akhil Bhan- Resume
 
ITPlusPoint | Delivering top-notch application development and business proce...
ITPlusPoint | Delivering top-notch application development and business proce...ITPlusPoint | Delivering top-notch application development and business proce...
ITPlusPoint | Delivering top-notch application development and business proce...
 

Recently uploaded

NIST Cybersecurity Framework (CSF) 2.0 Workshop
NIST Cybersecurity Framework (CSF) 2.0 WorkshopNIST Cybersecurity Framework (CSF) 2.0 Workshop
NIST Cybersecurity Framework (CSF) 2.0 WorkshopBachir Benyammi
 
Videogame localization & technology_ how to enhance the power of translation.pdf
Videogame localization & technology_ how to enhance the power of translation.pdfVideogame localization & technology_ how to enhance the power of translation.pdf
Videogame localization & technology_ how to enhance the power of translation.pdfinfogdgmi
 
Machine Learning Model Validation (Aijun Zhang 2024).pdf
Machine Learning Model Validation (Aijun Zhang 2024).pdfMachine Learning Model Validation (Aijun Zhang 2024).pdf
Machine Learning Model Validation (Aijun Zhang 2024).pdfAijun Zhang
 
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPA
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPAAnypoint Code Builder , Google Pub sub connector and MuleSoft RPA
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPAshyamraj55
 
20230202 - Introduction to tis-py
20230202 - Introduction to tis-py20230202 - Introduction to tis-py
20230202 - Introduction to tis-pyJamie (Taka) Wang
 
Linked Data in Production: Moving Beyond Ontologies
Linked Data in Production: Moving Beyond OntologiesLinked Data in Production: Moving Beyond Ontologies
Linked Data in Production: Moving Beyond OntologiesDavid Newbury
 
Comparing Sidecar-less Service Mesh from Cilium and Istio
Comparing Sidecar-less Service Mesh from Cilium and IstioComparing Sidecar-less Service Mesh from Cilium and Istio
Comparing Sidecar-less Service Mesh from Cilium and IstioChristian Posta
 
Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...
Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...
Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...Will Schroeder
 
9 Steps For Building Winning Founding Team
9 Steps For Building Winning Founding Team9 Steps For Building Winning Founding Team
9 Steps For Building Winning Founding TeamAdam Moalla
 
COMPUTER 10: Lesson 7 - File Storage and Online Collaboration
COMPUTER 10: Lesson 7 - File Storage and Online CollaborationCOMPUTER 10: Lesson 7 - File Storage and Online Collaboration
COMPUTER 10: Lesson 7 - File Storage and Online Collaborationbruanjhuli
 
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCostKubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCostMatt Ray
 
COMPUTER 10 Lesson 8 - Building a Website
COMPUTER 10 Lesson 8 - Building a WebsiteCOMPUTER 10 Lesson 8 - Building a Website
COMPUTER 10 Lesson 8 - Building a Websitedgelyza
 
UiPath Studio Web workshop series - Day 8
UiPath Studio Web workshop series - Day 8UiPath Studio Web workshop series - Day 8
UiPath Studio Web workshop series - Day 8DianaGray10
 
Computer 10: Lesson 10 - Online Crimes and Hazards
Computer 10: Lesson 10 - Online Crimes and HazardsComputer 10: Lesson 10 - Online Crimes and Hazards
Computer 10: Lesson 10 - Online Crimes and HazardsSeth Reyes
 
IaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdf
IaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdfIaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdf
IaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdfDaniel Santiago Silva Capera
 
Crea il tuo assistente AI con lo Stregatto (open source python framework)
Crea il tuo assistente AI con lo Stregatto (open source python framework)Crea il tuo assistente AI con lo Stregatto (open source python framework)
Crea il tuo assistente AI con lo Stregatto (open source python framework)Commit University
 
UiPath Platform: The Backend Engine Powering Your Automation - Session 1
UiPath Platform: The Backend Engine Powering Your Automation - Session 1UiPath Platform: The Backend Engine Powering Your Automation - Session 1
UiPath Platform: The Backend Engine Powering Your Automation - Session 1DianaGray10
 
Artificial Intelligence & SEO Trends for 2024
Artificial Intelligence & SEO Trends for 2024Artificial Intelligence & SEO Trends for 2024
Artificial Intelligence & SEO Trends for 2024D Cloud Solutions
 

Recently uploaded (20)

NIST Cybersecurity Framework (CSF) 2.0 Workshop
NIST Cybersecurity Framework (CSF) 2.0 WorkshopNIST Cybersecurity Framework (CSF) 2.0 Workshop
NIST Cybersecurity Framework (CSF) 2.0 Workshop
 
Videogame localization & technology_ how to enhance the power of translation.pdf
Videogame localization & technology_ how to enhance the power of translation.pdfVideogame localization & technology_ how to enhance the power of translation.pdf
Videogame localization & technology_ how to enhance the power of translation.pdf
 
Machine Learning Model Validation (Aijun Zhang 2024).pdf
Machine Learning Model Validation (Aijun Zhang 2024).pdfMachine Learning Model Validation (Aijun Zhang 2024).pdf
Machine Learning Model Validation (Aijun Zhang 2024).pdf
 
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPA
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPAAnypoint Code Builder , Google Pub sub connector and MuleSoft RPA
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPA
 
20230202 - Introduction to tis-py
20230202 - Introduction to tis-py20230202 - Introduction to tis-py
20230202 - Introduction to tis-py
 
Linked Data in Production: Moving Beyond Ontologies
Linked Data in Production: Moving Beyond OntologiesLinked Data in Production: Moving Beyond Ontologies
Linked Data in Production: Moving Beyond Ontologies
 
Comparing Sidecar-less Service Mesh from Cilium and Istio
Comparing Sidecar-less Service Mesh from Cilium and IstioComparing Sidecar-less Service Mesh from Cilium and Istio
Comparing Sidecar-less Service Mesh from Cilium and Istio
 
Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...
Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...
Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...
 
9 Steps For Building Winning Founding Team
9 Steps For Building Winning Founding Team9 Steps For Building Winning Founding Team
9 Steps For Building Winning Founding Team
 
201610817 - edge part1
201610817 - edge part1201610817 - edge part1
201610817 - edge part1
 
COMPUTER 10: Lesson 7 - File Storage and Online Collaboration
COMPUTER 10: Lesson 7 - File Storage and Online CollaborationCOMPUTER 10: Lesson 7 - File Storage and Online Collaboration
COMPUTER 10: Lesson 7 - File Storage and Online Collaboration
 
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCostKubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
 
COMPUTER 10 Lesson 8 - Building a Website
COMPUTER 10 Lesson 8 - Building a WebsiteCOMPUTER 10 Lesson 8 - Building a Website
COMPUTER 10 Lesson 8 - Building a Website
 
UiPath Studio Web workshop series - Day 8
UiPath Studio Web workshop series - Day 8UiPath Studio Web workshop series - Day 8
UiPath Studio Web workshop series - Day 8
 
Computer 10: Lesson 10 - Online Crimes and Hazards
Computer 10: Lesson 10 - Online Crimes and HazardsComputer 10: Lesson 10 - Online Crimes and Hazards
Computer 10: Lesson 10 - Online Crimes and Hazards
 
IaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdf
IaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdfIaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdf
IaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdf
 
Crea il tuo assistente AI con lo Stregatto (open source python framework)
Crea il tuo assistente AI con lo Stregatto (open source python framework)Crea il tuo assistente AI con lo Stregatto (open source python framework)
Crea il tuo assistente AI con lo Stregatto (open source python framework)
 
20230104 - machine vision
20230104 - machine vision20230104 - machine vision
20230104 - machine vision
 
UiPath Platform: The Backend Engine Powering Your Automation - Session 1
UiPath Platform: The Backend Engine Powering Your Automation - Session 1UiPath Platform: The Backend Engine Powering Your Automation - Session 1
UiPath Platform: The Backend Engine Powering Your Automation - Session 1
 
Artificial Intelligence & SEO Trends for 2024
Artificial Intelligence & SEO Trends for 2024Artificial Intelligence & SEO Trends for 2024
Artificial Intelligence & SEO Trends for 2024
 

Continuous Quality: What DevOps Means for QA

  • 1. + Continuous Quality: What DevOps Means for QA Jeff Sussna, Ingineering.IT @jeffsussna Copyright © 2013 Ingineering.IT, LLC. All Rights Reserved.
  • 2. My Career Jobs:  Dev: from compilers to content management systems  QA: automation through the ages  Ops: enterprise, managed hosting, ITIL, ops automation, cloud Design thinker with a penchant for software systems Inveterate boundary-spanner and synthesist Copyright © 2013 Ingineering.IT, LLC. All Rights Reserved.
  • 3. My Practice IT Service Innovation consultant  Clients: Small development shops and large enterprise IT Help Dev/QA/Ops work more closely with each other Help them better understand each other and their customers Copyright © 2013 Ingineering.IT, LLC. All Rights Reserved.
  • 4. The Old Definition of Quality The software meets the spec Copyright © 2013 Ingineering.IT, LLC. All Rights Reserved.
  • 5. …and then… Agile…  put the focus on delivering value  changed the development/QA relationship Cloud…  turned software into service Copyright © 2013 Ingineering.IT, LLC. All Rights Reserved.
  • 6. The New Definition of Quality The service helps customers accomplish their jobs-to-be-done Copyright © 2013 Ingineering.IT, LLC. All Rights Reserved.
  • 7. Service Co-creates Value Service provider helps customer create value-in-use Restaurant:  Not just about acquiring food  Also about a having a relaxing, romantic night out Cloud:  Not just about acquiring Linux instances  Also about cost-effectively operating elastic, resilient applications Copyright © 2013 Ingineering.IT, LLC. All Rights Reserved.
  • 8. Value Co-Creation is a Journey Customers engage via multiple touchpoints over time Customers judge service by the entirety of their experience "The food was great but the service was terrible" Copyright © 2013 Ingineering.IT, LLC. All Rights Reserved.
  • 9. Restaurant Journey:  Find a restaurant  Make a reservation  Park  Wait for a table, drink at the bar  Order  Eat  Pay  Recommend (or not) Touchpoints:  Yellow pages, phone, valet, bar, waiter, food, cashier, social media  

  • 10. Cloud Journey:  Understand a new service or feature  Provision instances  Import data  Get help/provide feedback  Incorporate updates  Deal with outages  Deprovision Touchpoints:  Website, console, API, documentation, blogs, email, phone, twitter  

  • 11. Software as Service Co-creation has four dimensions:  Functionality  Operability  Deliverability  Coherency Their quality is inseparable in the customer’s mind Copyright © 2013 Ingineering.IT, LLC. All Rights Reserved.
  • 12. Functionality Does the service help me do what I want to do (e.g., get paid on time)?  Track my time  Generate and submit invoices  Pester clients who haven't paid on time  Track my overall cash flow Copyright © 2013 Ingineering.IT, LLC. All Rights Reserved.
  • 13. Operability Is the service available when and how I need it?  Is it up when I need to bill my clients?  Does it perform whether I’m the only user or one of a million?  Does it let my data get lost or stolen?  Can I get help when I need it?  Can I get information when it’s down? Copyright © 2013 Ingineering.IT, LLC. All Rights Reserved.
  • 14. Deliverability Does the service respond as my needs change?  70% functionality OK on day 1, not in year 2  Today I want to manage invoices, tomorrow expenses  I want bugs fixed in hours, not days/weeks/months  I want changes delivered in an absorbable manner Copyright © 2013 Ingineering.IT, LLC. All Rights Reserved.
  • 15. Coherency Does the service engage me throughout my journey?  Discovery/understanding  Adoption/onboarding  Administration  Usage  Help/feedback  New functionality delivery  Outage management  Departure Copyright © 2013 Ingineering.IT, LLC. All Rights Reserved.
  • 16. Delivering Software as Service Service changes how software is built and operated  User-centered design  Test-driven development  Continuous delivery  MTTR over MTBF 21st-century IT builds in testing and learning-from-failure Copyright © 2013 Ingineering.IT, LLC. All Rights Reserved.
  • 17. Old QA Role Disintegrates Boundaries between QA and Design/Dev/Ops blur Automation takes over mechanical activities Copyright © 2013 Ingineering.IT, LLC. All Rights Reserved.
  • 18. Opportunity for New QA Role Representative of “service not software” perspective:  Four dimensions of software as service  Their inseparability from perspective of quality  Co-creation as fundamental unit of value Boundary-spanning mirror Copyright © 2013 Ingineering.IT, LLC. All Rights Reserved.
  • 19. New QA Mindset Lift one’s gaze above mechanics of testing Treat tests as code  Adopt development tools and practices Focus on building quality into the system  Tester => Quality Advocate #seewhatIdidthere Copyright © 2013 Ingineering.IT, LLC. All Rights Reserved.
  • 20. New QA Skills Understand and think about service  Outages involve communications, not just fixing servers Understand and think about ops Understand and think about process/automation Copyright © 2013 Ingineering.IT, LLC. All Rights Reserved.
  • 21. New QA Practices Deep participation in cross-functional teams Facilitation of service requirements understanding Outside-in testing Service delivery machine validation Copyright © 2013 Ingineering.IT, LLC. All Rights Reserved.
  • 22. Participation Good testers see systems and their parts Good testers ask probing questions  Did you think about…?  You forgot about… Good testers design good tests Engage that proficiency in design and test plan critiques Copyright © 2013 Ingineering.IT, LLC. All Rights Reserved.
  • 23. Requirements Understanding Bugs found earlier in the lifecycle are cheaper to fix Most important quality question:  Do design/dev/qa/ops understand each other and the customer? Techniques for modeling service requirements:  Customer Journey Mapping  Behavior-Driven Development Copyright © 2013 Ingineering.IT, LLC. All Rights Reserved.
  • 24. Outside-in Testing User testing validates usability Unit testing validates code Still need to test whether the thing does what it’s supposed to:  Functional tests  System tests  Integration tests Copyright © 2013 Ingineering.IT, LLC. All Rights Reserved.
  • 25. Continuous Outside-in Testing Continuous delivery challenges testers’ ability to keep up Solution:  Part software tools and design skills  Part agile process improvement Copyright © 2013 Ingineering.IT, LLC. All Rights Reserved.
  • 26. Delivery Machine Validation Operability and deliverability need outside-in testing too  BDD (not just for functional requirements) Co-creation necessitates designing for internal users  Service Blueprinting Copyright © 2013 Ingineering.IT, LLC. All Rights Reserved.
  • 27. New QA Value Unique holder of service understanding across all dimensions Antidote to confirmation bias  Everyone else is invested in what they designed and built  QA is invested in customer satisfaction Copyright © 2013 Ingineering.IT, LLC. All Rights Reserved.
  • 28. Continuous Quality Old: tail wagging the dog  Testing happens at the end  QA acts as bottleneck  Everyone hates QA New: continuous quality  Testing happens all the time  Entire process and organization infused with quality  QA acts as mirror Copyright © 2013 Ingineering.IT, LLC. All Rights Reserved.
  • 29. QA As Mirror DevOps is about spanning boundaries Service is about inseparable value Need to translate between common and specialist viewpoints QA can provide a mirror that helps everyone see:  The whole  Their relationship to it and to each other Copyright © 2013 Ingineering.IT, LLC. All Rights Reserved.
  • 30. Delivery and Co-Creation The product is not the functionality The product is the delivery and operation of functionality True co-creation dissolves customer/provider boundaries Copyright © 2013 Ingineering.IT, LLC. All Rights Reserved.
  • 31. The Ultimate Definition of Quality The ability to continuously deliver and operate functionality that helps customers accomplish their jobs-to-be-done Copyright © 2013 Ingineering.IT, LLC. All Rights Reserved.
  • 32. Props to @guidostompff 'Boundary-Spanning Mirroring' PhD thesis http://designinteams.com/
  • 33. More Info Contact me about Continuous Quality consulting services:  info@ingineering.it  @jeffsussna  http://www.linkedin.com/in/jeffsussna Read my thoughts on DevOps, Cloud, and Design Thinking:  http://blog.ingineering.it Copyright © 2013 Ingineering.IT, LLC. All Rights Reserved.

Editor's Notes

  1. Advantage of being around since rocks were young is you do lots of stuffMy career has spanned dev qa opsHad oppt to do a lot of each of them
  2. Now that I’m a consultant I spend my time help dev qa ops better understand and work more closely with each otherI do it from the perspective of my personal obsession with interdisciplinarity and synthesis
  3. When I look back to the early days of my career I remember applying for a QA Eng jobMechanicalNo input into goodness of softwareOnly input into spec if we couldn’t test
  4. Key to talk: we’re in the business of service not software
  5. Service helps customer accomplish something they’re trying to do
  6. A defining characteristic of service is value cocreationNot just exchange of money for value-filled thingsAmazon gets this
  7. Another defining characteristic of service is the customer journey
  8. Customer does much more than just eatInteracts with much more than just food
  9. Customer does much more than just spin up serversInteracts with much more than just API
  10. Tracking cash flow requires integration is part of larger JTBD
  11. Outage info points to JTBD
  12. Outage management engages me as ops too - JTBD
  13. Not only does service change what software is, but also how it’s deliveredTesting built inNew reliance on automationNew view of approach to failures
  14. Reimagine QA as something concerned with 4 dimensions and inseparability as characteristic of qualityThat plays profound role enabling devops altogether
  15. Adopt new mindsetLife gaze from traditional mechanical activities of execution and reportingNew job is quality advocacy
  16. Do game days test service or just softwareOps: performance scalability security resilienceAlso monitoring and support
  17. Deeply embed them Just because you have unit tests doesn’t mean their goodWhy would you want crabby testers telling you what’s wrong with everything?Think about good testers you’ve worked with
  18. Modeling requirements allows us to talk about them and uncover misunderstanding confusion gaps
  19. Opportunity for software engineering practice
  20. Most important is planning
  21. Do monitoring tools work properlyDoes deployment automation work properly
  22. When bugs get into the wild QA gets blamed
  23. Goal/challenge is to move from old to new
  24. Some good at css, some at chef. Diff langs need to contribute to larger wholeQuality includes optimizing at the right local-global level
  25. What is this whole thing that we’re delivering? Not func but op/dev of funcWhen you think about it, aws is part of your ops team
  26. Ultimate def of what QA needs to validate