Continuous Quality: What DevOps Means for QA

+
Continuous Quality: What
DevOps Means for QA
Jeff Sussna, Ingineering.IT
@jeffsussna
Copyright © 2013 Ingineering.IT, LLC.
All Rights Reserved.

My Career
Jobs:
 Dev: from compilers to content management systems
 QA: automation through the ages
 Ops: enterprise, managed hosting, ITIL, ops automation, cloud
Design thinker with a penchant for software systems
Inveterate boundary-spanner and synthesist
Copyright © 2013 Ingineering.IT, LLC. All Rights Reserved.

My Practice
IT Service Innovation consultant
 Clients: Small development shops and large enterprise IT
Help Dev/QA/Ops work more closely with each other
Help them better understand each other and their customers

The Old Definition of Quality
The software meets the spec

…and then…
Agile…
 put the focus on delivering value
 changed the development/QA relationship
Cloud…
 turned software into service

The New Definition of Quality
The service helps customers accomplish their jobs-to-be-done

Service Co-creates Value
Service provider helps customer create value-in-use
Restaurant:
 Not just about acquiring food
 Also about a having a relaxing, romantic night out
Cloud:
 Not just about acquiring Linux instances
 Also about cost-effectively operating elastic, resilient applications

Value Co-Creation is a Journey
Customers engage via multiple touchpoints over time
Customers judge service by the entirety of their experience
"The food was great but the service was terrible"

Restaurant
Journey:
 Find a restaurant
 Make a reservation
 Park
 Wait for a table, drink at the bar
 Order
 Eat
 Pay
 Recommend (or not)
Touchpoints:
 Yellow pages, phone, valet, bar, waiter, food, cashier, social media


Cloud
Journey:
 Understand a new service or feature
 Provision instances
 Import data
 Get help/provide feedback
 Incorporate updates
 Deal with outages
 Deprovision
Touchpoints:
 Website, console, API, documentation, blogs, email, phone, twitter


Software as Service
Co-creation has four dimensions:
 Functionality
 Operability
 Deliverability
 Coherency
Their quality is inseparable in the customer’s mind

Functionality
Does the service help me do what I want to do (e.g., get paid on
time)?
 Track my time
 Generate and submit invoices
 Pester clients who haven't paid on time
 Track my overall cash flow

Operability
Is the service available when and how I need it?
 Is it up when I need to bill my clients?
 Does it perform whether I’m the only user or one of a million?
 Does it let my data get lost or stolen?
 Can I get help when I need it?
 Can I get information when it’s down?

Deliverability
Does the service respond as my needs change?
 70% functionality OK on day 1, not in year 2
 Today I want to manage invoices, tomorrow expenses
 I want bugs fixed in hours, not days/weeks/months
 I want changes delivered in an absorbable manner

Coherency
Does the service engage me throughout my journey?
 Discovery/understanding
 Adoption/onboarding
 Administration
 Usage
 Help/feedback
 New functionality delivery
 Outage management
 Departure

Delivering Software as Service
Service changes how software is built and operated
 User-centered design
 Test-driven development
 Continuous delivery
 MTTR over MTBF
21st-century IT builds in testing and learning-from-failure

Old QA Role Disintegrates
Boundaries between QA and Design/Dev/Ops blur
Automation takes over mechanical activities

Opportunity for New QA Role
Representative of “service not software” perspective:
 Four dimensions of software as service
 Their inseparability from perspective of quality
 Co-creation as fundamental unit of value
Boundary-spanning mirror

New QA Mindset
Lift one’s gaze above mechanics of testing
Treat tests as code
 Adopt development tools and practices
Focus on building quality into the system
 Tester => Quality Advocate #seewhatIdidthere

New QA Skills
Understand and think about service
 Outages involve communications, not just fixing servers
Understand and think about ops
Understand and think about process/automation

New QA Practices
Deep participation in cross-functional teams
Facilitation of service requirements understanding
Outside-in testing
Service delivery machine validation

Participation
Good testers see systems and their parts
Good testers ask probing questions
 Did you think about…?
 You forgot about…
Good testers design good tests
Engage that proficiency in design and test plan critiques

Requirements Understanding
Bugs found earlier in the lifecycle are cheaper to fix
Most important quality question:
 Do design/dev/qa/ops understand each other and the customer?
Techniques for modeling service requirements:
 Customer Journey Mapping
 Behavior-Driven Development

Outside-in Testing
User testing validates usability
Unit testing validates code
Still need to test whether the thing does what it’s supposed to:
 Functional tests
 System tests
 Integration tests

Continuous Outside-in Testing
Continuous delivery challenges testers’ ability to keep up
Solution:
 Part software tools and design skills
 Part agile process improvement

Delivery Machine Validation
Operability and deliverability need outside-in testing too
 BDD (not just for functional requirements)
Co-creation necessitates designing for internal users
 Service Blueprinting

New QA Value
Unique holder of service understanding across all dimensions
Antidote to confirmation bias
 Everyone else is invested in what they designed and built
 QA is invested in customer satisfaction

Continuous Quality
Old: tail wagging the dog
 Testing happens at the end
 QA acts as bottleneck
 Everyone hates QA
New: continuous quality
 Testing happens all the time
 Entire process and organization infused with quality
 QA acts as mirror

QA As Mirror
DevOps is about spanning boundaries
Service is about inseparable value
Need to translate between common and specialist viewpoints
QA can provide a mirror that helps everyone see:
 The whole
 Their relationship to it and to each other

Delivery and Co-Creation
The product is not the functionality
The product is the delivery and operation of functionality
True co-creation dissolves customer/provider boundaries

The Ultimate Definition of Quality
The ability to continuously deliver and operate functionality that
helps customers accomplish their jobs-to-be-done

Props to @guidostompff
'Boundary-Spanning Mirroring' PhD thesis
http://designinteams.com/

More Info
Contact me about Continuous Quality consulting services:
 info@ingineering.it
 @jeffsussna
 http://www.linkedin.com/in/jeffsussna
Read my thoughts on DevOps, Cloud, and Design Thinking:
 http://blog.ingineering.it

Continuous Quality: What DevOps Means for QA

More Related Content

What's hot

Viewers also liked

Similar to Continuous Quality: What DevOps Means for QA

Recently uploaded

Continuous Quality: What DevOps Means for QA

Editor's Notes