TESTINGMIND
www.testingmind.com
2ND ANNUAL SOFTWARE TESTING
SYMPOSIUM
#STS18
Melbourne, Australia
May 03, 2018
TESTINGMIND
www.testingmind.com
WHAT GOT US HERE – WONT GET US THERE
A STORY OF TRANSFORMATIONS
• Mirco Hering
• APAC lead for Agile and DevOps
TESTINGMIND CONSULTING 2nd Annual Software Testing Symposium 2018 | Melbourne, Australia
ABOUT MEMirco Hering
APAC Lead for DevOps & Agile
@mircohering on Twitter
Upcoming BookBlog at http://notafactoryanymore.comAt work
• Run a team of passionate
transformation agents
• Create solutions for clients
• Fix problems
@mircohering #notafactoryanymore
TESTINGMIND CONSULTING 2nd Annual Software Testing Symposium 2018 | Melbourne, Australia
THE PROBLEM AT HAND
https://devops.com/still-fighting-problems-devops-15-years-ago/
@mircohering #notafactoryanymore
TESTINGMIND CONSULTING 2nd Annual Software Testing Symposium 2018 | Melbourne, Australia
AN ATTEMPT OF AN EXPLANATION
We are here
DevOps Practices & Tools
DevOps
Organisation
@mircohering #notafactoryanymore
TESTINGMIND CONSULTING 2nd Annual Software Testing Symposium 2018 | Melbourne, Australia
THE TRANSFORMATION LIFECYCLE
Stable State Stable State
Stable State
@mircohering #notafactoryanymore
TESTINGMIND CONSULTING 2nd Annual Software Testing Symposium 2018 | Melbourne, Australia
THE TRANSFORMATION LIFECYCLE
Ongoing stimulus
@mircohering #notafactoryanymore
TESTINGMIND CONSULTING 2nd Annual Software Testing Symposium 2018 | Melbourne, Australia
WORKING WITH THE WRONG MENTAL MODEL
@mircohering #notafactoryanymore
TESTINGMIND CONSULTING 2nd Annual Software Testing Symposium 2018 | Melbourne, Australia
MISALIGNED PRINCIPLES AND PROCESSES
“Rigth now, your company has
21st century Internet enabled business
processes,
Mid 20th century management
processes all built
Atop 19th century management
principles.”
- Gary Hamel, American Management
Expert
@mircohering #notafactoryanymore
TESTINGMIND CONSULTING 2nd Annual Software Testing Symposium 2018 | Melbourne, Australia
IT CAN BE MANAGED BASED ON
ENGINEERING/MANUFACTURING PRINCIPLES
Predictable Production Process allowing you
to measure Productivity and define output
Economies of Scale and effort of scaling
Based on functional specialisation of labour
Importance of upfront planning
Automation is improving productivity
@mircohering #notafactoryanymore
TESTINGMIND CONSULTING 2nd Annual Software Testing Symposium 2018 | Melbourne, Australia
NOT A FACTORY ANYMORE?
@mircohering #notafactoryanymore
TESTINGMIND CONSULTING 2nd Annual Software Testing Symposium 2018 | Melbourne, Australia
THREE DIMENSIONS OF CHANGE
Organising
and
managing
knowledge
workers in IT
Technology
Architectures
Ecosystem of
vendors and
applications
Rigorous
Continuous
Improvement
Innovation
Speed
Agility
Quality
@mircohering #notafactoryanymore
TESTINGMIND CONSULTING 2nd Annual Software Testing Symposium 2018 | Melbourne, Australia
TECHNOLOGY – ENABLING SMALL BATCHES
Optimum
batch size
(lowest total
cost)
Items per batch
Cost
Optimum
batch size
(lowest total
cost)
Items per batch
Cost
@mircohering #notafactoryanymore
TESTINGMIND CONSULTING 2nd Annual Software Testing Symposium 2018 | Melbourne, Australia
TECHNOLOGY – EVOLVING ARCHITECTURE
Consolidated
core system(s)
Access
Simplify
core
Access
Core as-is
Value Added
Services
Access
Core
Access
Modular
Core as-is
Access
Silo
“We need a simpler
core and we are also
going to simplify our
processes and
products”
“We need agility on
top of our core
systems to become
digitally enabled”
“We will try this out to
test if it will work for
us, putting it in the
core now is not
feasible”
“We have different
businesses with
different agendas,
one size does not fit
all; decouple”
“Our core systems
need more functions
/ we need to
consolidate across
our businesses”
1
2
3
4
5
The traditional stating
point / goal
@mircohering #notafactoryanymore
TESTINGMIND CONSULTING 2nd Annual Software Testing Symposium 2018 | Melbourne, Australia
TECHNOLOGY – GOVERNING DELIVERY
@mircohering #notafactoryanymore
TESTINGMIND CONSULTING 2nd Annual Software Testing Symposium 2018 | Melbourne, Australia
TECHNOLOGY – TEST AUTOMATION
16
TA
Approach
Framework
Test Data
Distribution
Execution
Reporting
Maintenance
• Orchestrated through CI/CD
process
• Parallel execution
• Execution reporting (e.g. SMTP)
• Metrics, trends, status
• Less than 15% of development effort
• Impacted by framework &
distribution
• Coding standards, guidelines
• Design & architecture patterns
• Tools
• Refresh, masking
• API driven
• Unit > Service Layer > UI
TESTINGMIND CONSULTING 2nd Annual Software Testing Symposium 2018 | Melbourne, Australia
TECHNOLOGY – TEST DATA
1
7
Test Data Management Service Components
Data Refresh
Refresh Data from source
(typically production)
environments
Data Masking /
Obfuscation
Protection of data to
meet data security and
compliance objectives
Data Generation
Synthesis
Generate realistic, high
quality data when
production data is not
available
Data Analytics
Data Mining
Extract data needed
for testing or identify
the right data
conditions needed to
test a functionality
Data Integration
Automation
Integrate data with
testing tools to
automate testing and
enable self-service test
data options
Data Sub-
Setting
Smaller slice of a
trusted source system
that meets the needs
of testing
Automated Environment
Provisioning
Automated Build and
Deployment
Service
Virtualization Integrated Test AutomationTest Data
Management
Test Data Management is an integral service capability of an end-to-end automation solution
TESTINGMIND CONSULTING 2nd Annual Software Testing Symposium 2018 | Melbourne, Australia
PEOPLE – FROM TESTERS TO QUALITY ENGINEERS
1
8
From…. Quality issues accumulate and are manually
captured at the end …
Transforming to…. Automation and process changes
capture quality issues at the point of origin and enable
faster, more frequent and less costly releases.
ST SIT UAT
ST SIT/UAT
ST SIT/UAT
ST SIT/UAT
Example Avg. 22 to 26 wks
Avg. 4 to 12 wks
Integrated Upstream Quality Focus Increase in Speed to Market
From…. Long release cycles, heavy SIT/UAT testing effort,
highly dependent functions …
Transforming to…. Shorter and more frequent release cycles,
greater independence, feature-driven enablement, merged
IT & Business testing.
TESTINGMIND CONSULTING 2nd Annual Software Testing Symposium 2018 | Melbourne, Australia
PEOPLE – TESTING OR QUALITY
It is a mistake to think of test automation as automating what you otherwise would do
manually. You are finding ways to assess the quality of your product in automated ways and
you will execute it way more often than you would do manual testing.
You are not automating testing, you automate the quality assessment.
1
9
• Underestimating the impact on infrastructure and the ecosystem
• Underestimating the data hunger
• Not thinking about the system
• Not integrating it into the Software Development Life Cycle
• Test automation is always an afterthought and not baked in as part of the original estimates
– and is hence quite often seen as a additional cost
• Good automation requires good skills – consider the ‘pond’ model
TESTINGMIND CONSULTING 2nd Annual Software Testing Symposium 2018 | Melbourne, Australia
ECOSYSTEM – THE EVALUATION CHALLENGE
Automation
Vendor A: 100 Dollar per day
Vendor B: 80 Dollars per day
AVG$
Total$
@mircohering #notafactoryanymore
TESTINGMIND CONSULTING 2nd Annual Software Testing Symposium 2018 | Melbourne, Australia
ECOSYSTEM – THE EVALUATION CHALLENGE -PART 2
First Time Resolution
Rate/Resolution TimeSLAs
@mircohering #notafactoryanymore
TESTINGMIND CONSULTING 2nd Annual Software Testing Symposium 2018 | Melbourne, Australia
SILVER BULLET OR SNAKE OIL?
@mircohering #notafactoryanymore
WANT TO LEARN MORE?
To receive the following:
• A copy of this presentation
• An excerpt of DevOps for the Modern Enterprise
• Videos and slides from my DevOps Enterprise talks
• Most popular blog posts from notafactoryanymore.com
• Get early access to any material coming out with “DevOps
For the Modern Enterprise”
Just pick up your phone, and send an email:
• To: mirco@SendYourSlides.com
• Subject: devops

Testingmind Conference Presentation

  • 1.
    TESTINGMIND www.testingmind.com 2ND ANNUAL SOFTWARETESTING SYMPOSIUM #STS18 Melbourne, Australia May 03, 2018
  • 2.
    TESTINGMIND www.testingmind.com WHAT GOT USHERE – WONT GET US THERE A STORY OF TRANSFORMATIONS • Mirco Hering • APAC lead for Agile and DevOps
  • 3.
    TESTINGMIND CONSULTING 2ndAnnual Software Testing Symposium 2018 | Melbourne, Australia ABOUT MEMirco Hering APAC Lead for DevOps & Agile @mircohering on Twitter Upcoming BookBlog at http://notafactoryanymore.comAt work • Run a team of passionate transformation agents • Create solutions for clients • Fix problems @mircohering #notafactoryanymore
  • 4.
    TESTINGMIND CONSULTING 2ndAnnual Software Testing Symposium 2018 | Melbourne, Australia THE PROBLEM AT HAND https://devops.com/still-fighting-problems-devops-15-years-ago/ @mircohering #notafactoryanymore
  • 5.
    TESTINGMIND CONSULTING 2ndAnnual Software Testing Symposium 2018 | Melbourne, Australia AN ATTEMPT OF AN EXPLANATION We are here DevOps Practices & Tools DevOps Organisation @mircohering #notafactoryanymore
  • 6.
    TESTINGMIND CONSULTING 2ndAnnual Software Testing Symposium 2018 | Melbourne, Australia THE TRANSFORMATION LIFECYCLE Stable State Stable State Stable State @mircohering #notafactoryanymore
  • 7.
    TESTINGMIND CONSULTING 2ndAnnual Software Testing Symposium 2018 | Melbourne, Australia THE TRANSFORMATION LIFECYCLE Ongoing stimulus @mircohering #notafactoryanymore
  • 8.
    TESTINGMIND CONSULTING 2ndAnnual Software Testing Symposium 2018 | Melbourne, Australia WORKING WITH THE WRONG MENTAL MODEL @mircohering #notafactoryanymore
  • 9.
    TESTINGMIND CONSULTING 2ndAnnual Software Testing Symposium 2018 | Melbourne, Australia MISALIGNED PRINCIPLES AND PROCESSES “Rigth now, your company has 21st century Internet enabled business processes, Mid 20th century management processes all built Atop 19th century management principles.” - Gary Hamel, American Management Expert @mircohering #notafactoryanymore
  • 10.
    TESTINGMIND CONSULTING 2ndAnnual Software Testing Symposium 2018 | Melbourne, Australia IT CAN BE MANAGED BASED ON ENGINEERING/MANUFACTURING PRINCIPLES Predictable Production Process allowing you to measure Productivity and define output Economies of Scale and effort of scaling Based on functional specialisation of labour Importance of upfront planning Automation is improving productivity @mircohering #notafactoryanymore
  • 11.
    TESTINGMIND CONSULTING 2ndAnnual Software Testing Symposium 2018 | Melbourne, Australia NOT A FACTORY ANYMORE? @mircohering #notafactoryanymore
  • 12.
    TESTINGMIND CONSULTING 2ndAnnual Software Testing Symposium 2018 | Melbourne, Australia THREE DIMENSIONS OF CHANGE Organising and managing knowledge workers in IT Technology Architectures Ecosystem of vendors and applications Rigorous Continuous Improvement Innovation Speed Agility Quality @mircohering #notafactoryanymore
  • 13.
    TESTINGMIND CONSULTING 2ndAnnual Software Testing Symposium 2018 | Melbourne, Australia TECHNOLOGY – ENABLING SMALL BATCHES Optimum batch size (lowest total cost) Items per batch Cost Optimum batch size (lowest total cost) Items per batch Cost @mircohering #notafactoryanymore
  • 14.
    TESTINGMIND CONSULTING 2ndAnnual Software Testing Symposium 2018 | Melbourne, Australia TECHNOLOGY – EVOLVING ARCHITECTURE Consolidated core system(s) Access Simplify core Access Core as-is Value Added Services Access Core Access Modular Core as-is Access Silo “We need a simpler core and we are also going to simplify our processes and products” “We need agility on top of our core systems to become digitally enabled” “We will try this out to test if it will work for us, putting it in the core now is not feasible” “We have different businesses with different agendas, one size does not fit all; decouple” “Our core systems need more functions / we need to consolidate across our businesses” 1 2 3 4 5 The traditional stating point / goal @mircohering #notafactoryanymore
  • 15.
    TESTINGMIND CONSULTING 2ndAnnual Software Testing Symposium 2018 | Melbourne, Australia TECHNOLOGY – GOVERNING DELIVERY @mircohering #notafactoryanymore
  • 16.
    TESTINGMIND CONSULTING 2ndAnnual Software Testing Symposium 2018 | Melbourne, Australia TECHNOLOGY – TEST AUTOMATION 16 TA Approach Framework Test Data Distribution Execution Reporting Maintenance • Orchestrated through CI/CD process • Parallel execution • Execution reporting (e.g. SMTP) • Metrics, trends, status • Less than 15% of development effort • Impacted by framework & distribution • Coding standards, guidelines • Design & architecture patterns • Tools • Refresh, masking • API driven • Unit > Service Layer > UI
  • 17.
    TESTINGMIND CONSULTING 2ndAnnual Software Testing Symposium 2018 | Melbourne, Australia TECHNOLOGY – TEST DATA 1 7 Test Data Management Service Components Data Refresh Refresh Data from source (typically production) environments Data Masking / Obfuscation Protection of data to meet data security and compliance objectives Data Generation Synthesis Generate realistic, high quality data when production data is not available Data Analytics Data Mining Extract data needed for testing or identify the right data conditions needed to test a functionality Data Integration Automation Integrate data with testing tools to automate testing and enable self-service test data options Data Sub- Setting Smaller slice of a trusted source system that meets the needs of testing Automated Environment Provisioning Automated Build and Deployment Service Virtualization Integrated Test AutomationTest Data Management Test Data Management is an integral service capability of an end-to-end automation solution
  • 18.
    TESTINGMIND CONSULTING 2ndAnnual Software Testing Symposium 2018 | Melbourne, Australia PEOPLE – FROM TESTERS TO QUALITY ENGINEERS 1 8 From…. Quality issues accumulate and are manually captured at the end … Transforming to…. Automation and process changes capture quality issues at the point of origin and enable faster, more frequent and less costly releases. ST SIT UAT ST SIT/UAT ST SIT/UAT ST SIT/UAT Example Avg. 22 to 26 wks Avg. 4 to 12 wks Integrated Upstream Quality Focus Increase in Speed to Market From…. Long release cycles, heavy SIT/UAT testing effort, highly dependent functions … Transforming to…. Shorter and more frequent release cycles, greater independence, feature-driven enablement, merged IT & Business testing.
  • 19.
    TESTINGMIND CONSULTING 2ndAnnual Software Testing Symposium 2018 | Melbourne, Australia PEOPLE – TESTING OR QUALITY It is a mistake to think of test automation as automating what you otherwise would do manually. You are finding ways to assess the quality of your product in automated ways and you will execute it way more often than you would do manual testing. You are not automating testing, you automate the quality assessment. 1 9 • Underestimating the impact on infrastructure and the ecosystem • Underestimating the data hunger • Not thinking about the system • Not integrating it into the Software Development Life Cycle • Test automation is always an afterthought and not baked in as part of the original estimates – and is hence quite often seen as a additional cost • Good automation requires good skills – consider the ‘pond’ model
  • 20.
    TESTINGMIND CONSULTING 2ndAnnual Software Testing Symposium 2018 | Melbourne, Australia ECOSYSTEM – THE EVALUATION CHALLENGE Automation Vendor A: 100 Dollar per day Vendor B: 80 Dollars per day AVG$ Total$ @mircohering #notafactoryanymore
  • 21.
    TESTINGMIND CONSULTING 2ndAnnual Software Testing Symposium 2018 | Melbourne, Australia ECOSYSTEM – THE EVALUATION CHALLENGE -PART 2 First Time Resolution Rate/Resolution TimeSLAs @mircohering #notafactoryanymore
  • 22.
    TESTINGMIND CONSULTING 2ndAnnual Software Testing Symposium 2018 | Melbourne, Australia SILVER BULLET OR SNAKE OIL? @mircohering #notafactoryanymore
  • 23.
    WANT TO LEARNMORE? To receive the following: • A copy of this presentation • An excerpt of DevOps for the Modern Enterprise • Videos and slides from my DevOps Enterprise talks • Most popular blog posts from notafactoryanymore.com • Get early access to any material coming out with “DevOps For the Modern Enterprise” Just pick up your phone, and send an email: • To: mirco@SendYourSlides.com • Subject: devops