My keynote in Czech Test 2015 about Challenges and Solutions to Quality in the Digitalized World. It's about trends and musts that help cope with quality assurance of the increasing complexity of our digitalized world.

1. Quality Challenges of the Digitalized World
– How Does Quality Assurance Change?
Kari Kakkonen, Director, Quality and Competences, Knowit Oy at Czech Test, Prague June 26, 2015

2. Kari Kakkonen, Knowit
• Speaks, train, coach and mentor regularly about
• ISTQB Advanced, Foundation and Agile Testing + Knowit
Quality Professional
• Quality & Test process and organization development
• Agile testing, Scrum, Kanban, Lean
• Metrics
• Leadership
• Test automation, mobile, cloud, DevOps
• Quality, Cost, Benefits
• Speaking & writing highlights
• EuroSTAR and Iqnite several times
• ASTQB in USA, OOP in Germany, TEST-IT in South-Africa,
Nordic Testing Days in Estonia, Testing Days in Czech,
Israel Testing Week
• Numerous times in Finland at Testing Assembly, Aalto
Testing Days, Tieturi Testing, Talentum Testing Forum,
Quality Assurance & Software Testing, ICT Expo, TestIT
Summit, Microsoft, HP, IBM, Borland etc. events
• Testing Experience magazine, Quality and Testing
magazine, Sytyke-magazine, Tietoviikko
• Education
• ISTQB Expert Level Test Management Full & Advanced Full &
Agile Tester certified
• SPICE provisionary assessor certified
• M.Sc, Helsinki University of Technology / Aalto-university
• Marketing studies, University of Wisconsin-Madison
• Professional achievements
• Wide spread of business domain knowledge
• Embedded, Industry, Public,
• Training, Telecom, Commerce,
• Insurance, Banking, Pension
• ISTQB Treasurer, Executive Committee 2015-
• Finnish Software Testing Board FiSTB, chairman
• TestausOSY/FAST founding member
• Knowit, Director, Quality and Competences
• Chairman of research project STX, Lappeenranta University of
Technology
• Finnish Software Measurement Association FiSMA ry ex-board
member
• Ranked in 100 most influential IT-persons in Finland
© Knowit Oy
Twitter: @kkakkonen
LinkedIn:
fi.linkedin.com/in/karika
kkonen/

3. 100+
Mobile
apps
20+
Extranet
services
50+
Intranet
services
25+
Web stores
500+
Web sites
100+
Service design
projects
Knowit – We are
known for our
work
We develop and grow
our customers’
business.
© Copyright Knowit Oy 2015 | Confidential | Version 1.0
#1
In Quality
Assurance

4. Revolution of User Experience

5. Mobile is the new reality – research tells us
25.6.2015 © Copyright Knowit Oy 2015 | Confidential | Version 1.0 5
www.shopify.com/blog
“Mobile Now Accounts for 50.3% of All Ecommerce Traffic”

6. Change of Business
1. Services are digitalized
2. Services processes are
digitalized
3. Internet of Things, Internet
of Everything

7. 7
Central
Quality Challenges
Security
Usability
Performance
Speed of software development

8. 8
Security Testing
Everything in focus

9. Security Testing
25.6.2015 © Copyright Knowit Oy 2015 | Confidential | Version 1.0 9
Preventive
Security
• Expert in the team
Application level
security
• Every level must be
secure
Interfaces
• Internet
• Machine-2-Machine
Testing
• Vulnerability scans
• Audits
Up-to-date
• OWASP etc.

10. 10
Usability testing
Think about the UX

11. 25.6.2015 © Copyright Knowit Oy 2013 | Confidential | Version 1.0 11
Surface
Skeleton
Structure
Scope
Strategy
Applied from source:
The Elements of
User Experience
By Jesse James Garrett
• Visual design
• Micro-interactions
• Service structure
• Page structure
• Content structure
• Content structure
• Information architecture
• Functionality
• Content needs
• Business needs
• User needs
Bottom-Up
Abstract
Concrete

12. Usability testing can be…
Usability
assessments
Comparing
service to User
Experience
Levels
Assessing
usability
problems
criticality
Reporting
usability
problems
Suggesting
solutions
25.6.2015 © Copyright Knowit Oy 2015 | Confidential | Version 1.0 12

13. User device coverage – check usability with test
automation
• Different devices, same functionality,
same user experience
• Responsiveness
• One test library, many test objects
• Little to maintain
• Increasing number of objects
• Start early
• Repeat daily / by build
• Tests differ just a little
• Reporting
• Also other interfaces than UI
© Copyright Knowit Oy 2015
| Confidential | Version 1.0
25.6.2015 13
Apps to be tested (Web)
Test automation from cloud or devices

14. Performance testing
Test and Measure, don’t get surprised

16. Performance testing
Testing tools
• Commercial
• Open source
Continuous
monitoring
• Common goal with
performance testing
Identify
bottlenecks
• Fix
• Tune
Test coverage
• Devices
• Cloud
• Scenarios
Repeatability
• Include in regression
tests
25.6.2015 © Copyright Knowit Oy 2015 | Confidential | Version 1.0 16

17. Speed of software development
Balance between speed and quality

18. Fast software development process
DevOps
• As a culture
• Technically
Agility
• In organization
• In projects
Test automation
• Supporting the Team
• Speeding the Process
• Expand the Coverage
Exploratory Testing
• Defect finding
effectiveness
• Efficiency
Competences
• Versatile: testing,
development,
operations, business
25.6.2015 © Copyright Knowit Oy 2015 | Confidential | Version 1.0 18

19. DevOps principles
• DevOps is a method which emphasizes
communication, cooperation and integration
between software development teams (Dev) and IT-
support / operations teams (Ops)
• Method requires agile software development process
• It connects operations organization to software
development by using virtualized environments
• Production environments are part of software
development and as agile as the software
development
25.6.2015 © Copyright Knowit Oy 2015 | Confidential | Version 1.0 19
Software
Development
Quality
Assurance
Operations /
Production
DevOps

20. 25.6.2015 © Copyright Knowit Oy 2015 | Confidential | Version 1.0 20
Reporting
DevOps
Automated tests and installation
DeployBuild
Integration Testing Performance
Agile development
End-user
monitoring
Production
DeployDeploy
Virtualized back-end services
Test data management
Development Services
Version
management
Continuous
Integration
Business
Code
TestingDevelopment

21. Agile goes forward, what fits together?
25.6.2015 © Copyright Knowit Oy 2015 | Confidential | Version 1.0 21
Scrum
Kanban
Waterfall, V-Model
Scaled Agile
Framework
Lean
Lean Six Sigma
DevOps
Continuous
Deployment
Hybrid

22. Knowit Research 2014:
Agile test planning on company level
Strategic level
No plans
Plan in high level
Plan in detail
• Almost a quarter doesn’t plan agile
testing at all (is that agile anymore…)
• Mainly testing is planned in high level
(this is maybe the definition of agile
testing…)
• 85 % of company level testing is
documented (to some extent)

23. Company level
• Not just
• Scrum
• Kanban
• …
• Organization level
• UVF
• M3.0
• SAFe
• Nexus
• Spotify Way
• Capo
• Agility Path
• Lean
• …
25.6.2015 © Copyright Knowit Oy 2013 | Confidential | Version 1.0 23

24. SAFe 3.0
24

25. Cloud (MS, IBM…)
Example of test automation and continuous
integration
25.6.2015 © Copyright Knowit Oy 2015 | Confidential
Robot Framework
Quality Center
Jmeter
Python libraries
Selenium
Jenkins
GIT
Quick Test Pro
SoapUI
Sharepoint
Assure

26. Agile Testing Quadrants
© Copyright Knowit Oy 2015 | Confidential 26
Source: Lisa
Crispin and
Janet Gregory

27. Use Exploratory testing
Source: Sinkkonen, Elisabeth Hendrickson, Cem Kaner, James Bach, Lisa Crispin
“In exploratory testing you learn from previous tests, and put that
learning into your next tests”
Plan
•Test charter
Test
session
•Notes
•Bugs
Debriefing
•Dashboard
Testing
Opinion-forming
Reporting
Designing
actions
Observations

28. Exploratory testing in Prague – find a park
• Test charter
• ”look for green”
26.6.2015 © Copyright Knowit Oy 2013 | Confidential | Version 1.0 28

29. Exploratory testing in Prague – find a park
• Test execution log
26.6.2015 © Copyright Knowit Oy 2013 | Confidential | Version 1.0 29

30. Exploratory testing in Prague – find a park
• Defect report
26.6.2015 © Copyright Knowit Oy 2013 | Confidential | Version 1.0 30

31. Learn versatilely about business, development,
testing, operations
Courses, certificates Coaching
Pair work Workshops
Increase
competences
25.6.2015 © Copyright Knowit Oy 2015 | Confidential | Version 1.0 31

32. Prototype and test

33. Quality assurance in
Digitalized World
Build Quality, then also Test
• Security
• Usability
• Performance
Get faster feedback
• Team velocity
• Support of testing
• Speed to Operations

34. Questions?
kari.kakkonen@knowit.fi