nt Session 
 
Presented by: 
Regg   
Po re 
 
 
Brought to you by: 
 
 
340 Corporate Way, Suite   Orange Park, FL 3...
Regg Struyk
Polarion Software
 
With twenty years of commercial software development and testing experience, Regg
Struyk h...
2/11/2014
1
QA Evolves
20 plus years software dev experience
Who am I?
Senior Product Manager @ MKS (PTC)
Group Product Ma...
2/11/2014
2
•Introduction
•Company Profile
•Speed of Light
•Big Data
•Mobile
•Tools•Tools
•Automation
•Resources
“We produ...
2/11/2014
3
Agile
– Get ready to test
The Speed of Light
– More time constraints
– Originated from development
Hybrids/Spi...
2/11/2014
4
Time Constraints - Deliver Faster
Pressure to test more with Limited
Resources
Focus becomes task completion v...
2/11/2014
5
The real goal of testing should be to improve the software
– A continuous search for information
What‘s going ...
2/11/2014
6
The big data construct
What is Big Data?
2/11/2014
7
“Data sets whose size is beyond the ability of commonly used tools to
process it within acceptable time”.
Big ...
2/11/2014
8
What is Big Data?
The big data construct theory
Source: Gartner
To gain the right insights, big data is typica...
2/11/2014
9
Data Volume
"I have too much data to analyze"
“We live in a world that is getting more digitized and more conn...
2/11/2014
10
Data Velocity
Speed at which new data is being created
Big Data – Velocity
The need for real-time analytics
a...
2/11/2014
11
The ability to implement a data driven testing process
Big Data – Velocity
Challenges
Improving response rate...
2/11/2014
12
structured/unstructured data
text
Big Data – Variety
text
sensor data
audio
video
click streams
log files
Pol...
2/11/2014
13
Traditional methods not enough
Big Data – Challenges
Traditional tools are inadequate
Lack of knowledge as to...
2/11/2014
14
Mobile Testing
Mobility application testers, face many critical
challenges compared to conventional software ...
2/11/2014
15
BYOD
Complexity added due to combination of multiple
Mobile– Challenges
Complexity added due to combination o...
2/11/2014
16
Emulation Software (Zap, Keynote)
Mobile– Solutions
Cloud Computing (Perfecto, Soasta)
Crowd sourcing
Perform...
2/11/2014
17
Tools
Unavailability of right tools
– Budget, availability and funding
– Type of Tools:
• DEFFECT TRACKING
• ...
2/11/2014
18
Tools - Solution
Network with other testers to find information about
l k llesser-known test tools.
Online QA...
2/11/2014
19
Test Automation
1. Test Automation Will Eliminate Human Testers1. Test Automation Will Eliminate Human Tester...
2/11/2014
20
Test Automation - Solutions
The right automation
– Right skill sets (programing scripts)
– Ensure we create r...
2/11/2014
21
Lessons
To handle the volume and ensure live data
i t ti Bi D t t ti d tintegration, Big Data testing needs t...
2/11/2014
22
Lessons
With nearly a third of all page views now coming
f bil d t bl t t t d t bfrom mobile and tablets, tes...
2/11/2014
23
Lessons
What did you learn????
Resources
“Game Testing Evolves” whitepaper @ Polarion
“Big Data Testing” whit...
Upcoming SlideShare
Loading in …5
×

Why Classic Software Testing Doesn’t Work Anymore

391 views
289 views

Published on

The classic software testing team is becoming increasingly obsolete. Traditional processes and tools just don’t meet today’s testing challenges. With the introduction of methodologies such as agile, testing processes with a "test last" philosophy cannot succeed in a rapid deployment environment. To exacerbate our testing difficulties, we now have to deal with "big data" which introduces an entirely new set of problems. In the past, we have relied on tools such as test automation to solve these problems; however, classic test automation simply will not suffice on its own and must be integrated with the right testing activities while being supported by correct procedures. When you combine these problems with inadequately defined requirements and limited resources, you have a recipe for testing disaster. Regg Struyk shares real-world examples and offers constructive ways to move away from traditional testing methods to a more integrated process using concepts such as test-driven development and TestOps.

Published in: Technology, Education
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
391
On SlideShare
0
From Embeds
0
Number of Embeds
69
Actions
Shares
0
Downloads
4
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Why Classic Software Testing Doesn’t Work Anymore

  1. 1.       nt Session    Presented by:  Regg    Po re      Brought to you by:      340 Corporate Way, Suite   Orange Park, FL 32073  888‐2 W5  Concurre 4/9/2014    12:45 PM          “Why Classic Software Testing   Doesn’t Work Anymore”      Struyk larion Softwa             300, 68‐8770 ∙ 904‐278‐0524 ∙ sqeinfo@sqe.com ∙ www.sqe.com 
  2. 2. Regg Struyk Polarion Software   With twenty years of commercial software development and testing experience, Regg Struyk has held many different positions ranging from the head of technical product management for Agfa HealthCare to, most recently, product evangelist for Polarion QA. Regg has developed for several software testing tools including test integrity, iTest, and Polarion QA. Dedicated to the domain of test management, Regg is continually analyzing testing trends and their potential impact on the discipline of software testing.
  3. 3. 2/11/2014 1 QA Evolves 20 plus years software dev experience Who am I? Senior Product Manager @ MKS (PTC) Group Product Manager @ Agfa Healthcare Product Manager @ Polarion for QA Software Polarion Software® Polarion® QA™ - www.polarion.com g @ Q 2
  4. 4. 2/11/2014 2 •Introduction •Company Profile •Speed of Light •Big Data •Mobile •Tools•Tools •Automation •Resources “We produce all significant documents for specifications, QA and test plans easily and rapidly with Polarion QA” Andreas Deuter , Phoenix Contact Electronics Waterfall – Test Specifications Traditional QA – Test Cases – Test Execution Polarion Software® Polarion® QA™ - www.polarion.com 4
  5. 5. 2/11/2014 3 Agile – Get ready to test The Speed of Light – More time constraints – Originated from development Hybrids/Spin-Offs – Scrum, Agile-Fall – Does not means getting rid of documentation Polarion Software® Polarion® QA™ - www.polarion.com Market Pressures – Regulatory – Quick check-in 5 The Speed of Light “Testing is not a phase on Agile teams, testing is a way of life. Agile teams should test continuously. It’s the only way to ensure that the feautures implemented during a given iteration or sprint are actually done” Polarion Software® Polarion® QA™ - www.polarion.com 6
  6. 6. 2/11/2014 4 Time Constraints - Deliver Faster Pressure to test more with Limited Resources Focus becomes task completion vs coverage Quality may suffer (may miss critical bugs) Examples: Madden Football 2008 – prematurely released Xbox – hardware crashes = LAWSUIT Lack of Planning – Still need to plan not a free for all What‘s going on QA – Pitfalls – More time constraints Lack of Documentation – Key component of Agile is removing clunky documentation – Does not means getting rid Daily Meeting Hassle – Waste of time????? Polarion Software® Polarion® QA™ - www.polarion.com – Quick check-in “I’m surprised at the name Agile —it should be called couch potato. There are too many meetings. It’s ironic that there are all these books explaining how simple it is.” 8
  7. 7. 2/11/2014 5 The real goal of testing should be to improve the software – A continuous search for information What‘s going on QA – Instead of pass or fail think in terms of whether or not tests provide valuable information Understanding project and goals – Define coverage requirements – Understand/establish goals of a sprint O f % Polarion Software® Polarion® QA™ - www.polarion.com “Out of 200 participants, 64 % said that switching to Agile Development was harder than it initially seemed.” 9 •Risk-based testing accommodates change (focus on main areas) – Reduce the scope of testing based on risk What‘s going on QA -Solutions •Careful automated testing can accomodate change* •Good unit testing is essential (limited bug-finding 30%) •System Testing will cover more on average Polarion Software® Polarion® QA™ - www.polarion.com Combine risk-based testing with maintainable automated regression testing at the system test level 10
  8. 8. 2/11/2014 6 The big data construct What is Big Data?
  9. 9. 2/11/2014 7 “Data sets whose size is beyond the ability of commonly used tools to process it within acceptable time”. Big Data – What is it? Transformation in how society processes information. Explosion of big data is just as transformative as the internet was. According to IBM, “Every day, we create 2.5 quintillion bytes of data — so much that 90% of the data in the world today has been created in Polarion Software® Polarion® QA™ - www.polarion.com so much that 90% of the data in the world today has been created in the last two years alone”. Gartner believes Big Data will drive a total IT spend of $34billion in 2013 rising to $232billion by 2016! 13 Big Data – Sources Polarion Software® Polarion® QA™ - www.polarion.com 14
  10. 10. 2/11/2014 8 What is Big Data? The big data construct theory Source: Gartner To gain the right insights, big data is typically broken down by three characteristics: Big Data – 3Vs? •Volume: How much data •Velocity: How fast data is processed •Variety: The various types of data Big data is high-volume, high-velocity and high-variety information assets that demand cost-effective, innovative forms of information processing for enhanced insight and decision making. Polarion Software® Polarion® QA™ - www.polarion.com processing for enhanced insight and decision making. 16
  11. 11. 2/11/2014 9 Data Volume "I have too much data to analyze" “We live in a world that is getting more digitized and more connected Big Data Volume - Challenges every day.” Data is now created by employees, customers, 3rd party Extracting information from large data sets Massive, complex and semi-structured Regulatory demands increase pressure on testing Consolidation of data from systems, apps, internet Mi i d t & d t i t it Polarion Software® Polarion® QA™ - www.polarion.com Missing data & data integrity 18
  12. 12. 2/11/2014 10 Data Velocity Speed at which new data is being created Big Data – Velocity The need for real-time analytics acquisition of data vs. actionable insight Polarion Software® Polarion® QA™ - www.polarion.com “Every business is now a digital business” 20
  13. 13. 2/11/2014 11 The ability to implement a data driven testing process Big Data – Velocity Challenges Improving response rates for decision making Expectations – when fast isn’t fast enough Polarion Software® Polarion® QA™ - www.polarion.com 21 Data Variety
  14. 14. 2/11/2014 12 structured/unstructured data text Big Data – Variety text sensor data audio video click streams log files Polarion Software® Polarion® QA™ - www.polarion.com g 23 80% data growth in images & video Number of data types Unstructured/semi Big Data – Variety Challenges Number of data types Unstructured/semi Volatility of changing data Sources of data Mobile devices Internet Email Polarion Software® Polarion® QA™ - www.polarion.com 24
  15. 15. 2/11/2014 13 Traditional methods not enough Big Data – Challenges Traditional tools are inadequate Lack of knowledge as to what to test and how much data to test Polarion Software® Polarion® QA™ - www.polarion.com Test strategies for structured vs unstructured data 25 Test early – Test fast Data driven testing Big Data – Solutions Data driven testing Develop a strong testing environment Multiple testing methodologies functional/non-functional Data Sampling Identify suitable sampling technique Polarion Software® Polarion® QA™ - www.polarion.com y p g q Include correct uses cases Right test data sets 26
  16. 16. 2/11/2014 14 Mobile Testing Mobility application testers, face many critical challenges compared to conventional software testingchallenges compared to conventional software testing Number of mobile phones will exceed world population by 2014 (7.3 billion) Mobile – Landscape y ( ) Polarion Software® Polarion® QA™ - www.polarion.com 28
  17. 17. 2/11/2014 15 BYOD Complexity added due to combination of multiple Mobile– Challenges Complexity added due to combination of multiple platforms each having numerous devices and various OS versions Various versions of each platform and device Mobile user: range from tech savvy to newbie Testing on various networks and network vendors Polarion Software® Polarion® QA™ - www.polarion.com CDMA and GSM with their 2G, 3G and 4G versions 29 Rapid Roll-out Schedule Expectations for apps Mobile– Challenges Expectations for apps Ultra competitive to release new apps Complexity added due to combination of multiple platforms each having numerous devices and various OS versions Polarion Software® Polarion® QA™ - www.polarion.com 30
  18. 18. 2/11/2014 16 Emulation Software (Zap, Keynote) Mobile– Solutions Cloud Computing (Perfecto, Soasta) Crowd sourcing Performance Testing (Loadster, Neotys) Polarion Software® Polarion® QA™ - www.polarion.com 31 Tools
  19. 19. 2/11/2014 17 Tools Unavailability of right tools – Budget, availability and funding – Type of Tools: • DEFFECT TRACKING • TEST CASE CREATION & Execution • Test Automation Integrations: lack there of– Integrations: lack there of – MS Excel & MS Word equation • Time consuming to manage and maintain • Metrics • Integrations Tools - Solution Test Management Software – Combines Test Cases, Test Executions, Defects, Test Automation – Replace or leverage MS Excel/Word – Customizable workflow – Automation– Automation – Integrations
  20. 20. 2/11/2014 18 Tools - Solution Network with other testers to find information about l k llesser-known test tools. Online QA/Testing forums Interoperable tools Automation
  21. 21. 2/11/2014 19 Test Automation 1. Test Automation Will Eliminate Human Testers1. Test Automation Will Eliminate Human Testers FALSEFALSE 2. What tests should be automated first?2. What tests should be automated first? Automate your testing procedure when you have lot of regression work. Test Automation - Implementation – Commitment can take time to set p & maintain• can take time to set-up & maintain • If not implemented correctly = $$$ Desire to automate tests across apps & platforms Most automated test tools do not tell you what to test Initial cost can be high • purchase and set-up 100% Automation does not exist!
  22. 22. 2/11/2014 20 Test Automation - Solutions The right automation – Right skill sets (programing scripts) – Ensure we create reusable tests • can take time to set-up & maintain – Unit test will provide highest ROI There is a large gap between the people who own automated test tools and the people who actually realize the benefits from test automation. 5 Lessons Learned
  23. 23. 2/11/2014 21 Lessons To handle the volume and ensure live data i t ti Bi D t t ti d tintegration, Big Data testing needs to empower developers, testers and enterprises with freedom to experiment and innovate. Lessons Successful software test automation is possible if f d t l i dd d d dfundamental issues are addressed and managed. Automated software testing is truly a different way of testing and requires adjustments to current test methods and organizational structures. However, thepayback from test automation can far outweigh the costs.
  24. 24. 2/11/2014 22 Lessons With nearly a third of all page views now coming f bil d t bl t t t d t bfrom mobile and tablets, testers need to embrace mobile. Mobile is no longer an area that a few UX people specialize in, and we need to start designing and testing everything for smartphones and tablets as well as computers. Lessons Development cycles will continue to accelerate b d i f t ( k tbased on various factors (market, process, technology), and quality expectations will need to meet the challenge through increased visibility, new ways of thinking combined with advances in software.
  25. 25. 2/11/2014 23 Lessons What did you learn???? Resources “Game Testing Evolves” whitepaper @ Polarion “Big Data Testing” whitepaper @ PolarionBig Data Testing whitepaper @ Polarion http://www.polarion.com/products/qa/index.php regg.struyk@polarion.com T itt R St k1Twitter: ReggStruyk1 LinkedIn: Software Test Case Management

×