More Related Content Similar to Key Findings from the World Quality Report 2012-13 at HP Discover (20) Key Findings from the World Quality Report 2012-13 at HP Discover1. World Quality Report 2012-13
Testing for the Future
Alain Mey
Capgemini Global & EMEA Alliance Manager,
HP Software
Stefan Gerstner
VP Global Testing Services Capgemini Group
2012/13
December 4, 2012
2. Introductions
Alain Mey
Capgemini Global & EMEA Alliance Manager, HP Software
Stefan Gerstner
VP Global Testing Services, Capgemini Group
World Quality Report 2012-13 | September 2012
Copyright © 2012 Capgemini and Sogeti. All Rights Reserved 2
3. Agenda
Introducing the World Quality Report
Key findings by topic
Quality Budgets
Testing Centers of Excellence
Mobile Testing
Cloud
Quality Assurance Resources
How can you leverage the WQR and its findings?
World Quality Report 2012-13 | September 2012
Copyright © 2012 Capgemini and Sogeti. All Rights Reserved 3
4. What is the World Quality Report?
Only global report for application quality
covering:
• Outsourcing trends, tools, technology, organization
• Industries: CPRD; E&U; FS; High-Tech; Public; TME
• Geographic analysis: ANZ; Brazil; China; France;
Germany; Netherlands; North America; Nordics; UK
Co-sponsored report by Capgemini, Sogeti & HP
• Capgemini Group (world leader in testing services)
• HP (World leader in application testing technologies
Research sample and methodology
• 1,553 telephone interviews with CIOs, CFOs,
IT directors and senior managers involved in
testing and QA based on a structured questionnaire.
Fourth year the report has been published
World Quality Report 2012-13 | September 2012
Copyright © 2012 Capgemini and Sogeti. All Rights Reserved 4
5. World Quality Report: Interviews by Region
WQR research
based on 1,553
telephone
interviews
covering 25
countries
Data from
questionnaire
and customer
quotes
World Quality Report 2012-13 | September 2012
Copyright © 2012 Capgemini and Sogeti. All Rights Reserved 5
6. World Quality Report: Interviews by Sector and
Job Title
Interviews by Sector Interviews by Job Title
World Quality Report 2012-13 | September 2012
Copyright © 2012 Capgemini and Sogeti. All Rights Reserved 6
7. 2012-13 World Quality Report Details
6 x 2 page sector analysis 9 x 2 page country analysis,
top trends
World Quality Report 2012-13 | September 2012
Copyright © 2012 Capgemini and Sogeti. All Rights Reserved 7
9. QA budgets: Stepping up
Majority of organizations expect testing budgets to rise between now
and 2015.
World Quality Report 2012-13 | September 2012
Copyright © 2012 Capgemini and Sogeti. All Rights Reserved 9
10. QA budgets: Stepping up
MY INVESTMENT GOING FORWARD WILL [FOCUS ON] BRINGING MY INTERNAL
TEAMS UP TO STANDARD, INCREASING THE LEVEL OF AUTOMATION USED, AND
DEVELOPING GREATER EFFICIENCIES THROUGHOUT. ALTHOUGH OUR TESTING
BUDGET IS NOT SEPARATE FROM THE WIDER IT BUDGET, THE LEVEL OF
MONITORING AND PERFORMANCE MEASURING HAS INCREASED – WE NEED TO
JUSTIFY OUR COSTS.
– AN AUTOMOTIVE BUSINESS, BRAZIL
THE PROBLEM IS INCREASING PRESSURE TO GET THINGS DONE YESTERDAY. WE
HAVE THIS CONFLICT BETWEEN TESTING ADEQUATELY TO OUR SATISFACTION AND
THE TESTING TIME EXPECTED BY THOSE ON THE [BUSINESS ] SIDE. INVESTMENT
HAS INCREASED BUT THEN EXPECTATION IN SPEED OF DELIVERY HAS ALSO
INCREASED. IT ’S NOT EASY, AS THE ORGANIZATION NEEDS TO BE ABLE TO JUMP
INTO A MUCH FASTER TESTING MODE WHILE MAINTAINING QUALITY STANDARDS.
– A MANUFACTURING BUSINESS, CZECH REPUBLIC
World Quality Report 2012-13 | September 2012
Copyright © 2012 Capgemini and Sogeti. All Rights Reserved 10
11. QA budgets: Stepping up
Figure 3
Areas of Greatest Increase in Spend
World Quality Report 2012-13 | September 2012
Copyright © 2012 Capgemini and Sogeti. All Rights Reserved 11
12. Testing Centers of Excellence: A growing priority
Organisations are turning to COEs to consolidate their QA operations in
response to market pressures, technology demands, and emerging models of
testing delivery.
World Quality Report 2012-13 | September 2012
Copyright © 2012 Capgemini and Sogeti. All Rights Reserved 12
13. Testing Centers of Excellence: A growing priority
[A TCOE ] IS SOMETHING I’M BUILDING HERE; IT REPRESENTS A SIGNIFICANT
CHANGE IN OUR APPROACH BUT ONE THAT I PERSONALLY BELIEVE TO BE VITAL
TO OUR BUSINESS SINCE I’VE COME FROM AN ORGANIZATION WITH AN OPERATING
TESTING CENTER. WE KNOW THAT IT CAN REDUCE COSTS AND OVERALL TIME OF
TESTING PRACTICES, AND GIVEN THE COMPETITIVE NATURE OF THE MARKET WE
OPERATE IN, IT’S CRITICAL TO GET SPEED AND COSTS DOWN WHILST GETTING
QUALITY UP. WE BELIEVE A TCOE CAN DO THIS.
– A MANUFACTURING BUSINESS, UK
FOR US, THE MOVE TOWARDS A TCOE IS PART OF THE WIDER UNION BETWEEN THE
BUSINESS AND IT SIDE OF THE ORGANIZATION. WE NOW HAVE A MUCH STRONGER
LINK BETWEEN THE TWO. DECENTRALIZED TESTING TEAMS JUST COULDN’T
PROVIDE THAT EFFICIENCY AND IT MEANT THAT WALLS EXISTED BETWEEN THE
TEAMS, WHEREAS SHARED EXPERIENCE AND BEST PRACTICE ARE FAR EASIER
WITHIN A TCOE FRAMEWORK.
– A FINANCIAL SERVICES BUSINESS, FINLAND
World Quality Report 2012-13 | September 2012
Copyright © 2012 Capgemini and Sogeti. All Rights Reserved 13
14. Testing Centers of Excellence: A growing priority
World Quality Report 2012-13 | September 2012
Copyright © 2012 Capgemini and Sogeti. All Rights Reserved 14
15. Mobile Testing: Behind the curve
Only 31% of firms
surveyed currently test
mobile applications.
Why? Many report
they do not have
appropriate tools or
expertise and limited
access to the devices
to test effectively
mobile applications.
World Quality Report 2012-13 | September 2012
Copyright © 2012 Capgemini and Sogeti. All Rights Reserved 15
16. Mobile Testing: Behind the curve
World Quality Report 2012-13 | September 2012
Copyright © 2012 Capgemini and Sogeti. All Rights Reserved 16
17. Mobile trends
Transactions in both B2C and B2B
• Products and services
• Account management
Internal processes (B2E)
• Business intelligence
• Process management
• Social collaboration
World Quality Report 2012-13 | September 2012
Copyright © 2012 Capgemini and Sogeti. All Rights Reserved 17
18. Mobile Testing is business as usual...
World Quality Report 2012-13 | September 2012
Copyright © 2012 Capgemini and Sogeti. All Rights Reserved 18
19. ... and Mobile Testing adds additional hot spots
World Quality Report 2012-13 | September 2012
Copyright © 2012 Capgemini and Sogeti. All Rights Reserved 19
20. Device fragmentation creates a need to perform
device level testing & UAT for mobile apps
Testing cycle for mobile application
Requirements
Analysis
Testing cycle for desktop application
Requirements
Analysis Test report Test Plan
analysis
Test report analysis Test Plan
Device
provisioning & Test Case Creation
device UAT
Execute regression Test case creation /
testing / automation
automation scripting Device Level Execute new
validation / manual features testing /
testing manual testing
Execute new
features testing /
manual testing Execute regression
testing /
automation
Unlike the desktop, the mobile space includes multiple platforms and form factors. These
complexities lead many organizations to test on real devices, and most do this manually
World Quality Report 2012-13 | September 2012
Copyright © 2012 Capgemini and Sogeti. All Rights Reserved 20
21. Why Mobile Testing is difficult? (1/2)
Touching the device Touch, swipe, pinch, rotate, move
Diversity in Android is the most fragmented, with 5 major versions corresponding to 9 API sets. In typical
platforms, OSs and markets for feature phones, there can be more than 100 different device types in the installed
devices base
Sophisticated user experiences involve touch, gestures, GPS location, audio, sensors
Automation
(accelerometers) and physical actions (touching the handset to NFC readers). Such interactions
challenges
can't be fully scripted or simulated, and may involve manual testing on real devices
Cellular networks are nondeterministic, and exhibit wide variations in bandwidth and latency.
Network performance Users in different locations accessing different operators can experience vastly different
application performance
Application
Mobile devices and applications are becoming more sophisticated, using techniques such as
complexity and
context, 3D graphics and gaming. Greater sophistication implies more complex testing
sophistication
New OS versions
Developers have no control over when new OS versions will appear, and when or whether users
often break
will upgrade. Thus, it's common for new OS releases to break existing native applications
applications
World Quality Report 2012-13 | September 2012
Copyright © 2012 Capgemini and Sogeti. All Rights Reserved 21
22. Why Mobile Testing is difficult? (2/2)
New technology Mobile is at the leading edge of new technologies, such as HTML5, that are not yet
risks well-understood in terms of testing
There is a performance difference of greater than one order of magnitude across
Performance
devices in the smartphone installed base. An app or HTML5 website that runs well
variations
on a top-end device may not be acceptable on a low-performance handset
Operators may modify mobile Web content to optimize network performance so
Operator
that desk-based testing may not show what a real-world handset user
intervention
experiences
Mobile applications and websites are used in a wide range of contexts, which
Contextual issues raises many new testing challenges. Applications dealing with critical or regulated
data may demand much more rigorous testing
Smartphones are acquiring a growing range of add-on devices, such as Bluetooth
Peripherals
peripherals, that are generally not accessible to testing tools
Many applications will be distributed via public app stores, where user reviews and
Testing user attitudes play a large part in determining whether an app is downloaded.
opinions Understanding user reactions to an app and its store collateral before it's
published may be valuable
World Quality Report 2012-13 | September 2012
Copyright © 2012 Capgemini and Sogeti. All Rights Reserved 22
23. Mobile Testing: Behind the curve
Quote on page 28
Retail Business UK
World Quality Report 2012-13 | September 2012
Copyright © 2012 Capgemini and Sogeti. All Rights Reserved 23
24. Mobile Testing: Behind the curve
World Quality Report 2012-13 | September 2012
Copyright © 2012 Capgemini and Sogeti. All Rights Reserved 24
25. Cloud: gathering pace
28% of testing currently occurs in a cloud environment and is set to rise to
39% by 2015.
Only 4% reported no plans for Cloud testing, down from 31%, 2 years ago.
World Quality Report 2012-13 | September 2012
Copyright © 2012 Capgemini and Sogeti. All Rights Reserved 25
26. Cloud: Gathering pace
THE CLOUD IS SOMETHING THAT HAS BECOME A SIGNIFICANT PRIORITY FOR OUR
ORGANIZATIONAL DIRECTION IN THE INFORMATION TECHNOLOGY DEPARTMENT.
WITH REGARDS TO TESTING, IT’S OPENING UP A WAY OF TURNING ON AND OFF A
STREAM OF NEW SKILLS AND ABILITIES IN A WAY THAT IS BOTH FAST AND COST-
EFFECTIVE WHEN WE NEED THEM. WE CAN REACH OUT AND GET EXACTLY WHAT
WE NEED WITHOUT DELAY.
– A FINANCIAL SERVICES BUSINESS, THE NETHERLANDS
WE EXPERIENCE PEAKS AND TROUGHS IN OUR TESTING PROGRAM, SO TAAS
PROVIDES A CERTAIN DEGREE OF FLEXIBILITY. WHILE SOME APPLICATIONS MAY BE
RUN OF THE MILL, SOME MAY HAVE MORE AD HOC REQUIREMENTS THAT PUSH US
TO MEET UNIQUE OR TEMPORARY NEEDS. TAAS ALLOWS US TO DO THIS WITHOUT
THE BURDEN OF GREAT COST OR ADDING TO PROJECT TIME.
– A RETAIL BUSINESS, DENMARK
World Quality Report 2012-13 | September 2012
Copyright © 2012 Capgemini and Sogeti. All Rights Reserved 26
27. Cloud: Gathering pace
World Quality Report 2012-13 | September 2012
Copyright © 2012 Capgemini and Sogeti. All Rights Reserved 27
28. Cloud: Gathering pace
World Quality Report 2012-13 | September 2012
Copyright © 2012 Capgemini and Sogeti. All Rights Reserved 28
29. Quality Assurance Resources: A question of
value
World Quality Report 2012-13 | September 2012
Copyright © 2012 Capgemini and Sogeti. All Rights Reserved 29
30. Quality Assurance Resources: A question of
value
OUTSOURCED TESTING WILL ALWAYS BE AN IMPORTANT OPTION, BUT HOW WE
SOURCE THOSE TESTERS DOES TEND TO DEPEND ON PROJECT OR COST, RATHER
THAN ANY SPECIFIC PREDEFINED RULES. WE USED TO USE INDIA-BASED
SUPPLIERS A LOT MORE THAN WE DO TODAY, BUT THE COST BENEFITS WERE NOT
BALANCED BY OUR INTERNAL CLIENT SATISFACTION FEEDBACK.
– A FINANCIAL SERVICES BUSINESS, AUSTRALIA
I’D LOVE TO BE ABLE TO PROFESSIONALIZE OUR TESTING FUNCTION – GIVE
PEOPLE A CAREER AND THE PAY TO MATCH AND THEY’LL STICK WITH YOU. BUT
THE REALITY IS QUITE THE OPPOSITE. TESTING IS SEE N AS A ‘NECESSARY EVIL’ IN
MANY ORGANIZATIONS. THIS MEANS ORGANIZATIONS NEVER GIVE [TESTERS] THE
PROPER ATTENTION [THEY] DESERVE AND WE END UP BUYING EXTERNAL EXPERTS
WHO MAY HAVE THE SKILLS, BUT NOT NECESSARILY THE INTIMATE KNOWLEDGE
OF OUR BUSINESS.
– A RETAIL BUSINESS, DENMARK
World Quality Report 2012-13 | September 2012
Copyright © 2012 Capgemini and Sogeti. All Rights Reserved 30
31. Quality Assurance Resources: A question of
value
World Quality Report 2012-13 | September 2012
Copyright © 2012 Capgemini and Sogeti. All Rights Reserved 31
32. How can you use the World Quality Report?
As supporting evidence for your budget
negotiations
Pro-actively address challenges around Mobile
Apps Testing
Discuss best suited delivery model
• TaaS
• TCoE
• Internal vs. external
Download the World Quality Report
• www.capgemini.com/testing
• www.sogeti.com/testing
World Quality Report 2012-13 | September 2012
Copyright © 2012 Capgemini and Sogeti. All Rights Reserved 32
33. Summary - key findings
• Quality budgets: stepping up
• Testing Centers of Excellence:
a growing priority
• Mobile Testing: behind the curve
• Cloud: gathering pace
• Quality Assurance Resources:
a question of value
Open House Tour TLI Testing | September & October 2012
Copyright © Capgemini 2012. All Rights Reserved 33
34. About Capgemini and Sogeti About HP
With around 120,000 people in 40 countries, Capgemini is one of the world's foremost providers of consulting, At HP, we don’t just believe in the power of technology. We believe in the
technology and outsourcing services. The Group reported 2011 global revenues of EUR 9.7 billion. power of people when technology works for them. To help you create. To
Together with its clients, Capgemini creates and delivers business and technology solutions that fit their make the digital tangible. To harness the power of human information. At
needs and drive the results they want. A deeply multicultural organization, Capgemini has developed its own HP, we work to make what you do matter even more.
way of working, the Collaborative Business Experience™, and draws on Rightshore®, its worldwide delivery Our Business Technology Optimization (BTO) products, along with our
model. new and complete approach to Application Lifecycle Management (ALM),
Sogeti, its wholly-owned subsidiary, is a leading provider of local professional services, bringing together help our customers to achieve better business outcomes.
more than 20,000 professionals in 15 countries and is present in over 100 locations in Europe, the US and
India.
Together, Capgemini and Sogeti have developed innovative, business-driven quality assurance (QA) and
testing services, combining best-in-breed testing methodologies (TMap® and TPI®) and the global delivery
model, Rightshore®, to help organizations achieve their testing and QA goals. Capgemini and Sogeti have
created one of the largest dedicated testing practices in the world, with over 9,500 test professionals and a TMap®, TMap NEXT®, TPI® and TPI NEXT® are registered trademarks of Sogeti, part
further 14,500 application specialists, notably through a common center of excellence with testing specialists of the Capgemini Group.
developed in India.
www.capgemini.com/testing
www.capgemini.com
www.sogeti.com/testing
www.hp.com
The information contained in this presentation is proprietary. It is for internal and intermediary use only.
Copyright ©2012 Capgemini and Sogeti. All rights reserved.
Rightshore® is a trademark belonging to Capgemini.
No part of this document may be modified, deleted or expanded by any process or means without prior written permission from Capgemini.