fourth EdItIon      2012-13                 world                 QualiTy                 reporT
CONTENTS4	Introduction                         6	    Executive Summary                 74	   About the Study  10       	  ...
 IntroductionIntroduction4	        WORLD QUALITY REPORT 2012-13
Introduction                                                Michel de Meijer                                     Matthew ...
Executive SummaryExecutive SummaryThis is the fourth year that Capgemini, Sogeti, and HP have produced theWorld Quality R...
Executive SummaryOur global survey was carried out againsta challenging economic landscape fororganizations of all sizes ...
Executive SummaryTesting as a Service (TaaS) – which can         are changing to support the core areas of        exist, ...
Executive Summary                                                                                                  Re-eva...
TheState ofQuality2012      12	   Quality Budgets: Stepping up      18	   Testing Centers of Excellence:            A grow...
The State of Quality 2012	   WORLD QUALITY REPORT 2012-13	        11
The State of Quality 2012Quality BudgetsQuality Budgets:Stepping up“Investment is going to be               Organizations ...
The State of Quality 2012                                                                                               Qu...
The State of Quality 2012Quality BudgetsIncreasing pressure to                           21% collect data manually and use...
The State of Quality 2012                                                           Quality Budgets                   “The...
The State of Quality 2012Quality Budgets                     “My investment going forward will [focus on] bringing my inte...
The State of Quality 2012                                                                                                 ...
The State of Quality 2012Testing Centers of ExcellenceTesting Centersof Excellence: Agrowing priorityConsolidation and sta...
The State of Quality 2012                                           Testing Centers of Excellence                   “[A TC...
The State of Quality 2012Testing Centers of Excellence            PLANS FOR A TESTING CENTER OF EXCELLENCE                ...
The State of Quality 2012                                                                                     Testing Cent...
The State of Quality 2012Testing Centers of ExcellenceBased on the survey responses, together         used by over a third...
The State of Quality 2012                                                                                      Testing Cen...
The State of Quality 2012Mobile TestingMobile Testing:Behind the curveMobile has certainly changed the game          as th...
The State of Quality 2012                                                                                                 ...
The State of Quality 2012Mobile Testing           Regional variations                              Regions with traditiona...
The State of Quality 2012                                                                                               Mo...
The State of Quality 2012Mobile Testing             Sector perspectives             A look at mobile testing by industry t...
The State of Quality 2012                                                                                                 ...
World Quality Report 2012-13
World Quality Report 2012-13
World Quality Report 2012-13
World Quality Report 2012-13
World Quality Report 2012-13
World Quality Report 2012-13
World Quality Report 2012-13
World Quality Report 2012-13
World Quality Report 2012-13
World Quality Report 2012-13
World Quality Report 2012-13
World Quality Report 2012-13
World Quality Report 2012-13
World Quality Report 2012-13
World Quality Report 2012-13
World Quality Report 2012-13
World Quality Report 2012-13
World Quality Report 2012-13
World Quality Report 2012-13
World Quality Report 2012-13
World Quality Report 2012-13
World Quality Report 2012-13
World Quality Report 2012-13
World Quality Report 2012-13
World Quality Report 2012-13
World Quality Report 2012-13
World Quality Report 2012-13
World Quality Report 2012-13
World Quality Report 2012-13
World Quality Report 2012-13
World Quality Report 2012-13
World Quality Report 2012-13
World Quality Report 2012-13
World Quality Report 2012-13
World Quality Report 2012-13
World Quality Report 2012-13
World Quality Report 2012-13
World Quality Report 2012-13
World Quality Report 2012-13
World Quality Report 2012-13
World Quality Report 2012-13
World Quality Report 2012-13
World Quality Report 2012-13
World Quality Report 2012-13
World Quality Report 2012-13
World Quality Report 2012-13
World Quality Report 2012-13
World Quality Report 2012-13
World Quality Report 2012-13
World Quality Report 2012-13
World Quality Report 2012-13
World Quality Report 2012-13
Upcoming SlideShare
Loading in...5
×

World Quality Report 2012-13

3,413

Published on

Published in: Technology, Business
0 Comments
4 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
3,413
On Slideshare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
Downloads
114
Comments
0
Likes
4
Embeds 0
No embeds

No notes for slide

Transcript of "World Quality Report 2012-13"

  1. 1. fourth EdItIon 2012-13 world QualiTy reporT
  2. 2. CONTENTS4 Introduction 6 Executive Summary 74 About the Study 10 40 54 The State of sector Country andQuality 2012 analysis Regional Analysis12 Quality Budgets: Stepping up 42 Consumer Products, 56 Australia and New Zealand Retail, and Distribution:18 Testing Centers of Excellence: Mobile and internet commerce 58 Brazil A growing priority drives sector transformation 60 China24 Mobile Testing: 44 Energy & Utilities: Commitment Behind the curve to improve quality of testing 62 France through increased investment30 Cloud: Gathering pace 64 Germany 46 Financial Services:36 Quality Assurance Resources: Evolving QA – Riding the 66 The Netherlands A question of value transformation wave 68 The Nordic Region 48 High Tech: Increasing focus on data-driven customer 70 North America service delivery 72 United Kingdom 50 Public Sector: Renewed focus on efficiency gains linked to budget reductions 52 Telecoms, Media, and Entertainment: Maximizing assets, unleashing growth, and transforming to succeed WORLD QUALITY REPORT 2012-13 3
  3. 3.  IntroductionIntroduction4 WORLD QUALITY REPORT 2012-13
  4. 4. Introduction Michel de Meijer Matthew Morgan Senior Vice President, Leader Global Service Line Vice President, Hybrid IT and Cloud Product Testing, Capgemini & Sogeti Marketing, Software, HPWelcome to the fourth edition of the World Quality Report, which, as in previousyears, has been designed to provide the most comprehensive assessment of thecurrent state of enterprise application quality and testing practices availablefrom around the world.To enhance the quality and relevance of the number of new emerging developments tools, services, and products developed todata gathered, we increased the number of in the market and an increased pace improve the efficiency and effectiveness ofinterviews to more than 1,550 CFOs, CIOs, of change. For example, since last year, enterprise applications.IT directors, and quality assurance (QA) many readers will have seen corporatedirectors around the globe, all of which were announcements about a growing number We hope you find that the research,carried out by phone. We also broadened of large-scale enterprise-wide managed analysis, and commentary containedour geographical sample, focused more testing service contracts being awarded, in this year’s report both informs yourintently on the ‘enterprise’ market, and and an increased uptake of Testing as a own testing and quality assuranceincluded additional questions reflecting Service (TaaS) solutions. decisions and perhaps challenges somethe significant changes in our technology of your current thinking. We also valuelandscape, such as the almost ubiquitous Similarly, many organizations are looking your comments and ideas, and inviteuptake of mobile communications. to develop Testing Centers of Excellence you to contact either of us or any of the (TCOEs), which use a standardized testing contributors to this report regarding anyChanges taking place in the IT market methodology, best practices, and tools, question you might have concerning testingincreasingly require scalable, robust, and together with a flexible pool of professional and quality assurance.reliable applications and infrastructure. resources to ensure high levels of qualityFurther, the disruptive effects of and risk mitigation across all applications. Finally, we would like to thank all of thecloud computing adoption, increased individuals who participated in the researchmobility, and device proliferation require Over the last four years, the World Quality and gave generously of their time. Withoutorganizations to develop applications that Report, published by the Capgemini them, it would not have been possible tocan seamlessly interface with an ever- Group and HP, has established itself as produce the World Quality Report 2012-13.broader range of distribution channels and the largest annual survey of applicationroutes-to-market. Our research is evidence quality and testing practices, with a growing You can also reach us by sending an emailthat a focus on quality management is reputation as a valuable benchmark for to: worldqualityreport.s.uk@capgemini.comincreasingly regarded as a strategic value. organizations around the world. The report or worldqualityreport.s.uk@sogeti.com is the result of close collaboration betweenWhile this year’s research confirms many Capgemini, Sogeti, and HP, both leaders inof the trends from previous editions of our respective fields of outsourced testingthe World Quality Report, there are also a and Application Lifecycle Management WORLD QUALITY REPORT 2012-13 5
  5. 5. Executive SummaryExecutive SummaryThis is the fourth year that Capgemini, Sogeti, and HP have produced theWorld Quality Report. Our aim remains to examine the current trends inenterprise application quality across different industries and geographies forthe purposes of providing actionable insight for decision making. Our hope isthat readers will learn from our report and apply their new understanding toimprove the effectiveness of Quality Assurance (QA) and testing within theirown organizations.6 WORLD QUALITY REPORT 2012-13
  6. 6. Executive SummaryOur global survey was carried out againsta challenging economic landscape fororganizations of all sizes and sectors.Many countries continue to experience low,stagnant, or negative rates of economicgrowth, and many governments haveapplied harsh austerity measures aimed atreducing public expenditure. Some sectors,particularly public and financial services, Murat Aksu Charlie Liare under greater scrutiny to get IT right Vice-President, Global Alliance Executive for HP Vice-President, Global Service Line Testingthe first time and ensure improved value Capgemini Capgeminifor money. Social media and the speed bywhich information is exchanged are alsochanging the way consumers and providers effectively and QA teams need to stay As today’s mobile users – customers andof applications interact with one another. alert to the new disruptive technologies employees – expect interaction at theirConsumers of applications enjoy an and ensure their skill levels are attuned to fingertips, anytime, anywhere, making aenhanced scrutiny over service providers, both market and internal expectations. We business mobile and “always on” should bewhereas the providers are looking for explore these findings in more detail below. higher up the corporate and IT agenda.ways to respond to the resulting pressure,while also keeping an eye on a potential Has QA been caught off guard New operating models such asmarket uptick. by mobility? cloud-based models and Testing Centers of Excellence haveWe have focused our research on the As mobile adoption has become almost come of ageimpact of this complex environment on ubiquitous in developed markets, thetesting and QA, a discipline that is itself business imperative for mobile business Cloud-based models for testing have had aalso undergoing a series of technological is clear. But our study indicates that speed relatively slow adoption rate, but evidenceand cultural changes. of adoption and proliferation of handheld from our study of sustained wider-scale devices, coupled with use of social media, adoption and falling barriers indicates thatRapidly evolving IT landscape seem to have caught enterprise testing organizations are on the verge of muchnecessitates innovation in the by surprise. Organizations may not be giving more extensive use. Testing in a cloudQA discipline mobile the priority it warrants. Only 31% environment, for example, is growing at a of respondents across the world currently healthy rate. Some 28% of our respondentsQuality Assurance is undergoing a quiet test mobile applications – a figure that does currently use this infrastructure model, abut steady evolution from in-house not deviate much from region to region, and proportion forecast to rise to 39% by 2015.testing generalists to a structured those surveyed readily admit to being ill Moreover, only 4% say that they will notand efficient discipline, with a greater equipped for mobile testing. This suggests use the cloud in some way over the nextinfluence within the overall application that QA has fallen behind the mainstream three years, down from 31% only two yearsdevelopment lifecycle. We observe mobile curve. ago, a clear indication that testing in thefrom our survey the emergence of a cloud is set to become a new norm, asmultifaceted discipline with an increasing Reasons given are multiple: many report issues surrounding cloud, such as security,range of operating models at its disposal, that they don’t have the ability to test or are addressed.that enables a streamlined and cost- effectively certify mobile applicationseffective output that is also aligned with because of the lack of appropriate tools, Equally significant is the adoption of otherbusiness needs. processes, or expertise, and limited access cloud-based services. Software as a to the necessary devices. Moreover, the Service (SaaS) continues to grow; almost The level of QA and testing investment focus is firmly on efficiency of performance, a third (31%) of firms’ testing software ishas proved to be resilient in adverse cited by 64% of firms, rather than currently provided on a SaaS basis – upeconomic environments in most markets, functionality, usability, or security. Overall, from 25% last year, and by 2015 we predictand has stretched to accommodate an this argues for an underestimation of the that almost half of firms will use theever-increasing workload. But now testing infrastructure challenges posed by the on-demand model.resources need to prove themselves more mobile era, or an inability to address them. WORLD QUALITY REPORT 2012-13 7
  7. 7. Executive SummaryTesting as a Service (TaaS) – which can are changing to support the core areas of exist, there is a greater harmonization,be a combination of cloud-based testing the business that will require an optimized with fewer major differences evidencedenvironment and tools, and on-demand test platform for efficient delivery of testing, across the globe, at least within theexecution – is a cost-effective operating which we regard as a positive step. enterprise markets. One example ismodel for ad hoc quality projects and also that the proportion of an organization’sfor longer-term testing provisioning that For the first time, the World Quality Report software development lifecycle budgetdemonstrates similar rates of adoption. We has established a worldwide benchmark that is invested in testing, averaged acrossfound that 78% of respondents are planning for testing hourly rates, internally the regions, is 18%, with remarkably fewto move to TaaS in the next two years, rising and externally. Firms are paying a global regional variations.to 89% by 2015. This rapid take-up of TaaS average hourly rate of around $55 per houris fueled by cost-reduction drivers and for in-house testers, compared to $53 Our perspective is that testing is becomingits flexibility as a complementary model for external support – a differential of $2 less of a regionally facing discipline andto more comprehensive outsourcing or per hour. Obviously, rates vary greatly from that, led by globally active companies, thein-house Testing Centers of Excellence region to region and can be affected by the gap between emerging markets and more(TCOEs). So while some still see the security scarcity of available skills in local markets. traditionally mature markets is narrowing.of the cloud as a continuing threat, their Does this represent the price of quality? This is no doubt a reflection of the rapidnumbers are dwindling as the advantages Only an individual business can make that investment in skilled resources andmore clearly outweigh the drawbacks. decision, when evaluated against other tools, led by India, that has taken place in prime resources. offshore and nearshore countries – China,For TCOEs that have been underutilized South East Asia, South America, and,in the past, their time too has arrived. Confidence in QA resources more recently, Eastern Europe, CentralUndoubtedly an efficient approach to is a concern America, and parts of Africa. All pointindustrializing testing activity, TCOEs to testing emerging as a more uniformact as a virtual command center, using a While budgets might be in reasonable global discipline.standardized approach and a flexible pool shape, confidence in testing resources isof available resources. Our data reveals not resounding. A majority of organizations World Quality Reportthat, this year, 60% of respondents plan characterize their internal teams as recommendationson – or are – developing a TCOE, up from “average” at best, in their knowledge of45% last year. In our experience, we see core testing processes and methodologies, For providers and users of QA and testing, wethe concept is now sufficiently advanced and not necessarily up to speed with the offer these suggestions on how to apply theand its benefits more clearly understood: latest testing tools and technologies. Their findings of this year’s World Quality Report.a TCOE enables organizations to meet the assessment of external testers is slightlychallenges of the business – speed-to- better, with a third of organizations scoring QA needs to be a formalized stepmarket and doing “more for less” – head on. their external testers’ knowledge and in the application lifecycle abilities as “above average”. But less than While QA’s status within the IT organizationCan we put a price on quality? 5% of firms are fully confident that their has improved compared to a decade ago, testers (internal or external) are “best in we still see instances of the testing functionTechnological innovations and initiatives are class”. Despite initiatives and investment, being considered as an afterthoughtincreasing the workload for ever-stretched especially over the last decade or so, there or viewed as a roadblock preventingQA teams, but the corresponding QA and are clearly lessons to be learned in terms of an application from being deployedtesting budgets appear to be weathering the either real or perceived quality of output for to production. IT can only be as agile oreconomic storms. In 2011, these budgets both providers and users of QA. efficient as its weakest component.were not supporting the challenges facedin the marketplace, but this year’s survey Simply put, as the competitive landscape As IT continues to reinvent itself with newfinds testing budgets growing at a stronger forces organizations to update and optimize processes, paradigms, and platforms, theserate than last year, with 42% reporting that their testing resources and drive down innovations rarely include aspects of QA.budgets had risen over the past 12 months. costs and time-to-market, the overall The fact that testing is not being broughtWhile 18% forecast a fall when looking quality of testing resources needs to along the journey of reinvention often resultsahead to 2015, some 53% optimistically keep pace or change, to satisfy or exceed in QA being the bottleneck. Organizationsexpect budgets to rise, indicating a new the perceived and increasingly complex need to understand the critical role of QA indegree of confidence that the business is requirements of organizations in the future. the application lifecycle and treat it with themore committed to investing in QA. same respect as its peers. The emergence of a more globalMoreover, the focus appears to have and uniform industry Mobility should become fullyshifted from “business as usual” tasks integrated into testing prioritiesto investment in transformational In our previous surveys, we have focused We were surprised by the relatively lowwork, to drive enhancements, and attention on the differences between the level of proactive structured testing59% of the budget is now spent on major countries or regions surveyed. This in this increasingly essential area ofcustomer-facing applications. So priorities year we notice that while variances still business connectivity. We believe that8 WORLD QUALITY REPORT 2012-13
  8. 8. Executive Summary Re-evaluate the diversity of skills required The perceived quality of testing resources has a bearing on value for money, and the unexceptional assessment of ability needs examination. Over time, we expect that businesses will be even more exacting in their expectations of any and all testing resources when it comes to complex business-critical applications. All providers, in-house, contract, and external, should take notice. QA has made great strides in becoming a professional discipline with a distinct career path and specific skills. But this is a moving target; testing training at the individual and team level needs to keep pace with the fast- changing landscape, and should be properly funded, rather than being sacrificed under the pressure of deadlines.mobile testing needs to be a fully integrated and corporate culture, appetite for risk, andelement of the QA discipline, so that the need for control versus flexibility all play a Budgets are currently at a strong level, amobile strategy of the enterprise takes part in determining whether a new operating clear recognition of the value of the testingtesting into account right from the start. model should be adopted and which one. function to the organization, yet this isThe strategy should consider the objectives in contrast to the perceived value of theof the business owner, how the mobile On the journey from in-house testing actual resources. This gap can certainlyapp is delivered, and the target user generalists and developers to a be bridged by clearly targeting those skillfor the app whether that be customers, professionalized discipline wanting to areas that are essential to the business.suppliers, or employees. Organizations adopt “Shift Left” principles that embedneed to accept the paradigm shift brought QA earlier in the traditional software Conclusionabout by mobility and embrace the new development lifecycle, organizations shouldnotion of quality for mobile apps, which critically appraise their current set-up and Our findings in this report indicate thatis a departure from traditional standards review how testing in a cloud environment, QA continues to make steady progressapplied to desktop applications. cloud-based TaaS, and TCOEs can provide on repositioning itself further up the those critical incremental efficiency and business value chain on the journey ofNo doubt, standardization of devices will quality improvements. reinvention, emerging from the shadows ofease the complexity over time, but in the development teams to a value-for-moneyshort term, proliferation of smartphone Pay closer attention to demonstrating and mature discipline. Looking ahead, weand mobile devices, the roll-out of 4G, QA’s business value for the money anticipate further strides in the adoptionand use of social media will only continue Value for money is, of course, subjective, of “Shift Left” initiatives and greater use ofto exacerbate the situation. As will the but what does constitute a good return new models such as crowd sourcing whichneed to focus on the user experience on your QA investment? One measure is leverages knowledge and manpower fromand functionality testing as well investment in QA/testing as a proportion an undefined pool of resources, to improveas performance. If organizations are to of the total IT budget, averaging at 18% value-for-money ratios and timelines. Butturn the mobile opportunity into a business across the world. Below 18%, organizations greater improvement will not come aboutadvantage, some will need to “skill up” or should examine if they are investing enough without significant investment – not only“skill out”. to achieve or sustain the quality required. financially, in terms of budgets and skills, Those spending more should review their but also culturally.Evaluate the breadth of cost-effective operational efficiency.operating models to see if it is a good fit But organizations will need to exert firmIn the quest for greater testing maturity But more precisely, companies need to but flexible control to work across a widerbalanced against return on investment, establish specific measures to evaluate range of delivery models and technologies.organizations have an increasing array of the return on their overall investment, Budgets will need to work harder to effectoperating models from which to choose and testing departments and service the changes required to respond to a rangeto achieve their goals. Companies should providers need to act “smarter” in their use of demands from time-to-market, costfirst define the business goals they wish of tools, models, and communication to efficiency, and improved quality. And QA willto achieve and then evaluate the myriad stakeholders to demonstrate the business need to keep up with market expectations,of models against these objectives. Each value they provide in order to justify the remaining focused on providing real,model has its advantages and shortcomings, proportional spend on QA. measurable business value. WORLD QUALITY REPORT 2012-13 9
  9. 9. TheState ofQuality2012 12 Quality Budgets: Stepping up 18 Testing Centers of Excellence: A growing priority 24 Mobile Testing: Behind the curve 30 Cloud: Gathering pace 36 Quality Assurance Resources: A question of value10 WORLD QUALITY REPORT 2012-13
  10. 10. The State of Quality 2012 WORLD QUALITY REPORT 2012-13 11
  11. 11. The State of Quality 2012Quality BudgetsQuality Budgets:Stepping up“Investment is going to be Organizations are trying to balance market uncertainty with the need tofocused on those elements position for eventual growth, as well as tackling the challenges of social media and mobile technologies. At the operational level, a complex web of factors isthat support our wider forcing organizations to update and optimize their testing resources and internalbusiness objectives, as processes to drive reduced time-to-market and lower costs, as well as improvethere is a need for [ testing] quality using new beneficial delivery models, such as cloud and TCOEs.and the business side to be As a result, the QA function and QA budgets, in particular, are under the spotlight,much closer. Decisions need because to achieve these stretch business targets, organizations need to leverageto be made more quickly and their QA capabilities more effectively. This requires a step-up in investment over theproduct development time short and medium term, even in times of economic uncertainty.is shrinking – we need the Indeed, this year’s survey finds testing budgets are expanding at a healthier ratetools and skills to be able than the incremental upturn seen in 2011-12, when growth was failing to keep paceto answer that need.” with an increasing workload, due to cost-cutting measures. This positive step points to a desire among businesses to invest for the future and meet internal and external demands quickly and effectively.A telecommunications business,The Netherlands An upward curve In general, the survey provides good news regarding QA budgets, given the economic backdrop. Close to half (42%) of QA budgets have increased over the last year, with only 11% reporting a decrease. The greatest proportion (44%) saw no movement either way, but in a time of uncertainty, maintaining the same level of investment in the quality function is, in itself, a testimony to the importance of QA to the organization. Overall, this is perhaps a more positive situation than might have been predicted. Moreover, this upward curve is expected to continue over the coming three years, indicating a degree of optimism. A healthy majority (53%) expect their budgets to rise between now and 2015, while just a fifth (18%) forecast a fall. This seems to be a universal trend with little variation by region or sector.12 WORLD QUALITY REPORT 2012-13
  12. 12. The State of Quality 2012 Quality BudgetsCHANGE IN TESTING BUDGETS FROM LAST YEAR, ASCOMPARED TO NOW AND IN 2015 FIGURE 1 3% 8% INCREASED SIGNIFICANTLY 8% INCREASED SLIGHTLY 45% 10% 4% 3% 6% STAYED THE SAME 7% 36% DECREASED SLIGHTLY NOW 2015 DECREASED SIGNIFICANTLY NO BUDGET TO TEST 44% 26%Base: 1553 Respondents The majority of organizations expect testing budgets to rise between now and 2015 WORLD QUALITY REPORT 2012-13 13
  13. 13. The State of Quality 2012Quality BudgetsIncreasing pressure to 21% collect data manually and use Excel to a commercial competitive edge,demonstrate results as a presentation layer. However, while or because it needs to meet highly Excel is universally available and a low-cost stringent governmental regulationsBudgets may be on the upward trend, solution, it does not support the richness in or safety requirements, e.g. publicbut there is no room for complacency. An functionality that a professional reporting transportation systems.increase in investment will intensify the and QA management product can provide.focus on QA productivity, a function that is In times of greater business scrutiny, QA Increased focus onviewed primarily as a necessary cost base. organizations’ best interests will be served external activities by providing constantly updated businessEven as testing spend rises, companies KPIs that track return on investment. The areas in which testing investmentstill need to maintain downward pressure is being focused reflect the competitiveon costs, as the move to consolidation The magic number for pressures organizations feel in anin TCOEs and on-demand testing via the quality budgets unpredictable climate. A robust two-fifthscloud both illustrate. Any investment (41%) of testing spend is being dedicatedneeds to achieve a return, which means Testing budgets tend to average out at to transformational work – testing newthat continued or intensified scrutiny by around 18% of total spend on organizations’ applications, as opposed to maintenancethe business units who pay for QA services software development lifecycle, and this of existing software. This is a positiveis likely, in order to ensure that new figure seems to extend across all global indication that, even amid continued globaltesting resources are optimally deployed regions and industry sectors surveyed. economic turbulence, firms are continuingand managed. On balance, this would seem to be the to invest in developing new products benchmark proportion of the software and services.One way to ensure budgets are being development budget that organizationsspent productively is to effectively need to commit to testing. Similarly, the greater proportion ofmeasure the added value of testing testing budgets (59%) are being spent onand clearly communicate this to the Spending much below this level runs the customer-facing applications rather thanbusiness stakeholders. The good news risk of developing applications that are on internal organizational applicationsis that only 15% of organizations do not inadequate from a quality perspective, (41%), a clear focus on external applicationshave a process to consistently collect and or applications that are outdated before that have a direct impact on sales andpresent their quality KPIs to the business. their release when the QA process takes revenue growth. Firms are also boostingThe vast majority of organizations are too long to bring a product to market their investment in TCOEs in ordercollecting this data using QA management ahead of the competition. Spend too to compress the time-to-market foror business intelligence products in far above this level, and organizations new products. Companies, no matter whatthe market, or using common office may need to question the efficiency of sector or region, are investing in innovationapplications such as Microsoft Excel. In their QA operations, or ask whether the in order to maintain and capture marketfact, 56% indicate that they automatically additional spend is justified because the share and ultimately drive revenue.gather and share metrics using Excel, and superior application quality contributesSPEND ON EMPLOYEE- VERSUS CUSTOMER-FACING APPLICATIONS,AND TRANSFORMATIONAL VERSUS MAINTENANCE WORK FIGURE 2 EMPLOYEE FACING NEW TRANSFORMATIONAL WORK 41% 41% VS VS CUSTOMER FACING MAINTENANCE WORK 59% 59% Base: 1,221 respondents14 WORLD QUALITY REPORT 2012-13
  14. 14. The State of Quality 2012 Quality Budgets “The problem is increasing pressure to get things done yesterday. We have this conflict between testing adequately toA manufacturing our satisfaction and the business, Czech Republic testing time expected by those on the [business] side. Investment has increased but then expectation in speed of delivery has also increased. It’s not easy, as the organization needs to be able to jump into a much faster testing mode while maintaining quality standards.” WORLD QUALITY REPORT 2012-13 15
  15. 15. The State of Quality 2012Quality Budgets “My investment going forward will [focus on] bringing my internal teams up to standard, increasing the level of automation used, and developing greater efficiencies throughout. Although our testing budget is not separate from the wider IT budget, the level of monitoring and performance measuring has increased – we need to justify our costs.” An automotive business, Brazil Investment priorities: As a result, organizations are seeing an Looking ahead to 2015, however, prioritiescurrent and future imperative to upgrade existing suites are set to shift from upgrading existing of testing tools. More than half (55%) of tools to buying external resource support.QA investment priorities are largely shaped respondents describe the upgrade process A smaller proportion (44%) of firms stillby the new market landscape where as the budgetary line item receiving the expect existing tools to be their number oneorganizations must achieve more with fewer greatest increase in spend, and in our investment focus three years from now,resources, find ways to reduce cost, and opinion this is driven by a desire to move to but it is secondary to purchasing externaldeal with increased competition from both newer application lifecycle platforms and resources to supplement their internal QAbricks-and-mortar and online channels. software-on-demand type models. team (46%). AREAS OF GREATEST INCREASE IN SPEND: NOW AND IN 2015 FIGURE 3 MORE INVESTMENT IN 55% NOW EXISTING TESTING TOOLS 44% 2015 INTERNAL PROFESSIONAL 46% NOW TESTING RESOURCES 44% 2015 EXTERNAL OUTSOURCED 43% NOW RESOURCES 46% 2015 TEST ENVIRONMENTS/ 39% NOW INFRASTRUCTURE 39% 2015 NEW TESTING TOOLS 19% NOW 22% 2015 QA BUDGET IS NOT 12% NOW INCREASING 13% 2015 Base: 1553 Respondents16 WORLD QUALITY REPORT 2012-13
  16. 16. The State of Quality 2012 Quality BudgetsNew product development will generate the when compared to other regions and to the world are radically reducing spend toneed for new testing skills, tools, and devices, global average (19%). The Nordics and North cope with the fallout from the worldwideprompting organizations to seek support America are at the other end of the scale, with economic crisis. Almost half (48%) of publicfrom external partners. Investment in new just 15% of firms giving top priority to buying organizations’ testing budgets increasedtesting tools will also rise, as new products new tools, although both regions indicate a from the previous year, the highestand services are introduced to the market. focus on upgrading existing tool sets, which proportion of all sectors studied, with 56%Some 22% see this as the top spend priority might suggest that they are looking to better also predicting an increase between nowfor 2015, up from 19% at present. leverage their shelfware and make their and 2015. existing investments go further. SouthernBudgets relatively consistent European firms are more heavily focused Our observation is that public bodies areacross regions on upgrading existing tools (63% compared investing in consolidating their testing to 55% on average), perhaps suggesting operations to drive cost efficiencies inThe greatest degree of consensus between that they are maturing from a more manual- the face of austerity. With a lingeringglobal regions in this year’s report is around intensive process to incorporate automation. legacy of failed major IT projects, QA hasQA budgets. Budget levels hover consistently to be stepped up to derive more fromaround the all-sample mean of 18% of total Looking ahead, Southern Europe will be existing capabilities and improve testing.software development spend across all but increasing its investment in outsourced Investment in existing testing tools is seenone of the geographic areas surveyed. The testing, with a majority (57%) of firms in as one of the solutions to remedy the failedexception is South America, where testing this region viewing this as their main QA IT projects issue. Over half (58%) of Publictakes up some 25% of the budget. investment priority for 2015, compared to Sector entities identify this as the greatest 46% overall. Just over half of respondents increase in spend.We believe that this is an indication of in the UK and Ireland, North America,targeted South American investment. As and Eastern Europe also expect external Manufacturing firms are the mostBrazil, in particular, continues to boom suppliers to receive the lion’s share of their optimistic among the sectors, anticipatingeconomically and expectations around testing spend three years from now, further an increase in testing budgets three yearsapplication needs are raised, so testing evidence that the global move towards from now: 58% expect spend to growinvestment is being enhanced. outsourcing will continue to grow. between now and 2015, against a mean of 53%. Although our experience has shownThe pattern is similarly consistent for budget Public Sector leads the way that manufacturing firms tend to spend lessincreases, with the exception of Asia. Asian for increased QA budget on IT overall compared to other sectors, itfirms appear to benefit from the most is an increase nonetheless. Some 30% ofaggressive budget expansion compared to The pattern of uniformity around testing manufacturers predict that new testingother regions, as was the case in 2011. Half budgets seen between the regions is tools will be the greatest spend in 2015,of Asian respondents saw an increase in repeated across the industries surveyed, compared to 22% on average. CPRD firmsbudget from last year, compared to the global with QA budgets in most sectors fairly are similarly optimistic in terms of budgetaverage of 42%. More still (60%) expect an evenly balanced around the average 18% outlook in 2015, with 57% anticipating anincrease over the next three years, against an of total software development spend. The increase, and the sector places the highestaverage of 53%. greatest disparities are in the Consumer priority on innovation, committing 45% of Products, Retail, and Distribution (CPRD) spend to new transformational work (all-The balance of spend between new and space (22%), reflecting a higher focus on the sample average 41%).maintenance work also presents a fairly quality of end-user consumer applications,uniform picture. Here again, South America and Financial Services (FS) at 20%, a Energy and Utilities (E&U) is an intriguingis the outlier, where firms are spending traditionally heavy investor in QA. exception in terms of current investmentalmost half (48%) of testing budget on priorities, spending notably more on bothtransformational work (average 41%). The Public Sector, surprisingly, is existing (65%) and new (23%) tools, and experiencing unexpected budget uplifts transportation firms are the most keenlyThe same goes for the split between spend on at a time when governments around the focused on outsourcing. More than halfcustomer- and employee-facing applications. (52%) see this as the fastest rising area ofSouth America again goes against the global spend, compared to a mean of 43%.trend, being the least externally focused 48%region, spending only 52% on customer In summary, against a gloomy backdrop of of publicapplications (compared to 59% on average). depressed economic activity and relative organizations’ financial instability affecting most regions testing budgetsInvestment priorities increased last surveyed, QA budgets have held theirvary by region year - the highest place in the IT overall spend. We believe this is a somewhat late recognition of the proportion of allWhen it comes to investment priorities, inherent value of testing and QA in ensuring sectors studiedthere are greater variations between robust business-critical applications asthe regions. Eastern Europe currently has an enabler of commercial growth andinvested far more in new testing tools (33%) public reputation. WORLD QUALITY REPORT 2012-13 17
  17. 17. The State of Quality 2012Testing Centers of ExcellenceTesting Centersof Excellence: Agrowing priorityConsolidation and standardization are continuing Testing automation, another aspect of a TCOE, isthemes for all organizations around the world and also scarce, with just 32% of test cases supportedthe testing function within these companies is not by automation. These figures indicate that companiesimmune to these trends. Aided by the emergence continue to perform QA as siloed projects withof new delivery models, technologies, and best minimal opportunity for standardization and sharingpractices, organizations are looking to streamline of best practices across the enterprise. Of note, thethe way in which their QA function is structured greater proportion of automated testing is conductedand run. TCOEs are undoubtedly an efficient approach offshore, a testament to the growing confidence into industrializing testing activity. They act as a moving this type of testing beyond national borders,virtual command center that uses a standardized with only 40% being performed onshore.testing methodology, best practices, automation,metrics, and tools, while managing a flexible pool Flexible resource pools that can ramp up and downof available resources, both internal and external, with the changing needs of the company is anotherto ensure high levels of quality across applications attribute of a TCOE. Our research shows that thebefore deployment and during production. TCOEs majority of organizations (51%) still run testing asalso provide visibility into the level of quality for any an in-house function, and only 13% have moved to asoftware system or project, helping IT management service fully managed by an external provider. Withmake deployment decisions based on business risk. the exception of North America, QA organizations have yet to truly embrace the cost benefitsGrowing from a low base of offshoring. The majority of testing resources (52%) are still based on home territory.We found that only 6% of companies around theworld have developed QA into a fully functional TCOE. Reasons for this global discrepancy vary from regionThis number is a significant increase, in terms of to region. In South America, government-imposedpercentage, from last year’s findings (4%), but the customs taxes and new rules governing the usesingle digit number still reflects a long road ahead for of offshore services discourage the use of thiscompanies to universally adopt the TCOE model. flexible pool of resources. In Europe, particularly in continental Europe, the lack of available offshoreOne of the key attributes of a TCOE is a centralized resources skilled in the native language hinders thepool of resources that can be leveraged across adoption of an offshore model, although we are nowmultiple projects in an organization. We found that seeing the UK and Nordics embrace this model atclose to 70% of organizations still rely on elements a faster pace. In other areas of the world, the laborof decentralized testing, while more than a quarter arbitrage does not yield adequate amounts of return(27%) of them describe their testing function as on investment to warrant an offshoring strategy.“highly decentralized”.18 WORLD QUALITY REPORT 2012-13
  18. 18. The State of Quality 2012 Testing Centers of Excellence “[A TCOE] is something I’m building here; it represents a significant change in our approach but one that I personally believe toA manufacturing be vital to our businessbusiness, United Kingdom since I’ve come from an organization with an operating testing center. We know that it can reduce costs and overall time of testing practices, and given the competitive nature of the market we operate in, it’s critical to get speed and costs down whilst getting quality up. We believe a TCOE can do this.” WORLD QUALITY REPORT 2012-13 19
  19. 19. The State of Quality 2012Testing Centers of Excellence PLANS FOR A TESTING CENTER OF EXCELLENCE FIGURE 4 NO PLANS IN PLACE 6% PLAN TO DEVELOP INTERNALLY MANAGED TCOE WITHIN THE NEXT 2 YEARS 15% PLAN TO USE A THIRD-PARTY 34% COMPANY WITH A TCOE CAPABILITY IN-HOUSE TCOE STARTED WITHIN LAST 2 YEARS, BUT NOT YET FULLY OPERATIONAL 21% IN-HOUSE TCOE IS FULLY OPERATIONAL 24% Base: 924 RespondentsAn appetite for change The percentage of organizations reporting mature economies. Add to this the 17% that they have no plans to set up a TCOE aiming to enhance the agility of their QAHowever, we are witnessing a growing has fallen significantly from last year (45% operations, and it is clear that speed is ainterest in centralizing QA operations. to 34%), and we predict that this will fall key driver of growing TCOE adoption.Survey respondents indicated a significant further over the coming few years as thetrend towards new investment in TCOEs, TCOE model gains recognition for delivering In an unpredictable global economy, andguided by the imperatives to reduce time- performance improvement. This trend begs with market and business expectationsto-market and costs, increase quality, the question – what is driving the surge in focused on producing better productsand gain better control of the whole interest in industrializing testing activity? faster, time-to-market has becomelifecycle process. essential in gaining or maintaining Less cost, more speed competitive edge and maximizing profits.Whilst only 6% of organizations currently As a result, development cycles arehave a fully fledged operational center, Firms are centralizing their QA being squeezed, and testing teams havethis presents a 50% increase from 2011 infrastructure in order to reduce time-to- to do more with less. A TCOE offers the(4%) – a small but significant step. On a market, bring down the cost of testing, and opportunity to meet these challengespositive note, almost two-thirds (60%) manage testing resources more effectively. head on.of firms are currently in the process of Compressing time-to-market is the primarybuilding or planning a TCOE, while a third of reason for TCOE implementation, cited Reducing the cost of testing activity isthese companies (21%) plan to leverage an by 36% of respondents, and especially next on the priority list, identified by justoutsourced partner. those in what might be called the more over a quarter (27%) of organizations,20 WORLD QUALITY REPORT 2012-13
  20. 20. The State of Quality 2012 Testing Centers of Excellence EXPECTED BENEFITS FROM IMPLEMENTING A TESTING CENTER OF EXCELLENCE FIGURE 5 REDUCED TIME-TO-MARKET REDUCED COST BETTER RESOURCE MANAGEMENT 36% 27% 25% (HW/SW AND PEOPLE) BETTER QUALITY STANDARDIZED TRACKING OF 22% 19% 17% PROJECTS THROUGH METRICS IMPROVED QA AGILITY TO SUPPORT BUSINESS OBJECTIVES Base: 1221 Respondentsclosely followed by the need to manage hardware, “For us, the move towards a TCOE is part of thesoftware, and human resources more efficiently and wider union between the business and IT side ofproductively (25%). the organization. We now have a much strongerIt is interesting to observe that cost – at least for link between the two, and while that meanseconomies such as North America – is not the we feel more able to understand and answeroverriding priority. Most organizations have been demands, it does increase pressure on testingfocused on cost cutting for many years, and costshave been pared down to a minimum, with strategies to be more efficient as a whole – decentralizedsuch as outsourcing delivering benefits. It should testing teams just couldn’t provide thatnot be a surprise, therefore, that companies in these efficiency and it meant that walls existedregions are now seeking advantages beyond thepurely financial. between the teams, whereas shared experience and best practice are far easier within aThese objectives for moving to a TCOE model prompt TCOE framework.”us to ask: why has efficiency in meeting time-to-market deadlines risen to the top of firms’ agendas?What is different in today’s market environment A financial services business, Finlandthat has rendered the decentralized approach lessdesirable for the QA function in some instances? WORLD QUALITY REPORT 2012-13 21
  21. 21. The State of Quality 2012Testing Centers of ExcellenceBased on the survey responses, together used by over a third of testers (37%), So as markets mature, cost becomes lesswith the experiences of our clients, we analysts (34%), and developers (29%). And of a driver, and the focus moves towardsbelieve there are three key factors at play: encouragingly, three-quarters (75%) of reducing time-to-market.the market environment, rising technology Agile testing is already being performedexpectations, and diversity offshore, a situation that seemed unlikely a Sectorsof delivery models. few years ago. Public organizations seem to be leading1. The market environment Offshore adoption: a key the way in TCOE adoption, because, alongContinuing market unpredictability means determinant for TCOE with Telecoms, Media, and Entertainmentthat organizations can ill afford to take implementation (TME), they can claim the lowest proportionthe emphasis away from cost reduction. of organizations with no plans to implementMany remain firmly in cost-saving mode Centralizing and professionalizing the QA a TCOE (30%). They have also experiencedwhere QA is concerned: some still aim to function undoubtedly presents a valuable the steepest drop in this proportion – downshave as much as 30-40% off their current opportunity to save costs in a tough markedly from 59% last year.testing budgets. economic climate. However, motives driving consolidation are, in part, a function of how This is likely being driven by the costThis report also notes that QA budgets far different regions have already leveraged reduction imperative that is behind manyare increasing but that more is being the cost benefits of offshoring. organizations’ TCOE plans. With spendingdemanded of teams and more complex austerity being implemented by manydevelopments are requested. So while As a rule, markets with greater levels of governments, public bodies are being askedbudgets are increasing, there is still a need offshoring – primarily North America and to do more with less, while maintainingto control costs. the UK – are less focused on cost reduction. service levels and quality. As an efficiency Offshoring has been considered, executed, driver, an onshore TCOE is a compellingIn parallel, there is considerable pressure and reviewed, and there is little remaining model, particularly as offshoring is ato develop applications and bring them to scope to further squeeze labor arbitrage. political anathema to many countries.market as rapidly as possible. At the same Therefore, other models are being exploited,time, firms also need to focus on innovation such as TCOE, for achieving better quality, For other sectors, the imperativeto prepare for an eventual upturn in the being quicker to market, and more is responding faster to consumerglobal economy. effectively using resources. Cost is not an expectations, changing market demands, easy win for the more mature markets – and time-to-market. Two-thirds (66%) of2. Rising technology expectations they have to look at other key benefits to CPRD respondents now operate a testingIntensifying this pressure on application drive more efficient and focused QA. center, or plan to – significantly up fromdevelopment is an increasingly 41% last year. Streamlining of supply chainscompetitive landscape, fuelled by Conversely, other regions with lower levels and application testing would appear togrowing customer demands. Customers of offshoring remain more keenly focused be complementary. And this upturn is also– both internal and external – expect to on cost reduction. This is understandable, true of the FS industry, where adoption is upaccess services and purchase products given that many are in the early stages of from 58% to 69% .whenever they want, on whatever device offshoring and thus have more resourcethey choose. To meet this demand for 24/7 costs to reduce. This difference is reflected Finally the High Tech sector, which includesavailability, there is mounting pressure to in our survey results. aerospace and defense companies,deliver applications as quickly, efficiently, appears to be closing down internal centersand cheaply as possible – and this is North America and the UK are the only two in favor of an outsourced model, as time-to-particularly evident in the mobile space regions where the majority of testers are market is a major issue in this sector. Thewhere expectations are well advanced and located either offshore or nearshore (55% sector had a remarkably high proportionare most likely going to increase over time. and 50% respectively). And they are the two of in-house facilities up and running last least cost-focused markets: only 15% and year (17% compared to an overall average3. Diversity of delivery models 16% of companies cite cost reduction as a of just 4%), while only 8% planned toIn response, QA departments are reason for establishing a TCOE. work with a third party. This year, thecapitalizing on the efficiency gains tables have turned. Just 6% now haveoffered by cloud technology, increasingly In South America, by contrast, offshoring their own TCOE, while 17% plan to use anoutsourcing testing to cloud environments, has been far less exploited, and (it should outsourced partner.and buying resources on demand, to be noted) in Brazil is actively discouragedreduce cost and time to bring applications by government policy. Only 38% of testers This year’s data indicates that TCOEs areto market. in the region are offshored, compared to gaining traction, with growing levels of 48% on average. Here, cost reduction is a adoption across the board. As a flexibleA further step to speed up the development far greater priority for TCOE operations than model, it offers significant advantagesprocess is the wider adoption of elsewhere: more than half of respondents and opportunities to fully leverageAgile methodology. This is currently being (54%) identify cost reduction as the primary existing resources. driver (compared to just 27% on average).22 WORLD QUALITY REPORT 2012-13
  22. 22. The State of Quality 2012 Testing Centers of ExcellencePercentage of offshoring compared to costreduction as A reason for TCOE FIGURE 6NORTH AMERICA UK AND IRELAND NORDICS 55% 15% 50% 16% 50% 44%WESTERN EUROPE SOUTHERN EUROPE EASTERN EUROPE 49% 29% 49% 25% 43% 32% ASIA SOUTH AMERICA AUSTRALIA AND NEW ZEALAND 36% 29% 38% 54% 42% 20% GLOBAL AVERAGE Percentage of testers offshore or nearshore 48% 27% Percentage of respondents citing cost reduction as reason for TCOE Base: 1553 Respondents Organizations are turning to TCOEs toconsolidate their QA operations in response to market pressures, technology demands, and emerging models of testing delivery WORLD QUALITY REPORT 2012-13 23
  23. 23. The State of Quality 2012Mobile TestingMobile Testing:Behind the curveMobile has certainly changed the game as the mobile era adds a further level of desktop application. Performance is farfor enterprise IT. Organizations around complexity to users’ needs, demands, ahead of functionality (cited by 48%)the world now need to deliver continuous and expectations. on firms’ priority list for mobile testing.access, anytime and anywhere, to Security, a perennial concern where mobilethousands of employees and millions Underprepared and ill equipped is concerned, is a priority for less thanof customers, over a bewildering array a fifth (18%) of organizations. This doesof devices. Our research suggests that organizations not mean that companies are no longer may not be giving mobile the priority concerned with security, but rather thatDue to cost pressures, firms are no longer it deserves. Only 31% of respondents the perspective on security has changedwilling to pay for, and thus control, their currently test mobile applications, and with the bring-your-own-device cultureemployees’ mobile-access devices, leading those surveyed readily admit to being ill and method of disseminating apps. Unliketo a bring-your-own-device culture. The equipped for mobile testing. This suggests with traditional desktops, companiessituation is further exacerbated because that QA has fallen behind the mobile curve. cannot control what apps are installed onorganizations are unwilling or unable personal devices, so the concept of securityto support the proliferation of personal A troubling two-thirds of organizations is shifting from a denial of access to thedevices such as smartphones and tablets (65%) do not have the right tools to test protection of sensitive data and ability tobeing used at work by employees. However, mobile applications, and equally concerning wipe clean certain apps or data.as the prevalence of non-company is the 52% who do not have access to theapproved devices becomes mainstream required devices. It would appear that Even lower on the radar is the certificationand employees demand access to the organizations either do not have a good of applications, cited by just 14%, despiteflexibility and productivity that mobile grasp of the infrastructure challenges high levels of inconsistency in the quality ofoffers, IT departments are now being posed by the mobile era or are unable to mobile apps on different operating systems.forced to embrace this as the new reality address these new demands. Meanwhile,and address the unique challenges of this a third of organizations lack the testing Specialist partners areparadigm shift. methodologies and processes (34%) and plugging the gap specialist expertise (29%) necessary toMobile has also turned IT security effectively certify mobile applications. The number one criterion when selectinginside out. A firm’s entire infrastructure external partners to assist with mobilecan no longer be tucked safely behind a Performance is the key testing is the capacity to test across severalvirtual private network and firewall. What’s to success networks, identified by almost two-thirdsmore, employees are prepared to download (62%) of organizations. This reflects theand try out apps on mobile devices with a Organizations that are conducting mobile need to ensure coverage, as businesseslatitude they would not consider acceptable testing seem to be working towards a seek partners who can deliver testing in aon a desktop or laptop. different set of quality standards as variety of environments. compared to traditional testing. TheIn addition, the need for mobility has primary focus is on the application’s Interestingly, cost reduction is less of areset expectations of what constitutes performance rather than its functionality. consideration (cited by only 25%), despiteapplication quality. With traditional Efficiency of performance was identified as being high on the list of factors driving bothsoftware, users expect flawless a focus for mobile testing activity by almost cloud and TCOE adoption. This may be duefunctionality first and foremost, but mobile two-thirds (64%) of firms. to a number of factors: mobile applicationsusers are seeking convenience. They expect are relatively cheap to produce, the focusrobust performance and usability on the The usual priorities – such as functionality is on managing risk rather than cost,move, and are more inclined to tolerate the and security – are being pushed lower and firms’ need to address their lack ofoccasional glitch along the way, as long as down the scale as companies place more readiness and expertise where mobile QAthe application performs well and is user- emphasis on the demands of end-users is concerned.friendly. QA teams may need to rethink on mobile platforms, which are differenttheir testing strategies and priorities from the demands of a traditional24 WORLD QUALITY REPORT 2012-13
  24. 24. The State of Quality 2012 Mobile TestingCHALLENGES TO MOBILE TESTING FIGURE 7 DO NOT HAVE THE RIGHT TOOLS TO TEST 65% DO NOT HAVE THE DEVICES READILY AVAILABLE 52% DO NOT HAVE THE RIGHT TESTING PROCESS/METHOD NO MOBILE TESTING 34% EXPERTS AVAILABLE 29% DO NOT HAVE IN-HOUSE TESTING ENVIRONMENT 19% 18% NOT ENOUGH TIME TO TESTBase: 485 RespondentsPRIORITIES WHEN TESTING MOBILE APPLICATIONS FIGURE 8 EFFICIENCY/PERFORMANCE (INCLUDING NETWORK-RELATED) FUNCTIONALITY 64% PORTABILITY 48% 46% USER INTERFACE/EASE OF USE 36% 31% COMPATIBILITY/ REGRESSION TESTING 18% SECURITY (PROTECTION OF SENSITIVE 14% DATA OVER THE PHONE OR AIR) CERTIFICATION OF APPLICATIONBase: 485 Respondents WORLD QUALITY REPORT 2012-13 25
  25. 25. The State of Quality 2012Mobile Testing Regional variations Regions with traditionally high levels of mobile adoption, with the exception of Asia, We might expect the strongest emphasis have – unsurprisingly – the highest levels on QA in the mobile arena to be seen in of mobile testing. Over one-third of firms Asia in particular, as smartphone and in Australia and New Zealand, the Nordics, tablet penetration are accelerating at a and Southern and Western Europe currently blistering pace in China. But the distinction test mobile applications. between East and West economies is not clear-cut. Indeed, in Asia, the proportion Regions displaying lower levels of mobile of organizations conducting mobile testing testing include Eastern Europe and the UK sits at 30%, close to the all-sample average and Ireland, where only a fifth of businesses of 31%. carry out mobile testing, and just a quarter do so in North and South America. Percentage of respondents testing mobile applications, by region FIGURE 9 26% 39% 22% 38% 38% 21% 30% 24% 37% WESTERN EUROPE NORDICS UK AND IRELAND EASTERN EUROPE NORTH AMERICA ASIA SOUTHERN EUROPE SOUTH AMERICA AUSTRALIA AND NEW ZEALAND Base: 1553 Respondents26 WORLD QUALITY REPORT 2012-13
  26. 26. The State of Quality 2012 Mobile TestingAsian challenges Greatest challenge when testing mobile applicationsThe explosion of the mobile market in Asia is and devicesreflected in the QA challenges and priorities faced FIGURE 10by organizations in the area. A greater percentageof Asian firms confirm they faced the common ALLmobile challenges as compared to all the companies RESPONDENTS ASIAon average. 79%Asian firms also place much more emphasis onportability and compatibility across devices whenconducting mobile testing. Portability is identifiedas a priority by 58% of businesses in this region,compared to less than half (46%) on average. 65%Compatibility is cited by almost as many (55%),compared to less than a third (31%) overall. 58%This focus might be driven by the far wider selectionof devices available to users in the region. In China, 52%for example, mobile customers can choose fromjust about any handset available worldwide. In theUS and Europe, networks offer only the handsetsfor which they have contractual arrangementswith manufacturers. As a result, the choice availableto customers is far more limited. 34% 29%Performance versus functionality 26% 18%Elsewhere, the emphasis on application performanceis particularly acute in the UK and Ireland (75%),against an average of 64%. This could be due toits being a more mature market where end-usersdemand more from their mobile apps. Consider thelandscape a decade ago, when having a browser on amobile device was touted as a new feature, whereastoday these same users want apps that are tailoredfor specific needs. The opposite is the case in SouthAmerica and Eastern Europe, where only around half NOT ENOUGH TIME TO TESTof organizations focus on performance. DO NOT HAVE THEInstead, South American firms are focused primarily RIGHT TOOLS TO TESTon functionality, and more than four-fifths identifythis as a priority, compared to just under half overall. DO NOT HAVE THE DEVICESMobile security is also a heightened concern in READILY AVAILABLEthe region, cited by 30% against an 18% average.Eastern European businesses are even more worried NO MOBILE TESTINGabout security (36%), and are equally concerned EXPERTS AVAILABLEwith application certification (compared to just 14%on average). Base: 485 Respondents WORLD QUALITY REPORT 2012-13 27
  27. 27. The State of Quality 2012Mobile Testing Sector perspectives A look at mobile testing by industry throws up some Public bodies are also the least focused on surprising results. Firstly, it is Manufacturing firms, performance, although this is still their greatest area rather than those from more consumer-facing of priority, cited by more than half of organizations industries, that are most likely to conduct mobile (53% compared to 64% average). The sector places testing (37% compared to 31% on average). a higher emphasis than average on security and user experience, in line with their public responsibilities. We might expect the TME sector to be most at ease with mobile technology. However, their QA demands Despite its reputation for innovation, the FS are uniquely complex, comprising the need to test industry faces some of the most acute mobile networks, equipment, and billing systems, among testing challenges. Almost three-quarters (72%) others that are applicable only to TME firms. As a of FS firms lack the right tools, compared to 65% result, just 30% of firms in the technology-intensive on average, and 38% lack mobile testing expertise, TME space currently test mobile applications, against 29% overall. Financial firms are the most equal to the all-sample mean. Some 62% lack the acutely concerned about application performance necessary tools to test mobile applications. This is with 72% citing this as a priority. only slightly better than average (65%), and almost one-third (32%) lack in-house expertise, compared User-friendliness is an understandable priority for to just 29% overall. Perhaps this is due to the TME the CPRD space. Ease of use and the user interface providers focusing their QA efforts on the mobile are a focus for 50% of CPRD firms (compared to 36% infrastructure and leaving the testing of mobile on average). devices to the handset manufacturers. Mobile is certainly having a significant impact on the Public Sector organizations (27%) are the slowest QA function and it seems that QA has been slow to to adopt mobile testing. This may not be quite so react to the expectations from within the business. surprising in a sector not traditionally viewed as We would expect swift reassessment of testing a leader in technological innovation. But it does priorities and increased investment over the coming present a major issue in an era when citizens year to handle the rush of mobile applications either increasingly expect to interact while on the move in-house or with specialist providers. with public service providers, just as they do with commercial brands. “It’s an area of our business that typifies how things are changing, in my opinion. The guys in marketing say ‘we need a mobile app’; sales or product execs agree and then send it across to development and testing to push it through in half the time we might actually need. It’s not going away either. More and more of our business is done online [in general] so it’s something we need to get hold of and understand quickly.” A retail business, United Kingdom28 WORLD QUALITY REPORT 2012-13
  28. 28. The State of Quality 2012 Mobile TestingPercentage of respondents who test mobileapplications or devices, by sector FIGURE 11 27% PUBLIC SECTOR 37% MANUFACTURING 35% ENERGY AND UTILITIES 35% HEALTH AND 34% CONSUMER 31% TRANSPORTATION LIFE SCIENCES PRODUCTS, RETAIL, AND DISTRIBUTION 31% FINANCIAL SERVICES 30% HIGH TECH 30% TELECOMS, MEDIA, AND ENTERTAINMENTBase: 1553 Respondentsgreatest challenge when testing mobileapplications or devices, by sector FIGURE 12CONSUMER PRODUCTS,RETAIL, AND DISTRIBUTION TRANSPORTATION ENERGY AND UTILITIES FINANCIAL SERVICES63% 50% 27% 23% 27% 20% 59% 56% 41% 22% 7% 26% 68% 42% 32% 29% 29% 23% 72% 50% 36% 38% 21% 14% TELECOMS, MEDIA, ANDMANUFACTURING HIGH TECH HEALTH AND LIFE SCIENCES ENTERTAINMENT63% 43% 40% 20% 26% 31% 62% 35% 50% 12% 23% 12% 66% 64% 22% 26% 22% 17% 62% 62% 30% 32% 12% 16% PUBLIC SECTOR DO NOT HAVE THE RIGHT No mobile testing experts TOOLS TO TEST available DO NOT HAVE THE DEVICES Do not have in-house testing READILY AVAILABLE environment DO NOT HAVE THE RIGHT TESTING PROCESS/METHOD Not enough time to test 64% 51% 40% 30% 13% 19%Base: 485 Respondents WORLD QUALITY REPORT 2012-13 29

×