This document provides guidance on integrating independent quality assurance and testing across the product lifecycle to shorten software development cycles. It discusses how outsourcing QA can help technology companies innovate faster by reducing development cycles, improving customer satisfaction, and reducing costs. The key benefits of outsourcing QA include focused in-house resources, reduced total cost of ownership, and reduced technology risks.
Bots for Quality: Augmenting QA's Scope in the Digital AgeCognizant
To help promote the digital journey, software quality assurance (QA) can be transformed through applying machine learning and predictive analytics to produce an insight-driven approach based on smart next-generation automation.
Candid Conversations With Product People: Using Continuous Customer Testing f...Aggregage
After weathering recessions with a wide range of iconic customers, CEO and Product Manager Luke Freiler has seen first hand the impact the Voice of the Customer has had in making or breaking tech companies during hard times. He's going to walk you through an agile process for continuous customer testing that saves you time and gives you full confidence in your products — no matter how many you're sending out the door this year.
Delight Your Customers with Four Eyes of Quality: Agile Quality Improvement S...Kaali Dass PMP, PhD.
IT Project success depends on realizing value realization and customer success. In addition to stimulating test environment and test data, projects need to focus on geographically distributed and culturally diversified people who can simulate End-user scenarios, User Experience, and Customer Outcomes
This presentation focuses on leveraging agile teams to simulate an end-user environment based on the team’s knowledge, understanding, and skills.
Enterprise Agile release planning is complicated when multiple agile teams work together to deliver combined capabilities, and the scope for a release span across multiple business functions, processes, and systems. This paper presents agile release planning models for large global organizations delivering business capabilities using IT projects.
Software organizations that want to maximize the yield of Software Testing find that choosing the right testing strategy is hard, and most testing managers are ill-prepared for this. The organization has to learn how to plan testing efforts based on the characteristics of each project and the many ways the software product is to be used. This tutorial is intended for Software professionals who are likely to be responsible for defining the strategy and planning of the testing effort and managing it through its life cycle. These roles are usually Testing Managers or Project Managers.
Traditional, full-code waterfall application development, with its focus on a sequential define-develop-test-deploy-maintain approach has given way for many enterprises to low-code/no-code development
Bots for Quality: Augmenting QA's Scope in the Digital AgeCognizant
To help promote the digital journey, software quality assurance (QA) can be transformed through applying machine learning and predictive analytics to produce an insight-driven approach based on smart next-generation automation.
Candid Conversations With Product People: Using Continuous Customer Testing f...Aggregage
After weathering recessions with a wide range of iconic customers, CEO and Product Manager Luke Freiler has seen first hand the impact the Voice of the Customer has had in making or breaking tech companies during hard times. He's going to walk you through an agile process for continuous customer testing that saves you time and gives you full confidence in your products — no matter how many you're sending out the door this year.
Delight Your Customers with Four Eyes of Quality: Agile Quality Improvement S...Kaali Dass PMP, PhD.
IT Project success depends on realizing value realization and customer success. In addition to stimulating test environment and test data, projects need to focus on geographically distributed and culturally diversified people who can simulate End-user scenarios, User Experience, and Customer Outcomes
This presentation focuses on leveraging agile teams to simulate an end-user environment based on the team’s knowledge, understanding, and skills.
Enterprise Agile release planning is complicated when multiple agile teams work together to deliver combined capabilities, and the scope for a release span across multiple business functions, processes, and systems. This paper presents agile release planning models for large global organizations delivering business capabilities using IT projects.
Software organizations that want to maximize the yield of Software Testing find that choosing the right testing strategy is hard, and most testing managers are ill-prepared for this. The organization has to learn how to plan testing efforts based on the characteristics of each project and the many ways the software product is to be used. This tutorial is intended for Software professionals who are likely to be responsible for defining the strategy and planning of the testing effort and managing it through its life cycle. These roles are usually Testing Managers or Project Managers.
Traditional, full-code waterfall application development, with its focus on a sequential define-develop-test-deploy-maintain approach has given way for many enterprises to low-code/no-code development
Understanding the who, what, why, and when of quality is essential in implementing an effective Quality Program. It requires a combination of distinct disciplines: Quality Assurance, Quality Control, and Quality Improvement. They are three unique disciplines which, when used together, can improve the efficiency and effectiveness of any organization leading to reduced cost and increased customer satisfaction.
Boast the Potential of DevOps with CI CDZoe Gilbert
DevOps CI/CD is the best practice of continuous integration, continuous delivery, and Deployment by optimizing the resources. Reading this blog, help you understand the key points of adopting the right attitude to the CI/CD approach to enable good quality software.
How to Optimize Your Software Development Process for Maximum Efficiency.pdfIT Service India
Efficiency in software development is not a one-time achievement; it's an ongoing commitment to refining your processes. By implementing clear requirements, agile practices, automation, and a focus on code quality, you can optimize your development process for maximum efficiency. Embrace a culture of continuous improvement, and your team will be better equipped to deliver high-quality software products faster than ever before.
How Can Quality Assurance Ensure Effective Application Development.Techugo
The digital revolution has entailed the building of myriad software applications working in tandem with hardware systems to deliver a better customer experience. The demands of the market not with standing, a software application development needs to stand tall on the crucible of customer satisfaction. The latter can only be achieved when a software application is validated for Quality Assurance. The importance of QA is felt all across the business since attributes/outcomes such as trust, credibility, customer satisfaction, and even ROI are determined by it.
Benefits and best practices outsource quality assuranceAlisha Henderson
Successful software development quality assurance plans start with a clear plan. Every such plan must involve well-defined standards, processes, and methods.
These must then be surely observed completely the development means. As the development project moves forward, significant audits designed to bolster compliance with these criteria must also be implemented.
Digitize your repetitive development processes to gain additional productivity and improve quality and costs.
Your Challenge
IT is increasingly under pressure to release applications quickly, reduce development, operations, and maintenance costs, while remaining relevant to business objectives.
Using a purely manual software development process can be useful when starting out and processes are not yet concrete. At a certain point, you will hit scalability limits around traceability and throughput. This will affect overall product quality.
Tests will increase over time as applications become larger and more complex, causing testing capacities to reach an upper bound slowing testing throughout. The urgency of release to market means a product may not be fully tested.
Our Advice
Critical Insight
Focus on instrumentation and integration when implementing your automated application development (AAD) tools to reduce maintenance costs, increase product quality, and improve project productivity. Instrumentation focuses on your tool’s execution, and integration refers to the movement and acceptance of development artifacts.
Don't try to automate if you don't have a good foundation based on quality and traceability first. Otherwise automation will make matters worse by amplifying gaps in your foundation. Assess your current development process for people, process, and technology gaps before deciding to implement AAD.
Impact and Result
Realize significant savings over time by reducing repetitive effort devoted to developing, testing, deploying, and maintaining applications.
Free up time to proactively optimize processes and tools or to take on innovation projects to help boost business initiatives. Innovation, a historically nice-to-have, can now be an integrated part of IT directives and mantras.
IT and the business can accurately measure the success of the development, testing, and deployment of the application to determine the need for future optimization projects to further reduce costs and improve productivity.
Why Software Testing is Crucial in Software Development_.pdfXDuce Corporation
Software testing is the process of verifying and then confirming that a software application or product
performs as expected or not. Testing has its own set of advantages like bug prevention, lower costs of
development, and comparatively better performance.
The software cannot be said to be bug-free from the start. Therefore, software developers might strive
to write code that will reduce the number and severity of flaws that are already there. However, the
majority of bugs are latent and only emerge when the conditions are right.
Defines software quality and provides detailed activities of SQA along with software change management delivered to post-graduate students of Object Oriented Software Engineering.
Metrics serve as important indicator of the efficiency and effectiveness of software process. Analysis of defined metrics helps identify area of improvement and devise subsequent actions.......Read more
Custom Software Development Cost, Process and Time.pdfJPLoft Solutions
Custom software development services provide unparalleled opportunities to realize your ideas. You get to decide how your software looks and acts; what functions it should perform. Furthermore, custom apps offer unsurpassed data security as only you can access its code; additionally, you can incorporate business expansion plans into its architecture for easier expansion of functionality over time.
In this webinar we will discuss:
- The profile of an organization that is Expert at Kubernetes on Azure and AKS
- How to get to Expert status
- The challenges along the way and how embracing Azure services can help
- A demo of deploying applications with velocity on AKS
Journey Through Four Stages of Kubernetes Deployment MaturityAltoros
In this webinar we will discuss a crawl, walk, run approach to continuous delivery (CD) for applications, point by point:
Where to start, how to advance, and how to reach the level of maximum automation.
How to orchestrate CI/CD processes along with routing and business continuity.
When the automation level is sufficient.
GitOps principles and their benefits.
What tools should be used to automate CI, CD, GitOps, Container Registry, Secrets management, etc
SGX: Improving Privacy, Security, and Trust Across Blockchain NetworksAltoros
These slides explain how to use Intel Software Garden Extensions (SGX) to improve privacy, security, trust, and transparency across blockchain networks that store sensitive data.
Using the Cloud Foundry and Kubernetes Stack as a Part of a Blockchain CI/CD ...Altoros
These slides exemplify how to employ the tools available through Cloud Foundry and Kubernetes to enable a continuous integration and continuous delivery pipeline on blockchain.
The combination of StackPointCloud with NetApp creates NetApp Kubernetes Service, the industry’s first complete Kubernetes platform for multi-cloud deployments and a complete cloud-based stack for Azure, Google Cloud, AWS, and NetApp HCI. Further, Trident is a fully supported open source project maintained by NetApp, designed from the ground up to help meet the sophisticated persistence demands of containerized applications.
With no built-in solutions for managing user accounts, Kubernetes has to rely on external systems for this. Can we use one UAA solution for both Cloud Foundry and Kubernetes authentication while building a hybrid deployment?
Troubleshooting .NET Applications on Cloud FoundryAltoros
These slides overview how logs can be employed to troubleshoot .NET app on Cloud Foundry, as well as how to use metrics to enable preventive maintenance.
Continuous Integration and Deployment with Jenkins for PCFAltoros
Jenkins has been the preferred tool for continuous integration and deployment for many years already due to it's smooth user experience, easy configuration, abundance of available plugins and integrations. During the talk we will tell about best practices on using Jenkins together with Cloud Foundry installations, accelerating cloud-native application delivery and packaging using combination of Docker and Jenkins and thoughtful configuration of CI/CD pipelines and keeping apps up-to-date on all CF environments.
At the Cloud Foundry Summit 2017 in Santa Clara, Altoros and GE Digital talked about a sensor-based solution for tracking luggage from registration to claim belt.
Navigating the Ecosystem of Pivotal Cloud Foundry TilesAltoros
For application developers, PCF tiles are arguably the easiest way to run Redis, Elasticsearch, Cassandra, or any other backing service with applications in the cloud.
Integrating AI into IoT networks is becoming a prerequisite for success in today’s data-driven digital ecosystems. The only way to keep up with IoT-generated data and gain the hidden insights it holds is using AI as the catalyst of IoT. Watch this slides to understand how IoT and AI may work together.
Over-Engineering: Causes, Symptoms, and TreatmentAltoros
If your are using Cloud Foundry, you are most obviously into the microservices architecture and cloud-native app development approach. These are definitely best practices in modern application development, but too much of a good thing is good for nothing. Overuse of these principles may lead to over-engineering, when an application is split into too much microservices and, as such, gets hard to maintain and support. This presentation highlights how far overuse of the microservices concept can go, what issues exist, and how these issues can be avoided.
SAP Sapphire 2024 - ASUG301 building better apps with SAP Fiori.pdfPeter Spielvogel
Building better applications for business users with SAP Fiori.
• What is SAP Fiori and why it matters to you
• How a better user experience drives measurable business benefits
• How to get started with SAP Fiori today
• How SAP Fiori elements accelerates application development
• How SAP Build Code includes SAP Fiori tools and other generative artificial intelligence capabilities
• How SAP Fiori paves the way for using AI in SAP apps
Accelerate your Kubernetes clusters with Varnish CachingThijs Feryn
A presentation about the usage and availability of Varnish on Kubernetes. This talk explores the capabilities of Varnish caching and shows how to use the Varnish Helm chart to deploy it to Kubernetes.
This presentation was delivered at K8SUG Singapore. See https://feryn.eu/presentations/accelerate-your-kubernetes-clusters-with-varnish-caching-k8sug-singapore-28-2024 for more details.
UiPath Test Automation using UiPath Test Suite series, part 3DianaGray10
Welcome to UiPath Test Automation using UiPath Test Suite series part 3. In this session, we will cover desktop automation along with UI automation.
Topics covered:
UI automation Introduction,
UI automation Sample
Desktop automation flow
Pradeep Chinnala, Senior Consultant Automation Developer @WonderBotz and UiPath MVP
Deepak Rai, Automation Practice Lead, Boundaryless Group and UiPath MVP
GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using Deplo...James Anderson
Effective Application Security in Software Delivery lifecycle using Deployment Firewall and DBOM
The modern software delivery process (or the CI/CD process) includes many tools, distributed teams, open-source code, and cloud platforms. Constant focus on speed to release software to market, along with the traditional slow and manual security checks has caused gaps in continuous security as an important piece in the software supply chain. Today organizations feel more susceptible to external and internal cyber threats due to the vast attack surface in their applications supply chain and the lack of end-to-end governance and risk management.
The software team must secure its software delivery process to avoid vulnerability and security breaches. This needs to be achieved with existing tool chains and without extensive rework of the delivery processes. This talk will present strategies and techniques for providing visibility into the true risk of the existing vulnerabilities, preventing the introduction of security issues in the software, resolving vulnerabilities in production environments quickly, and capturing the deployment bill of materials (DBOM).
Speakers:
Bob Boule
Robert Boule is a technology enthusiast with PASSION for technology and making things work along with a knack for helping others understand how things work. He comes with around 20 years of solution engineering experience in application security, software continuous delivery, and SaaS platforms. He is known for his dynamic presentations in CI/CD and application security integrated in software delivery lifecycle.
Gopinath Rebala
Gopinath Rebala is the CTO of OpsMx, where he has overall responsibility for the machine learning and data processing architectures for Secure Software Delivery. Gopi also has a strong connection with our customers, leading design and architecture for strategic implementations. Gopi is a frequent speaker and well-known leader in continuous delivery and integrating security into software delivery.
PHP Frameworks: I want to break free (IPC Berlin 2024)Ralf Eggert
In this presentation, we examine the challenges and limitations of relying too heavily on PHP frameworks in web development. We discuss the history of PHP and its frameworks to understand how this dependence has evolved. The focus will be on providing concrete tips and strategies to reduce reliance on these frameworks, based on real-world examples and practical considerations. The goal is to equip developers with the skills and knowledge to create more flexible and future-proof web applications. We'll explore the importance of maintaining autonomy in a rapidly changing tech landscape and how to make informed decisions in PHP development.
This talk is aimed at encouraging a more independent approach to using PHP frameworks, moving towards a more flexible and future-proof approach to PHP development.
Securing your Kubernetes cluster_ a step-by-step guide to success !KatiaHIMEUR1
Today, after several years of existence, an extremely active community and an ultra-dynamic ecosystem, Kubernetes has established itself as the de facto standard in container orchestration. Thanks to a wide range of managed services, it has never been so easy to set up a ready-to-use Kubernetes cluster.
However, this ease of use means that the subject of security in Kubernetes is often left for later, or even neglected. This exposes companies to significant risks.
In this talk, I'll show you step-by-step how to secure your Kubernetes cluster for greater peace of mind and reliability.
Welocme to ViralQR, your best QR code generator.ViralQR
Welcome to ViralQR, your best QR code generator available on the market!
At ViralQR, we design static and dynamic QR codes. Our mission is to make business operations easier and customer engagement more powerful through the use of QR technology. Be it a small-scale business or a huge enterprise, our easy-to-use platform provides multiple choices that can be tailored according to your company's branding and marketing strategies.
Our Vision
We are here to make the process of creating QR codes easy and smooth, thus enhancing customer interaction and making business more fluid. We very strongly believe in the ability of QR codes to change the world for businesses in their interaction with customers and are set on making that technology accessible and usable far and wide.
Our Achievements
Ever since its inception, we have successfully served many clients by offering QR codes in their marketing, service delivery, and collection of feedback across various industries. Our platform has been recognized for its ease of use and amazing features, which helped a business to make QR codes.
Our Services
At ViralQR, here is a comprehensive suite of services that caters to your very needs:
Static QR Codes: Create free static QR codes. These QR codes are able to store significant information such as URLs, vCards, plain text, emails and SMS, Wi-Fi credentials, and Bitcoin addresses.
Dynamic QR codes: These also have all the advanced features but are subscription-based. They can directly link to PDF files, images, micro-landing pages, social accounts, review forms, business pages, and applications. In addition, they can be branded with CTAs, frames, patterns, colors, and logos to enhance your branding.
Pricing and Packages
Additionally, there is a 14-day free offer to ViralQR, which is an exceptional opportunity for new users to take a feel of this platform. One can easily subscribe from there and experience the full dynamic of using QR codes. The subscription plans are not only meant for business; they are priced very flexibly so that literally every business could afford to benefit from our service.
Why choose us?
ViralQR will provide services for marketing, advertising, catering, retail, and the like. The QR codes can be posted on fliers, packaging, merchandise, and banners, as well as to substitute for cash and cards in a restaurant or coffee shop. With QR codes integrated into your business, improve customer engagement and streamline operations.
Comprehensive Analytics
Subscribers of ViralQR receive detailed analytics and tracking tools in light of having a view of the core values of QR code performance. Our analytics dashboard shows aggregate views and unique views, as well as detailed information about each impression, including time, device, browser, and estimated location by city and country.
So, thank you for choosing ViralQR; we have an offer of nothing but the best in terms of QR code services to meet business diversity!
Encryption in Microsoft 365 - ExpertsLive Netherlands 2024Albert Hoitingh
In this session I delve into the encryption technology used in Microsoft 365 and Microsoft Purview. Including the concepts of Customer Key and Double Key Encryption.
DevOps and Testing slides at DASA ConnectKari Kakkonen
My and Rik Marselis slides at 30.5.2024 DASA Connect conference. We discuss about what is testing, then what is agile testing and finally what is Testing in DevOps. Finally we had lovely workshop with the participants trying to find out different ways to think about quality and testing in different parts of the DevOps infinity loop.
Dev Dives: Train smarter, not harder – active learning and UiPath LLMs for do...UiPathCommunity
💥 Speed, accuracy, and scaling – discover the superpowers of GenAI in action with UiPath Document Understanding and Communications Mining™:
See how to accelerate model training and optimize model performance with active learning
Learn about the latest enhancements to out-of-the-box document processing – with little to no training required
Get an exclusive demo of the new family of UiPath LLMs – GenAI models specialized for processing different types of documents and messages
This is a hands-on session specifically designed for automation developers and AI enthusiasts seeking to enhance their knowledge in leveraging the latest intelligent document processing capabilities offered by UiPath.
Speakers:
👨🏫 Andras Palfi, Senior Product Manager, UiPath
👩🏫 Lenka Dulovicova, Product Program Manager, UiPath
Epistemic Interaction - tuning interfaces to provide information for AI supportAlan Dix
Paper presented at SYNERGY workshop at AVI 2024, Genoa, Italy. 3rd June 2024
https://alandix.com/academic/papers/synergy2024-epistemic/
As machine learning integrates deeper into human-computer interactions, the concept of epistemic interaction emerges, aiming to refine these interactions to enhance system adaptability. This approach encourages minor, intentional adjustments in user behaviour to enrich the data available for system learning. This paper introduces epistemic interaction within the context of human-system communication, illustrating how deliberate interaction design can improve system understanding and adaptation. Through concrete examples, we demonstrate the potential of epistemic interaction to significantly advance human-computer interaction by leveraging intuitive human communication strategies to inform system design and functionality, offering a novel pathway for enriching user-system engagements.
A tale of scale & speed: How the US Navy is enabling software delivery from l...sonjaschweigert1
Rapid and secure feature delivery is a goal across every application team and every branch of the DoD. The Navy’s DevSecOps platform, Party Barge, has achieved:
- Reduction in onboarding time from 5 weeks to 1 day
- Improved developer experience and productivity through actionable findings and reduction of false positives
- Maintenance of superior security standards and inherent policy enforcement with Authorization to Operate (ATO)
Development teams can ship efficiently and ensure applications are cyber ready for Navy Authorizing Officials (AOs). In this webinar, Sigma Defense and Anchore will give attendees a look behind the scenes and demo secure pipeline automation and security artifacts that speed up application ATO and time to production.
We will cover:
- How to remove silos in DevSecOps
- How to build efficient development pipeline roles and component templates
- How to deliver security artifacts that matter for ATO’s (SBOMs, vulnerability reports, and policy evidence)
- How to streamline operations with automated policy checks on container images
Observability Concepts EVERY Developer Should Know -- DeveloperWeek Europe.pdfPaige Cruz
Monitoring and observability aren’t traditionally found in software curriculums and many of us cobble this knowledge together from whatever vendor or ecosystem we were first introduced to and whatever is a part of your current company’s observability stack.
While the dev and ops silo continues to crumble….many organizations still relegate monitoring & observability as the purview of ops, infra and SRE teams. This is a mistake - achieving a highly observable system requires collaboration up and down the stack.
I, a former op, would like to extend an invitation to all application developers to join the observability party will share these foundational concepts to build on:
The Art of the Pitch: WordPress Relationships and SalesLaura Byrne
Clients don’t know what they don’t know. What web solutions are right for them? How does WordPress come into the picture? How do you make sure you understand scope and timeline? What do you do if sometime changes?
All these questions and more will be explored as we talk about matching clients’ needs with what your agency offers without pulling teeth or pulling your hair out. Practical tips, and strategies for successful relationship building that leads to closing the deal.
The Art of the Pitch: WordPress Relationships and Sales
How To Integrate Independent QA To Shorten Development Cycles
1. Abstract: Many organizations have turned to outsourcing to cut
costs. But how can you gain a strategic advantage from an
outsourcing initiative, not just cost savings? This white paper will
provide executives and managers of software product and service
companies with a ―How To‖ guide and best practices for integrating
independent Quality Assurance and testing across the product
lifecycle to shorten software development cycles.
2.
3.
4. Technology companies are challenged to innovate faster than their competition, to deliver their applications to
market first, and to re-balance their development priorities in order to accelerate revenue generation.
Business leaders often view software quality as a luxury - something that can be sacrificed, if necessary, for more
functionality, faster development, or lower costs. However, in practice, software development organizations that have
a firm commitment to quality can actually improve development cycles, reduce costs, and add new features with
greater ease. Organizations that develop low-quality software, whether for sale or for internal use, are always looking
backward, spending time and money on fixing defects in "finished"
products. In contrast, an organization that builds-in product quality
from the beginning can be forward-looking and innovative; it can A study commissioned by the U.S.
spend its resources on pursuing new opportunities. Department of Commerce's National
Institute of Standards and Technology
According to U.S. Department of Commerce, software bugs cost
(NIST) found that software defects
the U.S. economy an estimated $59.5 billion per year. $22.2 billion
cost the U.S. economy almost $60
could be eliminated with improved testing and earlier identification
billion annually. $22.2 billion could be
of errors.
eliminated with improved testing and
earlier identification of errors.
Customer expectations of flawless execution have become par for
the course. In today’s competitive economy, no software vendor
can afford the tarnished reputation or unhappy customers that
result from preventable software | defects. But when it comes down to traditional methods for assuring software
quality, companies are finding it increasingly difficult to give up the valuable time and strategic focus that get
consumed by resource-intensive internal test efforts—a problem that grows larger as development projects advance
in both scope and complexity. It’s no surprise that many companies are turning, with increasing popularity, to
outsourced methods of software quality assurance as a way to ensure the integrity of their products, the satisfaction
of their customers, and the long-term success of their growing development initiatives—all while keeping costs down.
As a software company, you face a unique set of development challenges— more platforms and configurations to
support, more releases to sustain, more testing to be completed, and more products to integrate.
Historically, QA is managed at a discrete project level and not coordinated across the product portfolio. As a result,
there are several challenges:
Limited strategic direction for QA at an organizational level
No standardization of tools/processes/test environment
No centralized reporting or service execution
A limited view of function level progress/quality
Product-release delays due to excessive testing time
5. Lack of quality testing resources, preventing quick application delivery
Product-release delays due to inability to quickly ramp-up QA team when needed
Absence of real-time data for managers to evaluate progress of testing
You need tools and practices that accelerate the development process to maintain and extend existing code as well
as to develop and deploy new code—while tightly managing the costs associated with numerous members of your
development team spread out across multiple locations. Your bottom line is dependent on your development team’s
ability to produce quality software.
When it comes to product lifecycle management, there are several primary objectives:
Shorten time-to-market timeframes
Software companies should rapidly narrow the gap between what the customer needs and what the
software delivers by increasing functionality, adaptability, usability, and manageability.
Deliver increased product feature sets
There should be a strategic release management plan that optimizes usage of the product
engineering and QA team.
Improve product quality
Software companies may use the testing and The emerging technology leaders are
validation services of an independent team with constantly improving speed, quality,
specialized skills that stays focused on delivering and cost parameters at each step of
a zero-defect product. the product lifecycle.
Constantly increase software development
efficiencies
To succeed in the next economic cycle in the software business, companies have to increase
efficiencies and productivity, and create and consolidate software assets into reusable components
and blocks.
Increase market share and customer/partner retention byfocusing on the strategic activities
and the core business
Software companies can leverage a thoughtful and balanced offshore strategy across the entire
product lifecycle to keep key people focused on strategic activities such as product management,
marketing, sales, and customer service.
6. The Informatics Computer School's Software Engineering course-text defines Software Quality as:
Conformance to clearly stated functional and performance requirements
Conformance to clearly documented development standards
Conformance to implicit characteristics that are expected of all professionally developed software
For any organization, defining what you mean by "quality" is an important first step. Too often, software development
organizations operate with a loose, general notion of quality and
tolerate defects that most engineering disciplines would not allow.
In contrast, a solid definition of quality that all team members
On average, developers make 100 to
understand and accept promotes thoroughness and attention to
150 errors for every thousand lines of
detail. In the domain of business applications, we can best define
code they write. Even if only a small
quality in terms of the target audience: the software users.
fraction—for example 10 %—of these
Development organizations with a quality focus know that a
errors are serious, then a relatively
"quality" application must do more than simply provide correct
small application of 20 thousand lines
results without crashing. Does the application meet stakeholder
of code will have roughly 200 serious
requirements? Is it usable? Secure? Scalable? Reliable? Easily
coding errors.
maintained? Easily extended? Simple to monitor?
James Juran's definition provides a good starting point: quality is
"fitness for use." Juran goes on to say that a product is not high-quality unless it adds value for both the consumer
and the manufacturer.
Put slightly differently, quality encompasses added value plus attention to detail. Both a luxury car and an entry-level
car will get you from point A to point B. But the luxury car offers features and capabilities that go beyond the
essentials of transportation: usability, safety, comfort, reliability, and so on. Product quality also reflects the process
behind the product. In the software world, a high-quality process can keep development organizations from losing
time reworking, re-factoring, and rewriting software. These organizations can produce more innovative and creative
products because they have more time to think about adding value and quality details.
Achieving quality results requires applying a high-quality process throughout development, integration, and testing.
This applies equally to projects involving packaged or homegrown applications, upgrades or greenfield development
work, and extending, integrating, and modernizing legacy applications (see Figure 1).
7. Figure 1: All modes of software development require the same attention to quality.
Design
New Develop
Development Test
Deploy
Integrated Customize
Extend Expose
Legacy Packaged
Expose
Systems Applications Test
Upgrade Deploy
But the bottom line of software quality is conformance to the user's needs⎯or requirements,, or wants. The point is,
any definition of software quality must take into consideration the end-user's perspective. An application may do all
that it was designed to do, and do it according to professional standards, but if users hate using it, find it difficult,
counterintuitive, or error-prone, then its quality is at best questionable.
Being free of defects is not enough. For a product to achieve quality, it must also provide exceptional functionality,
usability, and compatibility.
Software Quality Assurance cannot be treated by executives, managers, or engineers as one additional "Task" on
the way to the final launch, or a thing that can be taken care of
as the project proceeds, along the development and release
process. Team members closely associated with the application
"Rigorous inspections can remove up
are less likely to find the problems that an external party would
to 90% of errors before the first test
find easy to spot and report. Independent testing efforts ensure
case is run, but rigorous inspections
that your product is tested by individuals who will be just as
should not replace testing" - Robert L.
objective and unforgiving as your customers in the case of a
Glass, "Facts and Fallacies of
potential defect.
Software Engineering"
This is simply a fact, and not a judgment of or reflection on the
developers' talents. An individual who spends the most time
working on a project intended for an audience cannot simply step back and look at it as though through that
audience's eyes. The creator is too familiar with his work to view it objectively.
8. But this is why Quality Assurance experts exist⎯just asprofessional proofreaders, copy editors, test audiences, and
software do. These independent parties bring much-needed objectivity to the appraisal of a software development
project.
It is crucial that software Quality Assurance be independent of project management for it to be completely effective.
Outsourcing software Quality Assurance to an independent, unbiased source is the most effective way to achieve the
objectivity that a project needs.
When it comes to the practice of independent verification and validation, an independent QA team should be the one
final check-off point before allowing a system to go into operation; how it is done systematically and organizationally
varies, however.
The standardization of software engineering processes, margin compression, and competition are compelling
technology-centered companies to face the task of delivering more complex application development with limited and
decreasing time frames. Software leaders are looking at outsourcing processes across the entire software
development lifecycle, from engineering, to quality assurance and testing, to maintenance and porting.
Software Quality Assurance outsourcing can take a number of paths, testing for different capabilities and performing
different analyses, depending on the demands of the project, the users, and the software itself. But an outsourced
Quality Assurance and Testing initiative carried out by seasoned QA professionals will confer certain benefits:
1. Reduced Development Cycles
2. Improved Customer Satisfaction
3. Reduced Cost of Development
4. Reduced Cost of Maintenance
Coupled with an outsourced business model and its benefits, outsourcing software QA and testing became a
strategic R&D initiative for many technology companies. The expected benefits include:
5. Improved focus of in-house resources on strategic activities and innovation
6. Reduced Total Cost of Ownership (TCO)
7. Reduced technology risks
According to Gartner (2005), “20% of
So what is the best way to leverage QA and testing to shorten
outsourcing deals do not produce cost
development cycles? Build your own QA and testing department?
savings, with 50% of all outsourcing
Create a hybrid of in-house and outsourced resources? Or
projects falling short of delivering the
outsource entirely?
expected value and being deemed
unsuccessful".
As a software vendor, innovation is the foundation of your
success. The earlier in your software development process that
you apply world-class testing and quality assurance practices, the
more efficient your process will be. Advancing other critical path activities, such as internationalization and
9. localization, further upstream in the product lifecycle will also serve to improve efficiency. Software makers who
manage their product lifecycles as carefully as the process-mature manufacturers in, say, the automotive industry,
are sure to be winners as the high-tech industry moves into the next economic cycle.
Outsourcing the testing activity, properly applied, can address many of these issues while relieving management of
many of the burdens of process development, staffing, and ongoing control.
The reasons for outsourcing are many and varied among potential customers. Depending on the nature and maturity
of the company, these could range from the need for limited special purpose testing to taking over quality
management of a product/application portfolio. The former might be done to address specific gaps in the testing
framework, such as a lack of environments for conducting load testing, or areas of particular risk such as security.
The latter may be advisable if your internal resources are unable or have limited capabilities to justify, explain, or
quantify existing internal testing activities. As with any business decision, the choice to outsource and/or offshore
software testing must be viewed in terms of risk and reward. While this will vary from company to company, a few
basic principles should be applied when considering such a solution.
The first step should be to articulate the goals of the initiative. The basic rationale for undertaking this should be
stated explicitly and frame the entire process. A formal strategy statement may also be valuable here to lay out a
roadmap for proceeding, articulating critical risks and assumptions, estimating resource requirements, and
determining measures of success.
Some of the possible reasons for deciding to outsource include:
"...the average enterprise will
1. Deficiencies in the specialized skills and ultimately outsource 60% of
processes required to support testing application work offshore (circa
2008/09)."— META Group
2. Problems related to poor test practices such as
excessive expenditures on maintenance and
critical failures in production
3. Lack of test infrastructure including test environments and test beds
4. Insufficient testing activity to warrant a full-time testing staff because the portfolio is too small or
there are only a limited number of releases during a year
5. Lack of facilities or expertise to conduct special purpose testing
6. Independent validation and verification of development work performed by other vendors
One other area to note where an organization might gain significant long-term benefits from outsourcing is in
implementing test automation. Manual testing and tracking is a complicated enough task and, although using
automated test tools can help improve efficiency and control, they are not an immediate panacea. Such tools must
be carefully selected to match the technology and environments used by a system. The tools must be set up, scripts
should be written and maintained by experienced professionals who know how to configure the package, set up
databases, design test artifacts, determine and enforce privileges, establish library control procedures, train staff,
and enforce standards. By engaging a service provider who knows how to set up and manage these tools, a
10. company can gain valuable experience in using them without some of the pain associated with starting up such an
initiative internally.
The final aspect of making a decision about whether or not to outsource Quality Assurance and testing is to
determine the organization’s readiness to support testing. An evaluation of existing processes, skill sets, and
infrastructure is recommended in order to set the stage for planning a test outsourcing initiative. An important part of
any such assessment is how the existing QA or testing organization will interact with the vendor.
The chapter entitled, ―Five Phases of Integrating QA Into the Entire Global Software-development Cycle" provides
step-by-step guidelines for how to design and manage your Quality Assurance and testing program.
Based on the goals articulated by management, a formal framework for moving forward should be developed. Some
potential models for outsourcing that might be considered include:
1. Total off-site outsourcing of all testing functions for the entire portfolio including offloading the
existing testing staff to the vendor.
2. Conducting outsourced testing on a pilot basis with the intention of developing internal processes or
introducing automation.
3. Partial outsourcing of special purpose testing (e.g. performance or security) or testing level (e.g.
outsourcing system integration testing but not unit testing or user acceptance testing).
4. A vendor-managed internal testing program in which all work takes place on site, applications
remain in the customer environment, test beds are kept in repositories controlled by the client, but
the test management and staffing is provided by the vendor.
5. Third party validation of outsourced products, involving independent testing of applications built by
another vendor.
Total off-site outsourcing has the advantage of reducing internal staff expenditures while improving the efficiency and
effectiveness of the testing function. It leverages the knowledge of the vendor to the largest extent. This model tends
to work best for customers who have well-documented requirements and well-educated stakeholders. In the absence
of those conditions, there may be a ramp-up period in which the vendor will have to spend time organizing
requirements and familiarizing internal staff with their roles. In addition, the relationship between the vendor and
developers must be managed carefully; delays in the delivery of software will affect testing schedules and the
amount of testing, which can be performed.
The final consideration in a pure outsourcing arrangement is the fact that the product will be under the control of the
vendor during testing. Arrangements must be made to assure the security and integrity of programs and data during
this time. A risk worth mentioning with a pure outsourcing approach is that the client can become out of touch with
the quality management of its own products. It is incumbent on the customer to review, interpret, and act on test
artifacts and test results even if the entire testing operation is run by an off-site test team.
11. The major benefits of running an outsourcing pilot are that targets and costs can be controlled selectively and future
strategies can be modified based on the outcomes and findings from the pilot. When choosing pilot applications, it is
usually wise to find those of enough significance and size to make the activity meaningful but not those which are
business-critical. This allows clients to learn important lessons about testing applications in their domains without
putting core systems at risk. Establishing predetermined criteria for the success of the pilot is vital to extracting the
proper lessons from it.
Partial outsourcing of special purpose tests or by testing level can be an extremely effective way for an organization
with existing test infrastructure to enhance its quality management
program.
By engaging vendors with extensive
A vendor-managed internal testing program is a hybrid business
experience in a particular specialty,
model, which can serve multiple needs. In this scenario, all work
such as performance testing, a
takes place on-site and applications remain in the customer
company can leverage its knowledge
environment. There is greater control and much less risk of
and environments, reduce investment
compromising systems or data. The vendor may design and
in hard to find skills, minimize the
create testing procedures or use existing processes, but they can
need for expanding test infrastructure,
bring additional value to the organization by working with internal
and receive independent verification of
best practices and enhancing them. The vendor may also develop
results.
processes for controlling test beds, however it is recommended
that the repositories are designed and managed by the client’s
infrastructure team. Since the test management and staffing is
provided by the vendor, the customer does not have to invest in finding the skills needed to establish the test
infrastructure and staff the test teams. One other approach that is gaining popularity is third-party validation of
applications built by another vendor. This trend is pronounced with mid-market clients who have decided to
outsource most of their IT function. It can be a cost-effective option for organizations with little experience with
software development and testing or for those who have very few or small software projects. While it does not
contribute to the client developing its own testing processes, it is a good model for those who do not wish to invest in
that infrastructure. In addition, the test cases and test data are the property of the customer and can be reused in the
future should the outsourcing relationship be discontinued.
One issue that may arise when adopting this approach is contention among vendors. Performance of the testers can
be affected negatively by developers who do not provide good test requirements, cannot provide feedback on tests
in development, or fail to deliver on time. The client in this situation must maintain control of the process and must
take steps to assure that the vendors are working in a manner that supports the company’s goals.
At the end of the day, how does an executive determine the value of outsourced testing and whether it is worthwhile
to undertake? Potential buyers should realize that the benefits to be derived from test outsourcing may go beyond
those gained through outsourcing other functions.
Those who have survived in the tough conditions of the recent IT economy are well aware of quality issues and the
need to deliver fully tested products. The financial motivation to find and fix bugs before product release is not a new
notion, however. A study by noted industry analyst Capers Jones back in 1996 clearly showed the staggering costs
12. associated with post-release bug detection. Post-release bug detection usually occurs when software companies
decide to cut costs by reducing testing schedules in early phases of development.
Figure 2. Cost of repairing defects across application lifecycle
The fact that closer attention to testing is vital to keeping overall long-term costs down is often obscured by near-
term budget constraints. Here we run into a paradox: more testing resources and enhanced QA requirements are
paramount to a company's success, but their budgets simply cannot support increased spending.
There is a solution to this seeming contradiction. As Steven Brier, senior analyst and managing director of research
at Roberson Stephens, notes, the secret to success in the IT landscape these days is to focus on running the core
business while effectively using outsourcing options. "It's often cheaper and faster to give it to someone else to do,"
says Brier {UPSIDE, June 2001).
In terms of quality assurance, this means that developers can stay within their core competencies by unit testing with
highly specialized debugging tools, such as jtest® from ParaSoft®, while outsourcing functionality, compatibility, and
integration testing. This allows software engineering organizations to capitalize on what they do best, i.e. write code,
while delegating the task of testing how their product functions in the complete solution stack to an outside team with
proven expertise in quality assurance.
Additionally, in many companies, because of repetitious test processes, the turnover rate for test engineers is very
high. This means that an in-house test lab must continuously spend time and money to seek and recruit skilled
workers and train them for its purposes. In comparison, outsource test organizations offer clearly defined career
13. paths to their employees, reducing turnover. Plus, due to the nature of the business, projects in outsourced testing
labs are always in flux, which keeps career test engineers interested in working with and discovering upcoming
technologies. As a result, these engineers are highly trained, enthusiastic, possess a wide variety of core
competencies, and are efficient in carrying out complex test projects.
It is important to understand that the savings realized through more effective testing don’t automatically fall to the
bottom line. Even if an outsourcing arrangement does directly reduce the cost of testing, most IT organizations will
probably still spend their up to their budget limits. The savings can be and usually are spent on development and
other productive tasks in support of core business functions. But some of the payback of instituting an effective
testing program is derived in other ways, including:
More efficient staff and resource management
Ability to investigate test automation in a low-risk setting
Reduction of time spent on rework and repair
Minimizing and managing product risk
By enabling the software engineering department to make more effective use of the internal resources, both testing
and development benefit. Developers can focus on the job of developing the software without worrying about
planning, designing, and executing tests. This can shorten lead times to get products to market and result in better
systems because developers will be focused on just one task - engineering - which has to be the only core
competency. Dedicated, trained testing resources who understand the overall framework of quality management will
be much more effective in constructing and implementing an organized approach to validating software. The testing
process will be more efficient and effective. First of all, highly trained specialists will be better at this task.
Effectiveness will also be enhanced since an outsourcer is more likely to provide an unbiased assessment of product
quality simply because they are operating independently from the developers. Finally, the costly and time-consuming
task of setting up and managing test environments will no longer be a factor in determining resource allocations. A
good provider of testing services should either have the appropriate hardware and software infrastructure in place or
be able to generate it at less cost and in less time.
The management of special-purpose testing will also be enhanced. Performance and stress testing can be done with
more breadth and depth. Regression testing can also be enhanced. The key to effective regression testing is to
maintain existing test scripts so that they can be re-run when changes are made. However, most companies don’t
have experience in determining which scripts to keep, how to store them in an organized fashion or when to re-run
them. A qualified service provider will have these skills on staff or make them available, when needed.
Outsourcing to a service provider who has extensive experience with automated test tools also provides a buyer the
benefits of working with a partner knowledgeable in this field. They can assist with defining the tool set to be used,
installing and configuring the product, managing it on an ongoing basis, and training staff in its use. Automated tools
can also be used to improve regression testing, as they provide facilities for storing and managing test scripts and
test data. When one considers that a significant proportion of test tools purchased are never used effectively by the
buyers due to lack of experience, the return on investment is often gained simply by using the very products they
purchase. Then there are the benefits of better product quality. Put simply, it is advantageous and economical in the
long run to test software systems effectively to assure that they meet their requirements rather than correcting them
later.
14. The amount of time and money spent by software engineering departments on application ―maintenance‖ can
exceed 50% of the annual budget. This calculation also does not take into account the overall cost of poor quality
which includes the time and resources required to identify defects, determine their location and cause, repair them,
retest them, and promote them again into the production environment. Finally, there is what may be the largest cost
of poor quality: business lost and the company’s reputation tainted by exposing end-users and customers to
systems, which cannot perform their functions accurately and dependably. This could result in inaccurate order
processing and billing, unexecuted transactions, miscalculations, inability of customers to do business during high
volume periods, generation of inaccurate reports, etc. Any one of these circumstances can cause customers to
discontinue their relationship with a company, and they may very well convince their acquaintances to do so as well.
The last major benefit to be derived from working with a vendor to establish a sound testing program is better
information to support management decision-making. Prioritizing requirements provides a sound basis for evaluating
product risk and determining areas for concentrating quality management or improvement programs. Measurements
of test completion and coverage represent necessary information for determining if and when products go live. Root
cause analysis of incident reports can lead to significant improvements in the product definition and development
process. A vendor can make recommendations for making the development effort more efficient and improving the
quality of its deliverables. It is this benefit, which is not immediately quantifiable in monetary terms, which can
contribute to the long-term success of the enterprise by helping senior management assess risk, set priorities,
establish product improvement programs, and generally provide a real-world basis for evaluating investments in
technology.
The bottom line? In some cases, development organizations prefer having their own test departments for reasons
that go beyond financial considerations. For one, they believe close coordination between test and development
resources requires proximity. In these instances, a hybrid solution of outsourced test specialists working on-site,
backed by external lab-based testing, may be the optimal solution. In either case, today's budget-minded and high
velocity organizations avoid missed time-to-market opportunities and enjoy lower over¬all costs of quality assurance
through testing outsourcing.
Whether you are a software product company and sell software or develop software to use internally, it is important
to begin with the discovery of business opportunities across the application lifecycle.
Choosing the testing processes that software companies want to outsource is a critical component of their
outsourcing initiatives and can have a tremendous impact on their success. The stakeholders should take the current
inventory of development and testing processes, resources, and tools before they start identifying objectives,
expectations, and metrics to measure the success of the company.
15. To forge a successful outsourcing strategy, companies should conduct an outsourcing ―readiness‖ assessment. The
assessment should be equally applied to companies considering outsourcing processes across the entire product
lifecycle, or to those who wish to outsource QA and testing only.
The results of the assessment will provide an organization with much of the information required to make key
decisions and to focus management effort. The assessment can help examine the ability of a software development
organization to outsource by looking at the processes maturity, experience, tools, and ROI objectives.
Data for assessment may be gathered from:
1. Questionnaires
2. Interviews
3. Discussion with product managers and application owners
4. Study of product roadmaps, test plans, test coverage, test scripts inventory, and application/code
base inventory
Deliverables may include:
1. Outsourcing Readiness Report
2. ROI analysis
3. Application/Project Sequencing Plan
4. Engagement Roadmap
When reviewing test plans, test coverage, and test scripts inventory, the following criteria are recommended to
assess readiness of specific processes:
1. Skill alignment with current/future quality assurance and testing direction
2. Current cost of testing/quality assurance
3. Application maturity
4. Alignment to business (core, context, utility)
5. Business value
6. Functional completeness
7. Technical completeness
8. Rate of change
9. Test coverage
10. Perceived risk profile
16. Successful completion of this phase will provide the company with insight to areas of the software quality assurance
and engineering, which have room for improvement. Gap analysis naturally flows from the results of the
Assessment/Business Case phase and involves determining, documenting, and approving the variance between
business objectives and current capabilities.
At this stage, companies should analyze gathered materials with the goal of comparing its actual performance with
its potential performance, and of finding gaps that represent opportunities for improvement.
The objective is to study the differences between the current and desired state/best practices of the quality
assurance and testing processes, for the purpose of determining how to get from one state to a new state. Gap
analysis is undertaken as a means of bridging that space. Among the various methodologies used to perform gap
analysis is IDEF, a group of methods used to create a model of a system, analyze the model, create a model of a
desired version of the system, and to aid in the transition from one to the other.
With that information in hand, a company can pinpoint and address the quality/process gaps. The ultimate goal is to
identify and compare a list of initiatives, which you can begin implementing to improve the speed, quality, and cost
parameters at each step of the product lifecycle.
In this step, companies define a high-level project plan and a test strategy for reaching the desired QA destination.
The following steps outline the major components of a typical outsourced QA project.
1. Overview –Includes the objectives for QA on this project and background information.
2. Scope – Contains what will be covered and what will not be covered in the QA effort.
3. Assumptions – Includes any assumptions made in developing the QA Strategy.
4. Project Organization – Contains the project organization chart and how the QA Team fits within.
Roles and responsibilities respective to QA and other stakeholders. Specify Project Team
responsibilities for test infrastructures and data.
5. Project Steps – Address how the QA effort will be approached. Include findings from Proof of
Concept. Include one or more schematics, along with written descriptions of the proposed QA
workflow. Describe types of data and validation methods.
6. Levels of Testing – Define what types of testing will be performed such as Unit, Integration,
System, Data Validation, Performance, GUI and the entrance and exit criteria for each.
7. Testing Groups - Testing groups provide a breakdown of major functionality. List the testing groups
and clearly state what they include and what they do not include.
8. Technical Assessment – Content depends on current status of development, technical
infrastructure, and type of technologies and platforms. In this section, the following should be done:
Assess compatibility of data structures along with any code or software specific issues. Address
feasibility of test automation with recommendations for using home-grown or vendor tools at certain
17. stages of the development cycle. Include recommendations regarding what should be automated for
testing.
9. Test Infrastructure Requirements - Assess existing infrastructure processes and needs. Discuss
test infrastructure considerations and requirements for loading converted data and ongoing data load
processes. Include environment sizing considerations and depth of test data. Clearly state how the
test infrastructure is and is not analogous to the production infrastructure. Note any exceptions or
limitations.
10. Processes and Tools – Describe processes and tools to be used for change control, configuration
management, and discrepancy reporting. Describe what test automation will or may be used and
how it will or could be performed.
11. High Level QA Project Plan - Schedule with tasks and associated resources for successfully
completing testing.
After implementation of the project, companies have to return to their initial assessment and gap analysis to compare
the results of the project with the initial expectations.
18. What to Automate Quality of Software
What Tool to Use Impact of Failure Nature of System
Manual Tools Likelihood of Failure Strategic Significance
Test Tools Risks Application
Test Test
Objectives Infrastructure
Test
Strategy
Demonstrate Usability Code Control
Intrfaces Enviroments/Progression
Coverage Hardware
Stress/Load Software
Review Test
Resources
Strategy Requirements
Money Walkthroughs Completeness
Time Inspections Accuracy
People Desk Checks Stability
Skills
Priorities
Hardware
Software
19. During the execution phase the following metrics are used:
1. Planned test execution versus actual: are we on track? (could identify a risk for being on time)
2. Number of successful versus not successful tests (too low means inefficient testing due to low
quality of the test object)
3. Trend in outstanding number of defects (going up could mean a risk for being on time)
4. Burn rate of the test hours per phase (too high could identify a budget issue)
The metrics #1, #2, and #3 are essential to the success of an outsourced QA process and should be tracked weekly.
These metrics are mandatory for Quality Assurance projects with a duration of the execution phase longer than two
weeks.
By applying criteria to the metrics during a running test project, important signals can be derived as to whether the
outsourced testing project is going well and whether the quality of the test object is reasonable.
Towards completion of the project it is advisable to add the following metrics:
1. Has the test coverage reached sufficient level?
2. Is the percentage of successfully executed tests high enough?
3. Are the remaining defects below a certain level?
These metrics play an important role in the acceptance process.
When the execution phase is finished some additional metrics can be determined:
1. Number of hours spent on test execution per test case
2. Number of hours spent on test execution per defect found
3. Number of hours spent on testing as percentage of the whole project
When these metrics are collected for a number of comparable projects, they play an important role for supplier
management purposes (how efficient and effective is a test supplier) and in general to drive improvements.
When the project is over, it is recommended that companies analyze defects found after the deliverables were
accepted:
20. 1. Number of defects found after acceptance compared to before acceptance
2. Test specification progress per week (number of test cases written) versus planned
3. Defect solution turn around time (are all showstoppers solved within a week?)
4. Defect root cause analysis in all phases (are there particular weak spots in build?)
The following mistakes are the most common when it comes to outsourcing Quality Assurance and testing activities:
1. No ownership of product quality.
2. No overall test program design or goals.
3. Non-existent or ill-defined test plans and cases.
4. Testing that focuses narrowly on functional cases.
5. NO ad hoc, stress, or boundary testing.
6. Use of inconsistent or incorrect testing methodology.
7. Relying on inexperienced testers.
8. Improper use of tools and automation, resulting in lost time and reduced ROI.
9. No meaningful metrics for tracking bugs and driving quality back into development.
10. Incomplete regression cycle before software release.
Future State, and Required Roadmap
Software vendors often make decisions about what and how to outsource too quickly, without detailed expert
assessment of their current state, desired future state, and required path on how to get there. It is critical to plan
outsourcing strategy by considering a company’s current software engineering processes maturity, in-house team
size, available documentation, and experience with distributed software development, cultural fit, and executive and
staff support. It is recommended to access applications, source code base, and business processes, and to lay out a
matrix of how and in what sequence applications and processes can and/or should be outsourced, kept in-house, or
retired.
21. The cost expectations that determine the savings that organization can leverage as a result of outsourcing are often
over-estimated. Although they can be based on ROI projections, software vendors should consider the learning
curve for the offsite team to become productive. For several years now, the business press has been claiming that IT
work costing $40-$80 an hour in the United States can be done for $15-$25 an hour in India or Russia. If those
figures sound too good to be true, that’s because they usually are. An offsite team often will not become productive
(by onshore standards) for at least three to four months, or even longer for more complex projects. IT executives
should expect to pay an additional 5 percent to 15 percent on managing an offsite outsourcing program, at least
during the first year. The transition phase will add costs, too — planned expenses should cover due diligence,
transfer of knowledge, and extensive program management.
Once a company has decided to outsource, it usually rushes to get the outsourcing deal done and put the
transaction on a fast track. To do that, executives from the company and the outsourcer meet, agree, and then let
the next level of management sort out all the details. Since an outsourcing process is a complicated one, it is
advisable that various elements of a company remain involved in the original meetings and have a say in the
agreement. Top-level executives alone are rarely sufficient to review such a process effectively.
The essence of outsourcing is that a company transfers the process of application engineering to the outsourcer, and
then buys the results of that process. The quickest and most certain way to destroy an outsourcing process is to
dictate to the outsourcer how the process should be done. When a company tells the outsourcer what to do, it
removes from the outsourcer the ability to add value to the process. As the result, the outsourcer cannot deliver
quality software.
Many companies choose not to transfer the ownership of the outsourcing process to the outsourcer, and instead opt
to keep the process under company control. When the company assigns someone to oversee the process, this
person usually becomes overwhelmed and is unable to manage it, because it is an area that he or she has very little
knowledge about. As a result the outsourcing process fails.
The longer your contract terms, the more deep you get in it. Information-driven enterprises and ASPs should
construct a contract that will last a long time, but that will allow the service term's agreements to change during short
periods of time. It is preferable for a company to develop a close relationship with an outsourcing company, but to
sign short-term contracts that can be easily negotiated and, if necessary, renegotiated or even broken.
22. Companies new to the outsourcing process often do not assign the right people to manage the process. They often
assume that since all the details of an outsourcing contract are discussed and the contract is signed, that they do not
have to worry about the outsourcing process any longer. The right person for the job is someone who understands
the process, is able to focus on soley on the process, and who can interpret the results with the best interest of the
company in mind. He or she should have a different way of thinking, in respect to most business managers, and the
oversight to recognize any problems or mishaps before they surface.
Far too often, the company forgets that the outsourcer is a business asset and must be treated as such. Team
members across the globe are vital parts of a successful and prosperous business. If the outsourcer is treated
differently— for example, like one of many suppliers or one of many customers—then the one thing that the
outsourcer adds to a company will be lost.
It is preferred that companies inspect in person the physical premises where the software is to be
developed. This is an opportunity to check the security firewalls of their vendor’s buildings and work
area, the organization culture, the functioning of their networks, etc.
It is preferable that companies interview the team members who will work on their project. This will
help companies to judge the employees' level of reliability.
Companies should also check the offshore development company’s employee retention rate, and
also determine if the outsourcing vendor is working with any competing organizations. If this is the
case, companies should ensure that the teams working on their competitors’ projects do not have
access to their project information.
Any method of information transfer, such as e-mails, fax, electronic file exchange, instant
messenger, on-line meetings, paper documentation etc, should have parameters for usage clearly
defined.
All activities that will have to accompany the end/termination of the contract should be defined while
negotiating the contract terms and conditions. This includes the retrieval of any methods and
procedures, documents, source and executable code, company proprietary security or development
standards, code libraries, and data stored offsite.
Contracts should be framed, such that the offshore company takes responsibility for the actions of its
employees.
23. Companies should ensure that any project-related work is not subcontracted without approval. This
will help to protect a sensitive application or data within the application.
Companies should ensure that the vendor agrees not to use any of the company’s confidential
information for purposes like sales, marketing, or demo without prior approval.
Companies should ensure that only data related to the performance and reliability of the system is
transmitted over the Internet. Information from the database should not be disturbed during any part
of the project.
Companies should make sure that the system experts make modifications to the system only after
obtaining prior permission.
Exchange of passwords and other critical information should be made secure by encrypting them.
Companies should ensure that the data used during testing does not expose the real information of
the customers.
Unwanted data should be destroyed.
Companies should ensure that the vendor reports on replacements of team members, if any.
Companies should ensure that an original copy of the source code is maintained.
The business benefits of including quality-oriented activities in all phases of your software development lifecycle are
both broad and deep. These measures not only facilitate innovation and lower costs by increasing predictability,
reducing risk, and eliminating rework, but they can also help to differentiate your business from its competitors. Most
importantly, continuously ensuring quality will always cost less than ignoring quality considerations.
Renat Khasanshyn is CEO of Altoros Systems, LLC. www.altoros.com
24. 1. James M. Juran and Frank Gryna, Juran's Quality Control Handbook. McGraw-Hill, 1988
2. Software Quality Engineering: Testing, Quality Assurance, and Quantifiable Improvement, Jeff Tian
(Wiley-IEEE Computer Society Press, 2005)
3. Managing Successful IT Outsourcing Relationships, Petter Gottschalk and Hans Solli-saether (Jul
2005)
4. "Software Quality Assurance: Principles And Practice" by Nina S. Godbole Alpha Science
International, Ltd (2004)
5. Test Outsourcing: Risks and Rewards, 2006 Rich Yannetti 545 Fifth Avenue 4th Floor, Ste 401 New
York, NY 10017
6. Metrics for Outsourced Test Activities, Kees Blokland / Polteq IT Services B.V. (NL) 2004
7. Data Warehousing QA Challenges, Denise Copening, Olenick & Associates
8. http://en.wikipedia.org/wiki/Gap_analysis
9. Software Quality Assurance Improvement Plan: EPIcode Gap Analysis; May 2004, U.S. Department
of Energy
10. Facts and Fallacies of Software Engineering; Robert L. Glass; Addison-Wesley (2002)
11. See Watts S. Humphrey's article entitled "The Quality Attitude," published in the Software
Engineering Institute (SEI ) newsletter, (2004, no.3): http://www.sei.cmu.edu/news-at-
sei/columns/watts_new/watts-new.htm