In the software development lifecycle, application testing is crucial - but often given short shrift by companies allocating temporary resources. Test environment management services (TEMS) is a perfect solution, reducing testing costs and errors while conducting test/QA environment management, monitoring and maintenance and cloud infrastructure provisioning.
Key performance indicators to determine that an organisation has an effective, efficient, result oriented TEM function or is close to having one, based on best practise and ITIL/ISO standards.
Test Environment Management: A Critical Requirement for Effective CI/CDDevOps.com
est Environment (TEM) enables the efficient configuration, allocation, reporting, and management
of test environments.
Attend this webinar to get results and insights from the latest test environment research by Enterprise Management Associates (EMA) and the role that test environment management is playing in implementing effective CI/CD. The survey gathered insights and usage details from 160 of the largest North American enterprises. You'll hear Steve Hendrick, Research Director at EMA and Jeff Keyes, Director of Marketing at Plutora discuss the results and dive into the strategies, objectives, and experiences of large enterprises in using TEM tools. This research will show how proactive management of preproduction environments accelerates application development while generating significant cost savings across DevOps activities, resources, and staff. The result will be a roadmap for enterprises showing how best to leverage TEM technology.
This webinar summarizes the research findings into these key areas:
Test environment management strategies, priorities and maturity
Key functionality and top vendors providing capabilities
Real-world benefits with quantifiable results
Test Environment Management (TEM) is a function in the software delivery process which aids the software testing cycle by providing a validated, stable and usable test-environment to execute the test scenarios or replicate bugs.
This presentation about DevOps will help you understand what is DevOps, how is DevOps different from traditional IT, benefits of DevOps, the lifecycle of DevOps and tools used in DevOps processes. DevOps is one of the most trending IT jobs. It is a collaboration between development and operation teams which enables continuous delivery of applications and services to our end users. However, if you want to become a DevOps engineer, you must have knowledge of various DevOps tools (like Git, Maven, Selenium, Jenkins, Docker, Ansible, Nagios etc.) to achieve automation at each stage which helps in gaining Continuous Development, Continuous Integration, Continuous Testing and Continuous Monitoring in order to deliver a quality product to the client at a very fast pace. Now, let us get started and understand DevOps and does the various DevOps tools work.
Below are the topics explained in this DevOps presentation:
1. What is DevOps?
2. Benefits of DevOps
3. Lifecycle of DevOps
4. Tools in DevOps
Why learn DevOps?
Simplilearn’s DevOps training course is designed to help you become a DevOps practitioner and apply the latest in DevOps methodology to automate your software development lifecycle right out of the class. You will master configuration management; continuous integration deployment, delivery, and monitoring using DevOps tools such as Git, Docker, Jenkins, Puppet, and Nagios in a practical, hands-on and interactive approach. The DevOps training course focuses heavily on the use of Docker containers, a technology that is revolutionizing the way apps are deployed in the cloud today and is a critical skillset to master in the cloud age.
After completing the DevOps training course you will achieve hands-on expertise in various aspects of the DevOps delivery model. The practical learning outcomes of this Devops training course are:
An understanding of DevOps and the modern DevOps toolsets
The ability to automate all aspects of a modern code delivery and deployment pipeline using:
1. Source code management tools
2. Build tools
3. Test automation tools
4. Containerization through Docker
5. Configuration management tools
6. Monitoring tools
Who should take this course?
DevOps career opportunities are thriving worldwide. DevOps was featured as one of the 11 best jobs in America for 2017, according to CBS News, and data from Payscale.com shows that DevOps Managers earn as much as $122,234 per year, with DevOps engineers making as much as $151,461. DevOps jobs are the third-highest tech role ranked by employer demand on Indeed.com but have the second-highest talent deficit.
1. This DevOps training course will be of benefit the following professional roles:
2. Software Developers
3. Technical Project Managers
4. Architects
5. Operations Support
6. Deployment engineers
7. IT managers
8. Development managers
Learn more at https://www.simplilearn.com/cloud-computing/devops-practitioner-certification-training
Key performance indicators to determine that an organisation has an effective, efficient, result oriented TEM function or is close to having one, based on best practise and ITIL/ISO standards.
Test Environment Management: A Critical Requirement for Effective CI/CDDevOps.com
est Environment (TEM) enables the efficient configuration, allocation, reporting, and management
of test environments.
Attend this webinar to get results and insights from the latest test environment research by Enterprise Management Associates (EMA) and the role that test environment management is playing in implementing effective CI/CD. The survey gathered insights and usage details from 160 of the largest North American enterprises. You'll hear Steve Hendrick, Research Director at EMA and Jeff Keyes, Director of Marketing at Plutora discuss the results and dive into the strategies, objectives, and experiences of large enterprises in using TEM tools. This research will show how proactive management of preproduction environments accelerates application development while generating significant cost savings across DevOps activities, resources, and staff. The result will be a roadmap for enterprises showing how best to leverage TEM technology.
This webinar summarizes the research findings into these key areas:
Test environment management strategies, priorities and maturity
Key functionality and top vendors providing capabilities
Real-world benefits with quantifiable results
Test Environment Management (TEM) is a function in the software delivery process which aids the software testing cycle by providing a validated, stable and usable test-environment to execute the test scenarios or replicate bugs.
This presentation about DevOps will help you understand what is DevOps, how is DevOps different from traditional IT, benefits of DevOps, the lifecycle of DevOps and tools used in DevOps processes. DevOps is one of the most trending IT jobs. It is a collaboration between development and operation teams which enables continuous delivery of applications and services to our end users. However, if you want to become a DevOps engineer, you must have knowledge of various DevOps tools (like Git, Maven, Selenium, Jenkins, Docker, Ansible, Nagios etc.) to achieve automation at each stage which helps in gaining Continuous Development, Continuous Integration, Continuous Testing and Continuous Monitoring in order to deliver a quality product to the client at a very fast pace. Now, let us get started and understand DevOps and does the various DevOps tools work.
Below are the topics explained in this DevOps presentation:
1. What is DevOps?
2. Benefits of DevOps
3. Lifecycle of DevOps
4. Tools in DevOps
Why learn DevOps?
Simplilearn’s DevOps training course is designed to help you become a DevOps practitioner and apply the latest in DevOps methodology to automate your software development lifecycle right out of the class. You will master configuration management; continuous integration deployment, delivery, and monitoring using DevOps tools such as Git, Docker, Jenkins, Puppet, and Nagios in a practical, hands-on and interactive approach. The DevOps training course focuses heavily on the use of Docker containers, a technology that is revolutionizing the way apps are deployed in the cloud today and is a critical skillset to master in the cloud age.
After completing the DevOps training course you will achieve hands-on expertise in various aspects of the DevOps delivery model. The practical learning outcomes of this Devops training course are:
An understanding of DevOps and the modern DevOps toolsets
The ability to automate all aspects of a modern code delivery and deployment pipeline using:
1. Source code management tools
2. Build tools
3. Test automation tools
4. Containerization through Docker
5. Configuration management tools
6. Monitoring tools
Who should take this course?
DevOps career opportunities are thriving worldwide. DevOps was featured as one of the 11 best jobs in America for 2017, according to CBS News, and data from Payscale.com shows that DevOps Managers earn as much as $122,234 per year, with DevOps engineers making as much as $151,461. DevOps jobs are the third-highest tech role ranked by employer demand on Indeed.com but have the second-highest talent deficit.
1. This DevOps training course will be of benefit the following professional roles:
2. Software Developers
3. Technical Project Managers
4. Architects
5. Operations Support
6. Deployment engineers
7. IT managers
8. Development managers
Learn more at https://www.simplilearn.com/cloud-computing/devops-practitioner-certification-training
Structuring the right team for DevOps without Re-Organization. I presented this at DevOps Fusion 2015. Tips include rapid feedback loop, value stream analysis, etc.
"Shift Left" is a DevOps practice that provides an effective means to perform testing with or in parallel to development activities.
When shifting left, development, test and operations work together to plan, manage and execute automated and continuous testing to accelerate feedback to developers and improve the quality of changes early in the life-cycle. The rate of the accelerated feedback is determined by an organization’s desired outcomes for velocity of changes and capacity for feedback.
EduXFactor presents to you a comprehensive up-to-date DevOps certification program. This course will empower you with job-relevant skills and power you ahead in your career.
With this course, master various aspects of software development, operations, continuous integration, continuous delivery, automated configuration management, test, and deployment using DevOps tools like Git, Docker, Jenkins, Ansible, Kubernetes, Puppet & Nagios..
Packed with hands-on exercise for every module, this course is suitable for software developers, technical project managers, architects, operations support, deployment engineers, IT managers, and development managers.
In 2009 Patrick Dubois coined the term "DevOps" when he organised the first "DevOpsDays" In Ghent, Belgium. Since then the term has become a term to explain the collaboration between all organisational stakeholders in IT projects (developers, operations, QA, marketing, security, legal, …) to deliver high quality, reliable solutions where issues are tackled early on in the value stream.
But reality shows that many businesses that implement "DevOps" are actually talking about a collaboration between development, QA and operations (DQO). Solutions are being provided but lack the security and/or legal regulations causing hard-to-fix problems in production environments.
In this talk I will explain how the original idea of Patrick to include all stakeholders got reduced to development, QA and operations and why it's so difficult to apply security or compliance improvements in this model. I will also talk about ways to make the DQO model welcoming for security experts and legal teams and why "DevSecOps" is now the term to be used to ensure security is no longer omitted from the value process.
Finally we'll have a vote if we keep the term "DevOps" as an all-inclusive representation for all stakeholders or if we need to start using "DevSecOps" to ensure the business understands can no longer ignore the importance of security.
Devops core principles
CI/CD basics
CI/CD with asp.net core webapi and Angular app
Iac Why and What?
Demo using Azure and Azure Devops
Docker why and what ?
Demo using Azure and Azure Devops
Kubernetes why and what?
Demo using Azure and Azure Devops
Building a Test Automation Strategy for SuccessLee Barnes
Choosing an appropriate tool and building the right framework are typically thought of as the main challenges in implementing successful test automation. However, long term success requires that other key questions must be answered including:
- What are our objectives?
- How should we be organized?
- Will our processes need to change?
- Will our test environment support test automation?
- What skills will we need?
- How and when should we implement?
In this workshop, Lee will discuss how to assess your test automation readiness and build a strategy for long term success. You will interactively walk through the assessment process and build a test automation strategy based on input from the group. Attend this workshop and you will take away a blue print and best practices for building an effective test automation strategy in your organization.
• Understand the key aspects of a successful test automation function
• Learn how to assess your test automation readiness
• Develop a test automation strategy specific to your organization
A high level introduction to DevOps. Explains what it is, how popular DevOps has become, why DevOps is popular, how DevOps differs from traditional approaches and some next steps to implementation.
When DevOps talks meet DevOps tactics, companies find that Continuous Integration is the make or break point. And implementing CI is one thing, but sustainable CI takes a little bit more consideration. CI is not all about releases, it is also about knowing more about how your software delivery pipeline works, it's weak points, and how you are doing over time.
Join CloudBees and cPrime as we discuss best practices for facilitating DevOps pipelines with Jenkins Workflow and reveal how the workflow engine of Jenkins CI and “Agilecentric” Devops practices together, support complex control structures, shortens the development cycle, stabilizes environments and reduces defects.
Microsoft recently released Azure DevOps, a set of services that help developers and IT ship software faster, and with higher quality. These services cover planning, source code, builds, deployments, and artifacts. One of the great things about Azure DevOps is that it works great for any app and on any platform regardless of frameworks.
In this session, I will provide a hands on workshop guiding you through getting started with Azure Pipelines to build your application. Using continuous integration and deployment processes, you will leave with clear understanding and skills to get your applications up and running quickly in Azure DevOps and see the full benefits that CI/CD can bring to your organization.
Talk given by Kelly Currier, Agile Senior Director and Vladimir Gerasimov, Product Management Senior Manager at Salesforce, at STPCon in April 2016
Salesforce adopted agile methodologies over 7 years ago. Over the years, it has helped us to drive innovation, productivity and become the world’s #1 CRM solution. Salesforce has taken agile methodologies and created a unique approach called the Adaptive Delivery Methodology (ADM). During this session, we will provide an ADM overview and how it helps us deliver 3 major releases with hundreds of features every year. We will also cover how we approach testing and quality through ADM. At Salesforce, there is no such thing as throwing code over the fence for someone else to test. Developers and Quality Engineers, we all work together to ensure release quality.
Devops Intro - Devops for Unicorns & DevOps for HorsesBoonNam Goh
An introduction to DevOps including full-fledged DevOps (the so-called DevOps for Unicorns) and legacy application DevOps (the so-called DevOps for Horses).
Why On-Demand Provisioning Enables Tighter Alignment of Test and Production E...Cognizant
To improve their test environments and application quality, organizations are turning to the cloud for on-demand provisioning, as well as build and deployment automation.
Structuring the right team for DevOps without Re-Organization. I presented this at DevOps Fusion 2015. Tips include rapid feedback loop, value stream analysis, etc.
"Shift Left" is a DevOps practice that provides an effective means to perform testing with or in parallel to development activities.
When shifting left, development, test and operations work together to plan, manage and execute automated and continuous testing to accelerate feedback to developers and improve the quality of changes early in the life-cycle. The rate of the accelerated feedback is determined by an organization’s desired outcomes for velocity of changes and capacity for feedback.
EduXFactor presents to you a comprehensive up-to-date DevOps certification program. This course will empower you with job-relevant skills and power you ahead in your career.
With this course, master various aspects of software development, operations, continuous integration, continuous delivery, automated configuration management, test, and deployment using DevOps tools like Git, Docker, Jenkins, Ansible, Kubernetes, Puppet & Nagios..
Packed with hands-on exercise for every module, this course is suitable for software developers, technical project managers, architects, operations support, deployment engineers, IT managers, and development managers.
In 2009 Patrick Dubois coined the term "DevOps" when he organised the first "DevOpsDays" In Ghent, Belgium. Since then the term has become a term to explain the collaboration between all organisational stakeholders in IT projects (developers, operations, QA, marketing, security, legal, …) to deliver high quality, reliable solutions where issues are tackled early on in the value stream.
But reality shows that many businesses that implement "DevOps" are actually talking about a collaboration between development, QA and operations (DQO). Solutions are being provided but lack the security and/or legal regulations causing hard-to-fix problems in production environments.
In this talk I will explain how the original idea of Patrick to include all stakeholders got reduced to development, QA and operations and why it's so difficult to apply security or compliance improvements in this model. I will also talk about ways to make the DQO model welcoming for security experts and legal teams and why "DevSecOps" is now the term to be used to ensure security is no longer omitted from the value process.
Finally we'll have a vote if we keep the term "DevOps" as an all-inclusive representation for all stakeholders or if we need to start using "DevSecOps" to ensure the business understands can no longer ignore the importance of security.
Devops core principles
CI/CD basics
CI/CD with asp.net core webapi and Angular app
Iac Why and What?
Demo using Azure and Azure Devops
Docker why and what ?
Demo using Azure and Azure Devops
Kubernetes why and what?
Demo using Azure and Azure Devops
Building a Test Automation Strategy for SuccessLee Barnes
Choosing an appropriate tool and building the right framework are typically thought of as the main challenges in implementing successful test automation. However, long term success requires that other key questions must be answered including:
- What are our objectives?
- How should we be organized?
- Will our processes need to change?
- Will our test environment support test automation?
- What skills will we need?
- How and when should we implement?
In this workshop, Lee will discuss how to assess your test automation readiness and build a strategy for long term success. You will interactively walk through the assessment process and build a test automation strategy based on input from the group. Attend this workshop and you will take away a blue print and best practices for building an effective test automation strategy in your organization.
• Understand the key aspects of a successful test automation function
• Learn how to assess your test automation readiness
• Develop a test automation strategy specific to your organization
A high level introduction to DevOps. Explains what it is, how popular DevOps has become, why DevOps is popular, how DevOps differs from traditional approaches and some next steps to implementation.
When DevOps talks meet DevOps tactics, companies find that Continuous Integration is the make or break point. And implementing CI is one thing, but sustainable CI takes a little bit more consideration. CI is not all about releases, it is also about knowing more about how your software delivery pipeline works, it's weak points, and how you are doing over time.
Join CloudBees and cPrime as we discuss best practices for facilitating DevOps pipelines with Jenkins Workflow and reveal how the workflow engine of Jenkins CI and “Agilecentric” Devops practices together, support complex control structures, shortens the development cycle, stabilizes environments and reduces defects.
Microsoft recently released Azure DevOps, a set of services that help developers and IT ship software faster, and with higher quality. These services cover planning, source code, builds, deployments, and artifacts. One of the great things about Azure DevOps is that it works great for any app and on any platform regardless of frameworks.
In this session, I will provide a hands on workshop guiding you through getting started with Azure Pipelines to build your application. Using continuous integration and deployment processes, you will leave with clear understanding and skills to get your applications up and running quickly in Azure DevOps and see the full benefits that CI/CD can bring to your organization.
Talk given by Kelly Currier, Agile Senior Director and Vladimir Gerasimov, Product Management Senior Manager at Salesforce, at STPCon in April 2016
Salesforce adopted agile methodologies over 7 years ago. Over the years, it has helped us to drive innovation, productivity and become the world’s #1 CRM solution. Salesforce has taken agile methodologies and created a unique approach called the Adaptive Delivery Methodology (ADM). During this session, we will provide an ADM overview and how it helps us deliver 3 major releases with hundreds of features every year. We will also cover how we approach testing and quality through ADM. At Salesforce, there is no such thing as throwing code over the fence for someone else to test. Developers and Quality Engineers, we all work together to ensure release quality.
Devops Intro - Devops for Unicorns & DevOps for HorsesBoonNam Goh
An introduction to DevOps including full-fledged DevOps (the so-called DevOps for Unicorns) and legacy application DevOps (the so-called DevOps for Horses).
Why On-Demand Provisioning Enables Tighter Alignment of Test and Production E...Cognizant
To improve their test environments and application quality, organizations are turning to the cloud for on-demand provisioning, as well as build and deployment automation.
Creating and managing test environments best practices for test infrastructur...Knoldus Inc.
Explore best practices for setting up and managing test environments. Learn about infrastructure considerations, environment provisioning, configuration management, and infrastructure as code (IaC) principles. Gain insights to create stable, scalable, and reusable test environments.
The Business Case for On-Demand Test ServicesCognizant
On-demand services offer a superior alternative to traditional QA service delivery models by providing a pay-per-use approach and enabling greater operational agility.
Programming testing is the method involved with assessing and confirming that a product item or application does what it should do. The advantages of testing incorporate forestalling bugs, lessening improvement costs and further developing execution.
Programming testing is the method involved with assessing and confirming that a product item or application does what it should do. The advantages of testing incorporate forestalling bugs, lessening improvement costs and further developing execution.
Programming testing is the method involved with assessing and confirming that a product item or application does what it should do. The advantages of testing incorporate forestalling bugs, lessening improvement costs and further developing execution.
Test case prioritization techniques schedule test cases for execution in an order that attempts to increase their effectiveness in meeting some performance goal. Various goals are possible; one involves rate of fault detection | a measure of how quickly faults are detected within the testing process. An improved rate of fault detection during testing can provide faster feedback on the system under test, and let software engineers begin correcting faults earlier than might otherwise be possible.
ROLE OF iSAFE/iMobi IN SEAMLESS INTEGRATION OF THE DEVOPS ENVIRONMENTIndium Software
IP-led test automation framework supported by blueprint
for product development in Devops environment can
ensure automation in the true sense.
DevOps is fast becoming adopted as the environment for product
development. It facilitates closer integration of development and operations
teams, reducing the time needed to develop and deploy a product. However,
it is still in its early stages and the teams continue to work in silos due to the
different kinds of tools they need suited to their needs.
An IP-driven testing framework like iSAFE can be the bulwark on which the development, testing and operations teams can integrate more seamlessly,
as it provides one key feature needed when handling such a comprehensive
environment – traceability. The other advantages, of course, are reusability,
automated alerts and shorter testing periods, thus aiding in the quick time-to-market
needs of the organizations.
For Impetus’ White Papers archive, visit- http://www.impetus.com/whitepaper
In this white paper, Impetus focuses on how the power of the Cloud can be harnessed to address the software product testing challenges faced by organizations.
As cloud computing becomes of strategic importance in the enterprise, part of the solution is no longer on-premise but in the cloud, adding a layer of complexity. Edwin Chan demystifies performance testing of cloud systems and applications by addressing the following key questions: Is performance testing of cloud systems fundamentally different from testing on-premises applications? What are the best practices for performance testing of both cloud and on-premises systems? Performance testing of cloud systems is essentially the same as that of its on-premises counterpart with the exception of the key consideration of network latency. After clearing common misconceptions, Edwin shares the hot topic best practices—adopting an agile/lean methodology, conducting early performance testing, and automating the injection of test data. Discuss the challenges the testing team faces in these days of disruptive and fast-paced technology changes. Take back and apply some of the best practices that fit your organization’s need.
Using Adaptive Scrum to Tame Process Reverse Engineering in Data Analytics Pr...Cognizant
Organizations rely on analytics to make intelligent decisions and improve business performance, which sometimes requires reproducing business processes from a legacy application to a digital-native state to reduce the functional, technical and operational debts. Adaptive Scrum can reduce the complexity of the reproduction process iteratively as well as provide transparency in data analytics porojects.
It Takes an Ecosystem: How Technology Companies Deliver Exceptional ExperiencesCognizant
Experience is evolving into a strategy that reaches across technology companies. We offer guidance on the rise of experience and its role in business modernization, with details on how orgnizations can build the ecosystem to support it.
The Work Ahead: Transportation and Logistics Delivering on the Digital-Physic...Cognizant
The T&L industry appears poised to accelerate its long-overdue modernization drive, as the pandemic spurs an increased need for agility and resilience, according to our study.
Enhancing Desirability: Five Considerations for Winning Digital InitiativesCognizant
To be a modern digital business in the post-COVID era, organizations must be fanatical about the experiences they deliver to an increasingly savvy and expectant user community. Getting there requires a mastery of human-design thinking, compelling user interface and interaction design, and a focus on functional and nonfunctional capabilities that drive business differentiation and results.
The Work Ahead in Manufacturing: Fulfilling the Agility MandateCognizant
According to our research, manufacturers are well ahead of other industries in their IoT deployments but need to marshal the investment required to meet today’s intensified demands for business resilience.
The Work Ahead in Higher Education: Repaving the Road for the Employees of To...Cognizant
Higher-ed institutions expect pandemic-driven disruption to continue, especially as hyperconnectivity, analytics and AI drive personalized education models over the lifetime of the learner, according to our recent research.
Engineering the Next-Gen Digital Claims Organisation for Australian General I...Cognizant
In recent years, insurers have invested in technology platforms and process improvements to improve
claims outcomes. Leaders will build on this foundation across the claims landscape, spanning experience,
operations, customer service and the overall supply chain with market-differentiating capabilities to
achieve sustainable results.
Profitability in the Direct-to-Consumer Marketplace: A Playbook for Media and...Cognizant
Amid constant change, industry leaders need an upgraded IT infrastructure capable of adapting to audience expectations while proactively anticipating ever-evolving business requirements.
Green Rush: The Economic Imperative for SustainabilityCognizant
Green business is good business, according to our recent research, whether for companies monetizing tech tools used for sustainability or for those that see the impact of these initiatives on business goals.
Policy Administration Modernization: Four Paths for InsurersCognizant
The pivot to digital is fraught with numerous obstacles but with proper planning and execution, legacy carriers can update their core systems and keep pace with the competition, while proactively addressing customer needs.
The Work Ahead in Utilities: Powering a Sustainable Future with DigitalCognizant
Utilities are starting to adopt digital technologies to eliminate slow processes, elevate customer experience and boost sustainability, according to our recent study.
AI in Media & Entertainment: Starting the Journey to ValueCognizant
Up to now, the global media & entertainment industry (M&E) has been lagging most other sectors in its adoption of artificial intelligence (AI). But our research shows that M&E companies are set to close the gap over the coming three years, as they ramp up their investments in AI and reap rising returns. The first steps? Getting a firm grip on data – the foundation of any successful AI strategy – and balancing technology spend with investments in AI skills.
Operations Workforce Management: A Data-Informed, Digital-First ApproachCognizant
As #WorkFromAnywhere becomes the rule rather than the exception, organizations face an important question: How can they increase their digital quotient to engage and enable a remote operations workforce to work collaboratively to deliver onclient requirements and contractual commitments?
Five Priorities for Quality Engineering When Taking Banking to the CloudCognizant
As banks move to cloud-based banking platforms for lower costs and greater agility, they must seamlessly integrate technologies and workflows while ensuring security, performance and an enhanced user experience. Here are five ways cloud-focused quality assurance helps banks maximize the benefits.
Getting Ahead With AI: How APAC Companies Replicate Success by Remaining FocusedCognizant
Changing market dynamics are propelling Asia-Pacific businesses to take a highly disciplined and focused approach to ensuring that their AI initiatives rapidly scale and quickly generate heightened business impact.
The Work Ahead in Intelligent Automation: Coping with Complexity in a Post-Pa...Cognizant
Intelligent automation continues to be a top driver of the future of work, according to our recent study. To reap the full advantages, businesses need to move from isolated to widespread deployment.
The Work Ahead in Intelligent Automation: Coping with Complexity in a Post-Pa...
The Business Case for Test Environment Management Services
1. • Cognizant Reports
The Business Case for Test Environment
Management Services
Application test environments managed by temporary resources can cause
spikes in release costs, high rates of defects that leak into production
systems and consequent losses. Test environment services delivered via
a dedicated managed services approach represents a superior alternative
that delivers greater monetary benefits and fosters enhanced reputation
as well as brand loyalty.
Executive Summary Managing environment-related issues and
This much about testing is incontrovertible: defects consumes nearly 40% of the effort
Poor test environment configuration, downtime, involved in the software development lifecycle.1
and the unavailability of test environments can Apart from delaying testing cycles and creating
impact the quality of the testing process as well additional costs, the test management problems
as the code itself. This is caused primarily by noted above force managers to unnecessarily
the lack of ownership and management of test rush through the testing process, conduct tests
environments. in environments that are far from optimal (i.e.,
environments that do not approximate produc-
The reasons are manifold. For starters, most tion scenarios) and result in risk-based sign-off
IT organizations lack a dedicated testing team (i.e., acknowledging the risks involved in releasing
that ensures code quality from top to bottom. a product that is not completely tested).
In fact, many organizations temporarily divert
employees from development and other respon- Against this backdrop, test environment manage-
sibilities to manage test environments. Such ment services (TEMS) provides a consultative
a setup leads to a lack of accountability and approach, thus making it a superior alternative.
creates issues that often impact the quality of The TEMS framework helps organizations to
testing, which in turn compromises the quality of address common test environment issues and
software applications. Our experience suggests achieve efficiencies in demand fulfillment,
that the cost of QA project/activity is increased capacity utilization and environment availability.
by at least 20% to 25% due to diversion of efforts Delivered as an end-to-end managed service,
by testers and developers. TEMS includes creating a centralized, single
cognizant reports | june 2012
2. ownership for environment management, moni- cycle times, organizations must ensure that test
toring and maintenance along with cloud-based environments closely mimic real-time situations
infrastructure provisioning. and are highly available. Reduced release cycle
time does not provide an allowance for defect
TEMS is designed to: leakage into production, and regardless of the
project timeframe all testing steps involved until
• Support in-flight projects, planned projects release sign-off must be rigorously followed.
and new application releases.
• Manage existing test environments, build new Test environments have a vital role to play in deliv-
test environments and provide additional ering fully tested code and in ensuring confidence
capacity on demand via the cloud. in the successful release of the application. Our
• Help organizations establish the processes experience with organizations across industries
and controls that are required to place an suggest that poorly built test environments often
application environment in the cloud, provided limit the testing team’s ability to test applications
the application is properly cloud-enabled. for various scenarios and increases required
testing cycles, which in turn results in unneces-
The monetary and reputational benefits of switch- sary costs and allows defects to pass through to
ing to TEMS are significant as its per-project costs the production environments.
are lower than test projects without managed
services. However, organizations need to be aware Poor software quality costs organizations
that the TEMS model has its own limitations. They worldwide $500 billion annually, according to a
must also exercise due diligence in choosing the recent survey by Caper Jones & Associates LLC.2
right provider in order to achieve desired results. Defects seep in at various stages of an applica-
tion’s development lifecycle; importantly, finding
Why Managed Test Environment and fixing these bugs in the early stages costs
Services? less than remediation during the later stages.
Jeff and his team are excited about their first big According to Gartner, “The cost of fixing defects
assignment — testing a new business application ranges from a low of approximately $70 (cost to
to be released shortly. They have left no stone fix a defect at the requirements phase) to a high
unturned in their preparation. of $14,000 (cost to fix a defect in production).”3
However, on the day of testing, Jeff faces a Testing is considered an important but not very
few setbacks. First, he discovers that the test business-critical activity; hence, test environ-
environment was not built to the project ments within many organizations receive a low
specifications due to lack of capacity and priority. Test environments are typically managed
configurations. Second, by the time he gets the not by a dedicated unit but by a team of employees
configuration right, he is staring at an unsched- temporarily diverted from development and
uled infrastructure maintenance activity. He is production support activities. With no unit
forced to hurry through the testing process. As exclusively responsible for testing, there is a
a result, defects surface immediately after the lack of ownership and accountability. Further,
application is rolled out. such ad hoc testing teams often have limited
capabilities in building and managing test
Such incidents are common in testing centers environments, leading to unnecessary delays
where lack of ownership and poor management in creating test environments and other issues
of testing impact quality, allowing defects to pass (see sidebar, page 4) that significantly impact
through to production. the effectiveness of testing and ultimately the
business. It is therefore not surprising that in an
Issues in Test Environment Management Aberdeen Group study, only 42% of over 200
The growing criticality of business applica- organizations surveyed were happy with the per-
tions needs no substantiation. The increasing formance of their business-critical applications.4
complexity of today’s applications has expanded
the scope of testing, creating unforeseen The need for efficient test environments is
challenges for testing teams. To deal with the endorsed by an old but still relevant National
complexity and the pressure to reduce release Institute of Standards and Technology (NIST)
cognizant reports 2
3. study. According to the study, improving test • Setting up new environment: service on-board-
infrastructure can save U.S. organizations ing and build-out.
one-third of their annual QA costs.5 • Environment monitoring and management:
hardware, networking, systems software and
TEMS to the Rescue applications.
By applying a combination of consulting and man- • Infrastructure monitoring: servers, databases,
aged services, the TEMS framework addresses mainframes, etc.
common issues that plague test environment • Service request management: issue resolution
management. In most deployments, a dedicated and coordination.
TEMS team works closely with an organization’s • Release management: systematic updates,
IT department to create a centralized resource terminations and reports.
that owns, monitors and controls test environ-
ments. A real-time project dashboard typically Further, TEMS offered via the cloud can generate
enables day-to-day tracking of project milestones additional benefits by eliminating the need for
and metrics, while an integrated delivery model costly upfront Cap-Ex investments and providing
synthesizes the efforts of multiple project teams. on-demand access to test environments.
One of TEMS’ key attributes is that it can be TEMS: A Better Alternative
offered modularly as well as an end-to-end In the traditional model, the cost of test
service (see Figure 1). It can also be delivered environment management is borne by various
in multivendor scenarios and with minimum teams. When organizations switch to the TEMS
disruption to daily business activities in a rapid model, those costs become visible since under
and robust fashion by following best practices for TEMS a separate group is accountable for
change management. Organizations can evaluate providing testing services through a direct cost
their current test environment management allocation model. While TEMS merely makes
setup and choose the most suitable service. the relevant costs visible, organizations may
mistakenly believe it necessitates additional
TEMS provides a single contact point for costs. On the contrary, the monetary losses due
resolution of issues, queries, changes and all to environment downtime — paying testers for the
other requirements, thus creating greater idle time and for the additional time they work
accountability for losses due to test environ- to finish the task — are far greater than the cost
ment issues. Broadly, TEMS’ end-to-end service of creating a separate team that minimizes the
includes the following: incidence of environment issues.
TEMS Suite
Module Services
Environment Management • Manage/monitor SLAs.
• Plan/analyze environment requests.
• Facilitate design workshops.
• Liaise with testing/infrastructure teams.
• Create/update service reports.
• Identify improvement areas.
Environment Monitoring • Additional responsibility for monitoring servers and notification.
Environment Maintenance • Handle infrastructure support or any combination of technology support levels.
• Incident management, problem management, change management.
• Build and smoke-test environment.
Cloud Infrastructure • Provide hosting services (using partner vendor).
Provisioning/Management • Provision virtual servers and test environments.
Figure 1
cognizant reports 3
4. In this regard, TEMS offers a superior alternative than 80% of customer-invested tools and
to the traditional approach, creating new efficien- technologies, thereby lowering the total cost
cies in managing test environments and deliv- of ownership (TCO).
ering greater monetary benefits (as described • Increased environment availability: Through
below). Such advantages arise out of the fact proactive monitoring, TEMS ensures the
that ownership and accountability within TEMS health of the environment as well as its
rests with an exclusive team as opposed to the availability. In some cases, average uptime of
traditional model where a heterogeneous team is more than 90% has been achieved from just
accountable for test environment management 68%.6 Higher availability allows testers to
activities. Such a centralized, dedicated TEMS perform extensive and exhaustive testing
team therefore frees up ad hoc resources drawn which vastly improves result efficacy.
together from multiple groups and instead allows • Proper scheduling replaces chaos: Improper
them to focus on their core activities. scheduling of tests when the environment is
shared by multiple testing teams often delays
TEMS’ major benefits include: testing projects. This forces test managers to
conduct nonfunctional testing in scaled-down
• Improved build efficiencies: With a dedicated environments and/or non-functional testing
team of specialists running the show, new that relies on stubbed environments, result-
test environments can be built and deployed ing in projects being signed-off with numerous
rapidly. This minimizes the defects that caveats and application problems in produc-
arise due to inaccurate test environment tion. In contrast, by continuously providing
configuration (the rate of which is 30% environment availability metrics through daily
according to studies) and achieve build standardized reports, TEMS allows teams to
efficiencies of more than 80%. book the environment in advance. Further,
• Reuse prevents waste: As a “tool-agnostic” through proper scheduling with set start
framework, TEMS enables reuse rates of more and end dates and institutionalization of the
Quick Take
Drawbacks of Traditional Test Environment Management
• Test environments differ from production environments in terms of operating systems, patch
levels, software versions, configuration, etc. The wider the gap between the test and production
environments, the greater the chance of an application failing after being deployed or a defect
leaking into live systems. Further, configuration changes made in response to errors often go
untracked, causing errors when applications are moved into production.
• Loosely managed asset control necessitates undue investment in infrastructure. Inadequate
access control reduces confidence in the state of the environment and its configuration control.
Together, this introduces additional risks and adds unnecessary costs to the project.
• Simulating the production issue in the test environment to identify root cause issues is
challenging due to a lack of complete alignment between the test environment and production
versions across system component layers.
• Testing teams often clone production databases or extract data by writing scripts to create test
data. This approach takes a lot of effort, is prone to errors and may not meet data protection
policies. Often, this activity is not change-controlled and is non-auditable.
• The conventional approach of manually creating in-house testing environments that fully
mirror the complexities and multiplicities of real-time environment consumes a lot of resources.
This has prompted application developers to adopt the risky approach of conducting testing in
production environments. However, the Sarbanes–Oxley Act and other regulations mandates
restricted access to production systems. This has forced organizations across industries to
conduct testing in separate environments. Further, it is not uncommon for organizations that
cannot afford large-scale test environments to use sizing or extrapolation techniques during
performance testing to understand how an application behaves in the real world. This approach,
however, leaves too much room for potential problems after implementation.
cognizant reports 4
5. release management process, TEMS helps controlled access management; maintaining
deliver fully tested projects on time. up-to-date inventory of environments and
• Better utilization of infrastructure: In the infrastructure components; compliance with
traditional model, approximately 50% to 70% security and standards; etc., TEMS offers organi-
of the technology infrastructure earmarked zations a one-stop solution to their test environ-
for testing is underutilized, according to both ment issues.
anecdotal and published reports.7 Through
effective capacity planning, TEMS ensures TEMS Efficiencies
that testing projects receive the right amount Figure 2 illustrates the QA project efficiencies
of resources and test environments are that TEMS can generate, with the dark red
optimally utilized, prompting improved returns and dark green bands representing the lowest
on existing environments. For instance, (pre-TEMS) and best-in-class (post-TEMS)
incidents such as over-provisioning, which efficiency levels respectively. In our experi-
result in additional costs, can be prevented. ence, most of the current test environment
• Communication replaces commotion: In efficiencies fall in the D and E bands, with an
the absence of ownership, test environment average efficiency of 57%. Deploying TEMS has
downtime and changes are often not increased the average efficiency levels to 75%,
controlled and communicated properly. This and has the potential to increase it to 95%.
poses challenges in managing testing projects
and achieving KPIs. TEMS creates an effective However, it must be noted that the TEMS
communication mechanism among the key framework does not establish rigorous processes
stakeholders in the event of environment or controls and has lower SLA commitments
issues by deploying same-day text alerts, etc. when compared with production environments.
Additionally, its scope is limited to the staging
With a host of other benefits such as targeted environment and does not include the change
problem management and root cause analysis; controls at the production environment and
TEMS-Efficiency Meter
Commonly seen TEMS Partly SLA and OLA
Pre-TEMS (%) Implemented (%) in place (%)
QA Environment Availability
Post-TEMS: Highly available managed test environments reduces
project costs.
(91-100) A 95
(86-90) B
(81-85) C 85
(60-80) D
(41-60) E 57
(21-40) F
(1-20) G
Pre-TEMS: Poorly managed test environments raises project costs.
Note:
1. Bands G to D indicate improvements to QA environment can be rapid as low availability indicator means the environment
is unstable or poorly supported.
2. Bands C and B represent environments that get timely support and are stable, but require more processes and controls
to improve availability.
3. Band A represents environments that are stable and their availability is high as committed SLA/OLAs are in place.
Source: Cognizant Research Center
Figure 2
cognizant reports 5
6. release management process. These limita- Organizations planning to deploy TEMS must
tions are by design of the service and separate find a partner that understands business
TEMS from production environment service challenges and has prior experience in offering
management. They add value by lowering the cost managed testing services. Reputable consulting
of operation and enabling the test environment to firms that have rich domain experience and the
align itself with release management objectives. ability to work in multivendor scenarios should
be considered to ensure a smooth transfor-
Moving Forward: Embracing TEMS mation, create new process efficiencies and
A stable, reliable and flexible test environ- accelerate payback, culminating in instilling
ment is important for companies to cope with greater discipline in managing test environments.
today’s testing demands. Poor test environment This developent will help ensure that managers
management affects delivery schedules enjoy their testing assignments and assure
and results in increased risk of subsequent application code quality and business continuity
software failures as well as additional business across the enterprise.
expenditure. TEMS has much to offer organiza-
tions seeking to overcome environment-related
challenges in an easy and convenient fashion.
Footnotes
1
“Testing Environment Management Challenges With Virtualization,” ComputerWorldUK, Sept. 6, 2008.
2
“Software Quality in 2011: A Survey of the State of the Art,” Caper Jones & Associates LLC, August 31,
2011.
3
“Hype Cycle for Application Development, 2011,” Gartner, July 27, 2011.
4
“Application Performance Management: The Lifecycle Approach Brings IT and Business Together,”
Aberdeen Group, June 30, 2008.
5
“The Economic Impacts of Inadequate Infrastructure for Software Testing,” NIST, May 2002.
6
“UK Bank Hits Peak Performance with Cognizant TEMS,” Cognizant, 2011.
7
“Taking Testing to the Cloud,” Cognizant, March 2011.
Bibliography
1. Martin Perlin, "Notes from the Trenches: Obstacles and Challenges to IT Environment Stability,"
www.evolven.com, October 2011.
2. Wayne Ariola, "The Next Generation of Test Environment Management," Virtualization Journal,
July 12, 2011.
3. “Determining the 'Right' Number of Nonproduction Environments,” Gartner, July 4, 2011.
4. “Why On-Demand Provisioning Enables Tighter Alignment of Test and Production Environments,”
Cognizant, December 2011.
5. “Transforming a Telecom Service Provider’s Test Environment for Maximum Responsiveness and
Efficiency,” Cognizant, 2011.
cognizant reports 6