http://www.hcltech.com/enterprise-transformation-services/overview~ More on ETS
It is not only desirable but also necessary to assess the quality of testing being delivered by a vendor. Specific to software testing, there are some discerning metrics that one an look at, however it must be kept in mind that there are multiple factors that affect these metrics which are not necessarily under the control of testing team. The SLAs for testing initiatives can, and should, only be committed after a detailed understanding of the customer’s IT organization in terms of culture and process maturity and after analyzing the various trends among these metrics. This white paper lists some of the popular testing metrics and the factors one must keep in mind while reading in to their values.
Excerpts from the Paper
The estimates and planning for testing is based on certain assumptions and available historical data. However if there are higher number of disruptions (than anticipated) to testing in terms of environment unavailability or higher number of defects being found and fixed, the quality time available for testing the system would be less and hence higher number of defects slip through the testing stage. We must ensure that the data on defects on all subsequent stages are also available and are accurate. Production defects are usually handled by a separate Production support team and testing team is at times not given much insight in to this data. Also, since multiple projects and/or Programs would be going live, one after another, there are usually challenges in identifying which defects in Production can be attributed to which Project or Program. Inaccuracies in assignment would lead to inaccurate measure of test stage effectiveness.
This presentation gives you a walkthorugh on CTFL module 01.
Covers in detail about-
1. Fundamentals of testing
2. Terminologies in testing
3. Seven testing principles
4. Fundamental test process
RCA on Residual defects – Techniques for adaptive Regression testingIndium Software
One of the important issues associated with a system lifespan view that we have ignored in past
years is the effects of enduring defects – defects that persist undetected – across several
releases of a system. Many studies performed to date have evaluated regression testing
techniques under the limited context such as short term assessment which do not fully account
for the industry based solutions.
Reports estimate that regression testing consumes as much as 80% of the overall testing
budget and can consume up to 50% of the cost of software maintenance.
Aliaa delivered a session in the topic of “Test planning” using a new technique of delivering content through games and knowledge sharing instead of instructive technique. The session covered all test planning activities including defining test items, risk assessment techniques, testing strategies, planning for testing resources, testing scheduling, and test deliverables and the final test plan documents.
The session introduced to quality team at ITWorx (June , 2013)
This presentation gives you a walkthorugh on CTFL module 01.
Covers in detail about-
1. Fundamentals of testing
2. Terminologies in testing
3. Seven testing principles
4. Fundamental test process
RCA on Residual defects – Techniques for adaptive Regression testingIndium Software
One of the important issues associated with a system lifespan view that we have ignored in past
years is the effects of enduring defects – defects that persist undetected – across several
releases of a system. Many studies performed to date have evaluated regression testing
techniques under the limited context such as short term assessment which do not fully account
for the industry based solutions.
Reports estimate that regression testing consumes as much as 80% of the overall testing
budget and can consume up to 50% of the cost of software maintenance.
Aliaa delivered a session in the topic of “Test planning” using a new technique of delivering content through games and knowledge sharing instead of instructive technique. The session covered all test planning activities including defining test items, risk assessment techniques, testing strategies, planning for testing resources, testing scheduling, and test deliverables and the final test plan documents.
The session introduced to quality team at ITWorx (June , 2013)
The anonymised slides from an old (but hopefully still relevant) talk on the case for placing a strategic focus on design testability. The material covers the technical, process and organisational considerations arising from such a strategy and is predominantly a summary of the ideas presented in Brett Pettichord's 2001 "Design For Testability' paper available here. The presentation makes a case for why a high level of design testability can be seen as a critical success factor in achieving sustained agility.
Test Planning and Test Estimation TechniquesMurageppa-QA
In this Quality Assurance Training session, you will learn about Types of Testing , Test Strategy and Planning, and Test Estimation Techniques. Topic covered in this session are:
• Test Planning,
• Test Estimation Techniques
For more information, about this quality assurance training, visit this link: https://www.mindsmapped.com/courses/quality-assurance/software-testing-training-with-hands-on-project-on-e-commerce-application/
Moodle is a very flexible application with a large number of variables and roles. Testing upgrades and changes can be a challenge. This presentation should help attendees focus testing at their own workplace.
TOPS Technologies offer Professional Software Testing Training in Ahmedabad.
Ahmedabad Office (C G Road)
903 Samedh Complex,
Next to Associated Petrol Pump,
CG Road,
Ahmedabad 380009.
http://www.tops-int.com/live-project-training-software-testing.html
Most experienced IT Training Institute in Ahmedabad known for providing software testing course as per Industry Standards and Requirement.
Importance of a Test Management Tool for Your ProjectSarah Elson
The age we live in demands for RAD(Rapid Application Development) models where testing provides a sense of relief by delivering a sound quality check. From noting down the test scenarios and developing respective test cases, to collecting the test results and sharing them with the team. Turning a blind eye towards the tiniest aspect while testing may have a high impact on your project delivery, or in worst cases, it could even lead to postponement of your release date. Fortunately, we have test management tools available in the market to help orchestrate our release cycle.
YouTube Link: https://youtu.be/S2_AJP9Oeg0
**Test Automation Masters Program: https://www.edureka.co/masters-program/automation-testing-engineer-training **
This Edureka PPT on "Test Plan in Software Testing" will give you in-depth knowledge on how to create a Test Plan in Software Testing and why it is important. The following are the topics covered in the session:
Software Testing Documentation
What is Test Plan?
Benefits of Using Test Plan
Types of Test Plan
How to Write a Test Plan?
Test Plan Template / Test Plan Document
Software Testing Blog playlist: http://bit.ly/2UXwdJm
Selenium playlist: https://goo.gl/NmuzXE
Selenium Blog playlist: http://bit.ly/2B7C3QR
Follow us to never miss an update in the future.
YouTube: https://www.youtube.com/user/edurekaIN
Instagram: https://www.instagram.com/edureka_learning/
Facebook: https://www.facebook.com/edurekaIN/
Twitter: https://twitter.com/edurekain
LinkedIn: https://www.linkedin.com/company/edureka
Castbox: https://castbox.fm/networks/505?country=in
Software testing for project report .pdfKamal Acharya
Methods of Software Testing There are two basic methods of performing software testing: 1. Manual testing 2. Automated testing Manual Software Testing As the name would imply, manual software testing is the process of an individual or individuals manually testing software. This can take the form of navigating user interfaces, submitting information, or even trying to hack the software or underlying database. As one might presume, manual software testing is labor-intensive and slow.
The anonymised slides from an old (but hopefully still relevant) talk on the case for placing a strategic focus on design testability. The material covers the technical, process and organisational considerations arising from such a strategy and is predominantly a summary of the ideas presented in Brett Pettichord's 2001 "Design For Testability' paper available here. The presentation makes a case for why a high level of design testability can be seen as a critical success factor in achieving sustained agility.
Test Planning and Test Estimation TechniquesMurageppa-QA
In this Quality Assurance Training session, you will learn about Types of Testing , Test Strategy and Planning, and Test Estimation Techniques. Topic covered in this session are:
• Test Planning,
• Test Estimation Techniques
For more information, about this quality assurance training, visit this link: https://www.mindsmapped.com/courses/quality-assurance/software-testing-training-with-hands-on-project-on-e-commerce-application/
Moodle is a very flexible application with a large number of variables and roles. Testing upgrades and changes can be a challenge. This presentation should help attendees focus testing at their own workplace.
TOPS Technologies offer Professional Software Testing Training in Ahmedabad.
Ahmedabad Office (C G Road)
903 Samedh Complex,
Next to Associated Petrol Pump,
CG Road,
Ahmedabad 380009.
http://www.tops-int.com/live-project-training-software-testing.html
Most experienced IT Training Institute in Ahmedabad known for providing software testing course as per Industry Standards and Requirement.
Importance of a Test Management Tool for Your ProjectSarah Elson
The age we live in demands for RAD(Rapid Application Development) models where testing provides a sense of relief by delivering a sound quality check. From noting down the test scenarios and developing respective test cases, to collecting the test results and sharing them with the team. Turning a blind eye towards the tiniest aspect while testing may have a high impact on your project delivery, or in worst cases, it could even lead to postponement of your release date. Fortunately, we have test management tools available in the market to help orchestrate our release cycle.
YouTube Link: https://youtu.be/S2_AJP9Oeg0
**Test Automation Masters Program: https://www.edureka.co/masters-program/automation-testing-engineer-training **
This Edureka PPT on "Test Plan in Software Testing" will give you in-depth knowledge on how to create a Test Plan in Software Testing and why it is important. The following are the topics covered in the session:
Software Testing Documentation
What is Test Plan?
Benefits of Using Test Plan
Types of Test Plan
How to Write a Test Plan?
Test Plan Template / Test Plan Document
Software Testing Blog playlist: http://bit.ly/2UXwdJm
Selenium playlist: https://goo.gl/NmuzXE
Selenium Blog playlist: http://bit.ly/2B7C3QR
Follow us to never miss an update in the future.
YouTube: https://www.youtube.com/user/edurekaIN
Instagram: https://www.instagram.com/edureka_learning/
Facebook: https://www.facebook.com/edurekaIN/
Twitter: https://twitter.com/edurekain
LinkedIn: https://www.linkedin.com/company/edureka
Castbox: https://castbox.fm/networks/505?country=in
Software testing for project report .pdfKamal Acharya
Methods of Software Testing There are two basic methods of performing software testing: 1. Manual testing 2. Automated testing Manual Software Testing As the name would imply, manual software testing is the process of an individual or individuals manually testing software. This can take the form of navigating user interfaces, submitting information, or even trying to hack the software or underlying database. As one might presume, manual software testing is labor-intensive and slow.
Programming testing is the method involved with assessing and confirming that a product item or application does what it should do. The advantages of testing incorporate forestalling bugs, lessening improvement costs and further developing execution.
Programming testing is the method involved with assessing and confirming that a product item or application does what it should do. The advantages of testing incorporate forestalling bugs, lessening improvement costs and further developing execution.
Programming testing is the method involved with assessing and confirming that a product item or application does what it should do. The advantages of testing incorporate forestalling bugs, lessening improvement costs and further developing execution.
JIMS Vasant KunjII is the Top institute for BCA. JIMS is one of the Best BCA Colleges in Delhi which offers best placements in Top IT Companies in Delhi NCR. It is amongst the top A+ Category highest ranked colleges in Delhi, provides 3 years Regular Degree from UGC Approved University.
This unit of Software Testing is a part of BCA 5th sem syllabi.
Emergence of ITOA: An Evolution in IT Monitoring and ManagementHCL Technologies
IT operations analytics(ITOA) plays key role by providing intelligence that makes business sense out of the real-time data being generated by infrastructure components and applications.
USING FACTORY DESIGN PATTERNS IN MAP REDUCE DESIGN FOR BIG DATA ANALYTICSHCL Technologies
Though insights from Big Data gives a breakthrough to make better business decision, it poses its own set of challenges. This paper addresses the gap of Variety problem and suggest a way to seamlessly handle data processing even if there is change in data type/processing algorithm. It explores the various map reduce design patterns and comes out with a unified working solution (library). The library has the potential to ‘adapt’ itself to any data processing need which can be achieved by Map Reduce saving lot of man hours and enforce good practices in code.
HCL HELPS A US BASED WIRELINE TELECOM OPERATOR FOR BETTER LEAD-TO-CASH AND TH...HCL Technologies
The client is a privately-held competitive local exchange carrier, offering voice services, phone services, internet access, etc. to business customers primarily in California and Nevada. HCL's engagement included managed services application portfolio, CRM implementation and reporting. The client achieved seamless transition within three months with 100% off-shore presence using HCL's transition methodology
HCL HELPS A LEADING US TELECOM PROTECT ITS MARKET SHARE AND MAINTAIN HIGH LEV...HCL Technologies
The customer is a worldwide player in Networking & Cloud automation, Workflow automation and sustenance engineering. HCL took complete ownership of the customer's engineering services and saved costs by minimizing material overhead in thermal projections.
HCL suggests solutions to reduce airborne noise being emitted by vacuum cleaners. It has been seen that blowers used in vacuum cleaners are the main source of airborne noise and blade wakes are unavoidable in turbo machines.Focus of this whitepaper is to understand how to reduce sound intensity of vacuum cleaners and studying its effects on human hearing. ERS division in HCL proposes the design of a spiral enclosure for the blower in the vacuum cleaner. HCL suggests solutions to reduce airborne noise being emitted by vacuum cleaners. ERS division in HCL proposes the design of a spiral enclosure for the blower in the vacuum cleaner.
Comply is an IoT enabled, Small Pill Box, with a digital display that keeps track of your medical dosage level as well as the remaining pills. Offload complex tasks by pairing the solution with a smartphone/tablet app or a wearable fitness monitor. The collated data is analyzed from individual Comply units and then sent to the cloud.
Smart City solution providers will face challenges in increasing network load due to the huge amounts of video data flowing through their networks. For cost-effective analytics, distributed architecture with user control is just the right solution required. In Smart Cities with varying applications of video analytics solutions in fields such as security systems, utilities operators, and emergency response systems, it gives users a simple way to pick the feed they would like, instrument the analysis they want, and report the way they require in a simple-configurable manner.
With the advent of IoT and connected devices, there is an urgent need for a security framework that addresses major security goals of embedded devices. Security has to be an exercise built into the product development process instead of adding as an add-on feature.
Connected cars are fast becoming a reality and has the potential to change the way businesses are run. A connected car facilitates devices inside the car to connect with the computing and application servers and use computing power to access real time information and data. Use cases are explained for Transportation, Healthcare and Education fields along with the business models.
A Sigh of Relief for Patients with Chronic DiseasesHCL Technologies
This paper presents a solution for remote health monitoring for chronic diseases like chronic diseases like diabetes, asthma, cardiac arrhythmia, sleep disorders, and hypertension. It commences with the definitions for a better understanding of the terms used, and then excavates into perceiving what technology-enabled care is.
A simple solution that can utilize data, tap into social sentiments and provide business value to mobile users is much desired. Social data can be tapped for both society and business, and everyone is looking for an application that can address both. This paper analyzes a working solution, its tenets and features, and also indulge in a bit of future gazing.
A Novel Design Approach for Electronic Equipment - FEA Based MethodologyHCL Technologies
This paper describes the design approach established to study and simulate the vibration behavior of electronic products and provide good correlation between test data and FE simulation through a well calibrated analytical model. This established and validated approach/methodology has been practically implemented in various real time projects for various HCL clients, thereby eliminating/minimizing the actual hardware testing and prototyping efforts resulting in a significant reduction in turnaround time, increased product cost savings and improved productivity.
Due to the phenomenal development of Networking technology, applications and other services, IP networks are preferred for communication, but are more vulnerable to attacks. To cope with the growing menace of security threats, security systems have to be made more intelligent and robust by introducing Intrusion Detection Systems (IDS) in the security layers of a network.
This white paper explores the role of IDS to detect attacks accurately at an early stage to minimize the impact.
Manufacturing though is increasingly being outsourced, developed countries like Germany are toying with the idea of digitizing the entire process to bring down costs and enhance efficiencies. Learn how Germany is doing it, through Industry 4.0.
The financial services industry has never had a better opportunity to embrace a customer-centric approach to doing business. Raising the bar for customer experience can create clear competitive advantages, and a responsive digital channel offering is essential. Here, insiders from the banking industry, the insurance sector and HCL Technologies’ customer experience management principal discuss the challenges of remaining agile in the digital space.
http://www.hcltech.com/financial-services/cxstudio
Digital Customer Care Solutions, Smart Customer Care Solutions, Next Gen Cust...HCL Technologies
Many banking and financial institutions do not want to embrace digital CRM thinking things have remained same for long and will stay so. Though some are willing, they are reluctant because of organizational flexibility. Experts at HCL tell how organizations can overcome this impediment with HCL’s solutions
The Internet of Things. Wharton Guest Lecture by Sandeep Kishore – Corporate ...HCL Technologies
Internet became mainstream around 20 years ago and the rapid pace of technology development we have seen over these years is fascinating. We are now looking at the biggest revolution ever, in the world of connectivity - Internet of Things (IoT). Everything we can think of around us - at home, work, in the car, or at a retail store -- will be interconnected, exchanging data and information, thus leading to an extremely intelligent network of things. It will lead to richer user experience, improved efficiencies and higher collaboration across the ecosystem. Exciting times await us...
Be Digital or Be Extinct. Wharton Guest Lecture by Sandeep Kishore – Corporat...HCL Technologies
The era of Digital Darwinism is upon us. Businesses have no choice but to adopt digital technologies or disappear. Traditional businesses which do not leverage digital technologies risk becoming irrelevant or losing business to native digital companies which understand technology better. The good news is that most companies realize the importance of digital. However, the not so good news is that many still approach digital as “nice” or “cool to have” rather than treating it as an important aspect for business. Digital is no longer good to have: either be digital or be extinct – there is simply no other option.
3.0 Project 2_ Developing My Brand Identity Kit.pptxtanyjahb
A personal brand exploration presentation summarizes an individual's unique qualities and goals, covering strengths, values, passions, and target audience. It helps individuals understand what makes them stand out, their desired image, and how they aim to achieve it.
Enterprise Excellence is Inclusive Excellence.pdfKaiNexus
Enterprise excellence and inclusive excellence are closely linked, and real-world challenges have shown that both are essential to the success of any organization. To achieve enterprise excellence, organizations must focus on improving their operations and processes while creating an inclusive environment that engages everyone. In this interactive session, the facilitator will highlight commonly established business practices and how they limit our ability to engage everyone every day. More importantly, though, participants will likely gain increased awareness of what we can do differently to maximize enterprise excellence through deliberate inclusion.
What is Enterprise Excellence?
Enterprise Excellence is a holistic approach that's aimed at achieving world-class performance across all aspects of the organization.
What might I learn?
A way to engage all in creating Inclusive Excellence. Lessons from the US military and their parallels to the story of Harry Potter. How belt systems and CI teams can destroy inclusive practices. How leadership language invites people to the party. There are three things leaders can do to engage everyone every day: maximizing psychological safety to create environments where folks learn, contribute, and challenge the status quo.
Who might benefit? Anyone and everyone leading folks from the shop floor to top floor.
Dr. William Harvey is a seasoned Operations Leader with extensive experience in chemical processing, manufacturing, and operations management. At Michelman, he currently oversees multiple sites, leading teams in strategic planning and coaching/practicing continuous improvement. William is set to start his eighth year of teaching at the University of Cincinnati where he teaches marketing, finance, and management. William holds various certifications in change management, quality, leadership, operational excellence, team building, and DiSC, among others.
Kseniya Leshchenko: Shared development support service model as the way to ma...Lviv Startup Club
Kseniya Leshchenko: Shared development support service model as the way to make small projects with small budgets profitable for the company (UA)
Kyiv PMDay 2024 Summer
Website – www.pmday.org
Youtube – https://www.youtube.com/startuplviv
FB – https://www.facebook.com/pmdayconference
"𝑩𝑬𝑮𝑼𝑵 𝑾𝑰𝑻𝑯 𝑻𝑱 𝑰𝑺 𝑯𝑨𝑳𝑭 𝑫𝑶𝑵𝑬"
𝐓𝐉 𝐂𝐨𝐦𝐬 (𝐓𝐉 𝐂𝐨𝐦𝐦𝐮𝐧𝐢𝐜𝐚𝐭𝐢𝐨𝐧𝐬) is a professional event agency that includes experts in the event-organizing market in Vietnam, Korea, and ASEAN countries. We provide unlimited types of events from Music concerts, Fan meetings, and Culture festivals to Corporate events, Internal company events, Golf tournaments, MICE events, and Exhibitions.
𝐓𝐉 𝐂𝐨𝐦𝐬 provides unlimited package services including such as Event organizing, Event planning, Event production, Manpower, PR marketing, Design 2D/3D, VIP protocols, Interpreter agency, etc.
Sports events - Golf competitions/billiards competitions/company sports events: dynamic and challenging
⭐ 𝐅𝐞𝐚𝐭𝐮𝐫𝐞𝐝 𝐩𝐫𝐨𝐣𝐞𝐜𝐭𝐬:
➢ 2024 BAEKHYUN [Lonsdaleite] IN HO CHI MINH
➢ SUPER JUNIOR-L.S.S. THE SHOW : Th3ee Guys in HO CHI MINH
➢FreenBecky 1st Fan Meeting in Vietnam
➢CHILDREN ART EXHIBITION 2024: BEYOND BARRIERS
➢ WOW K-Music Festival 2023
➢ Winner [CROSS] Tour in HCM
➢ Super Show 9 in HCM with Super Junior
➢ HCMC - Gyeongsangbuk-do Culture and Tourism Festival
➢ Korean Vietnam Partnership - Fair with LG
➢ Korean President visits Samsung Electronics R&D Center
➢ Vietnam Food Expo with Lotte Wellfood
"𝐄𝐯𝐞𝐫𝐲 𝐞𝐯𝐞𝐧𝐭 𝐢𝐬 𝐚 𝐬𝐭𝐨𝐫𝐲, 𝐚 𝐬𝐩𝐞𝐜𝐢𝐚𝐥 𝐣𝐨𝐮𝐫𝐧𝐞𝐲. 𝐖𝐞 𝐚𝐥𝐰𝐚𝐲𝐬 𝐛𝐞𝐥𝐢𝐞𝐯𝐞 𝐭𝐡𝐚𝐭 𝐬𝐡𝐨𝐫𝐭𝐥𝐲 𝐲𝐨𝐮 𝐰𝐢𝐥𝐥 𝐛𝐞 𝐚 𝐩𝐚𝐫𝐭 𝐨𝐟 𝐨𝐮𝐫 𝐬𝐭𝐨𝐫𝐢𝐞𝐬."
In the Adani-Hindenburg case, what is SEBI investigating.pptxAdani case
Adani SEBI investigation revealed that the latter had sought information from five foreign jurisdictions concerning the holdings of the firm’s foreign portfolio investors (FPIs) in relation to the alleged violations of the MPS Regulations. Nevertheless, the economic interest of the twelve FPIs based in tax haven jurisdictions still needs to be determined. The Adani Group firms classed these FPIs as public shareholders. According to Hindenburg, FPIs were used to get around regulatory standards.
Company Valuation webinar series - Tuesday, 4 June 2024FelixPerez547899
This session provided an update as to the latest valuation data in the UK and then delved into a discussion on the upcoming election and the impacts on valuation. We finished, as always with a Q&A
Putting the SPARK into Virtual Training.pptxCynthia Clay
This 60-minute webinar, sponsored by Adobe, was delivered for the Training Mag Network. It explored the five elements of SPARK: Storytelling, Purpose, Action, Relationships, and Kudos. Knowing how to tell a well-structured story is key to building long-term memory. Stating a clear purpose that doesn't take away from the discovery learning process is critical. Ensuring that people move from theory to practical application is imperative. Creating strong social learning is the key to commitment and engagement. Validating and affirming participants' comments is the way to create a positive learning environment.
B2B payments are rapidly changing. Find out the 5 key questions you need to be asking yourself to be sure you are mastering B2B payments today. Learn more at www.BlueSnap.com.
2. 1. Abstract
It is not only desirable but also necessary to assess the quality of
testing being delivered by a vendor. Specific to software testing,
there are some discerning metrics that one can look at, however it
must be kept in mind that there are multiple factors that affect these
metrics which are not necessarily under the control of testing team.
The SLAs for testing initiatives can, and should, only be committed
after a detailed understanding of the customer’s IT organization in
terms of culture and process maturity and after analyzing the
various trends among these metrics. This white paper lists some of
the popular testing metrics and the factors one must keep in mind
while reading in to their values.
2. Introduction
This white paper discusses some of the popular metrics for testing
outsourcing engagements and the factors one must keep in mind
while looking at the values of these metrics.
Metric 1.Residual defects after a testing stage
?
Definition
The absolute number of defects that are detected after the testing
stage (owned by the vendor’s testing team).
The lower the number of defects found after the current testing
stage the better the quality of testing is considered.
Factors to consider
? requirements
Quality of
The ambiguity in the requirements results in misinterpretations and
misunderstandings, leading to ineffectiveness in defect detection.
The clearer the requirements the higher the chances of testing team
understanding it right and hence noticing any deviations or defects
in the system under test (SUT).
? development
Quality of
The planning for testing is usually done with an assumption that the
system will be thoroughly unit tested prior to handling it to the
testing team. However if the quality of the development process is
poor and if the unit testing is not thoroughly done, the testing team
is likely to be encountering more of unit-level defects and might be
2
3. pausing their testing for the defects (even in the fundamental
processes) to get fixed and hence wouldn’t be able to devote/focus
all of their time in looking for functional-level/system-
level/business-level defects.
? of Business users for reviews and clarifications
Availability
If the Business users are not available for answering the queries in
time, the testing team is likely to struggle in establishing the
intended behavior of the system under test (whether it is a defect or
not etc.) and hence some amount of productivity would be lost and
it is likely to have more defects remaining by the time the testing
window is used up.
? test cycles - Delay in defect fixes, higher defect fixes,
Incomplete
environment availability
The estimates and planning for testing is based on certain
assumptions and available historical data. However if there are
higher number of disruptions (than anticipated) to testing in terms
of environment unavailability or higher number of defects being
found and fixed, the quality time available for testing the system
would be less and hence higher number of defects slip through the
testing stage.
Metric 2. Test effectiveness of a testing stage (or) Defect
Containment effectiveness.
?
Definition
The % of system level defects that have slipped through the testing
stage (owned by the vendor’s testing team) and are detected at a later
testing stage or in Production.
The higher the effectiveness the lower the chances of defects being
found downstream.
Test Stage Effectiveness =
(Defects detected in the current testing stage)
-------------------------------------------------------------
(Detects detected in current testing stage + defects detected in all
subsequent stages)
Factors to consider
? of accurate defects data at all stages
Availability
We must ensure that the data on defects on all subsequent stages are
also available and are accurate. Production defects are usually
handled by a separate Production support team and testing team is
3
4. at times not given much insight in to this data. Also, since multiple
projects and/or Programs would be going live, one after another,
there are usually challenges in identifying which defects in
Production can be attributed to which Project or Program.
Inaccuracies in assignment would lead to inaccurate measure of test
stage effectiveness.
All the factors listed above for ‘Residual defects’ also apply for this
metric.
Metric 3. % improvement in test case productivity
?
Definitions
Test case Productivity = # of test cases developed per person per
day (or per hour)
% Improvement =
(Test case productivity in current year – last year)
= --------------------------------------------------------- x 100
(Initial Test Case Productivity last year)
Factors to consider
? changes
Nature of
The nature of changes that the system goes through might not
necessarily be comparable all the time. Depending on the nature of
changes the # of test cases required to test a unit of development
effort, and the amount of investigation/analysis required prior to
deciding or documenting the test cases would differ drastically.
? definition
Test case
The very measurement of a ‘test case’ itself could be a challenge. A
test case can range from very simple to very complex depending on
the specifics of the test objective. Hence a mere “count of the test
cases’ might not reflect the actual effort put in testing a particular
change or functionality. We must consider the complexity of test
cases as well. Also different teams might be following different
levels of documentation for ‘a test case’. For example a test case
with 12 conditions might be considered as a single test case by one
team while the other may split them in to 12 separate test cases with
one condition each. Normalizing the ‘unit’ of test case is very
critical to get this metric representing the real picture.
? on effort categorization
Ambiguity
Some people might consider test data set up as part of test case
4
5. development effort while some consider it as part of test execution
and set up. Different people following different notations would
lead to erroneous values and the data might not be comparable.
? Effectiveness of test cases
Testers might be incorrectly motivated towards creating ‘lots’ of
test cases in less time rather than taking time to think through the
changes and requirements and come up with good test cases (even
if they are only few) that are likely to find defects or are likely to give
the Business people more comfort/confidence.
?in experience/SME
Increase
Over a period of time a testing resource is likely to become more
knowledgeable about the SUT. Due to this he/she would be in a
position to better expect which test cases are likely to find a defect
and thus might cut down on test cases that are NOT likely to find
defects.
? Testing Resources
Change in
Whenever a resource is replaced by another it is likely that the new
resource would take more time for doing the analysis needed to
write the test cases. The higher the number of changes the lower the
test case productivity (and also test case effectiveness) in a way.
Metric 4. % reduction in testing cost
?
Definitions
(Testing cost per unit dev effort in curr year – last year)
= ----------------------------------------------------------
(Testing cost per unit development effort last year)
Factors to consider
? Lack of accurate measurement of development efforts
This metric heavily depends on the measurement of actual
development effort which might not be accurate. The number of
projects that have formal measurement units such as FP is relatively
few.
?
Testing effort variance with dev effort
Testing effort might not always directly proportional to the
development effort. For example a slight modification (with very
small development effort) to a legacy application might incur a lot
of testing effort factoring in the regression testing etc. Similarly
5
6. there may be lot of structural changes to the existing code with high
development efforts (for performance enhancement or some other
refactoring needs) but the (black box) testing effort might be not be
as high if the end functionality is not undergoing a drastic change.
? outsourcing
Scope of
When we engage with a customer for testing projects we might start
with a small set of applications or modules with in. However over a
period of time we would have started servicing more applications
or modules and the timesheet entries might not be granular enough
to distinguish between efforts for different modules. After some
point of time it might be difficult to extract the actual effort that is
spent on a particular module and hence to calculate the reduction in
effort for the same module.
? resources
Sharing of
Along the same lines as above it might be difficult to extract the
information as to which resource spent how much time on which
application as they might be working on different modules
interchangeably.
? complexity not comparable
Projects’
It is also common to encounter projects with complexities widely
differing from the projects that were used for base lining. Hence
comparing the data between these types of projects might not give
the real picture.
Metric 5. % Automated
?
Definitions
% of test cases automated
# of test cases automated
= --------------------------------------- x 100
Total # of test cases
Factors to consider
? case
Business
Not every application and at times not every module within an
application (and even not every test case within that module) has
business case for automation and they might continue to be tested
manually for multiple business reasons (e.g. possible replacement
of application with a COTS package in the next 6 months).
6
7. ? of automation
Feasibility
Depending on the technology constraints and the suitability of
tools that are available, some parts of an application might not be
technically feasible to automate even if we wanted/needed to.
?
Scope
It is advisable to measure % of automation under the revised scope:
“For the modules/applications that are known to give good RoI and
are technically feasible – what % is automated’.
? allocation
Resource
Industry survey reveals that over 60% of automation projects are
not successful and the major causes for this are: automation
attempted on an ad-hoc basis and people not dedicated for
automation. People also underestimate the need for an effective on-
going maintenance after the initial test bed is automated. If
automation is approached immaturely one can expect disruptions in
build and maintenance of automation due to lack of constant focus
and supporting skills/resources for automation.
? exploratory testing
Room for
At times it may be desirable to give allowance for a bit of
exploratory testing to check out if we can detect any anomalies that
could not be detected by regular testing techniques. Usually in
exploratory testing, test execution is attempted along the lines of a
few test objectives. This is then followed by documentation of the
attempts and results. This could lead to addition of a few test cases
which might not necessarily be automated but are desired to ensure
more probing in to the application behavior.
Metric 6. Requirements coverage
Definitions
% of requirements that are covered by test cases.
(# Of requirements covered by at least 1 test case)
= -------------------------------------------------------- x 100
(Total # of requirements)
Factors to consider
? requirements, traceability matrix
Format of
Not every team might be following standard notations while
documenting the requirements and hence one can expect challenges
in mapping test cases to free-format requirements.
7
8. ? Legacy systems with no documented requirements
Most of the legacy systems might not have any documentation on
the basic functionality thus making the reference point difficult to
establish.
?
Configuration Management
The team may not be having access to a configuration management
tool that enables keeping the test cases systematically mapped with
changing requirements. The challenge lies in managing not only the
version of requirements but also the corresponding version of test
cases.
Metric 7. Test case effectiveness
Definitions
(# of Test Cases that detected defects)
(Total # of Test cases)
Factors to consider
? application
Stability of
As the application or SUT stabilizes over a period of time the
number of defects in the system goes down and it require more
effort (test cases) to find remaining defects.
?
Risk
Where people try to over emphasize this metric and refrain from
writing more test cases - there may be a risk of not detecting some
of defects we could otherwise find (thus resulting in bad test stage
effectiveness). In order to reduce this risk people might try and
write more and more test cases even though the possibility of
finding a defect might be low.
Metric 8. Defect Rejection Ratio
A defect initially raised by a tester could be later rejected for multiple
reasons and the main objective of having this metric is to ensure the
testers correctly understand the application/requirements and do
their ground work well before everyone’s time and attention is
invested in solving the defect. Too many defects being rejected
results in inefficiency (due to time and effort spent on something
that wasn’t a problem).
8
9. ?
Definitions
(Defects rejected as invalid)
= ---------------------------------
(Total no. of defects raised)
Factors to consider
? rejection
Cause of
Lack of application knowledge is usually the cause for rejected
defects. However there could be other reasons as well, such as
misinterpretations, changes in the environments and defects
becoming non-reproducible. Hence we need to take in to
consideration the causes for rejection in order to correctly
understand the trends.
? cycle culture
Defect life
In some teams a defect is initially raised in the system as ‘New’
followed by a discussion and then ‘rejected’ off the system if it is
not considered a defect. However in some teams discussions are
held between testers and other stakeholders such as Business and
Development on the ‘possible’ defects and a defect is entered in to
the system only after it is confirmed during the discussions. Hence
there won’t be a rejection at all.
?
Fear of risk
In some cases it might not be very straightforward to decide on
whether a behavior is a defect or not and one has to choose between
risking a defect rejection but covering the ‘defect slippage’ and not
raising the defect (lower defect rejection) but higher defect slippage
in to the next stage. Usually people are more concerned about the
defect slippage and hence take the route of raising defects even if it
means higher defect rejection.
? requirements
Quality of
The higher the quality of requirements the higher the chances of
elimination of misunderstanding and lower the defect rejection
rate. Even if the defect rejection rate is higher it will at least help in
establishing the fact that tester’s knowledge was not sufficient and
that requirements could not be blamed.
? environment
Control of
At times the behavior of the system changes from the time a defect
is raised to the time the verification of the defect is done (‘it didn’t
work when I raised the defect but now it is working. Don’t know
9
10. how.’) If the environment is undergoing changes with out the
notice/control of testing team it would be hard to establish the
cause for the defect rejection.
Metric 9. Currency of knowledge database
?
Definitions
% of application knowledge that is documented.
(Knowledge level documented)
= ------------------------------------------------------------
(Total Knowledge of the Application or Module)
Factors to consider
? Quantifying ‘Knowledge’
The parameter ‘Knowledge’ is hard to quantify and is mostly a
qualitative measure. At times people do graduate their
applications/modules in to a list of functions/transactions/flows
and the documentation for these is measured relatively objectively.
However not every module might be conducive for this kind of
graduation and care must be taken to ensure that some sort of
classification is possible before we attempt to measure this metric.
? details expected in documentation
Depth of
Also a functionality/transaction can very well be documented at
different levels of depth – the deeper the documentation the higher
the time involved in it, ranging from a few minutes to a few days. We
must ensure that the expectations on ‘depth’ are understood
correctly before we commit a certain SLA on this metric.
? of changes
Frequency
During estimations for documentation efforts a certain amount of
changes to baseline requirements/functionality would be factored
in. However if the changes are much higher than estimated this has
an obvious impact on the actual effort for documentation.
?
Expectation on Time of Documentation
It is also important to know the actual ‘point’ in life cycle at which
the completed documentation is “expected”. If the documentation
is expected to be reviewed/refined at the end of each project (i.e. by
Phase and not by duration such as 3 months or 6 months) then it is
very likely that people do not attempt to document things ‘during’
the middle of a project thus avoiding any rework (due to changes in
10
11. requirements). However if the expectation is that the
documentation is updated every 3 months (or even 6 months) it is
possible that the project is going through some changes in
requirements and hence one should anticipate rework in
documentation.
?Projects on same application/module
Multiple
It is also likely that the same application might be undergoing
changes by multiple projects with a little bit of time gap between
each. This brings in all the complexity that applies to configuration
management of the ‘code’. The documentation must reflect the
changes with respect to the Projects so people of other projects also
know where to apply their changes and by how much. In the
absence of a configuration management tool the documentation
might be a little difficult to handle without leading to
confusion/rework.
? for documentation effort
Planning
Testing usually encounters tight schedules with racing against time
becoming the norm. Due to this, whether documentation of
application knowledge is a paid service or a value addition, one
cannot achieve it without factoring the time and effort (for
documentation) during the Project planning.
3. Conclusion
There are some discerning metrics for testing engagements which
can be considered for drafting SLAs. However for each metric one
must take in to consideration the factors (including other metrics)
that influence the value of metric and the scope of the testing team
in controlling those factors. This white paper lists down some of
the popular metrics and discusses about the factors that affect each
of these metrics.
One shouldn’t underestimate the fact that it usually takes some time
(a few months to over a year) in order to establish consistent trends
on these metrics. It is thus recommended that enough time is given
to assess the trends of these metrics before an SLA is worked upon
for the engagement.
Where applicable, ‘assumptions’ must clearly be documented to
indicate/reflect the influencing factors on the SLA’s being
committed.
11
12. Hello there. I am from HCL Technologies. We work behind the scenes, helping our
customers to shift paradigms and start revolutions. We use digital engineering to
build superhuman capabilities. We make sure that the rate of progress far
exceeds the price. And right now, 59000 of us bright sparks are busy developing
solutions for 500 customers in 20 countries across the world.
How can I help you?
transform@hcl.in
12