#Interactive Session by Seema Kohli, "Test Leadership in the Era of Artificial Intelligence" at #ATAGTR2023.

G lobal T esting Retreat
#ATAGTR2023
T E S T L E A D E R S H I P I N T H E E R A O F
A R T I F I C I A L I N T E L L I G E N C E
Author Name: Seema
Kohli
IBM India Pvt. Ltd
Global Testing Retreat
#ATAGTR2023

#ATAGTR2023
G lobal Testing Retreat
#ATAGTR2023
E V O L V I N G R O L E O F T E S T L E A D E R S H I P
Test Leadership role and strategy continues to evolve with the changing landscape and
technology
In this context:
 What does Artificial
Intelligence bring to table
when it comes to test
leadership?
 What shifts are foreseeable/
necessary to bring about the
successes?
Test
Leadership
Quality Reliability
Efficienc
y
Core Goals

#ATAGTR2023
#ATAGTR2023
T E S T L E A D E R S H I P : W H A T C H A N G E S W I T H A I ?
Generative AI is all set to bring-in transformational changes in the work horizon and
test leadership approach. Here, we delve into two pivotal themes pertinent to test
leadership in the era of AI.
Test Leadership Success
Factors
How is the test leadership
impacted
What do we need to know,
embrace and address to succeed
How will AI change the way test
leaders operate to succeed
How core tenets need to be
adapted
Embracing technological
advances

#ATAGTR2023
#ATAGTR2023
T E S T L E A D E R S H I P : H O W R E L E V A N T A R E
T R A D I T I O N A L S U C C E S S F A C T O R S
•:
Adaptability and Clarity for navigating
through and addressing the people factor.
 Navigating the AI phobia
 Assessing and managing AI-human
interactions
Traditional Success
factors
Adaptability
Clarity
Strategy
Expertise
Answer is a
Resounding YES
Are these
still
relevant?
How?
Expertise and Strategy as an enabler for
Decision-making.
 Continues to be an imperative in driving
successful outcomes.
 Scaling up to augment testing with AI
Let's look at
some
examples

#ATAGTR2023
#ATAGTR2023
Augmenting
Test Life Cycle
with AI
Documentation
Test Plans
Test Strategy
Test Reports
Test Deliverables
Test Scenarios
Test Data
Test Scripts
Data-Based
Decision making
Test Dashboards
Trend Analysis
Test Metrics
Utilities
Tooling Decisions
Template creation
Test Query Bots
E M B R A C I N G T H E T E C H N O L O G I C A L A D V A N C E S :
A U G M E N T A N D A C C E L E R A T E T E S T L I F E C Y C L E

#ATAGTR2023
#ATAGTR2023
T O O L S F O R B U I L D I N G A N D D E P L O Y I N G G E N A I
Off-the Shelf AI powered
tools
IBM Ignite
Katalon
Lambda Test
OpenText UFT One
Get ready-to-use solution
for your area of need
Infrastructure management
will depend on the
subscription plans
Build or Buy? Or Go Open
Source!
Cloud Based Platforms
IBM Watson
AWS Bedrock
Azure OpenAI
Google Cloud
Accelerated development
with pre-built tools and
services
Get Managed
Infrastructure, controlled
data and security features.
Framework-based
Development
LangChain
LlamaIndex
PyTorch
TensorFlow
Gain development control
using libraries and LLM APIs
Manage Infrastructure, review
and manage data and security
considerations
References: https://www.gartner.com/reviews/market/ai-augmented-software-testing-tools

#ATAGTR2023
#ATAGTR2023
P U B L I C L Y A V A I L A B L E P L A T F O R M S A S Y O U R P E R S O N A L
A S S I S T A N T !
Do’s
Use as a friendly assistant for search operations
Get template suggestions; test strategy, test plan, test traceability
Get assistance in decision making on tools/process selection. Get Tool
comparison and analysis
Use it for idea generation. Like: Ways to use GenAI for defect analysis and
prediction
Generate code snippets for common operations
Establish usage guidelines for your team: Data cleaning, removing IP,
confidential information before posting
Dont’s
Posting proprietary data: company data; financial data, personal data
Email edits, test plan/case refinements are pathways to information
leaks
Take the generated inputs at face value
ChatGPT
Bard
DALL-E2
Many
More!

#ATAGTR2023
#ATAGTR2023
G E N A I U S E C A S E S F O R S O F T W A R E T E S T I N G
Autogenerate minutes of meeting
Get extracts from lengthy docs via
summarization
Generate code faster with AI-
powered tools like GitHub Copilot
Efficient information retrieval via
chatbot
Retrieve data from vast knowledge
bases, code bases, status reports
Use as a handy project assistant for all your
information needs
Embed summarized data effortlessly in
test docs like test plans
Accelerate test planning
Focus solely on discussions and actions
No note taking, filtering, refining notes
Focus on application/system logic and
workflows
Get the mundane IDE, syntax, common
code addressed by your handy assistant

#ATAGTR2023
#ATAGTR2023
U S E C A S E D E M O
Versatile Chatbot
Assistant
 All these utilities are now
possible to be generated
in your workspaces with
tool of your choice
 One framework and vast
number of applications at
your disposal!
For the demo here, we are using LangChain framework

#ATAGTR2023
#ATAGTR2023
U S E C A S E D E M O

#ATAGTR2023
#ATAGTR2023
B E N E F I T S O F A I - P O W E R E D C H A T B O T
Versatile -- applied across multiple use
cases
Summarize lengthy
documents
Query knowledge base,
code base, test reports
Generate meeting
minutes
One Chatbot, Many Applications
Read multiple
file types

#ATAGTR2023
#ATAGTR2023
B E N E F I T S O F A I - P O W E R E D C H A T B O T : A S N A P S H O T O F
Q U A N T I T A T I V E B E N E F I T A N A L Y S I S
Use Case: Chatbot as an assistant to the delivery team
This is for Banking-Payments domain with a European Bank. Here, as part of the deliverables, team
needs to work through lengthy requirement documents and industry portals. Based on the inputs
gathered, the team participates in requirement jams and develops deliverables like use cases/ test
cases.
Checking through vast repository of documents is a time intensive effort. Having a tool that can sift
through the document repositories and give the inputs helps with effort savings and improving overall
team efficiencies.
Chatbot as an assistant to identify inputs from requirement documents and repositories while working on
project developers.
Refer a set of scenarios where this will be used in the table below.
Scenario Manual
Effort
(Hours)
Effort With
Chatbot
(Hours)
Effort
Savings
(%)
Get inputs while developing use cases 16 1 93%
Get inputs while developing test scenarios/cases/data/sample files 16 1 93%
Get inputs while developing sample files based on the test data 5 1 80%
Get inputs during meetings to check inputs on discussion items 1 0.5 50%

#ATAGTR2023
#ATAGTR2023
D E R I V E I M P A C T O N E S T I M A T I O N S A N D P L A N N I N G
Compare effort
As-Is vs New
Measure the efficiencies gained
Deploy New Process/Tool
With Gen AI at our disposal, the possibilities are endless. The world is your
canvas. How do we know the value derived?
Monitor the Changes
Embed the
differences
in
estimation
model and
planning

#ATAGTR2023
#ATAGTR2023
U S I N G L L M A P I S – D A T A S E C U R I T Y A N D P R I V A C Y ?
Data is stored for a stipulated period for
monitoring misuse (usually with an opt-out
option)
Data is not processed for training and fine-
tuning GPT models
LLM Source companies usually implement
security features (like SOC2 compliance in
case of OpenAI)
These must be reviewed when going for
enterprise-grade deployments
Privacy
Considerations when LLMs are accessed via API in your apps and
programs
Refer LLM security and privacy policies when you plan to deploy at enterprize level.
As an example, see OpenAIs privacy policy here: https://openai.com/enterprise-privacy
Security
Be aware of the LLM API policies Implement your own DS&P measures on your
data
Continuous Monitoring is
imperative
Points to Note
Reference
s
DS&P is assured not Guaranteed

#ATAGTR2023
#ATAGTR2023
W H A T A N D H O W
Hype Vs Reality
Testing Transformations
Charting the Future
Know the difference
Identify what is implementable
and how much with AI Identify the transformative
opportunities
Keep abreast with the shifts
and trends
Prepare to navigate in the
evolving landscape
Prioritize and Plan

#ATAGTR2023
#ATAGTR2023
E M B R A C I N G T H E T E C H N O L O G I C A L A D V A N C E S : H Y P E
V S R E A L I T Y
Hype Vs Reality
It is not a magical one-size fits
solution
It is innovations in progress at a rapid
pace
Not a silver bullet
More use cases will come along as we
continue to explore
It is revolutionary; not just transformative
Not a human replacement

#ATAGTR2023
#ATAGTR2023
T E S T I N G T R A N S F O R M A T I O N S
Plan &
Prioritize
Continuous
Improvemen
t
Pilot &
Rollout
Assess &
Strategize
Monitor &
Measure
Stay Tuned-
in on new
development
s
AI Use Cases
in the Industry
AI Tools and
frameworks
Innovations &
Trends
Transformation Life Cycle
Pre-AI Era
In the AI
Era
Plan &
Prioritize
Continuous
Improvemen
t
Pilot &
Rollout
Evaluate &
Prepare
Monitor &
Measure
Period
Changes happening at the speed of light; Stay ahead of the
curve

#ATAGTR2023
#ATAGTR2023
A U G M E N T A N D A C C E L E R A T E T E S T L I F E C Y C L E
Investment
(Cost /Effort)
Stakeholder
Buy-in
Change
Resistance
Evaluation
Upskilling
Tools and
Techniques
Data Sources &
Quality
Preparation
Security
Privacy
Ethicalit
y
Implementation
Charting the Future

#ATAGTR2023
#ATAGTR2023
Scale-up in
Know-how
Invest Time
to Learn
Identify
quick-wins
Strategize
and
Execute
Monitor
and
Measure
Spear-
ahead with
efficiencies
Onboard AI in
Testing: Step-by-
Step
T E S T L E A D E R S H I P S U C C E S S F A C T O R S
A D A P T I N G I N T H E C H A N G I N G L A N D S C A P E

#ATAGTR2023
#ATAGTR2023
C O N C L U S I O N
AI holds potential to
enhance Software testing
Not a one-size-fits-all
solution
Manage stakeholder
expectations
Plan based on human
capability and supplement
with AI
Know-how and right
perspective  enabler to
thrive

#ATAGTR2023
#ATAGTR2023

#ATAGTR2023
#ATAGTR2023
THANKS
Seema Kohli
www.linkedin.com/in/seema-kohli-aa752

#Interactive Session by Seema Kohli, "Test Leadership in the Era of Artificial Intelligence" at #ATAGTR2023.

Recommended

Recommended

More Related Content

Similar to #Interactive Session by Seema Kohli, "Test Leadership in the Era of Artificial Intelligence" at #ATAGTR2023.

Similar to #Interactive Session by Seema Kohli, "Test Leadership in the Era of Artificial Intelligence" at #ATAGTR2023. (20)

More from Agile Testing Alliance

More from Agile Testing Alliance (20)

Recently uploaded

Recently uploaded (20)

#Interactive Session by Seema Kohli, "Test Leadership in the Era of Artificial Intelligence" at #ATAGTR2023.