© 2020 Perforce Software, Inc.
Classification of AI and ML
Tools in DevOps
E R A N K I N S B R U N E R
3 | DevOps Next 2020 perforce.com
Confidentiality Statement
The information contained in this document is strictly confidential, privileged, and
only for the information of the intended recipient. The information contained in this
document may not be otherwise used, disclosed, copied, altered, or distributed
without the prior written consent of Perforce Software, Inc.
ABOUT ME:
Eran Kinsbruner
• Chief Evangelist, Product Manager, and Author at Perfecto by Perforce
• Blogger, Inventor, and Speaker
• 20+ years in software development & testing
• Author of:
• The Digital Quality Handbook
• Continuous Testing for DevOps Professionals
• Accelerating Software Quality
• Twitter: @ek121268
5 | DevOps Next 2020 perforce.com
Today’s Agenda
1
2
Introduction to AI/ML
DevOps Objectives and the Case for AI/ML
3 Classification of Key AI/ML Tools In DevOps
6 Q&A
5 Not Covered: RPA, Automated Code Reviews, Code Fuzzing
and ML, Defects Classifications, Test Management
4 Additional AI/ML Use Case Examples
6 | DevOps Next 2020 perforce.com
AI and ML
7 | DevOps Next 2020 perforce.com
• A compound of development (Dev) and operations (Ops), DevOps is the union of people, process, and technology to
continually provide value to customers.
• What does DevOps mean for teams? DevOps enables formerly siloed roles—development, IT operations, quality
engineering, and security—to coordinate and collaborate to produce better, more reliable products. By adopting a
DevOps culture along with DevOps practices and tools, teams gain the ability to better respond to customer needs,
increase confidence in the applications they build, and achieve business goals faster.
DevOps Objectives
Source: https://azure.microsoft.com/en-us/overview/what-is-devops/
8 | DevOps Next 2020 perforce.com
The Impact of AI and ML on DevOps
9 | DevOps Next 2020 perforce.com
What Roles Will AI and ML Play in DevOps?
10 | DevOps Next 2020 perforce.com
DIFFERENTIAL VISUAL DECLARATIVE
Classifications of AI and ML Tools
Image-Based Learning
• Screen Comparisons
• Look and Feel
• UI Design, Accessibility
Comparing application
versions over builds:
• Recognizing changes
• Classifying the differences
• Bugs vs. features
ANALYTICSSELF-HEALING
Specifying Test Intent
• Goal-Based Automation
• Natural Language Processing
• Domain-Specific Languages
Autocorrecting Test Scripts
• Element Location
• Improved Maintenance
• Improved Robustness
Autocorrecting Test Scripts
• Root Cause Analysis
• Test Selection/Prioritization
• Test Flakiness
11 | DevOps Next 2020 perforce.com
DIFFERENTIAL
• The tools in this segment aim to proactively and automatically identify code quality issues, regressions, and security
vulnerabilities through code scanning, unit test automated creations, etc.
Classification of AI and ML Tools in DevOps
12 | DevOps Next 2020 perforce.com
VISUAL
Classification of AI and ML Tools in DevOps
13 | DevOps Next 2020 perforce.com
DECLARATIVE
• Test Automation Methods:
• NLP (Functionize)
• MBTA (TestModeller.io)
• RPA (UIPath)
Classification of AI and ML Tools in DevOps
14 | DevOps Next 2020 perforce.com
DECLARATIVE
Classification of AI and ML Tools in DevOps
15 | DevOps Next 2020 perforce.com
SELF-HEALING
Classification of AI and ML Tools in DevOps
16 | DevOps Next 2020 perforce.com
ANALYTICS
Classification of AI and ML Tools in DevOps
17 | DevOps Next 2020 perforce.com
Classification of AI and ML Tools in DevOps – Chatbots Testing
18 | DevOps Next 2020 perforce.com
Classification of AI and ML Tools in DevOps – Observability
Clustering Logs by RulesLogs Filtered By Time and Message
19 | DevOps Next 2020 perforce.com
Classification of AI and ML Tools in DevOps – AIOps
Source: ScienceLogic Source: Medium.com
20 | DevOps Next 2020 perforce.com
Classification of AI and ML Tools in DevOps – TIA
21 | DevOps Next 2020 perforce.com
DevOps is Still DevOps – AI/ML Should Fit in the Process
Classify
Classify the
pain and
category
Identify
Identify a POC
that makes
sense (pain
subset criteria)
Validate
Validate
solutions (skills,
org. fit)
Expand
Expand
solution to
solve the bigger
problem
Move
Move to
additional
pains
22 | DevOps Next 2020 perforce.com
DevOps is Still DevOps – AI/ML Should Fit in the Process
Classify the pain
and category
Identify a POC
that makes sense
(pain subset
criteria)
Validate
solutions
(skills, org. fit)
Expand solution
to solve the
bigger problem
Move to
additional
pains
Pick a
solution
DIY – Do it Yourself
OSS – Open Source
COTS – Off the shelf
© 2020 Perforce Software, Inc.
UP NEXT…
Advancing the State of The Art in
AI and Testing
Thank You!

Classification of Advanced AI and ML Testing Tools - DevOps Next

  • 2.
    © 2020 PerforceSoftware, Inc. Classification of AI and ML Tools in DevOps E R A N K I N S B R U N E R
  • 3.
    3 | DevOpsNext 2020 perforce.com Confidentiality Statement The information contained in this document is strictly confidential, privileged, and only for the information of the intended recipient. The information contained in this document may not be otherwise used, disclosed, copied, altered, or distributed without the prior written consent of Perforce Software, Inc.
  • 4.
    ABOUT ME: Eran Kinsbruner •Chief Evangelist, Product Manager, and Author at Perfecto by Perforce • Blogger, Inventor, and Speaker • 20+ years in software development & testing • Author of: • The Digital Quality Handbook • Continuous Testing for DevOps Professionals • Accelerating Software Quality • Twitter: @ek121268
  • 5.
    5 | DevOpsNext 2020 perforce.com Today’s Agenda 1 2 Introduction to AI/ML DevOps Objectives and the Case for AI/ML 3 Classification of Key AI/ML Tools In DevOps 6 Q&A 5 Not Covered: RPA, Automated Code Reviews, Code Fuzzing and ML, Defects Classifications, Test Management 4 Additional AI/ML Use Case Examples
  • 6.
    6 | DevOpsNext 2020 perforce.com AI and ML
  • 7.
    7 | DevOpsNext 2020 perforce.com • A compound of development (Dev) and operations (Ops), DevOps is the union of people, process, and technology to continually provide value to customers. • What does DevOps mean for teams? DevOps enables formerly siloed roles—development, IT operations, quality engineering, and security—to coordinate and collaborate to produce better, more reliable products. By adopting a DevOps culture along with DevOps practices and tools, teams gain the ability to better respond to customer needs, increase confidence in the applications they build, and achieve business goals faster. DevOps Objectives Source: https://azure.microsoft.com/en-us/overview/what-is-devops/
  • 8.
    8 | DevOpsNext 2020 perforce.com The Impact of AI and ML on DevOps
  • 9.
    9 | DevOpsNext 2020 perforce.com What Roles Will AI and ML Play in DevOps?
  • 10.
    10 | DevOpsNext 2020 perforce.com DIFFERENTIAL VISUAL DECLARATIVE Classifications of AI and ML Tools Image-Based Learning • Screen Comparisons • Look and Feel • UI Design, Accessibility Comparing application versions over builds: • Recognizing changes • Classifying the differences • Bugs vs. features ANALYTICSSELF-HEALING Specifying Test Intent • Goal-Based Automation • Natural Language Processing • Domain-Specific Languages Autocorrecting Test Scripts • Element Location • Improved Maintenance • Improved Robustness Autocorrecting Test Scripts • Root Cause Analysis • Test Selection/Prioritization • Test Flakiness
  • 11.
    11 | DevOpsNext 2020 perforce.com DIFFERENTIAL • The tools in this segment aim to proactively and automatically identify code quality issues, regressions, and security vulnerabilities through code scanning, unit test automated creations, etc. Classification of AI and ML Tools in DevOps
  • 12.
    12 | DevOpsNext 2020 perforce.com VISUAL Classification of AI and ML Tools in DevOps
  • 13.
    13 | DevOpsNext 2020 perforce.com DECLARATIVE • Test Automation Methods: • NLP (Functionize) • MBTA (TestModeller.io) • RPA (UIPath) Classification of AI and ML Tools in DevOps
  • 14.
    14 | DevOpsNext 2020 perforce.com DECLARATIVE Classification of AI and ML Tools in DevOps
  • 15.
    15 | DevOpsNext 2020 perforce.com SELF-HEALING Classification of AI and ML Tools in DevOps
  • 16.
    16 | DevOpsNext 2020 perforce.com ANALYTICS Classification of AI and ML Tools in DevOps
  • 17.
    17 | DevOpsNext 2020 perforce.com Classification of AI and ML Tools in DevOps – Chatbots Testing
  • 18.
    18 | DevOpsNext 2020 perforce.com Classification of AI and ML Tools in DevOps – Observability Clustering Logs by RulesLogs Filtered By Time and Message
  • 19.
    19 | DevOpsNext 2020 perforce.com Classification of AI and ML Tools in DevOps – AIOps Source: ScienceLogic Source: Medium.com
  • 20.
    20 | DevOpsNext 2020 perforce.com Classification of AI and ML Tools in DevOps – TIA
  • 21.
    21 | DevOpsNext 2020 perforce.com DevOps is Still DevOps – AI/ML Should Fit in the Process Classify Classify the pain and category Identify Identify a POC that makes sense (pain subset criteria) Validate Validate solutions (skills, org. fit) Expand Expand solution to solve the bigger problem Move Move to additional pains
  • 22.
    22 | DevOpsNext 2020 perforce.com DevOps is Still DevOps – AI/ML Should Fit in the Process Classify the pain and category Identify a POC that makes sense (pain subset criteria) Validate solutions (skills, org. fit) Expand solution to solve the bigger problem Move to additional pains Pick a solution DIY – Do it Yourself OSS – Open Source COTS – Off the shelf
  • 23.
    © 2020 PerforceSoftware, Inc. UP NEXT… Advancing the State of The Art in AI and Testing
  • 24.