This is a talk I gave on 2 Sept 2020 as part of Sofy.ai's webinar series.

1. © Hee-Meng Foo 2020

2. © Hee-Meng Foo 2020
A little about me
● Worked on Genetic Algorithms and
optimization problems in grad school
● Did a course in Neural Networks there too
● Been building/leading Quality/Test
engineering teams for about 14 years
● Yahoo for 6+6 yrs, Climate Corp 2.5 yrs
● Started looking into Data Science, Machine
Learning and Deep Learning again in 2015
on the side
If you’d like to connect, just search for
“Heemeng” on Linkedin and look for this

3. © Hee-Meng Foo 2020
What
Is
Quality?

4. © Hee-Meng Foo 2020
Quality
is in the
eye of
the
beholder

5. © Hee-Meng Foo 2020
Quality is about user perception
“Quality means providing value to the customer; that is, offering conditions of
product use or service that meet or exceed customer’s expectations, yet are still
affordable. Quality also takes into account the reduction of waste that a product
may cause to the environment or human society, yet still allowing the
manufacturing company to maintain customer satisfaction.”
Enrique Diaz, Geneva Business News, Aug 2014
In “Software Quality: The Top 10 Metrics to Build Confidence”, John Lafleur listed
down “Defect removal efficiency” as one of the metrics

6. © Hee-Meng Foo 2020
The Stages of MTTR
Mean Time to Resolution
(MTTR)
MTT
Detection
MTT
Know
MTT
Fix
MTT
Verify
Triage Isolate Diagnose

7. © Hee-Meng Foo 2020
The Stages of MTTR
Mean Time to Resolution
(MTTR)
MTT
Detection
MTT
Know
MTT
Fix
MTT
Verify
Triage Isolate Diagnose

8. © Hee-Meng Foo 2020
Why Bug Triage?
● Takes up a lot of time (at least 30mins every day)
● Takes up expensive resource
○ Test/QA manager’s time
○ Engineering manager’s time
○ Lead engineer’s time
○ Product manager’s time
● What usually happens is that not urgent bugs pile up (dirty dishes analogy)

9. © Hee-Meng Foo 2020
What this talk is about
● An exploration into the use of ML techniques to perform bug
triage
○ Getting the data
○ Data exploration
○ Using traditional ML techniques
○ Using Tensorflow 2.0
○ Using BlazingText from AWS Sagemaker (very briefly)
○ Using Google’s AutoML (Natural Language) (very briefly)
Key assumption: you understand the basics of Machine Learning esp Classification

10. © Hee-Meng Foo 2020
A General Recipe
● Obtaining and processing training data
● Formatting the data into the form needed for training
● Understanding the data
● Try out various ML algorithms to understand how they perform
○ Small data sets - traditional ML algorithms
○ Large data sets - Deep Neural Networks
● Hyperparameter tuning
● Package up and deploy

11. © Hee-Meng Foo 2020
Material for this talk
● GitLab repo - https://gitlab.com/foohm71/octopus2
● Medium Articles:
○ Building a deployable Jira Bug Classification Engine in
Tensorflow
○ Building a deployable Jira Bug Classification Engine using
Amazon Sagemaker
○ Building a deployable Jira Bug Classification Engine using Google
AutoML

12. © Hee-Meng Foo 2020
Deep Dive

13. © Hee-Meng Foo 2020
Some Key Takeaways
● Getting and cleaning data can be challenging
● Getting a good ML or DNN algorithm takes some work
● A basic deployable ML engine is not difficult to create
○ Getting it to production grade is not so simple - DevOps skills needed
● Cloud services are making building ML engines easier and easier
● At the end of the day, you need to understand your data

14. © Hee-Meng Foo 2020
Q&A