A presentation for the Ottawa AI/ML Meetup Group, Sept 9, 2019. A person’s Facebook newsfeed from their vacation is to the real-world as Kaggle competitions are to industrial ML/AI projects. This presentation shares with the audience some common challenges in ML/AI that real-world practitioners face while doing their craft, along with suggestions on how to address them. This is a non-technical presentation but does assume some knowledge of data science process and ML/AI.

1. Ericsson Internal | 2018-02-21
Practicalchallenges in
ML workflows
Some tips from working with Ericsson data sets
and use cases.
Steven Rochefort SA OSS PDU OSS S&T BX 2019-03-04

2. Ericsson Internal | 2018-02-21
— Steve Rochefort is hands-on data scientist
team leader, experienced leader and telecom
professional.
— An academic background in mathematics and
marketing research has been applied in various
ways over a 30+ year career in industry.
— Software & System development for
mission critical systems
— Innovation/advanced R&D
— Machine Learning/Data Science/AI
— Most recently developing innovative use cases
for telecom data sets at Ericsson (3+ years)
Bio

3. Ericsson Internal | 2018-02-21
ProcessofDataScience
Data science process flowchart from "Doing Data Science", Cathy O'Neil and Rachel Schutt, 2013
From <https://en.wikipedia.org/wiki/Data_analysis#/media/File:Data_visualization_process_v1.png>

4. Ericsson Internal | 2018-02-21
AProcessofDataScienceatEricsson

5. Ericsson Internal | 2018-02-21
DataScienceEffortallocation
1. Develop UC
2. Gain access to sample data
and validate UC potential
based on data exploration,
and deliver a report
3. Develop ML model and test
it
4. Deploy model in context of
solving the UC and/or
deliver dashboard /
visualization on UC
1
2
3
4
5%
70%
15%
10%
2-3 people working 4-5 months is a
complete cycle

6. Ericsson Internal | 2018-02-21
—ML is about finding useful
correlation between data
(features) and results (labels)
—Correlation is very useful to make
predictions and inferences, only
as long as the correlations persist
—If you want causation, find an
expert to make you some rules
Tip#1:CorrelationisnotCausation

7. Ericsson Internal | 2018-02-21
—Find out what is in the data
—Relationships
—Patterns
—Find out how the data relates to
the label
—Then…start building models
Tip#2:Lookatthedata…alot

8. Ericsson Internal | 2018-02-21
—More data is better than less data
—All the data is never a good thing
—Sample the data statistically to
make some inferences and
models about the whole data
Tip#3:SampleTheData
You can drink the ocean … but only cup by cup

9. Ericsson Internal | 2018-02-21
—Simulation
—Creating realistic data from
physical models
—Alteration
—Take existing data and overlay
a pattern or experience
—Simplification
—Create parallel way to
generate data
Tip#4:Ifyoudon’thavedata,
makeyourown

10. Ericsson Internal | 2018-02-21
—It is hard not to get a result after
using a ML algorithm or
Python/R script (GIGO)
—Validate the result against your
common sense to ensure it is a
correct answer
Tip#5:Validateyourresults
Even if everything adds up it can still be wrong

11. Ericsson Internal | 2018-02-21
—Try your data against a lot of
different algorithms and see
which ones work better.
—Tuning hyperparameters will
give a small improvement, but
not a lot usually.
—Choose the simplest algorithm
that works the best
Tip#6:Mostalgorithmswork
prettygoodoutofthebox

12. Ericsson Internal | 2018-02-21
—A ML that improves overall
performance of a UC by 5% could
be fantastic … or complete
rubbish.
—The usefulness of an algorithm is
based on how it solves a real-life
problem
Tip#7:Assessthealgorithms
usefulness

13. Ericsson Internal | 2018-02-21
—For unbalanced data sets,
accuracy (TP/#observations) is
not very useful
—For a 99-1 dataset split the
algorithm that says
everything is the majority set
is 99% accurate
—Consider using the F1 Score as a
measure (link)
Tip#8:Classification:Accuracyisrarelya
goodmeasureofperformance

14. Ericsson Internal | 2018-02-21
—Comparing regression
algorithms using RMSE, MAE,
R2, etc are goods way to see
which algorithm is performing
better than others.
—However the real measure is
how well the algorithm meets
the use case expectations on
performance
Tip#9:Regression:AlgorithmPerformance
doesnotimplyUseCasePerformance

15. Ericsson Internal | 2018-02-21
—The real world is messy and
tends to mess up many a good
theory.
—On the other hand, just because a
solution is not perfect, it can be
useful sometimes.
—Weight the benefit against the
utility obtained, not the original
goal.
Tip#10:ImperfectionCanstillbe
useful