Machine Learning in the Data Science Context - Tech Talk to CSI

1. Big Data & Machine Learning
12/24/2017

2. Agenda
• Why Machine Learning ?
• What is Machine Learning ?
• Some ML Applications
• Data Science Pipeline
• Data -> Big Data
• Big Data -> Feature Selection
• Machine Learning Modelling
• Model Evaluation
• Inference/Analytics
• Summary
12/24/2017 2

3. Why Machine Learning ?
12/24/2017 3

4. … Why Machine Learning ?
12/24/2017 4

5. What is Machine Learning ?
12/24/2017 5

6. What is Machine Learning ?
12/24/2017 6

7. Some MLApplications
12/24/2017 7

8. Data Science Pipeline
12/24/2017 8

9. Data Science Pipeline
12/24/2017 9
Data Features Model Inference Decision
Evaluation

10. Data => Big Data
• Structured Data
• Unstructured Data
• Big Data
- Text Data
- Time Series Data
- Spatial/Location-based Data
- Image/Video/Audio Data
12/24/2017 10

11. Big Data
12/24/2017 11

12. Big Data => Feature Selection
• Simplification of models - easier to interpret
• Shorter training times
• To avoid the curse of dimensionality
• Enhanced generalization by reducing
overfitting(formally, reduction of variance)
12/24/2017 12

13. Big Data => Feature Selection Techniques
1. Subset selection
• Exhaustive
• Best first
• Simulated annealing
• Genetic algorithm
• Greedy forward selection
• Greedy backward elimination
• Particle swarm optimization
• Targeted projection pursuit
• Scatter Search
• Variable Neighborhood Search
2. Optimality criteria
3. Structure learning
12/24/2017 13

14. Machine Learning Modelling
Linear Vs Non-linear Models
12/24/2017 14

15. Linear Modelling
• Response = constant + parameter * predictor
+ ... + parameter * predictor
• Y = b o + b1X1 + b2X2 + ... + bkXk
• Y = b o + b1X1 + b2X1
2
12/24/2017 15

16. … Linear Modelling
12/24/2017 16

17. Non-Linear Modelling
• Models which are not Linear ;)
12/24/2017 17

18. Machine Learning Modelling
Deterministic Vs Stochastic Models
12/24/2017 18

19. Deterministic Modelling
12/24/2017 19
Modeling is done using deterministic
variables. Uncertainty is not captured

20. Stochastic Modelling
Models the real world uncertainty using
random variables
12/24/2017 20

21. Machine Learning Modelling
Parametric Vs Non- Parametric Models
12/24/2017 21

22. Parametric Modelling
• Data is behaved according to a probability
distribution
• No of parameters is constant
• Focused on group means
12/24/2017 22

23. Non-Parametric Modelling
• Do not assume a particular probability
distribution
• No of parameters grows with training
samples
• Focused on group medians
12/24/2017 23

24. Stochastic ML Modelling
Frequentist Vs Bayesian Models
12/24/2017 24

25. Frequentist ML Modelling
Maximum Likelihood Estimation(MLE)
You need to model your random variables realistically
- Discrete r.v
i.e : Bernouli/Binomial/Geometri/Poisson)
- Continuous r.v
i.e :
Uniform/Exponential/Gamma/Normal(Gaussian)
Explained in Regression Modeling – Probabilistic
Interpretation
12/24/2017 25

26. Bayesian ML Modelling
Very powerful modeling approach
Prior knowledge is incorporated 
Maximum-a-Posteriori(MAP)
12/24/2017 26

27. Bayesian ML Modelling
12/24/2017 27

28. Bayesian ML Modelling
12/24/2017 28

29. Bayesian ML Modelling
12/24/2017 29

30. Classification of ML Techniques
12/24/2017 30

31. Supervised Learning
• Given a training set of N example input–
output pairs (x1, y1), (x2, y2), . . . (xN, yN) ,
where each yj was generated by an
unknown function y = f(x),
• discover a function h that approximates the
true function f.
12/24/2017 31

32. Supervised Learning
12/24/2017 32

33. Supervised Learning
12/24/2017 33
• Regression
When the output y is a number
i.e : tomorrow’s temperature
• Classification
When the output y is one of a finite set of
values.
i.e : sunny, cloudy or rainy

34. Regression
12/24/2017 34

35. Regression – Optimization Approach
12/24/2017 35

36. Regression – Optimization Approach
12/24/2017 36

37. Regression – Probabilistic interpretation
12/24/2017 37

38. Maximum Likelihood function
Instead of maximizing L(θ), we can also maximize
any strictly increasing function of L(θ).
12/24/2017 38

39. Maximum Likelihood function
12/24/2017 39

40. Classification
12/24/2017 40
• Logistic Regression
i.e : Binary Classification y∈{0,1}
Hypothesis Representation

41. Unsupervised Learning
12/24/2017 41

42. Unsupervised Learning
12/24/2017 42
Reference

43. Model Evaluation – Bias Variance Tradeoff
12/24/2017 43

44. … Model Evaluation
Forecast Error = In-Sample Error + Model
Instability + Random Error
12/24/2017 44

45. Inference/Analytics
12/24/2017 45

46. Summary
12/24/2017 46

47. THANK YOU!