This document provides an overview of machine learning and big data concepts. It discusses machine learning applications and the data science pipeline of data collection, feature selection, model building, evaluation, and inference. Feature selection techniques for big data are covered, including subset selection and optimality criteria. Different machine learning modelling approaches are described, such as linear vs nonlinear, deterministic vs stochastic, parametric vs nonparametric, and frequentist vs Bayesian techniques. Supervised learning methods like regression and classification are explained. The document concludes with a discussion of model evaluation and performance.

1. Big Data & Machine Learning
12/24/2017

2. Agenda
• Why Machine Learning ?
• What is Machine Learning ?
• Some ML Applications
• Data Science Pipeline
• Data -> Big Data
• Big Data -> Feature Selection
• Machine Learning Modelling
• Model Evaluation
• Inference/Analytics
• Summary
12/24/2017 2

3. Why Machine Learning ?
12/24/2017 3

4. … Why Machine Learning ?
12/24/2017 4

5. What is Machine Learning ?
12/24/2017 5

6. What is Machine Learning ?
12/24/2017 6

7. Some MLApplications
12/24/2017 7

8. Data Science Pipeline
12/24/2017 8

9. Data Science Pipeline
12/24/2017 9
Data Features Model Inference Decision
Evaluation

10. Data => Big Data
• Structured Data
• Unstructured Data
• Big Data
- Text Data
- Time Series Data
- Spatial/Location-based Data
- Image/Video/Audio Data
12/24/2017 10

11. Big Data
12/24/2017 11

12. Big Data => Feature Selection
• Simplification of models - easier to interpret
• Shorter training times
• To avoid the curse of dimensionality
• Enhanced generalization by reducing
overfitting(formally, reduction of variance)
12/24/2017 12

13. Big Data => Feature Selection Techniques
1. Subset selection
• Exhaustive
• Best first
• Simulated annealing
• Genetic algorithm
• Greedy forward selection
• Greedy backward elimination
• Particle swarm optimization
• Targeted projection pursuit
• Scatter Search
• Variable Neighborhood Search
2. Optimality criteria
3. Structure learning
12/24/2017 13

14. Machine Learning Modelling
Linear Vs Non-linear Models
12/24/2017 14

15. Linear Modelling
• Response = constant + parameter * predictor
+ ... + parameter * predictor
• Y = b o + b1X1 + b2X2 + ... + bkXk
• Y = b o + b1X1 + b2X1
2
12/24/2017 15

16. … Linear Modelling
12/24/2017 16

17. Non-Linear Modelling
• Models which are not Linear ;)
12/24/2017 17

18. Machine Learning Modelling
Deterministic Vs Stochastic Models
12/24/2017 18

19. Deterministic Modelling
12/24/2017 19
Modeling is done using deterministic
variables. Uncertainty is not captured

20. Stochastic Modelling
Models the real world uncertainty using
random variables
12/24/2017 20

21. Machine Learning Modelling
Parametric Vs Non- Parametric Models
12/24/2017 21

22. Parametric Modelling
• Data is behaved according to a probability
distribution
• No of parameters is constant
• Focused on group means
12/24/2017 22

23. Non-Parametric Modelling
• Do not assume a particular probability
distribution
• No of parameters grows with training
samples
• Focused on group medians
12/24/2017 23

24. Stochastic ML Modelling
Frequentist Vs Bayesian Models
12/24/2017 24

25. Frequentist ML Modelling
Maximum Likelihood Estimation(MLE)
You need to model your random variables realistically
- Discrete r.v
i.e : Bernouli/Binomial/Geometri/Poisson)
- Continuous r.v
i.e :
Uniform/Exponential/Gamma/Normal(Gaussian)
Explained in Regression Modeling – Probabilistic
Interpretation
12/24/2017 25

26. Bayesian ML Modelling
Very powerful modeling approach
Prior knowledge is incorporated 
Maximum-a-Posteriori(MAP)
12/24/2017 26

27. Bayesian ML Modelling
12/24/2017 27

28. Bayesian ML Modelling
12/24/2017 28

29. Bayesian ML Modelling
12/24/2017 29

30. Classification of ML Techniques
12/24/2017 30

31. Supervised Learning
• Given a training set of N example input–
output pairs (x1, y1), (x2, y2), . . . (xN, yN) ,
where each yj was generated by an
unknown function y = f(x),
• discover a function h that approximates the
true function f.
12/24/2017 31

32. Supervised Learning
12/24/2017 32

33. Supervised Learning
12/24/2017 33
• Regression
When the output y is a number
i.e : tomorrow’s temperature
• Classification
When the output y is one of a finite set of
values.
i.e : sunny, cloudy or rainy

34. Regression
12/24/2017 34

35. Regression – Optimization Approach
12/24/2017 35

36. Regression – Optimization Approach
12/24/2017 36

37. Regression – Probabilistic interpretation
12/24/2017 37

38. Maximum Likelihood function
Instead of maximizing L(θ), we can also maximize
any strictly increasing function of L(θ).
12/24/2017 38

39. Maximum Likelihood function
12/24/2017 39

40. Classification
12/24/2017 40
• Logistic Regression
i.e : Binary Classification y∈{0,1}
Hypothesis Representation

41. Unsupervised Learning
12/24/2017 41

42. Unsupervised Learning
12/24/2017 42
Reference

43. Model Evaluation – Bias Variance Tradeoff
12/24/2017 43

44. … Model Evaluation
Forecast Error = In-Sample Error + Model
Instability + Random Error
12/24/2017 44

45. Inference/Analytics
12/24/2017 45

46. Summary
12/24/2017 46

47. THANK YOU!