![제 23회 보아즈(BOAZ) 빅데이터 컨퍼런스 - [MBOAX] : ABSA를 활용한 소비자 반응 분석 기반 운영 효율화 대시보드 설계](https://cdn.slidesharecdn.com/ss_thumbnails/3-1boaz23rdconferencemboax-260203102709-9d519923-thumbnail.jpg?width=640&height=640&fit=bounds)
A practical introduction to data science and machine learning
- 1.
- 2.
- 3.
- 4.
- 5. 1. Descriptive analytics • What happened? → Plots, summary statistics, alerts →Make data human-friendly akquinet fridge: daily energy consumption
- 6. 1. Descriptive analytics • What happened? → Plots, summary statistics, alerts →Make data human-friendly akquinet fridge: daily energy consumption Mean = 0.52 kWh
- 7. 1. Descriptive analytics • What happened? → Plots, summary statistics, alerts →Make data human-friendly akquinet fridge: daily energy consumption
- 8. 1. Descriptive analytics • What happened? → Plots, summary statistics, alerts →Make data human-friendly akquinet fridge: daily energy consumption
- 9.
- 10.
- 11.
- 12.
- 13.
- 14.
- 15. 4 types of data analytics 1. Descriptive –what happened? 2. Diagnostic – why did it happen? 3. Predictive – what will happen in the future? 4. Prescriptive – what should we do about it?
- 16. 4 types of data analytics 1. Descriptive –what happened? 2. Diagnostic – why did it happen? 3. Predictive – what will happen in the future? 4. Prescriptive – what should we do about it?
- 17.
- 18.
- 19. The problem of prediction • What do we mean by prediction? Information we use to make the prediction Quantity we want to predict y =f(X) X y ?? • In mathematical terms: Features Target variable
- 20.
- 21.
- 22.
- 23. How do we predict? $ … Size, location Pixel values {‘cat’, ‘not cat’} Usage, weather … kWh Diameter, weight{1c, 5c, 10c, 25c} ?? ?? ?? ??
- 24. How do we predict? $ … Size, location Pixel values {‘cat’, ‘not cat’} Usage, weather … kWh Diameter, weight{1c, 5c, 10c, 25c} ?? ?? ?? Manufacturer
- 25. How do we predict? $ … Size, location Pixel values {‘cat’, ‘not cat’} Usage, weather … kWh Diameter, weight{1c, 5c, 10c, 25c} ?? ?? Manufacturer Engineer, physical laws
- 26. How do we predict? $ … Size, location Pixel values {‘cat’, ‘not cat’} Usage, weather … kWh Diameter, weight{1c, 5c, 10c, 25c} Real estate agent ?? Manufacturer Engineer, physical laws
- 27. How do we predict? $ … Size, location Pixel values {‘cat’, ‘not cat’} Usage, weather … kWh Diameter, weight{1c, 5c, 10c, 25c} Real estate agent ?? Manufacturer Engineer, physical laws Rule based
- 28. How do we predict? $ … Size, location Pixel values {‘cat’, ‘not cat’} Usage, weather … kWh Diameter, weight{1c, 5c, 10c, 25c} Real estate agent ?? Manufacturer Engineer, physical laws Data!
- 29.
- 30.
- 31.
- 32. What can machine learning do? • Image, speech recognition (> 95% accuracy) • Self-driving vehicles •DeepMind defeated Go champion, • Recommendation engines, spam filters, fraud detection, text analysis… Google’s energy efficiency by 15%
- 33.
- 34.
- 35.
- 36.
- 37.
- 38.
- 39.
- 40. Formulate the problem Price E.g. house Size Location No. rooms Condition … • What do we want to predict? •What data could we use to do this? X y
- 41.
- 42.
- 43. Explore the data • Visualization, descriptive analytics 1. Individual variables 2.Relationships between variables
- 44. Explore the data • Visualization, descriptive analytics 1. Individual variables 2.Relationships between variables
- 45. Explore the data • Visualization, descriptive analytics 1. Individual variables 2.Relationships between variables
- 46.
- 47.
- 48.
- 49.
- 50. Clean the data • Missing entries • Outliers •Spurious values } • Remove? • Impute/correct? • Label and keep?
- 51.
- 52. Convert data types Neighbourhood 1 North Ames 2 North Ames 3Gilbert 4 Gilbert 5 Stone Brook 6 Gilbert 7 Gilbert North Ames Gilbert Stone Brook 1 1 0 0 2 1 0 0 3 0 1 0 4 0 1 0 5 0 0 1 6 0 1 0 7 0 1 0 Dummy (one-hot) encoding
- 53.
- 54.
- 55.
- 56. Feature engineering • Transform data to optimize learning → Construct new features →Transform features → Select features 2 x1 x2 x3 x4 x5
- 57.
- 58.
- 59. Choose and fit model • Linear models (generalized) • Tree-based models •Neural networks X y Model?? • Support vector machines, k-nearest neighbour, Bayesian networks, …
- 60.
- 61.
- 62.
- 63.
- 64.
- 65.
- 66.
- 67.
- 68.
- 69.
- 70.
- 71.
- 72.
- 73.
- 74.
- 75. Evaluating model performance • Cross validation Training set Model learns from this data Test setUsed to evaluate model performance
- 76.
- 77.
- 78.
- 79.
- 80.
- 81. House prices on Kaggle • Prediction error within 1%in most cases • Top 2% of all submissions on Kaggle
- 82. Summary • Types of data analytics • Machine learning for prediction •From data to predictive model → Data preparation → Modelling and cross validation Cat Data!
- 83.