![Use the Taoyuan City government to
provide open data and use data
science methods for predictive analysis
邹嘉骏 Jacky Zou
(大中華)台灣微軟資料科學應用者[專案研究員]
jackycj0830@gmail.com
國立中央大學通訊工程學系研究所(主攻物聯網、機器學習、人工智慧)
1](https://image.slidesharecdn.com/report-106553012-copy-180524175635/85/Report-106553012-copy-1-320.jpg)
More Related Content
Similar to Report 106553012 - copy
Similar to Report 106553012 - copy (20)
Recently uploaded
Recently uploaded (11)
Report 106553012 - copy
- 1. Use the Taoyuan City government to provide open data and use data science methods for predictive analysis 邹嘉骏 Jacky Zou (大中華)台灣微軟資料科學應用者[專案研究員] jackycj0830@gmail.com 國立中央大學通訊工程學系研究所(主攻物聯網、機器學習、人工智慧) 1
- 2. 大綱 • 前言：智慧城市、大數據 • 應用領域 • 應用方法(開放數據、分析工具) • 分析與預測 • 結語 2
- 5. 大數據的本質是什麼？ 5
- 11. 什麼是機器學習 • 機器學習(Machine Learning)是計算機科學的一門。 • 常利用着統計學的技巧，機器學習程式(Machine Learning Algorithms)能夠自動學習識別數據內的規律。憑着機器學習找 到的規律，電腦程式能作出高度準確的預測。 • Using known data, develop a model to predict unknown data. Known Data: Big enough archive, previous observations, past data Unknown Data: Unseen, not existing, future data Model: Known data + Algorithms (ML algorithms) 11
- 12. 12
- 13. Azure Machine Learning Service Data -> Predictive model -> Operational web API in minutes Blobs and Tables Hadoop (HDInsight) Relational DB (Azure SQL DB) Data Clients Model is now a web service that is callable Monetize the API through our marketplace API Integrated development environment for Machine Learning ML STUDIO
- 14. 14
- 15. Common Classes of Algorithms (Supervised|Unsupervised) Classification Regression Anomaly Detection Clustering 異質檢測迴歸分群分類
- 16. 機器學習流程 Acquire Data Understand Data Select "Relevant" Data Develop model Experiment and Evaluate Operate and Monitor 哪個環節是在機器 學習當中最為重要 的項目？ Data ? Model ? Result or Predict ?
- 18. 50°F 30°F 68°F 95°F1990 48°F 29°F 70°F 98°F2000 49°F 27°F 67°F 96°F2010 ? ? ? ?2020 … … … …… Known data Model Unknown data Weather forecast sample Using known data, develop a model to predict unknown data.
- 19. Model (Regression) 90°F -26°F 50°F 30°F 68°F 95°F1990 48°F 29°F 70°F 98°F2000 49°F 27°F 67°F 96°F2010 Using known data, develop a model to predict unknown data. Predict 2020 Summer
- 21. Azure ML的重點在於快速創造模型 • 以視覺結構方式呈現 • • 省下大量時間做黑手的工作 • • 有更充分的時間與精力在找出Key features • • 減少debug時間 • • 邏輯清楚易讀，交接交手方便 • 以快速的實驗來創造更棒的模型。 21
- 22. “” Azure Machine Learning Studio 22 Start here: studio.azureml.net
- 23. Project/ Modules Machine Learning Algorithms Custom Modules Clean data Data IO R/Python Support Region & Workspace Create Module/Exp/Proj ect … Run, Publish, etc Azure ML interface Description …
- 24. Azure ML 功能類別 • Data Format conversions • Feature Selection • Machine Learning • OpenCV Library Modules • R Language Modules • Python Language Modules • Statistical Functions • Text Analytics • Time Series Anomaly detection algorithms Classification algorithms Clustering algorithms Regression algorithms
- 25. Import data to Azure ML • Upload local file and use saved dataset • Import data via online data source • Import data via SQL Database • Use output from other experiment
- 26. Export data from Azure ML • Visualize data/output • Save as Dataset • Download to local • Open in Notebook • Export Data to Hive Query, Azure SQL Database, Azure Table, Blob Storage
- 27. Azure ML Modules • Azure ML modules • Execute R/Python Script • Imported custom modules (R) Create Custom Modules (R) (Web view)
- 29. Taoyuan Open Data & Azure Machine Learning Demo
- 31. ML Experiment Workflow Input data Data Transformation Train Model Define Model Split Data Score (prediction) Evaluate Model 31
- 33. ML Experiment Workflow Input data Data Transformation Train Model Define Model Split Data Score (prediction) Evaluate Model 33
- 34. Select Columns in Dataset 34
- 35. ML Experiment Workflow Input data Data Transformation Train Model Define Model Split Data Score (prediction) Evaluate Model 35
- 38. 結論 Garbage in, Gold out