Recommended
Recommended
More Related Content
Similar to awari-ds-aula1.pdf
Similar to awari-ds-aula1.pdf (20)
Recently uploaded
Recently uploaded (20)
awari-ds-aula1.pdf
- 1. Visão Geral de Data Science
- 2. 1. Definição de Data Science 2. Carreiras em Data Science 3. AI vs ML vs DL vs DS 4. Aplicações 5. Ferramentas de DS Programação
- 3. Definição Quais são os passos em um projeto de Data Science? - Definir a pergunta que queremos responder, os objetivos que queremos alcançar: definir o problema de negócio - Quais dados são pertinentes (muito dado é desperdício) e como cpodemos coletar? - Análise exploratória e processamento, tratar os dados, limpeza de dados, - Permitir que pessoas que não tem conhecimento consigam entender os dados, gráficos? - Modelagem de dados (IA, machine learning) - Comunicar as descobertas (visualização por dashboards) - Deploy, implementação de software, automatizar a nossa solução
- 4. Definição 1. Understand a Business Problem • Ask questions and define objectives
- 5. Definição 1. Understand a Business Problem • Ask questions and define objectives 2. Data Acquisition • Web Scraping • APIS • Direct download from URLs • Researches and crowdworkers (amazon Turk)
- 6. Definição 1. Understand a Business Problem • Ask questions and define objectives 2. Data Acquisition • Web Scraping • APIS • Direct download from URLs • Researches and crowdworkers (amazon Turk) 3. Data Preparation • Cleaning • Transformation • Very time consuming
- 7. Definição 1. Understand a Business Problem • Ask questions and define objectives 2. Data Acquisition • Web Scraping • APIS • Direct download from URLs • Researches and crowdworkers (amazon Turk) 3. Data Preparation • Cleaning • Transformation • Very time consuming 4. Exploratory Data Analysis • Visualization • Statistics • Distributions • Feature selection
- 8. Definição 1. Understand a Business Problem • Ask questions and define objectives 2. Data Acquisition • Web Scraping • APIS • Direct download from URLs • Researches and crowdworkers (amazon Turk) 3. Data Preparation • Cleaning • Transformation • Very time consuming 4. Exploratory Data Analysis • Visualization • Statistics • Distributions • Feature selection 5. Data Modeling • Machine Learning • Train a model that best fits the business requirements • Prediction models
- 9. Definição 5. Data Modeling • Machine Learning • Train a model that best fits the business requirements • Prediction models 1. Understand a Business Problem • Ask questions and define objectives 2. Data Acquisition • Web Scraping • APIS • Direct download from URLs • Researches and crowdworkers (amazon Turk) 3. Data Preparation • Cleaning • Transformation • Very time consuming 4. Exploratory Data Analysis • Visualization • Statistics • Distributions • Feature selection 6. Visualization and Communication • Documentation • Talk back to business • Reports and Dashboards 5. Data Modeling • Machine Learning • Train a model that best fits the business requirements • Prediction models
- 10. Definição 1. Understand a Business Problem • Ask questions and define objectives 2. Data Acquisition • Web Scraping • APIS • Direct download from URLs • Researches and crowdworkers (amazon Turk) 3. Data Preparation • Cleaning • Transformation • Very time consuming 4. Exploratory Data Analysis • Visualization • Statistics • Distributions • Feature selection 6. Visualization and Communication • Documentation • Talk back to business • Reports and Dashboards 7. Deploy and Maintenance • Production env • Real time analytics • Maintenance of Projects performance 5. Data Modeling • Machine Learning • Train a model that best fits the business requirements • Prediction models
- 11. Curva de Aprendizagem 1 2 3 4 5 6 7 Etapa mais complexa Etapa menos complexa
- 12. Curva de Aprendizagem 1 2 3 4 5 6 7 Etapa mais complexa Etapa menos complexa Data Modeling Deploy & Maintenance
- 13. Curva de Aprendizagem 1 2 3 4 5 6 7 Etapa mais complexa Etapa menos complexa Data Modeling Deploy & Maintenance
- 14. Quantidade de Código "The ML code is at the heart of a real-world ML production system, but that box often represents only 5% or less of the overall code of that total ML production system." Fonte: https://developers.google.com/machine-learning/crash-course/production-ml-systems
- 16. • Engenheira de Dados • Analista de Dados/Business Intelligence • Engenheira de ML/DL • Cientista de dados Carreiras em Data Science
- 17. Definição 1. Understand a Business Problem • Ask questions and define objectives 2. Data Acquisition • Web Scraping • APIS • Direct download from URLs • Researches and crowdworkers (amazon Turk) 3. Data Preparation • Cleaning • Transformation • Very time consuming 4. Exploratory Data Analysis • Visualization • Statistics • Distributions • Feature selection 6. Visualization and Communication • Documentation • Talk back to business • Reports and Dashboards 7. Deploy and Maintenance • Production env • Real time analytics • Maintenance of Projects performance 5. Data Modeling • Machine Learning • Train a model that best fits the business requirements • Prediction models
- 18. Definição: Engenheira de dados 1. Understand a Business Problem • Ask questions and define objectives 2. Data Acquisition • Web Scraping • APIS • Direct download from URLs • Researches and crowdworkers (amazon Turk) 3. Data Preparation • Cleaning • Transformation • Very time consuming 4. Exploratory Data Analysis • Visualization • Statistics • Distributions • Feature selection 6. Visualization and Communication • Documentation • Talk back to business • Reports and Dashboards 7. Deploy and Maintenance • Production env • Real time analytics • Maintenance of Projects performance 5. Data Modeling • Machine Learning • Train a model that best fits the business requirements • Prediction models
- 19. 1. Understand a Business Problem • Ask questions and define objectives 2. Data Acquisition • Web Scraping • APIS • Direct download from URLs • Researches and crowdworkers (amazon Turk) 3. Data Preparation • Cleaning • Transformation • Very time consuming 4. Exploratory Data Analysis • Visualization • Statistics • Distributions • Feature selection 6. Visualization and Communication • Documentation • Talk back to business • Reports and Dashboards 7. Deploy and Maintenance • Production env • Real time analytics • Maintenance of Projects performance 5. Data Modeling • Machine Learning • Train a model that best fits the business requirements • Prediction models Definição: Analista de dados
- 20. Definição: Engenheira de ML/DL 1. Understand a Business Problem • Ask questions and define objectives 2. Data Acquisition • Web Scraping • APIS • Direct download from URLs • Researches and crowdworkers (amazon Turk) 3. Data Preparation • Cleaning • Transformation • Very time consuming 4. Exploratory Data Analysis • Visualization • Statistics • Distributions • Feature selection 6. Visualization and Communication • Documentation • Talk back to business • Reports and Dashboards 7. Deploy and Maintenance • Production env • Real time analytics • Maintenance of Projects performance 5. Data Modeling • Machine Learning • Train a model that best fits the business requirements • Prediction models
- 21. Definição: Cientista de dados 1. Understand a Business Problem • Ask questions and define objectives 2. Data Acquisition • Web Scraping • APIS • Direct download from URLs • Researches and crowdworkers (amazon Turk) 3. Data Preparation • Cleaning • Transformation • Very time consuming 4. Exploratory Data Analysis • Visualization • Statistics • Distributions • Feature selection 6. Visualization and Communication • Documentation • Talk back to business • Reports and Dashboards 7. Deploy and Maintenance • Production env • Real time analytics • Maintenance of Projects performance 5. Data Modeling • Machine Learning • Train a model that best fits the business requirements • Prediction models
- 22. • Engenheira de Dados + Big Data • Analista de Dados/Business Intelligence + Big Data • Engenheira de ML/DL + Big Data • Cientista de Dados + Big Data Carreiras em Data Science
- 23. Definição 1. Understand a Business Problem • Ask questions and define objectives 2. Data Acquisition • Web Scraping • APIS • Direct download from URLs • Researches and crowdworkers (amazon Turk) 3. Data Preparation • Cleaning • Transformation • Very time consuming 4. Exploratory Data Analysis • Visualization • Statistics • Distributions • Feature selection 6. Visualization and Communication • Documentation • Talk back to business • Reports and Dashboards 7. Deploy and Maintenance • Production env • Real time analytics • Maintenance of Projects performance 5. Data Modeling • Machine Learning • Train a model that best fits the business requirements • Prediction models Big Data • Data too large for a spreadsheet • Requires alternative data-processing software
- 27. Palavras chaves em vagas de DS no Brasil
- 28. Palavras chaves em vagas de DS nos EUA
- 29. Pesquisa Data Hackers sobre mercado de Data Science • Base de dados • https://www.kaggle.com/datahackers/pesquisa-data-hackers-2019 • Análise detalhada • https://www.kaggle.com/jsaguiar/brazilian-ds-market
- 30. AI vs ML vs DL vs DS
- 33. Inteligência Artificial Machine Learning Deep Learning ”Imitar” a inteligência humana Artificial Intelligence
- 34. Inteligência Artificial Machine Learning Deep Learnin g Machine Learning Artificial Intelligence
- 35. Inteligência Artificial Machine Learning Deep Learnin g Algoritmos que ”aprendem”, modelos preditivos a partir de dados Machine Learning Artificial Intelligence
- 36. Inteligência Artificial Machine Learning Deep Learnin g Algoritmos que ”aprendem”, modelos preditivos a partir de dados xi - área (m2 ) yi - preço (USD) 60 100k 65 110k 70 150k 80 180k 100 200k 120 190k … … Machine Learning Artificial Intelligence
- 37. Inteligência Artificial Machine Learning Deep Learnin g Algoritmos que ”aprendem”, modelos preditivos a partir de dados x (área em m2 ) Dados Machine Learning Artificial Intelligence y (preço)
- 38. Inteligência Artificial Machine Learning Deep Learnin g Algoritmos que ”aprendem”, modelos preditivos a partir de dados Dados Modelo Machine Learning Artificial Intelligence x (área em m2 ) y (preço)
- 41. Inteligência Artificial Machine Learning Deep Learning Redes neurais com várias camadas treinadas com milhões de dados Deep Learning Machine Learning Artificial Intelligence
- 42. Inteligência Artificial Machine Learning Deep Learning Redes neurais com várias camadas treinadas com milhões de dados Aprendem características a partir de dados crus Deep Learning Machine Learning Artificial Intelligence
- 43. Inteligência Artificial Machine Learning Deep Learning Redes neurais com várias camadas treinadas com milhões de dados Aprendem características a partir de dados crus Deep Learning Machine Learning Artificial Intelligence
- 45. Aplicações
- 46. Aplicações Aplicações de ML • Uber – Preço dinâmico • Airbnb – Preço dinâmico • Youtube – Recomendação de próximo vídeo • Quinto Andar – Calculadora de Aluguel • Nubank – Análise de Crédito
- 47. Aplicações Aplicações de DL • Google - Google Translate • Android - Pesquisa por voz • Facebook - Reconhecimento facial • SkinVision App – Câncer de Pele • My Heritage – Colorização de Fotos
- 49. Aplicações de Data Visualization Link Data is Beautiful Aplicações
- 51. Anaconda: Kit de Ferramentas de Data Science
- 52. Imagens que vêm à cabeça quando falamos de programação
- 53. Em Data Science é Diferente
- 54. Em Data Science é Diferente
- 55. • https://nbviewer.jupyter.org/ Exemplos de Livros em Jupyter
- 56. Exemplos de Livros em Jupyter
- 58. • Nos contextualizarmos com o ambiente de desenvolvimento em data science remoto: Google Colab • Importar um utilitário para colorização de imagens • Primeiros passos com Python Objetivo da Atividade