SlideShare a Scribd company logo
THE PYTHON ECOSYSTEM FOR DATA
SCIENCE - LANDSCAPE OVERVIEW
Ananth Krishnamoorthy, Ph.D.
Outline Slides for Talk at Fifth Elephant 2017
25-Apr-2017
Summary
• In their day-to-day jobs, data science teams and data scientists face challenges in
many overlapping yet distinct areas such as Reporting, Data Processing &
Storage, Scientific Computing, ML Modelling, Application Development. To
succeed, Data science teams, especially small ones, need a deep appreciation of
these dependencies on their success.
• Python ecosystem for data science has a number of tools and libraries for various
aspects of data science, including Machine Learning, Cluster Computing,
Scientific Computing, etc.
• The idea of this talk is to understand what the Python data science ecosystem
offers (so that you don't reinvent it), what are some common gaps (so that you
don't go blue looking for answers).
• In this talk, we describe how different tools/libraries fit in the machine learning
model development and deployment workflow . This talk is about how these
different tools work (and don’t work) together with each other. It is intended as a
landscape survey of the python data science ecosystem, along with a mention of
some common gaps that practitioners may notice as they put together a stack
and/or an application for their company.
The most important trait of the Analytics 3.0 era is that not only online firms, but virtually any type of firm
in any industry, can participate in the data economy. Banks, industrial manufacturers, health care
providers, retailers—any company in any industry that is willing to exploit the possibilities—can all
develop data-based offerings for customers, as well as support internal decisions with big data.
Analytics 1.0 Analytics 2.0 Analytics 3.0
Data  Enterprise Data
 Structured transactional data
 Bring in web and social data
 Complex, large,
semistructured data sources
 GPS, Mobile Device, Clickstream,
Sensor data
 Unstructured, real time, streaming
Tools  Spreadsheets
 BI, OLAP
 ETL
 On-premise servers
 Visualization
 NoSQL
 Hadoop
 Machine Learning , Artificial
Intelligence
 On-Demand Everything
 Analytical Apps
 Integrated, Embedded models
Activity  Majority of analytical activity
was descriptive analytics, or
reporting
 Creating analytical models
was a time-consuming
“batch” process
 Visual analytics dominates
predictive and prescriptive
techniques
 Develop products, not
PowerPoints or reports
 Analytics integral to running the
business, strategic asset
 Rapid and agile insight delivery
 Analytical tools available at point of
decision
Source: THE RISE OF ANALYTICS 3.0, By Thomas H. Davenport, IIA, 2013
Evolving Role of Data Science Teams
Machine Learning vs Real World Data
Science
Machine Learning
Deployment
Application Development
Big Data Processing
Data Storage
ETL
Challenges faced by Data Science Teams
• Requires many more competencies than can be reasonably expected
from one person
• Challenges are greater for smaller teams and smaller companies, e.g.
startups
• Challenges create dependencies on other teams e.g. Development
• Dependencies slow down execution and benefits realization
Plethora of Choices
Reporting
Data
Processing
& Storage
Scientific
Computing
ML
Modelling
Application
Development
SQL
NoSQL
Graphdb
OLAP
ETL
Cluster
Computing
Stream
Processing
SQL
Charting
Statistics
Cloud
Front End
Microservices
Back End
ML
Deep Learning
Dim. Reduction
Signal
Processing
Optimization
Time Series
Analysis
Simulation
MapReduce
Data Science Workflow
ETL Process ModelStore Deploy
DATA SCIENTIST SKILLS
Infrastructure and Provisioning ???
Python Ecosystem
ETL Process ModelStore Deploy
Odo Blaze Pandas
Dask
Spark
Sklearn_Pandas
Scikit-learn
Keras
Spark MLlib
Bokeh
Jupyter
Review of Key Tools
(50% of talk time spent here, more slides to be added)
• Jupyter
• Pandas
• Scikit-Learn
• Keras / TensorFlow / Theano
• Matplotlib/Bokeh
• Blaze
• Odo
• Dask
• pySpark
We shall see some code snippets here, to
illustrate a few ideas
The idea is to know enough to pick the right
components for the job at hand
Use Case 1: Small Data
This use case will illustrate case of Small
Data i.e. Desktop / In-memory processing
Use Case 2: ‘Medium’ Data
This use case will illustrate case of Medium
Data with Out-of-core processing
Use Case 3: Big Data
This use case will illustrate case of Big Data
i.e cluster computing
What Works
• Sklearn’s Consistent API, wide variety of ML algorithms
• Sklearn Pipelines
• Scikit-Keras Integration
• Pandas for Data Analysis
• ….
• ….
Gaps – A Practitioner Perspective
• Uniform API Across Activities
• Separation of Data, Processing, and Instructions
• Single Data Structure Paradigm
• Support for in-memory, out-of-core, and distributed computing in same
paradigm e.g. SFrame
• ETL
• Push heavy lifting to backend systems
• Monitoring workflows
• Application development
• Bokeh
• Deployment
• Application
• Web Services

More Related Content

What's hot

Deep Learning - The Past, Present and Future of Artificial Intelligence
Deep Learning - The Past, Present and Future of Artificial IntelligenceDeep Learning - The Past, Present and Future of Artificial Intelligence
Deep Learning - The Past, Present and Future of Artificial Intelligence
Lukas Masuch
 
ONNX and MLflow
ONNX and MLflowONNX and MLflow
ONNX and MLflow
amesar0
 
Iot architecture
Iot architectureIot architecture
Iot architecture
Niranjan Kumar
 
The role of NLP & ML in Cognitive System by Sunantha Krishnan
The role of NLP & ML in Cognitive System by Sunantha KrishnanThe role of NLP & ML in Cognitive System by Sunantha Krishnan
The role of NLP & ML in Cognitive System by Sunantha Krishnan
sunanthakrishnan
 
Deep Learning: Application & Opportunity
Deep Learning: Application & OpportunityDeep Learning: Application & Opportunity
Deep Learning: Application & Opportunity
iTrain
 
Overview - IBM Big Data Platform
Overview - IBM Big Data PlatformOverview - IBM Big Data Platform
Overview - IBM Big Data Platform
Vikas Manoria
 
AI_1 Introduction of AI
AI_1 Introduction of AIAI_1 Introduction of AI
AI_1 Introduction of AI
Khushali Kathiriya
 
BUTLER project overview
BUTLER project overviewBUTLER project overview
BUTLER project overview
butler-iot
 
Cognitive computing
Cognitive computing Cognitive computing
Cognitive computing
Pratap Dangeti
 
Internet of Things A Vision, Architectural Elements, and Future Directions
Internet of Things A Vision, Architectural Elements, and Future Directions Internet of Things A Vision, Architectural Elements, and Future Directions
Internet of Things A Vision, Architectural Elements, and Future Directions
Mostafa Arjmand
 
IoT Design Principles
IoT Design PrinciplesIoT Design Principles
IoT Design Principles
ardexateam
 
Lecture 1. Introduction to AI and it's applications.ppt
Lecture 1. Introduction to AI and it's applications.pptLecture 1. Introduction to AI and it's applications.ppt
Lecture 1. Introduction to AI and it's applications.ppt
DebabrataPain1
 
Artificial Neural Networks for NIU session 2016 17
Artificial Neural Networks for NIU session 2016 17 Artificial Neural Networks for NIU session 2016 17
Artificial Neural Networks for NIU session 2016 17
Prof. Neeta Awasthy
 
Hyperautomation
HyperautomationHyperautomation
Hyperautomation
AkanshaBhatnagar7
 
IoT Cloud architecture
IoT Cloud architectureIoT Cloud architecture
IoT Cloud architecture
MachinePulse
 
(r)Evolution of Machine Learning
(r)Evolution of Machine Learning(r)Evolution of Machine Learning
(r)Evolution of Machine Learning
Pankaj Tirpude
 
Artificial intelligence- Logic Agents
Artificial intelligence- Logic AgentsArtificial intelligence- Logic Agents
Artificial intelligence- Logic Agents
Nuruzzaman Milon
 
Introduction to Neural Networks
Introduction to Neural NetworksIntroduction to Neural Networks
Introduction to Neural Networks
Databricks
 
Sensor Cloud
Sensor CloudSensor Cloud
Sensor Cloud
Debjyoti Ghosh
 
Machine Intelligence at Google Scale: TensorFlow
Machine Intelligence at Google Scale: TensorFlowMachine Intelligence at Google Scale: TensorFlow
Machine Intelligence at Google Scale: TensorFlow
DataWorks Summit/Hadoop Summit
 

What's hot (20)

Deep Learning - The Past, Present and Future of Artificial Intelligence
Deep Learning - The Past, Present and Future of Artificial IntelligenceDeep Learning - The Past, Present and Future of Artificial Intelligence
Deep Learning - The Past, Present and Future of Artificial Intelligence
 
ONNX and MLflow
ONNX and MLflowONNX and MLflow
ONNX and MLflow
 
Iot architecture
Iot architectureIot architecture
Iot architecture
 
The role of NLP & ML in Cognitive System by Sunantha Krishnan
The role of NLP & ML in Cognitive System by Sunantha KrishnanThe role of NLP & ML in Cognitive System by Sunantha Krishnan
The role of NLP & ML in Cognitive System by Sunantha Krishnan
 
Deep Learning: Application & Opportunity
Deep Learning: Application & OpportunityDeep Learning: Application & Opportunity
Deep Learning: Application & Opportunity
 
Overview - IBM Big Data Platform
Overview - IBM Big Data PlatformOverview - IBM Big Data Platform
Overview - IBM Big Data Platform
 
AI_1 Introduction of AI
AI_1 Introduction of AIAI_1 Introduction of AI
AI_1 Introduction of AI
 
BUTLER project overview
BUTLER project overviewBUTLER project overview
BUTLER project overview
 
Cognitive computing
Cognitive computing Cognitive computing
Cognitive computing
 
Internet of Things A Vision, Architectural Elements, and Future Directions
Internet of Things A Vision, Architectural Elements, and Future Directions Internet of Things A Vision, Architectural Elements, and Future Directions
Internet of Things A Vision, Architectural Elements, and Future Directions
 
IoT Design Principles
IoT Design PrinciplesIoT Design Principles
IoT Design Principles
 
Lecture 1. Introduction to AI and it's applications.ppt
Lecture 1. Introduction to AI and it's applications.pptLecture 1. Introduction to AI and it's applications.ppt
Lecture 1. Introduction to AI and it's applications.ppt
 
Artificial Neural Networks for NIU session 2016 17
Artificial Neural Networks for NIU session 2016 17 Artificial Neural Networks for NIU session 2016 17
Artificial Neural Networks for NIU session 2016 17
 
Hyperautomation
HyperautomationHyperautomation
Hyperautomation
 
IoT Cloud architecture
IoT Cloud architectureIoT Cloud architecture
IoT Cloud architecture
 
(r)Evolution of Machine Learning
(r)Evolution of Machine Learning(r)Evolution of Machine Learning
(r)Evolution of Machine Learning
 
Artificial intelligence- Logic Agents
Artificial intelligence- Logic AgentsArtificial intelligence- Logic Agents
Artificial intelligence- Logic Agents
 
Introduction to Neural Networks
Introduction to Neural NetworksIntroduction to Neural Networks
Introduction to Neural Networks
 
Sensor Cloud
Sensor CloudSensor Cloud
Sensor Cloud
 
Machine Intelligence at Google Scale: TensorFlow
Machine Intelligence at Google Scale: TensorFlowMachine Intelligence at Google Scale: TensorFlow
Machine Intelligence at Google Scale: TensorFlow
 

Similar to The Python ecosystem for data science - Landscape Overview

Proposed Talk Outline for Pycon2017
Proposed Talk Outline for Pycon2017 Proposed Talk Outline for Pycon2017
Proposed Talk Outline for Pycon2017
Dr. Ananth Krishnamoorthy
 
Self Study Business Approach to DS_01022022.docx
Self Study Business Approach to DS_01022022.docxSelf Study Business Approach to DS_01022022.docx
Self Study Business Approach to DS_01022022.docx
Shanmugasundaram M
 
Intelligently Automating Machine Learning, Artificial Intelligence, and Data ...
Intelligently Automating Machine Learning, Artificial Intelligence, and Data ...Intelligently Automating Machine Learning, Artificial Intelligence, and Data ...
Intelligently Automating Machine Learning, Artificial Intelligence, and Data ...
Ali Alkan
 
Agile data science
Agile data scienceAgile data science
Agile data science
Joel Horwitz
 
Big Data
Big DataBig Data
Big Data
NGDATA
 
Data science presentation
Data science presentationData science presentation
Data science presentation
MSDEVMTL
 
Big Data Analytics
Big Data AnalyticsBig Data Analytics
Big Data Analytics
Sreedhar Chowdam
 
Building Your Dream Machine Learning Team with Python Expertise
Building Your Dream Machine Learning Team with Python ExpertiseBuilding Your Dream Machine Learning Team with Python Expertise
Building Your Dream Machine Learning Team with Python Expertise
riyak40
 
Next generation of data scientist
Next generation of data scientistNext generation of data scientist
Next generation of data scientist
TanujaSomvanshi1
 
Python para Manual de Ciência de Dados
Python para Manual de Ciência de DadosPython para Manual de Ciência de Dados
Python para Manual de Ciência de Dados
Rafael Oliveira Bitcoin
 
Ch1IntroductiontoDataScience.pptx
Ch1IntroductiontoDataScience.pptxCh1IntroductiontoDataScience.pptx
Ch1IntroductiontoDataScience.pptx
AbderrahmanABID2
 
Top Big data Analytics tools: Emerging trends and Best practices
Top Big data Analytics tools: Emerging trends and Best practicesTop Big data Analytics tools: Emerging trends and Best practices
Top Big data Analytics tools: Emerging trends and Best practices
SpringPeople
 
2019 DSA 105 Introduction to Data Science Week 4
2019 DSA 105 Introduction to Data Science Week 42019 DSA 105 Introduction to Data Science Week 4
2019 DSA 105 Introduction to Data Science Week 4
Ferdin Joe John Joseph PhD
 
Introduction to Data Science - Week 4 - Tools and Technologies in Data Science
Introduction to Data Science - Week 4 - Tools and Technologies in Data ScienceIntroduction to Data Science - Week 4 - Tools and Technologies in Data Science
Introduction to Data Science - Week 4 - Tools and Technologies in Data Science
Ferdin Joe John Joseph PhD
 
Cognitive Computing - A Primer
Cognitive Computing - A PrimerCognitive Computing - A Primer
Cognitive Computing - A Primer
Marlabs
 
Big data and you
Big data and you Big data and you
Big data and you
IBM
 
10-Hot-Data-Analytics-Tre-8904178.ppsx
10-Hot-Data-Analytics-Tre-8904178.ppsx10-Hot-Data-Analytics-Tre-8904178.ppsx
10-Hot-Data-Analytics-Tre-8904178.ppsx
SangeetaTripathi8
 
The book of elephant tattoo
The book of elephant tattooThe book of elephant tattoo
The book of elephant tattoo
Mohamed Magdy
 
DATASCIENCE vs BUSINESS INTELLIGENCE.pptx
DATASCIENCE vs BUSINESS INTELLIGENCE.pptxDATASCIENCE vs BUSINESS INTELLIGENCE.pptx
DATASCIENCE vs BUSINESS INTELLIGENCE.pptx
OTA13NayabNakhwa
 
On Big Data
On Big DataOn Big Data
On Big Data
arttan2001
 

Similar to The Python ecosystem for data science - Landscape Overview (20)

Proposed Talk Outline for Pycon2017
Proposed Talk Outline for Pycon2017 Proposed Talk Outline for Pycon2017
Proposed Talk Outline for Pycon2017
 
Self Study Business Approach to DS_01022022.docx
Self Study Business Approach to DS_01022022.docxSelf Study Business Approach to DS_01022022.docx
Self Study Business Approach to DS_01022022.docx
 
Intelligently Automating Machine Learning, Artificial Intelligence, and Data ...
Intelligently Automating Machine Learning, Artificial Intelligence, and Data ...Intelligently Automating Machine Learning, Artificial Intelligence, and Data ...
Intelligently Automating Machine Learning, Artificial Intelligence, and Data ...
 
Agile data science
Agile data scienceAgile data science
Agile data science
 
Big Data
Big DataBig Data
Big Data
 
Data science presentation
Data science presentationData science presentation
Data science presentation
 
Big Data Analytics
Big Data AnalyticsBig Data Analytics
Big Data Analytics
 
Building Your Dream Machine Learning Team with Python Expertise
Building Your Dream Machine Learning Team with Python ExpertiseBuilding Your Dream Machine Learning Team with Python Expertise
Building Your Dream Machine Learning Team with Python Expertise
 
Next generation of data scientist
Next generation of data scientistNext generation of data scientist
Next generation of data scientist
 
Python para Manual de Ciência de Dados
Python para Manual de Ciência de DadosPython para Manual de Ciência de Dados
Python para Manual de Ciência de Dados
 
Ch1IntroductiontoDataScience.pptx
Ch1IntroductiontoDataScience.pptxCh1IntroductiontoDataScience.pptx
Ch1IntroductiontoDataScience.pptx
 
Top Big data Analytics tools: Emerging trends and Best practices
Top Big data Analytics tools: Emerging trends and Best practicesTop Big data Analytics tools: Emerging trends and Best practices
Top Big data Analytics tools: Emerging trends and Best practices
 
2019 DSA 105 Introduction to Data Science Week 4
2019 DSA 105 Introduction to Data Science Week 42019 DSA 105 Introduction to Data Science Week 4
2019 DSA 105 Introduction to Data Science Week 4
 
Introduction to Data Science - Week 4 - Tools and Technologies in Data Science
Introduction to Data Science - Week 4 - Tools and Technologies in Data ScienceIntroduction to Data Science - Week 4 - Tools and Technologies in Data Science
Introduction to Data Science - Week 4 - Tools and Technologies in Data Science
 
Cognitive Computing - A Primer
Cognitive Computing - A PrimerCognitive Computing - A Primer
Cognitive Computing - A Primer
 
Big data and you
Big data and you Big data and you
Big data and you
 
10-Hot-Data-Analytics-Tre-8904178.ppsx
10-Hot-Data-Analytics-Tre-8904178.ppsx10-Hot-Data-Analytics-Tre-8904178.ppsx
10-Hot-Data-Analytics-Tre-8904178.ppsx
 
The book of elephant tattoo
The book of elephant tattooThe book of elephant tattoo
The book of elephant tattoo
 
DATASCIENCE vs BUSINESS INTELLIGENCE.pptx
DATASCIENCE vs BUSINESS INTELLIGENCE.pptxDATASCIENCE vs BUSINESS INTELLIGENCE.pptx
DATASCIENCE vs BUSINESS INTELLIGENCE.pptx
 
On Big Data
On Big DataOn Big Data
On Big Data
 

Recently uploaded

🚂🚘 Premium Girls Call Guwahati 🛵🚡000XX00000 💃 Choose Best And Top Girl Servi...
🚂🚘 Premium Girls Call Guwahati  🛵🚡000XX00000 💃 Choose Best And Top Girl Servi...🚂🚘 Premium Girls Call Guwahati  🛵🚡000XX00000 💃 Choose Best And Top Girl Servi...
🚂🚘 Premium Girls Call Guwahati 🛵🚡000XX00000 💃 Choose Best And Top Girl Servi...
kuldeepsharmaks8120
 
potential usefulness of multi-agent maze-solving in general
potential usefulness of multi-agent maze-solving in generalpotential usefulness of multi-agent maze-solving in general
potential usefulness of multi-agent maze-solving in general
huseindihon
 
VIP Girls Call Mumbai 9910780858 Provide Best And Top Girl Service And No1 in...
VIP Girls Call Mumbai 9910780858 Provide Best And Top Girl Service And No1 in...VIP Girls Call Mumbai 9910780858 Provide Best And Top Girl Service And No1 in...
VIP Girls Call Mumbai 9910780858 Provide Best And Top Girl Service And No1 in...
44annissa
 
Towards an Analysis-Ready, Cloud-Optimised service for FAIR fusion data
Towards an Analysis-Ready, Cloud-Optimised service for FAIR fusion dataTowards an Analysis-Ready, Cloud-Optimised service for FAIR fusion data
Towards an Analysis-Ready, Cloud-Optimised service for FAIR fusion data
Samuel Jackson
 
Girls Call Chennai 000XX00000 Provide Best And Top Girl Service And No1 in City
Girls Call Chennai 000XX00000 Provide Best And Top Girl Service And No1 in CityGirls Call Chennai 000XX00000 Provide Best And Top Girl Service And No1 in City
Girls Call Chennai 000XX00000 Provide Best And Top Girl Service And No1 in City
solankikamal004
 
Celebrity Girls Call Noida 9873940964 Unlimited Short Providing Girls Service...
Celebrity Girls Call Noida 9873940964 Unlimited Short Providing Girls Service...Celebrity Girls Call Noida 9873940964 Unlimited Short Providing Girls Service...
Celebrity Girls Call Noida 9873940964 Unlimited Short Providing Girls Service...
ginni singh$A17
 
potential development of the A* search algorithm specifically
potential development of the A* search algorithm specificallypotential development of the A* search algorithm specifically
potential development of the A* search algorithm specifically
huseindihon
 
Verified Girls Call Andheri 9930245274 Unlimited Short Providing Girls Servic...
Verified Girls Call Andheri 9930245274 Unlimited Short Providing Girls Servic...Verified Girls Call Andheri 9930245274 Unlimited Short Providing Girls Servic...
Verified Girls Call Andheri 9930245274 Unlimited Short Providing Girls Servic...
revolutionary575
 
Best Girls Call Navi Mumbai 9930245274 Provide Best And Top Girl Service And ...
Best Girls Call Navi Mumbai 9930245274 Provide Best And Top Girl Service And ...Best Girls Call Navi Mumbai 9930245274 Provide Best And Top Girl Service And ...
Best Girls Call Navi Mumbai 9930245274 Provide Best And Top Girl Service And ...
sharonblush
 
Celebrity Girls Call Andheri 9930245274 Unlimited Short Providing Girls Servi...
Celebrity Girls Call Andheri 9930245274 Unlimited Short Providing Girls Servi...Celebrity Girls Call Andheri 9930245274 Unlimited Short Providing Girls Servi...
Celebrity Girls Call Andheri 9930245274 Unlimited Short Providing Girls Servi...
revolutionary575
 
Girls Call Vadodara 000XX00000 Provide Best And Top Girl Service And No1 in City
Girls Call Vadodara 000XX00000 Provide Best And Top Girl Service And No1 in CityGirls Call Vadodara 000XX00000 Provide Best And Top Girl Service And No1 in City
Girls Call Vadodara 000XX00000 Provide Best And Top Girl Service And No1 in City
gargnatasha985
 
社内勉強会資料_TransNeXt: Robust Foveal Visual Perception for Vision Transformers
社内勉強会資料_TransNeXt: Robust Foveal Visual Perception for Vision Transformers社内勉強会資料_TransNeXt: Robust Foveal Visual Perception for Vision Transformers
社内勉強会資料_TransNeXt: Robust Foveal Visual Perception for Vision Transformers
NABLAS株式会社
 
Beautiful Girls Call 9711199171 9711199171 Provide Best And Top Girl Service ...
Beautiful Girls Call 9711199171 9711199171 Provide Best And Top Girl Service ...Beautiful Girls Call 9711199171 9711199171 Provide Best And Top Girl Service ...
Beautiful Girls Call 9711199171 9711199171 Provide Best And Top Girl Service ...
janvikumar4133
 
VIP Kolkata Girls Call Kolkata 0X0000000X Doorstep High-Profile Girl Service ...
VIP Kolkata Girls Call Kolkata 0X0000000X Doorstep High-Profile Girl Service ...VIP Kolkata Girls Call Kolkata 0X0000000X Doorstep High-Profile Girl Service ...
VIP Kolkata Girls Call Kolkata 0X0000000X Doorstep High-Profile Girl Service ...
sukaniyasunnu
 
OpenMetadata Spotlight - OpenMetadata @ Aspire by Vinol Joy Dsouza
OpenMetadata Spotlight - OpenMetadata @ Aspire by Vinol Joy DsouzaOpenMetadata Spotlight - OpenMetadata @ Aspire by Vinol Joy Dsouza
OpenMetadata Spotlight - OpenMetadata @ Aspire by Vinol Joy Dsouza
OpenMetadata
 
New Girls Call Noida 9873940964 Unlimited Short Providing Girls Service Avail...
New Girls Call Noida 9873940964 Unlimited Short Providing Girls Service Avail...New Girls Call Noida 9873940964 Unlimited Short Providing Girls Service Avail...
New Girls Call Noida 9873940964 Unlimited Short Providing Girls Service Avail...
kinni singh$A17
 
🚂🚘 Premium Girls Call Nashik 🛵🚡000XX00000 💃 Choose Best And Top Girl Service...
🚂🚘 Premium Girls Call Nashik  🛵🚡000XX00000 💃 Choose Best And Top Girl Service...🚂🚘 Premium Girls Call Nashik  🛵🚡000XX00000 💃 Choose Best And Top Girl Service...
🚂🚘 Premium Girls Call Nashik 🛵🚡000XX00000 💃 Choose Best And Top Girl Service...
kuldeepsharmaks8120
 
Why_are_we_hypnotizing_ourselves-_ATeggin-1.pdf
Why_are_we_hypnotizing_ourselves-_ATeggin-1.pdfWhy_are_we_hypnotizing_ourselves-_ATeggin-1.pdf
Why_are_we_hypnotizing_ourselves-_ATeggin-1.pdf
Alexander Teggin
 
Female Service Girls Call Navi Mumbai 9930245274 Provide Best And Top Girl Se...
Female Service Girls Call Navi Mumbai 9930245274 Provide Best And Top Girl Se...Female Service Girls Call Navi Mumbai 9930245274 Provide Best And Top Girl Se...
Female Service Girls Call Navi Mumbai 9930245274 Provide Best And Top Girl Se...
dizzycaye
 
🚂🚘 Premium Girls Call Bangalore 🛵🚡000XX00000 💃 Choose Best And Top Girl Serv...
🚂🚘 Premium Girls Call Bangalore  🛵🚡000XX00000 💃 Choose Best And Top Girl Serv...🚂🚘 Premium Girls Call Bangalore  🛵🚡000XX00000 💃 Choose Best And Top Girl Serv...
🚂🚘 Premium Girls Call Bangalore 🛵🚡000XX00000 💃 Choose Best And Top Girl Serv...
bhupeshkumar0889
 

Recently uploaded (20)

🚂🚘 Premium Girls Call Guwahati 🛵🚡000XX00000 💃 Choose Best And Top Girl Servi...
🚂🚘 Premium Girls Call Guwahati  🛵🚡000XX00000 💃 Choose Best And Top Girl Servi...🚂🚘 Premium Girls Call Guwahati  🛵🚡000XX00000 💃 Choose Best And Top Girl Servi...
🚂🚘 Premium Girls Call Guwahati 🛵🚡000XX00000 💃 Choose Best And Top Girl Servi...
 
potential usefulness of multi-agent maze-solving in general
potential usefulness of multi-agent maze-solving in generalpotential usefulness of multi-agent maze-solving in general
potential usefulness of multi-agent maze-solving in general
 
VIP Girls Call Mumbai 9910780858 Provide Best And Top Girl Service And No1 in...
VIP Girls Call Mumbai 9910780858 Provide Best And Top Girl Service And No1 in...VIP Girls Call Mumbai 9910780858 Provide Best And Top Girl Service And No1 in...
VIP Girls Call Mumbai 9910780858 Provide Best And Top Girl Service And No1 in...
 
Towards an Analysis-Ready, Cloud-Optimised service for FAIR fusion data
Towards an Analysis-Ready, Cloud-Optimised service for FAIR fusion dataTowards an Analysis-Ready, Cloud-Optimised service for FAIR fusion data
Towards an Analysis-Ready, Cloud-Optimised service for FAIR fusion data
 
Girls Call Chennai 000XX00000 Provide Best And Top Girl Service And No1 in City
Girls Call Chennai 000XX00000 Provide Best And Top Girl Service And No1 in CityGirls Call Chennai 000XX00000 Provide Best And Top Girl Service And No1 in City
Girls Call Chennai 000XX00000 Provide Best And Top Girl Service And No1 in City
 
Celebrity Girls Call Noida 9873940964 Unlimited Short Providing Girls Service...
Celebrity Girls Call Noida 9873940964 Unlimited Short Providing Girls Service...Celebrity Girls Call Noida 9873940964 Unlimited Short Providing Girls Service...
Celebrity Girls Call Noida 9873940964 Unlimited Short Providing Girls Service...
 
potential development of the A* search algorithm specifically
potential development of the A* search algorithm specificallypotential development of the A* search algorithm specifically
potential development of the A* search algorithm specifically
 
Verified Girls Call Andheri 9930245274 Unlimited Short Providing Girls Servic...
Verified Girls Call Andheri 9930245274 Unlimited Short Providing Girls Servic...Verified Girls Call Andheri 9930245274 Unlimited Short Providing Girls Servic...
Verified Girls Call Andheri 9930245274 Unlimited Short Providing Girls Servic...
 
Best Girls Call Navi Mumbai 9930245274 Provide Best And Top Girl Service And ...
Best Girls Call Navi Mumbai 9930245274 Provide Best And Top Girl Service And ...Best Girls Call Navi Mumbai 9930245274 Provide Best And Top Girl Service And ...
Best Girls Call Navi Mumbai 9930245274 Provide Best And Top Girl Service And ...
 
Celebrity Girls Call Andheri 9930245274 Unlimited Short Providing Girls Servi...
Celebrity Girls Call Andheri 9930245274 Unlimited Short Providing Girls Servi...Celebrity Girls Call Andheri 9930245274 Unlimited Short Providing Girls Servi...
Celebrity Girls Call Andheri 9930245274 Unlimited Short Providing Girls Servi...
 
Girls Call Vadodara 000XX00000 Provide Best And Top Girl Service And No1 in City
Girls Call Vadodara 000XX00000 Provide Best And Top Girl Service And No1 in CityGirls Call Vadodara 000XX00000 Provide Best And Top Girl Service And No1 in City
Girls Call Vadodara 000XX00000 Provide Best And Top Girl Service And No1 in City
 
社内勉強会資料_TransNeXt: Robust Foveal Visual Perception for Vision Transformers
社内勉強会資料_TransNeXt: Robust Foveal Visual Perception for Vision Transformers社内勉強会資料_TransNeXt: Robust Foveal Visual Perception for Vision Transformers
社内勉強会資料_TransNeXt: Robust Foveal Visual Perception for Vision Transformers
 
Beautiful Girls Call 9711199171 9711199171 Provide Best And Top Girl Service ...
Beautiful Girls Call 9711199171 9711199171 Provide Best And Top Girl Service ...Beautiful Girls Call 9711199171 9711199171 Provide Best And Top Girl Service ...
Beautiful Girls Call 9711199171 9711199171 Provide Best And Top Girl Service ...
 
VIP Kolkata Girls Call Kolkata 0X0000000X Doorstep High-Profile Girl Service ...
VIP Kolkata Girls Call Kolkata 0X0000000X Doorstep High-Profile Girl Service ...VIP Kolkata Girls Call Kolkata 0X0000000X Doorstep High-Profile Girl Service ...
VIP Kolkata Girls Call Kolkata 0X0000000X Doorstep High-Profile Girl Service ...
 
OpenMetadata Spotlight - OpenMetadata @ Aspire by Vinol Joy Dsouza
OpenMetadata Spotlight - OpenMetadata @ Aspire by Vinol Joy DsouzaOpenMetadata Spotlight - OpenMetadata @ Aspire by Vinol Joy Dsouza
OpenMetadata Spotlight - OpenMetadata @ Aspire by Vinol Joy Dsouza
 
New Girls Call Noida 9873940964 Unlimited Short Providing Girls Service Avail...
New Girls Call Noida 9873940964 Unlimited Short Providing Girls Service Avail...New Girls Call Noida 9873940964 Unlimited Short Providing Girls Service Avail...
New Girls Call Noida 9873940964 Unlimited Short Providing Girls Service Avail...
 
🚂🚘 Premium Girls Call Nashik 🛵🚡000XX00000 💃 Choose Best And Top Girl Service...
🚂🚘 Premium Girls Call Nashik  🛵🚡000XX00000 💃 Choose Best And Top Girl Service...🚂🚘 Premium Girls Call Nashik  🛵🚡000XX00000 💃 Choose Best And Top Girl Service...
🚂🚘 Premium Girls Call Nashik 🛵🚡000XX00000 💃 Choose Best And Top Girl Service...
 
Why_are_we_hypnotizing_ourselves-_ATeggin-1.pdf
Why_are_we_hypnotizing_ourselves-_ATeggin-1.pdfWhy_are_we_hypnotizing_ourselves-_ATeggin-1.pdf
Why_are_we_hypnotizing_ourselves-_ATeggin-1.pdf
 
Female Service Girls Call Navi Mumbai 9930245274 Provide Best And Top Girl Se...
Female Service Girls Call Navi Mumbai 9930245274 Provide Best And Top Girl Se...Female Service Girls Call Navi Mumbai 9930245274 Provide Best And Top Girl Se...
Female Service Girls Call Navi Mumbai 9930245274 Provide Best And Top Girl Se...
 
🚂🚘 Premium Girls Call Bangalore 🛵🚡000XX00000 💃 Choose Best And Top Girl Serv...
🚂🚘 Premium Girls Call Bangalore  🛵🚡000XX00000 💃 Choose Best And Top Girl Serv...🚂🚘 Premium Girls Call Bangalore  🛵🚡000XX00000 💃 Choose Best And Top Girl Serv...
🚂🚘 Premium Girls Call Bangalore 🛵🚡000XX00000 💃 Choose Best And Top Girl Serv...
 

The Python ecosystem for data science - Landscape Overview

  • 1. THE PYTHON ECOSYSTEM FOR DATA SCIENCE - LANDSCAPE OVERVIEW Ananth Krishnamoorthy, Ph.D. Outline Slides for Talk at Fifth Elephant 2017 25-Apr-2017
  • 2. Summary • In their day-to-day jobs, data science teams and data scientists face challenges in many overlapping yet distinct areas such as Reporting, Data Processing & Storage, Scientific Computing, ML Modelling, Application Development. To succeed, Data science teams, especially small ones, need a deep appreciation of these dependencies on their success. • Python ecosystem for data science has a number of tools and libraries for various aspects of data science, including Machine Learning, Cluster Computing, Scientific Computing, etc. • The idea of this talk is to understand what the Python data science ecosystem offers (so that you don't reinvent it), what are some common gaps (so that you don't go blue looking for answers). • In this talk, we describe how different tools/libraries fit in the machine learning model development and deployment workflow . This talk is about how these different tools work (and don’t work) together with each other. It is intended as a landscape survey of the python data science ecosystem, along with a mention of some common gaps that practitioners may notice as they put together a stack and/or an application for their company.
  • 3. The most important trait of the Analytics 3.0 era is that not only online firms, but virtually any type of firm in any industry, can participate in the data economy. Banks, industrial manufacturers, health care providers, retailers—any company in any industry that is willing to exploit the possibilities—can all develop data-based offerings for customers, as well as support internal decisions with big data. Analytics 1.0 Analytics 2.0 Analytics 3.0 Data  Enterprise Data  Structured transactional data  Bring in web and social data  Complex, large, semistructured data sources  GPS, Mobile Device, Clickstream, Sensor data  Unstructured, real time, streaming Tools  Spreadsheets  BI, OLAP  ETL  On-premise servers  Visualization  NoSQL  Hadoop  Machine Learning , Artificial Intelligence  On-Demand Everything  Analytical Apps  Integrated, Embedded models Activity  Majority of analytical activity was descriptive analytics, or reporting  Creating analytical models was a time-consuming “batch” process  Visual analytics dominates predictive and prescriptive techniques  Develop products, not PowerPoints or reports  Analytics integral to running the business, strategic asset  Rapid and agile insight delivery  Analytical tools available at point of decision Source: THE RISE OF ANALYTICS 3.0, By Thomas H. Davenport, IIA, 2013 Evolving Role of Data Science Teams
  • 4. Machine Learning vs Real World Data Science Machine Learning Deployment Application Development Big Data Processing Data Storage ETL
  • 5. Challenges faced by Data Science Teams • Requires many more competencies than can be reasonably expected from one person • Challenges are greater for smaller teams and smaller companies, e.g. startups • Challenges create dependencies on other teams e.g. Development • Dependencies slow down execution and benefits realization
  • 6. Plethora of Choices Reporting Data Processing & Storage Scientific Computing ML Modelling Application Development SQL NoSQL Graphdb OLAP ETL Cluster Computing Stream Processing SQL Charting Statistics Cloud Front End Microservices Back End ML Deep Learning Dim. Reduction Signal Processing Optimization Time Series Analysis Simulation MapReduce
  • 7. Data Science Workflow ETL Process ModelStore Deploy DATA SCIENTIST SKILLS Infrastructure and Provisioning ???
  • 8. Python Ecosystem ETL Process ModelStore Deploy Odo Blaze Pandas Dask Spark Sklearn_Pandas Scikit-learn Keras Spark MLlib Bokeh Jupyter
  • 9. Review of Key Tools (50% of talk time spent here, more slides to be added) • Jupyter • Pandas • Scikit-Learn • Keras / TensorFlow / Theano • Matplotlib/Bokeh • Blaze • Odo • Dask • pySpark We shall see some code snippets here, to illustrate a few ideas The idea is to know enough to pick the right components for the job at hand
  • 10. Use Case 1: Small Data This use case will illustrate case of Small Data i.e. Desktop / In-memory processing
  • 11. Use Case 2: ‘Medium’ Data This use case will illustrate case of Medium Data with Out-of-core processing
  • 12. Use Case 3: Big Data This use case will illustrate case of Big Data i.e cluster computing
  • 13. What Works • Sklearn’s Consistent API, wide variety of ML algorithms • Sklearn Pipelines • Scikit-Keras Integration • Pandas for Data Analysis • …. • ….
  • 14. Gaps – A Practitioner Perspective • Uniform API Across Activities • Separation of Data, Processing, and Instructions • Single Data Structure Paradigm • Support for in-memory, out-of-core, and distributed computing in same paradigm e.g. SFrame • ETL • Push heavy lifting to backend systems • Monitoring workflows • Application development • Bokeh • Deployment • Application • Web Services

Editor's Notes

  1. Slide needs improvement 