SlideShare a Scribd company logo
1 of 19
Download to read offline
Esri International User Conference | San Diego, CA
Technical Workshops | Tuesday, July 12th


Python: Integrating Standard and
Third-Party Libraries
Jason Scheirer
Other Sessions
•   Building Tools with Python (Technical Workshop)

     -   Thursday 10:15AM 9

•   Getting Started With Map Algebra Using the Raster Calculator and Python (Demo Theater Presentation)

     -   Wednesday 10:00AM Exhibit Hall C (Spatial Analysis Demo Theater)

•   Python - Analyzing your GPS tracking data (Demo Theater Presentation)

     -   Thursday 11:00AM Exhibit Hall C (Spatial Analysis Demo Theater)

•   Python - Automating Geodatabase Administration (Tech Workshop 20 Minute)

     -   Thursday 11:05AM 23 B

•   Python - Creating an ArcGIS Script Tool (Demo Theater Presentation)

     -   Tuesday 4:00PM Exhibit Hall C (Spatial Analysis Demo Theater)

•   Python - Getting Started (Technical Workshop)

     -   Tuesday 1:30PM 2
     -   Thursday 8:30AM 2
Other Sessions (continued)
•   Python - Getting to know the Python Window (Demo Theater Presentation)

     -   Tuesday 10:00AM Exhibit Hall C (Spatial Analysis Demo Theater)

•   Python - Improving the Performance of your Script Tools (Tech Workshop 20 Minute)

     -   Tuesday 8:30AM 6 A

•   Python - Raster Analysis (Technical Workshop)

     -   Tuesday 3:15PM 6 C
     -   Wednesday 3:15PM 5 A/B

•   Python - Scripting for Map Automation (Technical Workshop)

     -   Tuesday 10:15AM 9
     -   Wednesday 3:15PM 9

•   Python - Spatial Analysis (Intermediate) (Technical Workshop)

     -   Tuesday 3:15PM 1 A/B
     -   Thursday 10:15AM 2
Other Sessions (yes, there are more)
•   Python - The Ease and Power of Cursors (Demo Theater Presentation)

     -   Tuesday 11:30AM Exhibit Hall C (Spatial Analysis Demo Theater)

•   Python Scripts - Using A Template (Demo Theater Presentation)

     -   Tuesday 11:00AM Exhibit Hall C (Spatial Analysis Demo Theater)

•   Road Ahead - Python Scripting Abilities

     -   Thursday 4:05PM 6 B
Python in ArcGIS
 •   Implementing geoprocessing script tools
      -   Creating an ArcGIS Script Tool (Tuesday 4:00PM)
      -   Building Tools with Python (Thursday 10:15AM)

 •   Automating workflows
      -   Gluing together other GP tools

      -   Scheduled tasks

 •   Integrating ArcGIS with other systems, other systems
     into ArcGIS
      -   Data interop: import/export
      -   Automating ArcGIS processes from other applications
      -   Preparing data from ArcGIS for consumption by other applications
Built-in Libraries
Batteries Included
 •   Libraries for many common tasks come with the
     language
     -   File formats: XML (DOM, SAX, ElementTree), JSON, CSV, .ZIP
         (zipfile)
     -   Networking: HTTP (urllib, urllib2), TCP/UDP (socket)
     -   Parallelism: threading, multiprocessing, subprocess
     -   Command line applications: optparse
     -   Paths, directories, files: sys, os, os.path
     -   User Interfaces: tkinter
     -   Data structures and algorithms: collections, itertools,
         heapq, struct, many more
     -   Date/time (time, datetime)
     -   Calling C++/C (ctypes)
Read the Docs!
•   Help file: ArcGIS 10.0 | Python 2.6 | Python Manuals
Read the Docs!
•   What’s installed: ArcGIS 10.0 | Python 2.6 | Module Docs
More Help

•   PyMOTW (Python Module of the Week)
    -   http://www.doughellmann.com/PyMOTW/
    -   Also available as a book
Example


            •   Summary Statistics in R
                -   Uses a third-party module for reading spatial data: maptools
                -   Only accepts shapefiles!
            •   Going to use two different system libraries:
                -   subprocess (Open/monitor other programs)
                -   re (Regular Expressions)



CopyFeatures to temporary Shapefile    Run R Script   Clean up temp file   Parse R Script’s Output




                                                                              Statistics!
Demo
But don’t reinvent the wheel!


  •   wget: Popular utility to download resources over
      HTTP/FTP
  •   Do I need it?
       -   urllib2 can download simple URLs
  •   But I need to follow links
       -   Don’t write a web spider in Python, use the –r option on
           wget
       -   Use subprocess, like in the previous example
Third-Party Libraries
The Cheese Shop
  •   http://pypi.python.org/pypi
  •   The official place for 3rd party Python modules
Some Interesting Modules
  •   PDF writing – reportlab
  •   Image manipulation – PIL
  •   (Messy) HTML – beautifulsoup
  •   Advanced graphing/plotting – matplotlib
  •   Advanced date/time handling – dateutil
  •   Excel
      -   Read – xlrd
      -   Write – xlwt
  •   Consuming .Net
      -   http://pythonnet.sourceforge.net/
  •   .docx Read/write – docx
      -   https://github.com/mikemaccana/python-docx
      -   This brings us to…
There’s No Installer!


  •   Find python.exe in your install, add it to %PATH%.
      Scripts too.
  •   Try pip:
       -   http://python-distribute.org/
  •   Or on the command line:
       -   python.exe setup.py install
  •   Look for it on an unofficial Python library builds site:
       -   http://www.lfd.uci.edu/~gohlke/pythonlibs/
Conclusion


  •   A lot of what you need is already there
  •   Amazing documentation at your fingertips, more online
  •   There are built-in ways of interfacing with non-Python
      utilities (great for using script tools to integrate into
      ArcGIS)
  •   There’s a HUGE ecosystem out there that probably
      already has what you need
Esri International User Conference 2011: Python: Integrating Standard and Third-Party Libraries

More Related Content

What's hot

Protocol libraries the right way
Protocol libraries the right wayProtocol libraries the right way
Protocol libraries the right way
Leo Zhou
 
Buzzwords Numba Presentation
Buzzwords Numba PresentationBuzzwords Numba Presentation
Buzzwords Numba Presentation
kammeyer
 
Numba: Flexible analytics written in Python with machine-code speeds and avo...
Numba:  Flexible analytics written in Python with machine-code speeds and avo...Numba:  Flexible analytics written in Python with machine-code speeds and avo...
Numba: Flexible analytics written in Python with machine-code speeds and avo...
PyData
 
Map Analytics in Starcraft II
Map Analytics in Starcraft IIMap Analytics in Starcraft II
Map Analytics in Starcraft II
gy8
 

What's hot (10)

Protocol libraries the right way
Protocol libraries the right wayProtocol libraries the right way
Protocol libraries the right way
 
Buzzwords Numba Presentation
Buzzwords Numba PresentationBuzzwords Numba Presentation
Buzzwords Numba Presentation
 
Scientific Plotting in Python
Scientific Plotting in PythonScientific Plotting in Python
Scientific Plotting in Python
 
Python VS GO
Python VS GOPython VS GO
Python VS GO
 
Object Detection with Tensorflow
Object Detection with TensorflowObject Detection with Tensorflow
Object Detection with Tensorflow
 
Numba: Flexible analytics written in Python with machine-code speeds and avo...
Numba:  Flexible analytics written in Python with machine-code speeds and avo...Numba:  Flexible analytics written in Python with machine-code speeds and avo...
Numba: Flexible analytics written in Python with machine-code speeds and avo...
 
Map Analytics in Starcraft II
Map Analytics in Starcraft IIMap Analytics in Starcraft II
Map Analytics in Starcraft II
 
Go: Beyond the Basics
Go: Beyond the BasicsGo: Beyond the Basics
Go: Beyond the Basics
 
FleetDB
FleetDBFleetDB
FleetDB
 
Accelerate Your Python* Code through Profiling, Tuning, and Compilation Part ...
Accelerate Your Python* Code through Profiling, Tuning, and Compilation Part ...Accelerate Your Python* Code through Profiling, Tuning, and Compilation Part ...
Accelerate Your Python* Code through Profiling, Tuning, and Compilation Part ...
 

Similar to Esri International User Conference 2011: Python: Integrating Standard and Third-Party Libraries

Python For Audio Signal Processing ( PDFDrive ).pdf
Python For Audio Signal Processing ( PDFDrive ).pdfPython For Audio Signal Processing ( PDFDrive ).pdf
Python For Audio Signal Processing ( PDFDrive ).pdf
shaikriyaz89
 
Monitoraggio del Traffico di Rete Usando Python ed ntop
Monitoraggio del Traffico di Rete Usando Python ed ntopMonitoraggio del Traffico di Rete Usando Python ed ntop
Monitoraggio del Traffico di Rete Usando Python ed ntop
PyCon Italia
 
How to integrate python into a scala stack
How to integrate python into a scala stackHow to integrate python into a scala stack
How to integrate python into a scala stack
Fliptop
 
Presto as a Service - Tips for operation and monitoring
Presto as a Service - Tips for operation and monitoringPresto as a Service - Tips for operation and monitoring
Presto as a Service - Tips for operation and monitoring
Taro L. Saito
 

Similar to Esri International User Conference 2011: Python: Integrating Standard and Third-Party Libraries (20)

Esri South Africa Python for Everyone
Esri South Africa Python for EveryoneEsri South Africa Python for Everyone
Esri South Africa Python for Everyone
 
Reproducibility and automation of machine learning process
Reproducibility and automation of machine learning processReproducibility and automation of machine learning process
Reproducibility and automation of machine learning process
 
Python ml
Python mlPython ml
Python ml
 
How to write a well-behaved Python command line application
How to write a well-behaved Python command line applicationHow to write a well-behaved Python command line application
How to write a well-behaved Python command line application
 
Python and GIS: Improving Your Workflow
Python and GIS: Improving Your WorkflowPython and GIS: Improving Your Workflow
Python and GIS: Improving Your Workflow
 
Overview of Modern Graph Analysis Tools
Overview of Modern Graph Analysis ToolsOverview of Modern Graph Analysis Tools
Overview of Modern Graph Analysis Tools
 
from ai.backend import python @ pycontw2018
from ai.backend import python @ pycontw2018from ai.backend import python @ pycontw2018
from ai.backend import python @ pycontw2018
 
PyData Boston 2013
PyData Boston 2013PyData Boston 2013
PyData Boston 2013
 
Cytoscape and External Data Analysis Tools
Cytoscape and External Data Analysis ToolsCytoscape and External Data Analysis Tools
Cytoscape and External Data Analysis Tools
 
Python For Audio Signal Processing ( PDFDrive ).pdf
Python For Audio Signal Processing ( PDFDrive ).pdfPython For Audio Signal Processing ( PDFDrive ).pdf
Python For Audio Signal Processing ( PDFDrive ).pdf
 
Monitoraggio del Traffico di Rete Usando Python ed ntop
Monitoraggio del Traffico di Rete Usando Python ed ntopMonitoraggio del Traffico di Rete Usando Python ed ntop
Monitoraggio del Traffico di Rete Usando Python ed ntop
 
Analysing GitHub commits with R
Analysing GitHub commits with RAnalysing GitHub commits with R
Analysing GitHub commits with R
 
Python in geospatial analysis
Python in geospatial analysisPython in geospatial analysis
Python in geospatial analysis
 
How to integrate python into a scala stack
How to integrate python into a scala stackHow to integrate python into a scala stack
How to integrate python into a scala stack
 
Travis Oliphant "Python for Speed, Scale, and Science"
Travis Oliphant "Python for Speed, Scale, and Science"Travis Oliphant "Python for Speed, Scale, and Science"
Travis Oliphant "Python for Speed, Scale, and Science"
 
Practical automation for beginners
Practical automation for beginnersPractical automation for beginners
Practical automation for beginners
 
Matplotlib presentation 20 apr2013 final
Matplotlib presentation 20 apr2013   finalMatplotlib presentation 20 apr2013   final
Matplotlib presentation 20 apr2013 final
 
Presto as a Service - Tips for operation and monitoring
Presto as a Service - Tips for operation and monitoringPresto as a Service - Tips for operation and monitoring
Presto as a Service - Tips for operation and monitoring
 
HiPEAC 2019 Tutorial - Maestro RTOS
HiPEAC 2019 Tutorial - Maestro RTOSHiPEAC 2019 Tutorial - Maestro RTOS
HiPEAC 2019 Tutorial - Maestro RTOS
 
Monitorama 2015 Netflix Instance Analysis
Monitorama 2015 Netflix Instance AnalysisMonitorama 2015 Netflix Instance Analysis
Monitorama 2015 Netflix Instance Analysis
 

Recently uploaded

EIS-Webinar-Prompt-Knowledge-Eng-2024-04-08.pptx
EIS-Webinar-Prompt-Knowledge-Eng-2024-04-08.pptxEIS-Webinar-Prompt-Knowledge-Eng-2024-04-08.pptx
EIS-Webinar-Prompt-Knowledge-Eng-2024-04-08.pptx
Earley Information Science
 
IAC 2024 - IA Fast Track to Search Focused AI Solutions
IAC 2024 - IA Fast Track to Search Focused AI SolutionsIAC 2024 - IA Fast Track to Search Focused AI Solutions
IAC 2024 - IA Fast Track to Search Focused AI Solutions
Enterprise Knowledge
 
Artificial Intelligence: Facts and Myths
Artificial Intelligence: Facts and MythsArtificial Intelligence: Facts and Myths
Artificial Intelligence: Facts and Myths
Joaquim Jorge
 

Recently uploaded (20)

TrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments
TrustArc Webinar - Stay Ahead of US State Data Privacy Law DevelopmentsTrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments
TrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments
 
08448380779 Call Girls In Civil Lines Women Seeking Men
08448380779 Call Girls In Civil Lines Women Seeking Men08448380779 Call Girls In Civil Lines Women Seeking Men
08448380779 Call Girls In Civil Lines Women Seeking Men
 
Axa Assurance Maroc - Insurer Innovation Award 2024
Axa Assurance Maroc - Insurer Innovation Award 2024Axa Assurance Maroc - Insurer Innovation Award 2024
Axa Assurance Maroc - Insurer Innovation Award 2024
 
A Domino Admins Adventures (Engage 2024)
A Domino Admins Adventures (Engage 2024)A Domino Admins Adventures (Engage 2024)
A Domino Admins Adventures (Engage 2024)
 
The Codex of Business Writing Software for Real-World Solutions 2.pptx
The Codex of Business Writing Software for Real-World Solutions 2.pptxThe Codex of Business Writing Software for Real-World Solutions 2.pptx
The Codex of Business Writing Software for Real-World Solutions 2.pptx
 
EIS-Webinar-Prompt-Knowledge-Eng-2024-04-08.pptx
EIS-Webinar-Prompt-Knowledge-Eng-2024-04-08.pptxEIS-Webinar-Prompt-Knowledge-Eng-2024-04-08.pptx
EIS-Webinar-Prompt-Knowledge-Eng-2024-04-08.pptx
 
Handwritten Text Recognition for manuscripts and early printed texts
Handwritten Text Recognition for manuscripts and early printed textsHandwritten Text Recognition for manuscripts and early printed texts
Handwritten Text Recognition for manuscripts and early printed texts
 
How to Troubleshoot Apps for the Modern Connected Worker
How to Troubleshoot Apps for the Modern Connected WorkerHow to Troubleshoot Apps for the Modern Connected Worker
How to Troubleshoot Apps for the Modern Connected Worker
 
Powerful Google developer tools for immediate impact! (2023-24 C)
Powerful Google developer tools for immediate impact! (2023-24 C)Powerful Google developer tools for immediate impact! (2023-24 C)
Powerful Google developer tools for immediate impact! (2023-24 C)
 
08448380779 Call Girls In Diplomatic Enclave Women Seeking Men
08448380779 Call Girls In Diplomatic Enclave Women Seeking Men08448380779 Call Girls In Diplomatic Enclave Women Seeking Men
08448380779 Call Girls In Diplomatic Enclave Women Seeking Men
 
Factors to Consider When Choosing Accounts Payable Services Providers.pptx
Factors to Consider When Choosing Accounts Payable Services Providers.pptxFactors to Consider When Choosing Accounts Payable Services Providers.pptx
Factors to Consider When Choosing Accounts Payable Services Providers.pptx
 
Scaling API-first – The story of a global engineering organization
Scaling API-first – The story of a global engineering organizationScaling API-first – The story of a global engineering organization
Scaling API-first – The story of a global engineering organization
 
Advantages of Hiring UIUX Design Service Providers for Your Business
Advantages of Hiring UIUX Design Service Providers for Your BusinessAdvantages of Hiring UIUX Design Service Providers for Your Business
Advantages of Hiring UIUX Design Service Providers for Your Business
 
IAC 2024 - IA Fast Track to Search Focused AI Solutions
IAC 2024 - IA Fast Track to Search Focused AI SolutionsIAC 2024 - IA Fast Track to Search Focused AI Solutions
IAC 2024 - IA Fast Track to Search Focused AI Solutions
 
Boost Fertility New Invention Ups Success Rates.pdf
Boost Fertility New Invention Ups Success Rates.pdfBoost Fertility New Invention Ups Success Rates.pdf
Boost Fertility New Invention Ups Success Rates.pdf
 
Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...
Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...
Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...
 
A Call to Action for Generative AI in 2024
A Call to Action for Generative AI in 2024A Call to Action for Generative AI in 2024
A Call to Action for Generative AI in 2024
 
[2024]Digital Global Overview Report 2024 Meltwater.pdf
[2024]Digital Global Overview Report 2024 Meltwater.pdf[2024]Digital Global Overview Report 2024 Meltwater.pdf
[2024]Digital Global Overview Report 2024 Meltwater.pdf
 
🐬 The future of MySQL is Postgres 🐘
🐬  The future of MySQL is Postgres   🐘🐬  The future of MySQL is Postgres   🐘
🐬 The future of MySQL is Postgres 🐘
 
Artificial Intelligence: Facts and Myths
Artificial Intelligence: Facts and MythsArtificial Intelligence: Facts and Myths
Artificial Intelligence: Facts and Myths
 

Esri International User Conference 2011: Python: Integrating Standard and Third-Party Libraries

  • 1. Esri International User Conference | San Diego, CA Technical Workshops | Tuesday, July 12th Python: Integrating Standard and Third-Party Libraries Jason Scheirer
  • 2. Other Sessions • Building Tools with Python (Technical Workshop) - Thursday 10:15AM 9 • Getting Started With Map Algebra Using the Raster Calculator and Python (Demo Theater Presentation) - Wednesday 10:00AM Exhibit Hall C (Spatial Analysis Demo Theater) • Python - Analyzing your GPS tracking data (Demo Theater Presentation) - Thursday 11:00AM Exhibit Hall C (Spatial Analysis Demo Theater) • Python - Automating Geodatabase Administration (Tech Workshop 20 Minute) - Thursday 11:05AM 23 B • Python - Creating an ArcGIS Script Tool (Demo Theater Presentation) - Tuesday 4:00PM Exhibit Hall C (Spatial Analysis Demo Theater) • Python - Getting Started (Technical Workshop) - Tuesday 1:30PM 2 - Thursday 8:30AM 2
  • 3. Other Sessions (continued) • Python - Getting to know the Python Window (Demo Theater Presentation) - Tuesday 10:00AM Exhibit Hall C (Spatial Analysis Demo Theater) • Python - Improving the Performance of your Script Tools (Tech Workshop 20 Minute) - Tuesday 8:30AM 6 A • Python - Raster Analysis (Technical Workshop) - Tuesday 3:15PM 6 C - Wednesday 3:15PM 5 A/B • Python - Scripting for Map Automation (Technical Workshop) - Tuesday 10:15AM 9 - Wednesday 3:15PM 9 • Python - Spatial Analysis (Intermediate) (Technical Workshop) - Tuesday 3:15PM 1 A/B - Thursday 10:15AM 2
  • 4. Other Sessions (yes, there are more) • Python - The Ease and Power of Cursors (Demo Theater Presentation) - Tuesday 11:30AM Exhibit Hall C (Spatial Analysis Demo Theater) • Python Scripts - Using A Template (Demo Theater Presentation) - Tuesday 11:00AM Exhibit Hall C (Spatial Analysis Demo Theater) • Road Ahead - Python Scripting Abilities - Thursday 4:05PM 6 B
  • 5. Python in ArcGIS • Implementing geoprocessing script tools - Creating an ArcGIS Script Tool (Tuesday 4:00PM) - Building Tools with Python (Thursday 10:15AM) • Automating workflows - Gluing together other GP tools - Scheduled tasks • Integrating ArcGIS with other systems, other systems into ArcGIS - Data interop: import/export - Automating ArcGIS processes from other applications - Preparing data from ArcGIS for consumption by other applications
  • 7. Batteries Included • Libraries for many common tasks come with the language - File formats: XML (DOM, SAX, ElementTree), JSON, CSV, .ZIP (zipfile) - Networking: HTTP (urllib, urllib2), TCP/UDP (socket) - Parallelism: threading, multiprocessing, subprocess - Command line applications: optparse - Paths, directories, files: sys, os, os.path - User Interfaces: tkinter - Data structures and algorithms: collections, itertools, heapq, struct, many more - Date/time (time, datetime) - Calling C++/C (ctypes)
  • 8. Read the Docs! • Help file: ArcGIS 10.0 | Python 2.6 | Python Manuals
  • 9. Read the Docs! • What’s installed: ArcGIS 10.0 | Python 2.6 | Module Docs
  • 10. More Help • PyMOTW (Python Module of the Week) - http://www.doughellmann.com/PyMOTW/ - Also available as a book
  • 11. Example • Summary Statistics in R - Uses a third-party module for reading spatial data: maptools - Only accepts shapefiles! • Going to use two different system libraries: - subprocess (Open/monitor other programs) - re (Regular Expressions) CopyFeatures to temporary Shapefile Run R Script Clean up temp file Parse R Script’s Output Statistics!
  • 12. Demo
  • 13. But don’t reinvent the wheel! • wget: Popular utility to download resources over HTTP/FTP • Do I need it? - urllib2 can download simple URLs • But I need to follow links - Don’t write a web spider in Python, use the –r option on wget - Use subprocess, like in the previous example
  • 15. The Cheese Shop • http://pypi.python.org/pypi • The official place for 3rd party Python modules
  • 16. Some Interesting Modules • PDF writing – reportlab • Image manipulation – PIL • (Messy) HTML – beautifulsoup • Advanced graphing/plotting – matplotlib • Advanced date/time handling – dateutil • Excel - Read – xlrd - Write – xlwt • Consuming .Net - http://pythonnet.sourceforge.net/ • .docx Read/write – docx - https://github.com/mikemaccana/python-docx - This brings us to…
  • 17. There’s No Installer! • Find python.exe in your install, add it to %PATH%. Scripts too. • Try pip: - http://python-distribute.org/ • Or on the command line: - python.exe setup.py install • Look for it on an unofficial Python library builds site: - http://www.lfd.uci.edu/~gohlke/pythonlibs/
  • 18. Conclusion • A lot of what you need is already there • Amazing documentation at your fingertips, more online • There are built-in ways of interfacing with non-Python utilities (great for using script tools to integrate into ArcGIS) • There’s a HUGE ecosystem out there that probably already has what you need