SlideShare a Scribd company logo
Performance
Enhancement
Tips
Don’t Write too much

Wen Chang Hsu
石頭湯
主菜們站出來
Who am I
 Wen   Chang Hsu ( you can find me via
  Tim)
 Dorm7 Software, 好客民宿
 Slide Note 我想讓它上線…..
 Repository   find timtan in GitHub
      https://github.com/timtan/python-
     performance-tips.git
    pip install -r requirement.txt
Profile
 Now   see profile_sample1.py
profile_sample1
Who is slow?
 python   -m cProfile -s cumulative profile_sample1.py

    -m cProfile means directly invoke the module
    -s          is the sort order.
The nature of cProfile
 Deterministic  profiling,
 Python interpreter have hook on each
  function call.
Summary
 Profilingfirst, don’t guess
 Use the command
 python -m cProfile -s cumulative
  profile_sample1.py
If you don’t have time
 Reduce    complex is better



 You   can use pypy !!
But !!
C   Extension is not available
 However, original standard library written
  in C are replaced with pure python
How Quora think
 http://www.quora.com/PyPy/Will-PyPy-
  be-the-standard-Python-implementation
 Page loading time boost 2x. But lxml, pyml
  cannot runs in PyPy.
 Communications between Cpython and
  PyPY
You can use Cython
 Compile python module to C code
 Compile the c Code to python module
 You change no code, 20% boost
Key Point to use Cython
In the example
 typemake in your shell
 How to write setup.py is a little tricky if you
  want to use cython and setuptools at the
  same time.
Summary
 PyPy   is good and near production
    C Extension is in experiment
 Cython is more realistic, and you can
 integrate it with existing C module easily
Parallel ?
 大量的資料做同樣的事
 Gevent,   multiprocessing, Thread
 Celery ( I won’t cover this today )
Reddit says
 Thread  in python sucks
 Multiprocessing is good
computation_parallel_example.py
7 second
computation_parallel_example_threading.py
7 seconds again
Why
 Python     Has a GIL
 The max function call is not preemptable,
  it is written in C
 The interrupter cannot yield form the
  function call
Multiprocessing
4s, faster than multithreading
Summary Multiprocessing
 It did fork process. (consumes memory)
 It can utilize all your core
Trick For Multiprocessing
 pool = multiprocessing.Pool(10)
 pool.map( function, data)


 Than
     you get 10 workers that will help you
 process data
e.g.

Error !!!
Summary of Multiprocessing
 It   did fork !!
      Previous data is duplicated, you should
 The only way to communicate data
  between process. It use IPC
 The argument, return value function
  should be pickable
PyCon TW needs U
不要再叫 Tim 了

More Related Content

What's hot

PHP, Under The Hood - DPC
PHP, Under The Hood - DPCPHP, Under The Hood - DPC
PHP, Under The Hood - DPC
Anthony Ferrara
 
Real World ChatOps
Real World ChatOpsReal World ChatOps
Real World ChatOps
VictorOps
 
Python
PythonPython
Top 10 python ide
Top 10 python ideTop 10 python ide
Top 10 python ide
Saravanakumar viswanathan
 
Metaprogramming in Ruby
Metaprogramming in RubyMetaprogramming in Ruby
Metaprogramming in Ruby
John Vlachoyiannis
 
ChatOps
ChatOpsChatOps
Errbit & Gitlab
Errbit & GitlabErrbit & Gitlab
Errbit & Gitlab
motioneleven
 
Project Linne 徵音梅林 -- Virtual Singer Sound-bank Processed with Python
Project Linne 徵音梅林 -- Virtual Singer Sound-bank Processed with PythonProject Linne 徵音梅林 -- Virtual Singer Sound-bank Processed with Python
Project Linne 徵音梅林 -- Virtual Singer Sound-bank Processed with Python
Yuan CHAO
 
Geekcamp ID 2015: Programmable Music
Geekcamp ID 2015: Programmable MusicGeekcamp ID 2015: Programmable Music
Geekcamp ID 2015: Programmable Music
Asep Bagja
 

What's hot (9)

PHP, Under The Hood - DPC
PHP, Under The Hood - DPCPHP, Under The Hood - DPC
PHP, Under The Hood - DPC
 
Real World ChatOps
Real World ChatOpsReal World ChatOps
Real World ChatOps
 
Python
PythonPython
Python
 
Top 10 python ide
Top 10 python ideTop 10 python ide
Top 10 python ide
 
Metaprogramming in Ruby
Metaprogramming in RubyMetaprogramming in Ruby
Metaprogramming in Ruby
 
ChatOps
ChatOpsChatOps
ChatOps
 
Errbit & Gitlab
Errbit & GitlabErrbit & Gitlab
Errbit & Gitlab
 
Project Linne 徵音梅林 -- Virtual Singer Sound-bank Processed with Python
Project Linne 徵音梅林 -- Virtual Singer Sound-bank Processed with PythonProject Linne 徵音梅林 -- Virtual Singer Sound-bank Processed with Python
Project Linne 徵音梅林 -- Virtual Singer Sound-bank Processed with Python
 
Geekcamp ID 2015: Programmable Music
Geekcamp ID 2015: Programmable MusicGeekcamp ID 2015: Programmable Music
Geekcamp ID 2015: Programmable Music
 

Viewers also liked

Internship
InternshipInternship
Internship
sanjay kiran
 
Main project
Main projectMain project
Main project
muhammed anees
 
Project
ProjectProject
Project
sanjay kiran
 
hr project
hr projecthr project
hr project
bignu000
 
Summer internship project hr mba mms
Summer internship project hr mba mmsSummer internship project hr mba mms
Summer internship project hr mba mms
Sria
 
Internship report
Internship reportInternship report
Internship report
Sajjad Ashraf
 
Cooking oil
Cooking oilCooking oil
Cooking oil
Hari Krishnan
 
Summer internship project report
Summer internship project reportSummer internship project report
Summer internship project report
Manish Singh
 
MBA Internship Report
MBA Internship ReportMBA Internship Report
MBA Internship Report
Ajesh U Bhanu
 
A study on employee job satisfaction h r final project
A study on employee job satisfaction h r final projectA study on employee job satisfaction h r final project
A study on employee job satisfaction h r final project
Projects Kart
 

Viewers also liked (10)

Internship
InternshipInternship
Internship
 
Main project
Main projectMain project
Main project
 
Project
ProjectProject
Project
 
hr project
hr projecthr project
hr project
 
Summer internship project hr mba mms
Summer internship project hr mba mmsSummer internship project hr mba mms
Summer internship project hr mba mms
 
Internship report
Internship reportInternship report
Internship report
 
Cooking oil
Cooking oilCooking oil
Cooking oil
 
Summer internship project report
Summer internship project reportSummer internship project report
Summer internship project report
 
MBA Internship Report
MBA Internship ReportMBA Internship Report
MBA Internship Report
 
A study on employee job satisfaction h r final project
A study on employee job satisfaction h r final projectA study on employee job satisfaction h r final project
A study on employee job satisfaction h r final project
 

Similar to Performance Enhancement Tips

PyCon2022 - Building Python Extensions
PyCon2022 - Building Python ExtensionsPyCon2022 - Building Python Extensions
PyCon2022 - Building Python Extensions
Henry Schreiner
 
Euro python2011 High Performance Python
Euro python2011 High Performance PythonEuro python2011 High Performance Python
Euro python2011 High Performance Python
Ian Ozsvald
 
Pythonic doesn't mean slow!
Pythonic doesn't mean slow!Pythonic doesn't mean slow!
Pythonic doesn't mean slow!
Ronan Lamy
 
Pypy is-it-ready-for-production-the-sequel
Pypy is-it-ready-for-production-the-sequelPypy is-it-ready-for-production-the-sequel
Pypy is-it-ready-for-production-the-sequel
Mark Rees
 
Running a Plone product on Substance D
Running a Plone product on Substance DRunning a Plone product on Substance D
Running a Plone product on Substance D
Makina Corpus
 
PyPy London Demo Evening 2013
PyPy London Demo Evening 2013PyPy London Demo Evening 2013
PyPy London Demo Evening 2013
Carl Friedrich Bolz
 
Writing Fast Code (JP) - PyCon JP 2015
Writing Fast Code (JP) - PyCon JP 2015Writing Fast Code (JP) - PyCon JP 2015
Writing Fast Code (JP) - PyCon JP 2015
Younggun Kim
 
Writing Fast Code - PyCon HK 2015
Writing Fast Code - PyCon HK 2015Writing Fast Code - PyCon HK 2015
Writing Fast Code - PyCon HK 2015
Younggun Kim
 
First Steps in Python Programming
First Steps in Python ProgrammingFirst Steps in Python Programming
First Steps in Python Programming
Dozie Agbo
 
PyPy - is it ready for production
PyPy - is it ready for productionPyPy - is it ready for production
PyPy - is it ready for production
Mark Rees
 
Advanced topics with python
Advanced topics with pythonAdvanced topics with python
Advanced topics with python
saipreethi16
 
Advanced topics with python
Advanced topics with pythonAdvanced topics with python
Advanced topics with python
saijohn1997
 
Infrastructure-as-Code with Pulumi - Better than all the others (like Ansible)?
Infrastructure-as-Code with Pulumi- Better than all the others (like Ansible)?Infrastructure-as-Code with Pulumi- Better than all the others (like Ansible)?
Infrastructure-as-Code with Pulumi - Better than all the others (like Ansible)?
Jonas Hecht
 
Rusty Python
Rusty PythonRusty Python
Rusty Python
RangHo Lee
 
Odoo profiler
Odoo profilerOdoo profiler
Odoo profiler
Angga Faizul Ikrom
 
Cfgmgmt Challenges aren't technical anymore
Cfgmgmt Challenges aren't technical anymoreCfgmgmt Challenges aren't technical anymore
Cfgmgmt Challenges aren't technical anymore
Julien Pivotto
 
Take a Stroll in the Bazaar
Take a Stroll in the BazaarTake a Stroll in the Bazaar
Take a Stroll in the Bazaar
Myles Braithwaite
 
Pythonpresent
PythonpresentPythonpresent
Pythonpresent
Chui-Wen Chiu
 
Multiprocessing with python
Multiprocessing with pythonMultiprocessing with python
Multiprocessing with python
Patrick Vergain
 
Introduction to Python
Introduction to PythonIntroduction to Python
Introduction to Python
Spotle.ai
 

Similar to Performance Enhancement Tips (20)

PyCon2022 - Building Python Extensions
PyCon2022 - Building Python ExtensionsPyCon2022 - Building Python Extensions
PyCon2022 - Building Python Extensions
 
Euro python2011 High Performance Python
Euro python2011 High Performance PythonEuro python2011 High Performance Python
Euro python2011 High Performance Python
 
Pythonic doesn't mean slow!
Pythonic doesn't mean slow!Pythonic doesn't mean slow!
Pythonic doesn't mean slow!
 
Pypy is-it-ready-for-production-the-sequel
Pypy is-it-ready-for-production-the-sequelPypy is-it-ready-for-production-the-sequel
Pypy is-it-ready-for-production-the-sequel
 
Running a Plone product on Substance D
Running a Plone product on Substance DRunning a Plone product on Substance D
Running a Plone product on Substance D
 
PyPy London Demo Evening 2013
PyPy London Demo Evening 2013PyPy London Demo Evening 2013
PyPy London Demo Evening 2013
 
Writing Fast Code (JP) - PyCon JP 2015
Writing Fast Code (JP) - PyCon JP 2015Writing Fast Code (JP) - PyCon JP 2015
Writing Fast Code (JP) - PyCon JP 2015
 
Writing Fast Code - PyCon HK 2015
Writing Fast Code - PyCon HK 2015Writing Fast Code - PyCon HK 2015
Writing Fast Code - PyCon HK 2015
 
First Steps in Python Programming
First Steps in Python ProgrammingFirst Steps in Python Programming
First Steps in Python Programming
 
PyPy - is it ready for production
PyPy - is it ready for productionPyPy - is it ready for production
PyPy - is it ready for production
 
Advanced topics with python
Advanced topics with pythonAdvanced topics with python
Advanced topics with python
 
Advanced topics with python
Advanced topics with pythonAdvanced topics with python
Advanced topics with python
 
Infrastructure-as-Code with Pulumi - Better than all the others (like Ansible)?
Infrastructure-as-Code with Pulumi- Better than all the others (like Ansible)?Infrastructure-as-Code with Pulumi- Better than all the others (like Ansible)?
Infrastructure-as-Code with Pulumi - Better than all the others (like Ansible)?
 
Rusty Python
Rusty PythonRusty Python
Rusty Python
 
Odoo profiler
Odoo profilerOdoo profiler
Odoo profiler
 
Cfgmgmt Challenges aren't technical anymore
Cfgmgmt Challenges aren't technical anymoreCfgmgmt Challenges aren't technical anymore
Cfgmgmt Challenges aren't technical anymore
 
Take a Stroll in the Bazaar
Take a Stroll in the BazaarTake a Stroll in the Bazaar
Take a Stroll in the Bazaar
 
Pythonpresent
PythonpresentPythonpresent
Pythonpresent
 
Multiprocessing with python
Multiprocessing with pythonMultiprocessing with python
Multiprocessing with python
 
Introduction to Python
Introduction to PythonIntroduction to Python
Introduction to Python
 

More from Tim (文昌)

Python的50道陰影
Python的50道陰影Python的50道陰影
Python的50道陰影
Tim (文昌)
 
Frontend django, Django Web 前端探索
Frontend django, Django Web 前端探索Frontend django, Django Web 前端探索
Frontend django, Django Web 前端探索
Tim (文昌)
 
Profile django
Profile djangoProfile django
Profile django
Tim (文昌)
 
Introduction to protocol buffer
Introduction to protocol bufferIntroduction to protocol buffer
Introduction to protocol buffer
Tim (文昌)
 
I18n
I18nI18n
Mock Introduction
Mock IntroductionMock Introduction
Mock Introduction
Tim (文昌)
 
Tainan.py, Experience about package
Tainan.py, Experience about packageTainan.py, Experience about package
Tainan.py, Experience about package
Tim (文昌)
 
pygame sharing pyhug
pygame sharing pyhugpygame sharing pyhug
pygame sharing pyhug
Tim (文昌)
 

More from Tim (文昌) (9)

Python的50道陰影
Python的50道陰影Python的50道陰影
Python的50道陰影
 
Frontend django, Django Web 前端探索
Frontend django, Django Web 前端探索Frontend django, Django Web 前端探索
Frontend django, Django Web 前端探索
 
Profile django
Profile djangoProfile django
Profile django
 
Introduction to protocol buffer
Introduction to protocol bufferIntroduction to protocol buffer
Introduction to protocol buffer
 
I18n
I18nI18n
I18n
 
Mock Introduction
Mock IntroductionMock Introduction
Mock Introduction
 
Ml weka
Ml wekaMl weka
Ml weka
 
Tainan.py, Experience about package
Tainan.py, Experience about packageTainan.py, Experience about package
Tainan.py, Experience about package
 
pygame sharing pyhug
pygame sharing pyhugpygame sharing pyhug
pygame sharing pyhug
 

Performance Enhancement Tips