PerformanceEnhancementTipsDon’t Write too muchWen Chang Hsu
石頭湯
主菜們站出來
Who am I Wen   Chang Hsu ( you can find me via  Tim) Dorm7 Software, 好客民宿 Slide Note 我想讓它上線…..
 Repository   find timtan in GitHub      https://github.com/timtan/python-     performance-tips.git    pip install -r r...
Profile Now   see profile_sample1.py
profile_sample1
Who is slow? python   -m cProfile -s cumulative profile_sample1.py    -m cProfile means directly invoke the module    -...
The nature of cProfile Deterministic  profiling, Python interpreter have hook on each  function call.
Summary Profilingfirst, don’t guess Use the command python -m cProfile -s cumulative  profile_sample1.py
If you don’t have time Reduce    complex is better You   can use pypy !!
But !!C   Extension is not available However, original standard library written  in C are replaced with pure python
How Quora think http://www.quora.com/PyPy/Will-PyPy-  be-the-standard-Python-implementation Page loading time boost 2x. ...
You can use Cython Compile python module to C code Compile the c Code to python module You change no code, 20% boost
Key Point to use Cython
In the example typemake in your shell How to write setup.py is a little tricky if you  want to use cython and setuptools...
Summary PyPy   is good and near production    C Extension is in experiment Cython is more realistic, and you can integr...
Parallel ? 大量的資料做同樣的事 Gevent,   multiprocessing, Thread Celery ( I won’t cover this today )
Reddit says Thread  in python sucks Multiprocessing is good
computation_parallel_example.py
7 second
computation_parallel_example_threading.py
7 seconds again
Why Python     Has a GIL The max function call is not preemptable,  it is written in C The interrupter cannot yield for...
Multiprocessing
4s, faster than multithreading
Summary Multiprocessing It did fork process. (consumes memory) It can utilize all your core
Trick For Multiprocessing pool = multiprocessing.Pool(10) pool.map( function, data) Than     you get 10 workers that wi...
e.g.
Error !!!
Summary of Multiprocessing It   did fork !!      Previous data is duplicated, you should The only way to communicate da...
PyCon TW needs U不要再叫 Tim 了
Performance Enhancement Tips
Performance Enhancement Tips
Upcoming SlideShare
Loading in...5
×

Performance Enhancement Tips

3,303

Published on

Some Small tip fro performance enhancement

1 Comment
2 Likes
Statistics
Notes
No Downloads
Views
Total Views
3,303
On Slideshare
0
From Embeds
0
Number of Embeds
5
Actions
Shares
0
Downloads
61
Comments
1
Likes
2
Embeds 0
No embeds

No notes for slide

Performance Enhancement Tips

  1. 1. PerformanceEnhancementTipsDon’t Write too muchWen Chang Hsu
  2. 2. 石頭湯
  3. 3. 主菜們站出來
  4. 4. Who am I Wen Chang Hsu ( you can find me via Tim) Dorm7 Software, 好客民宿 Slide Note 我想讓它上線…..
  5. 5.  Repository find timtan in GitHub  https://github.com/timtan/python- performance-tips.git  pip install -r requirement.txt
  6. 6. Profile Now see profile_sample1.py
  7. 7. profile_sample1
  8. 8. Who is slow? python -m cProfile -s cumulative profile_sample1.py  -m cProfile means directly invoke the module  -s is the sort order.
  9. 9. The nature of cProfile Deterministic profiling, Python interpreter have hook on each function call.
  10. 10. Summary Profilingfirst, don’t guess Use the command python -m cProfile -s cumulative profile_sample1.py
  11. 11. If you don’t have time Reduce complex is better You can use pypy !!
  12. 12. But !!C Extension is not available However, original standard library written in C are replaced with pure python
  13. 13. How Quora think http://www.quora.com/PyPy/Will-PyPy- be-the-standard-Python-implementation Page loading time boost 2x. But lxml, pyml cannot runs in PyPy. Communications between Cpython and PyPY
  14. 14. You can use Cython Compile python module to C code Compile the c Code to python module You change no code, 20% boost
  15. 15. Key Point to use Cython
  16. 16. In the example typemake in your shell How to write setup.py is a little tricky if you want to use cython and setuptools at the same time.
  17. 17. Summary PyPy is good and near production  C Extension is in experiment Cython is more realistic, and you can integrate it with existing C module easily
  18. 18. Parallel ? 大量的資料做同樣的事 Gevent, multiprocessing, Thread Celery ( I won’t cover this today )
  19. 19. Reddit says Thread in python sucks Multiprocessing is good
  20. 20. computation_parallel_example.py
  21. 21. 7 second
  22. 22. computation_parallel_example_threading.py
  23. 23. 7 seconds again
  24. 24. Why Python Has a GIL The max function call is not preemptable, it is written in C The interrupter cannot yield form the function call
  25. 25. Multiprocessing
  26. 26. 4s, faster than multithreading
  27. 27. Summary Multiprocessing It did fork process. (consumes memory) It can utilize all your core
  28. 28. Trick For Multiprocessing pool = multiprocessing.Pool(10) pool.map( function, data) Than you get 10 workers that will help you process data
  29. 29. e.g.
  30. 30. Error !!!
  31. 31. Summary of Multiprocessing It did fork !!  Previous data is duplicated, you should The only way to communicate data between process. It use IPC The argument, return value function should be pickable
  32. 32. PyCon TW needs U不要再叫 Tim 了
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×