Python Developer's
Daily Routine
Maxim Avanov (github.com/avanov)
Project structure
3

Project structure
.cei
/hbt
|-cei
- hbt
|

`-_ii_.y
- _nt_p

`-stpp
- eu.y
4

setup.py
fo stpol ipr stp fn_akgs
rm eutos mot eu, idpcae
stp
eu(
nm=Cei'
ae'hbt,
vrin'..'
eso=001,
pcae=idpcae(,
akgsf...
5

Bootstrapping
$ python setup.py install
$ python setup.py develop
$ python setup.py test
$ python setup.py regtister sd...
Package
management
7

pip commands
$ppisalPrmd
i ntl yai
$ppisalPrmd-ugae
i ntl yai -prd
$ppisalPrmd=.a -ugae
i ntl yai=152 -prd
$ppuisalPrmd...
8

pip commands
$ppfez
i ree
Bbl=.
ae=13
Cei=001
hbt=..
CfeSrp=108
ofecit=..
Jna=271
ij2=..
Mk=090
ao=..
..
.
9

pip-tools
$pprve
i-eiw
rqet=01. aalbe(o hv 01.)
euss=.34 vial yu ae .32

$pprve -at
i-eiw -uo
..<i isalotu>
. pp ntl up...
10

Cookiecutter
• Generates initial project files from defined templates;
• File names are also templates;
• Transparentl...
11

Cookiecutter template
.cei-epae
/hbttmlt
|-coicte.sn
- okeutrjo
`-{coicte.rjc_ae}
- {okeutrpoetnm}
|-{coicte.rjc_ae}
-...
12

cookiecutter.json
{
"rjc_ae:"yrjc"
poetnm" mpoet,
"rjc_eso" ".."
poetvrin: 001
}
13

Cookiecutter in action
$coicte {rjc_epaept}
okeutr poettmlt_ah
poetnm (eal i "yrjc"?mnw
rjc_ae dfut s mpoet) ye
poetvr...
We'd like to
isolate project
environments
15

virtualenv
$vruln ~vn/poetnm}
itaev /ev{rjc_ae
$suc ~vn/poetnm}bnatvt
ore /ev{rjc_ae/i/ciae

What if we need another P...
Still not happy
enough!
Introducing pyenv
18

pyenv - 5 steps to happiness
$gtcoegt/gtu.o/yupevgt~pev
i ln i:/ihbcmyu/yn.i /yn
$eh 'xotPEVRO=$OEpev'> ~.ahc
co epr Y...
19

pyenv - Rise and Shine!
$pevisalyn ntl l
$pevisal276
yn ntl ..
$pevisal333
yn ntl ..
$pevisalpp-..
yn ntl yy221
$pevrh...
20

The Happiness
$ pyenv versions
$ pyenv version
$ pyenv local 2.7.6 3.3.3
$ pyenv global 3.3.3
$ pyenv shell pypy-2.2.1...
21

...
$wihpto
hc yhn
/oegotrtrpevsispto
hm/hswie/yn/hm/yhn

$pto
yhn
Pto 273(7ad1fc Nv2 21,1:81)
yhn .. 8a9e09, o 4 03 8...
virtualenv meets
pyenv
23

pyenv-virtualenv
$gtcoegt/gtu.o/yupev
i ln i:/ihbcmyu/ynvruln.i ~pevpuispevvruln
itaevgt /yn/lgn/yn-itaev

$pevvruln p...
24

pyenv-virtualenv in action
$pevvrulns
yn itaev
{rjc_ae (rae fo /oegotrtrpevvrin/yy221
poetnm} cetd rm hm/hswie/yn/esos...
Well done!
Now, what about
other components?
Virtualization
28

Vagrant
• Written in Ruby
• Relies on VirtualBox
• Runs on Linux / OS X / Win
29

Juju
• Written in Python / Go
• Relies on LXC
• Linux only
30

Docker
• Written in Go
• Relies on LXC and AUFS
• Linux only
31

Vagrant's Success Story...
1. Put a Vagrantfile into the project root
2. $ vagrant up
3. $ vagrant ssh
4. Profit!
32

Vagrantfile
Vgatcniue"" d |ofg
arn.ofgr(2) o cni|
cni.mbx="rjc_ae
ofgv.o
poetnm"
cni.mbxul="tp/flsvgatpcmpeie4bx
ofgv....
33

...Vagrant's Success Story
1. Put a Vagrantfile into the project root
2. $ vagrant up
3. $ vagrant ssh
4. Profit! ..??...
34

Deployment tools
Chef (Ruby)
Puppet (Ruby)
Ansible (Python)
Salt (Python)
35

Ansible Success Story
1. Specify an inventory file
2. Specify a playbook
3. $ ansible-playbook ./playbook.yml -i ./hos...
36

Ansible inventory
[eeo-m
dvlpv]
12183.0
9.6.310
37

Ansible playbook
-hss dvlpv
ot: eeo-m
sd:ys
uo e
tss
ak:
nm:esr atccei u t dt
ae nue p ah s p o ae
ato:atudt_ah=e
cin ...
38

Ansible meets Vagrant
$asbepabo .pabo.m - .hss- nil-lyok /lyokyl i /ot v
uvgat- prmk -piae
arn c aaio -rvtky$OE.arn./n...
39

Ansible - Vagrant Integration
#Vgatie
arnfl
Vgatcniue"" d |ofg
arn.ofgr(2) o cni|

#..
.
cni.mpoiin:nil d |nil|
ofgv.r...
Python & Web
41

Web Server Gateway Interface
Standardized by PEP 333, PEP 3333

dfapiainevrn satrsos)
e plcto(nio, tr_epne:
satrsos(20...
42

WSGI Servers
• CherryPy WSGI Server
• gunicorn
• tornado.wsgi
• uWSGI
• Waitress

http://nichol.as/benchmark-of-python...
Frameworks
44

Heavyweights
• Django
• Twisted
• web2py
• Zope
45

Middleweights
• CherryPy
• Pyramid
• Tornado
• TurboGears
46

Flyweights
• Bottle
• Flask
• Webpy
Database Access
48

ORM
• SQLAlchemy ORM
• Django ORM (Django only)
• Pony ORM
• Peewee
49

Something else
• Native DB-API 2.0 (standardized with PEP 249)
• SQLAlchemy's raw queries
• mosql
50

SQL DDL Versioning & Data Migration
• South (Django only)
• Alembic (SQLAlchemy)
• Pyrseas
Background task
processing
52

Celery
• Out of the box solution for complex cases;
• Multiple brokers - RabbitMQ, Redis, RDBMS, SQS, MongoDB etc;
• T...
53

RQ (Redis Queue)
• Designed to have a low barrier to enter;
• Uses Redis' Pub/Sub feature.
Process managers
55

Process managers
• Supervisor
• Circus.io
Sentry
60

Testing tools
• unittest
• py.test
• nose
• WebTest
• coverage
• tox
Thank you!
Python Developer's Daily Routine
Python Developer's Daily Routine
Python Developer's Daily Routine
Upcoming SlideShare
Loading in …5
×

Python Developer's Daily Routine

1,266 views

Published on

Published in: Technology

Python Developer's Daily Routine

  1. 1. Python Developer's Daily Routine Maxim Avanov (github.com/avanov)
  2. 2. Project structure
  3. 3. 3 Project structure .cei /hbt |-cei - hbt | `-_ii_.y - _nt_p `-stpp - eu.y
  4. 4. 4 setup.py fo stpol ipr stp fn_akgs rm eutos mot eu, idpcae stp eu( nm=Cei' ae'hbt, vrin'..' eso=001, pcae=idpcae(, akgsfn_akgs) ts_ut=tss, etsie'et' tssrqie[pts' 'oeae] et_eur='yet, cvrg', isalrqie='yai=152] ntl_eurs[Prmd=.a' )
  5. 5. 5 Bootstrapping $ python setup.py install $ python setup.py develop $ python setup.py test $ python setup.py regtister sdist [bdist_egg] upload
  6. 6. Package management
  7. 7. 7 pip commands $ppisalPrmd i ntl yai $ppisalPrmd-ugae i ntl yai -prd $ppisalPrmd=.a -ugae i ntl yai=152 -prd $ppuisalPrmd i nntl yai
  8. 8. 8 pip commands $ppfez i ree Bbl=. ae=13 Cei=001 hbt=.. CfeSrp=108 ofecit=.. Jna=271 ij2=.. Mk=090 ao=.. .. .
  9. 9. 9 pip-tools $pprve i-eiw rqet=01. aalbe(o hv 01.) euss=.34 vial yu ae .32 $pprve -at i-eiw -uo ..<i isalotu> . pp ntl upt $pprve -itrcie i-eiw -neatv rqet=01. aalbe(o hv 01.) euss=.40 vial yu ae .32 Ugaenw []s [],[]l []i y prd o? Ye, No Al, Qut .. .
  10. 10. 10 Cookiecutter • Generates initial project files from defined templates; • File names are also templates; • Transparently works with remote VCS.
  11. 11. 11 Cookiecutter template .cei-epae /hbttmlt |-coicte.sn - okeutrjo `-{coicte.rjc_ae} - {okeutrpoetnm} |-{coicte.rjc_ae} - {okeutrpoetnm} | `-_ii_.y - _nt_p `-stpp - eu.y
  12. 12. 12 cookiecutter.json { "rjc_ae:"yrjc" poetnm" mpoet, "rjc_eso" ".." poetvrin: 001 }
  13. 13. 13 Cookiecutter in action $coicte {rjc_epaept} okeutr poettmlt_ah poetnm (eal i "yrjc"?mnw rjc_ae dfut s mpoet) ye poetvrin(eal i ".."?011 rjc_eso dfut s 001) .. $te .mnw re /ye .mnw /ye |-mnw - ye | `-_ii_.y - _nt_p `-stpp - eu.y
  14. 14. We'd like to isolate project environments
  15. 15. 15 virtualenv $vruln ~vn/poetnm} itaev /ev{rjc_ae $suc ~vn/poetnm}bnatvt ore /ev{rjc_ae/i/ciae What if we need another Python version? $vruln ~vn/poetnm}- pto3 itaev /ev{rjc_ae p yhn
  16. 16. Still not happy enough!
  17. 17. Introducing pyenv
  18. 18. 18 pyenv - 5 steps to happiness $gtcoegt/gtu.o/yupevgt~pev i ln i:/ihbcmyu/yn.i /yn $eh 'xotPEVRO=$OEpev'> ~.ahc co epr YN_OT"HM/yn" > /bsr $eh 'xotPT=$YN_OTbn$AH'> ~.ahc co epr AH"PEVRO/i:PT" > /bsr $eh 'vl"(yn ii -" > ~.ahc co ea $pev nt )' > /bsr $ee $HL xc SEL
  19. 19. 19 pyenv - Rise and Shine! $pevisalyn ntl l $pevisal276 yn ntl .. $pevisal333 yn ntl .. $pevisalpp-.. yn ntl yy221 $pevrhs yn eah
  20. 20. 20 The Happiness $ pyenv versions $ pyenv version $ pyenv local 2.7.6 3.3.3 $ pyenv global 3.3.3 $ pyenv shell pypy-2.2.1 $ pyenv whence 2to3
  21. 21. 21 ... $wihpto hc yhn /oegotrtrpevsispto hm/hswie/yn/hm/yhn $pto yhn Pto 273(7ad1fc Nv2 21,1:81) yhn .. 8a9e09, o 4 03 84:3 [yy221wt GC463 o lnx PP .. ih C ..] n iu2 >> >>
  22. 22. virtualenv meets pyenv
  23. 23. 23 pyenv-virtualenv $gtcoegt/gtu.o/yupev i ln i:/ihbcmyu/ynvruln.i ~pevpuispevvruln itaevgt /yn/lgn/yn-itaev $pevvruln pp-.. ~vn/poetnm} yn itaev yy221 /ev{rjc_ae
  24. 24. 24 pyenv-virtualenv in action $pevvrulns yn itaev {rjc_ae (rae fo /oegotrtrpevvrin/yy221 poetnm} cetd rm hm/hswie/yn/esospp-..) $suc ~pevvrin/poetnm}bnatvt ore /yn/esos{rjc_ae/i/ciae
  25. 25. Well done!
  26. 26. Now, what about other components?
  27. 27. Virtualization
  28. 28. 28 Vagrant • Written in Ruby • Relies on VirtualBox • Runs on Linux / OS X / Win
  29. 29. 29 Juju • Written in Python / Go • Relies on LXC • Linux only
  30. 30. 30 Docker • Written in Go • Relies on LXC and AUFS • Linux only
  31. 31. 31 Vagrant's Success Story... 1. Put a Vagrantfile into the project root 2. $ vagrant up 3. $ vagrant ssh 4. Profit!
  32. 32. 32 Vagrantfile Vgatcniue"" d |ofg arn.ofgr(2) o cni| cni.mbx="rjc_ae ofgv.o poetnm" cni.mbxul="tp/flsvgatpcmpeie4bx ofgv.o_r ht:/ie.arnu.o/rcs6.o" cni.mntok:rvt_ewr,i:"9.6.310 ofgv.ewr piaentok p 12183.0" ed n
  33. 33. 33 ...Vagrant's Success Story 1. Put a Vagrantfile into the project root 2. $ vagrant up 3. $ vagrant ssh 4. Profit! ..??? 5. Deployment
  34. 34. 34 Deployment tools Chef (Ruby) Puppet (Ruby) Ansible (Python) Salt (Python)
  35. 35. 35 Ansible Success Story 1. Specify an inventory file 2. Specify a playbook 3. $ ansible-playbook ./playbook.yml -i ./hosts
  36. 36. 36 Ansible inventory [eeo-m dvlpv] 12183.0 9.6.310
  37. 37. 37 Ansible playbook -hss dvlpv ot: eeo-m sd:ys uo e tss ak: nm:esr atccei u t dt ae nue p ah s p o ae ato:atudt_ah=e cin p paecceys http://www.ansibleworks.com/docs/modules.html
  38. 38. 38 Ansible meets Vagrant $asbepabo .pabo.m - .hss- nil-lyok /lyokyl i /ot v uvgat- prmk -piae arn c aaio -rvtky$OE.arn./neuepiaeky e=HM/vgatdiscr_rvt_e or $vgatpoiin arn rvso
  39. 39. 39 Ansible - Vagrant Integration #Vgatie arnfl Vgatcniue"" d |ofg arn.ofgr(2) o cni| #.. . cni.mpoiin:nil d |nil| ofgv.rvso asbe o asbe asbepabo ="/lyokyl nil.lyok .pabo.m" asbeivnoypt ="/ot" nil.netr_ah .hss asbevroe="" nil.ebs v ed n ed n
  40. 40. Python & Web
  41. 41. 41 Web Server Gateway Interface Standardized by PEP 333, PEP 3333 dfapiainevrn satrsos) e plcto(nio, tr_epne: satrsos(20O' ['otn-ye,'etpan)) tr_epne'0 K, (CnetTp' tx/li'] yed'el Wrdn il Hlo ol'
  42. 42. 42 WSGI Servers • CherryPy WSGI Server • gunicorn • tornado.wsgi • uWSGI • Waitress http://nichol.as/benchmark-of-python-web-servers
  43. 43. Frameworks
  44. 44. 44 Heavyweights • Django • Twisted • web2py • Zope
  45. 45. 45 Middleweights • CherryPy • Pyramid • Tornado • TurboGears
  46. 46. 46 Flyweights • Bottle • Flask • Webpy
  47. 47. Database Access
  48. 48. 48 ORM • SQLAlchemy ORM • Django ORM (Django only) • Pony ORM • Peewee
  49. 49. 49 Something else • Native DB-API 2.0 (standardized with PEP 249) • SQLAlchemy's raw queries • mosql
  50. 50. 50 SQL DDL Versioning & Data Migration • South (Django only) • Alembic (SQLAlchemy) • Pyrseas
  51. 51. Background task processing
  52. 52. 52 Celery • Out of the box solution for complex cases; • Multiple brokers - RabbitMQ, Redis, RDBMS, SQS, MongoDB etc; • Tasks scheduler; • Results backends.
  53. 53. 53 RQ (Redis Queue) • Designed to have a low barrier to enter; • Uses Redis' Pub/Sub feature.
  54. 54. Process managers
  55. 55. 55 Process managers • Supervisor • Circus.io
  56. 56. Sentry
  57. 57. 60 Testing tools • unittest • py.test • nose • WebTest • coverage • tox
  58. 58. Thank you!

×