Your SlideShare is downloading. ×
How we do python
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×

Introducing the official SlideShare app

Stunning, full-screen experience for iPhone and Android

Text the download link to your phone

Standard text messaging rates apply

How we do python

1,197
views

Published on

Come here and have a short presentation of the tools we use in BestOfMedia science team, in Grenoble, France.

Come here and have a short presentation of the tools we use in BestOfMedia science team, in Grenoble, France.

Published in: Technology

0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
1,197
On Slideshare
0
From Embeds
0
Number of Embeds
4
Actions
Shares
0
Downloads
0
Comments
0
Likes
1
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide

Transcript

  • 1. How We Do Python Python, a way © BestofMedia Grenoble, Science Team
  • 2. ● Python Runtime ● Development environment ○ bom_utils ○ Zero-Application-Server ○ AWS ● Packaging ● CI ● Deployment How does the science team speak python? © BestofMedia Grenoble, Science Team
  • 3. ● VirtualEnv vs GlobalEnv ● GlobalEnv ○ tainted on every install ○ high risk of base installation instability (python is highly used as system scripting) ● VirtualEnv & PYTHONPATH ○ Sandboxed: don't break the box ○ Can be legion Python Runtime © BestofMedia Grenoble, Science Team
  • 4. ● Ubuntu/Debian box with python2.6 ● Targeted sci-env ● Eclipse with pydev Et voilà! Development environment © BestofMedia Grenoble, Science Team
  • 5. ● timothy (easy hadoop using dumbo - to be open-sourced) ● common libs ○ bss (our proprietary documents warehouse) ○ ldap ○ mail ○ ... bom_utils © BestofMedia Grenoble, Science Team
  • 6. ● Boto (aws python lib) encapsulated into bom_utils ○ And now for some names dropping: ■ SNS, SQS, S3, EC2, EMR, CloudWatch AWS © BestofMedia Grenoble, Science Team
  • 7. ● One Application to serve them all ● Automatic logging configuration ● On AWS: ○ Automatic Monitoring & alerting ● to be open-sourced Zero-Application-Server © BestofMedia Grenoble, Science Team
  • 8. ● Once, there was Maven ○ easy to put to Jenkins/Hudson ○ BOM was all Maven world ○ but: ■ Verbose ■ not really suited for python ■ Heavy & sloooooooow ● Then, bdist-deb came to the rescue ○ pythonic ○ light ○ just a line of freestyle Jenkins job ○ to be open-sourced Packaging © BestofMedia Grenoble, Science Team
  • 9. ● One commit job, one release job ● unittest, pylint, coverage CI © BestofMedia Grenoble, Science Team
  • 10. ● One project: ○ one egg ○ one deb (can contain other eggs, like projects shipping their version of internal libs) ● Deb deployed using old internal conf server ○ migration to our new in house system in the pipe Deployment © BestofMedia Grenoble, Science Team
  • 11. Any more questions? © BestofMedia Grenoble, Science Team