@nzupan
Turbo-boosting
your Python development
Problem
Problem
Pull Request
Problem
Pull Request
Problem
Pull Request
Travis build
Problem
Pull Request
Travis build
Automatic Deployment
Problem
Pull Request
Travis build
Automatic Deployment
Sentry
Problem
Pull Request
Travis build
Automatic Deployment
Sentry
QA
Problem
Pull Request
Travis build
Automatic Deployment
Sentry
QA
Datadog
Papertrail
CODE REPOSITORY
Any DVCS is OK, but I prefer git
Main reason: GitHub’s UI
CONTINUOUS INTEGRATION
Run your tests often
Preferably on every commit
In a clean environment
Easy and fun CI for your Python packages
Travis CI
Hosted Continuous Integration
for the Open Source Community
+
BUILDS
SINGLE BUILD
SINGLE BUILD (CONT.)
Free for any public GitHub repo
Private repos supported too!
Already ran ~8 mio tests for ~78k
open-source projects
Supports:
C, C++, Clojure, Erlang, Go, Groovy,
Haskell, Java, JavaScript, Perl, PHP,
Python, Ruby, Scala
Preinstalled DBs:
MySQL, PostgreSQL, MongoDB,
CouchDB, Redis, Riak, RabbitMQ,
Memcached, Cassandra, Neo4J,
ElasticSearch, ...
GETTING STARTED
Sign in to travis-ci.org with GitHub account
Enable repository
Add .travis.yml to your repository
Push it
MINIMAL CONFIG
MORE CONTROL
PREINSTALLED SERVICES
HEADLESS TESTING
BRANCHES
BUILD NOTIFICATIONS
+ many more
STATUS IMAGES
STATUS IMAGES
https://travis-ci.org/[YOUR_GITHUB_USERNAME]/
[YOUR_PROJECT_NAME].png
PULL REQUESTS TESTING
HEROKU: CLOUD
APPLICATION PLATFORM
OUT-OF-THE-BOX
Add to .travis.yml:
AUTO-DEPLOYING PLONE
SENTRY: CLOUD ERROR
REPORTING
ERROR AGGREGATION
ERROR CONTEXT
SENTRY SETUP
Datadog: cloud monitoring
Datadog: cloud monitoring
MUNIN + ...
... + OPS TWITTER
OUT-OF-THE-BOX
EVENTS:
Fabric, Chef, Puppet
RSS
GitHub, Pivotal Tracker,
Redmine, etc.
Pingdom
Jenkins
HTTP API
METRICS
Ng...
CUSTOM METRICS
$ easy_install dogstatsd-python
from statsd import statsd
stasd.gauge(‘users.active’, 50)
statsd.counter(‘b...
CUSTOM APP METRICS
PAPERTRAIL: CLOUD LOG
AGGREGATION
WHO IS DOWNLOADING?
BONUS SLIDE:
CODE COVERAGE REPORTS
Problem
Pull Request
Travis build
Automatic Deployment
Sentry
QA
Datadog
Papertrail
bobtemplates.niteoweb
THE PLONE TEMPLATE
zc.buildout best practices
GenericSetup install profile
Zope 3 browser layer
z3c.jbot overrides folder
s...
THE PYRAMID TEMPLATE
zc.buildout best practices
SQLite for development, PostgreSQL for production
A sample view with a Cha...
Try it out:
https://github.com/niteoweb/bobtemplates.niteoweb/
Then, let’s generalize it!
Thanks!
Some images taken from sxc.hu
http://travis-ci.org/
http://coveralls.io/
https://papertrailapp.com/
http://www.dat...
Turbo boosting your python development
Turbo boosting your python development
Turbo boosting your python development
Turbo boosting your python development
Turbo boosting your python development
Turbo boosting your python development
Turbo boosting your python development
Turbo boosting your python development
Turbo boosting your python development
Turbo boosting your python development
Turbo boosting your python development
Turbo boosting your python development
Turbo boosting your python development
Turbo boosting your python development
Upcoming SlideShare
Loading in …5
×

Turbo boosting your python development

1,330 views
1,214 views

Published on

A talk describing how I use services like Travis CI, Coveralls, Papertrail, DataDog and others to make my life easier and more productive on Plone and Pyramid projects.
Personally, I regard time spent fixing preventable problems and doing routine tasks is time wasted stupidly. I'll show you how you can leverage some of the recently available "cloud" services to cut the amount of routine tasks in your day-to-day work.

This talk is somewhat based on the Travis CI talk I had last year on PloneConf in Arnhem, but extended to include other cloud services.

Published in: Technology
0 Comments
7 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
1,330
On SlideShare
0
From Embeds
0
Number of Embeds
11
Actions
Shares
0
Downloads
12
Comments
0
Likes
7
Embeds 0
No embeds

No notes for slide

Turbo boosting your python development

  1. 1. @nzupan Turbo-boosting your Python development
  2. 2. Problem
  3. 3. Problem Pull Request
  4. 4. Problem Pull Request
  5. 5. Problem Pull Request Travis build
  6. 6. Problem Pull Request Travis build Automatic Deployment
  7. 7. Problem Pull Request Travis build Automatic Deployment Sentry
  8. 8. Problem Pull Request Travis build Automatic Deployment Sentry QA
  9. 9. Problem Pull Request Travis build Automatic Deployment Sentry QA Datadog Papertrail
  10. 10. CODE REPOSITORY Any DVCS is OK, but I prefer git Main reason: GitHub’s UI
  11. 11. CONTINUOUS INTEGRATION Run your tests often Preferably on every commit In a clean environment
  12. 12. Easy and fun CI for your Python packages Travis CI
  13. 13. Hosted Continuous Integration for the Open Source Community
  14. 14. +
  15. 15. BUILDS
  16. 16. SINGLE BUILD
  17. 17. SINGLE BUILD (CONT.)
  18. 18. Free for any public GitHub repo
  19. 19. Private repos supported too!
  20. 20. Already ran ~8 mio tests for ~78k open-source projects
  21. 21. Supports: C, C++, Clojure, Erlang, Go, Groovy, Haskell, Java, JavaScript, Perl, PHP, Python, Ruby, Scala
  22. 22. Preinstalled DBs: MySQL, PostgreSQL, MongoDB, CouchDB, Redis, Riak, RabbitMQ, Memcached, Cassandra, Neo4J, ElasticSearch, Kestrel, SQLite3
  23. 23. GETTING STARTED Sign in to travis-ci.org with GitHub account Enable repository Add .travis.yml to your repository Push it
  24. 24. MINIMAL CONFIG
  25. 25. MORE CONTROL
  26. 26. PREINSTALLED SERVICES
  27. 27. HEADLESS TESTING
  28. 28. BRANCHES
  29. 29. BUILD NOTIFICATIONS + many more
  30. 30. STATUS IMAGES
  31. 31. STATUS IMAGES https://travis-ci.org/[YOUR_GITHUB_USERNAME]/ [YOUR_PROJECT_NAME].png
  32. 32. PULL REQUESTS TESTING
  33. 33. HEROKU: CLOUD APPLICATION PLATFORM
  34. 34. OUT-OF-THE-BOX
  35. 35. Add to .travis.yml:
  36. 36. AUTO-DEPLOYING PLONE
  37. 37. SENTRY: CLOUD ERROR REPORTING
  38. 38. ERROR AGGREGATION
  39. 39. ERROR CONTEXT
  40. 40. SENTRY SETUP
  41. 41. Datadog: cloud monitoring
  42. 42. Datadog: cloud monitoring
  43. 43. MUNIN + ...
  44. 44. ... + OPS TWITTER
  45. 45. OUT-OF-THE-BOX EVENTS: Fabric, Chef, Puppet RSS GitHub, Pivotal Tracker, Redmine, etc. Pingdom Jenkins HTTP API METRICS Nginx & Apache Amazon AWS & HP Cloud Cacti & Nagios PostgreSQL, MySQL, Cassandra, MongoDB, CouchDB, etc. Haproxy,Varnish, Memcached, RabbitMQ, Solr & ElasticSearch, etc.
  46. 46. CUSTOM METRICS $ easy_install dogstatsd-python from statsd import statsd stasd.gauge(‘users.active’, 50) statsd.counter(‘base_edit.rendered’) statsd.histogram: (‘catalog.query.time’, duration) statsd.set: (‘users.unique’, user_id)
  47. 47. CUSTOM APP METRICS
  48. 48. PAPERTRAIL: CLOUD LOG AGGREGATION
  49. 49. WHO IS DOWNLOADING?
  50. 50. BONUS SLIDE: CODE COVERAGE REPORTS
  51. 51. Problem Pull Request Travis build Automatic Deployment Sentry QA Datadog Papertrail
  52. 52. bobtemplates.niteoweb
  53. 53. THE PLONE TEMPLATE zc.buildout best practices GenericSetup install profile Zope 3 browser layer z3c.jbot overrides folder static/ resourceDirectory for serving static resources (images, CSS, JS, etc.) Sphinx documentation test suite with a solid test coverage Travis CI integration
  54. 54. THE PYRAMID TEMPLATE zc.buildout best practices SQLite for development, PostgreSQL for production A sample view with a Chameleon template A sample DB model static resources filled with Twitter Bootstrap niceties Sphinx documentation Exemplary test suite Travis CI & Coveralls integration Heroku continuous deployment
  55. 55. Try it out: https://github.com/niteoweb/bobtemplates.niteoweb/ Then, let’s generalize it!
  56. 56. Thanks! Some images taken from sxc.hu http://travis-ci.org/ http://coveralls.io/ https://papertrailapp.com/ http://www.datadoghq.com/ http://docs.fabfile.org/ https://pypi.python.org/pypi/githook https://www.heroku.com/ @nzupan

×