Buildout talk

751 views

Published on

Tach T

Published in: Technology
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
751
On SlideShare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
Downloads
4
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

Buildout talk

  1. 1. Buildout @Formspring @martincozziMonday, September 19, 2011
  2. 2. Formspring helps people find out more about each other by sharing interesting & personal responsesMonday, September 19, 2011
  3. 3. 26M usersMonday, September 19, 2011
  4. 4. 3.5B responsesMonday, September 19, 2011
  5. 5. Our StackMonday, September 19, 2011
  6. 6. ApplicationMonday, September 19, 2011
  7. 7. Memcache ApplicationMonday, September 19, 2011
  8. 8. Memcache Redis ApplicationMonday, September 19, 2011
  9. 9. Memcache Redis Application MySQLMonday, September 19, 2011
  10. 10. Memcache Redis Application MySQL CassandraMonday, September 19, 2011
  11. 11. Memcache Redis SQS Application MySQL CassandraMonday, September 19, 2011
  12. 12. Memcache Redis SQS Application MySQL Kestrel CassandraMonday, September 19, 2011
  13. 13. PHP PHP PHP PHP QWPHP QW QW QW QW Memcache Redis SQS Application MySQL Kestrel CassandraMonday, September 19, 2011
  14. 14. PHP PHP PHP PHP QWPHP QW QW QW QW Memcache Redis SQS Application MySQL Kestrel Cassandra Python Python Python Python Python QW QW QW QW QWMonday, September 19, 2011
  15. 15. Memcache Redis Python Python Python Python Python QW QW QW QW Kestrel MySQL QW CassandraMonday, September 19, 2011
  16. 16. Python Memcache QW Redis Kestrel MySQL Python Cassandra QWMonday, September 19, 2011
  17. 17. Python Memcache QW Redis Kestrel MySQL Python Cassandra QWMonday, September 19, 2011
  18. 18. Python Memcache QW Redis Kestrel MySQL Python Cassandra QWMonday, September 19, 2011
  19. 19. Monday, September 19, 2011
  20. 20. Monday, September 19, 2011
  21. 21. Monday, September 19, 2011
  22. 22. Monday, September 19, 2011
  23. 23. Monday, September 19, 2011
  24. 24. Monday, September 19, 2011
  25. 25. Workers.py class StreamPush: ... class SmileOnAnswers: ... class SmileConnection: ... class Inbox: ...Monday, September 19, 2011
  26. 26. Workers.py • ./python2.6 kestrel_worker : • Thread Count • Queue Name • Class NameMonday, September 19, 2011
  27. 27. Deploy Queue-Stream code.tar.gz Queue-Smiles Queue-InboxMonday, September 19, 2011
  28. 28. Problems • Big archiveMonday, September 19, 2011
  29. 29. Problems • Big archive • DependenciesMonday, September 19, 2011
  30. 30. Problems • Big archive • Dependencies • Package AMIMonday, September 19, 2011
  31. 31. Problems • Big archive • Dependencies • Package AMI • Difficult to maintain / collaborateMonday, September 19, 2011
  32. 32. What needs to be done • Organize CodebaseMonday, September 19, 2011
  33. 33. What needs to be done • Organize Codebase • Split in folders / packagesMonday, September 19, 2011
  34. 34. What needs to be done • Organize Codebase • Split in folders / packages • Manage dependencies automaticallyMonday, September 19, 2011
  35. 35. BuildoutMonday, September 19, 2011
  36. 36. Buildout • No installation neededMonday, September 19, 2011
  37. 37. Buildout • No installation needed • 3 easy stepsMonday, September 19, 2011
  38. 38. Buildout • Download bootstrap.pyMonday, September 19, 2011
  39. 39. Buildout • Download bootstrap.py • Run boostrap.pyMonday, September 19, 2011
  40. 40. Buildout • Download bootstrap.py • Run boostrap.py • Run bin/buildoutMonday, September 19, 2011
  41. 41. Buildout Recipe • Plugin mechanismMonday, September 19, 2011
  42. 42. Buildout Recipe • Plugin mechanism • Main RecipesMonday, September 19, 2011
  43. 43. Buildout Recipe • Plugin mechanism • Main Recipes • zc.recipe.eggMonday, September 19, 2011
  44. 44. Buildout Recipe • Plugin mechanism • Main Recipes • zc.recipe.egg • zc.recipe.cmmiMonday, September 19, 2011
  45. 45. Buildout Recipe • Plugin mechanism • Main Recipes • zc.recipe.egg • zc.recipe.cmmi • djangorecipeMonday, September 19, 2011
  46. 46. Organize Codebase |-- fisher | |-- bootstrap.py | |-- buildout.cfg | |-- setup.py | `-- src | `-- fisher | `-- __init__.py |-- flake | |-- bootstrap.py | |-- buildout.cfg | |-- setup.py | `-- src | `-- flake | |-- README | |-- __init__.py | |-- flake.py | `-- static | |-- favicon.ico | `-- robots.txt |-- kestrel | |-- bootstrap.py | |-- buildout.cfg | |-- setup.py | `-- src | `-- kestrelMonday, September 19, 2011
  47. 47. Identify DependenciesMonday, September 19, 2011
  48. 48. Identify DependenciesMonday, September 19, 2011
  49. 49. Identify DependenciesMonday, September 19, 2011
  50. 50. Identify DependenciesMonday, September 19, 2011
  51. 51. Identify DependenciesMonday, September 19, 2011
  52. 52. Identify DependenciesMonday, September 19, 2011
  53. 53. List DependenciesMonday, September 19, 2011
  54. 54. List DependenciesMonday, September 19, 2011
  55. 55. List DependenciesMonday, September 19, 2011
  56. 56. Run BuildoutMonday, September 19, 2011
  57. 57. Run BuildoutMonday, September 19, 2011
  58. 58. Run BuildoutMonday, September 19, 2011
  59. 59. Run BuildoutMonday, September 19, 2011
  60. 60. Run BuildoutMonday, September 19, 2011
  61. 61. Run BuildoutMonday, September 19, 2011
  62. 62. Run BuildoutMonday, September 19, 2011
  63. 63. Run BuildoutMonday, September 19, 2011
  64. 64. Run BuildoutMonday, September 19, 2011
  65. 65. Run BuildoutMonday, September 19, 2011
  66. 66. Run BuildoutMonday, September 19, 2011
  67. 67. Advantages • Flexibility with dependencies.Monday, September 19, 2011
  68. 68. Advantages • Flexibility with dependencies. • Build your own eggs.Monday, September 19, 2011
  69. 69. Advantages • Flexibility with dependencies. • Build your own eggs. • Test things !Monday, September 19, 2011
  70. 70. Eggs?Monday, September 19, 2011
  71. 71. Deploy Redis Queue-Stream Kestrel stream.tar.gz Queue-Smiles inbox.tar.gz smiles.tar.gz Queue-InboxMonday, September 19, 2011
  72. 72. Deploy Queue-Stream Kestrel Memcache PyYAML Queue-Smiles stream.tar.gz inbox.tar.gz smiles.tar.gz Queue-InboxMonday, September 19, 2011
  73. 73. Deploy Queue-Stream stream.tar.gz Fisher Queue-Smiles inbox.tar.gz smiles.tar.gz Kestrel Queue-InboxMonday, September 19, 2011
  74. 74. Advantages • Targeted deploys.Monday, September 19, 2011
  75. 75. Advantages • Targeted deploys. • Less failures.Monday, September 19, 2011
  76. 76. Advantages • Targeted deploys. • Less failures. • Less work for Ops.Monday, September 19, 2011
  77. 77. Advantages • Targeted deploys. • Less failures. • Less work for Ops. • Manage your versions.Monday, September 19, 2011
  78. 78. PyPi.python.org goes downMonday, September 19, 2011
  79. 79. Downtime ? • Recipe to proxy PyPi.Monday, September 19, 2011
  80. 80. Downtime ? • Recipe to proxy PyPi. • Cronjob.Monday, September 19, 2011
  81. 81. Downtime ? • Recipe to proxy PyPi. • Cronjob. • Local PyPi.Monday, September 19, 2011
  82. 82. So .. Buildout • Manage dependenciesMonday, September 19, 2011
  83. 83. So .. Buildout • Manage dependencies • Test libraries on the flyMonday, September 19, 2011
  84. 84. So .. Buildout • Manage dependencies • Test libraries on the fly • Easy to scale / deployMonday, September 19, 2011
  85. 85. So .. Buildout • Manage dependencies • Test libraries on the fly • Easy to scale / deploy • Highly Recommended =)Monday, September 19, 2011
  86. 86. Questions ? • Ask me on Formspring • formspring.me/martincozzi • formspring.me/formspringEngMonday, September 19, 2011
  87. 87. We are hiring • Ops • Front-end • Back-end martin@formspring.me rob@formspring.meMonday, September 19, 2011

×