Move out from AppEngineand Python PaaS
小海 StreetVoice / CTOtwitter.com / tzangmsgithub.com / tzangms
StreetVoice  recruiting
aboutGoogle App Engine
Prosyou don’t worry about the server           enviroment
Consyou worry about money
MyAudioCastrunning on app engine over One year
Just a simple website that host Podcasts
But burns money
What it use•   Google AppEngine•   Django 1.1•   S3•   CDN - CDNetworks    (sponsored by pixnet)
Current stats•   600+ podcasts•   900+ episodes•   220GB storage usage / month•   60TB bandwidth usage / month•   6000,000...
Audience                                         80%•   China: 80%•   Taiwan: 3%•   Other: 17%                16%         ...
Page views
AppEngine is GFWed
Why I moved out from             AppEngine•   Pricing•   Performance•   Deployment•   Reinvent the wheel
Pricing
Expensive
$4 a day?$120/month, you can rent Linode 1024 for 3
One Linode 1024     for now
One Linode 1024Cost you $39.95      you save $80
Performance
Frankly,not good
High error rate
What problem?•   INSERT logs, 2000+ ops•   Counter (20 shards), 8000+ ops
1000+ errors    per day
For now•   INSERT logs, 35000+ ops•   Counter (no shards), 60000+ ops
0 errors
Slow Deployment
After click deploy button
I have to wait for   60 seconds
and 60 seconds
more 60 seconds
zzz ...
10 minutes past...
Finally, it’s done!
I know it’s better now
Reinvent the wheel
Slow development
`pip` is useless
You can not do lots of things       in easier way
For example
Shard Counter?
Why waste your time?
Just use Redis it’s damn easy and stable
Cacheadd cache / invalidation
Why waste your time?
use `jonny-cache`or `cache-machine`   just add a few lines of code
Its a lot more easier for you to run       on Linode or other PaaS.
How to move outand keep your data.
Dump and Prepare your        data
Generate bulkloader.yamlappcfg.py create_bulkloader_config   --filename=bulkloader.yaml myaudiocast
Dump as csvappcfg.py download_data   --config_file=bulkloader.yaml   --filename=users.csv   --kind=User myaudiocast
write a scriptto read CSV and insert your data into database
The hardest problem     Foreign Key / relation
Foreign Key                     with parent  User     Podcast       Episode           urltzangms       1           1001   ...
What?All the key id of Podcast kind is 1
Some data can’t be dumped
Write it down before you dump your data.
How about Auth?OMG, I don’t have password for user!
Implement Google     OAuth2
The problem
Poor document   I can’t find `scope`       at that time
You need this.                   weird scopehttps://www.googleapis.com/auth/userinfo.profilehttps://www.googleapis.com/aut...
After almost half yearI still can’t find those scope!
Just use `email` to identify user        forget the `id` in DataStore
How about Heroku and  other python PaaS
Python PaaS•   heroku•   dotcloud•   gondor.io•   ep.io•   ...
Advantage                  against AppEngine•   Pricing•   Performance•   Deployment•   Dynamic configuration
Pricing /Performance
AppEngine: $120 / mo                 you can run ...•   heroku: 4 dynos•   gondor.io: 4 wsgi server•   dotcloud: 4 services
But,how about the performance?
StickerAction                             for example•   Running on Heroku•   One dyno (it means $0)•   30 reqs / sec•   N...
New Relic
One Dyno is good    enough      on heroku
Deployment
Faster than AppEngine!        Life is short
You can useany python packages
pip install `package-name`
You dont’ have toreinvent the wheel
pip install -r requirements.txtmysql-python             django-bootstrap-form==0.2django==1.3.1            werkzeugdjango-...
Dynamic configuration
you canswitch config online
heroku config:add SECRET_KEY=‘xxx’
That’s why I left  AppEngine
Live Demo
AppEngine is still good        but not for me
Calling for Proposals
StreetVoice  recruiting
Any Questions?
Upcoming SlideShare
Loading in...5
×

Move out from AppEngine, and Python PaaS alternatives

12,261

Published on

Published in: Technology
0 Comments
13 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
12,261
On Slideshare
0
From Embeds
0
Number of Embeds
4
Actions
Shares
0
Downloads
5
Comments
0
Likes
13
Embeds 0
No embeds

No notes for slide
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • Move out from AppEngine, and Python PaaS alternatives

    1. 1. Move out from AppEngineand Python PaaS
    2. 2. 小海 StreetVoice / CTOtwitter.com / tzangmsgithub.com / tzangms
    3. 3. StreetVoice recruiting
    4. 4. aboutGoogle App Engine
    5. 5. Prosyou don’t worry about the server enviroment
    6. 6. Consyou worry about money
    7. 7. MyAudioCastrunning on app engine over One year
    8. 8. Just a simple website that host Podcasts
    9. 9. But burns money
    10. 10. What it use• Google AppEngine• Django 1.1• S3• CDN - CDNetworks (sponsored by pixnet)
    11. 11. Current stats• 600+ podcasts• 900+ episodes• 220GB storage usage / month• 60TB bandwidth usage / month• 6000,000 plays / month
    12. 12. Audience 80%• China: 80%• Taiwan: 3%• Other: 17% 16% 4% China Taiwan Other
    13. 13. Page views
    14. 14. AppEngine is GFWed
    15. 15. Why I moved out from AppEngine• Pricing• Performance• Deployment• Reinvent the wheel
    16. 16. Pricing
    17. 17. Expensive
    18. 18. $4 a day?$120/month, you can rent Linode 1024 for 3
    19. 19. One Linode 1024 for now
    20. 20. One Linode 1024Cost you $39.95 you save $80
    21. 21. Performance
    22. 22. Frankly,not good
    23. 23. High error rate
    24. 24. What problem?• INSERT logs, 2000+ ops• Counter (20 shards), 8000+ ops
    25. 25. 1000+ errors per day
    26. 26. For now• INSERT logs, 35000+ ops• Counter (no shards), 60000+ ops
    27. 27. 0 errors
    28. 28. Slow Deployment
    29. 29. After click deploy button
    30. 30. I have to wait for 60 seconds
    31. 31. and 60 seconds
    32. 32. more 60 seconds
    33. 33. zzz ...
    34. 34. 10 minutes past...
    35. 35. Finally, it’s done!
    36. 36. I know it’s better now
    37. 37. Reinvent the wheel
    38. 38. Slow development
    39. 39. `pip` is useless
    40. 40. You can not do lots of things in easier way
    41. 41. For example
    42. 42. Shard Counter?
    43. 43. Why waste your time?
    44. 44. Just use Redis it’s damn easy and stable
    45. 45. Cacheadd cache / invalidation
    46. 46. Why waste your time?
    47. 47. use `jonny-cache`or `cache-machine` just add a few lines of code
    48. 48. Its a lot more easier for you to run on Linode or other PaaS.
    49. 49. How to move outand keep your data.
    50. 50. Dump and Prepare your data
    51. 51. Generate bulkloader.yamlappcfg.py create_bulkloader_config --filename=bulkloader.yaml myaudiocast
    52. 52. Dump as csvappcfg.py download_data --config_file=bulkloader.yaml --filename=users.csv --kind=User myaudiocast
    53. 53. write a scriptto read CSV and insert your data into database
    54. 54. The hardest problem Foreign Key / relation
    55. 55. Foreign Key with parent User Podcast Episode urltzangms 1 1001 /tzangms/1001/ lomeer 1 1001 /lomeer/1001/ william 1 1001 /william/1001/
    56. 56. What?All the key id of Podcast kind is 1
    57. 57. Some data can’t be dumped
    58. 58. Write it down before you dump your data.
    59. 59. How about Auth?OMG, I don’t have password for user!
    60. 60. Implement Google OAuth2
    61. 61. The problem
    62. 62. Poor document I can’t find `scope` at that time
    63. 63. You need this. weird scopehttps://www.googleapis.com/auth/userinfo.profilehttps://www.googleapis.com/auth/userinfo.email
    64. 64. After almost half yearI still can’t find those scope!
    65. 65. Just use `email` to identify user forget the `id` in DataStore
    66. 66. How about Heroku and other python PaaS
    67. 67. Python PaaS• heroku• dotcloud• gondor.io• ep.io• ...
    68. 68. Advantage against AppEngine• Pricing• Performance• Deployment• Dynamic configuration
    69. 69. Pricing /Performance
    70. 70. AppEngine: $120 / mo you can run ...• heroku: 4 dynos• gondor.io: 4 wsgi server• dotcloud: 4 services
    71. 71. But,how about the performance?
    72. 72. StickerAction for example• Running on Heroku• One dyno (it means $0)• 30 reqs / sec• No errors• Smooth
    73. 73. New Relic
    74. 74. One Dyno is good enough on heroku
    75. 75. Deployment
    76. 76. Faster than AppEngine! Life is short
    77. 77. You can useany python packages
    78. 78. pip install `package-name`
    79. 79. You dont’ have toreinvent the wheel
    80. 80. pip install -r requirements.txtmysql-python django-bootstrap-form==0.2django==1.3.1 werkzeugdjango-guardian==1.0.3 python-memcacheddjango-taggit==0.9.3 johnny-cachedjango-userena==1.0.2 django-debug-toolbardjango-pagination southdjango-annoying django-nextpage==0.4simples3tweepypillowrequestspython-dateutil<2.0django-extensionsbotodjango-storagesdjango-admin-tools
    81. 81. Dynamic configuration
    82. 82. you canswitch config online
    83. 83. heroku config:add SECRET_KEY=‘xxx’
    84. 84. That’s why I left AppEngine
    85. 85. Live Demo
    86. 86. AppEngine is still good but not for me
    87. 87. Calling for Proposals
    88. 88. StreetVoice recruiting
    89. 89. Any Questions?
    1. A particular slide catching your eye?

      Clipping is a handy way to collect important slides you want to go back to later.

    ×