ARCHITECTING            for the            CLOUDleonidas tsementzisaka @goldstein
# get social           #         awsuggr
# who’s talking               leonidas tsementzis               aka @goldstein*   software architect, engineer    [all maj...
# format* the problem* development* deployment* failure* limitations* conclusion* questions
# the problem* increasing/decreasing resources on the fly using auto scaling* availability* performance* multi server pain...
:development:
# your stack matters* the single most important aspect* cloud-ready open source libraries for major platforms* saves you a...
# memory* avoid application level variables/ sessions* centralized storage: ✔ fast ✔ scalable ✔ efficient                 ...
:storage:
# storage - single server             server
# storage - multi server                  server farm  server 1   server 2     server 3          server 4                 ...
# storage - multi server - S3                  server farm  server 1   server 2     server 3     server 4                 ...
# storage                STORAGE MIDDLEWARE                                                      t   e                    ...
# storageusing a pluggable storage middleware, we can create storageslike...:* local filesystem* network storage* Amazon S...
# storage...and hopefully we don’t have to:
# storage...but if we have to:* avoid using HEAD/GET requests to  check for existing files* store file list in memory inst...
:task queuing:
# task queuinguse message/task queues for long running operations:* image resizes* external api calls* low priority update...
# task queuing* organize tasks into different queues* organize queues into priority workers* scale workers using AWS auto ...
:database:
# database* Amazon RDS does the trick if you’re on MySQL or Oracle* shard early mark down table dependencies from the star...
:deployment:
# huh?* it’s your code* you know the dependencies* you know it’s breaking points* it’s your job to deal with deployment fa...
# requirements* 50+ deployments per day from n devs* secure* fast rollbacks on failure* zero downtime* dependency handling...
# continuous deployment dev        dev             git push/pull                                                  repo dev...
# where the magic happens
pull from master ->clone previous production for backup ->   run test suite (abort on failure) ->  deploy/compress static ...
# continuous deploymentassumptions:* master is always production safe  use pull request for large teams* bootstrapped pre-...
:failure:
# failure“Design for failure and  nothing will fail”       “Everything fails, all             the time”                  ~...
# failure* backup/restore strategy* bootstrapped AMIs* multi-AZ deployment
:limitations:
# limitations* disk I/O ✔ use multiple EBS in RAID config* database ✔ sharding ✔ multiple read-only ✔ clustering* ram ✔ me...
# recap* the problem* development* deployment* failure* limitations* conclusion* questions
:one more thing:
:vendor lock-in: if you’re still following,there’s no such thing on AWS
# vendor lock-in* S3 ✔ pluggable storages* EC2 ✔ normal unix box* DynamoDB ✔ fully compatible NoSQL* RDS ✔ fully compatibl...
:conclusion:
# conclusion* use best practices and you’ll be safe* your stack matters* Cloud != high availability* Cloud != high perform...
# questions? challenges?          ?      @goldstein      aka leonidas tsementzis      leotsem [at] gmail.com
# thank you          !      @goldstein      aka leonidas tsementzis      leotsem [at] gmail.com
Upcoming SlideShare
Loading in...5
×

Architecting for the cloud

818

Published on

Published in: Technology
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
818
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
5
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

Architecting for the cloud

  1. 1. ARCHITECTING for the CLOUDleonidas tsementzisaka @goldstein
  2. 2. # get social # awsuggr
  3. 3. # who’s talking leonidas tsementzis aka @goldstein* software architect, engineer [all major web/mobile platforms]* devOps [enthusiast, not a real sysadmin]* entrepreneur [n00b]
  4. 4. # format* the problem* development* deployment* failure* limitations* conclusion* questions
  5. 5. # the problem* increasing/decreasing resources on the fly using auto scaling* availability* performance* multi server painless deployment
  6. 6. :development:
  7. 7. # your stack matters* the single most important aspect* cloud-ready open source libraries for major platforms* saves you a lot of development time* rapid changes* can lock you in
  8. 8. # memory* avoid application level variables/ sessions* centralized storage: ✔ fast ✔ scalable ✔ efficient Amazon DynamoDB
  9. 9. :storage:
  10. 10. # storage - single server server
  11. 11. # storage - multi server server farm server 1 server 2 server 3 server 4 - scripts - static files
  12. 12. # storage - multi server - S3 server farm server 1 server 2 server 3 server 4 - scripts - static files
  13. 13. # storage STORAGE MIDDLEWARE t e s s /si e / a ddr l / l oca uncpathsite application S3 AP I
  14. 14. # storageusing a pluggable storage middleware, we can create storageslike...:* local filesystem* network storage* Amazon S3* Rackspace CloudFiles* database (BLOB)* GridFS (MongoDB)* FTP, SFTP* Azure
  15. 15. # storage...and hopefully we don’t have to:
  16. 16. # storage...but if we have to:* avoid using HEAD/GET requests to check for existing files* store file list in memory instead* use S3 “PRELOAD_METADATA”
  17. 17. :task queuing:
  18. 18. # task queuinguse message/task queues for long running operations:* image resizes* external api calls* low priority updates* intensive calculations* big data queues* preparing hot caches* indexing updates* logging
  19. 19. # task queuing* organize tasks into different queues* organize queues into priority workers* scale workers using AWS auto scaling - send custom alerts using AWS CloudWatch API* it’s all about prioritiesAmazon SQS
  20. 20. :database:
  21. 21. # database* Amazon RDS does the trick if you’re on MySQL or Oracle* shard early mark down table dependencies from the start, work around this while you grow
  22. 22. :deployment:
  23. 23. # huh?* it’s your code* you know the dependencies* you know it’s breaking points* it’s your job to deal with deployment failures* continuous deployment? yes please!
  24. 24. # requirements* 50+ deployments per day from n devs* secure* fast rollbacks on failure* zero downtime* dependency handling (restart services, migrate dbs etc.)
  25. 25. # continuous deployment dev dev git push/pull repo dev dev git pull master $: fab production deploy server farm 0.0.0.1 0.0.0.2 0.0.0.3 0.0.0.4
  26. 26. # where the magic happens
  27. 27. pull from master ->clone previous production for backup -> run test suite (abort on failure) -> deploy/compress static files on S3 -> pre-compile less etc -> install new dependencies -> run db migration scripts -> cleanup -> rollback if something fails -> backup live db -> restart services if required
  28. 28. # continuous deploymentassumptions:* master is always production safe use pull request for large teams* bootstrapped pre-configured AMIs* handle stale servers with caretools:
  29. 29. :failure:
  30. 30. # failure“Design for failure and nothing will fail” “Everything fails, all the time” ~ Amazon CTO
  31. 31. # failure* backup/restore strategy* bootstrapped AMIs* multi-AZ deployment
  32. 32. :limitations:
  33. 33. # limitations* disk I/O ✔ use multiple EBS in RAID config* database ✔ sharding ✔ multiple read-only ✔ clustering* ram ✔ memcache/redis replication
  34. 34. # recap* the problem* development* deployment* failure* limitations* conclusion* questions
  35. 35. :one more thing:
  36. 36. :vendor lock-in: if you’re still following,there’s no such thing on AWS
  37. 37. # vendor lock-in* S3 ✔ pluggable storages* EC2 ✔ normal unix box* DynamoDB ✔ fully compatible NoSQL* RDS ✔ fully compatible MySQL/Oracle
  38. 38. :conclusion:
  39. 39. # conclusion* use best practices and you’ll be safe* your stack matters* Cloud != high availability* Cloud != high performance* Cloud != magic (but it’s close)
  40. 40. # questions? challenges? ? @goldstein aka leonidas tsementzis leotsem [at] gmail.com
  41. 41. # thank you ! @goldstein aka leonidas tsementzis leotsem [at] gmail.com
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×