Architecting for the cloud

  • 753 views
Uploaded on

 

More in: Technology
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
No Downloads

Views

Total Views
753
On Slideshare
0
From Embeds
0
Number of Embeds
0

Actions

Shares
Downloads
4
Comments
0
Likes
1

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. ARCHITECTING for the CLOUDleonidas tsementzisaka @goldstein
  • 2. # get social # awsuggr
  • 3. # who’s talking leonidas tsementzis aka @goldstein* software architect, engineer [all major web/mobile platforms]* devOps [enthusiast, not a real sysadmin]* entrepreneur [n00b]
  • 4. # format* the problem* development* deployment* failure* limitations* conclusion* questions
  • 5. # the problem* increasing/decreasing resources on the fly using auto scaling* availability* performance* multi server painless deployment
  • 6. :development:
  • 7. # your stack matters* the single most important aspect* cloud-ready open source libraries for major platforms* saves you a lot of development time* rapid changes* can lock you in
  • 8. # memory* avoid application level variables/ sessions* centralized storage: ✔ fast ✔ scalable ✔ efficient Amazon DynamoDB
  • 9. :storage:
  • 10. # storage - single server server
  • 11. # storage - multi server server farm server 1 server 2 server 3 server 4 - scripts - static files
  • 12. # storage - multi server - S3 server farm server 1 server 2 server 3 server 4 - scripts - static files
  • 13. # storage STORAGE MIDDLEWARE t e s s /si e / a ddr l / l oca uncpathsite application S3 AP I
  • 14. # storageusing a pluggable storage middleware, we can create storageslike...:* local filesystem* network storage* Amazon S3* Rackspace CloudFiles* database (BLOB)* GridFS (MongoDB)* FTP, SFTP* Azure
  • 15. # storage...and hopefully we don’t have to:
  • 16. # storage...but if we have to:* avoid using HEAD/GET requests to check for existing files* store file list in memory instead* use S3 “PRELOAD_METADATA”
  • 17. :task queuing:
  • 18. # task queuinguse message/task queues for long running operations:* image resizes* external api calls* low priority updates* intensive calculations* big data queues* preparing hot caches* indexing updates* logging
  • 19. # task queuing* organize tasks into different queues* organize queues into priority workers* scale workers using AWS auto scaling - send custom alerts using AWS CloudWatch API* it’s all about prioritiesAmazon SQS
  • 20. :database:
  • 21. # database* Amazon RDS does the trick if you’re on MySQL or Oracle* shard early mark down table dependencies from the start, work around this while you grow
  • 22. :deployment:
  • 23. # huh?* it’s your code* you know the dependencies* you know it’s breaking points* it’s your job to deal with deployment failures* continuous deployment? yes please!
  • 24. # requirements* 50+ deployments per day from n devs* secure* fast rollbacks on failure* zero downtime* dependency handling (restart services, migrate dbs etc.)
  • 25. # continuous deployment dev dev git push/pull repo dev dev git pull master $: fab production deploy server farm 0.0.0.1 0.0.0.2 0.0.0.3 0.0.0.4
  • 26. # where the magic happens
  • 27. pull from master ->clone previous production for backup -> run test suite (abort on failure) -> deploy/compress static files on S3 -> pre-compile less etc -> install new dependencies -> run db migration scripts -> cleanup -> rollback if something fails -> backup live db -> restart services if required
  • 28. # continuous deploymentassumptions:* master is always production safe use pull request for large teams* bootstrapped pre-configured AMIs* handle stale servers with caretools:
  • 29. :failure:
  • 30. # failure“Design for failure and nothing will fail” “Everything fails, all the time” ~ Amazon CTO
  • 31. # failure* backup/restore strategy* bootstrapped AMIs* multi-AZ deployment
  • 32. :limitations:
  • 33. # limitations* disk I/O ✔ use multiple EBS in RAID config* database ✔ sharding ✔ multiple read-only ✔ clustering* ram ✔ memcache/redis replication
  • 34. # recap* the problem* development* deployment* failure* limitations* conclusion* questions
  • 35. :one more thing:
  • 36. :vendor lock-in: if you’re still following,there’s no such thing on AWS
  • 37. # vendor lock-in* S3 ✔ pluggable storages* EC2 ✔ normal unix box* DynamoDB ✔ fully compatible NoSQL* RDS ✔ fully compatible MySQL/Oracle
  • 38. :conclusion:
  • 39. # conclusion* use best practices and you’ll be safe* your stack matters* Cloud != high availability* Cloud != high performance* Cloud != magic (but it’s close)
  • 40. # questions? challenges? ? @goldstein aka leonidas tsementzis leotsem [at] gmail.com
  • 41. # thank you ! @goldstein aka leonidas tsementzis leotsem [at] gmail.com