Architecting for the cloud
Upcoming SlideShare
Loading in...5

Architecting for the cloud






Total Views
Views on SlideShare
Embed Views



0 Embeds 0

No embeds



Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
Post Comment
Edit your comment

Architecting for the cloud Architecting for the cloud Presentation Transcript

  • ARCHITECTING for the CLOUDleonidas tsementzisaka @goldstein
  • # get social # awsuggr
  • # who’s talking leonidas tsementzis aka @goldstein* software architect, engineer [all major web/mobile platforms]* devOps [enthusiast, not a real sysadmin]* entrepreneur [n00b]
  • # format* the problem* development* deployment* failure* limitations* conclusion* questions
  • # the problem* increasing/decreasing resources on the fly using auto scaling* availability* performance* multi server painless deployment
  • :development:
  • # your stack matters* the single most important aspect* cloud-ready open source libraries for major platforms* saves you a lot of development time* rapid changes* can lock you in
  • # memory* avoid application level variables/ sessions* centralized storage: ✔ fast ✔ scalable ✔ efficient Amazon DynamoDB
  • :storage:
  • # storage - single server server
  • # storage - multi server server farm server 1 server 2 server 3 server 4 - scripts - static files
  • # storage - multi server - S3 server farm server 1 server 2 server 3 server 4 - scripts - static files
  • # storage STORAGE MIDDLEWARE t e s s /si e / a ddr l / l oca uncpathsite application S3 AP I
  • # storageusing a pluggable storage middleware, we can create storageslike...:* local filesystem* network storage* Amazon S3* Rackspace CloudFiles* database (BLOB)* GridFS (MongoDB)* FTP, SFTP* Azure
  • # storage...and hopefully we don’t have to:
  • # storage...but if we have to:* avoid using HEAD/GET requests to check for existing files* store file list in memory instead* use S3 “PRELOAD_METADATA”
  • :task queuing:
  • # task queuinguse message/task queues for long running operations:* image resizes* external api calls* low priority updates* intensive calculations* big data queues* preparing hot caches* indexing updates* logging
  • # task queuing* organize tasks into different queues* organize queues into priority workers* scale workers using AWS auto scaling - send custom alerts using AWS CloudWatch API* it’s all about prioritiesAmazon SQS
  • :database:
  • # database* Amazon RDS does the trick if you’re on MySQL or Oracle* shard early mark down table dependencies from the start, work around this while you grow
  • :deployment:
  • # huh?* it’s your code* you know the dependencies* you know it’s breaking points* it’s your job to deal with deployment failures* continuous deployment? yes please!
  • # requirements* 50+ deployments per day from n devs* secure* fast rollbacks on failure* zero downtime* dependency handling (restart services, migrate dbs etc.)
  • # continuous deployment dev dev git push/pull repo dev dev git pull master $: fab production deploy server farm
  • # where the magic happens
  • pull from master ->clone previous production for backup -> run test suite (abort on failure) -> deploy/compress static files on S3 -> pre-compile less etc -> install new dependencies -> run db migration scripts -> cleanup -> rollback if something fails -> backup live db -> restart services if required
  • # continuous deploymentassumptions:* master is always production safe use pull request for large teams* bootstrapped pre-configured AMIs* handle stale servers with caretools:
  • :failure:
  • # failure“Design for failure and nothing will fail” “Everything fails, all the time” ~ Amazon CTO
  • # failure* backup/restore strategy* bootstrapped AMIs* multi-AZ deployment
  • :limitations:
  • # limitations* disk I/O ✔ use multiple EBS in RAID config* database ✔ sharding ✔ multiple read-only ✔ clustering* ram ✔ memcache/redis replication
  • # recap* the problem* development* deployment* failure* limitations* conclusion* questions
  • :one more thing:
  • :vendor lock-in: if you’re still following,there’s no such thing on AWS
  • # vendor lock-in* S3 ✔ pluggable storages* EC2 ✔ normal unix box* DynamoDB ✔ fully compatible NoSQL* RDS ✔ fully compatible MySQL/Oracle
  • :conclusion:
  • # conclusion* use best practices and you’ll be safe* your stack matters* Cloud != high availability* Cloud != high performance* Cloud != magic (but it’s close)
  • # questions? challenges? ? @goldstein aka leonidas tsementzis leotsem [at]
  • # thank you ! @goldstein aka leonidas tsementzis leotsem [at]