Your SlideShare is downloading. ×
It's Not Always Sunny in the Clouds
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×
Saving this for later? Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime – even offline.
Text the download link to your phone
Standard text messaging rates apply

It's Not Always Sunny in the Clouds

1,734
views

Published on

I discuss why I became disillusioned with cloud computing and why I ultimately came to embrace it, using the case study of OtherInbox.com

I discuss why I became disillusioned with cloud computing and why I ultimately came to embrace it, using the case study of OtherInbox.com

Published in: Technology, Education

2 Comments
2 Likes
Statistics
Notes
No Downloads
Views
Total Views
1,734
On Slideshare
0
From Embeds
0
Number of Embeds
4
Actions
Shares
0
Downloads
63
Comments
2
Likes
2
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide



  •      - so you either have to pay for RightScale, EY, Heroku, Scalr, etc.
         - or you have to roll your own
         - or you can install an open source project
         - either way, it's going to cost you time or treasure


  • Nine different sets of creds to manage
    Maybe YAML not so much

  • some people, like Gartner, think AWS still charges too much
  • - general AWS launch time
    - JRuby JVM

  • - For us, SQS, S3, EC2

  • You can get AWS SLAs, premium support
  • otherwise you’re too dependent on the VCS host
  • - Because nodes are ephemeral
    - We use Alertra & custom scripts & rightscale
    - we've written a ton of custom monitoring scrips

  • one log file turns out to be better than many
    40+ boxes, ephemeral so don’t save logs
    have not found a palatable alternative to splunk
    hoptoad FTW

  • - use to test \"can we hit SQS a lot\", now we test \"can a message get delivered in a timely fashion end-to-end\"
    - show snippet

  • but can be a lifesaver


  • - show idempotency code example
  • - show idempotency code example
  • - show idempotency code example


  • - esp if you have a task that HAS to be performed immediately
    - we use this for email delivery and generally is fast enough
  • you miss a lot of awesome aggregation that RDBMs does for you

  • - Cloud services give a lot of expected and unexpected errors
    - Write lots of exponential retry code

  • - Testing is going to be difficult
    - Need something like Paypal test gateway
    - Fakeweb shows promise
    I’m working on something
  • - connections for killed-off servers may hang around
    - lowering the timeout caused lots of issues for seemingly-unrelated pieces
  • \"A system like Kerberos which is ideal for multi-server multi-role environments requires too much overhead to setup”
  • - cloudkick may change this
    - it's bound to change



  • - forces you to loosely couple components
    - affords you trade offs - you can throw more servers at a problem

  • - very low activation energy

  • - very low activation energy








  • Transcript

    • 1. It’s not always sunny in the clouds Me, teetering Text
    • 2. Mike Subelsky @subelsky
    • 3. Mike Subelsky @subelsky
    • 4. 1 / 1 / 08 08:00:00 ET
    • 5. “Everything needs to be automated”
    • 6. Autoscaling is the easiest part
    • 7. Think carefully about credential management
    • 8. You really could use internal DNS
    • 9. It's maybe not that cheap
    • 10. Launching servers is not that fast
    • 11. You will become dependent on “glue” services
    • 12. You will become dependent on “glue” services
    • 13. You will depend on a distant faceless provider
    • 14. Use DVCS
    • 15. You will spend a lot of time on monitoring
    • 16. Your logs will runneth over
    • 17. Your logs will runneth over
    • 18. Write lots of “in-process tests”
    • 19. Write lots of “in-process tests” m = Message.find_last_slug_message secs = m.created_at - m.date_sent if secs > 600 ErrorNotifier.fatal(quot;Missed slug deliveryquot;,secs,m) end
    • 20. Snapshots are slow
    • 21. Snapshots are slow
    • 22. Rails will be the least of your worries
    • 23. Cloud services involve subtle-yet-massive tradeos
    • 24. SQS guarantees delivery at least once
    • 25. SQS guarantees delivery at least once def self.guard_against_repeat_delivery(smtp_message_id) begin create!(:smtp_message_id = smtp_message_id) return true rescue ActiveRecord::StatementInvalid end
    • 26. SQS guarantees delivery at least once def self.guard_against_repeat_delivery(smtp_message_id) begin create!(:smtp_message_id = smtp_message_id) return true rescue ActiveRecord::StatementInvalid rows_updated = raw_update(quot;UPDATE application_message_logs SET last_attempt_at = UTC_TIMESTAMP(), WHERE smtp_message_id = #{smtp_message_id} AND last_attempt_at DATE_SUB(UTC_TIMESTAMP(), INTERVAL 900 SECOND)quot;) end
    • 27. SQS guarantees delivery at least once def self.guard_against_repeat_delivery(smtp_message_id) begin create!(:smtp_message_id = smtp_message_id) return true rescue ActiveRecord::StatementInvalid rows_updated = raw_update(quot;UPDATE application_message_logs SET last_attempt_at = UTC_TIMESTAMP(), WHERE smtp_message_id = #{smtp_message_id} AND last_attempt_at DATE_SUB(UTC_TIMESTAMP(), INTERVAL 900 SECOND)quot;) if rows_updated 1 logger.warn(quot;Preventing duplicate msg delivery #{smtp_message_id}quot;) return false else return true end end
    • 28. Queue lengths inaccurate for 1000 items
    • 29. SQS not necessarily FIFO
    • 30. So you may not want a cloud queue
    • 31. SimpleDB optimized for writes, not reads
    • 32. You must code defensively
    • 33. There are no good quot;cloud sandboxesquot;
    • 34. Pay attention to MySQL timeouts
    • 35. quot;User account management is -not- ideal.quot; -Deacon Bradley
    • 36. You are locked-in to your provider
    • 37. Relational DB may not be the best choice
    • 38. Is there a benefit?
    • 39. Changes the way you write code
    • 40. You can start right away
    • 41. Pretty awesome redundancy
    • 42. Time for an example?
    • 43. US-EAST-1C o t Ta me SQS s a kin en e Fil ars g 3P S Postfix MailSender MailReceiver MailParser Master DB le Ra Fi w Mb x bo ox M Fil w e Ra S3 US-EAST-1B to Ta e SQS am skin en e Fil ars g S3 P Postfix MailSender MailReceiver MailParser Master DB le Ra Fi w Mb x bo ox M Fil w e Ra S3
    • 44. US-EAST-1C o t Ta me SQS s a kin en e Fil ars g 3P S Postfix MailSender MailReceiver MailParser Master DB le Ra Fi w Mb x bo ox M Fil w e Ra S3 X XX US-EAST-1B to Ta e SQS am skin en e Fil ars g S3 P Postfix MailSender MailReceiver MailParser Master DB le Ra Fi w Mb x bo ox M Fil w e Ra S3
    • 45. X US-EAST-1C o t Ta me SQS s a kin en e Fil ars g 3P S Postfix MailSender MailReceiver MailParser Master DB le Ra Fi w Mb x bo ox M Fil w e Ra S3 X US-EAST-1B to Ta e SQS am skin en e Fil ars g S3 P Postfix MailSender MailReceiver MailParser Master DB le Ra Fi w Mb x bo ox M Fil w e Ra S3
    • 46. US-EAST-1C X o t Ta me SQS s a kin en e Fil ars g 3P S Postfix MailSender MailReceiver MailParser Master DB le Ra Fi w Mb x bo ox M Fil w e Ra S3 US-EAST-1B X to Ta e SQS am skin en e Fil ars g S3 P Postfix MailSender MailReceiver MailParser Master DB le Ra Fi w Mb x bo ox M Fil w e Ra S3
    • 47. X US-EAST-1C SMTP Cloud o t Ta me SQS s a kin en e Fil ars g 3P S Postfix MailSender MailReceiver MailParser Master DB le Ra Fi w Mb x bo ox M Fil w e Ra S3 US-EAST-1B X to Ta e SQS am skin en e Fil ars g S3 P Postfix MailSender MailReceiver MailParser Master DB le Ra Fi w Mb x bo ox M Fil w e Ra S3
    • 48. Me,recovering
    • 49. Questions? mike@oib.com @subelsky subelsky.com

    ×