When I joined a startup already in progress as their first ops hire, I got a crash course in cloud operations. Running databases in EC2 without being on bare metal presents its own challenges; we also began using Hadoop and HBase on EMR, with tragicomic results. What monitoring existed was a twisty maze of half-measures, so improving our Mean Time To Lost Sleep required trying new tools and alerting strategies. And scaling performance meant relying on best practices and gut-feeling hunches. This talk will have appeal for those curious about AWS, about using MapReduce in the cloud, and about whether MongoDB is really "web scale". (Spoiler alert: lolol.) Come for the EC2 trivia; stay for the table-flipping. Notes and image credits: http://bridgetkromhout.com/speaking/2014/beyondthecode/notes/

1. Turtles All the
Way Down
Platform Ops in Public Cloud
Bridget Kromhout
@bridgetkromhout

2. @bridgetkromhout

3. We are the largest online video distributor of
international televised content streaming the
world's best movies, documentaries and TV
shows on demand with professional
subtitles.
@bridgetkromhout

4. Platform ops in public cloud?
Do you mean Platform
as a Service?
How is this different
from Infrastructure as a
Service?
@bridgetkromhout

5. @bridgetkromhout

6. @bridgetkromhout

7. (previous gig) SaaS Life
normal traffic
decision
to turn off
decision
to turn
back on
accidental removal
@bridgetkromhout

8. Platform?
@bridgetkromhout

9. @bridgetkromhout

10. @bridgetkromhout

11. @bridgetkromhout
AWS Regions*
(containing availability zones)
* for some values of
regions: Beijing &
Sydney too

12. Storage
No procurement delays!
All the IOPS!
No waiting!
Yes, cloud storage better
than the bad old days in
some ways, but with caveats.
@bridgetkromhout

13. Alphabet Soup: EBS, SSDs, pIOPS
Go with SSDs for your Elastic Block Store.
EBS-optimized instances = faster network
Provisioned IOPS: guaranteed, but prevent
bursting
@bridgetkromhout

14. Story Time!
Data stores and
sadness (as a service)
@bridgetkromhout

15. @bridgetkromhout
wow. such nosql. very
webscale.

16. @bridgetkromhout

17. “a single write
operation holds the
lock exclusively, and
no other read or write
operations may share
the lock.”
@bridgetkromhout

18. It’s 4am. Do you
know what your EMR
cluster is doing?
@bridgetkromhout

19. StatsD
monitoring != alerting
@bridgetkromhout

20. @bridgetkromhout

21. If it moves, we track it. Sometimes we’ll draw a graph
of something that isn’t moving yet, just in case it
decides to make a run for it. -- Ian Malpass, Etsy
@bridgetkromhout
measure all the things

22. So, back to this
platform stuff...
...how exactly do you
build and deploy it?
@bridgetkromhout

23. @bridgetkromhout

24. @bridgetkromhout

25. orchestration & config management
Current: Future possibilities:
@bridgetkromhout

26. kitten, not unicorn
@bridgetkromhout

27. @bridgetkromhout

28. “the
game
has
changed”
@littleidea
@bridgetkromhout

29. @bridgetkromhout

30. Questions? (and we’re hiring!)
@bridgetkromhout