Using the Amazon cloud requires a lot of moving parts like AMIs, ASGs, and ELBs. See how a small Netflix team developed web-based tools to abstract and clarify these cloudy components for use by hundreds of engineers. Presented at "Talk Cloudy to Me II" hosted by the Silicon Valley Cloud Computing Group in 2011. Apple Keynote version with animations is on Google Docs at http://bit.ly/netflixcloudtools

1. Building Cloud Tools for Netflix
@joesondow #netflixcloud
Sunday, September 25, 2011

2. Note takers, take note
http://www.slideshare.net/joesondow
@joesondow #netflixcloud
Sunday, September 25, 2011

3. Agenda
@joesondow #netflixcloud
Sunday, September 25, 2011

4. Agenda
Who am I?
@joesondow #netflixcloud
Sunday, September 25, 2011

5. Agenda
Who am I?
Why am I here?
@joesondow #netflixcloud
Sunday, September 25, 2011

6. Agenda
Who am I?
Why am I here?
Cloud deployment model
@joesondow #netflixcloud
Sunday, September 25, 2011

7. Agenda
Who am I?
Why am I here?
Cloud deployment model
Netflix Application Console
@joesondow #netflixcloud
Sunday, September 25, 2011

8. Agenda
Who am I?
Why am I here?
Cloud deployment model
Netflix Application Console
Why not the AWS Console?
@joesondow #netflixcloud
Sunday, September 25, 2011

9. Who am I?
@joesondow
Sunday, September 25, 2011

10. Who am I?
@joesondow
Sunday, September 25, 2011

11. Who am I?
Joe Sondow
@joesondow
Sunday, September 25, 2011

12. Who am I?
Joe Sondow
New York City
@joesondow
Sunday, September 25, 2011

13. Who am I?
Joe Sondow
New York City
Netflix since 2010
@joesondow
Sunday, September 25, 2011

14. Who am I?
Joe Sondow
New York City
Netflix since 2010
@joesondow
Sunday, September 25, 2011

15. Who am I?
Joe Sondow
New York City
Netflix since 2010
Cloud Tools
@joesondow
Sunday, September 25, 2011

16. Who am I?
Joe Sondow
New York City
Netflix since 2010
Cloud Tools
Grails
@joesondow
Sunday, September 25, 2011

17. Who am I?
Joe Sondow
New York City
Netflix since 2010
Cloud Tools
Grails
jQuery
@joesondow
Sunday, September 25, 2011

18. Why am I here?
Sunday, September 25, 2011

19. Why am I here?
Sunday, September 25, 2011

20. Why am I here?
Sell you something
Sunday, September 25, 2011

21. Why am I here?
Sell you something
Discuss business plans
Sunday, September 25, 2011

22. Why am I here?
Sell you something
Discuss business plans
Answer technical questions
Sunday, September 25, 2011

23. Why am I here?
Sell you something
Discuss business plans
Answer technical questions
Be a smaller fish in AWS
Sunday, September 25, 2011

24. Why am I here?
Sell you something
Discuss business plans
Answer technical questions
Be a smaller fish in AWS
Steal your engineers
Sunday, September 25, 2011

25. Cloud deployment model
Sunday, September 25, 2011

26. Cloud deployment model
Applications
Clusters
Sunday, September 25, 2011

27. Cloud deployment model
Sunday, September 25, 2011

28. Cloud deployment model
Auto Scaling
Group
Sunday, September 25, 2011

29. Cloud deployment model
Auto Scaling
Group
Launch
Configuration
Sunday, September 25, 2011

30. Cloud deployment model
Elastic Load
Balancer
Auto Scaling
Group
Launch
Configuration
Sunday, September 25, 2011

31. Cloud deployment model
Elastic Load
Balancer
Auto Scaling
Group
Launch
Configuration
Amazon Machine
Image
Sunday, September 25, 2011

32. Cloud deployment model
Elastic Load
Balancer
Auto Scaling
Group
Security
Group
Launch
Configuration
Amazon Machine
Image
Sunday, September 25, 2011

33. Cloud deployment model
Elastic Load
Balancer
Auto Scaling
Group
Security
Instances
Group
Launch
Configuration
Amazon Machine
Image
Sunday, September 25, 2011

34. Cloud deployment model
Elastic Load
Balancer
Auto Scaling
Group
Security
Instances
Group
Launch
Configuration
Amazon Machine
Image
Sunday, September 25, 2011

35. Cloud deployment model
Elastic Load
Balancer
Auto Scaling
Group
Security
Instances
Group
Launch
Configuration
Amazon Machine
Image
Sunday, September 25, 2011

36. Cloud deployment model
Elastic Load
Balancer
Auto Scaling
Group
Security
Instances
Group
Launch
Configuration
Amazon Machine
Image
Sunday, September 25, 2011

37. Cloud deployment model
Sunday, September 25, 2011

38. Cloud deployment model
Search
Sunday, September 25, 2011

39. Cloud deployment model
API Search
Sunday, September 25, 2011

40. Cloud deployment model
Ratings
API Search
Sunday, September 25, 2011

41. Cloud deployment model
Streaming Starts
Ratings
API Search
Sunday, September 25, 2011

42. Cloud deployment model
Streaming Starts
Ratings
API Search
Autocomplete
Sunday, September 25, 2011

43. Cloud deployment model
Sign Up
Streaming Starts
Ratings
API Search
Autocomplete
Sunday, September 25, 2011

44. Cloud deployment model
Sign Up
Streaming Starts
Ratings
Application
Application
Application
API Search
Autocomplete
Application
Application Application
Sunday, September 25, 2011

45. Inventing the Application
Sunday, September 25, 2011

46. Inventing the Application
Problem:
Application is not an Amazon concept
Sunday, September 25, 2011

47. Inventing the Application
Problem:
Application is not an Amazon concept
Solution:
Create an Application domain in SimpleDB
Enforce naming conventions on Amazon objects
Sunday, September 25, 2011

48. Fast Rollback
Sunday, September 25, 2011

49. Fast Rollback
Optimism causes outages
Sunday, September 25, 2011

50. Fast Rollback
Optimism causes outages
Production traffic is unique
Sunday, September 25, 2011

51. Fast Rollback
Optimism causes outages
Production traffic is unique
Keep old version running
Sunday, September 25, 2011

52. Fast Rollback
Optimism causes outages
Production traffic is unique
Keep old version running
Switch traffic to new version
Sunday, September 25, 2011

53. Fast Rollback
Optimism causes outages
Production traffic is unique
Keep old version running
Switch traffic to new version
Monitor results
Sunday, September 25, 2011

54. Fast Rollback
Optimism causes outages
Production traffic is unique
Keep old version running
Switch traffic to new version
Monitor results
Revert traffic quickly
Sunday, September 25, 2011

55. Fast Rollback
Sunday, September 25, 2011

56. Fast Rollback
api-frontend
api-usprod-v007
Sunday, September 25, 2011

57. Fast Rollback
api-frontend
api-usprod-v007 api-usprod-v008
Sunday, September 25, 2011

58. Fast Rollback
api-frontend
api-usprod-v007 api-usprod-v008
Sunday, September 25, 2011

59. Fast Rollback
api-frontend
api-usprod-v007 api-usprod-v008
Sunday, September 25, 2011

60. Fast Rollback
api-frontend
api-usprod-v007 api-usprod-v008
Sunday, September 25, 2011

61. Fast Rollback
api-frontend
api-usprod-v008
Sunday, September 25, 2011

62. Fast Rollback
Sunday, September 25, 2011

63. Fast Rollback
api-frontend
api-usprod-v007
Sunday, September 25, 2011

64. Fast Rollback
api-frontend
api-usprod-v007 api-usprod-v008
Sunday, September 25, 2011

65. Fast Rollback
api-frontend
api-usprod-v007 api-usprod-v008
Sunday, September 25, 2011

66. Fast Rollback
api-frontend
api-usprod-v007 api-usprod-v008
Sunday, September 25, 2011

67. Fast Rollback
api-frontend
api-usprod-v007
Sunday, September 25, 2011

68. Inventing the Cluster
Sunday, September 25, 2011

69. Inventing the Cluster
Problem:
Two ASGs with one function but different names
Sunday, September 25, 2011

70. Inventing the Cluster
Problem:
Two ASGs with one function but different names
Solution:
Append version number in reserved format
Parse ASG name to determine long-term “cluster”
Sunday, September 25, 2011

71. Netflix Application Console
Sunday, September 25, 2011

72. Netflix Application Console
Cloud interface
Sunday, September 25, 2011

73. Sunday, September 25, 2011

74. Sunday, September 25, 2011

75. Sunday, September 25, 2011

76. Sunday, September 25, 2011

77. Sunday, September 25, 2011

78. Sunday, September 25, 2011

79. Sunday, September 25, 2011

80. Sunday, September 25, 2011

81. Sunday, September 25, 2011

82. Sunday, September 25, 2011

83. Sunday, September 25, 2011

84. Sunday, September 25, 2011

85. Sunday, September 25, 2011

86. Sunday, September 25, 2011

87. REST API is easy in Grails
Sunday, September 25, 2011

88. REST API is easy in Grails
Sunday, September 25, 2011

89. REST API is easy in Grails
Sunday, September 25, 2011

90. REST API is easy in Grails
ApplicationController.groovy
Sunday, September 25, 2011

91. Why not the AWS console?
Sunday, September 25, 2011

92. Why not the AWS console?
No, not to keep me busy
Sunday, September 25, 2011

93. Why not the AWS console?
Sunday, September 25, 2011

94. Why not the AWS console?
Sunday, September 25, 2011

95. Why not the AWS console?
Hide keys
Sunday, September 25, 2011

96. Why not the AWS console?
Hide keys
Customize model
Sunday, September 25, 2011

97. Why not the AWS console?
Hide keys
Customize model
Automate workflow
Sunday, September 25, 2011

98. Why not the AWS console?
Hide keys
Customize model
Automate workflow
Enforce conventions
Sunday, September 25, 2011

99. Why not the AWS console?
Hide keys
Customize model
Automate workflow
Enforce conventions
Log changes
Sunday, September 25, 2011

100. Why not the AWS console?
Hide keys
Customize model
Automate workflow
Enforce conventions
Log changes
Integrate systems
Sunday, September 25, 2011

101. Why not the AWS console?
Hide keys
Customize model
Automate workflow
Enforce conventions
Log changes
Integrate systems
Create REST API
Sunday, September 25, 2011

102. Thank you
@joesondow
http://www.slideshare.net/joesondow
Sunday, September 25, 2011

103. Thank you
Questions?
@joesondow
http://www.slideshare.net/joesondow
Sunday, September 25, 2011