• Share
  • Email
  • Embed
  • Like
  • Save
  • Private Content
Asgard, the Grails App that Deploys Netflix to the Cloud
 

Asgard, the Grails App that Deploys Netflix to the Cloud

on

  • 20,504 views

Overview and technical exploration of Asgard, a graphical web console created by Netflix for cloud deployments and operations. Presented at the GR8 Conference in Copenhagen, Denmark, June 8, 2012.

Overview and technical exploration of Asgard, a graphical web console created by Netflix for cloud deployments and operations. Presented at the GR8 Conference in Copenhagen, Denmark, June 8, 2012.

Apple Keynote version with animations available at http://bit.ly/asgardgr8denmark

Statistics

Views

Total Views
20,504
Views on SlideShare
17,978
Embed Views
2,526

Actions

Likes
30
Downloads
409
Comments
4

22 Embeds 2,526

http://gr8conf.eu 1071
http://groovyflow.com 822
http://www.scoop.it 323
https://twitter.com 137
http://www.groovyflow.com 38
http://localhost 37
http://feeds.feedburner.com 23
http://lanyrd.com 19
http://www.mefeedia.com 17
http://gr8conf.org 11
http://www.linkedin.com 9
http://leonlohezic14.tumblr.com 7
http://webcache.googleusercontent.com 2
http://assets.txmblr.com 2
https://www.google.dk 1
http://www.tumblr.com 1
http://translate.googleusercontent.com 1
http://site7.miydim.com 1
https://twimg0-a.akamaihd.net 1
https://si0.twimg.com 1
http://50.19.243.188 1
https://www.linkedin.com 1
More...

Accessibility

Upload Details

Uploaded via as Apple Keynote

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel

14 of 4 previous next Post a comment

  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
  • Is there a video explaining the slides?
    Are you sure you want to
    Your message goes here
    Processing…
  • Matt, we use both ELBs and our own open source inter-instance load balancer software called Eureka. Asgard integrates with both ELBs and Eureka, and treats them as traffic sources to switch on and off for all the instances of an ASG with one button click. The AWS mechanism is to deregister all the ASG's instances from the ELBs that are associated with the ASG, and then to register all the instances with all the ELBs that are associated with the ASG. The other important detail is to suspend or resume the AddToLoadBalancer process on the ASG so that future instances launched by the Auto Scaling system will be registered with the ELB, or not, based on your wishes.

    Or, you can just run Asgard and click a button to enable or disable a full ASG.
    Are you sure you want to
    Your message goes here
    Processing…
  • We are using a similar manner to deploy by creating a new asg, adding it to the same ELB and then removing the old asg after all are healthy. I would like to do what you guys are doing and have two concurrent asgs, one new and one old, and be able to switch back if the deploy is bad. My question is how do you guys actually do that on the AWS level? Are you using ELB? Since one cannot change the load balancer of an active asg how can I direct traffic to a different ELBs without using DNS?

    Thanks
    Are you sure you want to
    Your message goes here
    Processing…
  • Asgard 1.0 is now open source. http://techblog.netflix.com/2012/06/asgard-web-based-cloud-management-and.html
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment
  • \n
  • \n
  • Asgard is a Grails application used by Netflix engineers to deploy code changes and manage resources in the Amazon cloud on a massive scale. This presentation will include slides and discussion about aspects of Asgard from both the user and developer points of view.\n\nThe Netflix cloud model of Applications and Clusters built on top of Amazon building blocks\nAsgard’s process for large deployments and fast rollback\nOpen sourcing Asgard on github in 2012\nComparison with the AWS Management Console\nTour of the user interface\nPublishing a REST interface API for other tools to call Asgard\nEnhancing Grails URLs to add region-specificity\nUsing external configuration to pull Netflix-specific details out of Asgard\nStoring metadata on cloud objects instead of a database\nUsing closures to help cache massive cloud metadata for multiple Amazon regions\nTango open source icons to improve the user experience\nJavaScript filters for large list pages and drop-downs\nMocking the AWS back-end for offline automated testing\n
  • Asgard is a Grails application used by Netflix engineers to deploy code changes and manage resources in the Amazon cloud on a massive scale. This presentation will include slides and discussion about aspects of Asgard from both the user and developer points of view.\n\nThe Netflix cloud model of Applications and Clusters built on top of Amazon building blocks\nAsgard’s process for large deployments and fast rollback\nOpen sourcing Asgard on github in 2012\nComparison with the AWS Management Console\nTour of the user interface\nPublishing a REST interface API for other tools to call Asgard\nEnhancing Grails URLs to add region-specificity\nUsing external configuration to pull Netflix-specific details out of Asgard\nStoring metadata on cloud objects instead of a database\nUsing closures to help cache massive cloud metadata for multiple Amazon regions\nTango open source icons to improve the user experience\nJavaScript filters for large list pages and drop-downs\nMocking the AWS back-end for offline automated testing\n
  • Asgard is a Grails application used by Netflix engineers to deploy code changes and manage resources in the Amazon cloud on a massive scale. This presentation will include slides and discussion about aspects of Asgard from both the user and developer points of view.\n\nThe Netflix cloud model of Applications and Clusters built on top of Amazon building blocks\nAsgard’s process for large deployments and fast rollback\nOpen sourcing Asgard on github in 2012\nComparison with the AWS Management Console\nTour of the user interface\nPublishing a REST interface API for other tools to call Asgard\nEnhancing Grails URLs to add region-specificity\nUsing external configuration to pull Netflix-specific details out of Asgard\nStoring metadata on cloud objects instead of a database\nUsing closures to help cache massive cloud metadata for multiple Amazon regions\nTango open source icons to improve the user experience\nJavaScript filters for large list pages and drop-downs\nMocking the AWS back-end for offline automated testing\n
  • Asgard is a Grails application used by Netflix engineers to deploy code changes and manage resources in the Amazon cloud on a massive scale. This presentation will include slides and discussion about aspects of Asgard from both the user and developer points of view.\n\nThe Netflix cloud model of Applications and Clusters built on top of Amazon building blocks\nAsgard’s process for large deployments and fast rollback\nOpen sourcing Asgard on github in 2012\nComparison with the AWS Management Console\nTour of the user interface\nPublishing a REST interface API for other tools to call Asgard\nEnhancing Grails URLs to add region-specificity\nUsing external configuration to pull Netflix-specific details out of Asgard\nStoring metadata on cloud objects instead of a database\nUsing closures to help cache massive cloud metadata for multiple Amazon regions\nTango open source icons to improve the user experience\nJavaScript filters for large list pages and drop-downs\nMocking the AWS back-end for offline automated testing\n
  • Asgard is a Grails application used by Netflix engineers to deploy code changes and manage resources in the Amazon cloud on a massive scale. This presentation will include slides and discussion about aspects of Asgard from both the user and developer points of view.\n\nThe Netflix cloud model of Applications and Clusters built on top of Amazon building blocks\nAsgard’s process for large deployments and fast rollback\nOpen sourcing Asgard on github in 2012\nComparison with the AWS Management Console\nTour of the user interface\nPublishing a REST interface API for other tools to call Asgard\nEnhancing Grails URLs to add region-specificity\nUsing external configuration to pull Netflix-specific details out of Asgard\nStoring metadata on cloud objects instead of a database\nUsing closures to help cache massive cloud metadata for multiple Amazon regions\nTango open source icons to improve the user experience\nJavaScript filters for large list pages and drop-downs\nMocking the AWS back-end for offline automated testing\n
  • Asgard is a Grails application used by Netflix engineers to deploy code changes and manage resources in the Amazon cloud on a massive scale. This presentation will include slides and discussion about aspects of Asgard from both the user and developer points of view.\n\nThe Netflix cloud model of Applications and Clusters built on top of Amazon building blocks\nAsgard’s process for large deployments and fast rollback\nOpen sourcing Asgard on github in 2012\nComparison with the AWS Management Console\nTour of the user interface\nPublishing a REST interface API for other tools to call Asgard\nEnhancing Grails URLs to add region-specificity\nUsing external configuration to pull Netflix-specific details out of Asgard\nStoring metadata on cloud objects instead of a database\nUsing closures to help cache massive cloud metadata for multiple Amazon regions\nTango open source icons to improve the user experience\nJavaScript filters for large list pages and drop-downs\nMocking the AWS back-end for offline automated testing\n
  • Asgard is a Grails application used by Netflix engineers to deploy code changes and manage resources in the Amazon cloud on a massive scale. This presentation will include slides and discussion about aspects of Asgard from both the user and developer points of view.\n\nThe Netflix cloud model of Applications and Clusters built on top of Amazon building blocks\nAsgard’s process for large deployments and fast rollback\nOpen sourcing Asgard on github in 2012\nComparison with the AWS Management Console\nTour of the user interface\nPublishing a REST interface API for other tools to call Asgard\nEnhancing Grails URLs to add region-specificity\nUsing external configuration to pull Netflix-specific details out of Asgard\nStoring metadata on cloud objects instead of a database\nUsing closures to help cache massive cloud metadata for multiple Amazon regions\nTango open source icons to improve the user experience\nJavaScript filters for large list pages and drop-downs\nMocking the AWS back-end for offline automated testing\n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n

Asgard, the Grails App that Deploys Netflix to the Cloud Asgard, the Grails App that Deploys Netflix to the Cloud Presentation Transcript

  • // Feed your brain GR8Conf Europe 2012 gr8.technologies.each{ June 6th-8th yourBrain << it Copenhagen, Denmark }The Grails App that Deploys Netflix to the CloudJoe Sondow, Netflix @joesondow #gr8conf
  • Slides onlinehttp://slideshare.net/joesondow @joesondow #gr8conf
  • Agenda
  • AgendaWho am I? Why am I here?
  • AgendaWho am I? Why am I here?Asgard, Screen shots
  • AgendaWho am I? Why am I here?Asgard, Screen shotsNetflix, Freedom and Responsibility
  • AgendaWho am I? Why am I here?Asgard, Screen shotsNetflix, Freedom and ResponsibilityRegional URLs
  • AgendaWho am I? Why am I here?Asgard, Screen shotsNetflix, Freedom and ResponsibilityRegional URLsCloud Deployment Model
  • AgendaWho am I? Why am I here?Asgard, Screen shotsNetflix, Freedom and ResponsibilityRegional URLsCloud Deployment ModelOffline Development
  • AgendaWho am I? Why am I here?Asgard, Screen shotsNetflix, Freedom and ResponsibilityRegional URLsCloud Deployment ModelOffline DevelopmentWhy not the AWS Console?
  • Who am I? @joesondow #gr8conf
  • Who am I? @joesondow #gr8conf
  • Who am I?Joe Sondow @joesondow #gr8conf
  • Who am I?Joe SondowNetflix since 2010 @joesondow #gr8conf
  • Who am I?Joe SondowNetflix since 2010Asgard lead @joesondow #gr8conf
  • Who am I?Joe SondowNetflix since 2010Asgard leadGrails @joesondow #gr8conf
  • Who am I?Joe SondowNetflix since 2010Asgard leadGrailsjQuery @joesondow #gr8conf
  • Why am I here? @joesondow #gr8conf
  • Why am I here?
  • Why am I here?Sell you something
  • Why am I here?Sell you somethingDiscuss business plans
  • Why am I here?Sell you somethingDiscuss business plansAnswer technical questions
  • Why am I here?Sell you somethingDiscuss business plansAnswer technical questionsBe a smaller fish in AWS
  • Why am I here?Sell you somethingDiscuss business plansAnswer technical questionsBe a smaller fish in AWSGive back to community
  • Why am I here?Sell you somethingDiscuss business plansAnswer technical questionsBe a smaller fish in AWSGive back to communitySteal your engineers
  • Asgard
  • Asgard
  • AsgardScreen shots
  • AsgardApplication list
  • AsgardAuto Scaling Group list
  • AsgardCluster deployment, ready for fast rollback
  • Asgard
  • AsgardCloud deployment
  • AsgardCloud deploymentCloud management
  • AsgardCloud deploymentCloud managementStarted 2010
  • AsgardCloud deploymentCloud managementStarted 2010Open source in 2012
  • AsgardCloud deploymentCloud managementStarted 2010Open source in 2012 Follow @NetflixOSS
  • AsgardCloud deploymentCloud managementStarted 2010Open source in 2012 Follow @NetflixOSS100’s of Jira tickets
  • AsgardCloud deploymentCloud managementStarted 2010Open source in 2012 Follow @NetflixOSS100’s of Jira ticketsActively developed
  • Joe Sondow Clay McCoy Jason Gritman @joesondow @claymccoy @jgritmanAsgard TeamJune 2012(The dolphin writes integration tests)
  • With more than 25 million streaming members in theUnited States, Canada, Latin America, the UnitedKingdom and Ireland, Netflix, Inc. (NASDAQ: NFLX) isthe worlds leading internet subscription service forenjoying movies and TV programs.(from http://ir.netflix.com)
  • Freedom and Responsibility
  • Freedom and Responsibility
  • Freedom and ResponsibilityCloud SOA
  • Freedom and ResponsibilityCloud SOA100’s of services
  • Freedom and ResponsibilityCloud SOA100’s of servicesSmall teams
  • Freedom and ResponsibilityCloud SOA100’s of servicesSmall teamsIndependent releases
  • Freedom and ResponsibilityCloud SOA100’s of servicesSmall teamsIndependent releasesControlled chaos
  • Regional URLs
  • Regional URLsLet’s see some code
  • Regional URLsCloud object URLs were unique, conventional, intuitiveDefault Grails URL mapping is excellent
  • Regional URLsLater, Netflix needed to use multiple Amazon regions
  • Regional URLsRegion is another URL dimensionAdd URL mapping, but don’t break existing links
  • Regional URLsRegionService.groovy (excerpt)
  • Regional URLsRegionService.groovy (excerpt)
  • Regional URLsSome controllers should never be regionalRegionService detects RegionAgnostic annotation
  • Cloud deployment model
  • Cloud deployment modelApplications and Clusters
  • Cloud deployment model
  • Cloud deployment model Auto Scaling Group
  • Cloud deployment model Auto Scaling Group Launch Configuration
  • Cloud deployment model Elastic Load Balancer Auto Scaling Group Launch Configuration
  • Cloud deployment model Elastic Load Balancer Auto Scaling Group Launch Configuration Amazon Machine Image
  • Cloud deployment model Elastic Load Balancer Auto Scaling Group Security Group Launch Configuration Amazon Machine Image
  • Cloud deployment model Elastic Load Balancer Auto Scaling Group Security Instances Group Launch Configuration Amazon Machine Image
  • Cloud deployment model Elastic Load Balancer Auto Scaling Group Security Instances Group Launch Configuration Amazon Machine Image
  • Cloud deployment model Elastic Load Balancer Auto Scaling Group Security Instances Group Launch Configuration Amazon Machine Image
  • Cloud deployment model Elastic Load Balancer Auto Scaling Group Security Instances Group Launch Configuration Amazon Machine Image
  • Cloud deployment model
  • Cloud deployment model Search
  • Cloud deployment model API Search
  • Cloud deployment model Ratings API Search
  • Cloud deployment modelStreaming Starts Ratings API Search
  • Cloud deployment modelStreaming Starts Ratings API SearchAutocomplete
  • Cloud deployment model Sign UpStreaming Starts Ratings API SearchAutocomplete
  • Cloud deployment model Sign UpStreaming Starts Ratings Application Application Application API SearchAutocomplete Application Application Application
  • Inventing the Application
  • Inventing the Application Problem: Application is not an Amazon concept Solution: Create an Application domain in SimpleDB Enforce naming conventions on Amazon objects
  • Fast Rollback
  • Fast RollbackOptimism causes outages
  • Fast RollbackOptimism causes outagesProduction traffic is unique
  • Fast RollbackOptimism causes outagesProduction traffic is uniqueKeep old version running
  • Fast RollbackOptimism causes outagesProduction traffic is uniqueKeep old version runningSwitch traffic to new version
  • Fast RollbackOptimism causes outagesProduction traffic is uniqueKeep old version runningSwitch traffic to new versionMonitor results
  • Fast RollbackOptimism causes outagesProduction traffic is uniqueKeep old version runningSwitch traffic to new versionMonitor resultsRevert traffic quickly
  • Fast Rollback
  • Fast Rollback api-frontend api-usprod-v007
  • Fast Rollback api-frontend api-usprod-v007 api-usprod-v008
  • Fast Rollback api-frontend api-usprod-v007 api-usprod-v008
  • Fast Rollback api-frontend api-usprod-v007 api-usprod-v008
  • Fast Rollback api-frontend api-usprod-v007 api-usprod-v008
  • Fast Rollback api-frontend api-usprod-v007
  • Inventing the Cluster
  • Inventing the Cluster Problem: Two ASGs with one function but different names Solution: Append version number in reserved format Parse ASG name to determine long-term “cluster”
  • Inventing the ClusterInstead of keeping a database in sync, use namingconventions to store the source in truth in Amazon’s APIapi Applicationapi-usprod Clusterapi-usprod-v007 Auto Scaling Groupapi-usprod-v008 Auto Scaling Group
  • Offline Development
  • Offline DevelopmentMakes on a plane
  • Offline DevelopmentMock dataMock behaviorSystem property switch offline=true
  • Mock Datahttp://asgardprod/us-east-1/autoScaling/list.json
  • Mock DataParse JSON
  • Mock BehaviorOverride Amazon Java client methods
  • System Propertygrails run-app -Doffline=true
  • Why not the AWS console?
  • Why not the AWS console?No, not to keep me busy
  • Why not the AWS console?
  • Why not the AWS console?
  • Why not the AWS console?Hide keys
  • Why not the AWS console?Hide keysCustomize model
  • Why not the AWS console?Hide keysCustomize modelEnforce conventions
  • Why not the AWS console?Hide keysCustomize modelEnforce conventionsAutomate workflow
  • Why not the AWS console?Hide keysCustomize modelEnforce conventionsAutomate workflowLog changes
  • Why not the AWS console?Hide keysCustomize modelEnforce conventionsAutomate workflowLog changesIntegrate systems
  • Why not the AWS console?Hide keysCustomize modelEnforce conventionsAutomate workflowLog changesIntegrate systemsCreate REST API
  • @NetflixOSS
  • @NetflixOSShttp://techblog.netflix.com
  • @NetflixOSShttp://techblog.netflix.comOpen source announcement...
  • @NetflixOSShttp://techblog.netflix.comOpen source announcement...
  • @NetflixOSSThank you
  • @NetflixOSSThank youQuestions? @joesondow slideshare.net/joesondow linkedin.com/in/joesondow