Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

WinOps Conf 2016 - Michael Greene - Release Pipelines

There are benefits to be gained when patterns and practices from developer techniques are applied to operations. Notably, a fully automated solution where infrastructure is managed as code and all changes are automatically validated before reaching production. This is a process shift that is recognized among industry innovators. For organizations already leveraging these processes, it should be clear how to leverage Microsoft platforms. For organizations that are new to the topic, it should be clear how to bring this process to your environment and what it means to your organizational culture. This presentation explains the components of a Release Pipeline for configuration as code, the value to operations, and solutions that are used when designing a new Release Pipeline architecture.

  • Login to see the comments

WinOps Conf 2016 - Michael Greene - Release Pipelines

  1. 1. The Release Pipeline Model Applied to Managing Windows Server and Microsoft Cloud Michael Greene Principal Program Manager Microsoft Enterprise Cloud CAT
  2. 2. whoami Michael Greene Principal Program Manager Enterprise Cloud Engineering CAT Team (Customer Feedback, Technical Enablement) CAT PM for PowerShell & DevOps 12 yrs at Microsoft 3 yrs BPOSD/O365D Ops @migreene Mgreenegit
  3. 3. When does a concept become a revolution? First, the new candidate must seem to resolve some outstanding and generally recognized problem that can be met in no other way. Second, the new paradigm must promise to preserve a relatively large part of the concrete problem-solving ability that has accrued to science through its predecessors. (The Structure of Scientific Revolutions, Thomas S. Kuhn)
  4. 4. Reading
  5. 5. Customer Landscape patterns & practices how do we include Windows Server? that’s a little more on the dev side, than IT Pro?
  6. 6. Treat Servers like PaaS service consistency fabric consistency Software As a Service consume service Platform As a Service deploy apps, jobs, data Infrastructure As a Service custom baseline manage binaries SQL Azure Azure App Services Azure Virtual Machines application consistency Increasing opportunity for configuration drift Physical/Virtual Servers
  7. 7. Configuration as Code • Lots of API's • Complex scenarios should be property:value • Living documentation • PowerShell DSC is the PLATFORM that all solutions can use to deploy and manage Windows Server • Azure Resource Manager is also Configuration as Code • Nothing replaces knowledge about the systems you intend to manage
  8. 8. The Release Pipeline
  9. 9. Who changed the environment? What did they change, exactly? When did the change occur?
  10. 10. What we did before Prototype in VM Submit change form Present to CAB Argue that the change is worthwhile Conflict oversight by committee Update docs
  11. 11. What we do now • Document everything in configuration as code • Including automated deployment and testing • Clone latest to workstation • Make change • Run tests locally • Push to Source • Who, What, When, Why (comments) • Merge with Master, resolve conflicts if needed
  12. 12. How will I catch problems at the earliest possible moment? Can elements be combined cleanly to produce the correct results? How will I be notified of a problem?
  13. 13. What we did before • Remote in on Saturday • Make sure servers are listed in RDCMan • Connect to servers and make changes • Reboot in order • Say a prayer / squeeze rabbit’s foot / perform tribal dance • Check that services still work correctly
  14. 14. What we do now • Trigger Build service • Webhook/agent, schedule, manual • Run scripts stored in Source • Validate quality • Perform work • Capture output * opportunity to implement Just Enough Administration
  15. 15. Engineer for Resiliency • What will this project require on Day 2? Day 30? Day 100? • Monitoring rules • Monitoring alerts • Automation runbooks • JEA endpoints • Onboard to backup services • Operational Validation Tests
  16. 16. How do we check for regulatory issues? How do I know this change will not cause an outage? Will this change work across every variation I have in my environments? Does this configuration meet our business requirements?
  17. 17. What we did before • Create VM • Install OS • Install app/service • Implement change • Manual, test OU, run script • Try app/service • Make an educated guess whether it still works
  18. 18. What we do now • Build runs Scripts from Source • Linting - Script Analyzer • Must meet guidelines set by organization • Unit Tests – Pester • Functionally should work as expected • Integration Tests – Pester / Kitchen • Should work across matrix of diverse combinations • Operational Validation – Pester / OVF • Service should do what it is supposed to do • TDD
  19. 19. Integration Testing with Pester All examples courtesy of the following blog post - $ADConfiguration = @{ Forest = @{ FQDN = 'pshirwin.local' ForestMode = 'Windows2012R2Forest' GlobalCatalogs = @( 'DC-DSC-01.pshirwin.local' ) SchemaMaster = 'DC-DSC-01.pshirwin.local' DomainNamingMaster = 'DC-DSC-01.pshirwin.local' } Domain = @{ NetBIOSName = 'PSHIRWIN' DomainMode = 'Windows2012R2Domain' RIDMaster = 'DC-DSC-01.pshirwin.local' PDCEmulator = 'DC-DSC-01.pshirwin.local' InfrastructureMaster = 'DC-DSC-01.pshirwin.local' DistinguishedName = 'DC=pshirwin,DC=local' DNSRoot = 'pshirwin.local' DomainControllers = @('DC-DSC-01') } Describe 'Active Directory configuration operational readiness' { Context 'Verifying Forest Configuration'{ it "Forest FQDN $($ADConfiguration.Forest.FQDN)" { $ADConfiguration.Forest.FQDN | Should be $SavedADReport.ForestInformation.RootDomain } it "ForestMode $($ADConfiguration.Forest.ForestMode)"{ $ADConfiguration.Forest.ForestMode | Should be $SavedADReport.ForestInformation.ForestMode.ToString() } } Context 'Verifying GlobalCatalogs'{ $ADConfiguration.Forest.GlobalCatalogs | ForEach-Object{ it "Server $($_) is a GlobalCatalog"{ $SavedADReport.ForestInformation.GlobalCatalogs.Contains($_) | Should be $true } } }
  20. 20. POSHSpec Describe 'Services' { Service w32time Status { Should Be Running } Service bits Status { Should Be Stopped } } Describe 'Files' { File C:inetpubwwwrootiisstart.htm { Should Exist } File C:inetpubwwwrootiisstart.htm { Should Contain 'text-align:center' } } Describe 'Registry' { Registry HKLM:SOFTWAREMicrosoftRpcClientProtocols { Should Exist } Registry HKLM:SYSTEMCurrentControlSetServicesTcpipParameters "SyncDomainWithMembership" { Should Be 1 } Registry 'HKLM:SOFTWARECallahan Auto' { Should Not Exist } } Describe 'Http' { TcpPort localhost 80 PingSucceeded { Should Be $true } TcpPort localhost 80 TcpTestSucceeded { Should Be $true } Http http://localhost StatusCode { Should Be 200 } Http http://localhost RawContent { Should Match 'X-Powered-By: ASP.NET' } Http http://localhost RawContent { Should Not Match 'X-Powered-By: Cobal' } } Describe 'Hotfix' { Hotfix KB3116900 { Should Not BeNullOrEmpty} Hotfix KB1112233 { Should BeNullOrEmpty} }
  21. 21. How do I make changes without granting long term administrative access? Does anyone need to sign-off before deployment? How do I keep services consistent across all my environments? Can I integrate service management?
  22. 22. What we did before • Request/create VM • Request/manage environmental changes • Install OS • Install app/service (or defer to app/service owner) • Onboard to patching, anti-malware, backup, monitoring • Validate app/service is working • Go Live
  23. 23. What we do now • Build runs scripts from Source • Potentially automatically releases when Tests pass • Deploy artifacts to environment • Simplify complex work through automation • Promote through stages • QA, Prod, Green/Blue
  24. 24. Getting Started Guide • Use Source Control • Don't expect to cutover instantly • Require tests • Keep options open • Old and new tools can be integrated
  25. 25. Make change visible • If change is happening fluidly, everyone needs to know • Your peers are not going to go check reports, the way they check Facebook • Notify all stakeholders in the way they collaborate Managing outages without randomizing servers
  26. 26. Map your requirements Plaster PSake Pester POSHSpec OVF PSDeploy Lability Community Projects PowerShell Slack
  27. 27. Finally Everyone must be onboard for long term success but it starts with one passionate engineer