Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

DevOps-as-a-Service: Towards Automating the Automation

3,724 views

Published on

My presentation from DevOps Enterprise Summit 2016 in San Francisco

Published in: Technology
  • Be the first to comment

DevOps-as-a-Service: Towards Automating the Automation

  1. 1. Accenture Architecture Services DevOps-as-a-Service: Towards Automating the Automation DevOps Enterprise Summit November 8, 2016 Keith Pleas DevOps Architecture Sr. Mgr. keith.pleas@Accenture.com
  2. 2. 2 Accenture’s Global DevOps Practice We support a large variety of industry standard tools – both open source and vendor/proprietary tools. 250 Clients on Accenture DevOps Platform + 100s more Clients with DevOps services 4400 DevOps Trained 1700+ DevOps experts 350 DevOps Certified 250+ DevOps Factory FTE 7 countries 15+Years heritage of high- performance CI/CD & DevOps 2000+ Members of DevOps community on Yammer sharing strategy, knowledge & collaboration 759 People deployed to AT&A Operations Domain alone 10%+ Cost reduction across IT delivery New features in hours not days DevOps Leader • Accenture developed ADOP DevOps platform (now open-sourced) • DevOps Managed Services(DCSC) • Development Lifecycle Management • Our in-house developed Cloud Platform • DevOps maturity assessment • IP/Frameworks: Methods and Tools Services • Assessment and Strategy • Application Architecture • DevOps Implementation – Continuous Integration – Continuous Delivery – Continuous Testing • Organizational Change • IT operating model transformation • Managed Service Components • Build & deployment automation • Software configuration management • Test automation • Environment management & monitoring • Cloud environment provisioning • Development & deployment architecture Key principles • Test early and often • Improve continuously • Automate everything • Cohesive teams • Strong source control • Deliver small increments • Experiment frequently without regret Enabling DevOps Professionals Global Expert Network Courses/Trainings DevOps Introduction • Introduction to various DevOps concepts and how to apply them • Illustrative examples/case studies DevOps Academy • Intensive 2 day classroom course • Continuous Integration and Delivery • Infrastructure as Code Multiple technology specific trainings Copyright © 2016 Accenture All rights reserved.
  3. 3. DevOps and Accenture 2009 2015201420132012201120102007 200820062005 Agile Conference Patrick Dubois & Andrew Clay Shafer Agile Infrastructure Qcon: Daniel Schauenberg Etsy 50 Deploys per day Velocity Conference John Alspaw & Paul Hammond: Flikr Deploy 10 time per day DevOps Days Belgium #DevOps Accenture DCSC Shared Service Created 2016 Copyright © 2016 Accenture All rights reserved. 3
  4. 4. The Joel Test (August 9, 2000) A friend from my distant past Copyright © 2016 Accenture All rights reserved. 4 • Do you use source control? • Can you make a build in one step? • Do you make daily builds? • Do you have a bug database? • Do you fix bugs before writing new code? • Do you have an up-to-date schedule? • Do you have a spec? • Do programmers have quiet working conditions? • Do you use the best tools money can buy? • Do you have testers? • Do new candidates write code during their interview? • Do you do hallway usability testing? www.joelonsoftware.com/articles/fog0000000043.html
  5. 5. What is DevOps? Is it culture? Copyright © 2016 Accenture All rights reserved.
  6. 6. 6 Our Challenges: Two Lenses • Technology – Automation – Software-defined Infrastructure – Automate the Automation – Software-defined DevOps – Templates, tools and stacks – Enterprise-specific concerns: licensing, security, support – Technical debt • People – Do what you do best - leverage others – Open Source culture – New paradigm for engaging Dev and Ops Copyright © 2016 Accenture All rights reserved.
  7. 7. Focus on Technology 7Copyright © 2016 Accenture All rights reserved.
  8. 8. Roger’s Curve with Moore’s “Chasm” How innovative are you? Copyright © 2016 Accenture All rights reserved. 8
  9. 9. Modern IT methodologies emphasize collaboration DevOps in context Copyright © 2016 Accenture All rights reserved. 9 IDEA PLAN DESIGN BUILD DEPLOY TEST RELEASE OPERATE DevOps Continuous Delivery Continuous Integration Execution Architecture Development Architecture Testing Strategy Ops Arch Design Thinking Lean Manufacturing Agile Development
  10. 10. 10 What about those “non-functional” requirements? Auditability, consistency, maintainability, modifiability, operability, recoverability, repeatability, resilience, reusability, robustness, scalability, security, supportability, scalability, testability, usability Copyright © 2016 Accenture All rights reserved. • How are you managing your own continuous delivery artifacts? • Are all of your build pipeline settings stored in source code control? • Can you rebuild your pipeline via automation? • Can you easily scale your pipeline? • Can you quickly onboard resources and make them productive? • Could you port your pipeline to a new build environment? • How are you managing your DevOps technical debt?
  11. 11. Our first “enabler” 11Copyright © 2016 Accenture All rights reserved.
  12. 12. 12 DevOps Maturity Model – Framework Copyright © 2016 Accenture All rights reserved. DELIVERY ORGANISATION AND METHODOLOGY • Release Frequency • Delivery Organisation • Delivery Process & Methodology • Metrics & Governance LEAN DELIVERY GOVERNANCE & PROCESSES • Governance Approach • Continuous Improvement • Product Management • Delivery & Change Control AUTOMATED RELEASE OF SOFTWARE • Source Code Version Control • Deployment Process • Deployment Artefact Management CONTINUOUS INTEGRATION • Traceability • Continuous Integration • Code Quality & Unit Test CONTINUOUS DELIVERY (INCLUDING AUTOMATED QA) • Test Strategy • Test Automation • Test Data Management • Deployment Pipelines AUTOMATED OPERATIONS • Monitoring • Organisational Considerations • Resilience SOFTWARE DEFINED INFRASTRUCTURE & CLOUD • Environments • Infrastructure Suitability • Responsiveness PLATFORM/APPLICATION ARCHITECTURE • Architecture • Impact of Component Failure • Business Enablement Architecture Methodology Lean Governance & Process Continuous Delivery Pipelines SW Defined Infrastructure Automated Release of SW Automated Operations Continuous Integration
  13. 13. 13 DevOps Maturity Model - Overview Copyright © 2016 Accenture All rights reserved. REPEATABLE CONSISTENT OPTIMISEDAD-HOC LEADING2 3 41 5 DELIVERY ORGANISATION AND METHODOLOGY Release scope poorly defined; subject to catastrophic and ad hoc change requests Releases duration exceeds business need; releases face disruptive change Release cadences well defined but exceeds business need; Requirements are stable; Release on demand, multi-speed releases; time-box meets to business need (eg: monthly) Small changes pushed through the pipeline ; Continuous deployment enables innovation LEAN DELIVERY GOVERNANCE & PROCESSES No defined or consistent applied delivery approach Delivery approach is well defined, but largely manually governed Industrialised project delivery; governance integrated into delivery and product management established Blame free culture embedded in governance that uses real-time data rather than manual inputs Continuous Delivery / Deployment supporting agile business change with automated governance AUTOMATED RELEASE OF SOFTWARE Mostly manual deployments Some CIs automated, environment tailoring required, no enterprise tools Fully automated Single-touch deployments into environments. Functioning environments can be build from nothing programmatically. Zero-touch zero- downtime deployments CONTINUOUS INTEGRATION No consistent use of version control – builds cannot be traced back to source code Source code consistently managed in VCS; releases traceable to source Developers integrate changes by checking into trunk on regular basis (daily) Build is typically green – if build breaks developers do not make other changes until resolved Build is typically green – if build breaks the CI tooling automatically reverses the failed change CONTINUOUS DELIVERY (INCL AUTOMATED QA) Fully manual test scripts Testers run a harness / suite Test harness / suite run automatically for some envs Automated test suites enforce a quality gate Tests run as functional monitoring AUTOMATED OPERATIONS No monitoring tools Tools in place, but not configured beyond basic OS checks Functional monitoring in place and DR is available Environment and application health monitored and proactively managed Service level monitoring (perf, usage) integrated with infrastructure and self-healing available SW DEFINED INFRASTRUCTURE & CLOUD Environments are managed ad-hoc without consistent blueprint Environments follow standards but are manually created Environments can be created automatically and follow common blueprint SW defined infrastructure that closely resembles Production throughout SDLC Utility based computing that leverages auto-scaling for several SDLC aspects PLATFORM/ APPLICATION ARCHITECTURE Big monolithic application architecture Architecture based on platforms, business requires IT for every change Resilient Architecture minimising single point of failures Modular and scalable architecture, that allows for some business self service configuration Microservice based architecture supported by end-to-end ownership in teams
  14. 14. 14 DevOps Maturity Model - Overview Copyright © 2016 Accenture All rights reserved. REPEATABLE CONSISTENT OPTIMISEDAD-HOC LEADING2 3 41 5
  15. 15. Our second “enabler” 15Copyright © 2016 Accenture All rights reserved.
  16. 16. Structure, Standardize, Automate The complete pipeline to operations with ADOP Copyright © 2016 Accenture All rights reserved. 16 IDE • Eclipse • X-code • … Git / Gerrit • Source code repository • Code branch management • Code review Jenkins • Continuous Integration • Multi- platform builds Code Analysis • Code quality • Coding standards rules • Compliance Deploy • Dev app store • Release repository • Environments Test • Component • Behavioural • Functional • Security • Performance Development Source Control Code Review Build Code Quality Deploy Testing A/B Test in Production Please visit github.com/Accenture for more information. Operations • Search & Analytics • Log Processing & Analysis • Visualization
  17. 17. 17 Fully automated solution for implementing ADOP Copyright © 2016 Accenture All rights reserved. https://hub.docker.com/ Availability Zone AWS region EU (Ireland) AWS AWS cloud Chef Server Public security group Public subnet Internet AWS AWS Cloud AWS CloudFormation CF template json VPC Single ADOP VM ADOP Provisioning Gerrit ELK Nexus Sonar Nginx Jenkins Platform Extensions Load Cartridge Tomcat Environment CloudInit
  18. 18. Fully distributed, highly available, auto-scaling, and self-healing ADOP platform Copyright © 2016 Accenture All rights reserved. 18
  19. 19. ADOP Cartridges Additional enablers 19Copyright © 2016 Accenture All rights reserved.
  20. 20. Dedicated versus Managed ADOP Dedicated Instance Single tenancy Supported by project or by DCSC Installed on-cloud or on client infrastructure Supports cartridge/blueprint configuration for various technologies Managed Service Multitenancy Provided as a managed service by DCSC Embeds ADLM and Innersource Costs recovered by service-charge Hosted primarily in AWS Configurable to support project requirements Clients Internal Projects Users Jira/Confluence 148 7 10,800 Innersource 69 2,820 10,600 Usage Jan 2016 20Copyright © 2016 Accenture All rights reserved.
  21. 21. Dedicated versus Managed ADOP Tools and Services Dedicated Instance On Premise (Roadmap) (Beta) Selenium AOWP Digital/01 (Beta) (Roadmap) + Others AVS Hybrid Managed Service (Roadmap) (Beta) Compatibility (Roadmap) 21Copyright © 2016 Accenture All rights reserved.
  22. 22. DCSC’s model • Service Catalog of pre-configured PMO tools and reporting, enabling rapid mobilization and low cost run • A number of processes benefit from a Business Process Management tool (Activiti), enabling an automated chasing and reporting • Services under continuous improvement, rolling out the latest technology to projects How to do shared services? Copyright © 2016 Accenture All rights reserved. 22 Founding principle for myPMO Services: Structure....Standardize…. Automate To reduce local team effort
  23. 23. • The spectrums: standardization, governance, processes • Consolidating and simplifying the tools portfolio & licensing • Maturity assessments and coaching clients • Experiment with ways to obtain better feedback from clients e.g. periodic surveys, feedback channels, user groups, social media • Geo-location • Metrics • Security, including VPN tunnels across trust boundaries – Detection (e.g. Security Monkey) rather than imposing restriction What are the tradeoffs with shared-services? Copyright © 2016 Accenture All rights reserved. 23
  24. 24. • Proof of concepts and MVPs • Onboarding and training • Starting point for your own DevOps stack • Starting point for your own shared services How might you use ADOP? Copyright © 2016 Accenture All rights reserved. 24
  25. 25. Focus on People 25Copyright © 2016 Accenture All rights reserved.
  26. 26. 26 Just one example from Jenkins World… Copyright © 2016 Accenture All rights reserved.
  27. 27. Benefits of a shared service model • Re-use of assets between teams • Sharing of knowledge between teams • Flexible workloads e.g. person X from project A spends and afternoon helping person Y on project B when needed • Creating strong culture that fosters shared values and creates a great and meaningful place to work • Being able to have our own culture and identity that we can control Focus on the People Copyright © 2016 Accenture All rights reserved. 27
  28. 28. Lessons learned from a decade of DCSC • Avoiding the HiPPO (Highest paid person’s opinion) • Specialization, cross-training, silos, and scalability • Trust versus Empowerment – johnclapham.wordpress.com/2016/04/22/inflicting-trust/ • Relationships and staying connected with a lack of face to face • Ensure internal product teams don’t end up in a silo and lose touch • Flexibility for mobility of individuals between roles and meet their needs balanced with staying on engagement through realization Focus on the People Copyright © 2016 Accenture All rights reserved. 28
  29. 29. Where do we go from here? 29Copyright © 2016 Accenture All rights reserved.
  30. 30. • Ever-increasing complexity • Work harder, work smarter, or get help • Structure, standardize, automate Evolution: Fact or Theory? Copyright © 2016 Accenture All rights reserved. 30
  31. 31. Large media client Complex DevOps tools landscape Copyright © 2016 Accenture All rights reserved. 31 Log Aggregation and Op. Dashboards Programme and Project Management Configuration Management Test Automation Requirements Management Functional Testing Continuous Integration Continuous Delivery Artefact Management Source Control Change Management Defect Management Continuous Integration/Build Build Orchestration JenkinsJenkins Risks and Issues Continuous Integration Build Deployment Orchestration Jenkins Code Validation Functional and API Tests Load Injection Design DocumentationTeam Dashboards MI and Reporting Security Tests Travis CI AWS S3 Github Jenkins Slack and Lita Zap Charles Box Composer BlazemeterPostmanSelenium Team Communications Slack Netsparker Development and Prototyping
  32. 32. Let’s look at those NFRs again… • Versioning, packaging, dependencies • Chef and Puppet had a head start and showed us the way • Cloud Formation templates • Security settings • Certificates • …. Where are our DevOps artifacts? Copyright © 2016 Accenture All rights reserved. 32 Auditability, consistency, maintainability, modifiability, operability, recoverability, repeatability, resilience, reusability, robustness, scalability, security, supportability, scalability, testability, usability
  33. 33. In general the same way • Repositories • DSLs (Domain-Specific-Languages) – Example: Gradle’s Groovy-based DSL for builds • Jenkins “Pipeline as Code”- jenkins.io/doc/book/pipeline-as-code/ • A new opportunity for automated testing How do we manage these? Copyright © 2016 Accenture All rights reserved. 33
  34. 34. Ah yes, software factories…did that end well? “Why I Hate Frameworks” – Benji Smith, September 30, 2005 • A hammer • A universal hammer • A hammer factory • A hammer-factory-building factory • A general-purpose tool-building factory factory factory Modification - the fatal flaw of MDA and software factories and… Automating DevOps Copyright © 2016 Accenture All rights reserved. 34discuss.joelonsoftware.com/?joel.3.219431.12 34
  35. 35. Putting it all together 35Copyright © 2016 Accenture All rights reserved.
  36. 36. • Technology – Do you have standards for your DevOps practices? – Do you rationalize your DevOps tools? – Are you considering your DevOps technical debt? • People – Are you attracting or building DevOps talent? – Are you making the best use of the talent you have? Does DevOps-as-a-Service make sense to you? Questions to consider Copyright © 2016 Accenture All rights reserved. 36
  37. 37. • www.Accenture.com/DevOps • ADOP Wiki and quickstart: accenture.github.io/adop-docker-compose/docs/quickstart/ • Mark Rendell DOES15 - www.youtube.com/watch?v=8WRRi6oui34 • “The Joel Test” - www.joelonsoftware.com/articles/fog0000000043.html • “Why I hate frameworks” - discuss.joelonsoftware.com/?joel.3.219431.12 • DevOps Maturity Model- DevOps@accenture.com • John Clapham “Inflicting Trust”: johnclapham.wordpress.com/2016/04/22/inflicting-trust/ • Accenture DevOps blog: www.accenture.com/us-en/blogs/blogs-accenture-devops • Mark Rendell (DCSC) blog: markosrendell.wordpress.com • Mirco Hering (Maturity Model) blog: notafactoryanymore.com Resources Copyright © 2016 Accenture All rights reserved. 37

×