LJC: "Chuck Norris Doesn't Do DevOps...but Java developers might benefit"

3,433 views
3,270 views

Published on

We all hear the term "DevOps" being thrown around on a daily basis, but what does it actually mean? With a little help from everyone's favourite 80's action hero, we'll undergo a whistle-stop tour of the philosophy, culture and tooling behind this buzzword, specifically aimed at Java developers. We'll also look at a real-world case study from Instant Access Technologies Ltd, and explore the key role that DevOps has played during a successful upgrade of the epoints customer loyalty platform to support increasing traffic.

The core discussion will focus on the challenges encountered as we moved from a JVM-based monolithic app deployed into a data centre on a 'big bang' schedule, to a platform of loosely-coupled components, all being continuously deployed into the Cloud. We will conclude the talk by recommending a learning path for Java developers who are interested to learn more about DevOps.

Published in: Technology
0 Comments
3 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
3,433
On SlideShare
0
From Embeds
0
Number of Embeds
5
Actions
Shares
0
Downloads
17
Comments
0
Likes
3
Embeds 0
No embeds

No notes for slide
  • organizations which design systems ... are constrained to produce designs which are copies of the communication structures of these organizationsFocus on flow: translate business ideas to a working service efficiently by examining how fast artifacts and work flows through the development lifecycleAmplify feedback loops: quickly learn about the system by seeing through fast feedback on what happens to results of changes
  • James Gough’s “The benefits are more than just the tests”Mash Badar’s “TDD at Scale” (slidesha.re/19P7kzS)
  • Jackie Stewart, 3 times F1 world champion, coined the phrase “Mechanical Sympathy” as a term for the driver and the machine working together in harmony. This can be summarised in that a driver does not need to know how to build an engine but they need to know the fundamentals of how one works to get the best out of it.
  • LJC: "Chuck Norris Doesn't Do DevOps...but Java developers might benefit"

    1. 1. Chuck Norris Doesn’t Need DevOps… but Java Developers might benefit Daniel Bryant CTO, Instant Access Technologies d.bryant@iatltd.com @taidevcouk
    2. 2. @taidevcouk14/03/2014
    3. 3. epoints.com 2012/13 Upgrade… @taidevcouk14/03/2014 • Increasing traffic – Scalability being stretched • Increasingly diverse requirements • Our starting point – Developers creating monolithic application – Manual QA – Operations deploying to data center
    4. 4. Core Changes… • Service-Oriented Architecture • Cloud-based deployments • DevOps Culture @taidevcouk14/03/2014
    5. 5. Core Changes… • Service-Oriented Architecture – Twitter’s Story (bit.ly/1j1WbmI) • Cloud-based deployments – “Cloud DHARMA”, 7th May at Skillsmatter • DevOps Culture – Today! @taidevcouk14/03/2014
    6. 6. What is DevOps? • “…a software development method that stresses communication, collaboration and integration between developers and IT professionals.” • “…aims to help an organization rapidly produce software products and services” http://en.wikipedia.org/wiki/DevOps 14/03/2014 @taidevcouk
    7. 7. What’s in a Name? • “Development / Operations” • Increasing cohesion between: – Business – Development – Quality Assurance – Operations “Bu-Dev-Qa-Ops”? 14/03/2014 @taidevcouk
    8. 8. yeah…‘DevOps’ sounds way cooler… 14/03/2014 @taidevcouk
    9. 9. Two Sides of the Same ‘DevOps’ Coin… • Business-driven – ‘Creating organizational alignment’ – Value stream mapping – Waste analysis • Pragmatic Tech – Pushing Agile/XP into QA and Ops – Cross-functional empowerment – Tooling 14/03/2014 @taidevcouk
    10. 10. So, What’s Chuck Norris Doing Here? 14/03/2014 @taidevcouk
    11. 11. Chuck Norris doesn’t need DevOps… …as a one-man army he codes with one hand, tests with the other and deploys with his beard 14/03/2014 @taidevcouk The rest of us… …work in teams to develop software
    12. 12. Software Development Teams • Developers • Quality Assurance • Operations @taidevcouk14/03/2014
    13. 13. Software Development Teams • Developers • Quality Assurance • Operations @taidevcouk14/03/2014
    14. 14. @taidevcouk14/03/2014
    15. 15. Software Development Teams • Developers • Quality Assurance • Operations @taidevcouk14/03/2014
    16. 16. @taidevcouk14/03/2014
    17. 17. Software Development Teams • Developers • Quality Assurance • Operations @taidevcouk14/03/2014
    18. 18. @taidevcouk14/03/2014
    19. 19. This silo mentality has to stop 14/03/2014 @taidevcouk
    20. 20. Culture is Vital • Culture drives behaviour, drives culture… – “Communication, simplicity, feedback, courage” – Everyone is responsible for delivery – Continuous experimentation and learning • Not easy to change culture – The hardest part of DevOps… – …but you will learn new things 14/03/2014 @taidevcouk
    21. 21. Changing Culture • Greenfield – Flickr’s story (slidesha.re/sHpYV) – “Why other people don’t get it” • Sandro Mancuso (slidesha.re/1bcStpe) • Enterprise – “The Pheonix Project” • Gene Kim et al 14/03/2014 @taidevcouk
    22. 22. Changing Culture • Create an effective team… • “Habits of highly effective technical teams” – Martijn Verburg (bit.ly/1aF9SnK) • “Patterns of Effective Teams” – Dan North (vimeo.com/68226771) 14/03/2014 @taidevcouk
    23. 23. Chuck Norris doesn’t do iterative development… …all applications Chuck Norris creates are right first time, every time 14/03/2014 @taidevcouk The rest of us… …need to enable agility (for both the business and technical teams)
    24. 24. We all do Continuous Integration, right? • Continuous Deployment/Delivery – Jez Humble and Dave Farley – Great InfoQ Video (bit.ly/XugWi8) • Create a “build pipeline” – Goal is fast feedback • Continuous Deployment – Production? 14/03/2014 @taidevcouk
    25. 25. Our Build Pipeline Jenkins, with plugins… • Build Pipeline – wiki.jenkins-ci.org/display/JENKINS/Build+Pipeline+Plugin • Parameterized build – wiki.jenkins-ci.org/display/JENKINS/Parameterized+Build • Promoted Builds Plugin – wiki.jenkins-ci.org/display/JENKINS/Promoted+Builds+Plugin • Chuck Norris plugin… – wiki.jenkins-ci.org/display/JENKINS/ChuckNorris+Plugin 14/03/2014 @taidevcouk
    26. 26. Our Build Pipeline • Component Build – Compile – Unit Tests (surefire) – Integration Tests (failsafe) • Deployment onto QA Cloud – Python Scripts + Chef to provision – Verify success using Python 14/03/2014 @taidevcouk
    27. 27. Our Build Pipeline • Acceptance Tests – Cucumber and Selenium – Work in progress… • Performance Tests – Jmeter – Jenkins Jmeter performance plugin • Staging / Live Deployment – Human-based conditional operation 14/03/2014 @taidevcouk
    28. 28. Gotchas • Managing dependencies in SOA is hard, very hard • Branching – Gitflow, Branch Per Feature, or Mainline • Migrating data can be challenging – Liquibase / Flyway – MongoDB / Solr Schema versions in data – Wooga case study (bit.ly/1egArDC) – Shout out to Daniel Josefsson @ Shazam for tips! 14/03/2014 @taidevcouk
    29. 29. Chuck Norris doesn’t do QA… …Chuck Norris can test an entire application with a single assert (and get 110% code coverage) 14/03/2014 @taidevcouk The rest of us… …need high-quality automated QA
    30. 30. Automating QA • Unit testing is essential • Intra-component integration testing – Spock is awesome (code.google.com/p/spock) – Utilise embedded datastore/middleware • Inter-component integration testing – The hardest part of SOA… 14/03/2014 @taidevcouk
    31. 31. Automating QA • Consumer-based Testing – Brandon Byars (bit.ly/1lmcoaD) • Performance – JMeter + plugins (jmeter-plugins.org) – BlazeMeter • Website – Selenium (www.seleniumhq.org) – Geb (www.gebish.org) 14/03/2014 @taidevcouk
    32. 32. Automating QA • Make it easy for everyone to execute • Include within the build pipeline • Make people care – fail the build! • “Agile Testing” by Lisa Crispin, Janet Gregory 14/03/2014 @taidevcouk
    33. 33. Chuck Norris doesn’t need an OS… …his keyboard has two keys, 0 and 1 14/03/2014 @taidevcouk The rest of us… …need to provision bare metal, and also be comfortable with the OS
    34. 34. Say No To Snowflakes! • Infrastructure as Code – Version control everything • Automate all provisioning – Chef, Puppet, SaltStack, Python, AWS CLI • Play with Vagrant (www.vagrantup.com) – “providers” are super cool 14/03/2014 @taidevcouk
    35. 35. Thinking/Acting Like A Sysadmin • Learn Linux fundamentals • Diagnostic skills – top, iotop, iostat, netstat, vmstat – Java utils: jps, jstat, jmap, jhat – “DevOps Troubleshooting” by K. Rankin • Maybe grow a beard… 14/03/2014 @taidevcouk
    36. 36. Chuck Norris doesn’t fail… …he just finds a new way in which reality is broken 14/03/2014 @taidevcouk The rest of us… …should plan for failure
    37. 37. Failure “Everything fails all the time [in the cloud]” Werner Vogels, CTO, Amazon.com • 21st Century Application Architecture – www.skillsmatter.com (bit.ly/10jAdSV) 14/03/2014 @taidevcouk
    38. 38. Black Swan Theory “…is a metaphor that describes an event that comes as a surprise, has a major effect, and is often inappropriately rationalized after the fact with the benefit of hindsight” Nassim Nicholas Taleb, The Black Swan http://amzn.to/1gaIqxz 14/03/2014 @taidevcouk
    39. 39. Failure Pre-Cloud we designed for success… and handled failure. With Cloud we design for failure… and handle success. Credit to John Bassil for inspiration here! (@johnbassil) 14/03/2014 @taidevcouk
    40. 40. Design for Failure • Design patterns – Asynchronous communication – Timeouts / retries – Bulkheads / circuit-breakers • Inspiration – Chris Richardson (slidesha.re/1ft3vsg) – Netflix (bit.ly/1h5GMid) 14/03/2014 @taidevcouk
    41. 41. 14/03/2014 @taidevcouk
    42. 42. Antifragile • The opposite of fragile? – Robust… – Antifragile… • Netflix are best-in-class – bit.ly/1gs5n3q • System must be robust first! 14/03/2014 @taidevcouk
    43. 43. All arrays Chuck Norris creates are of infinite size… …as Chuck Norris knows no bounds 14/03/2014 @taidevcouk The rest of us… …should manage our resources and cultivate ‘mechanical sympathy’
    44. 44. Mechanical Sympathy • Be aware of deployment platform properties • …especially if in the cloud – Block Storage IOPS (100 vs 49k) – 1Gbps Network ( < 125MB/s vs > 400MB/s) – “Noisy neighbours” • Monitor everything (more on this later) 14/03/2014 @taidevcouk
    45. 45. When Chuck Norris throws Exceptions… …everybody knows about it because they land outside of the data center 14/03/2014 @taidevcouk The rest of us… …should log all errors (and other vital information for diagnostic purposes)
    46. 46. Logging • Log pretty much everything – Use appropriate levels • Make comments searchable/machine readable – Good tips (bit.ly/hweqm4) • Use centralised logging – Especially if in the cloud – Logstash, Loggly, Papertrail 14/03/2014 @taidevcouk
    47. 47. Chuck Norris doesn’t worry about application downtime… …Chuck Norris’ production servers are so scared they constantly ping him 14/03/2014 @taidevcouk The rest of us… …should monitor everything
    48. 48. Monitor All The Things! • Infrastructure monitoring – Nagios – Zabbix – Splunk – AppDynamics 14/03/2014 @taidevcouk
    49. 49. Metrics 14/03/2014 @taidevcouk • Dropwizard’s Metrics – metrics.codahale.com • Netflix’s Servo – github.com/Netflix/servo • Etsy’s StatsD – github.com/etsy/statsd/wiki
    50. 50. Health Checks 14/03/2014 @taidevcouk
    51. 51. Gauges, Counters, Meters, Timers… 14/03/2014 @taidevcouk
    52. 52. 14/03/2014 @taidevcouk
    53. 53. 14/03/2014 @taidevcouk
    54. 54. Inspirational Metrics Companies • Several trail-blazers – Etsy (www.codeascraft.com) – Netflix (techblog.netflix.com) 14/03/2014 @taidevcouk
    55. 55. In Summary… • DevOps is driving Agile/XP into QA and Ops • Faster, leaner and more effective software – The ability to rapidly experiment is awesome! • There are some real benefits behind the buzz • Now is the time to step-up as a developer 14/03/2014 @taidevcouk
    56. 56. The Developer’s DevOps Action Plan • Think about your company culture • Explore continuous delivery • Learn Linux basics • Automate provisioning • Design for failure • Cultivate mechanical sympathy • Improve logging/metrics 14/03/2014 @taidevcouk
    57. 57. Thanks For Listening • Massive thanks to all the IAT team – DevOps gurus, Jamie Clarkson and Anjani Phuyal • Questions / comments? – d.bryant@iatltd.com – @taidevcouk • Devoxx UK 12-13th June – www.devoxx.co.uk 14/03/2014 @taidevcouk

    ×