Last year we talked about DevOps, what it was, why it was important and how to get started. Boy, was it scary. Now we’re wiser. More battle-scarred. The scale of the challenge for application writers exploiting cloud and DevOps is clearer, but so is the path forward. Understanding the DevOps approach is important but equally you must understand specific deployment technologies. How to exploit them and how they effect the design of applications. Whether creating simple applications or sophisticated microservice architectures many of the challenges are the same.
Presented at JAXLondon 2015 with Steve Poole
Boost Fertility New Invention Ups Success Rates.pdf
JAXLondon 2015 "DevOps and the Cloud: All Hail the (Developer) King"
1. DevOps and the Cloud:
All Hail the Developer King
Developer Mode:
Daniel Bryant
@danielbryantuk
Steve Poole
@spoole167
2. Outline
• On the previous episode of JAX London
– The rise of DevOps and the promise of microservices
• Today
– The reality of microservices, containers and DevOps
– A modern architecture of an application
• Monolitically Complex -> modularly complex++
• It’s real, it makes sense, it’s scary
– Tooling, continual learning and befriending ops
• Your choices…
3. I (we) am the one who knocks…
Steve Poole
IBM Developer
@spoole167
Daniel Bryant
Principal Consultant,
OpenCredo
@danielbryantuk
Making Java Real Since Version 0.9
Open Source Advocate
DevOps Practitioner (whatever that means!)
Driving Change
“Biz-dev-QA-ops”
Leading change in organisations
All over Docker, Mesos, k8s, Go, Java
InfoQ, DZone, Voxxed contributor
4. 2014 -> Moving to DevOps: Easy, Hard
or Just Plain Terrifying
• Extending agility across your IT org
• From waterfall to agile, (and agile-fall in Ops)
• Breaking down the silos
• CI/CD is key (automate all the things)
• Business need to react -> DevOps and Cloud
• Recommended Ruby, Docker and Vagrant
5. This Year
• Containers are mainstream
– Orchestration platforms are
where the action is at
– And the container bolt-ons are
evolving (storage, networking)
• #DOES14 showed enterprises
are embracing DevOps
– ‘Bimodal IT’ is really a thing
https://www.flickr.com/photos/tristantaussac/
6. Part 1- Painful Lessons…
https://www.flickr.com/photos/sarahmstewart/
7. All I hear is microservices…
“In computing, microservicesis a software
architecture style in which complex
applications are composed of small,
independent processes communicating with
each other using language-agnostic APIs. These
services are small, highly decoupled and
focus on doing a small task, facilitating a
modularapproach to system-building.”
https://en.wikipedia.org/wiki/Microservices
8. Microservices
Turn applications into small,
independent, highly decoupled,
modular services
https://www.flickr.com/photos/daikrieg/
You want to make
my life more
complicated?
15. A simple upgrade or a major impact?
lost revenue or going out of business?
16. https://www.flickr.com/photos/24151087@N00/
What lessons have we learnt?
Sharing datastores sounds like it saves effort but
introduces cohesion between applications
Big-bang versioning of applications means putting
existing unchanged usecases at risk
Scaling is challenging when you try to duplicate whole
systems
Infrastructure – it’s much more important than we
realized
25. Dynamic Development
Capacity
Predefined static
VM’s LPARs etc
OpenStack
Cloud(s)
Docker
Cloud
Infrastructure as Code
Chef, Puppet, UCD …
OS
Infra On Prem Data
Centres
Cloud Providers
SoftLayer / Amazon etc
Config Containerized
Applications
Continuous Availability
Mesos etc
Deploy
Pipeline
Block Architecture of Hybrid Cloud Dev SaaSPrimary Audience
GIT / Jenkins / Junit …. Selenium, Jmeter…
✔
✔
✔
✔
✔
Application DIY
26. Dynamic Development
Capacity
Predefined static
VM’s LPARs etc
OpenStack
Cloud(s)
Docker
Cloud
Infrastructure as Code
Chef, Puppet, UCD …
OS
Infra On Prem Data Centres Cloud Providers
SoftLayer / Amazon etc
Config Containerized
Applications
Continuous Availability
Mesos etc
Deploy
Pipeline
Block Architecture of Hybrid Cloud Dev SaaSPrimary Audience
GIT / Jenkins / Junit …. Selenium, Jmeter…
Other static
hosts (‘BYOD’)
DIY
Application DIY
27. Dynamic Development
Capacity
Predefined static
VM’s LPARs etc
OpenStack
Cloud(s)
Docker
Cloud
Infrastructure as Code
Chef, Puppet, UCD …
OS
Infra On Prem Data Centres Cloud Providers
SoftLayer / Amazon etc
Config Containerized
Applications
Continuous Availability
Mesos etc
Deploy
Pipeline
Block Architecture of Hybrid Cloud Dev SaaSPrimary Audience
GIT / Jenkins / Junit …. Selenium, Jmeter…
Other static
hosts (‘BYOD’)
DIY
Compliance
/ Security ContainersVM Images
Application DIY
28. Dynamic Development
Capacity
Predefined static
VM’s LPARs etc
OpenStack
Cloud(s)
Docker
Cloud
Infrastructure as Code
Chef, Puppet, UCD …
OS
Infra On Prem Data Centres Cloud Providers
SoftLayer / Amazon etc
Config Containerized
Applications
Continuous Availability
Mesos etc
Deploy
Pipeline
Block Architecture of Hybrid Cloud Dev SaaSPrimary Audience
GIT / Jenkins / Junit …. Selenium, Jmeter…
Other static
hosts (‘BYOD’)
DIY
Compliance
/ Security ContainersVM Images
Application DIY
29. Tooling Status
• Cloud OS technology is mainstream
• As you rise up the stack the quality (and options )
vary
• Your main challenge:
– Create a robust (self-healing), secure, compliant
environment with a bag of parts that is still evolving
• Roll your own?
– We still do
31. You do understand about security and
compliance right?
https://www.flickr.com/photos/adulau/
32. Where’s your data?
Legal restrictions on data location
Vary by country even within the EU.
Different rules depending on types of data
You already know this?
But now you’re putting the data in the
cloud.
DO YOU understand where its going
Can you control / manage / audit the
situation?
Now it’s your problem.
Having fun finding tools to help
33. How’s your security knowledge?
• Again – now it’s your problem
– Your code is running in the cloud
– You created the services & the containers
– Are they secure?
– How do you test?
– Are you sure?
– Are those web services you’re buying secure?
• How much do you know about networking?
– Not enough…
• Cyber crime is big business – you will get targeted.
https://www.flickr.com/photos/61423903@N06/
34. Cybercrime
The estimated cost of Cybercrime is $100 billion per year
500 million victims per year
600,000 facebook accounts compromised per day
28% of cyber attacks are around SQL injection
Only 5% of successful attacks are against Banks, Credit, Financials
etc 35% of successful attacks are against businesses in general
50% of attacks come from hackers – just for the fun if it
60% caused losses to the business
35. Talk to your Ops team
• They are your best friends.
– They know about security and networking.
• You need to know too
– They know (some) of the answers
• It’s a whole new domain for you
– It’s not a new problem for them
• Time to learn
37. The Results of the Survey Are In…
• Puppet Labs 2015 State of DevOps
– Available: puppetlabs.com/2015-devops-report
• Accelerates deployment
– High performers 30x more deploys
– Code committed to production 200x faster
• Prevents failures and streamlines recovery
– High performers 60x fewer failures
– Recovery 168x faster
41. “Dev-on-call”
An occasional spike to the head
is a good thing...
...metaphorically speaking
• You build it, you run it
– Shared responsibility
– Communication
13/10/2015 @danielbryantuk
46. “It” Happens
• Failure happens all the time in the cloud
– Distributed systems are complex
– Container/cluster tech promotes transiency
• Get used to failing… literally
– Run ‘Gamedays’
– Disaster in Recovery Training (DiRT)
– bit.ly/1L43U4H
49. False Start, Dead, or Alive and Kicking?
• We see lots of ‘DevOps’ teams in the wild
– They often look suspiciously similar to ops teams
that are using Puppet/Chef
• Advanced companies leaning towards
– (Full stack) developers
– Platform (tooling) teams
– Site Reliability Engineers (SREs)
54. The Elephant in the Room
• Security…
• “What is Rugged All About?”
– gotocon.com/dl/goto-london-
2015/slides/JoshuaCorman_WhatIsRuggedAllAbo
ut.pdf
• “The Future of Containers”
– www.youtube.com/watch?v=_jBTHyo0mEQ
56. Be Wary of Shiny SaaS
• SaaS tools can be like crystal
– Easy to get started
– Initially you have fun
– Encourages you to attend the
wrong (root cause) parties
– Soon you’re selling Grandma
for the next (support) hit
– You eventually lose friends
(and your teeth may fall out) https://www.flickr.com/photos/perfected/
57. Key Messages
• Containers are here to stay
– You need to learn how to design applications in a new
way (‘microservices’)
• The stack is still not baked – but is getting there
• “Safety first”
– It’s a wild world out there – you must understand
about security, networking and the the rise of the
cyber criminal
• Make your ops team your best friend
58. Other Tech Tips
• Golang dominates infrastructure and tooling
• Learn about clustering and scheduling
– Mesos, Kubernetes, Nomad
– Borg, Omega, Google’s DaaC
• Network skills essential (virtualization, overlays, OSI)
• Multi-cloud appears a solid bet
– Neutrality / abstraction
– Resource managers, OpenStack, Terraform, Ansible…
Simple 1 server setup. Everyones happy – no Ops in sight
Add a new application and server setup. Everyones happy – now load balancing required etc Ops team show up.. You’re a bit more stressed
Scaling up – multiple instances of the applications , data replication, bigger load balancer – more stress
Now move some of the workload to a different location (say the cloud) more work, more cross location calls – security issues, failing systems + much more stress to simply keep It
Throw h In an application update with scheme change…