Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Nelson
automated, multi-region container deployment
Hello.
!
verizon.github.io/nelson/
Problem
• Provisioning applications is still too slow (bare metal or cloud).
• Runtime traffic control systems are medieval...
Lessons
• Automate every part of the system.
• Testing a distributed system locally is a fable.
• Emergent properties. Sca...
Goals
• Use the minimally powerful components.
• System elements should be awesome at just one thing.
• Reduce overall pla...
Goals
• All application specifications are checked in.
• Build. Deployment. Alerting etc.
• Reduce deployment time 2 minute...
Build Better.
Nomad.
Nomad
• Use a farm of servers as a single resource pool: RAM, CPU, etc
• Typically used at larger scale, becoming more com...
Envoy.
Envoy
• Fast L4 and L7 proxy solving many practical ops concerns.
• Open-sourced end of 2016; blossomed since.
• Lyft, Goo...
Nelson.
– Vice Admiral Horatio Nelson, 1758-1805
“Desperate affairs require desperate remedies.”
– Vice Admiral Horatio Nelson, 1758-1805
“Desperate affairs require desperate remedies.”
#opslife
Overview
• Github driven developer workflow (.com or enterprise).
• Choose whatever build / CI system you want.
• State of ...
Lifecycle.
Deployment is the easy part.
based on

consul
typical state
user 

activated
pending

GC
pluggable
borrowed

time
garbage

collection
Graph 

Pruning
X
X
Upgraded!
last two

major revsXX X
last two

featuresXX X
Namespaces.
machines
scheduler
namespaces
namespaces
entirely 

virtual!
root 

namespace
qa/unstable
qa/staging/tim
Discovery & Routing.
Discovery.
• Discovery protocol written to Consul KV for every stack
• We call this Lighthouse protocol
• Application depe...
Routing.
• Non-prescriptive approach to routing tier implementation.
• Provides a control plane protocol describe routing ...
http://timperrett.com/2017/05/13/nomad-with-envoy-and-consul
embeded 

envoy
http://timperrett.com/2017/05/13/nomad-with-envoy-and-consul
sidecar 

envoy
http://timperrett.com/2017/05/13/nomad-with-envoy-and-consul
host-based 
envoy
Challenges
• Non-trivial level of investment and execution.
• Tight integration with Hashistack is both pro or con.
• Cont...
Future Work
• Aim to open-source supporting and complimentary tools.
• Consul / Envoy integration. Cost analysis subsystem...
Summary
• Fully automated application lifecycle: no manual housekeeping.
• Choose whatever CI setup best fits your team.
• ...
EOF
timperrett
verizon.github.io/nelson/
Nelson: Automated multi-region container deployment
Nelson: Automated multi-region container deployment
Nelson: Automated multi-region container deployment
Nelson: Automated multi-region container deployment
Nelson: Automated multi-region container deployment
Upcoming SlideShare
Loading in …5
×

Nelson: Automated multi-region container deployment

1,822 views

Published on

Hashicorp Nomad and Lyft Envoy have been brought together as fundamental building blocks for Nelson, a new open source project from Verizon Labs that enables blazing fast continuous delivery, fully automated lifecycle and cleanup for applications, along with dynamic runtime traffic management. Automatically shift your production traffic back and forth between different versions of your immutable service infrastructure without any outages. Encrypt all your traffic transparently and provide best-in-class security without your engineering staff having to learn how your infrastructure and security sausages are made.

Published in: Internet
  • Just got my check for $500, Sometimes people don't believe me when I tell them about how much you can make taking paid surveys online... So I took a video of myself actually getting paid $500 for paid surveys to finally set the record straight. I'm not going to leave this video up for long, so check it out now before I take it down!  http://ishbv.com/surveys6/pdf
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • I'll share this on Twitter...GET FUNDING NOW...Instantly send your startups pitchdeck to over 5700 of VC's and Angel's with just 1 click. Visit: Angelvisioninvestors.com
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here

Nelson: Automated multi-region container deployment

  1. 1. Nelson automated, multi-region container deployment
  2. 2. Hello. ! verizon.github.io/nelson/
  3. 3. Problem • Provisioning applications is still too slow (bare metal or cloud). • Runtime traffic control systems are medieval at best. • Coupling CI and CD creates monolithic operational systems. • These systems do everything. This is a distinct problem. • Current market solutions limited or hard to adopt. • Most teams have brittle, painful automation nobody wants to use. • Many teams attempt CD ignorant of the side-effects.
  4. 4. Lessons • Automate every part of the system. • Testing a distributed system locally is a fable. • Emergent properties. Scaling issues etc. • Uniformity is highly desirable and wildly advantageous. • Beautiful, unique snowflakes are however, inevitable. • Automated lifecycle management is required.
  5. 5. Goals • Use the minimally powerful components. • System elements should be awesome at just one thing. • Reduce overall platform complexity. • Increase responsibility of engineering teams. Break it, you bought it. • Decentralize process gatekeepers. • No build team. No ticket filing for deployments. No configuration management.
  6. 6. Goals • All application specifications are checked in. • Build. Deployment. Alerting etc. • Reduce deployment time 2 minutes or less. • Support multi-DC topologies from the get-go. • Automatic credential management and secure-introduction • Transparent, strong encryption for application I/O on the wire.
  7. 7. Build Better.
  8. 8. Nomad.
  9. 9. Nomad • Use a farm of servers as a single resource pool: RAM, CPU, etc • Typically used at larger scale, becoming more common. • Blazing fast: only placement without provisioning. • Integration with Vault, so secure-introduction works OOTB. • Monolithic resource manager & scheduler [1] • Several open-source & commercial alternatives: Mesos, k8s etc [1] https://research.google.com/pubs/pub43438.html
  10. 10. Envoy.
  11. 11. Envoy • Fast L4 and L7 proxy solving many practical ops concerns. • Open-sourced end of 2016; blossomed since. • Lyft, Google, IBM et al all actively contributing. • Make applications dumb; invest in a single element of routing infra • Retries, Circuit Breaking, TLS Encryption etc • Integrate horizontally, not vertically • Integrate with whatever discovery system you want via APIs.
  12. 12. Nelson.
  13. 13. – Vice Admiral Horatio Nelson, 1758-1805 “Desperate affairs require desperate remedies.”
  14. 14. – Vice Admiral Horatio Nelson, 1758-1805 “Desperate affairs require desperate remedies.” #opslife
  15. 15. Overview • Github driven developer workflow (.com or enterprise). • Choose whatever build / CI system you want. • State of the art runtime routing via Envoy. • Secure introduction for safe distribution of credentials from Vault. • Integrated with Nomad; target any datacenter running a scheduler. • Integrated alert definition with Prometheus.
  16. 16. Lifecycle. Deployment is the easy part.
  17. 17. based on
 consul
  18. 18. typical state
  19. 19. user 
 activated
  20. 20. pending
 GC
  21. 21. pluggable
  22. 22. borrowed
 time
  23. 23. garbage
 collection
  24. 24. Graph 
 Pruning
  25. 25. X X Upgraded!
  26. 26. last two
 major revsXX X
  27. 27. last two
 featuresXX X
  28. 28. Namespaces.
  29. 29. machines
  30. 30. scheduler
  31. 31. namespaces
  32. 32. namespaces entirely 
 virtual!
  33. 33. root 
 namespace
  34. 34. qa/unstable
  35. 35. qa/staging/tim
  36. 36. Discovery & Routing.
  37. 37. Discovery. • Discovery protocol written to Consul KV for every stack • We call this Lighthouse protocol • Application dependencies are declared a-priori. • You cannot route to that which you do not tell Nelson about. • Makes for awesome auditing and security. • Language implementations need only consume the protocol.
  38. 38. Routing. • Non-prescriptive approach to routing tier implementation. • Provides a control plane protocol describe routing actions. • Typically implemented with Envoy, but you can choose. • Minor application changes required. • Incentivized these with tracing and context propagation. • Models traffic shifting as a time vs traffic policy curve.
  39. 39. http://timperrett.com/2017/05/13/nomad-with-envoy-and-consul embeded 
 envoy
  40. 40. http://timperrett.com/2017/05/13/nomad-with-envoy-and-consul sidecar 
 envoy
  41. 41. http://timperrett.com/2017/05/13/nomad-with-envoy-and-consul host-based 
envoy
  42. 42. Challenges • Non-trivial level of investment and execution. • Tight integration with Hashistack is both pro or con. • Containerizing legacy applications can be “interesting”. • Migration can be a challenge if not collocated with “the new world”. • Small organizations better served by existing solutions.
  43. 43. Future Work • Aim to open-source supporting and complimentary tools. • Consul / Envoy integration. Cost analysis subsystem. • Make Nelson easier to extend for third-parties • eDSL for workflows, externalize policy algebra • General “plugin” system is a possibility • Listen to the community feedback.
  44. 44. Summary • Fully automated application lifecycle: no manual housekeeping. • Choose whatever CI setup best fits your team. • Secure your deployments. • Transparent mTLS and rotating credentials. • Automatic Vault policy management. • Provide rigor to your application Death Star.
  45. 45. EOF timperrett verizon.github.io/nelson/

×