Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Pull, don’t push!
Architectures for monitoring and configuration in a
microservices era
Julian Dunn, Director of Product M...
• Modular, self-contained, pre-fabricated components
• Neighbors share components
• Complex shares services as a whole
Orchestration
An ordered set of operations
Across a set of independent machines
Connected to an orchestrator only via a
network.
Humans acting on Microsoft Visio acting on
machines
Humans acting on code acting on machines
An ordered set of operations
Defined in code
Across a set of independent machines
Connected to an orchestrator only via a
...
mylaptop:~$ ./disable-load-balancer.sh
mylaptop:~$ ssh db01 do-database-migration.sh
mylaptop:~$ for i in app01 app02; do
...
Problems with Orchestration
Resilience Scalability
Deployment Technical
Operational Cognitive
Deployment Resilience
for i in app01 app02 app03; do
do-deploy.sh –server $i
done
Deployment Resilience
for i in app01 app02 app03; do
do-deploy.sh –server $i
if $? != 0; then
failed=$i
break
end
done
# w...
Operational Resilience
Operational Resilience
Orchestration Backplane – must be up at all times!
Application Plane – delegated resilience to the ...
Operational Resilience
Orchestration Backplane
Application Plane
Orchestration Backplane
Cognitive Scalability
Cognitive Scalability
Technical Scalability
Mainframes
Time Sharing
Client/Server
Web 1.0
Web 2.0
Cloud
Internet of
Things
Edge
Time
Distributed
Centralized
The Futur...
Distributed Devices Need Distributed Management
• Adaptive
Learning
• Configuration
Updates
• Software
Updates
Distributed, Autonomous Systems
Make progress towards promised
desired state
Expose interfaces to allow others to
verify p...
The Design of Sensu
and
The Design of Habitat
The Design of Sensu vs. Traditional “Monitoring”
Nagios master
Agent
1
Agent
2
1. Poll
(orchestrate)
2. Run
checks
1. Run
...
Habitat supervisor in a nutshell
•Network-connected supervision system
•Like systemd+consul/etcd (process supervision with...
sensu-
backend
hab-sup
sensu-
backend
hab-sup
sensu-
backend
hab-sup
backend.default
sensu-
agent
hab-sup
agent.default
--...
Let’s See it in Action!
Demo: Sensu running under Habitat
• Modern architectures demand a
choreographed rather than an
orchestrated approach
• At scale, fleet management and
cognit...
Pull, don’t push: Architectures for monitoring and configuration in a microservices era
Pull, don’t push: Architectures for monitoring and configuration in a microservices era
Pull, don’t push: Architectures for monitoring and configuration in a microservices era
Pull, don’t push: Architectures for monitoring and configuration in a microservices era
Pull, don’t push: Architectures for monitoring and configuration in a microservices era
Pull, don’t push: Architectures for monitoring and configuration in a microservices era
Pull, don’t push: Architectures for monitoring and configuration in a microservices era
Upcoming SlideShare
Loading in …5
×

Pull, don’t push: Architectures for monitoring and configuration in a microservices era

142 views

Published on

Applications today are increasingly being designed using a share-nothing, microservices architecture that is resilient to the failure of individual components, even when built atop cloud infrastructure that can suffer infrequent-but-massive outages. Yet we still see many supporting tools for application monitoring, observability, configuration management and release management using a centralized “orchestration” approach that depends on pushing changes to unreliable distributed systems.

In this Sensu Summit 2018 talk, Chef's Julian Dunn & Fletcher Nichol give you a primer about promise theory and the autonomous actor model that underlies the design of products like Sensu and Habitat, why it leads to not only higher overall system reliability but human comprehension for easier operations. They argue that you should consider designing all of your applications and supporting systems in this way. They may even show a demo or two to illustrate how inverting the design radically changes the notion of “application release orchestration”, so that you can retain orchestration-type semantics even with an eventually-consistent system design.

Published in: Technology
  • Be the first to comment

  • Be the first to like this

Pull, don’t push: Architectures for monitoring and configuration in a microservices era

  1. 1. Pull, don’t push! Architectures for monitoring and configuration in a microservices era Julian Dunn, Director of Product Marketing, Chef @julian_dunn Fletcher Nichol, Senior Software Development Engineer, Chef @fnichol
  2. 2. • Modular, self-contained, pre-fabricated components • Neighbors share components • Complex shares services as a whole
  3. 3. Orchestration
  4. 4. An ordered set of operations Across a set of independent machines Connected to an orchestrator only via a network.
  5. 5. Humans acting on Microsoft Visio acting on machines Humans acting on code acting on machines
  6. 6. An ordered set of operations Defined in code Across a set of independent machines Connected to an orchestrator only via a network.
  7. 7. mylaptop:~$ ./disable-load-balancer.sh mylaptop:~$ ssh db01 do-database-migration.sh mylaptop:~$ for i in app01 app02; do > ssh $i do-deployment.sh > done mylaptop:~$ ./enable-load-balancer.sh
  8. 8. Problems with Orchestration Resilience Scalability Deployment Technical Operational Cognitive
  9. 9. Deployment Resilience for i in app01 app02 app03; do do-deploy.sh –server $i done
  10. 10. Deployment Resilience for i in app01 app02 app03; do do-deploy.sh –server $i if $? != 0; then failed=$i break end done # what goes down here? # roll back $failed? # roll back all others? # ignore it?
  11. 11. Operational Resilience
  12. 12. Operational Resilience Orchestration Backplane – must be up at all times! Application Plane – delegated resilience to the backplane
  13. 13. Operational Resilience Orchestration Backplane Application Plane Orchestration Backplane
  14. 14. Cognitive Scalability
  15. 15. Cognitive Scalability
  16. 16. Technical Scalability
  17. 17. Mainframes Time Sharing Client/Server Web 1.0 Web 2.0 Cloud Internet of Things Edge Time Distributed Centralized The Future Is Distributed
  18. 18. Distributed Devices Need Distributed Management • Adaptive Learning • Configuration Updates • Software Updates
  19. 19. Distributed, Autonomous Systems Make progress towards promised desired state Expose interfaces to allow others to verify promises Can promise to take certain behaviors in the face of failure of others
  20. 20. The Design of Sensu and The Design of Habitat
  21. 21. The Design of Sensu vs. Traditional “Monitoring” Nagios master Agent 1 Agent 2 1. Poll (orchestrate) 2. Run checks 1. Run checks Agent 1 Agent 2 Sensu Backend 2. Post data
  22. 22. Habitat supervisor in a nutshell •Network-connected supervision system •Like systemd+consul/etcd (process supervision with lifecycle hooks + shared state for reactive realtime change management) •Eventually-consistent global state using SWIM masterless (peer-to-peer) membership protocol
  23. 23. sensu- backend hab-sup sensu- backend hab-sup sensu- backend hab-sup backend.default sensu- agent hab-sup agent.default --bind sensu:backend.default Resolve symbol “sensu” in configs to properties of service group backend.default
  24. 24. Let’s See it in Action! Demo: Sensu running under Habitat
  25. 25. • Modern architectures demand a choreographed rather than an orchestrated approach • At scale, fleet management and cognitive complexity is the biggest problem • Habitat and Sensu are both examples of edge-centric, autonomous actor systems, and they work well together 😺

×