Slides from my talk at GOTOpia November 2020 about the right balance between choreography (event-driven communication) and orchestration (command-driven communication)
Monitoring and Orchestration of your Microservices Landscape with Kafka and Z...Bernd Ruecker
Slides from a joined meetup of Confluent/Kafka and Camunda/Zeebe in October 2019.
Code is here: https://github.com/berndruecker/flowing-retail/tree/master/kafka
Slides from my talk at QCon London on March 3rd 2020. Abstract: Integrating microservices or other components is hard, as it involves taming distributed systems. New API technologies are great, but can't magically solve all underlying challenges. This talk distills real-life experiences around typical architecture patterns. You will understand why you have to carefully think about boundaries and responsibilities of all your components. Further you will see why balancing orchestration and choreography is essential to avoid chaos. We also need to talk about idempotency, long-running and event-driven services. Don’t worry if you are new here, I will use easy to understand examples. In the end you will have gained a better feeling how to make your API smarter.
Complex event flows in distributed systems (QCon London 2019)Bernd Ruecker
Slides from my talk at QCon London on 5th of March 2019. More information can be found here: https://berndruecker.io/complex-event-flows-in-distributed-systems/
Abstract: Event-driven architectures enable nicely decoupled microservices and are fundamental for decentral data management. However, using peer-to-peer event chains to implement complex end-to-end logic crossing service boundaries can accidentally increase coupling. Extracting such business logic into dedicated services reduces coupling and allows to keep sight of larger-scale flows - without violating bounded contexts, harming service autonomy or introducing god services. Service boundaries get clearer and service APIs get smarter by focusing on their potentially long running nature. I will demonstrate how the new generation of lightweight and highly-scalable state machines ease the implementation of long running services. Based on my real-life experiences, I will share how to handle complex logic and flows which require proper reactions on failures, timeouts and compensating actions and provide guidance backed by code examples to illustrate alternative approaches.
Monitoring and Orchestration of your Microservices Landscape with Kafka and Z...Bernd Ruecker
Slides from a joined meetup of Confluent/Kafka and Camunda/Zeebe in October 2019.
Code is here: https://github.com/berndruecker/flowing-retail/tree/master/kafka
Slides from my talk at QCon London on March 3rd 2020. Abstract: Integrating microservices or other components is hard, as it involves taming distributed systems. New API technologies are great, but can't magically solve all underlying challenges. This talk distills real-life experiences around typical architecture patterns. You will understand why you have to carefully think about boundaries and responsibilities of all your components. Further you will see why balancing orchestration and choreography is essential to avoid chaos. We also need to talk about idempotency, long-running and event-driven services. Don’t worry if you are new here, I will use easy to understand examples. In the end you will have gained a better feeling how to make your API smarter.
Complex event flows in distributed systems (QCon London 2019)Bernd Ruecker
Slides from my talk at QCon London on 5th of March 2019. More information can be found here: https://berndruecker.io/complex-event-flows-in-distributed-systems/
Abstract: Event-driven architectures enable nicely decoupled microservices and are fundamental for decentral data management. However, using peer-to-peer event chains to implement complex end-to-end logic crossing service boundaries can accidentally increase coupling. Extracting such business logic into dedicated services reduces coupling and allows to keep sight of larger-scale flows - without violating bounded contexts, harming service autonomy or introducing god services. Service boundaries get clearer and service APIs get smarter by focusing on their potentially long running nature. I will demonstrate how the new generation of lightweight and highly-scalable state machines ease the implementation of long running services. Based on my real-life experiences, I will share how to handle complex logic and flows which require proper reactions on failures, timeouts and compensating actions and provide guidance backed by code examples to illustrate alternative approaches.
Kafka Summit 2020: If an event is published to a topic and no one is around t...Bernd Ruecker
Slides from my talk "If an event is published to a topic and no one is around to consume it, does it make a sound?" at Kafka Summit Live in August 2020.
See recording here: https://www.confluent.de/resources/kafka-summit-2020/if-an-event-is-published-to-a-topic-and-no-one-is-around-to-consume-it-does-it-make-a-sound/
Jfokus 2018: Lost in transaction - Strategies to deal with (in-)consistency ...Bernd Ruecker
Slides from my talk at Jfokus 2018 about how to avoid getting lost in transaction. Code is here: https://github.com/flowing/flowing-retail and https://github.com/flowing/flowing-retail/tree/master/payment-rest.
Serverless Days 2019 - Lost in transactionBernd Ruecker
Slides from my talk at Serverless Days Hamburg 2019 about how to avoid getting lost in transaction, how to coordinate multiple functions and how Sagas might help.
Source code is here: https://github.com/berndruecker/trip-booking-saga-serverless
Java User Group Erfurt 2018: Zeebe.io - Event-driven Microservice OrchestrationBernd Ruecker
Slides from my talk at JUG Erfurt on 22nd or march 2018.
Sample code available: https://github.com/flowing/flowing-retail/tree/zeebe
Recording in German available: https://youtu.be/lZIe02um5eI
Camunda Con Live 2020 Keynote - Microservice Orchestration and IntegrationBernd Ruecker
Slides from my talk at Camunda Con Live on 24th of April 2020 about orchestrating and integrating microservices and the connection of choreography, observability and workflow automation
O'Reilly SA NYC 2018: Complex event flows in distributed systemsBernd Ruecker
Slides from my talk at O'Reilly Software Architecture Conference in New York City on 28th of February 2018. Code from the demo is available: http://github.com/flowing/flowing-retail
Camunda Day New York 2019: Workflow Automation in Microservices Architectures...camunda services GmbH
Many Camunda users start to adopt microservices. In this presentation Bernd Rücker, Co-founder of Camunda, shares experiences and best practices on how to apply BPMN and the Camunda platform in your microservices architecture. `He’ll tackle questions around how you can slice end-to-end business processes into appropriate pieces, how many engines you should operate and how to keep in control.
Kafka Summit 2020: If an event is published to a topic and no one is around t...Bernd Ruecker
Slides from my talk "If an event is published to a topic and no one is around to consume it, does it make a sound?" at Kafka Summit Live in August 2020.
See recording here: https://www.confluent.de/resources/kafka-summit-2020/if-an-event-is-published-to-a-topic-and-no-one-is-around-to-consume-it-does-it-make-a-sound/
Jfokus 2018: Lost in transaction - Strategies to deal with (in-)consistency ...Bernd Ruecker
Slides from my talk at Jfokus 2018 about how to avoid getting lost in transaction. Code is here: https://github.com/flowing/flowing-retail and https://github.com/flowing/flowing-retail/tree/master/payment-rest.
Serverless Days 2019 - Lost in transactionBernd Ruecker
Slides from my talk at Serverless Days Hamburg 2019 about how to avoid getting lost in transaction, how to coordinate multiple functions and how Sagas might help.
Source code is here: https://github.com/berndruecker/trip-booking-saga-serverless
Java User Group Erfurt 2018: Zeebe.io - Event-driven Microservice OrchestrationBernd Ruecker
Slides from my talk at JUG Erfurt on 22nd or march 2018.
Sample code available: https://github.com/flowing/flowing-retail/tree/zeebe
Recording in German available: https://youtu.be/lZIe02um5eI
Camunda Con Live 2020 Keynote - Microservice Orchestration and IntegrationBernd Ruecker
Slides from my talk at Camunda Con Live on 24th of April 2020 about orchestrating and integrating microservices and the connection of choreography, observability and workflow automation
O'Reilly SA NYC 2018: Complex event flows in distributed systemsBernd Ruecker
Slides from my talk at O'Reilly Software Architecture Conference in New York City on 28th of February 2018. Code from the demo is available: http://github.com/flowing/flowing-retail
Camunda Day New York 2019: Workflow Automation in Microservices Architectures...camunda services GmbH
Many Camunda users start to adopt microservices. In this presentation Bernd Rücker, Co-founder of Camunda, shares experiences and best practices on how to apply BPMN and the Camunda platform in your microservices architecture. `He’ll tackle questions around how you can slice end-to-end business processes into appropriate pieces, how many engines you should operate and how to keep in control.
Process automation technology progressed massively over the last years and can handle modern scalability and resiliency requirements. It is not the dinosaur some people think it is!
Now there are super interesting combinations between Kafka and a workflow engine possible, such as starting or influencing process instances upon certain events in a stream, orchestrating microservices by writing commands as Kafka records, or using Kafka to collect information about decentralized process executions.
In this talk, I will briefly walk you through these possibilities presenting real-life use cases and some sample code using open source tools.
We’re slicing and dicing systems into ever-shrinking pieces like microservices or serverless functions. End-to-end processes now often require multiple components to be properly integrated - but of course, without coupling them too tightly. We see many companies slicing up their core business processes in the pursuit of modern architectures and running into unanticipated challenges along the way. Spoiler alert: there’s no magic fix.
This keynote will foster your understanding of how workflows can be implemented and monitored in typical microservices architectures. You will gain a better sense of the positives and negatives of different approaches, and I will take you on a journey into how to migrate existing legacy systems to this exciting new (and decoupled) world.
SVCC Developing Asynchronous, Message-Driven Microservices Chris Richardson
The microservice architecture functionally decomposes an application into a set of services. Each service has its own private database that’s only accessible indirectly through the services API. Consequently, implementing queries and transactions that span multiple services is challenging.
In this presentation, you will learn how to solve these distributed data management challenges using asynchronous messaging. I describe how to implement transactions using sagas, which are sequences of local transactions, coordinated using messages. You will learn how to implement queries using Command Query Responsibility Segregation (CQRS), which uses events to maintain replicas. I describe the key role that messaging plays a microservice architecture.
Given at Silicon Valley Code Camp 2018
Using event sourcing (backed by Azure table storage) as a persistence mechanism to store durable state for azure functions and using Event Grid to communicate between them
Complex Event Flows in Distributed Systems (Bernd Ruecker, Camunda) Kafka Sum...confluent
Event-driven architectures enable nicely decoupled microservices and are fundamental for decentral data management. However, using peer-to-peer event chains to implement complex end-to-end logic crossing service boundaries can accidentally increase coupling. Extracting such business logic into dedicated services reduces coupling and allows to keep sight of larger-scale flows – without violating bounded contexts, harming service autonomy or introducing god services. Service boundaries get clearer and service APIs get smarter by focusing on their potentially long running nature. I will demonstrate how the new generation of lightweight and highly-scalable state machines ease the implementation of long running services. Based on my real-life experiences, I will share how to handle complex logic and flows which require proper reactions on failures, timeouts and compensating actions and provide guidance backed by code examples to illustrate alternative approaches.
#hacksummit 2016 - event-driven microservices – Events on the outside, on the...Chris Richardson
This is my presentation from #hacksummit 2016
This presentation looks at the importance of events and the role that they play in applications. We describe how events are a key application integration mechanism and how they are used by applications to communicate with the outside world. You will learn how the microservices inside a system can use events to maintain data consistency. We discuss how easy it is to implement both of these mechanisms by developing your core business logic using an event-centric approach known as event sourcing.
How Events Are Reshaping Modern SystemsJonas Bonér
Event-driven architecture and design have been getting a lot of attention in recent years. It’s an old concept that has been around for decades, so why this sudden peak of interest?
In this talk, we will explore the nature of events, what it means to be event-driven, and how we can unleash the power of events. The goal is to arm you with a solid theoretical understanding of how to design an event-driven system, what tools and techniques you can use to reap the most benefit from its design, and perhaps most importantly, what to avoid.
We'll discuss how an event-driven design can help:
- drive autonomy
- reduce risk
- increase certainty
- increase loose coupling
- increase scalability
- increase resilience
- increase traceability
Skeptics should definitely attend.
Audio from this sessions available at http://bit.ly/angelhacksf_paypal_audio - December 1st, 2012 presentation on PayPal products at AngelHack Bay Area (PayPal HQ, San Jose, CA)
The internet of things requires a different architectural model than what we've used to build Web 2.0. This presentation makes a proposal for what that architecture could look like and presents a working example based on the connected car platform Fuse (http://joinfuse.com)
Developing event-driven microservices with event sourcing and CQRS (svcc, sv...Chris Richardson
Modern, cloud-native applications typically use a microservices architecture in conjunction with NoSQL and/or sharded relational databases. However, in order to successfully use this approach you need to solve some distributed data management problems including how to maintain consistency between multiple databases without using 2PC.
In this talk you will learn more about these issues and how to solve them by using an event-driven architecture. We will describe how event sourcing and Command Query Responsibility Segregation (CQRS) are a great way to realize an event-driven architecture. You will learn about a simple yet powerful approach for building, modern, scalable applications.
Mucon: Not Just Events: Developing Asynchronous MicroservicesChris Richardson
The microservice architecture functionally decomposes an application into a set of services. Each service has its own private database that’s only accessible indirectly through the services API. Consequently, implementing queries and transactions that span multiple services is challenging. In this presentation, you will learn how to solve these distributed data management challenges using asynchronous messaging. Chris will share with you how to implement transactions using sagas, which are sequences of local transactions. You will learn how to coordinate sagas using either events or command messages. Chris will also explore how to implement queries using Command Query Responsibility Segregation (CQRS), which uses events to maintain easily queried replicas.
Event sourcing dates back hundreds of years. This is how Functional Programmers think about event sourcing.
I love comments and questions. @britishdan / danielc@wix.com
JUG Frankfurt - Orchestration of microservicesBernd Ruecker
Slides from my talk held at JUG Frankfurt on 27th of September 2017 about microservice orchestration. Source code for the live hacking is here: https://github.com/flowing/flowing-retail
Developing functional domain models with event sourcing (oakjug, sfscala)Chris Richardson
Event sourcing persists each entity as a sequence of state changing events. An entity’s current state is derived by replaying those events. Event sourcing is a great way to implement event-driven microservices. When one service updates an entity, the new events are consumed by other services, which then update their own state.
In this talk we describe how to implement business logic using event sourcing. You will learn how to write functional, immutable domain models in Scala. We will compare and contrast a hybrid OO/FP design with a purely functional approach.
Similar to GOTOpia 2020 - Balancing Choreography and Orchestration (20)
WeAreDevelopers Live 2024 - Mastering long-running processes in modern archit...Bernd Ruecker
Slides from my talk at WeAreDevelopers Live on the Java Day about long-running processes. Code can be found here: https://github.com/berndruecker/customer-onboarding-camunda-8-springboot
CraftConf: Surviving the hyperautomation low code bubblBernd Ruecker
Slides from my talk at CraftConf Budapest in May 2023 about how developers can embrace and shape low code applications in their organizations to help the business automate more
Slides from my talk "Microservices Architectures" at JAX Mainz on 3rd of May 2022 (https://jax.de/software-architecture/understanding-communication-patterns-in-microservices-architectures/)
Webinar "Communication Between Loosely Coupled Microservices"Bernd Ruecker
Slides from the Camunda webinar "Communication Between Loosely Coupled Microservices" in February 2021. Recording can be found online: https://page.camunda.com/wb-communication-between-microservices
CamundaCon 2020 Keynote - The Return of Process AutomationBernd Ruecker
Slides from my keynote at CamundaCon Live 2020.2
Title: The Return of Process Automation!
Abstract: This keynote will foster your understanding of how (business) processes can generally be implemented and monitored. I will compare different approaches, from batches over streaming, to workflow engines. You will understand the impact on agility and what is different in modern architectures, as well as learning about choreography and orchestration. You will learn about criteria that have crystalized as success factors in many real-life scenarios.
You will also understand the failures of BPM and process automation tooling in the past, which often lead to skepticism amongst different stakeholders.
Essentials of Automations: Optimizing FME Workflows with ParametersSafe Software
Are you looking to streamline your workflows and boost your projects’ efficiency? Do you find yourself searching for ways to add flexibility and control over your FME workflows? If so, you’re in the right place.
Join us for an insightful dive into the world of FME parameters, a critical element in optimizing workflow efficiency. This webinar marks the beginning of our three-part “Essentials of Automation” series. This first webinar is designed to equip you with the knowledge and skills to utilize parameters effectively: enhancing the flexibility, maintainability, and user control of your FME projects.
Here’s what you’ll gain:
- Essentials of FME Parameters: Understand the pivotal role of parameters, including Reader/Writer, Transformer, User, and FME Flow categories. Discover how they are the key to unlocking automation and optimization within your workflows.
- Practical Applications in FME Form: Delve into key user parameter types including choice, connections, and file URLs. Allow users to control how a workflow runs, making your workflows more reusable. Learn to import values and deliver the best user experience for your workflows while enhancing accuracy.
- Optimization Strategies in FME Flow: Explore the creation and strategic deployment of parameters in FME Flow, including the use of deployment and geometry parameters, to maximize workflow efficiency.
- Pro Tips for Success: Gain insights on parameterizing connections and leveraging new features like Conditional Visibility for clarity and simplicity.
We’ll wrap up with a glimpse into future webinars, followed by a Q&A session to address your specific questions surrounding this topic.
Don’t miss this opportunity to elevate your FME expertise and drive your projects to new heights of efficiency.
Software Delivery At the Speed of AI: Inflectra Invests In AI-Powered QualityInflectra
In this insightful webinar, Inflectra explores how artificial intelligence (AI) is transforming software development and testing. Discover how AI-powered tools are revolutionizing every stage of the software development lifecycle (SDLC), from design and prototyping to testing, deployment, and monitoring.
Learn about:
• The Future of Testing: How AI is shifting testing towards verification, analysis, and higher-level skills, while reducing repetitive tasks.
• Test Automation: How AI-powered test case generation, optimization, and self-healing tests are making testing more efficient and effective.
• Visual Testing: Explore the emerging capabilities of AI in visual testing and how it's set to revolutionize UI verification.
• Inflectra's AI Solutions: See demonstrations of Inflectra's cutting-edge AI tools like the ChatGPT plugin and Azure Open AI platform, designed to streamline your testing process.
Whether you're a developer, tester, or QA professional, this webinar will give you valuable insights into how AI is shaping the future of software delivery.
Dev Dives: Train smarter, not harder – active learning and UiPath LLMs for do...UiPathCommunity
💥 Speed, accuracy, and scaling – discover the superpowers of GenAI in action with UiPath Document Understanding and Communications Mining™:
See how to accelerate model training and optimize model performance with active learning
Learn about the latest enhancements to out-of-the-box document processing – with little to no training required
Get an exclusive demo of the new family of UiPath LLMs – GenAI models specialized for processing different types of documents and messages
This is a hands-on session specifically designed for automation developers and AI enthusiasts seeking to enhance their knowledge in leveraging the latest intelligent document processing capabilities offered by UiPath.
Speakers:
👨🏫 Andras Palfi, Senior Product Manager, UiPath
👩🏫 Lenka Dulovicova, Product Program Manager, UiPath
Kubernetes & AI - Beauty and the Beast !?! @KCD Istanbul 2024Tobias Schneck
As AI technology is pushing into IT I was wondering myself, as an “infrastructure container kubernetes guy”, how get this fancy AI technology get managed from an infrastructure operational view? Is it possible to apply our lovely cloud native principals as well? What benefit’s both technologies could bring to each other?
Let me take this questions and provide you a short journey through existing deployment models and use cases for AI software. On practical examples, we discuss what cloud/on-premise strategy we may need for applying it to our own infrastructure to get it to work from an enterprise perspective. I want to give an overview about infrastructure requirements and technologies, what could be beneficial or limiting your AI use cases in an enterprise environment. An interactive Demo will give you some insides, what approaches I got already working for real.
GraphRAG is All You need? LLM & Knowledge GraphGuy Korland
Guy Korland, CEO and Co-founder of FalkorDB, will review two articles on the integration of language models with knowledge graphs.
1. Unifying Large Language Models and Knowledge Graphs: A Roadmap.
https://arxiv.org/abs/2306.08302
2. Microsoft Research's GraphRAG paper and a review paper on various uses of knowledge graphs:
https://www.microsoft.com/en-us/research/blog/graphrag-unlocking-llm-discovery-on-narrative-private-data/
Accelerate your Kubernetes clusters with Varnish CachingThijs Feryn
A presentation about the usage and availability of Varnish on Kubernetes. This talk explores the capabilities of Varnish caching and shows how to use the Varnish Helm chart to deploy it to Kubernetes.
This presentation was delivered at K8SUG Singapore. See https://feryn.eu/presentations/accelerate-your-kubernetes-clusters-with-varnish-caching-k8sug-singapore-28-2024 for more details.
Encryption in Microsoft 365 - ExpertsLive Netherlands 2024Albert Hoitingh
In this session I delve into the encryption technology used in Microsoft 365 and Microsoft Purview. Including the concepts of Customer Key and Double Key Encryption.
The Art of the Pitch: WordPress Relationships and SalesLaura Byrne
Clients don’t know what they don’t know. What web solutions are right for them? How does WordPress come into the picture? How do you make sure you understand scope and timeline? What do you do if sometime changes?
All these questions and more will be explored as we talk about matching clients’ needs with what your agency offers without pulling teeth or pulling your hair out. Practical tips, and strategies for successful relationship building that leads to closing the deal.
Connector Corner: Automate dynamic content and events by pushing a buttonDianaGray10
Here is something new! In our next Connector Corner webinar, we will demonstrate how you can use a single workflow to:
Create a campaign using Mailchimp with merge tags/fields
Send an interactive Slack channel message (using buttons)
Have the message received by managers and peers along with a test email for review
But there’s more:
In a second workflow supporting the same use case, you’ll see:
Your campaign sent to target colleagues for approval
If the “Approve” button is clicked, a Jira/Zendesk ticket is created for the marketing design team
But—if the “Reject” button is pushed, colleagues will be alerted via Slack message
Join us to learn more about this new, human-in-the-loop capability, brought to you by Integration Service connectors.
And...
Speakers:
Akshay Agnihotri, Product Manager
Charlie Greenberg, Host
21. Why is it so tempting?
Service
A
Event Bus
A
B
Service
B
C
Service
C
D
E
F
Service
D
G
Service
…
Service
…
Service …
Service
…
Service …
Service …
Service
…
@berndruecker
22. Why is it so tempting?
Service
A
Event Bus
A
B
Service
B
C
Service
C
D
E
F
Service
D
G
Service
…
Service
…
Service …
Service
…
Service …
Service …
Service
…
Adding is easy!
You can „buy“ a shorter
initial time-to-value
by choreography.
It yields in technical debt.
@berndruecker
33. Using a workflow engine
Workflow Engine
Scheduler
Durable State
Glue Code Whatever you
need…
Workflow Definition
Workflow Engine:
Is stateful
Can wait
Can retry
Can escalate
Can compensate
Provides visibility
34. Now it is easy to change the process flow
@berndruecker
39. Commands in disguise
The Customer Needs To Be
Sent A Message To Confirm
Address Change
Event
Send
Message
Wording of
Sender
Wording of
recipient
40. Checkout Order Payment
Event-driven:
Decision to couple is on the receiving side
Command-driven
Decision to couple is on the sending side
Direction of dependency
Retrieve
Payment
Order
placed
Payment
received
Direction of dependency
50. # Orchestration != central
# Choreography != decoupled
# Orchestration = Command-driven
# Choreography = Event-driven
# You need to balance both!
# It is mostly about responsibility and the direction of coupling
@berndruecker
51. Want to learn more?
https://learning.oreilly.com/get-learning/?code=PPAER20