Microservice Architecture with CQRS and Event SourcingBen Wilcock
In this slide deck I'll introduce you to the Command and Query Responsibility Segregation [CQRS] and Event Sourcing [ES] patterns for software architecture and explain why I think they're worth investing a little of your time in if you're building Microservices that need to scale gracefully.
Presentation slides from sessions discussing Event Source data storage and read-model projections.
Source code from demos at: https://bitbucket.org/csharpfritz/nerddinner-cqrs
Introduction to Event Sourcing and Challenges such as consistency, validation and concurrent updates.
The talk has been delivered at the Microservice Meetup in Munich.
Building Cloud-Native App Series - Part 2 of 11
Microservices Architecture Series
Event Sourcing & CQRS,
Kafka, Rabbit MQ
Case Studies (E-Commerce App, Movie Streaming, Ticket Booking, Restaurant, Hospital Management)
Event-driven architecture (EDA) is a software architecture pattern promoting the production, detection, consumption of, and reaction to events.
This architectural pattern may be applied by the design and implementation of applications and systems which transmit events among loosely coupled software components and services.
In this session you’ll learn how to create a loosely coupled architecture for your business that has the domain at the core. You’ll learn the basics of EDA, and also learn how we are transforming our architecture at Unibet.com to become event driven, and what benefits it will bring to our business. The session will cover technologies such as JMS, XML, JSON, Google Protocol Buffers, ActiveMQ and Spring.
Apache kafka 모니터링을 위한 Metrics 이해 및 최적화 방안SANG WON PARK
Apache Kafak의 빅데이터 아키텍처에서 역할이 점차 커지고, 중요한 비중을 차지하게 되면서, 성능에 대한 고민도 늘어나고 있다.
다양한 프로젝트를 진행하면서 Apache Kafka를 모니터링 하기 위해 필요한 Metrics들을 이해하고, 이를 최적화 하기 위한 Configruation 설정을 정리해 보았다.
[Apache kafka 모니터링을 위한 Metrics 이해 및 최적화 방안]
Apache Kafka 성능 모니터링에 필요한 metrics에 대해 이해하고, 4가지 관점(처리량, 지연, Durability, 가용성)에서 성능을 최적화 하는 방안을 정리함. Kafka를 구성하는 3개 모듈(Producer, Broker, Consumer)별로 성능 최적화를 위한 …
[Apache Kafka 모니터링을 위한 Metrics 이해]
Apache Kafka의 상태를 모니터링 하기 위해서는 4개(System(OS), Producer, Broker, Consumer)에서 발생하는 metrics들을 살펴봐야 한다.
이번 글에서는 JVM에서 제공하는 JMX metrics를 중심으로 producer/broker/consumer의 지표를 정리하였다.
모든 지표를 정리하진 않았고, 내 관점에서 유의미한 지표들을 중심으로 이해한 내용임
[Apache Kafka 성능 Configuration 최적화]
성능목표를 4개로 구분(Throughtput, Latency, Durability, Avalibility)하고, 각 목표에 따라 어떤 Kafka configuration의 조정을 어떻게 해야하는지 정리하였다.
튜닝한 파라미터를 적용한 후, 성능테스트를 수행하면서 추출된 Metrics를 모니터링하여 현재 업무에 최적화 되도록 최적화를 수행하는 것이 필요하다.
Microservice Architecture with CQRS and Event SourcingBen Wilcock
In this slide deck I'll introduce you to the Command and Query Responsibility Segregation [CQRS] and Event Sourcing [ES] patterns for software architecture and explain why I think they're worth investing a little of your time in if you're building Microservices that need to scale gracefully.
Presentation slides from sessions discussing Event Source data storage and read-model projections.
Source code from demos at: https://bitbucket.org/csharpfritz/nerddinner-cqrs
Introduction to Event Sourcing and Challenges such as consistency, validation and concurrent updates.
The talk has been delivered at the Microservice Meetup in Munich.
Building Cloud-Native App Series - Part 2 of 11
Microservices Architecture Series
Event Sourcing & CQRS,
Kafka, Rabbit MQ
Case Studies (E-Commerce App, Movie Streaming, Ticket Booking, Restaurant, Hospital Management)
Event-driven architecture (EDA) is a software architecture pattern promoting the production, detection, consumption of, and reaction to events.
This architectural pattern may be applied by the design and implementation of applications and systems which transmit events among loosely coupled software components and services.
In this session you’ll learn how to create a loosely coupled architecture for your business that has the domain at the core. You’ll learn the basics of EDA, and also learn how we are transforming our architecture at Unibet.com to become event driven, and what benefits it will bring to our business. The session will cover technologies such as JMS, XML, JSON, Google Protocol Buffers, ActiveMQ and Spring.
Apache kafka 모니터링을 위한 Metrics 이해 및 최적화 방안SANG WON PARK
Apache Kafak의 빅데이터 아키텍처에서 역할이 점차 커지고, 중요한 비중을 차지하게 되면서, 성능에 대한 고민도 늘어나고 있다.
다양한 프로젝트를 진행하면서 Apache Kafka를 모니터링 하기 위해 필요한 Metrics들을 이해하고, 이를 최적화 하기 위한 Configruation 설정을 정리해 보았다.
[Apache kafka 모니터링을 위한 Metrics 이해 및 최적화 방안]
Apache Kafka 성능 모니터링에 필요한 metrics에 대해 이해하고, 4가지 관점(처리량, 지연, Durability, 가용성)에서 성능을 최적화 하는 방안을 정리함. Kafka를 구성하는 3개 모듈(Producer, Broker, Consumer)별로 성능 최적화를 위한 …
[Apache Kafka 모니터링을 위한 Metrics 이해]
Apache Kafka의 상태를 모니터링 하기 위해서는 4개(System(OS), Producer, Broker, Consumer)에서 발생하는 metrics들을 살펴봐야 한다.
이번 글에서는 JVM에서 제공하는 JMX metrics를 중심으로 producer/broker/consumer의 지표를 정리하였다.
모든 지표를 정리하진 않았고, 내 관점에서 유의미한 지표들을 중심으로 이해한 내용임
[Apache Kafka 성능 Configuration 최적화]
성능목표를 4개로 구분(Throughtput, Latency, Durability, Avalibility)하고, 각 목표에 따라 어떤 Kafka configuration의 조정을 어떻게 해야하는지 정리하였다.
튜닝한 파라미터를 적용한 후, 성능테스트를 수행하면서 추출된 Metrics를 모니터링하여 현재 업무에 최적화 되도록 최적화를 수행하는 것이 필요하다.
Debezium is a Kafka Connect plugin that performs Change Data Capture from your database into Kafka. This talk demonstrates how this can be leveraged to move your data from one database platform such as MySQL to PostgreSQL. A working example is available on GitHub (github.com/gh-mlfowler/debezium-demo).
CQRS and Event Sourcing, An Alternative Architecture for DDDDennis Doomen
Most of us will be familiar with the standard 3- or 4-layer architecture you often see in larger enterprise systems. Some are already practicing Domain Driven Design and work together with the business to clarify the domain concepts. Perhaps you’ve noticed that is difficult to get the intention of the 'verbs' from that domain into this standard architecture. If performance is an important requirement as well, then you might have discovered that an Object-Relational Mapper and a relational database are not always the best solution.
One of the main reasons for this is the fact that the interests of a consistent domain that takes into account the many business rules, and those of data reporting and presentation are conflicting. That’s why Betrand Meyer introduced the Command Query Separation principle.
An architecture based on this principle combined with the Event Sourcing concept provides the ideal architecture for building high-performance systems designed using DDD. Well-known bloggers like Udi Dahan and Greg Young have already spent quite a lot of of posts on this, and this year’s Developer Days had some coverage as well.
But how do you build such a system with the. NET framework? Is it really as complex as some claim, or is just different work?
A Visual Introduction to Event Sourcing and CQRS by Lorenzo NicoraOpenCredo
Slides from Lorenzo's speech at Haufe-Lexware Microservice Architecture Day, in Freiburg. A pictorial introduction to the concepts of Command and Event Sourcing, and CQRS, as persistence model for modern, scalable and distributed applications.
Distributed Transactions is a key concept for Micro Services based Apps and Saga Design Pattern helps out over here. However, developers struggle to shift their mindset from CRUD based design to Event Sourcing / CQRS concept. To solve this problem we are introducing the concept of Event Storming and Event Storming Process map.
Building Cloud-Native App Series - Part 11 of 11
Microservices Architecture Series
Service Mesh - Observability
- Zipkin
- Prometheus
- Grafana
- Kiali
Producer Performance Tuning for Apache KafkaJiangjie Qin
Kafka is well known for high throughput ingestion. However, to get the best latency characteristics without compromising on throughput and durability, we need to tune Kafka. In this talk, we share our experiences to achieve the optimal combination of latency, throughput and durability for different scenarios.
This modern engineering technique has grown from good old SOA (Service Oriented Architecture) with features like REST (vs. old SOAP) support, NoSQL databases and the Event driven/reactive approach sprinkled in.
Microservices
The criticism
Evolutionary approach
Best practices
Create a Separate Database for Each Service
Rely on contracts between services
Deploy in Containers
Treat Servers as Volatile
Related techniques and patterns
Design patterns
Integration techniques
Deployment of microservices
Serverless - Function as a Service
Continuous Deployment
Related technologies
Microservices based e-commerce platforms
Technologies that empower microservices achitecture
Distributed logging and monitoring
Case Studies: Re-architecting the monolith
Kafka as an event store - is it good enough?Guido Schmutz
Event Sourcing and CQRS are two popular patterns for implementing a Microservices architectures. With Event Sourcing we do not store the state of an object, but instead store all the events impacting its state. Then to retrieve an object state, we have to read the different events related to a certain object and apply them one by one. CQRS (Command Query Responsibility Segregation) on the other hand is a way to dissociate writes (Command) and reads (Query). Event Sourcing and CQRS are frequently grouped and used together to form something bigger. While it is possible to implement CQRS without Event Sourcing, the opposite is not necessarily correct. In order to implement Event Sourcing, an efficient Event Store is needed. But is that also true when combining Event Sourcing and CQRS? And what is an event store in the first place and what features should it implement?
This presentation will first discuss what functionalities an event store should offer and then present how Apache Kafka can be used to implement an event store. But is Kafka good enough or do specific event store solutions such as AxonDB or Event Store provide a better solution?
Developing event-driven microservices with event sourcing and CQRS (phillyete)Chris Richardson
Modern, cloud-native applications typically use a microservices architecture in conjunction with NoSQL and/or sharded relational databases. However, in order to successfully use this approach you need to solve some distributed data management problems including how to maintain consistency between multiple databases without using 2PC. In this talk you will learn more about these issues and how to solve them by using an event-driven architecture. We will describe how event sourcing and Command Query Responsibility Separation (CQRS) are a great way to realize an event-driven architecture. You will learn about a simple yet powerful approach for building, modern, scalable applications.
Event-Driven Microservices architecture has gained a lot of attention recently. The trend in the industry is to move away from Monolithic applications to Microservices to innovate faster. While Microservices have their benefits, implementing them is hard. This talk focuses on the challenges faced and how to solve them.
It covers topics like using Domain Driven Design to break functionality into small parts. Various communication patterns among Microservices are also discussed.
One major drawback is the problem of distributed data management, as each Microservice has its own database. Event-Driven Architecture enables a way to make microservices work together and the talks show how to use architectural patterns like Event Sourcing & CQRS to implement them.
Another implementation challenge is to manage transactions that update entities owned by multiple services in an eventually consistent fashion. This challenge is solved using sagas, which can be thought of as Long running transactions that use compensating actions to handle failures.
The objective of the talk is to show how to implement highly distributed Event Driven Microservices architecture that are scalable and easy to maintain.
Event-driven architecture is a versatile approach to designing and integrating complex software systems. These systems tend to be easier to model and build. Event-driven architecture is not a new concept, but as more organizations contemplate microservices, this approach to system design has become appropriate in more situations and is worth a fresh look.
Debezium is a Kafka Connect plugin that performs Change Data Capture from your database into Kafka. This talk demonstrates how this can be leveraged to move your data from one database platform such as MySQL to PostgreSQL. A working example is available on GitHub (github.com/gh-mlfowler/debezium-demo).
CQRS and Event Sourcing, An Alternative Architecture for DDDDennis Doomen
Most of us will be familiar with the standard 3- or 4-layer architecture you often see in larger enterprise systems. Some are already practicing Domain Driven Design and work together with the business to clarify the domain concepts. Perhaps you’ve noticed that is difficult to get the intention of the 'verbs' from that domain into this standard architecture. If performance is an important requirement as well, then you might have discovered that an Object-Relational Mapper and a relational database are not always the best solution.
One of the main reasons for this is the fact that the interests of a consistent domain that takes into account the many business rules, and those of data reporting and presentation are conflicting. That’s why Betrand Meyer introduced the Command Query Separation principle.
An architecture based on this principle combined with the Event Sourcing concept provides the ideal architecture for building high-performance systems designed using DDD. Well-known bloggers like Udi Dahan and Greg Young have already spent quite a lot of of posts on this, and this year’s Developer Days had some coverage as well.
But how do you build such a system with the. NET framework? Is it really as complex as some claim, or is just different work?
A Visual Introduction to Event Sourcing and CQRS by Lorenzo NicoraOpenCredo
Slides from Lorenzo's speech at Haufe-Lexware Microservice Architecture Day, in Freiburg. A pictorial introduction to the concepts of Command and Event Sourcing, and CQRS, as persistence model for modern, scalable and distributed applications.
Distributed Transactions is a key concept for Micro Services based Apps and Saga Design Pattern helps out over here. However, developers struggle to shift their mindset from CRUD based design to Event Sourcing / CQRS concept. To solve this problem we are introducing the concept of Event Storming and Event Storming Process map.
Building Cloud-Native App Series - Part 11 of 11
Microservices Architecture Series
Service Mesh - Observability
- Zipkin
- Prometheus
- Grafana
- Kiali
Producer Performance Tuning for Apache KafkaJiangjie Qin
Kafka is well known for high throughput ingestion. However, to get the best latency characteristics without compromising on throughput and durability, we need to tune Kafka. In this talk, we share our experiences to achieve the optimal combination of latency, throughput and durability for different scenarios.
This modern engineering technique has grown from good old SOA (Service Oriented Architecture) with features like REST (vs. old SOAP) support, NoSQL databases and the Event driven/reactive approach sprinkled in.
Microservices
The criticism
Evolutionary approach
Best practices
Create a Separate Database for Each Service
Rely on contracts between services
Deploy in Containers
Treat Servers as Volatile
Related techniques and patterns
Design patterns
Integration techniques
Deployment of microservices
Serverless - Function as a Service
Continuous Deployment
Related technologies
Microservices based e-commerce platforms
Technologies that empower microservices achitecture
Distributed logging and monitoring
Case Studies: Re-architecting the monolith
Kafka as an event store - is it good enough?Guido Schmutz
Event Sourcing and CQRS are two popular patterns for implementing a Microservices architectures. With Event Sourcing we do not store the state of an object, but instead store all the events impacting its state. Then to retrieve an object state, we have to read the different events related to a certain object and apply them one by one. CQRS (Command Query Responsibility Segregation) on the other hand is a way to dissociate writes (Command) and reads (Query). Event Sourcing and CQRS are frequently grouped and used together to form something bigger. While it is possible to implement CQRS without Event Sourcing, the opposite is not necessarily correct. In order to implement Event Sourcing, an efficient Event Store is needed. But is that also true when combining Event Sourcing and CQRS? And what is an event store in the first place and what features should it implement?
This presentation will first discuss what functionalities an event store should offer and then present how Apache Kafka can be used to implement an event store. But is Kafka good enough or do specific event store solutions such as AxonDB or Event Store provide a better solution?
Developing event-driven microservices with event sourcing and CQRS (phillyete)Chris Richardson
Modern, cloud-native applications typically use a microservices architecture in conjunction with NoSQL and/or sharded relational databases. However, in order to successfully use this approach you need to solve some distributed data management problems including how to maintain consistency between multiple databases without using 2PC. In this talk you will learn more about these issues and how to solve them by using an event-driven architecture. We will describe how event sourcing and Command Query Responsibility Separation (CQRS) are a great way to realize an event-driven architecture. You will learn about a simple yet powerful approach for building, modern, scalable applications.
Event-Driven Microservices architecture has gained a lot of attention recently. The trend in the industry is to move away from Monolithic applications to Microservices to innovate faster. While Microservices have their benefits, implementing them is hard. This talk focuses on the challenges faced and how to solve them.
It covers topics like using Domain Driven Design to break functionality into small parts. Various communication patterns among Microservices are also discussed.
One major drawback is the problem of distributed data management, as each Microservice has its own database. Event-Driven Architecture enables a way to make microservices work together and the talks show how to use architectural patterns like Event Sourcing & CQRS to implement them.
Another implementation challenge is to manage transactions that update entities owned by multiple services in an eventually consistent fashion. This challenge is solved using sagas, which can be thought of as Long running transactions that use compensating actions to handle failures.
The objective of the talk is to show how to implement highly distributed Event Driven Microservices architecture that are scalable and easy to maintain.
Event-driven architecture is a versatile approach to designing and integrating complex software systems. These systems tend to be easier to model and build. Event-driven architecture is not a new concept, but as more organizations contemplate microservices, this approach to system design has become appropriate in more situations and is worth a fresh look.
Domain-driven design is a collaborative process involving both domain experts and software practitioners that attempts to address issues of complexity in software. This process is described in the book Domain-Driven Design (Addison-Wesley 2004) written by Eric Evans. Domain-driven design starts with the assertion that (for almost all software) complexity is in the domain, not in the technology. Accordingly, we must let technology play a supporting role. Domain-driven design attempts to focus on and distill the core domain for a given project.
Philosopher and scientist Alfred Korzybski said, "The map is not the territory." As such, a person practicing domain-driven design does not attempt to model reality. Instead, domain experts and software practitioners use a mental model as a tool for solving problems within a given domain. The domain experts and software practitioners collaborate to explore and develop this model. No software of any reasonable scope has just one model. We will look at the concept of a bounded context within which each model can be isolated and explored. Within a bounded context, collaborators must speak a ubiquitous language in order to reason about and discuss the model.
We will also talk about domain-driven design's building block patterns including entities, value objects, aggregates, repositories, services, and domain events. We will look at domain-driven design practices including supple design, strategic design, and distillation of the core. We will see how test-driven development can be used as a means of exploring the model. Examples in PHP will be provided of the building block patterns as well as other techniques including closure of operations, intention revealing interfaces, side-effect free functions, and assertions.
Domain-Driven Design (DDD) is very useful set of tools to tackle complexity in a software projects. However, many software developers never heard of it, yet most of the one who do emphasize too much on the technical implementation.
This slide will explain what is DDD and why, and also what is its core.
An Introduction to Domain Driven Design for Product Managersr4isstatic
A presentation to the BBC Product Management community, introducing the concepts, processes and techniques of Domain Driven Design, including Domain Modelling and URL design. Also talks briefly about the Semantic W
Agile development and domain driven designJacopo Romei
Agile development is getting every year more popular around the world while a less known methodology is the Domain Driven Design (DDD) which defines a few rules to be followed to empower the agile team to raise communication effectiveness. Agile methods and DDD are perfectly matching and used together can solve many problems we are all too sadly used to.
Domain-driven design is a collaborative process involving both domain experts and software practitioners. This high-level overview takes a look at the driving principles behind domain-driven design. It also explores domain-driven design's building block patterns, supple design, strategic design, and distillation of the core.
An Introduction to Domain Driven Design focusing on the concepts of Bounded Context, Strategic & Tactical Design, CQRS, Ubiquitous Language, Hexagonal Architecture, Event Sourcing, Task - based UIs
Security, Privacy & Convenience – key drivers for mobile adoption from a cons...Subho Halder
Appknox will talk about bole security and privacy from a consumer perspective and discuss things that businesses need to keep in mind. While time to market is an important thing, security should not be replaced by convenience.
A Research Paper review presentation on "Government Controlled Mobile VoIP", presented by Md. Habibur Rahman, BIT0216, Institute of Information Technology University of Dhaka.
Modelling a complex domain with Domain-Driven DesignNaeem Sarfraz
Domain-Driven Design is an approach to modelling business complexity explicitly in your software. This deck of slides runs through the key concepts focusing on both the strategic and tactical aspects of DDD.
Event sourcing for IoT and mobile - JAX London 2017Lorenzo Nicora
When you have to scale and cope with unreliability of clients and network, event sourcing still works well, but you need to take an approach different from the mainstream
Voxxed Bristol 2017 - From C to Q, one event at a time: Event Sourcing illust...OpenCredo
Event Sourcing is becoming a popular persistence pattern. But when to trade simplicity of relational databases for scalability and high throughput? Analysing the rationale behind Event Sourcing and separating Commands from Queries, will help you make an informed decision aware of benefits and risks.
Event Sourcing is becoming a popular persistence pattern for scalable applications, reactive architectures and microservices. The subject is rather new, and we have no well defined, “gang-of-four”-like design patterns yet. Nevertheless, most of the introductory materials imply a particular implementation style. This presentation explains Event Sourcing and CQRS without assuming a particular implementation, to give you the freedom of choosing the pattern most appropriate for your business case. It’s a story, following the much-loved format of a visual picture book. Starting from relational and normalised persistent model, through the issues you face when scaling and moving to asynchronous, distributed application. How to get around these limits with event or command sourcing, why you have to use CQRS and what are your options for queries. The journey ends with benefits, drawbacks and, most importantly, with lessons learned “at the coal face” from real projects.
How Fast Data Is Turned into Fast Information and Timely Action (OOW 2014)Lucas Jellema
Fast data is big data, continuously streaming in, from which information is to be learned in (near) real time. This session demonstrates how Oracle Event Processing is used to analyze live streams of data to find patterns, deviations, and aggregates. The findings are reported in the form of business events that are pushed in live dashboards to Oracle Business Activity Monitoring, which also evaluates business rules on the business events and takes action when required. Examples to be demonstrated in this session include car sensors, website traffic, Twitter feeds, and bank run detection. Oracle SOA Suite 12c, WebSockets, Oracle Application Development Framework (Oracle ADF) active data visualization tools components, and JMS are used to process, forward, and act.
Stream processing with Apache Flink - Maximilian Michels Data ArtisansEvention
Apache Flink is an open source platform for distributed stream and batch data processing. At its core, Flink is a streaming dataflow engine which provides data distribution, communication, and fault tolerance for distributed computations over data streams. On top of this core, APIs make it easy to develop distributed data analysis programs. Libraries for graph processing or machine learning provide convenient abstractions for solving large-scale problems. Apache Flink integrates with a multitude of other open source systems like Hadoop, databases, or message queues. Its streaming capabilities make it a perfect fit for traditional batch processing as well as state of the art stream processing.
Sync or swim: the challenge of complex offline appsOutSystems
Video: https://www.outsystems.com/learn/courses/56/webinar-the-challenge-of-complex-offline-apps/
The requirement is usually stated in very simple terms: deliver an app that is available in all connectivity conditions. However, the solution can be anything but simple.
When data management rears its ugly head, things get really tricky fast..How are you going to synchronize that data?
Conflicts will most definitely occur and the strategy to approach them depends on how clever your implementation is.
There is no silver bullet! But we can help you think it through. If you were on hand when we showed you how to discover some of the most common sync patterns and techniques, you got a preview.
Make no mistake, though: we're going beyond the common and we're delving deep. So, if you weren’t around for Create Your Offline App, no worries - we’ll get you fully up to speed. We're not going to feed you for a day, we're here to teach you how to fish!
In this webinar you will:
- Review some of the common sync patterns OutSystems handles.
- Understand how the synchronization process happens in the background.
- Learn how to translate offline-first thinking into development best practices.
- Learn how to implement a more advanced sync scenario.
Free Online training: https://www.outsystems.com/learn/courses/
Follow us on Twitter http://www.twitter.com/OutSystemsDev
Like us on Facebook http://www.Facebook.com/OutSystemsDev
Fast data is big data, continuously streaming in, from which information is to be learned in (near) real time. This session demonstrates how Oracle Event Processing is used to analyze live streams of data to find patterns, deviations, and aggregates. The findings are reported in the form of business events that are pushed in live dashboards to Oracle Business Activity Monitoring, which also evaluates business rules on the business events and takes action when required. Examples to be demonstrated in this session include car sensors, website traffic, Twitter feeds, and bank run detection. Oracle SOA Suite 12c, WebSockets, Oracle Application Development Framework (Oracle ADF) active data visualization tools components, and JMS are used to process, forward, and act.
Similar to A visual introduction to Event Sourcing and CQRS (9)
Enhancing Research Orchestration Capabilities at ORNL.pdfGlobus
Cross-facility research orchestration comes with ever-changing constraints regarding the availability and suitability of various compute and data resources. In short, a flexible data and processing fabric is needed to enable the dynamic redirection of data and compute tasks throughout the lifecycle of an experiment. In this talk, we illustrate how we easily leveraged Globus services to instrument the ACE research testbed at the Oak Ridge Leadership Computing Facility with flexible data and task orchestration capabilities.
Custom Healthcare Software for Managing Chronic Conditions and Remote Patient...Mind IT Systems
Healthcare providers often struggle with the complexities of chronic conditions and remote patient monitoring, as each patient requires personalized care and ongoing monitoring. Off-the-shelf solutions may not meet these diverse needs, leading to inefficiencies and gaps in care. It’s here, custom healthcare software offers a tailored solution, ensuring improved care and effectiveness.
In software engineering, the right architecture is essential for robust, scalable platforms. Wix has undergone a pivotal shift from event sourcing to a CRUD-based model for its microservices. This talk will chart the course of this pivotal journey.
Event sourcing, which records state changes as immutable events, provided robust auditing and "time travel" debugging for Wix Stores' microservices. Despite its benefits, the complexity it introduced in state management slowed development. Wix responded by adopting a simpler, unified CRUD model. This talk will explore the challenges of event sourcing and the advantages of Wix's new "CRUD on steroids" approach, which streamlines API integration and domain event management while preserving data integrity and system resilience.
Participants will gain valuable insights into Wix's strategies for ensuring atomicity in database updates and event production, as well as caching, materialization, and performance optimization techniques within a distributed system.
Join us to discover how Wix has mastered the art of balancing simplicity and extensibility, and learn how the re-adoption of the modest CRUD has turbocharged their development velocity, resilience, and scalability in a high-growth environment.
Paketo Buildpacks : la meilleure façon de construire des images OCI? DevopsDa...Anthony Dahanne
Les Buildpacks existent depuis plus de 10 ans ! D’abord, ils étaient utilisés pour détecter et construire une application avant de la déployer sur certains PaaS. Ensuite, nous avons pu créer des images Docker (OCI) avec leur dernière génération, les Cloud Native Buildpacks (CNCF en incubation). Sont-ils une bonne alternative au Dockerfile ? Que sont les buildpacks Paketo ? Quelles communautés les soutiennent et comment ?
Venez le découvrir lors de cette session ignite
May Marketo Masterclass, London MUG May 22 2024.pdfAdele Miller
Can't make Adobe Summit in Vegas? No sweat because the EMEA Marketo Engage Champions are coming to London to share their Summit sessions, insights and more!
This is a MUG with a twist you don't want to miss.
A Comprehensive Look at Generative AI in Retail App Testing.pdfkalichargn70th171
Traditional software testing methods are being challenged in retail, where customer expectations and technological advancements continually shape the landscape. Enter generative AI—a transformative subset of artificial intelligence technologies poised to revolutionize software testing.
AI Pilot Review: The World’s First Virtual Assistant Marketing SuiteGoogle
AI Pilot Review: The World’s First Virtual Assistant Marketing Suite
👉👉 Click Here To Get More Info 👇👇
https://sumonreview.com/ai-pilot-review/
AI Pilot Review: Key Features
✅Deploy AI expert bots in Any Niche With Just A Click
✅With one keyword, generate complete funnels, websites, landing pages, and more.
✅More than 85 AI features are included in the AI pilot.
✅No setup or configuration; use your voice (like Siri) to do whatever you want.
✅You Can Use AI Pilot To Create your version of AI Pilot And Charge People For It…
✅ZERO Manual Work With AI Pilot. Never write, Design, Or Code Again.
✅ZERO Limits On Features Or Usages
✅Use Our AI-powered Traffic To Get Hundreds Of Customers
✅No Complicated Setup: Get Up And Running In 2 Minutes
✅99.99% Up-Time Guaranteed
✅30 Days Money-Back Guarantee
✅ZERO Upfront Cost
See My Other Reviews Article:
(1) TubeTrivia AI Review: https://sumonreview.com/tubetrivia-ai-review
(2) SocioWave Review: https://sumonreview.com/sociowave-review
(3) AI Partner & Profit Review: https://sumonreview.com/ai-partner-profit-review
(4) AI Ebook Suite Review: https://sumonreview.com/ai-ebook-suite-review
Climate Science Flows: Enabling Petabyte-Scale Climate Analysis with the Eart...Globus
The Earth System Grid Federation (ESGF) is a global network of data servers that archives and distributes the planet’s largest collection of Earth system model output for thousands of climate and environmental scientists worldwide. Many of these petabyte-scale data archives are located in proximity to large high-performance computing (HPC) or cloud computing resources, but the primary workflow for data users consists of transferring data, and applying computations on a different system. As a part of the ESGF 2.0 US project (funded by the United States Department of Energy Office of Science), we developed pre-defined data workflows, which can be run on-demand, capable of applying many data reduction and data analysis to the large ESGF data archives, transferring only the resultant analysis (ex. visualizations, smaller data files). In this talk, we will showcase a few of these workflows, highlighting how Globus Flows can be used for petabyte-scale climate analysis.
How to Position Your Globus Data Portal for Success Ten Good PracticesGlobus
Science gateways allow science and engineering communities to access shared data, software, computing services, and instruments. Science gateways have gained a lot of traction in the last twenty years, as evidenced by projects such as the Science Gateways Community Institute (SGCI) and the Center of Excellence on Science Gateways (SGX3) in the US, The Australian Research Data Commons (ARDC) and its platforms in Australia, and the projects around Virtual Research Environments in Europe. A few mature frameworks have evolved with their different strengths and foci and have been taken up by a larger community such as the Globus Data Portal, Hubzero, Tapis, and Galaxy. However, even when gateways are built on successful frameworks, they continue to face the challenges of ongoing maintenance costs and how to meet the ever-expanding needs of the community they serve with enhanced features. It is not uncommon that gateways with compelling use cases are nonetheless unable to get past the prototype phase and become a full production service, or if they do, they don't survive more than a couple of years. While there is no guaranteed pathway to success, it seems likely that for any gateway there is a need for a strong community and/or solid funding streams to create and sustain its success. With over twenty years of examples to draw from, this presentation goes into detail for ten factors common to successful and enduring gateways that effectively serve as best practices for any new or developing gateway.
Quarkus Hidden and Forbidden ExtensionsMax Andersen
Quarkus has a vast extension ecosystem and is known for its subsonic and subatomic feature set. Some of these features are not as well known, and some extensions are less talked about, but that does not make them less interesting - quite the opposite.
Come join this talk to see some tips and tricks for using Quarkus and some of the lesser known features, extensions and development techniques.
Globus Compute wth IRI Workflows - GlobusWorld 2024Globus
As part of the DOE Integrated Research Infrastructure (IRI) program, NERSC at Lawrence Berkeley National Lab and ALCF at Argonne National Lab are working closely with General Atomics on accelerating the computing requirements of the DIII-D experiment. As part of the work the team is investigating ways to speedup the time to solution for many different parts of the DIII-D workflow including how they run jobs on HPC systems. One of these routes is looking at Globus Compute as a way to replace the current method for managing tasks and we describe a brief proof of concept showing how Globus Compute could help to schedule jobs and be a tool to connect compute at different facilities.
We describe the deployment and use of Globus Compute for remote computation. This content is aimed at researchers who wish to compute on remote resources using a unified programming interface, as well as system administrators who will deploy and operate Globus Compute services on their research computing infrastructure.
Navigating the Metaverse: A Journey into Virtual Evolution"Donna Lenk
Join us for an exploration of the Metaverse's evolution, where innovation meets imagination. Discover new dimensions of virtual events, engage with thought-provoking discussions, and witness the transformative power of digital realms."
Globus Connect Server Deep Dive - GlobusWorld 2024Globus
We explore the Globus Connect Server (GCS) architecture and experiment with advanced configuration options and use cases. This content is targeted at system administrators who are familiar with GCS and currently operate—or are planning to operate—broader deployments at their institution.
First Steps with Globus Compute Multi-User EndpointsGlobus
In this presentation we will share our experiences around getting started with the Globus Compute multi-user endpoint. Working with the Pharmacology group at the University of Auckland, we have previously written an application using Globus Compute that can offload computationally expensive steps in the researcher's workflows, which they wish to manage from their familiar Windows environments, onto the NeSI (New Zealand eScience Infrastructure) cluster. Some of the challenges we have encountered were that each researcher had to set up and manage their own single-user globus compute endpoint and that the workloads had varying resource requirements (CPUs, memory and wall time) between different runs. We hope that the multi-user endpoint will help to address these challenges and share an update on our progress here.
First Steps with Globus Compute Multi-User Endpoints
A visual introduction to Event Sourcing and CQRS
1. A visual introduction to
Event Sourcing and CQRS
1
@nicusX
https://opencredo.com/author/lorenzo/
Lorenzo Nicora
Senior Consultant @ OpenCredo
2. 2
A couple of concepts from DDD
Aggregate
(Current) State
of the Aggregate
Lorenzo Nicora Intro to Event Sourcing and CQRS
3. 3
Once upon a time…
Everything
is synchronous
Request - Response
Lorenzo Nicora Intro to Event Sourcing and CQRS
4. 4
Scaling up…
Updates —> Locks —> Contention!
<— Block <—
Lorenzo Nicora Intro to Event Sourcing and CQRS
5. 5
Let’s go Asynchronous
Pwd —> “secret”
Pwd —> “12345”
Pwd —> “54321”
Pwd —> “secret”
Pwd —> “54321”
Pwd —> “12345”
===>
Out of
Order
Asynchronous, Message-driven
Request/Response
ACID Transaction
Distributed, Message-based
—> No order guaranteed
Lorenzo Nicora Intro to Event Sourcing and CQRS
6. 6
Command Sourcing
💡
• Append Only —> No Contention
• Build State from Command history
Write fast,
Think later
K/V Store
Distributed
Lorenzo Nicora Intro to Event Sourcing and CQRS
7. Command
“Submit Order!”
—> A request (imperative sentence)
—> May fail
—> May affect multiple Aggregates
7
Commands vs Events
✉
Rebuild Aggregate State
from Commands
Lorenzo Nicora Intro to Event Sourcing and CQRS
8. 8
Event
“Order submitted”
—> Statement of facts (past tense)
—> Never fails
—> May affect a single Aggregate
✉
Rebuild Aggregate State
from Events
Lorenzo Nicora Intro to Event Sourcing and CQRS
9. 9
Commands to Events
(DDD patterns: Aggregate / Process Manager)
X
Y
Z
Lorenzo Nicora Intro to Event Sourcing and CQRS
10. 10
Command > Event Sourcing
💡
Think a little,
Write,
Think later
Lorenzo Nicora Intro to Event Sourcing and CQRS
12. 12
Additional Benefits
Easy
Eventual Business Consistency
—> Corrective Events
Robust to data corruption
(bugs, fat fingers…)
—> Rebuild state ignoring wrong events
Lorenzo Nicora Intro to Event Sourcing and CQRS
16. 16
Retrieving the State
How do I retrieve the State?
“Get details of Order ‘AB123’”
❔
not very efficient, but…
…may work
Lorenzo Nicora Intro to Event Sourcing and CQRS
17. 17
Querying (Searching) the State
❓
❓ How do query the State?
“Get all Orders delivered to ‘SE1 0NZ’”
❓
❓
Lorenzo Nicora Intro to Event Sourcing and CQRS
22. 22
Materialised Views (of State)
Latest (known) State
(Persistent)
Rebuildable
from Events
In Memory
K/V Store
Graph
…
RDBMS
Delayed
💡
Lorenzo Nicora Intro to Event Sourcing and CQRS
23. 23
Materialised View of State
Query a RDBMS?!?
Wasn’t it the old way?
❓
RDBMS is just one of our options:
easy to use, easily become a bottleneck
Lorenzo Nicora Intro to Event Sourcing and CQRS
24. 24
Materialised Views of State
* Views are optimised for
specific query use cases
—> multiple Views from same Events
* Updated asynchronously, delayed
—> to Scale
-> may reorder Events
Lorenzo Nicora Intro to Event Sourcing and CQRS
25. 25
Materialised Views of State
* Views can be rebuilt from Events
Event Log
is our Source of Truth
* Easy to evolve or fix
—> change or fix logic;
rebuild view from events
(not the View)
Lorenzo Nicora Intro to Event Sourcing and CQRS
27. 27
Indexes
Search Engines
K/V Stores
• Optimised for querying (less for retrieving)
• Latest State; rebuild on the fly
💡
Lorenzo Nicora Intro to Event Sourcing and CQRS
29. 29
Hybrid solutions: e.g. Snapshots
• Speed up rebuilding the current State
• Use recent Events to rebuild up-to-date
💡 Long delayed
Lorenzo Nicora Intro to Event Sourcing and CQRS
33. 33
Lesson Learned #1
If you put data in…
…you will eventually
have to get them out!
The “Query” side
is not secondary
Lorenzo Nicora Intro to Event Sourcing and CQRS
34. 34
Lessons Learned #2
In old days:
normalising one DB
to support as many queries as possible
With CQRS:
multiple denormalised “data stores”
optimised for different queries
No single “Q” implementation
for all your queries
Lorenzo Nicora Intro to Event Sourcing and CQRS
35. 35
Lessons Learned #3
A central, shared Event Store
may not be the best option
No Event-sourced Monolith
Prefer persistence
per Bounded-Context
Lorenzo Nicora Intro to Event Sourcing and CQRS
36. +++ Summing up +++
36
Lorenzo Nicora Intro to Event Sourcing and CQRS
37. 37
ES/CQRS Optimal Use Cases
High Volume
Low Latency writes
(big data)
Event Sourcing + CQRS
😋
Out-of-order Commands/Events
(IoT)
Lorenzo Nicora Intro to Event Sourcing and CQRS
38. 38
ES/CQRS Drawbacks
x No “One-Size-Fits-All”
—> Multiple “Q” implementations
x Delayed reads
x No ACID Transactions
x Additional complexity (!)
🙁
Lorenzo Nicora Intro to Event Sourcing and CQRS
39. 39
ES/CQRS Benefits
+ No “One-Size-Fits-All”
—> “Q” are optimised for use cases
+ Eventual (Business) Consistency
+ History, Temporal queries
+ Robust to data corruption
😀
Lorenzo Nicora Intro to Event Sourcing and CQRS