This document discusses developing polyglot persistence applications that use both SQL and NoSQL databases. It describes using Redis as a cache to optimize queries by storing denormalized data in Redis materialized views. It also covers how to synchronize data between MySQL and Redis reliably by queuing database events in MySQL and processing them asynchronously to update Redis. This avoids consistency issues but ensures both databases eventually converge.
Futures and Rx Observables: powerful abstractions for consuming web services ...Chris Richardson
A modular, polyglot architecture has many advantages but it also adds complexity since each incoming request typically fans out to multiple distributed services. For example, in an online store application the information on a product details page - description, price, recommendations, etc - comes from numerous services. To minimize response time and improve scalability, these services must be invoked concurrently. However, traditional concurrency mechanisms are low-level, painful to use and error-prone.
In this talk you will learn about some powerful yet easy to use abstractions for consuming web services asynchronously. We will compare the various implementations of futures that are available in Java, Scala and JavaScript. You will learn how to use reactive observables, which are asynchronous data streams, to access web services from both Java and JavaScript. We will describe how these mechanisms let you write asynchronous code in a very straightforward, declarative fashion.
Microservices are an essential enabler of agility but developing and deploying them is a challenge. In order for microservices to be loosely coupled,each service must have its own datastore. This makes it difficult to maintain data consistency across services.
Deploying microservices is also a complex problem since an application typically consists of 10s or 100s of services, written in a variety of languages and frameworks. In this presentation, you will learn how to solve these problems by using an event-driven architecture to maintain data consistency and by using Docker to simplify deployment.
There is no such thing as a microservice! (oracle code nyc)Chris Richardson
This is a keynote I gave at Oracle Code 2017 in New York:
https://developer.oracle.com/code/newyork
The microservice architecture is becoming increasingly popular. However, frequent references to using “a microservice to solve a problem” suggest that the concept is not universally well understood.
In this talk we define the microservice architecture as an architectural style and explain what that actually means. You will learn why the architecture that you pick for your application matters. We describe how the primary goal of the microservice architecture is to enable continuous delivery/deployment and how it achieves that.
You will learn how to solve key challenges with decomposing an application into microservices. We describe why there is no such thing as a microservice!
Microservices pattern language (microxchg microxchg2016)Chris Richardson
My talk from http://microxchg.io/2016/index.html.
Here is the video - https://www.youtube.com/watch?v=1mcVQhbkA2U
When architecting an enterprise Java application, you need to choose between the traditional monolithic architecture consisting of a single large WAR file, or the more fashionable microservices architecture consisting of many smaller services. But rather than blindly picking the familiar or the fashionable, it’s important to remember what Fred Books said almost 30 years ago: there are no silver bullets in software. Every architectural decision has both benefits and drawbacks. Whether the benefits of one approach outweigh the drawbacks greatly depends upon the context of your particular project. Moreover, even if you adopt the microservices architecture, you must still make numerous other design decisions, each with their own trade-offs.
A software pattern is an ideal way of describing a solution to a problem in a given context along with its tradeoffs. In this presentation, we describe a pattern language for microservices. You will learn about patterns that will help you decide when and how to use microservices vs. a monolithic architecture. We will also describe patterns that solve various problems in a microservice architecture including inter-service communication, service registration and service discovery.
YOW! Perth: Cubes, Hexagons, Triangles, and More: Understanding the Microserv...Chris Richardson
YOW! Perth 2019 presentation
The microservice architecture is becoming increasing important. But what is it exactly? Why should you care about microservices? And, what do you need to do to ensure that your organization uses the microservice architecture successfully? In this talk, I’ll answer these and other questions using shapes as visual metaphors. You will learn about the motivations for the microservice architecture and why simply adopting microservices is insufficient. I describe essential characteristics of microservices, You will learn how a successful microservice architecture consist of loosely coupled services with stable APIs that communicate asynchronous. I will cover strategies for effectively testing microservices.
Gluecon: Using sagas to maintain data consistency in a microservice architectureChris Richardson
The microservice architecture structures an application as a set of loosely coupled, collaborating services. Maintaining data consistency is challenging since each service has its own database to ensure loose coupling. To make matters worse, for a variety of reasons distributed transactions using JTA are not an option for modern applications.
In this talk we describe an alternative transaction model known as a saga. You will learn about the benefits and drawbacks of using sagas. We describe how sagas are eventually consistent rather than ACID and what this means for developers. You will learn how to design and implement sagas in a Java application.
ArchSummit Shenzhen - Using sagas to maintain data consistency in a microserv...Chris Richardson
This is a talk I gave at QCON ArchSummit in Shenzhen.
The microservice architecture structures an application as a set of loosely coupled, collaborating services. Maintaining data consistency is challenging since each service has its own database to ensure loose coupling. To make matters worse, for a variety of reasons distributed transactions using JTA are not an option for modern applications.
In this talk we describe an alternative transaction model known as a saga. You will learn about the benefits and drawbacks of using sagas. We describe how sagas are eventually consistent rather than ACID and what this means for developers. You will learn how to design and implement sagas in a Java application.
#JaxLondon keynote: Developing applications with a microservice architectureChris Richardson
The micro-service architecture, which structures an application as a set of small, narrowly focused, independently deployable services, is becoming an increasingly popular way to build applications. This approach avoids many of the problems of a monolithic architecture. It simplifies deployment and let’s you create highly scalable and available applications. In this keynote we describe the micro-service architecture and how to use it to build complex applications. You will learn how techniques such as Command Query Responsibility Segregation (CQRS) and Event Sourcing address the key challenges of developing applications with this architecture. We will also cover some of the various frameworks such as Spring Boot that you can use to implement micro-services.
Futures and Rx Observables: powerful abstractions for consuming web services ...Chris Richardson
A modular, polyglot architecture has many advantages but it also adds complexity since each incoming request typically fans out to multiple distributed services. For example, in an online store application the information on a product details page - description, price, recommendations, etc - comes from numerous services. To minimize response time and improve scalability, these services must be invoked concurrently. However, traditional concurrency mechanisms are low-level, painful to use and error-prone.
In this talk you will learn about some powerful yet easy to use abstractions for consuming web services asynchronously. We will compare the various implementations of futures that are available in Java, Scala and JavaScript. You will learn how to use reactive observables, which are asynchronous data streams, to access web services from both Java and JavaScript. We will describe how these mechanisms let you write asynchronous code in a very straightforward, declarative fashion.
Microservices are an essential enabler of agility but developing and deploying them is a challenge. In order for microservices to be loosely coupled,each service must have its own datastore. This makes it difficult to maintain data consistency across services.
Deploying microservices is also a complex problem since an application typically consists of 10s or 100s of services, written in a variety of languages and frameworks. In this presentation, you will learn how to solve these problems by using an event-driven architecture to maintain data consistency and by using Docker to simplify deployment.
There is no such thing as a microservice! (oracle code nyc)Chris Richardson
This is a keynote I gave at Oracle Code 2017 in New York:
https://developer.oracle.com/code/newyork
The microservice architecture is becoming increasingly popular. However, frequent references to using “a microservice to solve a problem” suggest that the concept is not universally well understood.
In this talk we define the microservice architecture as an architectural style and explain what that actually means. You will learn why the architecture that you pick for your application matters. We describe how the primary goal of the microservice architecture is to enable continuous delivery/deployment and how it achieves that.
You will learn how to solve key challenges with decomposing an application into microservices. We describe why there is no such thing as a microservice!
Microservices pattern language (microxchg microxchg2016)Chris Richardson
My talk from http://microxchg.io/2016/index.html.
Here is the video - https://www.youtube.com/watch?v=1mcVQhbkA2U
When architecting an enterprise Java application, you need to choose between the traditional monolithic architecture consisting of a single large WAR file, or the more fashionable microservices architecture consisting of many smaller services. But rather than blindly picking the familiar or the fashionable, it’s important to remember what Fred Books said almost 30 years ago: there are no silver bullets in software. Every architectural decision has both benefits and drawbacks. Whether the benefits of one approach outweigh the drawbacks greatly depends upon the context of your particular project. Moreover, even if you adopt the microservices architecture, you must still make numerous other design decisions, each with their own trade-offs.
A software pattern is an ideal way of describing a solution to a problem in a given context along with its tradeoffs. In this presentation, we describe a pattern language for microservices. You will learn about patterns that will help you decide when and how to use microservices vs. a monolithic architecture. We will also describe patterns that solve various problems in a microservice architecture including inter-service communication, service registration and service discovery.
YOW! Perth: Cubes, Hexagons, Triangles, and More: Understanding the Microserv...Chris Richardson
YOW! Perth 2019 presentation
The microservice architecture is becoming increasing important. But what is it exactly? Why should you care about microservices? And, what do you need to do to ensure that your organization uses the microservice architecture successfully? In this talk, I’ll answer these and other questions using shapes as visual metaphors. You will learn about the motivations for the microservice architecture and why simply adopting microservices is insufficient. I describe essential characteristics of microservices, You will learn how a successful microservice architecture consist of loosely coupled services with stable APIs that communicate asynchronous. I will cover strategies for effectively testing microservices.
Gluecon: Using sagas to maintain data consistency in a microservice architectureChris Richardson
The microservice architecture structures an application as a set of loosely coupled, collaborating services. Maintaining data consistency is challenging since each service has its own database to ensure loose coupling. To make matters worse, for a variety of reasons distributed transactions using JTA are not an option for modern applications.
In this talk we describe an alternative transaction model known as a saga. You will learn about the benefits and drawbacks of using sagas. We describe how sagas are eventually consistent rather than ACID and what this means for developers. You will learn how to design and implement sagas in a Java application.
ArchSummit Shenzhen - Using sagas to maintain data consistency in a microserv...Chris Richardson
This is a talk I gave at QCON ArchSummit in Shenzhen.
The microservice architecture structures an application as a set of loosely coupled, collaborating services. Maintaining data consistency is challenging since each service has its own database to ensure loose coupling. To make matters worse, for a variety of reasons distributed transactions using JTA are not an option for modern applications.
In this talk we describe an alternative transaction model known as a saga. You will learn about the benefits and drawbacks of using sagas. We describe how sagas are eventually consistent rather than ACID and what this means for developers. You will learn how to design and implement sagas in a Java application.
#JaxLondon keynote: Developing applications with a microservice architectureChris Richardson
The micro-service architecture, which structures an application as a set of small, narrowly focused, independently deployable services, is becoming an increasingly popular way to build applications. This approach avoids many of the problems of a monolithic architecture. It simplifies deployment and let’s you create highly scalable and available applications. In this keynote we describe the micro-service architecture and how to use it to build complex applications. You will learn how techniques such as Command Query Responsibility Segregation (CQRS) and Event Sourcing address the key challenges of developing applications with this architecture. We will also cover some of the various frameworks such as Spring Boot that you can use to implement micro-services.
A Pattern Language for Microservices (@futurestack)Chris Richardson
When architecting an application, you need to choose between the traditional monolithic architecture consisting of a single large application, or the more fashionable microservices architecture consisting of many smaller services. But rather than blindly picking the familiar or the fashionable, it's important to remember what Fred Books said almost 30 years ago: there are no silver bullets in software. Every architectural decision has both benefits and drawbacks. Whether the benefits of one approach outweigh the drawbacks greatly depends upon the context of your particular project. Moreover, even if you adopt the microservices architecture, you must still make numerous other design decisions, each with their own trade-offs.
Developing microservices with aggregates (SpringOne platform, #s1p)Chris Richardson
The Domain Model pattern is a great way to develop complex business logic. Unfortunately, a typical domain model is a tangled, birds nest of classes. It can’t be decomposed into microservices. Moreover, business logic often relies on ACID transactions to maintain consistency.
Fortunately, there is a solution to this problem: aggregates. An aggregate is an often overlooked modeling concept from the must read book Domain Driven Design. In this talk you will learn how aggregates enable you to develop business logic for the modern world of microservices and NoSQL. We will describe how to use aggregates to design modular business logic that can be partitioned into microservices. You will learn how aggregates enable you to use eventual consistency instead of ACID. We will describe the design of a microservice that is built using aggregates, and Spring Cloud.
Decompose that WAR? A pattern language for microservices (@QCON @QCONSP)Chris Richardson
When architecting an enterprise Java application, you need to choose between the traditional monolithic architecture consisting of a single large WAR file, or the more fashionable microservices architecture consisting of many smaller services. But rather than blindly picking the familiar or the fashionable, it's important to remember what Fred Books said almost 30 years ago: there are no silver bullets in software. Every architectural decision has both benefits and drawbacks. Whether the benefits of one approach outweigh the drawbacks greatly depends upon the context of your particular project. Moreover, even if you adopt the microservices architecture, you must still make numerous other design decisions, each with their own trade-offs.
A software pattern is an ideal way of describing a solution to a problem in a given context along with its tradeoffs. In this presentation, we describe a pattern language for microservices. You will learn about patterns that will help you decide when and how to use microservices vs. a monolithic architecture. We will also describe patterns that solve various problems in a microservice architecture including inter-service communication, service registration and service discovery.
NodeJS: the good parts? A skeptic’s view (jax jax2013)Chris Richardson
JavaScript used to be confined to the browser. But these days, it's becoming increasingly popular in server-side applications in the form of Node.js. Node.js provides event-driven, non-blocking I/O model that supposedly makes it easy to build scalable network application. In this talk you will learn about the consequences of combining the event-driven programming model with a prototype-based, weakly typed, dynamic language. We will share our perspective as a server-side Java developer who wasn’t entirely happy about JavaScript in the browser, let alone on the server. You will learn how to use Node.js effectively in modern, polyglot applications.
Watch the video: http://www.youtube.com/watch?v=CN0jTnSROsk&feature=youtu.be
Polyglot persistence for Java developers: time to move out of the relational ...Chris Richardson
Relational databases have long been considered the one true way to persist enterprise data. Even today, they are an excellent choice for many applications. But for some applications NoSQL databases are a viable alternative. They can simplify the persistence of complex data models and offer significantly better scalability, and performance. But using NoSQL databases is very different than the ACID/SQL/JDBC/JPA world that we have become accustomed to. They have different and unfamiliar APIs and a very different and usually limited transaction model. So what’s a Java developer to do?
We're exploring "Reactive Architectures" at Senacor Technologies. This is our view on this topic - why should we go reactive and how.
It might be less than in pure theory but we need a practical approach to this topic. One that fits our company and our customers. For both worlds, we can't just throw away the existing technical steps and our skillets.
Another focus of the talk is on a few topics which did hurt us in the past but usually nobody talks about them in reactive presentations.
Building microservices with Scala, functional domain models and Spring BootChris Richardson
In this talk you will learn about a modern way of designing applications that’s very different from the traditional approach of building monolithic applications that persist mutable domain objects in a relational database.We will talk about the microservice architecture, it’s benefits and drawbacks and how Spring Boot can help. You will learn about implementing business logic using functional, immutable domain models written in Scala. We will describe event sourcing and how it’s an extremely useful persistence mechanism for persisting functional domain objects in a microservices architecture.
Building microservices with Scala, functional domain models and Spring Boot (...Chris Richardson
In this talk you will learn about a modern way of designing applications that’s very different from the traditional approach of building monolithic applications that persist mutable domain objects in a relational database.We will talk about the microservice architecture, it’s benefits and drawbacks and how Spring Boot can help. You will learn about implementing business logic using functional, immutable domain models written in Scala. We will describe event sourcing and how it’s an extremely useful persistence mechanism for persisting functional domain objects in a microservices architecture.
Serverless architecture allows us to build and run applications with nearly no infrastructure configuration. Isn’t it a dream of a developer to concentrate on writing software and not be distracted by infrastructure duties?
CQRS and Event Sourcing are patterns which allow us to benefit from this approach.
In this talk we will dive into the world of Serverless computing for Java developers using Amazon Web Services and of course we will take a look at some existing pitfalls.
Building and deploying microservices with event sourcing, CQRS and Docker (QC...Chris Richardson
In this talk we share our experiences developing and deploying a microservices-based application. You will learn about the distributed data management challenges that arise in a microservices architecture. We will describe how we solved them using event sourcing to reliably publish events that drive eventually consistent workflows and pdate CQRS-based views. You will also learn how we build and deploy the application using a Jenkins-based deployment pipeline that creates Docker images that run on Amazon EC2.
Developing functional domain models with event sourcing (oakjug, sfscala)Chris Richardson
Event sourcing persists each entity as a sequence of state changing events. An entity’s current state is derived by replaying those events. Event sourcing is a great way to implement event-driven microservices. When one service updates an entity, the new events are consumed by other services, which then update their own state.
In this talk we describe how to implement business logic using event sourcing. You will learn how to write functional, immutable domain models in Scala. We will compare and contrast a hybrid OO/FP design with a purely functional approach.
Building and deploying microservices with event sourcing, CQRS and Docker (Ha...Chris Richardson
In this talk we share our experiences developing and deploying a microservices-based application. You will learn about the distributed data management challenges that arise in a microservices architecture. We will describe how we solved them using event sourcing to reliably publish events that drive eventually consistent workflows and pdate CQRS-based views. You will also learn how we build and deploy the application using a Jenkins-based deployment pipeline that creates Docker images that run on Amazon EC2.
Microservice Architecture with CQRS and Event SourcingBen Wilcock
In this slide deck I'll introduce you to the Command and Query Responsibility Segregation [CQRS] and Event Sourcing [ES] patterns for software architecture and explain why I think they're worth investing a little of your time in if you're building Microservices that need to scale gracefully.
Developing functional domain models with event sourcing (sbtb, sbtb2015)Chris Richardson
Event sourcing persists each entity as a sequence of state changing event. An entity’s current state is derived by replaying the events. Event sourcing is a great way to implement event-driven micro services. When one service updates an entity, the new events are consumed by other services, which then update their own state. In this talk we describe how to implement business logic using a domain model that is based on event sourcing. You will learn how to write functional, immutable domain models in Scala. We will compare and contrast a hybrid OO/FP design with a purely functional approach. You will learn how Domain Driven Design concepts such as bounded contexts and aggregates fit in with event-driven microservices.
Building and deploying microservices with event sourcing, CQRS and Docker (Me...Chris Richardson
In this talk we share our experiences developing and deploying a microservices-based application. You will learn about the distributed data management challenges that arise in a microservices architecture. We will describe how we solved them using event sourcing to reliably publish events that drive eventually consistent workflows and pdate CQRS-based views. You will also learn how we build and deploy the application using a Jenkins-based deployment pipeline that creates Docker images that run on Amazon EC2.
Developing and deploying applications with Spring Boot and Docker (@oakjug)Chris Richardson
This presentation was given at Oakjug.
Describes why Spring Boot is an excellent choice for building microservices.
Talks about the various ways that Docker can simplify development and deployment.
Discusses how docker-compose makes the life of a developer easier.
Map, Flatmap and Reduce are Your New Best Friends: Simpler Collections, Concu...Chris Richardson
Higher-order functions such as map(), flatmap(), filter() and reduce() have their origins in mathematics and ancient functional programming languages such as Lisp. But today they have entered the mainstream and are available in languages such as JavaScript, Scala and Java 8. They are well on their way to becoming an essential part of every developer’s toolbox.
In this talk you will learn how these and other higher-order functions enable you to write simple, expressive and concise code that solve problems in a diverse set of domains. We will describe how you use them to process collections in Java and Scala. You will learn how functional Futures and Rx (Reactive Extensions) Observables simplify concurrent code. We will even talk about how to write big data applications in a functional style using libraries such as Scalding.
Decomposing applications for deployability and scalability #springone2gx #s12gxChris Richardson
Today, there are several trends that are forcing application architectures to evolve. Users expect a rich, interactive and dynamic user experience on a wide variety of clients including mobile devices. Applications must be highly scalable, highly available and run on cloud environments. Organizations often want to frequently roll out updates, even multiple times a day. Consequently, it’s no longer adequate to develop simple, monolithic web applications that serve up HTML to desktop browsers.
In this talk we describe the limitations of a monolithic architecture. You will learn how to use the scale cube to decompose your application into a set of narrowly focused, independently deployable back-end services and an HTML 5 client. We will also discuss the role of technologies such as NodeJS and AMQP brokers. You will learn how a modern PaaS such as Cloud Foundry simplifies the development and deployment of this style of application.
A Pattern Language for Microservices (@futurestack)Chris Richardson
When architecting an application, you need to choose between the traditional monolithic architecture consisting of a single large application, or the more fashionable microservices architecture consisting of many smaller services. But rather than blindly picking the familiar or the fashionable, it's important to remember what Fred Books said almost 30 years ago: there are no silver bullets in software. Every architectural decision has both benefits and drawbacks. Whether the benefits of one approach outweigh the drawbacks greatly depends upon the context of your particular project. Moreover, even if you adopt the microservices architecture, you must still make numerous other design decisions, each with their own trade-offs.
Developing microservices with aggregates (SpringOne platform, #s1p)Chris Richardson
The Domain Model pattern is a great way to develop complex business logic. Unfortunately, a typical domain model is a tangled, birds nest of classes. It can’t be decomposed into microservices. Moreover, business logic often relies on ACID transactions to maintain consistency.
Fortunately, there is a solution to this problem: aggregates. An aggregate is an often overlooked modeling concept from the must read book Domain Driven Design. In this talk you will learn how aggregates enable you to develop business logic for the modern world of microservices and NoSQL. We will describe how to use aggregates to design modular business logic that can be partitioned into microservices. You will learn how aggregates enable you to use eventual consistency instead of ACID. We will describe the design of a microservice that is built using aggregates, and Spring Cloud.
Decompose that WAR? A pattern language for microservices (@QCON @QCONSP)Chris Richardson
When architecting an enterprise Java application, you need to choose between the traditional monolithic architecture consisting of a single large WAR file, or the more fashionable microservices architecture consisting of many smaller services. But rather than blindly picking the familiar or the fashionable, it's important to remember what Fred Books said almost 30 years ago: there are no silver bullets in software. Every architectural decision has both benefits and drawbacks. Whether the benefits of one approach outweigh the drawbacks greatly depends upon the context of your particular project. Moreover, even if you adopt the microservices architecture, you must still make numerous other design decisions, each with their own trade-offs.
A software pattern is an ideal way of describing a solution to a problem in a given context along with its tradeoffs. In this presentation, we describe a pattern language for microservices. You will learn about patterns that will help you decide when and how to use microservices vs. a monolithic architecture. We will also describe patterns that solve various problems in a microservice architecture including inter-service communication, service registration and service discovery.
NodeJS: the good parts? A skeptic’s view (jax jax2013)Chris Richardson
JavaScript used to be confined to the browser. But these days, it's becoming increasingly popular in server-side applications in the form of Node.js. Node.js provides event-driven, non-blocking I/O model that supposedly makes it easy to build scalable network application. In this talk you will learn about the consequences of combining the event-driven programming model with a prototype-based, weakly typed, dynamic language. We will share our perspective as a server-side Java developer who wasn’t entirely happy about JavaScript in the browser, let alone on the server. You will learn how to use Node.js effectively in modern, polyglot applications.
Watch the video: http://www.youtube.com/watch?v=CN0jTnSROsk&feature=youtu.be
Polyglot persistence for Java developers: time to move out of the relational ...Chris Richardson
Relational databases have long been considered the one true way to persist enterprise data. Even today, they are an excellent choice for many applications. But for some applications NoSQL databases are a viable alternative. They can simplify the persistence of complex data models and offer significantly better scalability, and performance. But using NoSQL databases is very different than the ACID/SQL/JDBC/JPA world that we have become accustomed to. They have different and unfamiliar APIs and a very different and usually limited transaction model. So what’s a Java developer to do?
We're exploring "Reactive Architectures" at Senacor Technologies. This is our view on this topic - why should we go reactive and how.
It might be less than in pure theory but we need a practical approach to this topic. One that fits our company and our customers. For both worlds, we can't just throw away the existing technical steps and our skillets.
Another focus of the talk is on a few topics which did hurt us in the past but usually nobody talks about them in reactive presentations.
Building microservices with Scala, functional domain models and Spring BootChris Richardson
In this talk you will learn about a modern way of designing applications that’s very different from the traditional approach of building monolithic applications that persist mutable domain objects in a relational database.We will talk about the microservice architecture, it’s benefits and drawbacks and how Spring Boot can help. You will learn about implementing business logic using functional, immutable domain models written in Scala. We will describe event sourcing and how it’s an extremely useful persistence mechanism for persisting functional domain objects in a microservices architecture.
Building microservices with Scala, functional domain models and Spring Boot (...Chris Richardson
In this talk you will learn about a modern way of designing applications that’s very different from the traditional approach of building monolithic applications that persist mutable domain objects in a relational database.We will talk about the microservice architecture, it’s benefits and drawbacks and how Spring Boot can help. You will learn about implementing business logic using functional, immutable domain models written in Scala. We will describe event sourcing and how it’s an extremely useful persistence mechanism for persisting functional domain objects in a microservices architecture.
Serverless architecture allows us to build and run applications with nearly no infrastructure configuration. Isn’t it a dream of a developer to concentrate on writing software and not be distracted by infrastructure duties?
CQRS and Event Sourcing are patterns which allow us to benefit from this approach.
In this talk we will dive into the world of Serverless computing for Java developers using Amazon Web Services and of course we will take a look at some existing pitfalls.
Building and deploying microservices with event sourcing, CQRS and Docker (QC...Chris Richardson
In this talk we share our experiences developing and deploying a microservices-based application. You will learn about the distributed data management challenges that arise in a microservices architecture. We will describe how we solved them using event sourcing to reliably publish events that drive eventually consistent workflows and pdate CQRS-based views. You will also learn how we build and deploy the application using a Jenkins-based deployment pipeline that creates Docker images that run on Amazon EC2.
Developing functional domain models with event sourcing (oakjug, sfscala)Chris Richardson
Event sourcing persists each entity as a sequence of state changing events. An entity’s current state is derived by replaying those events. Event sourcing is a great way to implement event-driven microservices. When one service updates an entity, the new events are consumed by other services, which then update their own state.
In this talk we describe how to implement business logic using event sourcing. You will learn how to write functional, immutable domain models in Scala. We will compare and contrast a hybrid OO/FP design with a purely functional approach.
Building and deploying microservices with event sourcing, CQRS and Docker (Ha...Chris Richardson
In this talk we share our experiences developing and deploying a microservices-based application. You will learn about the distributed data management challenges that arise in a microservices architecture. We will describe how we solved them using event sourcing to reliably publish events that drive eventually consistent workflows and pdate CQRS-based views. You will also learn how we build and deploy the application using a Jenkins-based deployment pipeline that creates Docker images that run on Amazon EC2.
Microservice Architecture with CQRS and Event SourcingBen Wilcock
In this slide deck I'll introduce you to the Command and Query Responsibility Segregation [CQRS] and Event Sourcing [ES] patterns for software architecture and explain why I think they're worth investing a little of your time in if you're building Microservices that need to scale gracefully.
Developing functional domain models with event sourcing (sbtb, sbtb2015)Chris Richardson
Event sourcing persists each entity as a sequence of state changing event. An entity’s current state is derived by replaying the events. Event sourcing is a great way to implement event-driven micro services. When one service updates an entity, the new events are consumed by other services, which then update their own state. In this talk we describe how to implement business logic using a domain model that is based on event sourcing. You will learn how to write functional, immutable domain models in Scala. We will compare and contrast a hybrid OO/FP design with a purely functional approach. You will learn how Domain Driven Design concepts such as bounded contexts and aggregates fit in with event-driven microservices.
Building and deploying microservices with event sourcing, CQRS and Docker (Me...Chris Richardson
In this talk we share our experiences developing and deploying a microservices-based application. You will learn about the distributed data management challenges that arise in a microservices architecture. We will describe how we solved them using event sourcing to reliably publish events that drive eventually consistent workflows and pdate CQRS-based views. You will also learn how we build and deploy the application using a Jenkins-based deployment pipeline that creates Docker images that run on Amazon EC2.
Developing and deploying applications with Spring Boot and Docker (@oakjug)Chris Richardson
This presentation was given at Oakjug.
Describes why Spring Boot is an excellent choice for building microservices.
Talks about the various ways that Docker can simplify development and deployment.
Discusses how docker-compose makes the life of a developer easier.
Map, Flatmap and Reduce are Your New Best Friends: Simpler Collections, Concu...Chris Richardson
Higher-order functions such as map(), flatmap(), filter() and reduce() have their origins in mathematics and ancient functional programming languages such as Lisp. But today they have entered the mainstream and are available in languages such as JavaScript, Scala and Java 8. They are well on their way to becoming an essential part of every developer’s toolbox.
In this talk you will learn how these and other higher-order functions enable you to write simple, expressive and concise code that solve problems in a diverse set of domains. We will describe how you use them to process collections in Java and Scala. You will learn how functional Futures and Rx (Reactive Extensions) Observables simplify concurrent code. We will even talk about how to write big data applications in a functional style using libraries such as Scalding.
Decomposing applications for deployability and scalability #springone2gx #s12gxChris Richardson
Today, there are several trends that are forcing application architectures to evolve. Users expect a rich, interactive and dynamic user experience on a wide variety of clients including mobile devices. Applications must be highly scalable, highly available and run on cloud environments. Organizations often want to frequently roll out updates, even multiple times a day. Consequently, it’s no longer adequate to develop simple, monolithic web applications that serve up HTML to desktop browsers.
In this talk we describe the limitations of a monolithic architecture. You will learn how to use the scale cube to decompose your application into a set of narrowly focused, independently deployable back-end services and an HTML 5 client. We will also discuss the role of technologies such as NodeJS and AMQP brokers. You will learn how a modern PaaS such as Cloud Foundry simplifies the development and deployment of this style of application.
Events on the outside, on the inside and at the core (jfokus jfokus2016)Chris Richardson
This is the talk I gave at JFokus 2016 on event-driven microservices.
This presentation looks at the importance of events and the role that they play in applications. We describe how events are a key application integration mechanism and how they are used by applications to communicate with the outside world. You will learn how the microservices inside a system can use events to maintain data consistency. We discuss how easy it is to implement both of these mechanisms by developing your core business logic using an event-centric approach known as event sourcing.
Building and deploying microservices with event sourcing, CQRS and Docker (Be...Chris Richardson
In this talk we share our experiences developing and deploying a microservices-based application. You will learn about the distributed data management challenges that arise in a microservices architecture. We will describe how we solved them using event sourcing to reliably publish events that drive eventually consistent workflows and pdate CQRS-based views. You will also learn how we build and deploy the application using a Jenkins-based deployment pipeline that creates Docker images that run on Amazon EC2.
This talk was given at the Berlin Microxchg conference and the Munich microservices meetup.
Slides: Polyglot Persistence for the MongoDB, MySQL & PostgreSQL DBASeveralnines
Polyglot Persistence for the MongoDB, PostgreSQL & MySQL DBA
The introduction of DevOps in organisations has changed the development process, and perhaps introduced some challenges. Developers, in addition to their own preferred programming languages, also have their own preference for backend storage.The former is often referred to as polyglot languages and the latter as polyglot persistence.
Having multiple storage backends means your organization will become more agile on the development side and allows choice to the developers but it also imposes additional knowledge on the operations side. Extending your infrastructure from only MySQL, to deploying other storage backends like MongoDB and PostgreSQL, implies you have to also monitor, manage and scale them. As every storage backend excels at different use cases, this also means you have to reinvent the wheel for every one of them.
This webinar covers the four major operational challenges for MySQL, MongoDB & PostgreSQL:
Deployment
Management
Monitoring
Scaling
And how to deal with them
SPEAKER
Art van Scheppingen is a Senior Support Engineer at Severalnines. He’s a pragmatic MySQL and Database expert with over 15 years experience in web development. He previously worked at Spil Games as Head of Database Engineering, where he kept a broad vision upon the whole database environment: from MySQL to Couchbase, Vertica to Hadoop and from Sphinx Search to SOLR. He regularly presents his work and projects at various conferences (Percona Live, FOSDEM) and related meetups.
This webinar is based upon the experience Art had while writing our How to become a ClusterControl DBA blog series and implementing multiple storage backends to ClusterControl. To view all the blogs of the ‘Become a ClusterControl DBA’ series visit: http://severalnines.com/blog-categories/clustercontrol
Polygot persistence for Java Developers - August 2011 / @OakjugChris Richardson
Relational databases have long been considered the one true way to persist enterprise data. But today, NoSQL databases are emerging as a viable alternative for many applications. They can simplify the persistence of complex data models and offer significantly better scalability, and performance. But NoSQL databases are very different than the ACID/SQL/JDBC/JPA world that we have become accustomed to. In this presentation, you will learn about our experience implementing a use case from POJOs in Action using popular NoSQL databases: Redis, MongoDB, and Cassandra. We will compare and contrast each database’s data model and Java API. You will learn about the benefits and drawbacks of using NoSQL.
NoSQL databases such as Redis, MongoDB and Cassandra are emerging as a compelling choice for many applications. They can simplify the persistence of complex data models and offer significantly better scalability and performance. However, using a NoSQL database means giving up the benefits of the relational model such as SQL, constraints and ACID transactions. For some applications, the solution is polyglot persistence: using SQL and NoSQL databases together.
In this talk, you will learn about the benefits and drawbacks of polyglot persistence and how to design applications that use this approach. We will explore the architecture and implementation of an example application that uses MySQL as the system of record and Redis as a very high-performance database that handles queries from the front-end. You will learn about mechanisms for maintaining consistency across the various databases.
Developing applications with Cloud Services (Devnexus 2013)Chris Richardson
Cloud computing isn’t just about application deployment. There are also a growing number of cloud-based web services that you can use to develop your application. One of the most well known is Amazon’s Simple Storage Service. But there are many others including web services for messaging, relational and NoSQL databases, email and telephony. Using these services allows you to build highly scalable applications without the pain and cost of having to develop and operate your own infrastructure.
In this presentation, you will learn about the benefits and drawbacks of these Web services; their typical use cases and how to use them. We will describe a location aware, telephony application that is built using cloud services. You will learn about strategies for building resilient, fault tolerant applications that consume cloud services.
Developing applications with Cloud Services #javaone 2012Chris Richardson
Cloud computing isn't just about application deployment. There are also a growing number of cloud-based web services that you can use to develop your application. One of the most well known is Amazon's Simple Storage Service. But there are many others including web services for messaging, relational and NoSQL databases, email and telephony. Using these services allows you to build highly scalable applications without the pain and cost of having to develop and operate your own infrastructure.
In this presentation, you will learn about the benefits and drawbacks of these Web services; their typical use cases and how to use them. We will describe a location aware, telephony application that is built using cloud services. You will learn about strategies for building resilient, fault tolerant applications that consume cloud services.
Developing polyglot applications on Cloud Foundry (#oredev 2012)Chris Richardson
Developing web applications used to be simple. Your single war-file web application served up HTML to a desktop browser and used a relational database. Today however, web applications are much more complex: the front-end uses HTML5 and NodeJS, the middle tier is decomposed into multiple services, and the back-end uses a mix of SQL and NoSQL databases. Developing these kind of applications can be challenging since there are so many moving parts that need to be correctly installed and configured. Deployment is even more difficult.
In this talk, you will learn why we need to build applications with this architectural style and how Cloud Foundry, which is modern, multi-lingual, multi-service, extensible open-source PaaS, can help. We will talk about how to develop modern applications that run on Cloud Foundry and cover what’s new and different about the cloud environment. You will learn how your application can consume the various services that are provided by Cloud Foundry. We will discuss the various ways of using Cloud Foundry including the Micro Cloud that runs on a laptop as well as the hosted CloudFoundry.com.
NoSQL databases such as Redis, MongoDB and Cassandra are emerging as a compelling choice for many applications. They can simplify the persistence of complex data models and offer significantly better scalability and performance. However, using a NoSQL database means giving up the benefits of the relational model such as SQL, constraints and ACID transactions. For some applications, the solution is polyglot persistence: using SQL and NoSQL databases together.
In this talk, you will learn about the benefits and drawbacks of polyglot persistence and how to design applications that use this approach. We will explore the architecture and implementation of an example application that uses MySQL as the system of record and Redis as a very high-performance database that handles queries from the front-end. You will learn about mechanisms for maintaining consistency across the various databases.
Decomposing applications for scalability and deployability - svcc sv_code_ca...Chris Richardson
Today, there are several trends that are forcing application architectures to evolve. Users expect a rich, interactive and dynamic user experience on a wide variety of clients including mobile devices. Applications must be highly scalable, highly available and run on cloud environments. Organizations often want to frequently roll out updates, even multiple times a day. Consequently, it’s no longer adequate to develop simple, monolithic web applications that serve up HTML to desktop browsers.
In this talk we describe the limitations of a monolithic architecture. You will learn how to use the scale cube to decompose your application into a set of narrowly focused, independently deployable back-end services and an HTML 5 client. We will also discuss the role of technologies such as NodeJS and AMQP brokers. You will learn how a modern PaaS such as Cloud Foundry simplifies the development and deployment of this style of application.
Polyglot Persistence - Two Great Tastes That Taste Great TogetherJohn Wood
The days of the relational database being a one-stop-shop for all of your persistence needs are over. Although NoSQL databases address some issues that can’t be addressed by relational databases, the opposite is true as well. The relational database offers an unparalleled feature set and rock solid stability. One cannot underestimate the importance of using the right tool for the job, and for some jobs, one tool is not enough. This talk focuses on the strength and weaknesses of both relational and NoSQL databases, the benefits and challenges of polyglot persistence, and examples of polyglot persistence in the wild.
These slides were presented at WindyCityDB 2010.
Decomposing applications for scalability and deployability (devnexus 2013)Chris Richardson
Today, there are several trends that are forcing application architectures to evolve. Users expect a rich, interactive and dynamic user experience on a wide variety of clients including mobile devices. Applications must be highly scalable, highly available and run on cloud environments. Organizations often want to frequently roll out updates, even multiple times a day. Consequently, it’s no longer adequate to develop simple, monolithic web applications that serve up HTML to desktop browsers.
In this talk we describe the limitations of a monolithic architecture. You will learn how to use the scale cube to decompose your application into a set of narrowly focused, independently deployable back-end services and an HTML 5 client. We will also discuss the role of technologies such as NodeJS and AMQP brokers. You will learn how a modern PaaS such as Cloud Foundry simplifies the development and deployment of this style of application.
Improving application design with a rich domain model (springone 2007)Chris Richardson
A classic from 2007. This is a presentationthat I gave at SpringOne in Antwerp, Belgium. It describes show to improve application design by using a rich domain model
The rise of NoSQL is characterized with confusion and ambiguity; very much like any fast-emerging organic movement in the absence of well-defined standards and adequate software solutions. Whether you are a developer or an architect, many questions come to mind when faced with the decision of where your data should be stored and how it should be managed. The following are some of these questions: What does the rise of all these NoSQL technologies mean to my enterprise? What is NoSQL to begin with? Does it mean "No SQL"? Could this be just another fad? Is it a good idea to bet the future of my enterprise on these new exotic technologies and simply abandon proven mature Relational DataBase Management Systems (RDBMS)? How scalable is scalable? Assuming that I am sold, how do I choose the one that fit my needs best? Is there a middle ground somewhere? What is this Polyglot Persistence I hear about? The answers to these questions and many more is the subject of this talk along with a survey of the most popular of NoSQL technologies. Be there or be square.
NoSQL databases such as Redis, MongoDB and Cassandra are emerging as a compelling choice for many applications. They can simplify the persistence of complex data models and offer significantly better scalability and performance. However, using a NoSQL database means giving up the benefits of the relational model such as SQL, constraints and ACID transactions. For some applications, the solution is polyglot persistence: using SQL and NoSQL databases together.
In this talk, you will learn about the benefits and drawbacks of polyglot persistence and how to design applications that use this approach. We will explore the architecture and implementation of an example application that uses MySQL as the system of record and Redis as a very high-performance database that handles queries from the front-end. You will learn about mechanisms for maintaining consistency across the various databases.
NoSQL databases such as Redis, MongoDB and Cassandra are emerging as a compelling choice for many applications. They can simplify the persistence of complex data models and offer significantly better scalability and performance. However, using a NoSQL database means giving up the benefits of the relational model such as SQL, constraints and ACID transactions. For some applications, the solution is polyglot persistence: using SQL and NoSQL databases together. In this talk, you will learn about the benefits and drawbacks of polyglot persistence and how to design applications that use this approach. We will explore the architecture and implementation of an example application that uses MySQL as the system of record and Redis as a very high-performance database that handles queries from the front-end. You will learn about mechanisms for maintaining consistency across the various databases.
Developing applications with Cloud Services (jax jax2013)Chris Richardson
Cloud computing isn't just about application deployment. There are also a growing number of cloud-based web services that you can use to develop your application. One of the most well known is Amazon's Simple Storage Service. But there are many others including web services for messaging, relational and NoSQL databases, email and telephony. Using these services allows you to build highly scalable applications without the pain and cost of having to develop and operate your own infrastructure. In this presentation, you will learn about the benefits and drawbacks of these Web services; their typical use cases and how to use them. We will describe a location aware, telephony application that is built using cloud services. You will learn about strategies for building resilient, fault tolerant applications that consume cloud services.
Solving distributed data management problems in a microservice architecture (...Chris Richardson
This is a talk that I gave the SFBay Area Microservices and Cloud Native Apps meetup: https://www.meetup.com/microservices/events/241894841/
In a microservice architecture, each service has a private datastore in order to ensure that the services are loosely coupled. While loose coupling is essential, this architecture makes managing transactions and querying difficult. You can’t use traditional distributed transactions. Nor can you use distributed queries. In this talk you will learn about some patterns for solving these problems: Sagas, API Composition and Command Query Responsibility Segregation (CQRS).
Handling Eventual Consistency in JVM Microservices with Event Sourcing (javao...Chris Richardson
This is the talk that the Kenny Bastani and I gave at JavaOne 2016.
When you’re building JVM applications in a microservice architecture, managing state becomes a distributed systems problem. Instead of being able to manage state as transactions inside the boundaries of a single monolithic application, a microservice must be able to manage consistency by using transactions that are distributed across a network of many different applications and databases. This session explores the problems of data consistency and high availability in JVM-based microservices and how to use event sourcing to solve these problems.
Microservices + Events + Docker = A Perfect Trio by Docker Captain Chris Rich...Docker, Inc.
Microservices are an essential enabler of agility but developing and deploying them is a challenge. In order for microservices to be loosely coupled,each service must have its own datastore. This makes it difficult to maintain data consistency across services.
Deploying microservices is also a complex problem since an application typically consists of 10s or 100s of services, written in a variety of languages and frameworks.
In this presentation, you will learn how to solve these problems by using an event-driven architecture to maintain data consistency and by using Docker to simplify deployment.
QConPlus 2021: Minimizing Design Time Coupling in a Microservice ArchitectureChris Richardson
Delivering large, complex software rapidly, frequently and reliably requires a loosely coupled organization. DevOps teams should rarely need to communicate and coordinate in order to get work done. Conway's law states that an organization and the architecture that it develops mirror one another. Hence, a loosely coupled organization requires a loosely coupled architecture.
In this presentation, you will learn about design-time coupling in a microservice architecture and why it's essential to minimize it. I describe how to design service APIs to reduce coupling. You will learn how to minimize design-time coupling by applying a version of the DRY principle. I describe how key microservices patterns potentially result in tight design time coupling and how to avoid it.
YOW London - Considering Migrating a Monolith to Microservices? A Dark Energy...Chris Richardson
This is a talk I gave at YOW! London 2022.
Let's imagine that you are responsible for an aging monolithic application that's critical to your business. Sadly, getting changes into production is a painful ordeal that regularly causes outages. And to make matters worse, the application's technology stack is growing increasingly obsolete. Neither the business nor the developers are happy. You need to modernize your application and have read about the benefits of microservices. But is the microservice architecture a good choice for your application?
In this presentation, I describe the dark energy and dark matter forces (a.k.a. concerns) that you must consider when deciding between the monolithic and microservice architectural styles. You will learn about how well each architectural style resolves each of these forces. I describe how to evaluate the relative importance of each of these forces to your application. You will learn how to use the results of this evaluation to decide whether to migrate to the microservice architecture.
[Meetup] a successful migration from elastic search to clickhouseVianney FOUCAULT
Paris Clickhouse meetup 2019: How Contentsquare successfully migrated to Clickhouse !
Discover the subtleties of a migration to Clickhouse. What to check before hand, then how to operate clickhouse in Production
How Nationwide and Tasktop Achieved Continuous Visibility Across the DevOps L...Tasktop
During this on-demand webinar, Tasktop CEO Dr. Mik Kersten, and Nationwide Technology Director, Carmen DeArdo, overview the conceptual framework that Nationwide and Tasktop created to provide end-to-end “continuous visibility” across the DevOps lifecycle.
JFokus: Cubes, Hexagons, Triangles, and More: Understanding MicroservicesChris Richardson
The microservice architecture is becoming increasing important. But what is it exactly? Why should you care about microservices? And, what do you need to do to ensure that your organization uses the microservice architecture successfully? In this talk, I’ll answer these and other questions using shapes as visual metaphors. You will learn about the motivations for the microservice architecture and why simply adopting microservices is insufficient. I describe essential characteristics of microservices, You will learn how a successful microservice architecture consist of loosely coupled services with stable APIs that communicate asynchronous. I will cover strategies for effectively testing microservices.
InfluxQL is a powerful query language for InfluxDB, and TICKScript is a domain specific language used by Kapacitor to define tasks involving the extraction, transformation and loading of data and also involving the tracking of arbitrary changes and detection of events within data. The combination of these two can make your monitoring apps powerful. During this session, InfluxData Engineer Michael DeSa will share best practices for using these powerful tools. Prerequisite: Intro To Kapacitor.
Saturn 2018: Managing data consistency in a microservice architecture using S...Chris Richardson
A revised and extended version that I gave at Saturn 2018.
The services in a microservice architecture must be loosely coupled and so cannot share database tables. What’s more, two phase commit (a.k.a. a distributed transaction) is not a viable option for modern applications. Consequently, a microservices application must use the Saga pattern, which maintains data consistency using a series of local transactions.
In this presentation, you will learn how sagas work and how they differ from traditional transactions. We describe how to use sagas to develop business logic in a microservices application. You will learn effective techniques for orchestrating sagas and how to use messaging for reliability. We will describe the design of a saga framework for Java and show a sample application.
OReilly SACON2018 - Events on the outside, on the inside, and at the coreChris Richardson
Events are very much on the edge of traditional applications, which use them as an application integration mechanism. The classic example is an ecommerce system. When a customer places an order, the order management application publishes an event, which triggers the fulfillment application to action. But today, microservices and DDD—which is a great foundation for microservices—are at the core of the application.
Events play an essential role in modern applications. Chris Richardson explains why events are a key application integration mechanism and how they are used by applications to communicate with the outside world. You’ll learn how the microservices inside an application use events to maintain data consistency and discover how to go one step further and make events an integral part of your domain logic.
Building and Deploying Microservices with Event Sourcing, CQRS and DockerC4Media
Video and slides synchronized, mp3 and slide download available at URL http://bit.ly/1Qyjicw.
Chris Richardson shares his experiences developing and deploying a microservices-based application. Filmed at qconsf.com.
Chris Richardson is a developer and architect. He is a Java Champion and the author of POJOs in Action, which describes how to build enterprise Java applications with frameworks such as Spring and Hibernate.
Back to the future : SQL 92 for Elasticsearch ? @nosqlmatters Dublin 2014Lucian Precup
What if we would try to make Elasticsearch SQL 92 compliant (http://www.contrib.andrew.cmu.edu/~shadow/sql/sql1992.txt)? This wouldn't serve that much nowadays, you would say. Well, we actually tried to do the exercise and we have some interesting conclusions. While we take Elasticsearch as an example for this "side by side", the issues we are addressing also apply to nosql in general. With this unusual exercise, we take the occasion to compare relational databases / sql with Elasticsearch / nosql on all the levels : functionality, semantics, performance and user experience.
Developing event-driven microservices with event sourcing and CQRS (svcc, sv...Chris Richardson
Modern, cloud-native applications typically use a microservices architecture in conjunction with NoSQL and/or sharded relational databases. However, in order to successfully use this approach you need to solve some distributed data management problems including how to maintain consistency between multiple databases without using 2PC.
In this talk you will learn more about these issues and how to solve them by using an event-driven architecture. We will describe how event sourcing and Command Query Responsibility Segregation (CQRS) are a great way to realize an event-driven architecture. You will learn about a simple yet powerful approach for building, modern, scalable applications.
Similar to Developing polyglot persistence applications (gluecon 2013) (20)
A common microservice architecture anti-pattern is more the merrier. It occurs when an organization team builds an excessively fine-grained architecture, e.g. one service-per-developer. In this talk, you will learn about the criteria that you should consider when deciding service granularity. I'll discuss the downsides of a fine-grained microservice architecture. You will learn how sometimes the solution to a design problem is simply a JAR file.
Dark Energy, Dark Matter and the Microservices Patterns?!Chris Richardson
Dark matter and dark energy are mysterious concepts from astrophysics that are used to explain observations of distant stars and galaxies. The Microservices pattern language - a collection of patterns that solve architecture, design, development, and operational problems — enables software developers to use the microservice architecture effectively. But how could there possibly be a connection between microservices and these esoteric concepts from astrophysics?
In this presentation, I describe how dark energy and dark matter are excellent metaphors for the competing forces (a.k.a. concerns) that must be resolved by the microservices pattern language. You will learn that dark energy, which is an anti-gravity, is a metaphor for the repulsive forces that encourage decomposition into services. I describe how dark matter, which is an invisible matter that has a gravitational effect, is a metaphor for the attractive forces that resist decomposition and encourage the use of a monolithic architecture. You will learn how to use the dark energy and dark matter forces as guide when designing services and operations.
Dark energy, dark matter and microservice architecture collaboration patternsChris Richardson
Dark energy and dark matter are useful metaphors for the repulsive forces, which encourage decomposition into services, and the attractive forces, which resist decomposition. You must balance these conflicting forces when defining a microservice architecture including when designing system operations (a.k.a. requests) that span services.
In this talk, I describe the dark energy and dark matter forces. You will learn how to design system operations that span services using microservice architecture collaboration patterns: Saga, Command-side replica, API composition, and CQRS patterns. I describe how each of these patterns resolve the dark energy and dark matter forces differently.
It sounds dull but good architecture documentation is essential. Especially when you are actively trying to improve your architecture.
For example, I spend a lot time helping clients modernize their software architecture. More often than I like, I’m presented with a vague and lifeless collection of boxes and lines. As a result, it’s sometimes difficult to discuss the architecture in a meaningful and productive way. In this presentation, I’ll describe techniques for creating minimal yet effective documentation for your application’s microservice architecture. In particular, you will learn how documenting scenarios can bring your architecture to life.
Using patterns and pattern languages to make better architectural decisions Chris Richardson
This is a presentation that gave at the O'Reilly Software Architecture Superstream: Software Architecture Patterns.
The talk's focus is the microservices pattern language.
However, it also shows how thinking with the pattern mindset - context/problem/forces/solution/consequences - leads to better technically decisions.
The microservices architecture offers tremendous benefits, but it’s not a silver bullet. It also has some significant drawbacks. The microservices pattern language—a collection of patterns that solve architecture, design, development, and operational problems—enables software developers to apply the microservices architecture effectively. I provide an overview of the microservices architecture and examines the motivations for the pattern language, then takes you through the key patterns in the pattern language.
Rapid, reliable, frequent and sustainable software development requires an architecture that is loosely coupled and modular.
Teams need to be able complete their work with minimal coordination and communication with other teams.
They also need to be able keep the software’s technology stack up to date.
However, the microservice architecture isn’t always the only way to satisfy these requirements.
Yet, neither is the monolithic architecture.
In this talk, I describe loose coupling and modularity and why they are is essential.
You will learn about three architectural patterns: traditional monolith, modular monolith and microservices.
I describe the benefits, drawbacks and issues of each pattern and how well it supports rapid, reliable, frequent and sustainable development.
You will learn some heuristics for selecting the appropriate pattern for your application.
Events to the rescue: solving distributed data problems in a microservice arc...Chris Richardson
To deliver a large complex application rapidly, frequently and reliably, you often must use the microservice architecture.
The microservice architecture is an architectural style that structures the application as a collection of loosely coupled services.
One challenge with using microservices is that in order to be loosely coupled each service has its own private database.
As a result, implementing transactions and queries that span services is no longer straightforward.
In this presentation, you will learn how event-driven microservices address this challenge.
I describe how to use sagas, which is an asynchronous messaging-based pattern, to implement transactions that span services.
You will learn how to implement queries that span services using the CQRS pattern, which maintain easily queryable replicas using events.
A pattern language for microservices - June 2021 Chris Richardson
The microservice architecture is growing in popularity. It is an architectural style that structures an application as a set of loosely coupled services that are organized around business capabilities. Its goal is to enable the continuous delivery of large, complex applications. However, the microservice architecture is not a silver bullet and it has some significant drawbacks.
The goal of the microservices pattern language is to enable software developers to apply the microservice architecture effectively. It is a collection of patterns that solve architecture, design, development and operational problems. In this talk, I’ll provide an overview of the microservice architecture and describe the motivations for the pattern language. You will learn about the key patterns in the pattern language.
Mucon 2021 - Dark energy, dark matter: imperfect metaphors for designing micr...Chris Richardson
In order to explain certain astronomical observations, physicists created the mysterious concepts of dark energy and dark matter.
Dark energy is a repulsive force.
It’s an anti-gravity that is forcing matter apart and accelerating the expansion of the universe.
Dark matter has the opposite attraction effect.
Although it’s invisible, dark matter has a gravitational effect on stars and galaxies.
In this presentation, you will learn how these metaphors apply to the microservice architecture.
I describe how there are multiple repulsive forces that drive the decomposition of your application into services.
You will learn, however, that there are also multiple attractive forces that resist decomposition and bind software elements together.
I describe how as an architect you must find a way to balance these opposing forces.
Skillsmatter CloudNative eXchange 2020
The microservice architecture is a key part of cloud native.
An essential principle of the microservice architecture is loose coupling.
If you ignore this principle and develop tightly coupled services the result will mostly likely be yet another "microservices failure story”.
Your application will be brittle and have all of disadvantages of both the monolithic and microservice architectures.
In this talk you will learn about the different kinds of coupling and how to design loosely coupled microservices.
I describe how to minimize design time and increase the productivity of your DevOps teams.
You will learn how how to reduce runtime coupling and improve availability.
I describe how to improve availability by minimizing the coupling caused by your infrastructure.
DDD SoCal: Decompose your monolith: Ten principles for refactoring a monolith...Chris Richardson
This is a talk I gave at DDD SoCal.
1. Make the most of your monolith
2. Adopt microservices for the right reasons
3. It’s not just architecture
4. Get the support of the business
5. Migrate incrementally
6. Know your starting point
7. Begin with the end in mind
8. Migrate high-value modules first
9. Success is improved velocity and reliability
10. If it hurts, don’t do it
Decompose your monolith: Six principles for refactoring a monolith to microse...Chris Richardson
This was a talk I gave at the CTO virtual summit on July 28th. It describes 6 principles for refactoring to a microservice architecture.
1. Make the most of your monolith
2. Adopt microservices for the right reasons
3. Migrate incrementally
4. Begin with the end in mind
5. Migrate high-value modules first
6. Success is improved velocity and reliability
The microservice architecture is becoming increasingly important. But what is it exactly? Why should you care about microservices? And, what do you need to do to ensure that your organization uses the microservice architecture successfully? In this talk, I’ll answer these and other questions. You will learn about the motivations for the microservice architecture and why simply adopting microservices is insufficient. I describe essential characteristics of microservices, You will learn how a successful microservice architecture consists of loosely coupled services with stable APIs that communicate asynchronously.
Decompose your monolith: strategies for migrating to microservices (Tide)Chris Richardson
This is a presentation that I gave at Tide.co, London - January 2020
A typical mission-critical enterprise application is a large, complex monolith developed by large team. Software delivery is usually slow, and the team struggles to keep up with the demands of the business. Consequently, many enterprise applications are good candidates to be migrated to the microservice architecture. But how do you know whether it makes sense to migrate to microservices? And, how to get there? In this presentation, I describe when you should consider migrating to microservices. You will learn strategies for migrating a monolith application to a microservice architecture. I explain how to implement new functionality as services. You will learn how to incrementally break apart a monolith one service at a time.
The primary goal of the microservice architecture is to enable the rapid, reliable delivery of software with DevOps. One of the pillars of DevOps is automated testing, yet many organizations attempt to adopt microservices while still doing manual testing. What’s more, the microservice architecture has its own distinctive automated testing challenges.
This presentation describes how to descend the testing pyramid and replace slow, brittle, end-to-end tests with faster, more reliable tests for individual services. You will learn how to write tests that ensure that service APIs evolve while preserving backward compatibility. You’ll learn how, by running these tests in a deployment pipeline, you will fully benefit from microservices.
Oracle CodeOne 2019: Decompose Your Monolith: Strategies for Migrating to Mic...Chris Richardson
A typical mission-critical enterprise application is a large, complex monolith developed by a large team. Software delivery is usually slow, and the team struggles to keep up with the demands of the business. Consequently, many enterprise applications are good candidates to be migrated to the microservice architecture. But how do you know whether it makes sense to migrate to microservices and how to get there?
This session describes when you should consider migrating to microservices. You will learn strategies for migrating a monolith application to a microservice architecture. The presentation explains how to implement new functionality as services, and you will also learn how to incrementally break apart a monolith, one service at a time.
UiPath Test Automation using UiPath Test Suite series, part 4DianaGray10
Welcome to UiPath Test Automation using UiPath Test Suite series part 4. In this session, we will cover Test Manager overview along with SAP heatmap.
The UiPath Test Manager overview with SAP heatmap webinar offers a concise yet comprehensive exploration of the role of a Test Manager within SAP environments, coupled with the utilization of heatmaps for effective testing strategies.
Participants will gain insights into the responsibilities, challenges, and best practices associated with test management in SAP projects. Additionally, the webinar delves into the significance of heatmaps as a visual aid for identifying testing priorities, areas of risk, and resource allocation within SAP landscapes. Through this session, attendees can expect to enhance their understanding of test management principles while learning practical approaches to optimize testing processes in SAP environments using heatmap visualization techniques
What will you get from this session?
1. Insights into SAP testing best practices
2. Heatmap utilization for testing
3. Optimization of testing processes
4. Demo
Topics covered:
Execution from the test manager
Orchestrator execution result
Defect reporting
SAP heatmap example with demo
Speaker:
Deepak Rai, Automation Practice Lead, Boundaryless Group and UiPath MVP
Builder.ai Founder Sachin Dev Duggal's Strategic Approach to Create an Innova...Ramesh Iyer
In today's fast-changing business world, Companies that adapt and embrace new ideas often need help to keep up with the competition. However, fostering a culture of innovation takes much work. It takes vision, leadership and willingness to take risks in the right proportion. Sachin Dev Duggal, co-founder of Builder.ai, has perfected the art of this balance, creating a company culture where creativity and growth are nurtured at each stage.
"Impact of front-end architecture on development cost", Viktor TurskyiFwdays
I have heard many times that architecture is not important for the front-end. Also, many times I have seen how developers implement features on the front-end just following the standard rules for a framework and think that this is enough to successfully launch the project, and then the project fails. How to prevent this and what approach to choose? I have launched dozens of complex projects and during the talk we will analyze which approaches have worked for me and which have not.
Slack (or Teams) Automation for Bonterra Impact Management (fka Social Soluti...Jeffrey Haguewood
Sidekick Solutions uses Bonterra Impact Management (fka Social Solutions Apricot) and automation solutions to integrate data for business workflows.
We believe integration and automation are essential to user experience and the promise of efficient work through technology. Automation is the critical ingredient to realizing that full vision. We develop integration products and services for Bonterra Case Management software to support the deployment of automations for a variety of use cases.
This video focuses on the notifications, alerts, and approval requests using Slack for Bonterra Impact Management. The solutions covered in this webinar can also be deployed for Microsoft Teams.
Interested in deploying notification automations for Bonterra Impact Management? Contact us at sales@sidekicksolutionsllc.com to discuss next steps.
Accelerate your Kubernetes clusters with Varnish CachingThijs Feryn
A presentation about the usage and availability of Varnish on Kubernetes. This talk explores the capabilities of Varnish caching and shows how to use the Varnish Helm chart to deploy it to Kubernetes.
This presentation was delivered at K8SUG Singapore. See https://feryn.eu/presentations/accelerate-your-kubernetes-clusters-with-varnish-caching-k8sug-singapore-28-2024 for more details.
State of ICS and IoT Cyber Threat Landscape Report 2024 previewPrayukth K V
The IoT and OT threat landscape report has been prepared by the Threat Research Team at Sectrio using data from Sectrio, cyber threat intelligence farming facilities spread across over 85 cities around the world. In addition, Sectrio also runs AI-based advanced threat and payload engagement facilities that serve as sinks to attract and engage sophisticated threat actors, and newer malware including new variants and latent threats that are at an earlier stage of development.
The latest edition of the OT/ICS and IoT security Threat Landscape Report 2024 also covers:
State of global ICS asset and network exposure
Sectoral targets and attacks as well as the cost of ransom
Global APT activity, AI usage, actor and tactic profiles, and implications
Rise in volumes of AI-powered cyberattacks
Major cyber events in 2024
Malware and malicious payload trends
Cyberattack types and targets
Vulnerability exploit attempts on CVEs
Attacks on counties – USA
Expansion of bot farms – how, where, and why
In-depth analysis of the cyber threat landscape across North America, South America, Europe, APAC, and the Middle East
Why are attacks on smart factories rising?
Cyber risk predictions
Axis of attacks – Europe
Systemic attacks in the Middle East
Download the full report from here:
https://sectrio.com/resources/ot-threat-landscape-reports/sectrio-releases-ot-ics-and-iot-security-threat-landscape-report-2024/
JMeter webinar - integration with InfluxDB and GrafanaRTTS
Watch this recorded webinar about real-time monitoring of application performance. See how to integrate Apache JMeter, the open-source leader in performance testing, with InfluxDB, the open-source time-series database, and Grafana, the open-source analytics and visualization application.
In this webinar, we will review the benefits of leveraging InfluxDB and Grafana when executing load tests and demonstrate how these tools are used to visualize performance metrics.
Length: 30 minutes
Session Overview
-------------------------------------------
During this webinar, we will cover the following topics while demonstrating the integrations of JMeter, InfluxDB and Grafana:
- What out-of-the-box solutions are available for real-time monitoring JMeter tests?
- What are the benefits of integrating InfluxDB and Grafana into the load testing stack?
- Which features are provided by Grafana?
- Demonstration of InfluxDB and Grafana using a practice web application
To view the webinar recording, go to:
https://www.rttsweb.com/jmeter-integration-webinar
Dev Dives: Train smarter, not harder – active learning and UiPath LLMs for do...UiPathCommunity
💥 Speed, accuracy, and scaling – discover the superpowers of GenAI in action with UiPath Document Understanding and Communications Mining™:
See how to accelerate model training and optimize model performance with active learning
Learn about the latest enhancements to out-of-the-box document processing – with little to no training required
Get an exclusive demo of the new family of UiPath LLMs – GenAI models specialized for processing different types of documents and messages
This is a hands-on session specifically designed for automation developers and AI enthusiasts seeking to enhance their knowledge in leveraging the latest intelligent document processing capabilities offered by UiPath.
Speakers:
👨🏫 Andras Palfi, Senior Product Manager, UiPath
👩🏫 Lenka Dulovicova, Product Program Manager, UiPath
Key Trends Shaping the Future of Infrastructure.pdfCheryl Hung
Keynote at DIGIT West Expo, Glasgow on 29 May 2024.
Cheryl Hung, ochery.com
Sr Director, Infrastructure Ecosystem, Arm.
The key trends across hardware, cloud and open-source; exploring how these areas are likely to mature and develop over the short and long-term, and then considering how organisations can position themselves to adapt and thrive.
Neuro-symbolic is not enough, we need neuro-*semantic*Frank van Harmelen
Neuro-symbolic (NeSy) AI is on the rise. However, simply machine learning on just any symbolic structure is not sufficient to really harvest the gains of NeSy. These will only be gained when the symbolic structures have an actual semantics. I give an operational definition of semantics as “predictable inference”.
All of this illustrated with link prediction over knowledge graphs, but the argument is general.
Epistemic Interaction - tuning interfaces to provide information for AI supportAlan Dix
Paper presented at SYNERGY workshop at AVI 2024, Genoa, Italy. 3rd June 2024
https://alandix.com/academic/papers/synergy2024-epistemic/
As machine learning integrates deeper into human-computer interactions, the concept of epistemic interaction emerges, aiming to refine these interactions to enhance system adaptability. This approach encourages minor, intentional adjustments in user behaviour to enrich the data available for system learning. This paper introduces epistemic interaction within the context of human-system communication, illustrating how deliberate interaction design can improve system understanding and adaptation. Through concrete examples, we demonstrate the potential of epistemic interaction to significantly advance human-computer interaction by leveraging intuitive human communication strategies to inform system design and functionality, offering a novel pathway for enriching user-system engagements.
1. Developing polyglot persistence
applications
Chris Richardson
Author of POJOs in Action
Founder of the original CloudFoundry.com
@crichardson
chris.richardson@springsource.com
http://plainoldobjects.com
22. @crichardson
Redis is great but there are
tradeoffs
• No rich queries: PK-based access only
• Limited transaction model:
• Read first and then execute updates as batch
• Difficult to compose code
• Data must fit in memory
• Single-threaded server: run multiple with client-side sharding
• Missing features such as access control, ...
Hope you
don’t
need
ACID
or rich
queries
one day
23. @crichardson
The future is polyglot
IEEE Software Sept/October 2010 - Debasish Ghosh / Twitter @debasishg
e.g. Netflix
• RDBMS
• SimpleDB
• Cassandra
• Hadoop/Hbase
26. @crichardson
Polyglot Pattern: replicate from
MySQL to NoSQL
Query
Service
Update
Service
MySQL
Database
Reader Writer
Redis
System
of
RecordMaterialized
view query() update()
replicate
and
denormalize
28. @crichardson
Finding available restaurants
Available restaurants =
Serve the zip code of the delivery address
AND
Are open at the delivery time
public interface AvailableRestaurantRepository {
List<AvailableRestaurant>
findAvailableRestaurants(Address deliveryAddress, Date deliveryTime);
...
}
30. @crichardson
Finding available restaurants on Monday, 6.15pm
for 94619 zipcode
Straightforward three-way join
select r.*
from restaurant r
inner join restaurant_time_range tr
on r.id =tr.restaurant_id
inner join restaurant_zipcode sa
on r.id = sa.restaurant_id
where ’94619’ = sa.zip_code
and tr.day_of_week=’monday’
and tr.openingtime <= 1815
and 1815 <= tr.closingtime
31. @crichardson
How to scale this query?
• Query caching is usually ineffective
• MySQL Master/slave replication is straightforward BUT
• Complexity of slaves
• Risk of replication failing
• Limitations of MySQL master/slave replication
32. @crichardson
Query denormalized copy stored
in Redis
Order
taking
Restaurant
Management
MySQL
Database
CONSUMER
RESTAURANT
OWNER
Redis
System
of
RecordCopy
findAvailable()
update()
33. @crichardson
BUT how to implement findAvailableRestaurants()
with Redis?!
?
select r.*
from restaurant r
inner join restaurant_time_range tr
on r.id =tr.restaurant_id
inner join restaurant_zipcode sa
on r.id = sa.restaurant_id
where ’94619’ = sa.zip_code
and tr.day_of_week=’monday’
and tr.openingtime <= 1815
and 1815 <= tr.closingtime
K1 V1
K2 V2
... ...
34. @crichardson
ZRANGEBYSCORE = Simple
SQL Query
ZRANGEBYSCORE myset 1 6
select value
from sorted_set
where key = ‘myset’
and score >= 1
and score <= 6
=
key value score
sorted_set
35. @crichardson
select r.*
from restaurant r
inner join restaurant_time_range tr
on r.id =tr.restaurant_id
inner join restaurant_zipcode sa
on r.id = sa.restaurant_id
where ’94619’ = sa.zip_code
and tr.day_of_week=’monday’
and tr.openingtime <= 1815
and 1815 <= tr.closingtime
select value
from sorted_set
where key = ?
and score >= ?
and score <= ?
?
How to transform the SELECT
statement?
We need to denormalize
36. @crichardson
Simplification #1:
Denormalization
Restaurant_id Day_of_week Open_time Close_time Zip_code
1 Monday 1130 1430 94707
1 Monday 1130 1430 94619
1 Monday 1730 2130 94707
1 Monday 1730 2130 94619
2 Monday 0700 1430 94619
…
SELECT restaurant_id
FROM time_range_zip_code
WHERE day_of_week = ‘Monday’
AND zip_code = 94619
AND 1815 < close_time
AND open_time < 1815
Simpler query:
§ No joins
§ Two = and two <
40. @crichardson
zip_dow open_time_restaurant_id close_time
94707:Monday 1130_1 1430
94619:Monday 1130_1 1430
94707:Monday 1730_1 2130
94619:Monday 1730_1 2130
94619:Monday 0700_2 1430
...
Sorted Set [ Entry:Score, …]
[1130_1:1430, 1730_1:2130]
[0700_2:1430, 1130_1:1430, 1730_1:2130]
Using a Redis sorted set as an index
Key
94619:Monday
94707:Monday
94619:Monday 0700_2 1430
41. @crichardson
Finding restaurants that have not yet closed
ZRANGEBYSCORE 94619:Monday 1815 2359
è
{1730_1}
1730 is before 1815 è Ajanta is open
Delivery timeDelivery zip and day
Sorted Set [ Entry:Score, …]
[1130_1:1430, 1730_1:2130]
[0700_2:1430, 1130_1:1430, 1730_1:2130]
Key
94619:Monday
94707:Monday
46. @crichardson
Two-Phase commit is not an
option
• Redis does not support it
• Even if it did, 2PC is best avoided http://www.infoq.com/articles/ebay-scalability-best-practices
48. @crichardson
Updating Redis #FAIL
begin MySQL transaction
update MySQL
update Redis
rollback MySQL transaction
Redis has update
MySQL does not
begin MySQL transaction
update MySQL
commit MySQL transaction
<<system crashes>>
update Redis
MySQL has update
Redis does not
49. @crichardson
Updating Redis reliably
Step 1 of 2
begin MySQL transaction
update MySQL
queue CRUD event in MySQL
commit transaction
ACID
Event Id
Operation: Create, Update, Delete
New entity state, e.g. JSON
50. @crichardson
Updating Redis reliably
Step 2 of 2
for each CRUD event in MySQL queue
get next CRUD event from MySQL queue
If CRUD event is not duplicate then
Update Redis (incl. eventId)
end if
begin MySQL transaction
mark CRUD event as processed
commit transaction
55. @crichardson
Summary
• Each SQL/NoSQL database = set of tradeoffs
• Polyglot persistence: leverage the strengths of SQL and
NoSQL databases
• Use Redis as a distributed cache
• Store denormalized data in Redis for fast querying
• Reliable database synchronization required