A modular, polyglot architecture has many advantages but it also adds complexity since each incoming request typically fans out to multiple distributed services. For example, in an online store application the information on a product details page - description, price, recommendations, etc - comes from numerous services. To minimize response time and improve scalability, these services must be invoked concurrently. However, traditional concurrency mechanisms are low-level, painful to use and error-prone.
In this talk you will learn about some powerful yet easy to use abstractions for consuming web services asynchronously. We will compare the various implementations of futures that are available in Java, Scala and JavaScript. You will learn how to use reactive observables, which are asynchronous data streams, to access web services from both Java and JavaScript. We will describe how these mechanisms let you write asynchronous code in a very straightforward, declarative fashion.
GotoChgo 2019: Not Just Events: Developing Asynchronous MicroservicesChris Richardson
The microservice architecture functionally decomposes an application into a set of services. Each service has its own private database that’s only accessible indirectly through the services API. Consequently, implementing queries and transactions that span multiple services is challenging.
In this presentation, you will learn how to solve these distributed data management challenges using asynchronous messaging. Chris will share with you how to implement transactions using sagas, which are sequences of local transactions. You will learn how to coordinate sagas using either events or command messages. Chris will also explore how to implement queries using Command Query Responsibility Segregation (CQRS), which uses events to maintain easily queried replicas.
SpringOne Platform 2017
Ryan Baxter, Pivotal
You have heard and seen great things about Spring Cloud and you decide it is time to dive in and try it out yourself. You fire up your browser head to Google and land on the Spring Cloud homepage. Then it hits you, where do you begin? What do each of these projects do? Do you need to use all of them or can you be selective? The number of projects under the Spring Cloud umbrella has grown immensely over the past couple of years and if you are a newcomer to the Spring Cloud ecosystem it can be quite daunting to sift through the projects to find what you need. By the end of this talk you will leave with a solid understanding of the Spring Cloud projects, how to use them to build cloud native apps, and the confidence to get started!
In this slide deck I try to explore the meaning of the often misinterpreted sentence "You build it, you run it". Starting with its origin in a 2006 interview with Werner Vogels and a short description of the misinterpretation of that phrase that can be seen quite often these days in companies that try to pick up concepts like DevOps without really getting the idea behind it, I then start a bit longer journey.
It starts with the history and evolution of markets and IT. Based on that I dive deeper into the effects of uncertainty which these days is often predominant for companies facing narrow and highly dynamic markets. I try to approach from a financial point of view as well as from a risk management point of view (yes, even if we have become agile, managing risk and financial aspects have not gone away).
The basic result is that we need to go faster (in terms of cycle times) without compromising quality. I also add a quick litmus test to figure out where your own company currently is located as reality never is as simple as a slide deck - to avoid turning a thought model that should support reasoning into a dogma.
Having the task set and understanding that IT must support going faster without compromising quality, I look at DevOps as it targets exactly the same goal. After a short explanation of what DevOps actually means (we face a perfect confusion of ideas when talking about DevOps) and what the effects are if you do it seriously, I focus on the effects for the process organization.
The traditional process organization - while usually being set up nicely for cost efficiency - is very poor in terms of cycle times. In order to shorten cycle times those organization will eventually become cross-functional, autonomous teams with end-to-end responsibility. And while those teams have all skills and the full authority for all their activities, they also need to take over the full responsibility for their actions - their is no one left to blame ;)
And this naturally leads to "(You decide it,) you build it, you run it" which in the end is a required organizational prerequisite for going fast in a DevOps way.
There is a lot left out in the slides that would also be worth talking about (after all it is just a 60 min talk) and of course the voice track is missing, but I still hope that those slides are of some value for you.
Apache Kafka is the de facto standard for data streaming to process data in motion. With its significant adoption growth across all industries, I get a very valid question every week: When NOT to use Apache Kafka? What limitations does the event streaming platform have? When does Kafka simply not provide the needed capabilities? How to qualify Kafka out as it is not the right tool for the job?
This session explores the DOs and DONTs. Separate sections explain when to use Kafka, when NOT to use Kafka, and when to MAYBE use Kafka.
No matter if you think about open source Apache Kafka, a cloud service like Confluent Cloud, or another technology using the Kafka protocol like Redpanda or Pulsar, check out this slide deck.
A detailed article about this topic:
https://www.kai-waehner.de/blog/2022/01/04/when-not-to-use-apache-kafka/
Have you ever wondered what the relative differences are between two of the more popular open source, in-memory data stores and caches? In this session, we will describe those differences and, more importantly, provide live demonstrations of the key capabilities that could have a major impact on your architectural Java application designs.
Estimating Development Security Maturity in About an HourPriyanka Aash
The session describes a simple method of estimating a development team’s security maturity, i.e. how well they make a secure software product, by looking at five key factors. The factors and a simple rating system will be shown coupled with real-world samples. Applicable usage scenarios as well as comparison to other security maturity models will be given.
(Source: RSA USA 2016-San Francisco)
GotoChgo 2019: Not Just Events: Developing Asynchronous MicroservicesChris Richardson
The microservice architecture functionally decomposes an application into a set of services. Each service has its own private database that’s only accessible indirectly through the services API. Consequently, implementing queries and transactions that span multiple services is challenging.
In this presentation, you will learn how to solve these distributed data management challenges using asynchronous messaging. Chris will share with you how to implement transactions using sagas, which are sequences of local transactions. You will learn how to coordinate sagas using either events or command messages. Chris will also explore how to implement queries using Command Query Responsibility Segregation (CQRS), which uses events to maintain easily queried replicas.
SpringOne Platform 2017
Ryan Baxter, Pivotal
You have heard and seen great things about Spring Cloud and you decide it is time to dive in and try it out yourself. You fire up your browser head to Google and land on the Spring Cloud homepage. Then it hits you, where do you begin? What do each of these projects do? Do you need to use all of them or can you be selective? The number of projects under the Spring Cloud umbrella has grown immensely over the past couple of years and if you are a newcomer to the Spring Cloud ecosystem it can be quite daunting to sift through the projects to find what you need. By the end of this talk you will leave with a solid understanding of the Spring Cloud projects, how to use them to build cloud native apps, and the confidence to get started!
In this slide deck I try to explore the meaning of the often misinterpreted sentence "You build it, you run it". Starting with its origin in a 2006 interview with Werner Vogels and a short description of the misinterpretation of that phrase that can be seen quite often these days in companies that try to pick up concepts like DevOps without really getting the idea behind it, I then start a bit longer journey.
It starts with the history and evolution of markets and IT. Based on that I dive deeper into the effects of uncertainty which these days is often predominant for companies facing narrow and highly dynamic markets. I try to approach from a financial point of view as well as from a risk management point of view (yes, even if we have become agile, managing risk and financial aspects have not gone away).
The basic result is that we need to go faster (in terms of cycle times) without compromising quality. I also add a quick litmus test to figure out where your own company currently is located as reality never is as simple as a slide deck - to avoid turning a thought model that should support reasoning into a dogma.
Having the task set and understanding that IT must support going faster without compromising quality, I look at DevOps as it targets exactly the same goal. After a short explanation of what DevOps actually means (we face a perfect confusion of ideas when talking about DevOps) and what the effects are if you do it seriously, I focus on the effects for the process organization.
The traditional process organization - while usually being set up nicely for cost efficiency - is very poor in terms of cycle times. In order to shorten cycle times those organization will eventually become cross-functional, autonomous teams with end-to-end responsibility. And while those teams have all skills and the full authority for all their activities, they also need to take over the full responsibility for their actions - their is no one left to blame ;)
And this naturally leads to "(You decide it,) you build it, you run it" which in the end is a required organizational prerequisite for going fast in a DevOps way.
There is a lot left out in the slides that would also be worth talking about (after all it is just a 60 min talk) and of course the voice track is missing, but I still hope that those slides are of some value for you.
Apache Kafka is the de facto standard for data streaming to process data in motion. With its significant adoption growth across all industries, I get a very valid question every week: When NOT to use Apache Kafka? What limitations does the event streaming platform have? When does Kafka simply not provide the needed capabilities? How to qualify Kafka out as it is not the right tool for the job?
This session explores the DOs and DONTs. Separate sections explain when to use Kafka, when NOT to use Kafka, and when to MAYBE use Kafka.
No matter if you think about open source Apache Kafka, a cloud service like Confluent Cloud, or another technology using the Kafka protocol like Redpanda or Pulsar, check out this slide deck.
A detailed article about this topic:
https://www.kai-waehner.de/blog/2022/01/04/when-not-to-use-apache-kafka/
Have you ever wondered what the relative differences are between two of the more popular open source, in-memory data stores and caches? In this session, we will describe those differences and, more importantly, provide live demonstrations of the key capabilities that could have a major impact on your architectural Java application designs.
Estimating Development Security Maturity in About an HourPriyanka Aash
The session describes a simple method of estimating a development team’s security maturity, i.e. how well they make a secure software product, by looking at five key factors. The factors and a simple rating system will be shown coupled with real-world samples. Applicable usage scenarios as well as comparison to other security maturity models will be given.
(Source: RSA USA 2016-San Francisco)
Introducing envoy-based service mesh at Booking.comIvan Kruglov
Service mesh is a dedicated layer of a company's infrastructure which should simplify communication between services and make it reliable, secure and observable.
In this talk, we'll go deep into Booking.com's case study of introducing service mesh. We will discover the reasons and objectives of the project. Why Envoy was selected as the base rather than other available options. Find out what is the setup and features of the homegrown control plane. We will expand on what service is provided for developers and how they safely deploy potentially dangerous configuration changes. Finally, we will talk about pitfalls met along the way.
Integrating security events from all of your network and security systems is critical to solving problems quickly and keeping your environment secure. The ideal solution puts you in control and continues to work even during a DDoS attack. In this session, you'll learn how you can use the Akamai SIEM Integration product to feed security events from Akamai Cloud Security products into your environment. Also in this session, we’ll demonstrate how to use Luna administration tools to set up user, API, and security settings giving you the benefits of having Akamai security events integrated into your overall security event monitoring solution.
Simplifying Distributed Transactions with Sagas in Kafka (Stephen Zoio, Simpl...confluent
Microservices are seen as the way to simplify complex systems, until you need to coordinate a transaction across services, and in that instant, the dream ends. Transactions involving multiple services can lead to a spaghetti web of interactions. Protocols such as two-phase commit come with complexity and performance bottlenecks. The Saga pattern involves a simplified transactional model. In sagas, a sequence of actions are executed, and if any action fails, a compensating action is executed for each of the actions that have already succeeded. This is particularly well suited to long-running and cross-microservice transactions. In this talk we introduce the new Simple Sagas library (https://github.com/simplesourcing/simplesagas). Built using Kafka streams, it provides a scalable fault tolerance event-based transaction processing engine. We walk through a use case of coordinating a sequence of complex financial transactions. We demonstrate the easy to use DSL, show how the system copes with failure, and discuss this overall approach to building scalable transactional systems in an event-driven streaming context.
The Multiplatform App Architecture offers the possibility to create mobile applications for multiple mobile platforms and at the same time offer the flexibility to use all native functionality of the mobile operating systems to realize an optimal user experience. Combined with a powerful development environment and a comprehensive programming language it offers an excellent way to develop and maintain rich mobile applications.
Capgemini helps customers to achieve mobile excellence and realizes mobile applications in an agile way using this architecture.
stackconf 2022: Data Management in Kubernetes – Backup, DR, HANETWAYS
Kubernetes is everywhere now, right? You see how companies are embracing this technology more and more, it’s like the Kardashians or Rosalía or TikTok; MAINSTREAM! BUT, as companies evolve and grow their container environments, they realize that stateful apps require more than a robust Kubernetes distribution, data is the key of their apps. Applications are as important as the data that they use. Managing 1M bank transactions is not the same as using bridge images, so you have to be very careful on how you control and use that data. Portworx is here to help with those problems. In this session, we will see how Portworx is able to manage volumes in the cloud with Kubernetes in seconds AND migrate that data from cluster to cluster in minutes, even a Disaster Recovery environment with ZERO data loss and NO downtime. Do you want to see it live? join this session!
(GAM201) Scalable Game Architectures That Don't Break the Bank | AWS re:Inven...Amazon Web Services
In this session, AWS shares best practices for mobile, console, and MMO games that can scale from 1,000 to 1,000,000 users. See how to create a game backend using Amazon EC2 and AWS Elastic Beanstalk. Learn about database scaling challenges, and how to use Amazon DynamoDB and Amazon ElastiCache to address them. And, hear how to deliver game assets efficiently using Amazon S3 and Amazon CloudFront. Then, hear from Roope Kangas, Lead Server Developer and co-founder at Grand Cru, about their journey launching and cost-optimizing Supernauts on AWS. Grand Cru used load testing to validate their system before launch, enabling them to reach 1 million users in 6 days. Then, after launch, the team optimized their architecture based on system metrics to cut their AWS costs by more than half.
De KubeCon à ContainerDays, eBPF a le vent en poupe dans le monde Cloud Native. Mais de quoi s’agit-il, pourquoi cette technologie est-elle révolutionnaire, et qu’est-ce qu’elle peut m’apporter concrètement?
À travers des exemples concrets appliqués aux domaines de l’observabilité, du réseau et de la sécurité, cette session explique les tenants d’eBPF et ses avantages concrets pour connecter et sécuriser les applications Cloud Native.
Vous y découvrirez comment démarrer votre aventure avec eBPF, avec des outils vous permettant de bénéficier de ses super-pouvoirs en toute simplicité.
MicroCPH - Managing data consistency in a microservice architecture using SagasChris Richardson
The services in a microservice architecture must be loosely coupled and so cannot share database tables. What’s more, two-phase commit (aka a distributed transaction) is not a viable option for modern applications. Consequently, a microservices application must use the Saga pattern, which maintains data consistency using a series of local transactions.
In this presentation, you will learn how sagas work and how they differ from traditional transactions. We describe how to use sagas to develop business logic in a microservices application. You will learn effective techniques for orchestrating sagas and how to use messaging for reliability. We will describe the design of a saga framework for Java and show a sample application.
How to build an ETL pipeline with Apache Beam on Google Cloud DataflowLucas Arruda
Nowadays more and more companies are searching for insights with potential to grow their business by analyzing large amounts of data from many different systems. However, in order to reach this level of Big Data Analysis it's necessary to build an ETL pipeline that allows us to process raw data coming from different sources into an appropriate format that is possible to use against visualization tools such as Tableau.
This kind of data processing can be done by a variety of tools and in this presentation I show how to do it by using an unified programming model created by Google and open-sourced as the name of Apache Beam. We will build a simple pipeline that will be executed in the Cloud by a fully-managed service called Google Cloud Dataflow.
Top 5 Event Streaming Use Cases for 2021 with Apache KafkaKai Wähner
Apache Kafka and Event Streaming are two of the most relevant buzzwords in tech these days. Ever wonder what the predicted TOP 5 Event Streaming Architectures and Use Cases for 2021 are? Check out the following presentation. Learn about edge deployments, hybrid and multi-cloud architectures, service mesh-based microservices, streaming machine learning, and cybersecurity.
On-demand video recording: https://videos.confluent.io/watch/XAjxV3j8hzwCcEKoZVErUJ
End-to-end Streaming Between gRPC Services Via Kafka with John FallowsHostedbyConfluent
"You have built and deployed your gRPC micro-services architecture and now you need to expand beyond request-response to add streaming. Helpfully, gRPC has built-in support for bidirectional streaming. You probably already have the event streams in Kafka and have been tasked with figuring out the integration. You might even build a prototype to better understand the performance, security and scalability challenges.
I will discuss some of these challenges in detail and also describe solutions to help meet them, including a straightforward approach to bridge gRPC streams to and from Apache Kafka.
This session is targeted towards developers interested in learning how to integrate gRPC with Kafka event streaming; securely, reliably and scalably."
Consuming web services asynchronously with Futures and Rx Observables (svcc, ...Chris Richardson
A modular, polyglot architecture has many advantages but it also adds complexity since each incoming request typically fans out to multiple distributed services. For example, in an online store application the information on a product details page - description, price, recommendations, etc - comes from numerous services. To minimize response time and improve scalability, these services must be invoked concurrently. However, traditional concurrency mechanisms are low-level, painful to use and error-prone. In this talk you will learn about some powerful yet easy to use abstractions for consuming web services asynchronously. We will compare the various implementations of futures that are available in Java, Scala and JavaScript. You will learn how to use reactive observables, which are asynchronous data streams, to access web services from both Java and JavaScript. We will describe how these mechanisms let you write asynchronous code in a very straightforward, declarative fashion.
Futures and Rx Observables: powerful abstractions for consuming web services ...Chris Richardson
A modular, polyglot architecture has many advantages but it also adds complexity since each incoming request typically fans out to multiple distributed services. For example, in an online store application the information on a product details page - description, price, recommendations, etc - comes from numerous services. To minimize response time and improve scalability, these services must be invoked concurrently. However, traditional concurrency mechanisms are low-level, painful to use and error-prone.
In this talk you will learn about some powerful yet easy to use abstractions for consuming web services asynchronously. We will compare the various implementations of futures that are available on the JVM. You will learn how to access web services using reactive observables (RxJava), which are asynchronous data streams. We will describe how these mechanisms let you write asynchronous code in a very straightforward, declarative fashion.
Introducing envoy-based service mesh at Booking.comIvan Kruglov
Service mesh is a dedicated layer of a company's infrastructure which should simplify communication between services and make it reliable, secure and observable.
In this talk, we'll go deep into Booking.com's case study of introducing service mesh. We will discover the reasons and objectives of the project. Why Envoy was selected as the base rather than other available options. Find out what is the setup and features of the homegrown control plane. We will expand on what service is provided for developers and how they safely deploy potentially dangerous configuration changes. Finally, we will talk about pitfalls met along the way.
Integrating security events from all of your network and security systems is critical to solving problems quickly and keeping your environment secure. The ideal solution puts you in control and continues to work even during a DDoS attack. In this session, you'll learn how you can use the Akamai SIEM Integration product to feed security events from Akamai Cloud Security products into your environment. Also in this session, we’ll demonstrate how to use Luna administration tools to set up user, API, and security settings giving you the benefits of having Akamai security events integrated into your overall security event monitoring solution.
Simplifying Distributed Transactions with Sagas in Kafka (Stephen Zoio, Simpl...confluent
Microservices are seen as the way to simplify complex systems, until you need to coordinate a transaction across services, and in that instant, the dream ends. Transactions involving multiple services can lead to a spaghetti web of interactions. Protocols such as two-phase commit come with complexity and performance bottlenecks. The Saga pattern involves a simplified transactional model. In sagas, a sequence of actions are executed, and if any action fails, a compensating action is executed for each of the actions that have already succeeded. This is particularly well suited to long-running and cross-microservice transactions. In this talk we introduce the new Simple Sagas library (https://github.com/simplesourcing/simplesagas). Built using Kafka streams, it provides a scalable fault tolerance event-based transaction processing engine. We walk through a use case of coordinating a sequence of complex financial transactions. We demonstrate the easy to use DSL, show how the system copes with failure, and discuss this overall approach to building scalable transactional systems in an event-driven streaming context.
The Multiplatform App Architecture offers the possibility to create mobile applications for multiple mobile platforms and at the same time offer the flexibility to use all native functionality of the mobile operating systems to realize an optimal user experience. Combined with a powerful development environment and a comprehensive programming language it offers an excellent way to develop and maintain rich mobile applications.
Capgemini helps customers to achieve mobile excellence and realizes mobile applications in an agile way using this architecture.
stackconf 2022: Data Management in Kubernetes – Backup, DR, HANETWAYS
Kubernetes is everywhere now, right? You see how companies are embracing this technology more and more, it’s like the Kardashians or Rosalía or TikTok; MAINSTREAM! BUT, as companies evolve and grow their container environments, they realize that stateful apps require more than a robust Kubernetes distribution, data is the key of their apps. Applications are as important as the data that they use. Managing 1M bank transactions is not the same as using bridge images, so you have to be very careful on how you control and use that data. Portworx is here to help with those problems. In this session, we will see how Portworx is able to manage volumes in the cloud with Kubernetes in seconds AND migrate that data from cluster to cluster in minutes, even a Disaster Recovery environment with ZERO data loss and NO downtime. Do you want to see it live? join this session!
(GAM201) Scalable Game Architectures That Don't Break the Bank | AWS re:Inven...Amazon Web Services
In this session, AWS shares best practices for mobile, console, and MMO games that can scale from 1,000 to 1,000,000 users. See how to create a game backend using Amazon EC2 and AWS Elastic Beanstalk. Learn about database scaling challenges, and how to use Amazon DynamoDB and Amazon ElastiCache to address them. And, hear how to deliver game assets efficiently using Amazon S3 and Amazon CloudFront. Then, hear from Roope Kangas, Lead Server Developer and co-founder at Grand Cru, about their journey launching and cost-optimizing Supernauts on AWS. Grand Cru used load testing to validate their system before launch, enabling them to reach 1 million users in 6 days. Then, after launch, the team optimized their architecture based on system metrics to cut their AWS costs by more than half.
De KubeCon à ContainerDays, eBPF a le vent en poupe dans le monde Cloud Native. Mais de quoi s’agit-il, pourquoi cette technologie est-elle révolutionnaire, et qu’est-ce qu’elle peut m’apporter concrètement?
À travers des exemples concrets appliqués aux domaines de l’observabilité, du réseau et de la sécurité, cette session explique les tenants d’eBPF et ses avantages concrets pour connecter et sécuriser les applications Cloud Native.
Vous y découvrirez comment démarrer votre aventure avec eBPF, avec des outils vous permettant de bénéficier de ses super-pouvoirs en toute simplicité.
MicroCPH - Managing data consistency in a microservice architecture using SagasChris Richardson
The services in a microservice architecture must be loosely coupled and so cannot share database tables. What’s more, two-phase commit (aka a distributed transaction) is not a viable option for modern applications. Consequently, a microservices application must use the Saga pattern, which maintains data consistency using a series of local transactions.
In this presentation, you will learn how sagas work and how they differ from traditional transactions. We describe how to use sagas to develop business logic in a microservices application. You will learn effective techniques for orchestrating sagas and how to use messaging for reliability. We will describe the design of a saga framework for Java and show a sample application.
How to build an ETL pipeline with Apache Beam on Google Cloud DataflowLucas Arruda
Nowadays more and more companies are searching for insights with potential to grow their business by analyzing large amounts of data from many different systems. However, in order to reach this level of Big Data Analysis it's necessary to build an ETL pipeline that allows us to process raw data coming from different sources into an appropriate format that is possible to use against visualization tools such as Tableau.
This kind of data processing can be done by a variety of tools and in this presentation I show how to do it by using an unified programming model created by Google and open-sourced as the name of Apache Beam. We will build a simple pipeline that will be executed in the Cloud by a fully-managed service called Google Cloud Dataflow.
Top 5 Event Streaming Use Cases for 2021 with Apache KafkaKai Wähner
Apache Kafka and Event Streaming are two of the most relevant buzzwords in tech these days. Ever wonder what the predicted TOP 5 Event Streaming Architectures and Use Cases for 2021 are? Check out the following presentation. Learn about edge deployments, hybrid and multi-cloud architectures, service mesh-based microservices, streaming machine learning, and cybersecurity.
On-demand video recording: https://videos.confluent.io/watch/XAjxV3j8hzwCcEKoZVErUJ
End-to-end Streaming Between gRPC Services Via Kafka with John FallowsHostedbyConfluent
"You have built and deployed your gRPC micro-services architecture and now you need to expand beyond request-response to add streaming. Helpfully, gRPC has built-in support for bidirectional streaming. You probably already have the event streams in Kafka and have been tasked with figuring out the integration. You might even build a prototype to better understand the performance, security and scalability challenges.
I will discuss some of these challenges in detail and also describe solutions to help meet them, including a straightforward approach to bridge gRPC streams to and from Apache Kafka.
This session is targeted towards developers interested in learning how to integrate gRPC with Kafka event streaming; securely, reliably and scalably."
Consuming web services asynchronously with Futures and Rx Observables (svcc, ...Chris Richardson
A modular, polyglot architecture has many advantages but it also adds complexity since each incoming request typically fans out to multiple distributed services. For example, in an online store application the information on a product details page - description, price, recommendations, etc - comes from numerous services. To minimize response time and improve scalability, these services must be invoked concurrently. However, traditional concurrency mechanisms are low-level, painful to use and error-prone. In this talk you will learn about some powerful yet easy to use abstractions for consuming web services asynchronously. We will compare the various implementations of futures that are available in Java, Scala and JavaScript. You will learn how to use reactive observables, which are asynchronous data streams, to access web services from both Java and JavaScript. We will describe how these mechanisms let you write asynchronous code in a very straightforward, declarative fashion.
Futures and Rx Observables: powerful abstractions for consuming web services ...Chris Richardson
A modular, polyglot architecture has many advantages but it also adds complexity since each incoming request typically fans out to multiple distributed services. For example, in an online store application the information on a product details page - description, price, recommendations, etc - comes from numerous services. To minimize response time and improve scalability, these services must be invoked concurrently. However, traditional concurrency mechanisms are low-level, painful to use and error-prone.
In this talk you will learn about some powerful yet easy to use abstractions for consuming web services asynchronously. We will compare the various implementations of futures that are available on the JVM. You will learn how to access web services using reactive observables (RxJava), which are asynchronous data streams. We will describe how these mechanisms let you write asynchronous code in a very straightforward, declarative fashion.
NodeJS: the good parts? A skeptic’s view (oredev, oredev2013)Chris Richardson
JavaScript used to be confined to the browser. But these days, it becoming increasingly popular in server-side applications in the form of NodeJS. NodeJS provides event-driven, non-blocking I/O model that supposedly makes it easy to build scalable network application. In this talk you will learn about the consequences of combining the event-driven programming model with a prototype-based, weakly typed, dynamic language. We will share our perspective as a server-side Java developer who wasn’t entirely happy about JavaScript in the browser, let alone on the server. You will learn how to use NodeJS effectively in modern, polyglot applications.
Decompose That WAR! Architecting for Adaptability, Scalability, and Deployabi...Chris Richardson
It’s no longer acceptable to develop large, monolithic, single-language, single-framework Web applications. In this session, you will learn how to use the scale cube to decompose your monolithic Web application into a set of narrowly focused, independently deployable services. The presentation discusses how a modular architecture makes it easy to adopt newer and better languages and technologies. You will learn about the various communication mechanisms—synchronous and asynchronous—that these services can use.
Decompose That WAR! Architecting for Adaptability, Scalability, and Deployabi...Chris Richardson
It’s no longer acceptable to develop large, monolithic, single-language, single-framework Web applications. In this session, you will learn how to use the scale cube to decompose your monolithic Web application into a set of narrowly focused, independently deployable services. The presentation discusses how a modular architecture makes it easy to adopt newer and better languages and technologies. You will learn about the various communication mechanisms—synchronous and asynchronous—that these services can use.
NodeJS: the good parts? A skeptic’s view (devnexus2014)Chris Richardson
JavaScript used to be confined to the browser. But these days, it becoming increasingly popular in server-side applications in the form of NodeJS. NodeJS provides event-driven, non-blocking I/O model that supposedly makes it easy to build scalable network application.
In this talk you will learn about the consequences of combining the event-driven programming model with a prototype-based, weakly typed, dynamic language. We will share our perspective as a server-side Java developer who wasn’t entirely happy about JavaScript in the browser, let alone on the server. You will learn how to use NodeJS effectively in modern, polyglot applications.
NodeJS: the good parts? A skeptic’s view (jmaghreb, jmaghreb2013)Chris Richardson
JavaScript used to be confined to the browser. But these days, it becoming increasingly popular in server-side applications in the form of NodeJS. NodeJS provides event-driven, non-blocking I/O model that supposedly makes it easy to build scalable network application. In this talk you will learn about the consequences of combining the event-driven programming model with a prototype-based, weakly typed, dynamic language. We will share our perspective as a server-side Java developer who wasn’t entirely happy about JavaScript in the browser, let alone on the server. You will learn how to use NodeJS effectively in modern, polyglot applications.
Microservices + Events + Docker = A Perfect Trio by Docker Captain Chris Rich...Docker, Inc.
Microservices are an essential enabler of agility but developing and deploying them is a challenge. In order for microservices to be loosely coupled,each service must have its own datastore. This makes it difficult to maintain data consistency across services.
Deploying microservices is also a complex problem since an application typically consists of 10s or 100s of services, written in a variety of languages and frameworks.
In this presentation, you will learn how to solve these problems by using an event-driven architecture to maintain data consistency and by using Docker to simplify deployment.
Microservices are an essential enabler of agility but developing and deploying them is a challenge. In order for microservices to be loosely coupled,each service must have its own datastore. This makes it difficult to maintain data consistency across services.
Deploying microservices is also a complex problem since an application typically consists of 10s or 100s of services, written in a variety of languages and frameworks. In this presentation, you will learn how to solve these problems by using an event-driven architecture to maintain data consistency and by using Docker to simplify deployment.
#JaxLondon keynote: Developing applications with a microservice architectureChris Richardson
The micro-service architecture, which structures an application as a set of small, narrowly focused, independently deployable services, is becoming an increasingly popular way to build applications. This approach avoids many of the problems of a monolithic architecture. It simplifies deployment and let’s you create highly scalable and available applications. In this keynote we describe the micro-service architecture and how to use it to build complex applications. You will learn how techniques such as Command Query Responsibility Segregation (CQRS) and Event Sourcing address the key challenges of developing applications with this architecture. We will also cover some of the various frameworks such as Spring Boot that you can use to implement micro-services.
Developing applications with a microservice architecture (SVforum, microservi...Chris Richardson
Here is the version of my microservices talk that that I gave on September 17th at the SVforum Cloud SIG/Microservices meetup.
To learn more see http://microservices.io and http://plainoldobjects.com
Developing applications with a microservice architecture (svcc)Chris Richardson
The micro-service architecture, which structures an application as a set of small, narrowly focused, independently deployable services, is becoming an increasingly popular way to build applications. This approach avoids many of the problems of a monolithic architecture. It simplifies deployment and let’s you create highly scalable and available applications. In this talk we describe the micro-service architecture and how to use it to build complex applications. You will learn how techniques such as Command Query Responsibility Segregation (CQRS) and Event Sourcing address the key challenges of developing applications with this architecture. We will also cover some of the various frameworks such as NodeJS and Spring Boot that you can use to implement micro-services.
Building microservices with Scala, functional domain models and Spring Boot (...Chris Richardson
In this talk you will learn about a modern way of designing applications that’s very different from the traditional approach of building monolithic applications that persist mutable domain objects in a relational database.We will talk about the microservice architecture, it’s benefits and drawbacks and how Spring Boot can help. You will learn about implementing business logic using functional, immutable domain models written in Scala. We will describe event sourcing and how it’s an extremely useful persistence mechanism for persisting functional domain objects in a microservices architecture.
YOW2018 - Events and Commands: Developing Asynchronous MicroservicesChris Richardson
The microservice architecture functionally decomposes an application into a set of services. Each service has its own private database that’s only accessible indirectly through the services API. Consequently, implementing queries and transactions that span multiple services is challenging.
In this presentation, you will learn how to solve these distributed data management challenges using asynchronous messaging. I describe how to implement transactions using sagas, which are sequences of local transactions, coordinated using messages. You will learn how to implement queries using Command Query Responsibility Segregation (CQRS), which uses events to maintain replicas. I describe how to use event sourcing, which is an event-centric approach to business logic and persistence, in a microservice architecture.
Mucon: Not Just Events: Developing Asynchronous MicroservicesChris Richardson
The microservice architecture functionally decomposes an application into a set of services. Each service has its own private database that’s only accessible indirectly through the services API. Consequently, implementing queries and transactions that span multiple services is challenging. In this presentation, you will learn how to solve these distributed data management challenges using asynchronous messaging. Chris will share with you how to implement transactions using sagas, which are sequences of local transactions. You will learn how to coordinate sagas using either events or command messages. Chris will also explore how to implement queries using Command Query Responsibility Segregation (CQRS), which uses events to maintain easily queried replicas.
Saturn 2018: Managing data consistency in a microservice architecture using S...Chris Richardson
A revised and extended version that I gave at Saturn 2018.
The services in a microservice architecture must be loosely coupled and so cannot share database tables. What’s more, two phase commit (a.k.a. a distributed transaction) is not a viable option for modern applications. Consequently, a microservices application must use the Saga pattern, which maintains data consistency using a series of local transactions.
In this presentation, you will learn how sagas work and how they differ from traditional transactions. We describe how to use sagas to develop business logic in a microservices application. You will learn effective techniques for orchestrating sagas and how to use messaging for reliability. We will describe the design of a saga framework for Java and show a sample application.
These are the slides that were presented at Red Hat's workshop: Achieving True Integration Agility with Containers, Microservices and APIs. Seattle, WA, October 26, 2017
NodeJS: the good parts? A skeptic’s view (jax jax2013)Chris Richardson
JavaScript used to be confined to the browser. But these days, it's becoming increasingly popular in server-side applications in the form of Node.js. Node.js provides event-driven, non-blocking I/O model that supposedly makes it easy to build scalable network application. In this talk you will learn about the consequences of combining the event-driven programming model with a prototype-based, weakly typed, dynamic language. We will share our perspective as a server-side Java developer who wasn’t entirely happy about JavaScript in the browser, let alone on the server. You will learn how to use Node.js effectively in modern, polyglot applications.
Watch the video: http://www.youtube.com/watch?v=CN0jTnSROsk&feature=youtu.be
Similar to Futures and Rx Observables: powerful abstractions for consuming web services asynchronously (#springone #s2gx) (20)
A common microservice architecture anti-pattern is more the merrier. It occurs when an organization team builds an excessively fine-grained architecture, e.g. one service-per-developer. In this talk, you will learn about the criteria that you should consider when deciding service granularity. I'll discuss the downsides of a fine-grained microservice architecture. You will learn how sometimes the solution to a design problem is simply a JAR file.
YOW London - Considering Migrating a Monolith to Microservices? A Dark Energy...Chris Richardson
This is a talk I gave at YOW! London 2022.
Let's imagine that you are responsible for an aging monolithic application that's critical to your business. Sadly, getting changes into production is a painful ordeal that regularly causes outages. And to make matters worse, the application's technology stack is growing increasingly obsolete. Neither the business nor the developers are happy. You need to modernize your application and have read about the benefits of microservices. But is the microservice architecture a good choice for your application?
In this presentation, I describe the dark energy and dark matter forces (a.k.a. concerns) that you must consider when deciding between the monolithic and microservice architectural styles. You will learn about how well each architectural style resolves each of these forces. I describe how to evaluate the relative importance of each of these forces to your application. You will learn how to use the results of this evaluation to decide whether to migrate to the microservice architecture.
Dark Energy, Dark Matter and the Microservices Patterns?!Chris Richardson
Dark matter and dark energy are mysterious concepts from astrophysics that are used to explain observations of distant stars and galaxies. The Microservices pattern language - a collection of patterns that solve architecture, design, development, and operational problems — enables software developers to use the microservice architecture effectively. But how could there possibly be a connection between microservices and these esoteric concepts from astrophysics?
In this presentation, I describe how dark energy and dark matter are excellent metaphors for the competing forces (a.k.a. concerns) that must be resolved by the microservices pattern language. You will learn that dark energy, which is an anti-gravity, is a metaphor for the repulsive forces that encourage decomposition into services. I describe how dark matter, which is an invisible matter that has a gravitational effect, is a metaphor for the attractive forces that resist decomposition and encourage the use of a monolithic architecture. You will learn how to use the dark energy and dark matter forces as guide when designing services and operations.
Dark energy, dark matter and microservice architecture collaboration patternsChris Richardson
Dark energy and dark matter are useful metaphors for the repulsive forces, which encourage decomposition into services, and the attractive forces, which resist decomposition. You must balance these conflicting forces when defining a microservice architecture including when designing system operations (a.k.a. requests) that span services.
In this talk, I describe the dark energy and dark matter forces. You will learn how to design system operations that span services using microservice architecture collaboration patterns: Saga, Command-side replica, API composition, and CQRS patterns. I describe how each of these patterns resolve the dark energy and dark matter forces differently.
It sounds dull but good architecture documentation is essential. Especially when you are actively trying to improve your architecture.
For example, I spend a lot time helping clients modernize their software architecture. More often than I like, I’m presented with a vague and lifeless collection of boxes and lines. As a result, it’s sometimes difficult to discuss the architecture in a meaningful and productive way. In this presentation, I’ll describe techniques for creating minimal yet effective documentation for your application’s microservice architecture. In particular, you will learn how documenting scenarios can bring your architecture to life.
Using patterns and pattern languages to make better architectural decisions Chris Richardson
This is a presentation that gave at the O'Reilly Software Architecture Superstream: Software Architecture Patterns.
The talk's focus is the microservices pattern language.
However, it also shows how thinking with the pattern mindset - context/problem/forces/solution/consequences - leads to better technically decisions.
The microservices architecture offers tremendous benefits, but it’s not a silver bullet. It also has some significant drawbacks. The microservices pattern language—a collection of patterns that solve architecture, design, development, and operational problems—enables software developers to apply the microservices architecture effectively. I provide an overview of the microservices architecture and examines the motivations for the pattern language, then takes you through the key patterns in the pattern language.
Rapid, reliable, frequent and sustainable software development requires an architecture that is loosely coupled and modular.
Teams need to be able complete their work with minimal coordination and communication with other teams.
They also need to be able keep the software’s technology stack up to date.
However, the microservice architecture isn’t always the only way to satisfy these requirements.
Yet, neither is the monolithic architecture.
In this talk, I describe loose coupling and modularity and why they are is essential.
You will learn about three architectural patterns: traditional monolith, modular monolith and microservices.
I describe the benefits, drawbacks and issues of each pattern and how well it supports rapid, reliable, frequent and sustainable development.
You will learn some heuristics for selecting the appropriate pattern for your application.
Events to the rescue: solving distributed data problems in a microservice arc...Chris Richardson
To deliver a large complex application rapidly, frequently and reliably, you often must use the microservice architecture.
The microservice architecture is an architectural style that structures the application as a collection of loosely coupled services.
One challenge with using microservices is that in order to be loosely coupled each service has its own private database.
As a result, implementing transactions and queries that span services is no longer straightforward.
In this presentation, you will learn how event-driven microservices address this challenge.
I describe how to use sagas, which is an asynchronous messaging-based pattern, to implement transactions that span services.
You will learn how to implement queries that span services using the CQRS pattern, which maintain easily queryable replicas using events.
A pattern language for microservices - June 2021 Chris Richardson
The microservice architecture is growing in popularity. It is an architectural style that structures an application as a set of loosely coupled services that are organized around business capabilities. Its goal is to enable the continuous delivery of large, complex applications. However, the microservice architecture is not a silver bullet and it has some significant drawbacks.
The goal of the microservices pattern language is to enable software developers to apply the microservice architecture effectively. It is a collection of patterns that solve architecture, design, development and operational problems. In this talk, I’ll provide an overview of the microservice architecture and describe the motivations for the pattern language. You will learn about the key patterns in the pattern language.
QConPlus 2021: Minimizing Design Time Coupling in a Microservice ArchitectureChris Richardson
Delivering large, complex software rapidly, frequently and reliably requires a loosely coupled organization. DevOps teams should rarely need to communicate and coordinate in order to get work done. Conway's law states that an organization and the architecture that it develops mirror one another. Hence, a loosely coupled organization requires a loosely coupled architecture.
In this presentation, you will learn about design-time coupling in a microservice architecture and why it's essential to minimize it. I describe how to design service APIs to reduce coupling. You will learn how to minimize design-time coupling by applying a version of the DRY principle. I describe how key microservices patterns potentially result in tight design time coupling and how to avoid it.
Mucon 2021 - Dark energy, dark matter: imperfect metaphors for designing micr...Chris Richardson
In order to explain certain astronomical observations, physicists created the mysterious concepts of dark energy and dark matter.
Dark energy is a repulsive force.
It’s an anti-gravity that is forcing matter apart and accelerating the expansion of the universe.
Dark matter has the opposite attraction effect.
Although it’s invisible, dark matter has a gravitational effect on stars and galaxies.
In this presentation, you will learn how these metaphors apply to the microservice architecture.
I describe how there are multiple repulsive forces that drive the decomposition of your application into services.
You will learn, however, that there are also multiple attractive forces that resist decomposition and bind software elements together.
I describe how as an architect you must find a way to balance these opposing forces.
Skillsmatter CloudNative eXchange 2020
The microservice architecture is a key part of cloud native.
An essential principle of the microservice architecture is loose coupling.
If you ignore this principle and develop tightly coupled services the result will mostly likely be yet another "microservices failure story”.
Your application will be brittle and have all of disadvantages of both the monolithic and microservice architectures.
In this talk you will learn about the different kinds of coupling and how to design loosely coupled microservices.
I describe how to minimize design time and increase the productivity of your DevOps teams.
You will learn how how to reduce runtime coupling and improve availability.
I describe how to improve availability by minimizing the coupling caused by your infrastructure.
DDD SoCal: Decompose your monolith: Ten principles for refactoring a monolith...Chris Richardson
This is a talk I gave at DDD SoCal.
1. Make the most of your monolith
2. Adopt microservices for the right reasons
3. It’s not just architecture
4. Get the support of the business
5. Migrate incrementally
6. Know your starting point
7. Begin with the end in mind
8. Migrate high-value modules first
9. Success is improved velocity and reliability
10. If it hurts, don’t do it
Decompose your monolith: Six principles for refactoring a monolith to microse...Chris Richardson
This was a talk I gave at the CTO virtual summit on July 28th. It describes 6 principles for refactoring to a microservice architecture.
1. Make the most of your monolith
2. Adopt microservices for the right reasons
3. Migrate incrementally
4. Begin with the end in mind
5. Migrate high-value modules first
6. Success is improved velocity and reliability
The microservice architecture is becoming increasingly important. But what is it exactly? Why should you care about microservices? And, what do you need to do to ensure that your organization uses the microservice architecture successfully? In this talk, I’ll answer these and other questions. You will learn about the motivations for the microservice architecture and why simply adopting microservices is insufficient. I describe essential characteristics of microservices, You will learn how a successful microservice architecture consists of loosely coupled services with stable APIs that communicate asynchronously.
JFokus: Cubes, Hexagons, Triangles, and More: Understanding MicroservicesChris Richardson
The microservice architecture is becoming increasing important. But what is it exactly? Why should you care about microservices? And, what do you need to do to ensure that your organization uses the microservice architecture successfully? In this talk, I’ll answer these and other questions using shapes as visual metaphors. You will learn about the motivations for the microservice architecture and why simply adopting microservices is insufficient. I describe essential characteristics of microservices, You will learn how a successful microservice architecture consist of loosely coupled services with stable APIs that communicate asynchronous. I will cover strategies for effectively testing microservices.
Decompose your monolith: strategies for migrating to microservices (Tide)Chris Richardson
This is a presentation that I gave at Tide.co, London - January 2020
A typical mission-critical enterprise application is a large, complex monolith developed by large team. Software delivery is usually slow, and the team struggles to keep up with the demands of the business. Consequently, many enterprise applications are good candidates to be migrated to the microservice architecture. But how do you know whether it makes sense to migrate to microservices? And, how to get there? In this presentation, I describe when you should consider migrating to microservices. You will learn strategies for migrating a monolith application to a microservice architecture. I explain how to implement new functionality as services. You will learn how to incrementally break apart a monolith one service at a time.
Kubernetes & AI - Beauty and the Beast !?! @KCD Istanbul 2024Tobias Schneck
As AI technology is pushing into IT I was wondering myself, as an “infrastructure container kubernetes guy”, how get this fancy AI technology get managed from an infrastructure operational view? Is it possible to apply our lovely cloud native principals as well? What benefit’s both technologies could bring to each other?
Let me take this questions and provide you a short journey through existing deployment models and use cases for AI software. On practical examples, we discuss what cloud/on-premise strategy we may need for applying it to our own infrastructure to get it to work from an enterprise perspective. I want to give an overview about infrastructure requirements and technologies, what could be beneficial or limiting your AI use cases in an enterprise environment. An interactive Demo will give you some insides, what approaches I got already working for real.
UiPath Test Automation using UiPath Test Suite series, part 3DianaGray10
Welcome to UiPath Test Automation using UiPath Test Suite series part 3. In this session, we will cover desktop automation along with UI automation.
Topics covered:
UI automation Introduction,
UI automation Sample
Desktop automation flow
Pradeep Chinnala, Senior Consultant Automation Developer @WonderBotz and UiPath MVP
Deepak Rai, Automation Practice Lead, Boundaryless Group and UiPath MVP
DevOps and Testing slides at DASA ConnectKari Kakkonen
My and Rik Marselis slides at 30.5.2024 DASA Connect conference. We discuss about what is testing, then what is agile testing and finally what is Testing in DevOps. Finally we had lovely workshop with the participants trying to find out different ways to think about quality and testing in different parts of the DevOps infinity loop.
Builder.ai Founder Sachin Dev Duggal's Strategic Approach to Create an Innova...Ramesh Iyer
In today's fast-changing business world, Companies that adapt and embrace new ideas often need help to keep up with the competition. However, fostering a culture of innovation takes much work. It takes vision, leadership and willingness to take risks in the right proportion. Sachin Dev Duggal, co-founder of Builder.ai, has perfected the art of this balance, creating a company culture where creativity and growth are nurtured at each stage.
Generating a custom Ruby SDK for your web service or Rails API using Smithyg2nightmarescribd
Have you ever wanted a Ruby client API to communicate with your web service? Smithy is a protocol-agnostic language for defining services and SDKs. Smithy Ruby is an implementation of Smithy that generates a Ruby SDK using a Smithy model. In this talk, we will explore Smithy and Smithy Ruby to learn how to generate custom feature-rich SDKs that can communicate with any web service, such as a Rails JSON API.
Transcript: Selling digital books in 2024: Insights from industry leaders - T...BookNet Canada
The publishing industry has been selling digital audiobooks and ebooks for over a decade and has found its groove. What’s changed? What has stayed the same? Where do we go from here? Join a group of leading sales peers from across the industry for a conversation about the lessons learned since the popularization of digital books, best practices, digital book supply chain management, and more.
Link to video recording: https://bnctechforum.ca/sessions/selling-digital-books-in-2024-insights-from-industry-leaders/
Presented by BookNet Canada on May 28, 2024, with support from the Department of Canadian Heritage.
Connector Corner: Automate dynamic content and events by pushing a buttonDianaGray10
Here is something new! In our next Connector Corner webinar, we will demonstrate how you can use a single workflow to:
Create a campaign using Mailchimp with merge tags/fields
Send an interactive Slack channel message (using buttons)
Have the message received by managers and peers along with a test email for review
But there’s more:
In a second workflow supporting the same use case, you’ll see:
Your campaign sent to target colleagues for approval
If the “Approve” button is clicked, a Jira/Zendesk ticket is created for the marketing design team
But—if the “Reject” button is pushed, colleagues will be alerted via Slack message
Join us to learn more about this new, human-in-the-loop capability, brought to you by Integration Service connectors.
And...
Speakers:
Akshay Agnihotri, Product Manager
Charlie Greenberg, Host
Dev Dives: Train smarter, not harder – active learning and UiPath LLMs for do...UiPathCommunity
💥 Speed, accuracy, and scaling – discover the superpowers of GenAI in action with UiPath Document Understanding and Communications Mining™:
See how to accelerate model training and optimize model performance with active learning
Learn about the latest enhancements to out-of-the-box document processing – with little to no training required
Get an exclusive demo of the new family of UiPath LLMs – GenAI models specialized for processing different types of documents and messages
This is a hands-on session specifically designed for automation developers and AI enthusiasts seeking to enhance their knowledge in leveraging the latest intelligent document processing capabilities offered by UiPath.
Speakers:
👨🏫 Andras Palfi, Senior Product Manager, UiPath
👩🏫 Lenka Dulovicova, Product Program Manager, UiPath
Neuro-symbolic is not enough, we need neuro-*semantic*Frank van Harmelen
Neuro-symbolic (NeSy) AI is on the rise. However, simply machine learning on just any symbolic structure is not sufficient to really harvest the gains of NeSy. These will only be gained when the symbolic structures have an actual semantics. I give an operational definition of semantics as “predictable inference”.
All of this illustrated with link prediction over knowledge graphs, but the argument is general.
The Art of the Pitch: WordPress Relationships and SalesLaura Byrne
Clients don’t know what they don’t know. What web solutions are right for them? How does WordPress come into the picture? How do you make sure you understand scope and timeline? What do you do if sometime changes?
All these questions and more will be explored as we talk about matching clients’ needs with what your agency offers without pulling teeth or pulling your hair out. Practical tips, and strategies for successful relationship building that leads to closing the deal.
LF Energy Webinar: Electrical Grid Modelling and Simulation Through PowSyBl -...DanBrown980551
Do you want to learn how to model and simulate an electrical network from scratch in under an hour?
Then welcome to this PowSyBl workshop, hosted by Rte, the French Transmission System Operator (TSO)!
During the webinar, you will discover the PowSyBl ecosystem as well as handle and study an electrical network through an interactive Python notebook.
PowSyBl is an open source project hosted by LF Energy, which offers a comprehensive set of features for electrical grid modelling and simulation. Among other advanced features, PowSyBl provides:
- A fully editable and extendable library for grid component modelling;
- Visualization tools to display your network;
- Grid simulation tools, such as power flows, security analyses (with or without remedial actions) and sensitivity analyses;
The framework is mostly written in Java, with a Python binding so that Python developers can access PowSyBl functionalities as well.
What you will learn during the webinar:
- For beginners: discover PowSyBl's functionalities through a quick general presentation and the notebook, without needing any expert coding skills;
- For advanced developers: master the skills to efficiently apply PowSyBl functionalities to your real-world scenarios.
Essentials of Automations: Optimizing FME Workflows with ParametersSafe Software
Are you looking to streamline your workflows and boost your projects’ efficiency? Do you find yourself searching for ways to add flexibility and control over your FME workflows? If so, you’re in the right place.
Join us for an insightful dive into the world of FME parameters, a critical element in optimizing workflow efficiency. This webinar marks the beginning of our three-part “Essentials of Automation” series. This first webinar is designed to equip you with the knowledge and skills to utilize parameters effectively: enhancing the flexibility, maintainability, and user control of your FME projects.
Here’s what you’ll gain:
- Essentials of FME Parameters: Understand the pivotal role of parameters, including Reader/Writer, Transformer, User, and FME Flow categories. Discover how they are the key to unlocking automation and optimization within your workflows.
- Practical Applications in FME Form: Delve into key user parameter types including choice, connections, and file URLs. Allow users to control how a workflow runs, making your workflows more reusable. Learn to import values and deliver the best user experience for your workflows while enhancing accuracy.
- Optimization Strategies in FME Flow: Explore the creation and strategic deployment of parameters in FME Flow, including the use of deployment and geometry parameters, to maximize workflow efficiency.
- Pro Tips for Success: Gain insights on parameterizing connections and leveraging new features like Conditional Visibility for clarity and simplicity.
We’ll wrap up with a glimpse into future webinars, followed by a Q&A session to address your specific questions surrounding this topic.
Don’t miss this opportunity to elevate your FME expertise and drive your projects to new heights of efficiency.
UiPath Test Automation using UiPath Test Suite series, part 4DianaGray10
Welcome to UiPath Test Automation using UiPath Test Suite series part 4. In this session, we will cover Test Manager overview along with SAP heatmap.
The UiPath Test Manager overview with SAP heatmap webinar offers a concise yet comprehensive exploration of the role of a Test Manager within SAP environments, coupled with the utilization of heatmaps for effective testing strategies.
Participants will gain insights into the responsibilities, challenges, and best practices associated with test management in SAP projects. Additionally, the webinar delves into the significance of heatmaps as a visual aid for identifying testing priorities, areas of risk, and resource allocation within SAP landscapes. Through this session, attendees can expect to enhance their understanding of test management principles while learning practical approaches to optimize testing processes in SAP environments using heatmap visualization techniques
What will you get from this session?
1. Insights into SAP testing best practices
2. Heatmap utilization for testing
3. Optimization of testing processes
4. Demo
Topics covered:
Execution from the test manager
Orchestrator execution result
Defect reporting
SAP heatmap example with demo
Speaker:
Deepak Rai, Automation Practice Lead, Boundaryless Group and UiPath MVP
UiPath Test Automation using UiPath Test Suite series, part 4
Futures and Rx Observables: powerful abstractions for consuming web services asynchronously (#springone #s2gx)
1. @crichardson
Consuming web services asynchronously
with Futures
and Rx Observables
Chris Richardson
Author of POJOs in Action
Founder of the original CloudFoundry.com
@crichardson
chris@chrisrichardson.net
http://plainoldobjects.com
9. @crichardson
Agenda
The need for concurrency
Simplifying concurrent code with Futures
Taming callback hell with JavaScript promises
Consuming asynchronous streams with Reactive Extensions
15. @crichardson
Application architecture
Product Info Service
Desktop
browser
Native mobile
client
REST
REST
REST
Recomendation Service
Review Service
Front end server
API gatewayREST
Mobile
browser
Web Application
HTML
17. @crichardson
Handling getProductDetails() - sequentially
Front-end server
Product Info Service
Recommendations
Service
Review
Service
getProductInfo()
getRecommendations()
getReviews()
getProductDetails()
Higher response time :-(
18. @crichardson
Handling getProductDetails() - in parallel
Front-end server
Product Info Service
Recommendations
Service
Review
Service
getProductInfo()
getRecommendations()
getReviews()
getProductDetails()
Lower response time :-)
19. @crichardson
Implementing a concurrent REST client
Thread-pool based approach
executorService.submit(new Callable(...))
Simpler but less scalable - lots of idle threads consuming memory
Event-driven approach
NIO with completion callbacks
More complex but more scalable
20. @crichardson
The front-end server must handle partial
failure of backend services
Front-end server
Product Info Service
Recommendations
Service
Review
Service
getProductInfo()
getRecommendations()
getReviews()
getProductDetails()
X
How to provide a
good user
experience?
21. @crichardson
Agenda
The need for concurrency
Simplifying concurrent code with Futures
Taming callback hell with JavaScript promises
Consuming asynchronous streams with Reactive Extensions
22. @crichardson
Futures are a great concurrency
abstraction
http://en.wikipedia.org/wiki/Futures_and_promises
24. @crichardson
Benefits
Client does not know how the asynchronous operation is implemented
Client can invoke multiple asynchronous operations and gets a Future for each
one.
25. @crichardson
REST client using Spring @Async
@Component
class ProductInfoServiceImpl extends ProducInfoService {
val restTemplate : RestTemplate = ...
@Async
def getProductInfo(productId: Long) = {
new AsyncResult(restTemplate.getForObject(....)...)
}
}
Execute asynchronously in thread pool
A Future
containing a value
trait ProductInfoService {
def getProductInfo(productId: Long): java.util.concurrent.Future[ProductInfo]
}
26. @crichardson
ProductDetailsService
@Component
class ProductDetailsService
@Autowired()(productInfoService: ProductInfoService,
reviewService: ReviewService,
recommendationService: RecommendationService) {
def getProductDetails(productId: Long): ProductDetails = {
val productInfoFuture = productInfoService.getProductInfo(productId)
}
}
val recommendationsFuture = recommendationService.getRecommendations(productId)
val reviewsFuture = reviewService.getReviews(productId)
val productInfo = productInfoFuture.get(300, TimeUnit.MILLISECONDS)
val recommendations =
recommendationsFuture.get(10, TimeUnit.MILLISECONDS)
val reviews = reviewsFuture.get(10, TimeUnit.MILLISECONDS)
ProductDetails(productInfo, recommendations, reviews)
28. @crichardson
Not bad but...
val productInfo =
productInfoFuture.get(300, TimeUnit.MILLISECONDS)
Blocks Tomcat thread until Future
completes
Not so scalable :-(
29. @crichardson
... and also...
Java Futures work well for a single-level of asynchronous execution
BUT
#fail for more complex, scalable scenarios
Difficult to compose and coordinate multiple concurrent operations
http://techblog.netflix.com/2013/02/rxjava-netflix-api.html
30. @crichardson
Better: Futures with callbacks no blocking!
val f : Future[Int] = Future { ... }
f onSuccess {
case x : Int => println(x)
}
f onFailure {
case e : Exception => println("exception thrown")
}
Guava ListenableFutures, Spring 4 ListenableFuture
Java 8 CompletableFuture, Scala Futures
31. @crichardson
Even better: Composable Futures
val f1 = Future { ... ; 1 }
val f2 = Future { ... ; 2 }
val f4 = f2.map(_ * 2)
assertEquals(4, Await.result(f4, 1 second))
val fzip = f1 zip f2
assertEquals((1, 2), Await.result(fzip, 1 second))
Transforms Future
Combines two futures
Scala, Java 8 CompletableFuture (partially)
32. @crichardson
Scala futures are Monads
def callB() : Future[...] = ...
def callC() : Future[...] = ...
def callD() : Future[...] = ...
val result = for {
(b, c) <- callB() zip callC();
d <- callD(b, c)
} yield d
result onSuccess { .... }
Two calls execute in parallel
And then invokes D
Get the result of D
33. @crichardson
Scala Future + RestTemplate
import scala.concurrent.Future
@Component
class ProductInfoService {
def getProductInfo(productId: Long): Future[ProductInfo] = {
Future { restTemplate.getForObject(....) }
}
}
Executes in thread pool
Scala Future
34. @crichardson
Scala Future + RestTemplate
class ProductDetailsService @Autowired()(....) {
def getProductDetails(productId: Long) = {
val productInfoFuture = productInfoService.getProductInfo(productId)
val recommendationsFuture = recommendationService.getRecommendations(productId)
val reviewsFuture = reviewService.getReviews(productId)
for (((productInfo, recommendations), reviews) <-
productInfoFuture zip recommendationsFuture zip reviewsFuture)
yield ProductDetails(productInfo, recommendations, reviews)
}
}
Non-blocking!
35. @crichardson
Async Spring MVC + Scala Futures
@Controller
class ProductController ... {
@RequestMapping(Array("/productdetails/{productId}"))
@ResponseBody
def productDetails(@PathVariable productId: Long) = {
val productDetails =
productDetailsService.getProductDetails(productId)
val result = new DeferredResult[ProductDetails]
productDetails onSuccess {
case r => result.setResult(r)
}
productDetails onFailure {
case t => result.setErrorResult(t)
}
result
}
Convert Scala Future to
Spring MVC
DeferredResult
37. @crichardson
About the Reactor pattern
Defined by Doug Schmidt in 1995
Pattern for writing scalable servers
Alternative to thread-per-connection model
Single threaded event loop dispatches events on handles (e.g. sockets, file
descriptors) to event handlers
38. @crichardson
Reactor pattern structure
Event Handler
handle_event(type)
get_handle()
Initiation Dispatcher
handle_events()
register_handler(h)
select(handlers)
for each h in handlers
h.handle_event(type)
end loop
handle
Synchronous Event
Demultiplexer
select()
owns
notifies
uses
handlers
Application
creates
40. @crichardson
New in Spring 4
Mirrors RestTemplate
Methods return a ListenableFuture = JDK 7 Future + callback methods
Can use HttpComponents NIO-based AsyncHttpClient
Spring AsyncRestTemplate
41. @crichardson
Using the AsyncRestTemplate
http://hc.apache.org/httpcomponents-asyncclient-dev/
val asyncRestTemplate = new AsyncRestTemplate(
new HttpComponentsAsyncClientHttpRequestFactory())
override def getProductInfo(productId: Long) = {
val listenableFuture =
asyncRestTemplate.getForEntity("{baseUrl}/productinfo/{productId}",
classOf[ProductInfo],
baseUrl, productId)
toScalaFuture(listenableFuture).map { _.getBody }
}
Convert to Scala Future and get entity
42. @crichardson
Converting ListenableFuture to Scala Future
implicit def toScalaFuture[T](f : ListenableFuture[T]) : Future[T] = {
val p = promise[T]
f.addCallback(new ListenableFutureCallback[T] {
def onSuccess(result: T) { p.success(result)}
def onFailure(t: Throwable) { p.failure(t) }
})
p.future
}
The producer side of Scala Futures
Supply outcome
Return future
44. @crichardson
If recommendation service is down...
Never responds front-end server waits indefinitely
Consumes valuable front-end server resources
Page never displayed and customer gives up
Returns an error
Error returned to the front-end server error page is displayed
Customer gives up
45. @crichardson
Fault tolerance at Netflix
Network timeouts and retries
Invoke remote services via a bounded thread pool
Use the Circuit Breaker pattern
On failure:
return default/cached data
return error to caller
Implementation: https://github.com/Netflix/Hystrix
http://techblog.netflix.com/2012/02/fault-tolerance-in-high-volume.html
46. @crichardson
Using Hystrix
@Component
class ProductInfoServiceImpl extends ProductInfoService {
val restTemplate = RestTemplateFactory.makeRestTemplate()
val baseUrl = ...
class GetProductInfoCommand(productId: Long)extends
HystrixCommand[ProductInfo](....) {
override def run() =
restTemplate.
getForEntity("{baseUrl}/productinfo/{productId}",
classOf[ProductInfo],
baseUrl, productId).getBody
}
def getProductInfoUsingHystrix(productId: Long) : Future[ProductInfo] = {
new GetProductInfoCommand(productId).queue()
}
}
Runs in thread pool
Returns JDK Future
48. @crichardson
How to handling partial failures?
productDetailsFuture zip
recommendationsFuture zip reviewsFuture
Fails if any Future has failed
49. @crichardson
Handling partial failures
val recommendationsFuture =
recommendationService.
getRecommendations(userId,productId).
recover {
case _ => Recommendations(List())
}
“catch-like” Maps Throwable to value
50. @crichardson
Implementing a Timeout
resultFuture onSuccess { case r => result.setResult(r) }
resultFuture onFailure { case t => result.setErrorResult(t) }
No timeout - callbacks might never be
invoked :-(
Await.result(resultFuture, timeout)
Blocks until timeout:-(
51. @crichardson
Non-blocking Timeout
http://eng.42go.com/future-safefuture-timeout-cancelable/
object TimeoutFuture {
def apply[T](future: Future[T], onTimeout: => Unit = Unit)
(implicit ec: ExecutionContext, after: Duration): Future[T] = {
val timer = new HashedWheelTimer(10, TimeUnit.MILLISECONDS)
val promise = Promise[T]()
val timeout = timer.newTimeout(new TimerTask {
def run(timeout: Timeout){
onTimeout
promise.failure(new TimeoutException(s"Future timed out after ${after.toMillis}ms"))
}
}, after.toNanos, TimeUnit.NANOSECONDS)
Future.firstCompletedOf(Seq(future, promise.future)).
tap(_.onComplete { case result => timeout.cancel() })
}
}
val future = ...
val timedoutFuture = TimeoutFuture(future)(executionContext, 200.milleseconds)
Timer fails
promise
Outcome of first
completed
52. @crichardson
Using the Akka Circuit Breaker
import akka.pattern.CircuitBreaker
val breaker =
new CircuitBreaker(actorSystem.scheduler,
maxFailures = 1,
callTimeout = 100.milliseconds,
resetTimeout = 1.minute)
val resultFuture = breaker.
withCircuitBreaker{ asynchronousOperationReturningFuture() }
53. @crichardson
Limiting # of simultaneous requests
val limiter =
new ConcurrencyLimiter(maxConcurrentRequests=10, maxQueueSize=30)
val resultFuture = limiter.withLimit {
asynchronousOperationReturningFuture()
}
class ConcurrencyLimiter ...
val concurrencyManager : ActorRef = ...
def withLimit[T](body : => Future[T])(...) =
(concurrencyManager ?
ConcurrentExecutionManager.Request { () => body }).mapTo[T]
54. @crichardson
Putting it all together
@Component
class ProductInfoServiceImpl @Autowired()(...) extends ProductService {
val limiter = new ConcurrencyLimiter(...)
val breaker = new CircuitBreaker(...)
override def getProductInfo(productId: Long) = {
...
breaker.withCircuitBreaker {
limiter.withLimit {
TimeoutFuture { ... AsyncRestTemplate.get ... }
}
}
}
55. @crichardson
Agenda
The need for concurrency
Simplifying concurrent code with Futures
Taming callback hell with JavaScript promises
Consuming asynchronous streams with Reactive Extensions
62. @crichardson
Messy callback code
getProductDetails = (req, res) ->
productId = req.params.productId
result = {productId: productId}
makeHandler = (key) ->
(error, clientResponse, body) ->
if clientResponse.statusCode != 200
res.status(clientResponse.statusCode)
res.write(body)
res.end()
else
result[key] = JSON.parse(body)
if (result.productInfo and result.recommendations and result.reviews)
res.json(result)
request("http://localhost:3000/productdetails/" + productId, makeHandler("productInfo"))
request("http://localhost:3000/recommendations/" + productId, makeHandler('recommendations'))
request("http://localhost:3000/reviews/" + productId, makeHandler('reviews'))
app.get('/productinfo/:productId', getProductInfo)
Returns a callback
function
Have all three requests
completed?
63. @crichardson
Simplifying code with Promises (a.k.a.
Futures)
Functions return a promise - no callback parameter
A promise represents an eventual outcome
Use a library of functions for transforming and composing promises
Promises/A+ specification - http://promises-aplus.github.io/promises-spec
64. @crichardson
Basic promise API
promise2 = promise1.then(fulfilledHandler, errorHandler, ...)
called when promise1 is fulfilled
returns a promise or value
resolved with outcome of callback
called when promise1 is rejected
returns a promise or value
65. About when.js
Rich API = Promises spec + use full
extensions
Creation:
when.defer()
when.reject()...
Combinators:
all, some, join, map, reduce
Other:
Calling non-promise code
timeout
....
https://github.com/cujojs/when
75. @crichardson
Agenda
The need for concurrency
Simplifying concurrent code with Futures
Taming callback hell with JavaScript promises
Consuming asynchronous streams with Reactive Extensions
76. @crichardson
Let’s imagine you have a stream of trades
and
you need to calculate the rolling average
price of each stock
80. @crichardson
Introducing Reactive Extensions (Rx)
The Reactive Extensions (Rx) is a library for composing
asynchronous and event-based programs using
observable sequences and LINQ-style query operators.
Using Rx, developers represent asynchronous data
streams with Observables , query asynchronous
data streams using LINQ operators , and .....
https://rx.codeplex.com/
82. @crichardson
RxJava core concepts
class Observable<T> {
Subscription subscribe(Observer<T> observer)
...
}
interface Observer<T> {
void onNext(T args)
void onCompleted()
void onError(Throwable e)
}
Notifies
An asynchronous stream of
items
Used to unsubscribe
83. @crichardson
Comparing Observable to...
Observer pattern - similar but adds
Observer.onComplete()
Observer.onError()
Iterator pattern - mirror image
Push rather than pull
Future - similar but
Represents a stream of multiple values
87. @crichardson
Combining observables
class Observable<T> {
static <R,T1,T2>
Observable<R> zip(Observable<T0> o1,
Observable<T1> o2,
Func2<T1,T2,R> reduceFunction)
...
}
Invoked with pairs of items from
o1 and o2
Stream of results from
reduceFunction
88. @crichardson
Creating observables from data
class Observable<T> {
static Observable<T> from(Iterable<T> iterable]);
static Observable<T> from(T items ...]);
static Observable<T> from(Future<T> future]);
...
}
89. @crichardson
Creating observables from event sources
Observable<T> o = Observable.create(
new SubscriberFunc<T> () {
Subscription onSubscribe(Observer<T> obs) {
... connect to event source....
return new Subscriber () {
void unsubscribe() { ... };
};
});
Called once for
each Observer
Called when
unsubscribing
Arranges to call obs.onNext/onComplete/...
105. @crichardson
Using merge()
merge()
APPL : 401
IBM : 405
CAT : 405 ...
APPL: 403
APPL : 401 IBM : 405 CAT : 405 APPL: 403
Observable<Observable<AveragePrice>>
Observable<AveragePrice>
106. @crichardson
RxJS / NodeJS example
var rx = require("rx");
function tailFile (fileName) {
var self = this;
var o = rx.Observable.create(function (observer) {
var watcher = self.tail(fileName, function (line) {
observer.onNext(line);
});
return function () {
watcher.close();
}
});
return o.map(parseLogLine);
}
function parseLogLine(line) { ... }
107. @crichardson
Rx in the browser - implementing
completion
TextChanges(input)
.DistinctUntilChanged()
.Throttle(TimeSpan.FromMilliSeconds(10))
.Select(word=>Completions(word))
.Switch()
.Subscribe(ObserveChanges(output));
Your Mouse is a Database
http://queue.acm.org/detail.cfm?id=2169076
Ajax call returning
Observable
Change events Observable
Display completions
108. @crichardson
Summary
Consuming web services asynchronously is essential
Scala-style composable Futures are a powerful concurrency abstraction
Rx Observables are even more powerful
Rx Observables are a unifying abstraction for a wide variety of use cases