This document discusses microservices and alternatives to monolithic architectures. It begins by outlining the costs of software maintenance for monoliths and reasons why architectures need to change to adapt quickly. Alternatives like SOA are discussed, but microservices are presented as a way to break applications into fine-grained, independent services. However, the document cautions that simply breaking a monolith into smaller pieces does not automatically achieve decoupling, and improper use of microservices could result in even greater complexity. Autonomous services with explicit boundaries and asynchronous communication are emphasized as important characteristics to achieve decoupling.
Scylla Summit 2022: How to Migrate a Counter Table for 68 Billion RecordsScyllaDB
In this talk, we will discuss Happn's war story about migrating a Cassandra 2.1 cluster containing more than 68 Billion records in a counter table to ScyllaDB Open Source.
To watch all of the recordings hosted during Scylla Summit 2022 visit our website here: https://www.scylladb.com/summit.
Use App Configuration to store all the settings for your application and secure their accesses in one place.
Centralize management and distribution of hierarchical configuration data for different environments and geographies
Dynamically change application settings without the need to redeploy or restart an application
At the core its a key-value store
Supports history
Great fit for Event-driven microservices architecture
Control feature availability in real-time
Cloud Native Implementation of the “External configuration store” pattern
https://www.meetup.com/Stockholm-Azure-Meetup/events/265524268/
[APIdays INTERFACE 2021] The Evolution of API Security for Client-side Applic...WSO2
Client-side applications are becoming an increasingly popular technology to build applications owing to the advanced user experience that they provide consumers. Authentication and API authorization for these applications are also becoming equally popular topics that many developers have a hard time getting their heads around.
Check these slides, where Johann Nallathamby, Head of Solutions Architecture for IAM at WSO2, will attempt to demystify some complexities and misconceptions surrounding this topic and help you better understand the most important features to consider when choosing an authentication and API authorization solution for client-side applications.
These slides will review:
- The broader classification of client-side applications and their legacy and more recent authentication and API authorization patterns
- Sender-constrained token patterns
- Solution patterns being employed to improve user experience in client-side applications
by Darin Briskman, Technical Evangelist, AWS
Database Freedom means being able to use the database engine that’s right for you as your needs evolve. Being locked into a specific technology can prevent you from achieving your mission. Fortunately, AWS Database Migration Service makes it easy to switch between different database engines. We’ll look at how to use Schema Migration Tool with DMS to switch from a commercial database to open source. You’ll need a laptop with a Firefox or Chrome browser.
This slide deck describes some of the best practices found when running Oracle Database inside a Docker container. Those best practices are general observations collected over time and may not reflect your actual environment or current situation.
Scylla Summit 2022: How to Migrate a Counter Table for 68 Billion RecordsScyllaDB
In this talk, we will discuss Happn's war story about migrating a Cassandra 2.1 cluster containing more than 68 Billion records in a counter table to ScyllaDB Open Source.
To watch all of the recordings hosted during Scylla Summit 2022 visit our website here: https://www.scylladb.com/summit.
Use App Configuration to store all the settings for your application and secure their accesses in one place.
Centralize management and distribution of hierarchical configuration data for different environments and geographies
Dynamically change application settings without the need to redeploy or restart an application
At the core its a key-value store
Supports history
Great fit for Event-driven microservices architecture
Control feature availability in real-time
Cloud Native Implementation of the “External configuration store” pattern
https://www.meetup.com/Stockholm-Azure-Meetup/events/265524268/
[APIdays INTERFACE 2021] The Evolution of API Security for Client-side Applic...WSO2
Client-side applications are becoming an increasingly popular technology to build applications owing to the advanced user experience that they provide consumers. Authentication and API authorization for these applications are also becoming equally popular topics that many developers have a hard time getting their heads around.
Check these slides, where Johann Nallathamby, Head of Solutions Architecture for IAM at WSO2, will attempt to demystify some complexities and misconceptions surrounding this topic and help you better understand the most important features to consider when choosing an authentication and API authorization solution for client-side applications.
These slides will review:
- The broader classification of client-side applications and their legacy and more recent authentication and API authorization patterns
- Sender-constrained token patterns
- Solution patterns being employed to improve user experience in client-side applications
by Darin Briskman, Technical Evangelist, AWS
Database Freedom means being able to use the database engine that’s right for you as your needs evolve. Being locked into a specific technology can prevent you from achieving your mission. Fortunately, AWS Database Migration Service makes it easy to switch between different database engines. We’ll look at how to use Schema Migration Tool with DMS to switch from a commercial database to open source. You’ll need a laptop with a Firefox or Chrome browser.
This slide deck describes some of the best practices found when running Oracle Database inside a Docker container. Those best practices are general observations collected over time and may not reflect your actual environment or current situation.
Introduced in 2004, the Data Distribution Service (DDS) has been steadily growing in popularity and adoption. Today, DDS is at the heart of a large number of mission and business critical systems, such as, Air Traffic Control and Management, Train Control Systems, Energy Production Systems, Medical Devices, Autonomous Vehicles, Smart Cities and NASA’s Kennedy Space Centre Launch System.
Considered the technological trends toward data-centricity and the rate of adoption, tomorrow, DDS will be at the at the heart of an incredible number of Industrial IoT systems.
To help you become an expert in DDS and exploit your skills in the growing DDS market, we have designed the DDS in Action webcast series. This series is a learning journey through which you will (1) discover the essence of DDS, (2) understand how to effectively exploit DDS to architect and program distributed applications that perform and scale, (3) learn the key DDS programming idioms and architectural patterns, (4) understand how to characterise DDS performances and configure for optimal latency/throughput, (5) grow your system to Internet scale, and (6) secure you DDS system.
La arquitectura impulsada por eventos (EDA) será el corazón del ecosistema de MAPFRE. Para seguir siendo competitivas, las empresas de hoy dependen cada vez más del análisis de datos en tiempo real, lo que les permite obtener información y tiempos de respuesta más rápidos. Los negocios con datos en tiempo real consisten en tomar conciencia de la situación, detectar y responder a lo que está sucediendo en el mundo ahora.
How to Take Advantage of an Enterprise Data Warehouse in the CloudDenodo
Watch full webinar here: [https://buff.ly/2CIOtys]
As organizations collect increasing amounts of diverse data, integrating that data for analytics becomes more difficult. Technology that scales poorly and fails to support semi-structured data fails to meet the ever-increasing demands of today’s enterprise. In short, companies everywhere can’t consolidate their data into a single location for analytics.
In this Denodo DataFest 2018 session we’ll cover:
Bypassing the mandate of a single enterprise data warehouse
Modern data sharing to easily connect different data types located in multiple repositories for deeper analytics
How cloud data warehouses can scale both storage and compute, independently and elastically, to meet variable workloads
Presentation by Harsha Kapre, Snowflake
Learn how SQL Server on AWS gives you complete control over every setting, without the maintenance, backup and patching requirements of traditional on-site solutions. Discover how to provision and monitor your SQL Server databases in both Amazon RDS and Amazon EC2, and how to optimise scalability, performance, availability, security and disaster recovery.
This presentation gives an overview of the Apache Ignite project. It explains Ignite in relation to its architecture, scaleability, caching, datagrid and machine learning abilities.
Links for further information and connecting
http://www.amazon.com/Michael-Frampton/e/B00NIQDOOM/
https://nz.linkedin.com/pub/mike-frampton/20/630/385
https://open-source-systems.blogspot.com/
Webinar slides: How to Migrate from Oracle DB to MariaDBSeveralnines
Watch this webinar replay as we walk you through all you need to know to plan and execute a successful migration from Oracle database to MariaDB.
Over the years MariaDB has gained Enterprise support and maturity to run critical and complex data transaction systems. With the recent version, MariaDB has added some great new features such as SQL_Mode=Oracle compatibility, making the transition process easier than ever before.
Whether you’re planning to migrate from Oracle database to MariaDB manually or with the help of a commercial tool to automate the entire migration process, you need to know all the possible bottlenecks and methods involved in the process and the results validation.
Migrating from Oracle database to MariaDB can come with a number of benefits: lower cost of ownership, access to and use of an open source database engine, tight integration with the web, wide circle of MariaDB database professionals and more.
Find out how it could benefit your organisation!
AGENDA
- A brief introduction to the platform
- Oracle vs MariaDB
- Platform support
- Installation process
- Database access
- Backup process
- Controlling query execution
- Security
- Replication options
- Migration
- Planning and development strategy
- Assessment or preliminary check
- Data type mapping
- Migration tools
- Migration process
- Testing
- Post-migration
- Monitoring and Alerting
- Performance Management
- Backup Management
- High availability
- Upgrades
- Scaling
- Staff training
SPEAKER
Bartlomiej Oles, Senior Support Engineer at Severalnines, is a MySQL and Oracle DBA, with over 15 years experience in managing highly available production systems at IBM, Nordea Bank, Acxiom, Lufthansa, and other Fortune 500 companies. In the past five years, his focus has been on building and applying automation tools to manage multi-datacenter database environments.
Kafka for Real-Time Replication between Edge and Hybrid CloudKai Wähner
Not all workloads allow cloud computing. Low latency, cybersecurity, and cost-efficiency require a suitable combination of edge computing and cloud integration.
This session explores architectures and design patterns for software and hardware considerations to deploy hybrid data streaming with Apache Kafka anywhere. A live demo shows data synchronization from the edge to the public cloud across continents with Kafka on Hivecell and Confluent Cloud.
Kafka High Availability in multi data center setup with floating Observers wi...HostedbyConfluent
Enabling High Availability in cluster setup that spawns different data centers is challenging but it is even more if we are using just two data-centers. Not ideal for Kafka HA at all. But this is reality for most organizations as they are using the same data-centers previously used for database HA.
In this presentation we will see how to use Kafka Observer feature to address this challenge with additional tweak to distribute load evenly among Observers and ordinary Brokers and make them floating between data-centers. The whole demo is supported by Infrastructure as a code automation trough Ansible.
Aurora MySQL Backtrack을 이용한 빠른 복구 방법 - 진교선 :: AWS Database Modernization Day 온라인Amazon Web Services Korea
발표영상 다시보기: https://kr-resources.awscloud.com/data-databases-and-analytics/aurora-mysql-backtrack%EC%9D%84-%EC%9D%B4%EC%9A%A9%ED%95%9C-%EB%B9%A0%EB%A5%B8-%EB%B3%B5%EA%B5%AC-%EB%B0%A9%EB%B2%95-%EC%A7%84%EA%B5%90%EC%84%A0-aws-database-modernization-day-%EC%98%A8%EB%9D%BC%EC%9D%B8-2
Aurora MySQL은 기존 MySQL의 운영에 추가한 많은 기능들을 제공해 드리고 있습니다. 이 중 복구에 관련된 기능인 Aurora MySQL PITR과 Backtrack에 대한 소개를 드리고자 합니다. 두 기능을 통해 운영 중 일어날 수 있는 rollback 상황에서, 어떠한 방식으로 복구를 할 수 있는지 실습해보실 수 있습니다.
Introduced in 2004, the Data Distribution Service (DDS) has been steadily growing in popularity and adoption. Today, DDS is at the heart of a large number of mission and business critical systems, such as, Air Traffic Control and Management, Train Control Systems, Energy Production Systems, Medical Devices, Autonomous Vehicles, Smart Cities and NASA’s Kennedy Space Centre Launch System.
Considered the technological trends toward data-centricity and the rate of adoption, tomorrow, DDS will be at the at the heart of an incredible number of Industrial IoT systems.
To help you become an expert in DDS and exploit your skills in the growing DDS market, we have designed the DDS in Action webcast series. This series is a learning journey through which you will (1) discover the essence of DDS, (2) understand how to effectively exploit DDS to architect and program distributed applications that perform and scale, (3) learn the key DDS programming idioms and architectural patterns, (4) understand how to characterise DDS performances and configure for optimal latency/throughput, (5) grow your system to Internet scale, and (6) secure you DDS system.
La arquitectura impulsada por eventos (EDA) será el corazón del ecosistema de MAPFRE. Para seguir siendo competitivas, las empresas de hoy dependen cada vez más del análisis de datos en tiempo real, lo que les permite obtener información y tiempos de respuesta más rápidos. Los negocios con datos en tiempo real consisten en tomar conciencia de la situación, detectar y responder a lo que está sucediendo en el mundo ahora.
How to Take Advantage of an Enterprise Data Warehouse in the CloudDenodo
Watch full webinar here: [https://buff.ly/2CIOtys]
As organizations collect increasing amounts of diverse data, integrating that data for analytics becomes more difficult. Technology that scales poorly and fails to support semi-structured data fails to meet the ever-increasing demands of today’s enterprise. In short, companies everywhere can’t consolidate their data into a single location for analytics.
In this Denodo DataFest 2018 session we’ll cover:
Bypassing the mandate of a single enterprise data warehouse
Modern data sharing to easily connect different data types located in multiple repositories for deeper analytics
How cloud data warehouses can scale both storage and compute, independently and elastically, to meet variable workloads
Presentation by Harsha Kapre, Snowflake
Learn how SQL Server on AWS gives you complete control over every setting, without the maintenance, backup and patching requirements of traditional on-site solutions. Discover how to provision and monitor your SQL Server databases in both Amazon RDS and Amazon EC2, and how to optimise scalability, performance, availability, security and disaster recovery.
This presentation gives an overview of the Apache Ignite project. It explains Ignite in relation to its architecture, scaleability, caching, datagrid and machine learning abilities.
Links for further information and connecting
http://www.amazon.com/Michael-Frampton/e/B00NIQDOOM/
https://nz.linkedin.com/pub/mike-frampton/20/630/385
https://open-source-systems.blogspot.com/
Webinar slides: How to Migrate from Oracle DB to MariaDBSeveralnines
Watch this webinar replay as we walk you through all you need to know to plan and execute a successful migration from Oracle database to MariaDB.
Over the years MariaDB has gained Enterprise support and maturity to run critical and complex data transaction systems. With the recent version, MariaDB has added some great new features such as SQL_Mode=Oracle compatibility, making the transition process easier than ever before.
Whether you’re planning to migrate from Oracle database to MariaDB manually or with the help of a commercial tool to automate the entire migration process, you need to know all the possible bottlenecks and methods involved in the process and the results validation.
Migrating from Oracle database to MariaDB can come with a number of benefits: lower cost of ownership, access to and use of an open source database engine, tight integration with the web, wide circle of MariaDB database professionals and more.
Find out how it could benefit your organisation!
AGENDA
- A brief introduction to the platform
- Oracle vs MariaDB
- Platform support
- Installation process
- Database access
- Backup process
- Controlling query execution
- Security
- Replication options
- Migration
- Planning and development strategy
- Assessment or preliminary check
- Data type mapping
- Migration tools
- Migration process
- Testing
- Post-migration
- Monitoring and Alerting
- Performance Management
- Backup Management
- High availability
- Upgrades
- Scaling
- Staff training
SPEAKER
Bartlomiej Oles, Senior Support Engineer at Severalnines, is a MySQL and Oracle DBA, with over 15 years experience in managing highly available production systems at IBM, Nordea Bank, Acxiom, Lufthansa, and other Fortune 500 companies. In the past five years, his focus has been on building and applying automation tools to manage multi-datacenter database environments.
Kafka for Real-Time Replication between Edge and Hybrid CloudKai Wähner
Not all workloads allow cloud computing. Low latency, cybersecurity, and cost-efficiency require a suitable combination of edge computing and cloud integration.
This session explores architectures and design patterns for software and hardware considerations to deploy hybrid data streaming with Apache Kafka anywhere. A live demo shows data synchronization from the edge to the public cloud across continents with Kafka on Hivecell and Confluent Cloud.
Kafka High Availability in multi data center setup with floating Observers wi...HostedbyConfluent
Enabling High Availability in cluster setup that spawns different data centers is challenging but it is even more if we are using just two data-centers. Not ideal for Kafka HA at all. But this is reality for most organizations as they are using the same data-centers previously used for database HA.
In this presentation we will see how to use Kafka Observer feature to address this challenge with additional tweak to distribute load evenly among Observers and ordinary Brokers and make them floating between data-centers. The whole demo is supported by Infrastructure as a code automation trough Ansible.
Aurora MySQL Backtrack을 이용한 빠른 복구 방법 - 진교선 :: AWS Database Modernization Day 온라인Amazon Web Services Korea
발표영상 다시보기: https://kr-resources.awscloud.com/data-databases-and-analytics/aurora-mysql-backtrack%EC%9D%84-%EC%9D%B4%EC%9A%A9%ED%95%9C-%EB%B9%A0%EB%A5%B8-%EB%B3%B5%EA%B5%AC-%EB%B0%A9%EB%B2%95-%EC%A7%84%EA%B5%90%EC%84%A0-aws-database-modernization-day-%EC%98%A8%EB%9D%BC%EC%9D%B8-2
Aurora MySQL은 기존 MySQL의 운영에 추가한 많은 기능들을 제공해 드리고 있습니다. 이 중 복구에 관련된 기능인 Aurora MySQL PITR과 Backtrack에 대한 소개를 드리고자 합니다. 두 기능을 통해 운영 중 일어날 수 있는 rollback 상황에서, 어떠한 방식으로 복구를 할 수 있는지 실습해보실 수 있습니다.
Down to Stack Traces, up from Heap DumpsAndrei Pangin
Глубже стек-трейсов, шире хип-дампов
Stack trace и heap dump - не просто инструменты отладки; это потайные дверцы к самым недрам виртуальной Java машины. Доклад будет посвящён малоизвестным особенностям JDK, так или иначе связанным с обоходом хипа и стеками потоков.
Мы разберём:
- как снимать дампы в продакшне без побочных эффектов;
- как работают утилиты jmap и jstack изнутри, и в чём хитрость forced режима;
- почему все профилировщики врут, и как с этим бороться;
- познакомимся с новым Stack-Walking API в Java 9;
- научимся сканировать Heap средствами JVMTI;
- узнаем о недокументированных функциях Хотспота и других интересных штуках.
Understanding the mysteries of the CSS property value syntaxRuss Weakley
This presentation will reveal the wonders of the W3C CSS property and value syntax. Along the way we will explore the mysterious double pipes, pound signs, double ampersands and more. You will come away a CSS syntax expert, able to understand any CSS property / value syntax no matter how complex... or your money back!
Developing rich multimedia applications with Kurento: a tutorial for Java Dev...Luis Lopez
This presentation contains a tutorial devoted to showing how Java developers can create rich multimedia applications with Kurento. Java developers will find natural Kurento development model, which is based on standard Java EE technologies and is inspired on the WWW Servlet model.
If you have ever developed a Web application, you may be familiar with this scheme. At the browser, HTML and JavaScript code is in charge of user interaction and generates HTTP requests to the server. This code is usually programmed with the help of APIs such as jQuery, DOM, XHR or others. Upon reception, HTTP requests are processed by some kind of server side technology (e. g.. PHP, Java, Ruby, etc.) using service APIs providing features such as DB access, communications, transactions, XML parsing, and others. As a result, an HTTP response is issued and sent back to the client. Following this scheme, both server and client side APIs are just capabilities simplifying developer work and providing abstractions for programming faster and more efficiently.
Kurento technologies adapt to the Web development model so that, from a programmer perspective, Kurento can be seen just as an additional set of APIs. Developers does not need to learn novel programming schemes and can reuse all their knowledge and previous background on WWW application development. When you need multimedia, just use Kurento APIs. For the rest, use your preferred APIs or reuse previous code. Kurento APIs have been designed for simplicity and Web developers will find them familiar and intuitive. Most of the low level details related to codecs, formats, protocols, profiles and containers are abstracted by the framework. Programmers just concentrate on specifying the sequence of processing steps that they want to execute on the media flows.
Microservices give us many options. We can pick different technologies, mix synchronous and asynchronous integration techniques or embrace different deployment patterns. But they also give us different options in how we think about securing our systems. Done right, and microservices can increase the security of your vital data and processes. Done wrong, and you can increase the surface area of attack. This talk will discuss the importance of defence in depth, discussing the many different ways in which you can secure your fine-grained, distributed architectures.
elasticRTC -- how to have your own WebRTC cloud scaling to be billions in min...Luis Lopez
Creating WebRTC applications is simple, but making them scalable in a robust and efficient way isn’t. Due to this, many application developers do not dare to use their own infrastructures and instead prefer to leverage third party clouds for creating their scalable WebRTC services. These clouds are private and usually expose some kind of API that developers consume for accessing WebRTC communication capabilities such as group-communications, media recording or media transcoding. This model is very convenient for creating simple vertical applications, but it also has some drawbacks. First, these clouds billing models are based on pay-per-minute schemes that are not always compatible with the business models desired by application developers. Second, the exposed APIs tend to be restrictive and limit the freedom of developers for innovating. Third, these clouds, being fully private, cannot be extended or customized for specific needs developers may have.
In this talk, we propose an alternative solution to private WebRTC PaaS models based on leveraging novel cloud orchestration technologies such as Cloud Formation or Heat Templates for enabling developers to create their very own platform on top of public clouds. We analyze the technological ingredients required for enabling such models and present some experiments and production deployments showing how any developer may bring her own IaaS cloud for growing on it a full featured WebRTC platform with elastic scalability and full control on the billing and on the underlying technology evolution.
Token Authentication for Java ApplicationsStormpath
Everyone building a web application that supports user login is concerned with security. How do you securely authenticate users and keep their identity secure? With the huge growth in Single Page Applications (SPAs), JavaScript and mobile applications, how do you keep users safe even though these are 'unsafe' client environments?
This presentation will demystify HTTP Authentication and explain how the Next Big Thing - Token Authentication - can be used to secure web applications on the JVM, REST APIs, and 'unsafe' clients while supporting security best practices and even improving your application's performance and scale.
Microservices are an essential enabler of agility but developing and deploying them is a challenge. In order for microservices to be loosely coupled,each service must have its own datastore. This makes it difficult to maintain data consistency across services.
Deploying microservices is also a complex problem since an application typically consists of 10s or 100s of services, written in a variety of languages and frameworks. In this presentation, you will learn how to solve these problems by using an event-driven architecture to maintain data consistency and by using Docker to simplify deployment.
Developing microservices with aggregates (SpringOne platform, #s1p)Chris Richardson
The Domain Model pattern is a great way to develop complex business logic. Unfortunately, a typical domain model is a tangled, birds nest of classes. It can’t be decomposed into microservices. Moreover, business logic often relies on ACID transactions to maintain consistency.
Fortunately, there is a solution to this problem: aggregates. An aggregate is an often overlooked modeling concept from the must read book Domain Driven Design. In this talk you will learn how aggregates enable you to develop business logic for the modern world of microservices and NoSQL. We will describe how to use aggregates to design modular business logic that can be partitioned into microservices. You will learn how aggregates enable you to use eventual consistency instead of ACID. We will describe the design of a microservice that is built using aggregates, and Spring Cloud.
Handling Eventual Consistency in JVM Microservices with Event Sourcing (javao...Chris Richardson
This is the talk that the Kenny Bastani and I gave at JavaOne 2016.
When you’re building JVM applications in a microservice architecture, managing state becomes a distributed systems problem. Instead of being able to manage state as transactions inside the boundaries of a single monolithic application, a microservice must be able to manage consistency by using transactions that are distributed across a network of many different applications and databases. This session explores the problems of data consistency and high availability in JVM-based microservices and how to use event sourcing to solve these problems.
This presentation on building servers explains what is Netty, why choosing it and shows how with very little code you can build an asynchronous app server.
This is a small introduction to microservices. you can find the differences between microservices and monolithic applications. You will find the pros and cons of microservices. you will also find the challenges (Business/ technical) that you may face while implementing microservices.
Concurrency at Scale: Evolution to Micro-ServicesRandy Shoup
Most large-scale web companies have evolved their system architecture from a monolithic application and monolithic database to a set of loosely coupled micro-services. Using examples from Google, eBay, and KIXEYE, this talk outlines the pros and cons of these different stages of evolution, and makes practical suggestions about when and how other organizations should consider migrating to micro-services. It concludes with some more advanced implications of a micro-services architecture, including SLAs, cost-allocation, and vendor-customer relationships within the organization.
“ The Microservices architecture has many appealing qualities, but the road towards it has painful traps for the unwary. This book will help you figure out if this path is for you, and how to avoid those traps on your journey.”
—Martin Fowler Chief Scientist, ThoughtWorks
the cloud computing and its importance in modern day is mentioned in this ppt. and all necessary document required to study the cloud computing is been added this ppt along with required documents.
Exploring the drivers behind the Microservices hype, and defining the prerequisites in architecture and infrastructure needed before contemplating this path.
Presented at the first Sydney Microservices Meetup - Small Talk.
The microservice architectural style is an approach to developing a single application as a suite of small services, each running in its own process and communicating with lightweight mechanisms, often an HTTP resource API.
In this slide we have discussed, Monolithic application vs Microservices, applicable scenarios for adopting the architectural pattern, when we need microservices, what are the benefits, case study of an e-commerce platform by compartmentalizing the scopes into different sample microservices and Docker implementations.
The full talk has been recorded here: https://youtu.be/tNlp7HS533g
Phil Green - We're migrating to the cloud - Who needs service managementitSMF UK
This presentation explored the importance of service
management in the cloud and explore what is needed to build an operating model for the governance, assurance, and day to day operation of cloud services.
MuCon 2015 - Microservices in Integration ArchitectureKim Clark
Discusses the how microservices fit into the ever evolving integration architecture, looking at how these concepts are often seen very differently through the eyes of enterprises with different lanscapes.
Comparing Service-Oriented Architecture (SOA), Microservices and Service-Based Architecture (SBA - SOA and Microservices Hybrid) patterns.
Also discussing coupling and cohesion concepts in relation to the systems design.
Software Architecture and Architectors: useless VS valuableComsysto Reply GmbH
Abstract:
This talk introduces definitions of system architecture and proposes a way to achieve "good enough" architecture covers project requirements
Andrei will show several cases from real projects, where wrong, missing or over-sophisticated architecture decisions really hurt the development teams:
Painful sharing: do shared modules increase reusability or will be the source of problems?
Non-extensible extensibility: too sophisticated configuration hurts
Over fine-grained: incorrect splitting to microservices can make life even harder as with monolith
Cargo cult: blindly following patterns and rules can produce an unmaintainable system
Freestyle architecture: what happens if teams completely ignore architecture
Improve with less intelligence: smart endpoint and dumb pipes
We are looking forward to meet many of you in person and have great discussions around this topic!
https://www.meetup.com/de-DE/meetup-group-tfyvuydp/
This is a must-read for all engineers interested in developing a Micro services architecture. Turn your monolithic server into a prolific and multiple instance solution! Includes well-known example such as Netflix. Please contact me for more details.
How do effective large-scale service ecosystems work? Keynote Presentation at Istanbul Tech Talks 2018
How to Design Services
* Systems of record
* Interface specification
* Interface backward / forward compatibility
Service Ecosystems
* Layered services
* "Standardization" through encouragement
* Vendor-customer relationships between teams
Operating and Deploying Services
* Data Migration
* Automated Pipelines
* Incremental Deployment
* Feature Flags
Are you jumping on the microservices bandwagon? When and when not to adopt micro services architecture? If you must, what are the considerations? This slidedeck will help answer a few of those questions...
Similar to The "Why", "What" and "How" of Microservices (20)
Dev Dives: Train smarter, not harder – active learning and UiPath LLMs for do...UiPathCommunity
💥 Speed, accuracy, and scaling – discover the superpowers of GenAI in action with UiPath Document Understanding and Communications Mining™:
See how to accelerate model training and optimize model performance with active learning
Learn about the latest enhancements to out-of-the-box document processing – with little to no training required
Get an exclusive demo of the new family of UiPath LLMs – GenAI models specialized for processing different types of documents and messages
This is a hands-on session specifically designed for automation developers and AI enthusiasts seeking to enhance their knowledge in leveraging the latest intelligent document processing capabilities offered by UiPath.
Speakers:
👨🏫 Andras Palfi, Senior Product Manager, UiPath
👩🏫 Lenka Dulovicova, Product Program Manager, UiPath
The Art of the Pitch: WordPress Relationships and SalesLaura Byrne
Clients don’t know what they don’t know. What web solutions are right for them? How does WordPress come into the picture? How do you make sure you understand scope and timeline? What do you do if sometime changes?
All these questions and more will be explored as we talk about matching clients’ needs with what your agency offers without pulling teeth or pulling your hair out. Practical tips, and strategies for successful relationship building that leads to closing the deal.
Connector Corner: Automate dynamic content and events by pushing a buttonDianaGray10
Here is something new! In our next Connector Corner webinar, we will demonstrate how you can use a single workflow to:
Create a campaign using Mailchimp with merge tags/fields
Send an interactive Slack channel message (using buttons)
Have the message received by managers and peers along with a test email for review
But there’s more:
In a second workflow supporting the same use case, you’ll see:
Your campaign sent to target colleagues for approval
If the “Approve” button is clicked, a Jira/Zendesk ticket is created for the marketing design team
But—if the “Reject” button is pushed, colleagues will be alerted via Slack message
Join us to learn more about this new, human-in-the-loop capability, brought to you by Integration Service connectors.
And...
Speakers:
Akshay Agnihotri, Product Manager
Charlie Greenberg, Host
Neuro-symbolic is not enough, we need neuro-*semantic*Frank van Harmelen
Neuro-symbolic (NeSy) AI is on the rise. However, simply machine learning on just any symbolic structure is not sufficient to really harvest the gains of NeSy. These will only be gained when the symbolic structures have an actual semantics. I give an operational definition of semantics as “predictable inference”.
All of this illustrated with link prediction over knowledge graphs, but the argument is general.
GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using Deplo...James Anderson
Effective Application Security in Software Delivery lifecycle using Deployment Firewall and DBOM
The modern software delivery process (or the CI/CD process) includes many tools, distributed teams, open-source code, and cloud platforms. Constant focus on speed to release software to market, along with the traditional slow and manual security checks has caused gaps in continuous security as an important piece in the software supply chain. Today organizations feel more susceptible to external and internal cyber threats due to the vast attack surface in their applications supply chain and the lack of end-to-end governance and risk management.
The software team must secure its software delivery process to avoid vulnerability and security breaches. This needs to be achieved with existing tool chains and without extensive rework of the delivery processes. This talk will present strategies and techniques for providing visibility into the true risk of the existing vulnerabilities, preventing the introduction of security issues in the software, resolving vulnerabilities in production environments quickly, and capturing the deployment bill of materials (DBOM).
Speakers:
Bob Boule
Robert Boule is a technology enthusiast with PASSION for technology and making things work along with a knack for helping others understand how things work. He comes with around 20 years of solution engineering experience in application security, software continuous delivery, and SaaS platforms. He is known for his dynamic presentations in CI/CD and application security integrated in software delivery lifecycle.
Gopinath Rebala
Gopinath Rebala is the CTO of OpsMx, where he has overall responsibility for the machine learning and data processing architectures for Secure Software Delivery. Gopi also has a strong connection with our customers, leading design and architecture for strategic implementations. Gopi is a frequent speaker and well-known leader in continuous delivery and integrating security into software delivery.
Generating a custom Ruby SDK for your web service or Rails API using Smithyg2nightmarescribd
Have you ever wanted a Ruby client API to communicate with your web service? Smithy is a protocol-agnostic language for defining services and SDKs. Smithy Ruby is an implementation of Smithy that generates a Ruby SDK using a Smithy model. In this talk, we will explore Smithy and Smithy Ruby to learn how to generate custom feature-rich SDKs that can communicate with any web service, such as a Rails JSON API.
LF Energy Webinar: Electrical Grid Modelling and Simulation Through PowSyBl -...DanBrown980551
Do you want to learn how to model and simulate an electrical network from scratch in under an hour?
Then welcome to this PowSyBl workshop, hosted by Rte, the French Transmission System Operator (TSO)!
During the webinar, you will discover the PowSyBl ecosystem as well as handle and study an electrical network through an interactive Python notebook.
PowSyBl is an open source project hosted by LF Energy, which offers a comprehensive set of features for electrical grid modelling and simulation. Among other advanced features, PowSyBl provides:
- A fully editable and extendable library for grid component modelling;
- Visualization tools to display your network;
- Grid simulation tools, such as power flows, security analyses (with or without remedial actions) and sensitivity analyses;
The framework is mostly written in Java, with a Python binding so that Python developers can access PowSyBl functionalities as well.
What you will learn during the webinar:
- For beginners: discover PowSyBl's functionalities through a quick general presentation and the notebook, without needing any expert coding skills;
- For advanced developers: master the skills to efficiently apply PowSyBl functionalities to your real-world scenarios.
GraphRAG is All You need? LLM & Knowledge GraphGuy Korland
Guy Korland, CEO and Co-founder of FalkorDB, will review two articles on the integration of language models with knowledge graphs.
1. Unifying Large Language Models and Knowledge Graphs: A Roadmap.
https://arxiv.org/abs/2306.08302
2. Microsoft Research's GraphRAG paper and a review paper on various uses of knowledge graphs:
https://www.microsoft.com/en-us/research/blog/graphrag-unlocking-llm-discovery-on-narrative-private-data/
DevOps and Testing slides at DASA ConnectKari Kakkonen
My and Rik Marselis slides at 30.5.2024 DASA Connect conference. We discuss about what is testing, then what is agile testing and finally what is Testing in DevOps. Finally we had lovely workshop with the participants trying to find out different ways to think about quality and testing in different parts of the DevOps infinity loop.
Transcript: Selling digital books in 2024: Insights from industry leaders - T...BookNet Canada
The publishing industry has been selling digital audiobooks and ebooks for over a decade and has found its groove. What’s changed? What has stayed the same? Where do we go from here? Join a group of leading sales peers from across the industry for a conversation about the lessons learned since the popularization of digital books, best practices, digital book supply chain management, and more.
Link to video recording: https://bnctechforum.ca/sessions/selling-digital-books-in-2024-insights-from-industry-leaders/
Presented by BookNet Canada on May 28, 2024, with support from the Department of Canadian Heritage.
Software Delivery At the Speed of AI: Inflectra Invests In AI-Powered QualityInflectra
In this insightful webinar, Inflectra explores how artificial intelligence (AI) is transforming software development and testing. Discover how AI-powered tools are revolutionizing every stage of the software development lifecycle (SDLC), from design and prototyping to testing, deployment, and monitoring.
Learn about:
• The Future of Testing: How AI is shifting testing towards verification, analysis, and higher-level skills, while reducing repetitive tasks.
• Test Automation: How AI-powered test case generation, optimization, and self-healing tests are making testing more efficient and effective.
• Visual Testing: Explore the emerging capabilities of AI in visual testing and how it's set to revolutionize UI verification.
• Inflectra's AI Solutions: See demonstrations of Inflectra's cutting-edge AI tools like the ChatGPT plugin and Azure Open AI platform, designed to streamline your testing process.
Whether you're a developer, tester, or QA professional, this webinar will give you valuable insights into how AI is shaping the future of software delivery.
Smart TV Buyer Insights Survey 2024 by 91mobiles.pdf91mobiles
91mobiles recently conducted a Smart TV Buyer Insights Survey in which we asked over 3,000 respondents about the TV they own, aspects they look at on a new TV, and their TV buying preferences.
Smart TV Buyer Insights Survey 2024 by 91mobiles.pdf
The "Why", "What" and "How" of Microservices
1. The “Why”, “What” and “How”
of Microservices
Jeppe Cramon - @jeppec
Chief Architect - INPAY
2. WHY WOULD WE WANT TO USE
MICROSERVICES?
And what are the alternatives?
3. Cost of Software Development
• The 60/60 rule*
• The cost of software maintenance consumes 40%-80%
(60 % in average)
• Of this, 60% is consumed by enhancements
• And only 17% is consumed by error corrections
* "Frequently Forgotten Fundamental Facts about Software Engineering" by Robert L. Glass, IEEE Software May/June 2001
4. What are the sources of enhancements?
• New business requirements
• Changed processes (e.g. direct booking)
• Changed calculations (new pricing models)
• New features (space yield)
• New corporate strategies
• Ability to sell individual “components” of our application
• Management decisions to purchase 3rd party applications
• Upgrades/replacement to existing 3rd party systems
• Replacing existing functionality in our application with new components/systems (e.g. best
of breed or golf course deals)
9. WE ALL WANT TO MOVE FAST
But can’t always move fast -
for various reasons…
10. WHAT IS THE MAIN TECHNICAL,
BUSINESS AND ARCHITECTURAL
IMPEDIMENT TO OBTAINING HIGH
VELOCITY?
11. High Coupling
• Coupling is a measure of dependencies:
• Efferent coupling – who do you depend on
• Afferent coupling – who depends on you
• 4 different aspects on coupling:
• Temporal coupling (processing time of Y affects X – if Y is down can X continue?)
• Platform coupling (e.g. using platform specific protocols)
• Spatial coupling (where on the network are the Y instances and how does X know?)
• Behavioral coupling (how much about Y’s behavior does X understand)
X Y
14. Coupling causes ripple effect –
much like circles in the water
• You want to change a little thing and all of a sudden you need to change 25
other seemingly unrelated things
• Zero coupling is impossible
• Question is what is the right level of coupling?
• This highly depends on how likely the component/system/service is it to change and
what parts that change together
19. AS AN UMBRELLA NAME WE CALL
THESE SYSTEMS MONOLITHS
Which roughly translates to mean that they deployed as one
unit
20. Characteristics of a monolith?
• Often an indivisible unit
• Hard to adapt to new technology
• Low Reusability
• Functionality of a part cannot be reused alone
• Slow Delivery train
• Introducing a new feature often requires coordination with other features to
deliver all of them at the same time
• Higher and higher maintenance cost over time
• Change to control flow is often hard
• Starting the application often takes a long time
• Testing the application often takes a long time
• Forces high requirements on mental capacity to keep the entire
monolith in your head
• Tends to grow bigger and bigger and bigger
• Reliability
• When one thing fails => potentially everything fails (e.g. due to
OutOfMemoryException)
21. MONOLITHS ARE UNFAIRLY
CATEGORIZED AS BEING BAD
Many of the problems mentioned before can be solved using
design principles that we will investigate in this talk
22. Advantages of Monolith
• Can take advantage of locality:
• In memory calls and avoid distributed transactions
• Can perform Joins with other components SQL tables
• Can take advantage of development IDE’s
• Refactoring
• Code completion
• Code searching
• Less spinning wheels
• Less deployment and monitoring requirements
25. To many this is SOA
Data
Storage
Data
Storage
Data
Storage
Data
Service
Data
Service
Data
Service
Activity
Service
Activity
Service
Process
Service
Process
Service
Client Client
Client
Data
Service
If one service is
down, you risk
everything is down
26. Classic Service reuse multiplies our direct
and especially indirect dependencies
which creates high coupling
My new
Service that
wants to reuse
other services
Service that is
going to be
reused
Another
Service that is
going to be
reused
Reusable
Service
Reusable
Service
Reusable
Service
Reusable
Service
Reusable
Service
DB
service
27. The cost/benefit of reuse
• Layered SOA has typically been recommended because it increases reuse
• Really, this is use and not reuse.
• Use results in more coupling
• The thesis is that the more we reuse, the faster we will be done
• But the thesis rest on the false assumption that writing code is the most expensive
part of a project
• Not all code takes the same amount of time to write. Some code is very trivial while other
code is very hard to write
• The real money and time consumers on any project are:
• Figuring out what the customer needs
• The time it takes for the customer to figure what they really needed and the resulting rework
• Meetings
• UI, Integration work, databases, webservices
• Fix bugs
• Debugging
• Deployment
• Test
• Ship
Yes
No
Failed
?
The more
dependencies we have,
the worse this is and
the more expensive it
becomes
28. Where to go?
• Monoliths easily result in:
• Tight coupling
• Slow release train - All or nothing deployment
• Low reusability
• Reliability issues
• SOA done traditionally easily result in:
• A hard coupled architecture due to reusability and integration
patterns
• Both runtime (temporally) and design time (contracts)
• Performance issues
• Reliability issues
• Slow release train – requires a lot of governance
• High latency (2 way remote communication)
• High accidental complexity
• Compensations due to lack of transactions
• ESB’s to try and hide the mess
30. SOME SAY A MICROSERVICE IS THE
SAME AS A SOA SERVICE – JUST
SMALLER
That of course requires us to answer what a Service is ;-)
31. THE FINE-GRAINED, STATELESS, SELF- CONTAINED
NATURE OF MICROSERVICES CREATES DECOUPLING
BETWEEN DIFFERENT PARTS OF A CODE BASE AND IS
WHAT MAKES THEM
EASY TO UPDATE, REPLACE, REMOVE, OR AUGMENT.
Microservices: The resurgence of SOA principles and an
alternative to the monolith
32. IT SEEMS THE ARGUMENT IS THAT
JUST BY MAKING THINGS FINE
GRAINED
We by magic achieve decoupling
33. Let’s transform our monolith to microservices
“The” DB
UI
Logic
Microservices
Data Access
Microservices
UI-3
MS-3
MS-C
UI-2
MS-D
UI-4
MS-4MS-2
MS-B MS-E
UI-1 UI-5
MS-5MS-1
MS-A
34. THERE IS VALUE IN MAKING
THINGS SMALLER
For one thing it is easier to reason about them in isolation
35. Going small has its advantages
Monolith
Microservice Microservice
Microservice MicroserviceMicroservice
Microservice
Microservice
41. Nano Services
Unless we have a very good reason for doing so,
we risk building services
that are so fine-grained
that their costs outweigh their utility*
*Read Arnon Rotem-Gal-Oz’s Nano Services Anti Pattern:http://arnon.me/wp-content/uploads/2010/10/Nanoservices.pdf
42. IS YOUR MICROSERVICE
VALUABLE?
The value of a microservice must
exceed the cost of building &
operating it.
Microservices entail costs for serializations, deserializations,
security, communication, maintenance, configuration,
deployment, monitoring, etc.
43. FOR NOW ON WE WILL USE
SERVICE AND MICROSERVICE
INTERCHANGEABLY
A Service CAN be broken into smaller parts, but let’s leave
that out of the discussion for now…
44. Too small services
With too small services there’s a big risk that what’s left of coherence goes
out the window and we get:
• Communication-related and Layered coupling
• E.g. Logic and persistence are not the same service
• Temporal coupling
• Our service can not operate if it is unable to communicate with the services it depends upon
• Behavioral coupling
• Our services assume behaviors with regards to what and who.
• The sender/client determines what to do and knows something about how the receiver/server
should satisfy the request
47. Microservice characteristics
• Small
• Single Responsibility Principle (SRP)
• Own process
• Valuable
• Replaceable/Upgradeable
• Autonomous
• Independent
• Encapsulated
• Composable
• Testable
• Fast startup/shutdown
• Client friendly
48. SRP
This sounds really good but what about cross
Service relationships?
Customer Orders
Products
49. BEWARE…
When we break up big things into small
pieces we invariably push the complexity to
their interaction.
Michael Feathers
https://michaelfeathers.silvrback.com/microservices-until-macro-complexity
50. Let’s zoom in on the data tier
MS-A
“The” DB
MS-C
MS-D
MS-B
MS-E
51. IF OUR MONOLITHS CODE IS COUPLED AND MESSY
Chances are that the data(base) model is equally coupled
and messy
60. Let’s refactor the data tier
UI
Logic
Microservices
Data Access
Microservices
UI-3
MS-3
MS-C
UI-2
MS-D
UI-4
MS-4MS-2
MS-B MS-E
UI-1 UI-5
MS-5MS-1
MS-A
A’s
DB
B’s
DB
C’s
DB
D’s
DB
E’s
DB
MS-C
MS-D
MS-B
MS-E
MS-A A’s
DB
B’s
DB
C’s
DB
D’s
DB
E’s
DB
62. Things that are not Services
• A Service with only functionality (and no data) is a FUNCTION
• Like: check if order is valid
• A Service that only has data is a DATABASE
• Like: Entity CRUD
• A database already has a nice API - we don’t need to bubble wrap it with REST or Asynchronous messages
• Don’t split the atom – we need cohesion as well as decoupling!
• If we want datastore abstraction (so we can swap out Postgresql with Mongo or Redis) there this little
pattern called Respository.
This is typically seen in a lot of layered SOA usages where a function calls a
function that calls a function that calls a database
63. 4 tenets of Service Orientation
1. Services share contract and schema, not class or type
• Encapsulation improved. We need not worry about how things are implemented
(languages or platform dependent information)
2. Service interaction is controlled by a policy
• Controls the rules/form of technical communication between services take place (e.g.
using encryption incl. algorithm and keys)
3. Services have explicit boundaries
• A Service is defined by its Boundary
• What is on the inside and which is on the outside ownership wise (data and logic)
• A boundary tells us what parts make up the service and how the service interacts with
the outside world (as defined by the service contract – see point 1 and 2).
• The internal implementation and platform of a Service (what’s inside the boundary) is
irrelevant
4. Services are autonomous
• Encapsulation & Cohesion at a bigger scale.
• A service is autonomous if it doesn’t rely on other services to complete its job
• A service can be rewritten, redeployed and replaced without interfering with other
services as long as schema/contracts are upheld
64. This is different from Functions & Components
• Functions are very intimately linked with code that uses them
• Components form a higher modularization level than functions
• But they’re still intimate with the application they’re added to
• Services are yet another modularization level than components.
• They’re standalone, own their own data and are loosely coupled to their surroundings
65. Let’s refactor the “microservices”
UI
Microservices
UI-3
MS-3
UI-2 UI-4
MS-4MS-2
UI-1 UI-5
MS-5MS-1
1’s
DB
2’s
DB
3’s
DB
4’s
DB
5’s
DB
68. Synchronous calls lower our tolerance for faults
• When you get an IO error
• When servers crash or restarts
• When databases are down
• When deadlocks occurs in our databases
• Do you retry?
With synchronous style Service interaction we can loose business data, there’s no automatic retry or we risk creating
data more than once because idempotence* often is an after though
68
Client Server
Duplicated
Response
Duplicated Request
Processing
Response
Request Processing
The same message can be
processed more than once
*Idempotence describes the quality of an
operation in which result and state does not
change if the operation is performed more
than 1 time
71. SOA PRINCIPLE
SERVICES ARE AUTONOMOUS
Autonomy means that our service is independent and self-
contained and as far as possible doesn’t directly depend on
other services to be functional.
77. Clarification of Autonomy vs. Authority
Definition of Autonomy
A service is autonomous if it doesn’t directly depend on another application to
complete its work. It can determine on its own what to do.
Definition of Authority
A service is the authority if other services needs to ask it for data or instruct it to
perform a task on their behalf for them to complete their job
Autonomy
Authority
78. IF WE KEEP UP THE SAME HABITS
THAT RESULTED IN MESSY
MONOLITHS
We will end up with microservice spaghetti!
79. Monolith A Monolith B
Monolith C Monolith D
Service
A.1
Service
A.2
Service
A.3
Service
A.4
Service
B.1
Service
B.2
Service
B.3
Service
B.4
Service
D.1
Service
D.4
Service
D.3
Service
D.2
Service
C.4
Service
C.3
Service
C.2
Service
C.1
DB DB
DB DB
Service transformation
81. IF WE CONTINUE DOWN
THIS PATH WE HAVE GONE
FROM BAD TO WORSE
82. THIS MEANS THAT WE NEED TO TAKE
CARE OF HOW WE DESIGN OUR DATA
MODELS & THEREBY OUR SERVICE
BOUNDARIES
83. GUIDANCE CAN BE FOUND IN
Pat Hellands
“Life Beyond Distributed Transactions? An Apostate ‘s Opinion”
Link: http://www-db.cs.wisc.edu/cidr/cidr2007/papers/cidr07p15.pdf
84. Life Beyond Distributed Transactions?
According to Pat Helland, we must find the solution to our problem by looking
at:
1. How do we split our data / services
2. How do we identify our data
3. How do we communicate between our services
85. 1. How do we split our data / services
Data must be collected in pieces called aggregates. These aggregates should
be limited in size (but not smaller), so that, after a transaction they are
consistent.
Rule of thumb:
One transaction involves only one aggregate.
87. Aggregates
Invoice
InvoiceLine
*
Account *
What:
• Cluster coherent Entities and Value Objects,
with complex associations into Aggregates
with well defined boundaries.
• Choose one entity to be root and control
access to objects inside the boundary
through the root.
• External objects hold references to the root
• Aggregates only refer to other aggregates
by identity (their id)
Motivation:
Control invariants and consistency through the aggregate root.
Enables: Loading schemes, coarse grained locking and…
Ensuring consistency & transactional boundaries for Distributed
scenarios
Root
*
*
91. In this case we can fix it
All use cases shows that
LegalEntity and Address
are created, changed
and deleted together
Parent - Child
92. An improved aggregate design
Address
turned out to
NOT be an
Entity
LegalEntity is an Entity
and forms the Root of our
LegalEntity aggregate
93. THIS GIVES US THIS RULE OF
THUMB
1 use case = 1 transaction = 1 aggregate
94. With this improved design
“Client” that wants
to use our
LegalEntity
Service
LegalEntity Service
95. 2. How do we identify our data
According to Pat Helland we need to be able to uniquely identify each
Aggregate using an ID.
• This ID will usually a UUID/GUID
• Aggregates refer to each other by their ID
• they NEVER use memory pointers, join tables or remote calls
{21EC2020-3AEA-4069-A2DD-08002B30309D}
2122 (approximately 5.3×1036) combinations
97. 3. How we communicate between our services
• What do we do when our use case involves more than one aggregate and
therefore likely more than one service?
98. WITH CROSS SERVICE INTEGRATION
WE’RE BOUND BY THE LAWS OF
DISTRIBUTED COMPUTING
99. The 8 Fallacies of Distributed Computing
These fallacies are assumptions architects, designers and developers of
distributed systems are likely to make. The fallacies will be proven
wrong in the long run - resulting in all sorts of troubles and pains for
the solution and architects who made the assumptions.
1. The network is reliable.
2. Latency is zero.
3. Bandwidth is infinite.
4. The network is secure.
5. Topology doesn't change.
6. There is one administrator.
7. Transport cost is zero.
8. The network is homogeneous.
See http://www.rgoarchitects.com/Files/fallacies.pdf for a walkthrough of the fallacies and why they’re fallacies
100. A DISTRIBUTED SYSTEM IS ONE
WHERE A MACHINE I’VE NEVER
HEARD OF CAN CAUSE MY PROGRAM
TO FAIL.
— Leslie Lamport
101. 2 WAY (SYNCHRONOUS) COMMUNICATION IS THE
ROOT OF MANY PROBLEMS BECAUSE IT CREATES A
STRONG TEMPORAL COUPLING
(AS WELL AS BEHAVIORAL COUPLING)
Substituting synchronous with synchronous-
over-asynchronous doesn’t solve the core
problem
102. Change Address Response
Consumer
Change Address Request
Provider
RPC or Request/Response - Synchronous 2 way communication
Remote Procedure Call
Request/Reply – Asynchronous 2 way communication
Consumer Provider
Change Address Request
Change Address
Request
Change Address
Reply
Change Address
Reply
Request
Channel
Reply
Channel
VS.
103. THE PROBLEM ISN’T SO MUCH SYNCHRONOUS VS
ASYNCHRONOUS
The question is it two-way or one-way communication and what’s the use
case (e.g. is there a user sitting and waiting)
104. Essential complexity of 2 way integration
Component
C
Compnent
B
Component
A
UI
Service Service
B:Service()
call C:Service()
call A:Service()
commit()
Service
Local transaction between
System A, B and C
105. Accidental complexity from 2 way style integration
Component
C
Component
B
System A
UI
Service Service Service
Local transaction
between System B and C
B:Service()
call C:Service()
call A:Service()
if (A:Call-Failed:Too-Busy?)
Wait-A-While()
call A:Service()
if (A:Call-Failed:Too-Busy?)
Wait-A-Little-While-Longer()
call A:Service()
if (A:Call-Failed:IO-Error?)
Save-We-Need-Check-If-Call-A-Succeded-After-All
AND We-Need-To-Retry call C:Service and call B:Service
AND Tell-Customer-That-This-Operation-Perhaps-Went-Well
if (A:Call-Went-Well?)
commit()
106. HOW CAN WE ENSURE CONSISTENCY
WHEN WE’RE GOING DISTRIBUTED?
Can’t we just use distributed transactions?
(XA / 2 phase commit)
107. Distributed Transactions…. Oh my
Sales system
Sales Delivery
system Deliveries
Customer/C
RM system Customer
SAP
Book-
keeping
Complete
Purchase
Transaction
Coordinator Transactional
Resource
Prepare
Phase
Commit
Phase
2 Phase Commit
108. What’s wrong with distributed transactions?
• Transactions lock resources while active
• Services are autonomous
• Can’t be expected to finish within a certain time interval
• Locking keeps other transactions from completing their job
• Locking doesn’t scale
• And what about timeouts?
• X Phase Commit is fragile by design
110. THIS FORM OF COORDINATED
INTEGRATION BETWEEN
MULTIPLE SERVICES IS KNOWN
AS ORCHESTRATION
111. Consequence: Availability goes down
(without additional instances of each service)
Service
A
Service
B
Service
C
Availability: 99% Availability: 99% Availability: 99%
Combined availability: 97%
112. Synchronous calls are the crystal meth of programming
At first you make good progress but then the sheer horror
becomes evident when you realise the scalability
limitations and how the brittleness holds back both
performance and development flexibility. By then it is too
late to save.
http://www.infoq.com/news/2014/10/thompson-reactive-manifesto-2
We need the reactive properties and then apply
protocols for the message interactions. Without
considering the protocols of interaction this world
of micro-services will become a coordination
nightmare.
Martin Thompson
113. DECIDE IF YOU CAN LIVE WITH THE
CONSEQUENCES OF COUPLING
SERVICES TO EACH OTHER USING
REQUEST/RESPONSE
Different situations – different tradeoffs
114. HOW TO GET LOOSE COUPLING
BETWEEN SERVICES?
It highly depends on how you integrate them
116. IF WE WANT TO DECOUPLE OUR SERVICES AS MUCH
AS POSSIBLE
THEN WE NEED TO LOOK TOWARDS
COMPOSITE UI’S
AND
EVENTS
117. WHAT’S A COMPOSITE UI
A Composite UI is a way to allow different services to
participate an applications UI without revealing their
internals and thereby removing the need for other
services to know the this services internal data
This helps us keep coupling low and encapsulation high
118. Applications
• Applications are tailored solutions for certain groups of users.
Example
• Departments will typically have UI’s tailed to the responsibility of a department (which
often cut across Service responsibilities)
• Managers which see different reports from all areas of a company
• Sales representatives which have solutions that sometimes can work in offline mode
• Customers using e.g. Android/iOS apps to access their portfolio / home-banking /
appointments / etc.
119. Applications and Services
iOS Homebanking Call center support portal
Bank Backoffice application
Customer information service
Legal and contract information service
Accounts service
Credit card service
Mortgage loans service
120. Who owns the UI?
• For a service to be fully autonomous
is must be self contained – which
means it must:
• Own its UI
• Own its Business Logic
• Own its Data model & storage
User interface
Business Logic
Data model &
storage
Service
A similar approach is available under the name Self Contained Systems http://scs-architecture.org/
121. If a Service doesn’t own its UI we often find the need for
a Gateway or Backend For a Frontend (BFF)
• Experience shows if you get
your Service API wrong the
first time around, it is really
expensive to fix it*
• The granularity of APIs
provided by microservices is
often different than what a
client needs*
* See http://thenewstack.io/microservices-calls-robust-api-management-tools/
122. Gateway or Backend For a Frontend (BFF)
• Unfortunately Gateways & BFF’s
introduce a lot of coupling
between the Gateway and the
underlying services
• If an underlying service changes
its contract in a non-backwards
compatible way the gateway
needs to change
• In these cases the clients of the
gateway may also need to change
since the changes can permeate
upwards (the gateway abstraction is
leaky)
* See http://thenewstack.io/microservices-calls-robust-api-management-tools/
124. Application UI’s
• An applications is a composition of different services
• The composition can be:
• Resource Oriented Client Architecture (ROCA) style service
integration (http://roca-style.org/)
• Mashup of Service UI components in a dedicated Application –
aka. Composite UI
Both solutions involve
integration via Services
web interfaces to minimize
coupling to other services.
Image from http://scs-architecture.org/
125. ROCA style integration
Simple hyperlinks can be used to navigate between services
UI’s.
Image from http://scs-architecture.org/
126. ROCA style integration
A redirection can be used to ensure
navigation works in both directions.
Image from http://scs-architecture.org/
127. ROCA style integration
Hyperlinks can also support the
dynamic inclusion of content that is served by another
application into the web interface of a service.
Image from http://scs-architecture.org/
129. A SERVICE OWNS IT UI IN ALL
CONTEXTS AND FOR ALL COMPOSITE
UI’S
Not just for HTML clients
130. Invoice Composite UI example
InvoiceHeader
Order:ShippingI
nfo
Invoice:
InvoiceNumber
Invoice:
Data and Due
date
Order:
RelationInformation
Order:Item-
Qty
Product:Ite
m
Product:
Description
Order:
Item-Unit-Price
Order:
Item-
Total-
Price
Order:Total
Billing:Balance
All Services
participate at
the UI level for
each individual
Item in the
Order
131. Coupling matrix*
* Modified version of Ian Robinson’s matrix: http://iansrobinson.com/2009/04/27/temporal-and-behavioural-coupling/
Behavioral
coupling
Temporal
coupling
Low High
Low
High
Event oriented Command oriented
Emergency services Distributed 3 layer
132. TO BREAK TEMPORAL COUPLING &
BEHAVIORAL COUPLING
SERVICES NEEDS TO COMMUNICATE
ASYNCHROUSLY USING BUSINESS EVENTS
MESSAGES
Services communicate facts without making
assumptions about what other services intend
to do with the events
133. Let’s make the implicit explicit!
Old wisdom seems to have been forgotten. Let’s introduce:
Domain Events
Which:
• Signal that something has happened
• Closely aligned to the Domain Model
• Are handled by a messaging system
• They are in the past tense:
• CustomerBilled
• ParcelShipped
• CustomerCreated
• ReviewCreated
• CommentAdded
• CommentDeleted
134. Event Message
An Event message is non-prescriptive of what should happen in other services. It leaves this open to
the recipients, so that they themselves determine what to do based on occurrence of the event.
Events always carry a name in its past-tense form: OrderWasAccepted, OrderHasShipped,
CustomerWasReimbursed
Other qualities
• Immutable, i.e. content cannot be changed
• Always carries the ID of the Aggregate it relates to
• An event can and will typically will be published to multiple consumers.
• The publisher of the event does not know who the recipients are
• And it doesn’t know what the recipients intend to do with the event
• The service issuing the Event is the owner of the Event contract
“An Event describes something that HAS
happened”
135. Business Event as XML Message
<OrderWasAccepted>
<CustomerId>50D1F244-ABBC-4EC7-BDCA-E4934C124A89</CustomerId>
<OrderId>C199322A-01F1-4E56-918E-7A63529F8FA3</OrderId>
<ShippingAddress> ... </ShippingAddress>
<BillingAddress> ... </BillingAddress>
<Items>
<Item ProductId="4CD22C4B-600C-4477-B5BF-48ABDEE4DA61" Amount="100"
AmountUnit="Pieces" UnitPrice="100,10" UnitCurrency="EUR"/>
<Item ProductId="56E6BD19-660C-464A-9120-100DAF579855" Amount="10"
AmountUnit="Litres" UnitPrice="56,95" UnitCurrency="CHF"/>
</Items>
</OrderWasAccepted>
137. Events are often the side effect of Commands
A Command message is prescriptive of what should happen. This is a stronger form of coupling than Events.
A Command’s primary goal is to capture USER INTENT
A Command supports a single usecase and targets a single Aggregate
Commands always carry a name in its imperative form: CreateOrder, ShipOrder, CancelOrder, ReimburseCustomer, etc.
Other qualities
• Immutable, i.e. content cannot be changed
• Always carries the ID of the Business Object that it relates to
• A command can only be handled by ONE service
• The Operations that consume Commands don’t return values – but can throw exceptions
• Therefore Commands should be handled using 2 way communication, so the exception can be communicated back to the sender
“A command describes a Task that you want someone else to
carry out for you and the recipient can reject the Command”
138. Commands & Events
Commands mutate Aggregate/Business Object state which results in one or
more Events being published
Command Event(s)
AcceptOrder OrderAccepted
ShipOrder OrderShipped
AddComment CommentAdded
QuarantineReview ReviewQuarantined
UnquarantineReview ReviewUnquarantined
140. One way Messaging
• One way messaging is asynchronous and breaks
temporal coupling
• Messages can be exchanged between services over a
message channel
• The Message channel is responsible for delivering the
message(s) to the relevant parties (consumers). If
something goes wrong, the message will be put back on
the Message channel (rollback) and will be resent later*
Sender
Address Changed
Event
Receiver
Address Changed Event
Channel
Asynchronous Communication – Aka Messaging
* This is smart pipes and “dumb” endpoints – An alternative is dumb pipes and smart endpoints
141. SPLITTING INTO SMALLER AND MORE
FOCUSED SERVICES HAS MANY
ADVANTAGES
But it leaves us with the challenge on what
to do when processes span multiple
services
142. THE CHANGE OF FOCUS FROM
TECHNICAL SHORT TRANSACTIONS
To long running business transactions
143. BUSINESS EVENTS MESSAGES
AND BUSINESS PROCESSES
By publishing Events messages from our Services
we can communicate with each other and also
drive Business Processes
143
144. This form of architecture is called an
Event Driven Architecture (EDA)
146. Using Business Events to drive Business Processes
Sales Service
Shipping
Billing
Sales
Customers
MessageChannel
Online Ordering System
Web Shop
(Composite UI)
Billing Service
Shipping Service
Order
Accepted
Event
AcceptOrder
Command
The sales
fulfillment
processing
can now
begin…
147. Business Events example
Sales Service
Order
Accepted
Billing Service
Online Ordering System
Order
Accepted
Customer
Billed
MessageChannel
We use the Order Accepted event message published from the Sales Service
to drive the Billing of the customer.
The billing part of the process also use Business Events, in this case Customer Billed
Event, to indicate that its part of the process is completed.
Because we use asynchronous messaging we can still accept orders in the sales service
even though the invoicing services is down. The Order Accepted event message will
remain in the Message Channel until the Billing Service is ready to process it.
148. What about race conditions?
• Usually they are a natural part of the business and needs to be considered as
a natural part of the process – technology and 2 Phase Commit wont solve
this*
• This challenges how we look at the process
• Based on the questions we ask ourselves
• And what we promise customers
• Example:
• Is it really not possible to purchase Taxfree even though the seating system doesn’t know
our booking or concrete seating?
• This is known as being causally consistent (a level of eventual consistent)
* See http://www.udidahan.com/2010/08/31/race-conditions-dont-exist/
149. Eventual consistency can be our friend
• Consistency is within each Service
• Eventual consistency is between Services
• Like in the real world
Sales Invoicing
Inventory Shipping
Order
Accepted
Invoice Customer
Checks Inventory for availability Books truck driver
Order
Packaged
Fetch Package from Inventory
Customer
Invoiced
151. Choreographed Event Driven Processes
Online Ordering System
Sales Service
Order
Accepted
Invoicing Service
Order Fulfilment
(Saga/
Process-Manager)
Shipping Service
MessageChannel
Order
Accepted
Order
Accepted
Customer
Billed
Customer
Billed
Order
Approved
Order
Approved
Works as a Finite
State Machine
(WorkFlow)
handling the life
cycle of Shipping
and thereby forms a
very central new
Aggregate in the
System
152. Process Managers
• Process Managers are essential to the coordination and monitoring
of long running business processes/transactions
• They work as a Finite State Machines (WorkFlow) which handling the
life cycle of Process (e.g. Shipping an Order) and thereby forms a
very central new Aggregate in the System
• They can include manual steps/person intervention
• Sometimes these Process Managers belong naturally within a
specific Business capability and other times they are truly a thing by
themselves and therefore often form a new business capabilities
Many companies derive their competitive advantages from their Processes.
A Process Manager allows you coordinate Business Processes on the basis of
Events
153. Choreography is very different from the classical
orchestrated integration process
154. What should Events look like?
• Keep Events Small
• Only include relevant information (time, id’s unless you use events for
duplication)
• If possible, don’t include unnecessary id’s for aggregates in other services
• Think Inverse relationships based on what feels most natural
• Order doesn’t need to know the Shipping Id, but Shipping can know the Order Id
• Anchor Events to Time
• Include information about how long you can trust the Event’s change
• E.g. a price is valid until 8pm tomorrow night
• Reveal the intention and time anchoring as part of the Events
name
157. Many perspectives on data
Online Retail System
Product
Unit Price
Promotional Price
Promotion End Date
Stock Keeping Unit (SKU)
Quantity On Hand (QOH)
Location Code
Price
Quantity Ordered
Name
The lifecycle of the data is VERY important!
Customer
Pricing
Inventory
Sales
Management Reporting
158. Different perspectives on entities
With in a given Domain, e.g. Retail, there will exist multiple bounded contexts/sub-domains/business
capabilities such as:
• Product management
• Purchase
• Sales
• Pricing
• Inventory
• Shipping
• Support
• Accounting
• Management
Each of these lines of business have very specific and unique needs which are relevant for them alone in order
to conduct their business. They might use the same name for the entities they’re interested in or they might use
different names for the same logical entity.
159. Smaller models & clear data ownership
Retail System
Pricing
Product
ProductID
Unit Price
Promotional
Price
…
Pricing
Inventory
Product
ProductID
SKU
QOH
Location Code
…
Inventory
Sales
Product
ProductID
Name
Description
Quantity
Ordered
…
Sales
Shared Entity identitySOA:
Service
161. These splits should be fairly small (but not too small)
and share as little as possible with each other
Sales Pricing InventoryShipping
✕ ✕ ✕
162. What’s a macro architecture
• It’s the static/stable(r) parts of your architecture
• Which are very costly to refactor and change
• Business capabilities are stable
• Therefore we should strive to align services with business
capabilities / bounded contexts (DDD)
163. BOUNDED CONTEXT
The setting in which a word or a
statement appears that determines its
meaning
Ensures that our
is clear and precise
164. DDD DOESN’T ENFORCE
PROBLEM DOMAIN AND
SOLUTION DOMAIN
ALIGNMENT FOR BOUNDED
CONTEXTS
Which is why we prefer to use the term Business Capability
165. Service and Business Capability alignment
“The advantage of business capabilities is their remarkable level
of stability. If we take a typical insurance organisation, it will likely
have sales, marketing, policy administration, claims management,
risk assessment, billing, payments, customer service, human
resource management, rate management, document management,
channel management, commissions management, compliance, IT
support and human task management capabilities. In fact, any
insurance organisation will very likely have many of these
capabilities.”
See http://bill-poole.blogspot.dk/2008/07/business-
capabilities.html
166. A Service is
• The technical authority for a given business capability
• It is the owner of all the data and business rules that support this business
capability – everywhere (including the UI)
• It forms a single source of truth for that capability
• This form of business and IT alignment ensures that we can maintain service
Autonomy & Encapsulation
169. Services, Applications and code
• Each Service and Application is maintained within its own Git repository
• Common IT Operations libraries and infrastructure are maintained in separate
Git repositories
170. So what’s inside a Service source repository?
• Autonomous Components
• Libraries
• Adapters
• Front-end UI components
• API contracts (mostly Events)
• Schemas
• Build file(s)
172. Service and deployment
• A Service represents a logical responsibility
boundary
• Logical responsibility and physical deployment of a Service DOES
NOT have to be 1-to-1
• It’s too constraining
• We need more degrees of freedom
• Philippe Krutchen 4+1 views of architecture: Logical and Physical designs
should be independent of each other
A service needs to be deployed everywhere its data is needed
173. Service deployment
• Many services can be deployed to the same physical server
• Many services can be deployed in the same application
• Application boundary is a Process boundary which is a physical boundary
• A Service is a logical boundary
• Service deployment is not restricted to tiers either
• Part of service A and B can be deployed to the Web tier
• Another part of Service A and B can be deployed to the backend/app-service
tier of the same application
• The same service can be deployed to multiple tiers / multiple applications
• ie. applications and services are not the same and does not share the
same boundaries
• Multiple services can be “deployed” to the same UI page (service mashup)
• Multiple services can cooperate to fulfill a use-case (e.g. a workflow or a
business process)
174. Service
Autonomous
Component
1..*
Is implemented by
A Service is the technical authority of
a specific Business Capability
e.g. Sales, Shipping, Billing
Services support business processes.
Business processes naturally span multiple services, but
there will always be a single service that is the actual
authority on the business process.
Service vs Autonomous Components
175. Also known as Microservices
Service
Autonomous
Component
1..*
Is implemented by
Service vs Autonomous Components
Autonomous-Components/Microservices are a division of Services along Transactional
boundaries (a transaction stays within the boundary of a Microservice)
Microservices are the individually logical deployable units of a Service with their own
Endpoints. Could e.g. be the split between Read and Write models (CQRS) - each would be
their own Microservice
176. Services are the corner stone
• We talk in terms of Services/business capabilities and the processes/use-
cases they support
• Autonomous-Components/Microservices are an implementation detail
• They are much less stable (which is a good thing – it means they’re easier to replace)
• With regards to other Services
• subscribe to events from
• send commands to (less common)
• call operations (in rare occasions)
177. Where does that leave Systems?
• Systems and Services are in my
opinion the same
• For a service to be autonomous is
must be self contained – which
means it must:
• Own its UI
• Own its Business Logic
• Own its Data model & storage
• Which is similar to how Self
Contained Systems are/were build
User interface
Business Logic
Data model &
storage
Service / System
178. Services/Bounded Contexts and Aggregates
Sales
Service
PSP
ServiceVirtual Banking
Service
Finance
Service
Customer
customerId
…
Contract
contractId
customerId
…
VBFeeSchedule
contractId
…
PSPFeeSchedule
contractId
…
BillingTemplate
contractId
…
181. Commands and Events
public class RegisterBankCmd extends AbstractCommand {
@TargetAggregateIdentifier
public final BankId bankId;
public final String name;
public final Country countryOfOperation;
public RegisterBankCmd(BankId bankId, String name, Country countryOfOperation) {
Objects.requireNonNull(bankId, "bankId");
Objects.requireNonNull(name, "name");
Objects.requireNonNull(countryOfOperation, "countryOfOperation");
this.bankId = bankId;
this.name = name;
this.countryOfOperation = countryOfOperation;
}
}
182. Commands and Events
The generic business event interface includes the
name of the topic that subscribers must use
public class BankRegistered extends AbstractEvent implements BankEvent {
@AggregateIdentifier
public final BankId bankId;
public final String name;
public final Country countryOfOperation;
public BankCreated(BankId bankId, String name, Country countryOfOperation) {
this.bankId = bankId;
this.name = name;
this.countryOfOperation = countryOfOperation;
}
}
public interface BankEvent extends Serializable {
TopicName TOPIC_NAME = BankingServiceId.ID.topicName("bank-events");
}
183. CQRS
A single model cannot be
appropriate for reporting,
searching and transactional
behavior
Greg Young, 2008
184. Query/Read models
• To offload the Aggregate models, i.e. the transactional
write model, we introduce the concept of Read Models
• Query/Read Models are projections of one or more
Aggregate models which store and represent data in the
best possible way for the given Query use cases
UI Application Domain Write
model
Commands – Change data
UI Application Read
models
Queries – Ask for data (no side effects)
185. Commands, Events and Query Models
Read model
Read model
Events
UI
Domain modelQuery/Read model
”AcceptOrder”
command
”OrderAccepted”
event
”Find all
Accepted Orders”
Query
Commands are Imperative: DoStuff
Events are Past tense: StuffDone
187. Event Sourcing
Aggregates track their own Domain Events
and derive state from them
Time
07:39
Time
07:40
Time
07:41
Time
07:45
Time
07:46
Time
07:50
188. Relational Database EventStore example
Time
07:39
Time
07:40
Time
07:41
Time
07:45
Time
07:46
Time
07:50
Type Aggregate
Identifier
Sequence
Number
Timestamp Event
Identifier
EventType SerializedEvent
Order 14237 0 2014-01-06 7:39 {Guid-1} OrderCreated <serialized event>…
Order 14237 1 2014-01-06 7:40 {Guid-2} ProductAdded <serialized event>…
Order 14237 2 2014-01-06 7:41 {Guid-3} ProductAdded <serialized event>…
Order 14237 3 2014-01-06 7:45 {Guid-4} ProductRemoved <serialized event>…
Order 14237 4 2014-01-06 7:46 {Guid-5} ProductAdded <serialized event>…
Order 14237 5 2014-01-06 7:50 {Guid-6} OrderAccepted <serialized event>…
Order 14238 0 2014-01-07 9:10 {Guid-X} OrderCreated <serialized event>…
DomainEvents Table
189. Event Replaying
Type Aggregate
Identifier
Sequence
Number
Timestamp Event
Identifier
EventType SerializedEvent
Order 14237 0 2014-01-06 7:39 {Guid-1} OrderCreated <serialized event>…
Order 14237 1 2014-01-06 7:40 {Guid-2} ProductAdded <serialized event>…
Order 14237 2 2014-01-06 7:41 {Guid-3} ProductAdded <serialized event>…
Order 14237 3 2014-01-06 7:45 {Guid-4} ProductRemoved <serialized event>…
Order 14237 4 2014-01-06 7:46 {Guid-5} ProductAdded <serialized event>…
Order 14237 5 2014-01-06 7:50 {Guid-6} OrderAccepted <serialized event>…
Order
Accepted: false
190. Event Replaying
Type Aggregate
Identifier
Sequence
Number
Timestamp Event
Identifier
EventType SerializedEvent
Order 14237 0 2014-01-06 7:39 {Guid-1} OrderCreated <serialized event>…
Order 14237 1 2014-01-06 7:40 {Guid-2} ProductAdded <serialized event>…
Order 14237 2 2014-01-06 7:41 {Guid-3} ProductAdded <serialized event>…
Order 14237 3 2014-01-06 7:45 {Guid-4} ProductRemoved <serialized event>…
Order 14237 4 2014-01-06 7:46 {Guid-5} ProductAdded <serialized event>…
Order 14237 5 2014-01-06 7:50 {Guid-6} OrderAccepted <serialized event>…
Order
Orderline
Accepted: false
191. Event Replaying
Type Aggregate
Identifier
Sequence
Number
Timestamp Event
Identifier
EventType SerializedEvent
Order 14237 0 2014-01-06 7:39 {Guid-1} OrderCreated <serialized event>…
Order 14237 1 2014-01-06 7:40 {Guid-2} ProductAdded <serialized event>…
Order 14237 2 2014-01-06 7:41 {Guid-3} ProductAdded <serialized event>…
Order 14237 3 2014-01-06 7:45 {Guid-4} ProductRemoved <serialized event>…
Order 14237 4 2014-01-06 7:46 {Guid-5} ProductAdded <serialized event>…
Order 14237 5 2014-01-06 7:50 {Guid-6} OrderAccepted <serialized event>…
Order
Orderline
Orderline
Accepted: false
192. Event Replaying
Type Aggregate
Identifier
Sequence
Number
Timestamp Event
Identifier
EventType SerializedEvent
Order 14237 0 2014-01-06 7:39 {Guid-1} OrderCreated <serialized event>…
Order 14237 1 2014-01-06 7:40 {Guid-2} ProductAdded <serialized event>…
Order 14237 2 2014-01-06 7:41 {Guid-3} ProductAdded <serialized event>…
Order 14237 3 2014-01-06 7:45 {Guid-4} ProductRemoved <serialized event>…
Order 14237 4 2014-01-06 7:46 {Guid-5} ProductAdded <serialized event>…
Order 14237 5 2014-01-06 7:50 {Guid-6} OrderAccepted <serialized event>…
Order
Orderline
Accepted: false
193. Event Replaying
Type Aggregate
Identifier
Sequence
Number
Timestamp Event
Identifier
EventType SerializedEvent
Order 14237 0 2014-01-06 7:39 {Guid-1} OrderCreated <serialized event>…
Order 14237 1 2014-01-06 7:40 {Guid-2} ProductAdded <serialized event>…
Order 14237 2 2014-01-06 7:41 {Guid-3} ProductAdded <serialized event>…
Order 14237 3 2014-01-06 7:45 {Guid-4} ProductRemoved <serialized event>…
Order 14237 4 2014-01-06 7:46 {Guid-5} ProductAdded <serialized event>…
Order 14237 5 2014-01-06 7:50 {Guid-6} OrderAccepted <serialized event>…
Order
Orderline
Orderline
Accepted: false
194. Event Replaying
Type Aggregate
Identifier
Sequence
Number
Timestamp Event
Identifier
EventType SerializedEvent
Order 14237 0 2014-01-06 7:39 {Guid-1} OrderCreated <serialized event>…
Order 14237 1 2014-01-06 7:40 {Guid-2} ProductAdded <serialized event>…
Order 14237 2 2014-01-06 7:41 {Guid-3} ProductAdded <serialized event>…
Order 14237 3 2014-01-06 7:45 {Guid-4} ProductRemoved <serialized event>…
Order 14237 4 2014-01-06 7:46 {Guid-5} ProductAdded <serialized event>…
Order 14237 5 2014-01-06 7:50 {Guid-6} OrderAccepted <serialized event>…
Order
Orderline
Orderline
Accepted: true
195. Snapshotting
Type Aggregate
Identifier
Latest
Sequence
Number
Timestamp Serialized Aggregate
Order 14237 4 2014-01-06 7:46 <serialized aggregate>…
Product 15918 1 2014-01-07 7:40 <serialized aggregate>…
Type Aggregate
Identifier
Sequence
Number
Timestamp Event
Identifier
EventType SerializedEvent
Order 14237 0 2014-01-06 7:39 {Guid-1} OrderCreated <serialized event>…
Order 14237 1 2014-01-06 7:40 {Guid-2} ProductAdded <serialized event>…
Order 14237 2 2014-01-06 7:41 {Guid-3} ProductAdded <serialized event>…
Order 14237 3 2014-01-06 7:45 {Guid-4} ProductRemoved <serialized event>…
Order 14237 4 2014-01-06 7:46 {Guid-5} ProductAdded <serialized event>…
Order 14237 5 2014-01-06 7:50 {Guid-6} OrderAccepted <serialized event>…
Snapshot Table
Domain Events Table
196. Full CQRS
With EventSourcing
UI Domain
Event
Store
(can e.g. be a real
EventStore DB or
a Relational DB)
Commands – Change data
Commands Events
SQL DB Document DB Graph DB
UI Data
Queries – Ask for data
Events
Query Build
Our single
source of truth
197. Aggregate
public class Bank extends InPayEventSourcedAggregate {
@AggregateIdentifier
private BankId bankId;
@EventSourcedMember
private Map<BankAccountId, BankAccount> bankAccounts = new HashMap<>();
public Bank(BankId bankId, String name, Country countryOfOperation) {
apply(new BankRegistered(bankId, name, countryOfOperation));
}
public void addBankAccount(BankAccountId bankAccountId, …) {
if (!bankAccounts.containsKey(bankAccountId)) {
apply(new BankAccountAdded(bankId,
bankAccountId,
…));
}
}
@EventSourcingHandler
private void on(BankRegistered e) {
bankId = e.getBankId();
}
@EventSourcingHandler
private void on(BankAccountAdded e) {
BankAccount bankAccount =
bankAccounts.put(e.getBankAccountId(), new BankAccount(e.getBankId(), e.getBankAccountId(), …));
}
}
198. Service project structure sales_service
sales_api
sales_contract_ac
sales_customer_ac
frontend
sales_contract_manager_adapters
contract
adapters
sales/modules
customer
psp_service
psp_api
psp_fees_ac
…
frontend
psp_contract_manager_adapters
fees
adapters
psp/modules
fx
Legend:
Service/Business-Capability
External Event/Command Contracts
Autonomous Component
Autonomous Component Adapter
Angular.js Module
Builds into a Java JAR
Uses Spring MVC/REST
All Java 8 artifacts are built
using Gradle
199. Autonomous Component
• Can be deployed alone or co-located
• Usually deployed together with one or more adapters
• Works transparently in a clustered environment
• Completely Spring free
• Only depends on our Core infrastructure library
• Common types and Id’s
• Bus infrastructure
• Service lookup
• CQRS building blocks (backed by Axon framework)
200. AC in code public class PSPAgreementAc extends HzBackedAutonomousComponent {
public static AutonomousComponentId SERVICE_AC_ID = PSP_SERVICE_ID.ac("psp_agreement_ac");
…
public PSPAgreementAc(CurrencyConverter currencyConverter) {
this.currencyConverter = currencyConverter;
}
@Override
public void onInitialize(IConfigureACEnvironment acSetup) {
acSetup.withAutonomousComponentId(SERVICE_AC_ID).usingServiceDataSource()
.withBusConfiguration(cfg -> {
cfg.getAxonContext()
.subscribeAnnotatedCommandHandler(new TemplateCmdHandler(
cfg.getAxonContext().eventSourcedRepository(PSPTemplate.class),
currencyConverter));
….
manageLifecycleFor(templateViewRepository = new TemplateViewRepository(cfg, currencyConverter));
})
.runOnBusStartup((bus, axonContext) -> {
bus.registerAxonReplayableTopicPublisher(InternalTemplateEvents.TOPIC_NAME,
replayFromAggregate(PSPTemplate.class)
.dispatchAggregateEventsOfType(InternalTemplateEvents.class));
bus.subscribeTopic(SERVICE_AC_ID.topicSubscriber("ContractEvents"),
ExternalContractEvents.TOPIC_NAME,
new SalesTopicSubscription(bus));
});
}
public TemplateViewRepository getTemplateViewRepository() { return templateViewRepository; }
}
201. Distributed in memory views
public class PSPFeeScheduleViewRepository extends HzBackedDataRepository<ContractId, PSPFeeScheduleView> implements Lifecycle {
private final Bus bus;
private Subscription subscription;
public PSPFeeScheduleViewRepository(HazelcastInstance hazelcastInstance, String serviceId, Bus bus) {
super(hazelcastInstance, serviceId);
this.bus = bus;
}
@Override
public void start() {
subscription = bus.subscribeTopic(TopicSubscription.FromBeginning(PSPFeeScheduleAc.SERVICE_AC_ID.subscriberId("-FeeScheduleView"),
PSPFeeScheduleRelated.TOPIC_NAME),
new BusMessageHandlerDelegator<>(this));
}
@Override
public void shutdown() {
if (subscription != null)subscription.cancel();
}
public PSPFeeScheduleView load(ContractId contractId) {
final PSPFeeScheduleView payOutFeeScheduleView = cache.get(contractId);
if (payOutFeeScheduleView == null)
throw new ValidationException(PSPFeeScheduleMessages.feeScheduleDoesNotExist.validationMsg(contractId));
return payOutFeeScheduleView;
}
@BusMessagePayloadHandler
private void on(PSPFeeScheduleCreated event) {
cache.put(event.contractId, new PSPFeeScheduleView(event));
}
@BusMessagePayloadHandler
private void on(PSPFeeScheduleDiscarded event) {
cache.remove(event.contractId);
}
...
}
203. The core of the infrastructure
• It’s the core infrastructure that allows services/AC’s to communicate using
messages using our Federated Bus.
• This means we don’t need to deploy any Message Brokers (such as
RabbitMQ, ActiveMQ) – the Bus supports:
• Broadcast
• Notifications
• Non durable Queues
• Durable Queues
• Non-durable Topics
206. Non-durable Topics and Client handled subscriptions
• Highly resilient pattern for an Event Driven Architecture that’s backed by AC’s that
use EventSourcing.
• In this model the publisher of the Events is responsible for the durability of all its
Events, typically to an EventStore/EventLog.
• Each client (subscriber) maintains durable information of the last event it received
from each publisher.
• When ever the client starts up it makes a subscription to the publisher where it
states from which point in time it wants events published.
• This effectively means that publisher can remain simple and the client (subscriber)
can remain simple and we don’t need additional sophisticated broker infrastructure
such as Kafka+ZooKeeper.
207. Client handled subscriptions
Publisher
Subscriber
A
Local storage
EventStore
Subscriber
B
Local storage
Topic
Subscription
Topic
Subscription
TopicSubscriptionHandler
TopicSubscriptionHandler
EventEvent
Event Event
EventBus
Event
Event
211. Autonomous Components can be co-deployed together
with Application backends
contract_manager (Spring Boot fat–jar)
sales_contract_ac
sales_customer_ac
sales_contract_manager_adapters
psp_api
psp_fees_ac
psp_contract_manager_adapters
frontend
sales_api
app
libs
contract
customer
fees
212. Application in code
@Configuration
@ComponentScan(basePackages = { "com.inpay.contractmanager",
"com.inpay.adapters",
"com.inpay.itops.spring" })
public class Application extends InpaySpringBootApplication {
public Application() {
super();
}
@Override
protected String getApplicationName() {
return "ContractManager";
}
@Override
protected Collection<AutonomousComponent> getAutonomousComponentsHostedInThisApplication() {
CurrencyExchangeRateAc currencyExchangeRateAc = new CurrencyExchangeRateAc();
return list(
new IDMCoreAc(),
new PSPFeeScheduleAc(currencyExchangeRateAc.getCurrencyConverter()),
new VBFeeScheduleAc(currencyExchangeRateAc.getCurrencyConverter()),
new ContractAc(),
new CustomersAc(),
currencyExchangeRateAc
);
}
public static void main(String[] args) {
SpringApplication.run(Application.class, args);
}
}
213. What about Security/Identity management (IDM)?
• That’s an AC as well, belonging to IT operations
• Technical security belongs with IDM
• May this caller call this endpoint
• Is this User authenticated
• Simple Authorization: Does this User have Privilege A, B and C
• Business related Authorization belongs in the designated Business ACs
• May this user approve a Contract with Fees below X, Y and Z
215. AC, autonomy and “shared” data
Service
DB
DB
Autonomous
Component
Autonomous
Component
Autonomous
Component
Autonomous
Component DB
216. 50 shades of AC Autonomy*
Endpoint Process Database Storage
Shared Shared Shared Shared
Own Shared Shared Shared
Own Own Shared Shared
Own Shared Own Shared
Own Own Own Shared
Own Own Own Own
Lower Autonomy
Higher Autonomy
* No RPC in use!
219. Service A Service B Service C
IT-OPS
Application
App
Application
App logic
(Layered, CQRS,…)
lication
Storage
UI Components
Services & Composite UI’s
220. Widget Widget
Page
Widget
Service A Service B Service C
Widget
Widget
Widget
Service A
Service B
Service C
Widget
Service C
• Overall structure of the page
is “owned” by the application.
• Each widget and the view model
it binds to is delivered by the
underlying Service.
Page layout
221. AngularJS as Composite UI Framework
• Misko Hevery – Don’t fight HTML, extend it!
• For our purpose each widget maps naturally to a directive.
• AngularJS is modular in nature which aligns nicely with
our needs having multiple Services taking part
in an application.
222. Angular Controllers & Services (1 of 2)
• An angular service is normally responsible for gathering
and posting data to the back-end.
• There will be at least one angular service per Service
that participates in the application
• In angular a service is injected in a controller. Not vice
versa.
• Yet application in/output must be coordinated in some
way through the services
• In angular controllers are NOT singletons.
• Each directive get its own instance of a controller,
meaning there are numerous controller instances at
playWhat are the consequences of this with regards to a composite UI?
• An angular service is normally responsible for gathering and
posting data to the back-end.
• There will be at least one angular service per Service that participates in
the application
• In angular a service is injected in a controller. Not vice versa.
• Yet application in/output must be coordinated in some
way through the services
• In angular controllers are NOT singletons.
• Each directive get its own instance of a controller, meaning there are
numerous controller instances at play
223. Angular Controllers & Services (2 of 2)
• Gathering and posting data through angular services
needs to happen in a coordinated manner.
• Client side events
• The controllers instantiated through directives needs to be
kept in sync and caution must be made to avoid
numerous round-trips to the backend Services
• There is a need to have a controller that drives the overall
use-case for a single page. The application page
controller.
• The application page controller must not know the data
structure of any of the participating angular services.
• Gathering and posting data through angular services needs to
happen in a coordinated manner.
• Client side events
• The controllers instantiated through directives needs to be kept in
sync and caution must be made to avoid numerous round-trips to the
backend Services
• There is a need to have a controller that drives the overall use-case
for a single page. The application page controller.
• The application page controller must not know the data structure of
any of the participating angular services.