Should you be using an event driven architecture?Jeppe Cramon
What is an event-driven architecture, what can it do for your architecture, and is it a prerequisite for microservices?
In this talk, we'll see what events are and how they can solve many of the coupling and integration problems that you will experience in a distributed (micro)service landscape.
We'll get into the different usages for events and showcase them based on real world projects.
After this talk, you will know what events are, how they can be used and some of the requirements they put on your design and the way you thinking about software design.
Alternative microservices - one size doesn't fit allJeppe Cramon
A different look at microservices. Is there only one way to approach microservices, what are the pit falls, what protocols and communication patterns should we use. How do we discover service boundaries? What role does applications and the UI play?
Should you be using an event driven architecture - IDA IT (short version)Jeppe Cramon
What is an event-driven architecture, what can it do for your architecture, and is it a prerequisite for microservices?
In this talk, we'll see what events are and how they can solve many of the coupling and integration problems that you will experience in a distributed (micro)service landscape.
We'll get into the different usages for events and showcase them based on real world projects.
After this talk, you will know what events are, how they can be used and some of the requirements they put on your design and the way you thinking about software design.
What 5 years of implementation microservices has taught meJeppe Cramon
This is a sum up of what 15 year of implementation distributed systems has taught me, because many of the mistakes that I see people make with microservices are essentially a repetition of the mistakes that were made more than 10 years ago with layered SOA. We will look at the challenges experienced with poorly designed service boundaries, bad UI integration and what a solution to these challenges may look like in the form of logical services aligned with business capabilities, autonomous components and composite UI's
Should you be using an event driven architecture?Jeppe Cramon
What is an event-driven architecture, what can it do for your architecture, and is it a prerequisite for microservices?
In this talk, we'll see what events are and how they can solve many of the coupling and integration problems that you will experience in a distributed (micro)service landscape.
We'll get into the different usages for events and showcase them based on real world projects.
After this talk, you will know what events are, how they can be used and some of the requirements they put on your design and the way you thinking about software design.
Alternative microservices - one size doesn't fit allJeppe Cramon
A different look at microservices. Is there only one way to approach microservices, what are the pit falls, what protocols and communication patterns should we use. How do we discover service boundaries? What role does applications and the UI play?
Should you be using an event driven architecture - IDA IT (short version)Jeppe Cramon
What is an event-driven architecture, what can it do for your architecture, and is it a prerequisite for microservices?
In this talk, we'll see what events are and how they can solve many of the coupling and integration problems that you will experience in a distributed (micro)service landscape.
We'll get into the different usages for events and showcase them based on real world projects.
After this talk, you will know what events are, how they can be used and some of the requirements they put on your design and the way you thinking about software design.
What 5 years of implementation microservices has taught meJeppe Cramon
This is a sum up of what 15 year of implementation distributed systems has taught me, because many of the mistakes that I see people make with microservices are essentially a repetition of the mistakes that were made more than 10 years ago with layered SOA. We will look at the challenges experienced with poorly designed service boundaries, bad UI integration and what a solution to these challenges may look like in the form of logical services aligned with business capabilities, autonomous components and composite UI's
Not all the WEB APIs are RESTful, some are just plain RPC.
REST is an architecture style.
The video attached to these slides.
https://www.youtube.com/watch?v=M28aBbtdWj4
In this presentation, I will explain event driven architecture, describe the different types of events, demonstrate how events can be related and orchestrated, and provide a basic understanding of how this method can drive the architecture of enterprise systems. In addition to understanding the concepts of event driven architecture, we will explore a working sample built using an open-source .NET messaging framework called MassTransit.
We have identified issues related to composition of a business process and discussed the requirements for event-driven composition and event-driven service-oriented architecture.
Mail is received as a commodity from the cloud, also Collaboration. However, in many client meetings we often hear the question, where are we heading with the hundreds of Notes applications? Which strategy is most effective and cost efficient at the same time? Is cloud a practical answer? With sound and proven methodology Notes applications can be transformed into valuable web applications in the cloud. It turns out that today the time has come for cloud platforms. A side view of large customer projects, already transforming their Notes applications to the cloud - for example to IBM SoftLayer - is helpful. This Track helps you understand that strategies that are implemented and lets you understand the costs and risks involved.
Practical introduction to DDD, CQRS and Event SourcingDennis Doomen
After several in-depth talks about Event Sourcing, I realized that there's a large group of developers that may have heard about Domain-Driven Design, Command-Query Responsibility Segregation and Event Sourcing, but have not really connected the dots yet. So in this talk, I'd like to take a practical example of a simple domain and gradually introduce functional requirements to see how the principles behind DDD affect the way your entities are going to protect the business rules. After that, I'll introduce some real-world non-functional requirements and see if and how Event Sourcing and/or CQRS may or may not help to accomplish those.
Decomposing the Monolith (Riga Dev Days 2019)Dennis Doomen
A micro-service architecture is a great strategy for decomposing a monolith. In this talk, I’ll show you some of the pros and cons of micro-services and how you can leverage OWIN, .NET, Event Sourcing and the Onion Architecture to gradually move your monolith into a bright new future.
If I have to name a single hype in software architecture land then I would have to mention the micro-service architecture. Micro-services are supposed to be small, have a very focused purpose, can be deployed independently, are completely self-supporting and loosely coupled. Ideally, micro-services are technology agnostic, but hey, we’re in the .NET space, aren’t we? And they are not a goal, but a means to an end. In fact, a micro-service architecture has many benefits and are a great strategy for decomposing a monolith. So how do you build a micro-service? What technologies does the .NET realm offer for us? And what if you don’t want to deploy them independently? In this talk, I’ll show you some of the pros and cons of micro-services and how you can leverage OWIN, .NET, Event Sourcing and the Onion Architecture to gradually move your monolith into a bright new future.
Event Sourcing from the Trenches (with examples from .NET)Dennis Doomen
Over the years I've spoken many times about what Event Sourcing is and shared many of the good, the bad and the ugly parts of it in blog posts and various talks. However, I've never talked about how to actually build a system based on this architecture style. I keep getting the same questions over and over again. Like when to apply Event Sourcing and at what architectural level. How to deal with transactional boundaries within and outside the domain. How to build projections that are autonomous, reliable and self-supporting. How to deal with upgrades and blue-green deployments. But also on how to handle bugs, design mistakes and crashing projections. Having made a lot of these mistakes myself over these years, it's time to share my current thoughts and opinions about this. Since the .NET space has a pretty rich set of open-source projections to support this, the examples and code will be .NET. But the concepts are universal, so don't let that scare you off.
In this Meetup Arik Lerner – Liveperson Team lead of Java Automation, Performance & Resilience , will talk about How we measure our services, By End2End testing which become one of the most critical Monitor tool in LP .
Over 200K tests runs per day providing statistics and insights into the problem as they happen.
Arik will go through different topics and stages of the journey and share details that led to current results .
Part of the menu topics are : The Awakens of the End2End Insights
• How we measure our services using synthetic user experience
• Measuring through analytics & insights
• How we collect our data
• How we debug our services? Hint: video recording, HAR (Http archive), KIbana , Dashboard analytics & insights
• Future logs App correlation with End2End data
• Our tools: Selenium, Jenkins and cutting edge technologies such as Kafka & ELK (Elastic search, Logstash and Kibana)
In this Meetup, Arik will host Ali AbuAli- NOC Team Leader , who will talk about the e2e usage on his day 2 day work.
In this Meetup Arik Lerner – Liveperson Team lead of Java Automation, Performance & Resilience , will talk about How we measure our services, By End2End testing which become one of the most critical Monitor tool in LP .
Over 200K tests runs per day providing statistics and insights into the problem as they happen.
Arik will go through different topics and stages of the journey and share details that led to current results .
Part of the menu topics are : The Awakens of the End2End Insights
• How we measure our services using synthetic user experience
• Measuring through analytics & insights
• How we collect our data
• How we debug our services? Hint: video recording, HAR (Http archive), KIbana , Dashboard analytics & insights
• Future logs App correlation with End2End data
• Our tools: Selenium, Jenkins and cutting edge technologies such as Kafka & ELK (Elastic search, Logstash and Kibana)
In this Meetup, Arik will host Ali AbuAli- NOC Team Leader , who will talk about the e2e usage on his day 2 day work.
A talk through the journey we've been through at Snowplow thinking about event data, starting with our focus on web and then mobile analytics, and exploring our current and future technical and analytic approaches
No software engineer is exempt from writing batch jobs, its just part of every software app stack. Lack of a standards and a reusable batch architecture in the Java platform has resulted in the proliferation of many one-off in-house batch solutions.
In this talk I presented about Spring Batch Framework, how it could help an organization to standardize their batching needs. I also talked about a Real World Use Case in Dealer dot com. How we used Spring Batch along with Spring Integration to solve our job concurrency, data flow control, job resiliency & additional requirements - thus enabling us to build beautiful batch jobs.
Not all the WEB APIs are RESTful, some are just plain RPC.
REST is an architecture style.
The video attached to these slides.
https://www.youtube.com/watch?v=M28aBbtdWj4
In this presentation, I will explain event driven architecture, describe the different types of events, demonstrate how events can be related and orchestrated, and provide a basic understanding of how this method can drive the architecture of enterprise systems. In addition to understanding the concepts of event driven architecture, we will explore a working sample built using an open-source .NET messaging framework called MassTransit.
We have identified issues related to composition of a business process and discussed the requirements for event-driven composition and event-driven service-oriented architecture.
Mail is received as a commodity from the cloud, also Collaboration. However, in many client meetings we often hear the question, where are we heading with the hundreds of Notes applications? Which strategy is most effective and cost efficient at the same time? Is cloud a practical answer? With sound and proven methodology Notes applications can be transformed into valuable web applications in the cloud. It turns out that today the time has come for cloud platforms. A side view of large customer projects, already transforming their Notes applications to the cloud - for example to IBM SoftLayer - is helpful. This Track helps you understand that strategies that are implemented and lets you understand the costs and risks involved.
Practical introduction to DDD, CQRS and Event SourcingDennis Doomen
After several in-depth talks about Event Sourcing, I realized that there's a large group of developers that may have heard about Domain-Driven Design, Command-Query Responsibility Segregation and Event Sourcing, but have not really connected the dots yet. So in this talk, I'd like to take a practical example of a simple domain and gradually introduce functional requirements to see how the principles behind DDD affect the way your entities are going to protect the business rules. After that, I'll introduce some real-world non-functional requirements and see if and how Event Sourcing and/or CQRS may or may not help to accomplish those.
Decomposing the Monolith (Riga Dev Days 2019)Dennis Doomen
A micro-service architecture is a great strategy for decomposing a monolith. In this talk, I’ll show you some of the pros and cons of micro-services and how you can leverage OWIN, .NET, Event Sourcing and the Onion Architecture to gradually move your monolith into a bright new future.
If I have to name a single hype in software architecture land then I would have to mention the micro-service architecture. Micro-services are supposed to be small, have a very focused purpose, can be deployed independently, are completely self-supporting and loosely coupled. Ideally, micro-services are technology agnostic, but hey, we’re in the .NET space, aren’t we? And they are not a goal, but a means to an end. In fact, a micro-service architecture has many benefits and are a great strategy for decomposing a monolith. So how do you build a micro-service? What technologies does the .NET realm offer for us? And what if you don’t want to deploy them independently? In this talk, I’ll show you some of the pros and cons of micro-services and how you can leverage OWIN, .NET, Event Sourcing and the Onion Architecture to gradually move your monolith into a bright new future.
Event Sourcing from the Trenches (with examples from .NET)Dennis Doomen
Over the years I've spoken many times about what Event Sourcing is and shared many of the good, the bad and the ugly parts of it in blog posts and various talks. However, I've never talked about how to actually build a system based on this architecture style. I keep getting the same questions over and over again. Like when to apply Event Sourcing and at what architectural level. How to deal with transactional boundaries within and outside the domain. How to build projections that are autonomous, reliable and self-supporting. How to deal with upgrades and blue-green deployments. But also on how to handle bugs, design mistakes and crashing projections. Having made a lot of these mistakes myself over these years, it's time to share my current thoughts and opinions about this. Since the .NET space has a pretty rich set of open-source projections to support this, the examples and code will be .NET. But the concepts are universal, so don't let that scare you off.
In this Meetup Arik Lerner – Liveperson Team lead of Java Automation, Performance & Resilience , will talk about How we measure our services, By End2End testing which become one of the most critical Monitor tool in LP .
Over 200K tests runs per day providing statistics and insights into the problem as they happen.
Arik will go through different topics and stages of the journey and share details that led to current results .
Part of the menu topics are : The Awakens of the End2End Insights
• How we measure our services using synthetic user experience
• Measuring through analytics & insights
• How we collect our data
• How we debug our services? Hint: video recording, HAR (Http archive), KIbana , Dashboard analytics & insights
• Future logs App correlation with End2End data
• Our tools: Selenium, Jenkins and cutting edge technologies such as Kafka & ELK (Elastic search, Logstash and Kibana)
In this Meetup, Arik will host Ali AbuAli- NOC Team Leader , who will talk about the e2e usage on his day 2 day work.
In this Meetup Arik Lerner – Liveperson Team lead of Java Automation, Performance & Resilience , will talk about How we measure our services, By End2End testing which become one of the most critical Monitor tool in LP .
Over 200K tests runs per day providing statistics and insights into the problem as they happen.
Arik will go through different topics and stages of the journey and share details that led to current results .
Part of the menu topics are : The Awakens of the End2End Insights
• How we measure our services using synthetic user experience
• Measuring through analytics & insights
• How we collect our data
• How we debug our services? Hint: video recording, HAR (Http archive), KIbana , Dashboard analytics & insights
• Future logs App correlation with End2End data
• Our tools: Selenium, Jenkins and cutting edge technologies such as Kafka & ELK (Elastic search, Logstash and Kibana)
In this Meetup, Arik will host Ali AbuAli- NOC Team Leader , who will talk about the e2e usage on his day 2 day work.
A talk through the journey we've been through at Snowplow thinking about event data, starting with our focus on web and then mobile analytics, and exploring our current and future technical and analytic approaches
No software engineer is exempt from writing batch jobs, its just part of every software app stack. Lack of a standards and a reusable batch architecture in the Java platform has resulted in the proliferation of many one-off in-house batch solutions.
In this talk I presented about Spring Batch Framework, how it could help an organization to standardize their batching needs. I also talked about a Real World Use Case in Dealer dot com. How we used Spring Batch along with Spring Integration to solve our job concurrency, data flow control, job resiliency & additional requirements - thus enabling us to build beautiful batch jobs.
Oracle Cloud services products, including Planning and Budget Cloud Service (PBCS), enables companies to focus on their own business instead of spending money and resources on maintaining big IT infrastructures. It also gives them the possibility to be connected 24x7 from any place in the world.
But what happens if this company already has an ODI on-premise infrastructure and they want to integrate the new PBCS with it? Can we use our existing ODI on-premise? How hard is to accomplish this?
This session will show how to use your ODI on-premise to integrate and orchestrate your PBCS seamlessly.
Slides for my GGX 2014 talk "Grails and the real time world". The code is available here: https://github.com/lmivan/ggx2014
-----------------
In a hyper-connected world the concept ""Real Time"" is used more and more every day. With the traditional Grails architecture it's difficult to achieve this, so we need to use a different approach. The answer is to use message driven architectures that will allow us to achieve the goal and also build fast, decoupled and easy to test applications.
In this talk you'll see a different type of architecture that will help you to serve content in real-time to a lot of clients in a fast and easy to scale way. You'll see some examples of how to achieve this using Spring Integration and integrate with external systems like websockets and XMPP in an easy and decoupled way.
CDC Tests - Integration Tests cant be made simpler than this!Ramya Authappan
CDCs are “Asynchronous Integration Tests”.CDC stands for Consumer Driven Contract. We can now forget about bringing up all the microservices to test the integration between different microservices.This makes integration tests as simple as unit tests. This is a must-have test for all microservices.
Stateful Stream Processing at In-Memory SpeedJamie Grier
This presentation describes results from a real-world system where I used Apache Flink's stateful stream processing capabilities to eliminate the key-value store bottleneck and the burden of the Lambda Architecture while also improving accuracy and gaining huge improvements in hardware efficiency!
Updated: Should you be using an Event Driven ArchitectureJeppe Cramon
What is an event-driven architecture, what can it do for your architecture, and is it a prerequisite for microservices?
In this talk, we'll see what events are and how they can solve many of the coupling and integration problems that you will experience in a distributed (micro)service landscape.
We'll get into the different usages for events and showcase them based on real world projects.
After this talk, you will know what events are, how they can be used and some of the requirements they put on your design and the way you thinking about software design.
Talk given at DDDx London on 27th of April about how to implement long running processes in Domain Driven Design properly. Describes patterns like Process Manager and Saga.
AWS re:Invent 2016: How Fulfillment by Amazon (FBA) and Scopely Improved Resu...Amazon Web Services
We’ll share an overview of leveraging serverless architectures to support high performance data intensive applications. Fulfillment by Amazon (FBA) built the Seller Inventory Authority Platform (IAP) using Amazon DynamoDB Streams, AWS Lambda functions, Amazon Elasticsearch Service, and Amazon Redshift to improve results and reduce costs. Scopely will share how they used a flexible logging system built on Kinesis, Lambda, and Amazon Elasticsearch to provide high-fidelity reporting on hotkeys in Memcached and DynamoDB, and drastically reduce the incidence of hotkeys. Both of these customers are using managed services and serverless architecture to build scalable systems that can meet the projected business growth without a corresponding increase in operational costs.
In many database applications we first log data and then, a few hours or days later, we start analyzing it. But in a world that’s moving faster and faster, we sometimes need to analyze what is happening NOW.
Azure Stream Analytics allows you to analyze streams of data via a new Azure service. In this session you will see how to get started using this new service. From event hubs on the input side over temporal SQL queries: the demo’s in this session will show you end to end how to get started with Azure Stream Analytics.
EPM environments are generally supported by a Data Warehouse, however, we often see that those DWs are not optimized for the EPM tools. During the years, we have witnessed that modeling a DW thinking about the EPM tools may greatly increase the overall architecture performance.
The most common situation found in several projects is that the people that develops the data warehouse does not have a great knowledge about EPM tools and vice-versa. This may create a big gap between those two concepts which may severally impact performance.
This session will show a lot of techniques to model the right Data Warehouse for EPM tools. We will discuss how to improve performance using partitioned tables, create hierarchical queries with “Connect by Prior”, the correct way to use Multi-Period tables for block data load using Pivot/Unpivot and more. And if you want to go ever further, we will show you how to leverage all those techniques using ODI, which will create the perfect mix to perform any process between your DW and EPM environments.
Near Real-Time Data Analysis With FlyData FlyData Inc.
This document describes our products. FlyData makes it easy to load data automatically and continuously to Amazon Redshift. You can also refer to our HP ( http://flydata.com/ ) for more information.
Similar to Event Driven Architecture (Integration Tech Event 2019) (20)
In software engineering, the right architecture is essential for robust, scalable platforms. Wix has undergone a pivotal shift from event sourcing to a CRUD-based model for its microservices. This talk will chart the course of this pivotal journey.
Event sourcing, which records state changes as immutable events, provided robust auditing and "time travel" debugging for Wix Stores' microservices. Despite its benefits, the complexity it introduced in state management slowed development. Wix responded by adopting a simpler, unified CRUD model. This talk will explore the challenges of event sourcing and the advantages of Wix's new "CRUD on steroids" approach, which streamlines API integration and domain event management while preserving data integrity and system resilience.
Participants will gain valuable insights into Wix's strategies for ensuring atomicity in database updates and event production, as well as caching, materialization, and performance optimization techniques within a distributed system.
Join us to discover how Wix has mastered the art of balancing simplicity and extensibility, and learn how the re-adoption of the modest CRUD has turbocharged their development velocity, resilience, and scalability in a high-growth environment.
Providing Globus Services to Users of JASMIN for Environmental Data AnalysisGlobus
JASMIN is the UK’s high-performance data analysis platform for environmental science, operated by STFC on behalf of the UK Natural Environment Research Council (NERC). In addition to its role in hosting the CEDA Archive (NERC’s long-term repository for climate, atmospheric science & Earth observation data in the UK), JASMIN provides a collaborative platform to a community of around 2,000 scientists in the UK and beyond, providing nearly 400 environmental science projects with working space, compute resources and tools to facilitate their work. High-performance data transfer into and out of JASMIN has always been a key feature, with many scientists bringing model outputs from supercomputers elsewhere in the UK, to analyse against observational or other model data in the CEDA Archive. A growing number of JASMIN users are now realising the benefits of using the Globus service to provide reliable and efficient data movement and other tasks in this and other contexts. Further use cases involve long-distance (intercontinental) transfers to and from JASMIN, and collecting results from a mobile atmospheric radar system, pushing data to JASMIN via a lightweight Globus deployment. We provide details of how Globus fits into our current infrastructure, our experience of the recent migration to GCSv5.4, and of our interest in developing use of the wider ecosystem of Globus services for the benefit of our user community.
Enhancing Project Management Efficiency_ Leveraging AI Tools like ChatGPT.pdfJay Das
With the advent of artificial intelligence or AI tools, project management processes are undergoing a transformative shift. By using tools like ChatGPT, and Bard organizations can empower their leaders and managers to plan, execute, and monitor projects more effectively.
Enterprise Resource Planning System includes various modules that reduce any business's workload. Additionally, it organizes the workflows, which drives towards enhancing productivity. Here are a detailed explanation of the ERP modules. Going through the points will help you understand how the software is changing the work dynamics.
To know more details here: https://blogs.nyggs.com/nyggs/enterprise-resource-planning-erp-system-modules/
top nidhi software solution freedownloadvrstrong314
This presentation emphasizes the importance of data security and legal compliance for Nidhi companies in India. It highlights how online Nidhi software solutions, like Vector Nidhi Software, offer advanced features tailored to these needs. Key aspects include encryption, access controls, and audit trails to ensure data security. The software complies with regulatory guidelines from the MCA and RBI and adheres to Nidhi Rules, 2014. With customizable, user-friendly interfaces and real-time features, these Nidhi software solutions enhance efficiency, support growth, and provide exceptional member services. The presentation concludes with contact information for further inquiries.
First Steps with Globus Compute Multi-User EndpointsGlobus
In this presentation we will share our experiences around getting started with the Globus Compute multi-user endpoint. Working with the Pharmacology group at the University of Auckland, we have previously written an application using Globus Compute that can offload computationally expensive steps in the researcher's workflows, which they wish to manage from their familiar Windows environments, onto the NeSI (New Zealand eScience Infrastructure) cluster. Some of the challenges we have encountered were that each researcher had to set up and manage their own single-user globus compute endpoint and that the workloads had varying resource requirements (CPUs, memory and wall time) between different runs. We hope that the multi-user endpoint will help to address these challenges and share an update on our progress here.
OpenFOAM solver for Helmholtz equation, helmholtzFoam / helmholtzBubbleFoamtakuyayamamoto1800
In this slide, we show the simulation example and the way to compile this solver.
In this solver, the Helmholtz equation can be solved by helmholtzFoam. Also, the Helmholtz equation with uniformly dispersed bubbles can be simulated by helmholtzBubbleFoam.
Listen to the keynote address and hear about the latest developments from Rachana Ananthakrishnan and Ian Foster who review the updates to the Globus Platform and Service, and the relevance of Globus to the scientific community as an automation platform to accelerate scientific discovery.
SOCRadar Research Team: Latest Activities of IntelBrokerSOCRadar
The European Union Agency for Law Enforcement Cooperation (Europol) has suffered an alleged data breach after a notorious threat actor claimed to have exfiltrated data from its systems. Infamous data leaker IntelBroker posted on the even more infamous BreachForums hacking forum, saying that Europol suffered a data breach this month.
The alleged breach affected Europol agencies CCSE, EC3, Europol Platform for Experts, Law Enforcement Forum, and SIRIUS. Infiltration of these entities can disrupt ongoing investigations and compromise sensitive intelligence shared among international law enforcement agencies.
However, this is neither the first nor the last activity of IntekBroker. We have compiled for you what happened in the last few days. To track such hacker activities on dark web sources like hacker forums, private Telegram channels, and other hidden platforms where cyber threats often originate, you can check SOCRadar’s Dark Web News.
Stay Informed on Threat Actors’ Activity on the Dark Web with SOCRadar!
Code reviews are vital for ensuring good code quality. They serve as one of our last lines of defense against bugs and subpar code reaching production.
Yet, they often turn into annoying tasks riddled with frustration, hostility, unclear feedback and lack of standards. How can we improve this crucial process?
In this session we will cover:
- The Art of Effective Code Reviews
- Streamlining the Review Process
- Elevating Reviews with Automated Tools
By the end of this presentation, you'll have the knowledge on how to organize and improve your code review proces
Paketo Buildpacks : la meilleure façon de construire des images OCI? DevopsDa...Anthony Dahanne
Les Buildpacks existent depuis plus de 10 ans ! D’abord, ils étaient utilisés pour détecter et construire une application avant de la déployer sur certains PaaS. Ensuite, nous avons pu créer des images Docker (OCI) avec leur dernière génération, les Cloud Native Buildpacks (CNCF en incubation). Sont-ils une bonne alternative au Dockerfile ? Que sont les buildpacks Paketo ? Quelles communautés les soutiennent et comment ?
Venez le découvrir lors de cette session ignite
Innovating Inference - Remote Triggering of Large Language Models on HPC Clus...Globus
Large Language Models (LLMs) are currently the center of attention in the tech world, particularly for their potential to advance research. In this presentation, we'll explore a straightforward and effective method for quickly initiating inference runs on supercomputers using the vLLM tool with Globus Compute, specifically on the Polaris system at ALCF. We'll begin by briefly discussing the popularity and applications of LLMs in various fields. Following this, we will introduce the vLLM tool, and explain how it integrates with Globus Compute to efficiently manage LLM operations on Polaris. Attendees will learn the practical aspects of setting up and remotely triggering LLMs from local machines, focusing on ease of use and efficiency. This talk is ideal for researchers and practitioners looking to leverage the power of LLMs in their work, offering a clear guide to harnessing supercomputing resources for quick and effective LLM inference.
Software Engineering, Software Consulting, Tech Lead.
Spring Boot, Spring Cloud, Spring Core, Spring JDBC, Spring Security,
Spring Transaction, Spring MVC,
Log4j, REST/SOAP WEB-SERVICES.
Into the Box Keynote Day 2: Unveiling amazing updates and announcements for modern CFML developers! Get ready for exciting releases and updates on Ortus tools and products. Stay tuned for cutting-edge innovations designed to boost your productivity.
May Marketo Masterclass, London MUG May 22 2024.pdfAdele Miller
Can't make Adobe Summit in Vegas? No sweat because the EMEA Marketo Engage Champions are coming to London to share their Summit sessions, insights and more!
This is a MUG with a twist you don't want to miss.
Enhancing Research Orchestration Capabilities at ORNL.pdfGlobus
Cross-facility research orchestration comes with ever-changing constraints regarding the availability and suitability of various compute and data resources. In short, a flexible data and processing fabric is needed to enable the dynamic redirection of data and compute tasks throughout the lifecycle of an experiment. In this talk, we illustrate how we easily leveraged Globus services to instrument the ACE research testbed at the Oak Ridge Leadership Computing Facility with flexible data and task orchestration capabilities.
Top Features to Include in Your Winzo Clone App for Business Growth (4).pptxrickgrimesss22
Discover the essential features to incorporate in your Winzo clone app to boost business growth, enhance user engagement, and drive revenue. Learn how to create a compelling gaming experience that stands out in the competitive market.
Field Employee Tracking System| MiTrack App| Best Employee Tracking Solution|...informapgpstrackings
Keep tabs on your field staff effortlessly with Informap Technology Centre LLC. Real-time tracking, task assignment, and smart features for efficient management. Request a live demo today!
For more details, visit us : https://informapuae.com/field-staff-tracking/
12. @jeppec
What’s wrong with distributed transactions?
• Transactions lock resources while active
• Services are autonomous
• Can’t be expected to finish within a certain time interval
• Locking keeps other transactions from completing their job
• Locking doesn’t scale
• X Phase Commit is fragile by design
13. @jeppec
Learnings
The bad
• Task/Activity services needs to perform UPDATES across multiple data/entity
services. Requires distributed transactions
• The more synchronous request-response remote calls you have to make the
more it hurts Performance and latency.
• Robustness is lower. If one data/entity services is down it can take down
many other services.
• Coupling is higher. Multiple task/activity services manage the same CRUD
data/entity services
• Cohesion is likely lower as multiple task/activity services need to replicate
entity related logic
20. @jeppec
Accidental complexity from distributed service integration
Component
Warehouse
Component
Sales
Billing System
UI
Send-Invoice
Process-Paid-
Order
Reserve-
Items
Local transaction between 2
Components
23. @jeppec
Synchronous calls lower our tolerance for faults
• When you get an IO error
• When servers crash or restarts
• When databases are down
• When deadlocks occurs in our databases
• Do you retry?
With synchronous style Service interaction we can loose business data if there’s no automatic retry
Or we risk creating data more than once if the operation isn’t idempotent*
Client Server
Duplicated Response
Duplicated Request
Processing
Response
Request Processing
The same message can be
processed more than once
*Idempotence describes the quality of an operation
in which result and state does not change if the
operation is performed more than 1 time
24. @jeppec
Sales:Process-Paid-Order()
call Warehouse:Reserve-Items()
call Billing:Send-Invoice()
if (Billing:Call-Failed:Too-Busy?)
Wait-A-While()
call Billing:Send-Invoice()
if (Billing:Call-Failed:Too-Busy?)
Wait-A-Little-While-Longer()
call Billing:Send-Invoice()
if (Billing:Call-Failed:IO-Error?)
Save-We-Need-Check-If-Call-Billing-Succeded-After-All
AND We-Need-To-Retry call Sales:Process-Paid-Order and call Warehouse:Reserve-Items
AND Tell-Customer-That-This-Operation-Perhaps-Went-Well
if (Billing:Call-Went-Well?)
commit()
Accidental complexity from distributed service integration
Component
Warehouse
Component
Sales
Billing System
UI
Send-Invoice
Process-Paid-
Order
Reserve-
Items
Local transaction between 2
Components
29. @jeppec
Using Business Events to drive Business Processes
Sales Service
Shipping
Billing
Sales
Customers
MessageChannel
Online Ordering System
Web Shop
(Composite UI)
Billing Service
Warehouse Service
<<External>>
Order Paid
MarkOrderAsPai
d
The sales
fulfillment
processing can
now begin…
Cmd Handler
Order Paid
Apply
30. @jeppec
Choreographed Event Driven Processes
Sales Service
Order Paid
Billing Service
Shipping Service
Warehouse Service
Online Ordering System
MessageChannel(e.g.aTopic)
Order Paid
Customer
Invoiced
Order Paid
Items
Reserved
Order Paid
Shipping process
works as a Finite
State Machine
(WorkFlow)
handling the life
cycle of Shipping and
thereby forms a very
central new
Aggregate in the
System
Items
Reserved
31. @jeppec
Things are not quite the same
In a distributed systems the order in which messages arrive is not
guaranteed
In a distributed systems message delivery can and will fail!
Messages can depending on guarantees be delivered:
• At Most Once – If you don’t care about loosing messages
• Page visits
• Ad views
• Exactly Once
• Not really possible
• At Least Once
• For everything else – which is why:
Everything that can handle messages must be built with idempotency in mind!
32. @jeppec
Message Handling
public class OrderShippingProcess extends BusMessageHandler {
@BusMessageHandler
private void on(OrderPaid orderPaid, ItemsReserved itemsReserved) {
ShippingDetails shippingDetails = getShippingDetailsForOrder(orderPaid.orderId);
….
printShippingLabel(orderPaid.orderId, shippingDetails.address);
}
…
} Must also be idempotent
38. @jeppec
If we primarily model around nouns/entities we can
easily violate the SRP
Where a change to requirements
is likely to require changes
to multiple entity classes
38
52. @jeppec
Order Id
• Customer/customer-id
Linked to Order Id:
• Order Delivery address
• Order Shipping method
Linked to Order Id:
• Order item quantity
• Order item subtotal
• Order shipping price
• Order total
56. @jeppec
Service
AC - 1
AC - 2AC - 2
AC - 3
AC - 3AC - 4
Broker
Broker
(Kafka,
RabbitMQ,
JMS)
AC - 1AC - 1
AC - 1
AC - 2
AC - 2
AC - 3
AC - 3
AC - 4
AC - 4
AC - 4
57. @jeppec
Service
AC - 1
AC - 1AC - 1
AC - 1
AC - 2
AC - 2
AC - 2
AC - 2
AC - 3
AC - 3
AC - 3
AC - 3
AC - 4
AC - 4
AC - 4
AC - 4
QBus
59. @jeppec
Topics
Bus features
• Decouples publisher from subscribers
• Provides temporal decoupling
• If a subscriber is unavailable it will receive its messages when it comes
online
AC - 4
AC - 1
AC - 3
61. @jeppec
Topics
Bus features
But how do we handle:
• Coding errors in Subscribers?
• New Subscribers that didn’t exist when the events were originally
published?
AC - 4
AC - 1
AC - 3
62. @jeppec
Client handled subscriptions
• Highly resilient pattern for an Event Driven Architecture that’s backed by
Event-Sourced AC’s
• In this model the publisher of the Events is responsible for the durability of
all its Events, typically to an EventStore/EventLog.
• Each client (subscriber) maintains durable information of the last event it has
received from each publisher.
• When ever the client starts up it makes a subscription to the publisher
where it states from which point in time it wants events published/streamed
to it.
• This effectively means that publisher can remain simple and the client
(subscriber) can remain simple and we don’t need additional sophisticated
broker infrastructure such as Kafka+ZooKeeper.
63. @jeppec
Client handled subscriptions
Publisher
Subscriber A
Local storage
EventStore
Subscriber B
Local storage
Topic
Subscription
Topic
Subscription
TopicSubscriptionHandler
TopicSubscriptionHandler
EventEvent
Event Event
EventBus
Event
Event
Distributed Event Bus,
which ensures that
live events published
on an AC node in the
cluster can be seen
by all AC’s of the
same type
Singe Instance
Subscriber, which
ensures that only
one instance of
Subscriber B has
an active
subscription(s).
Other instances of
the same
subscriber are
hot-standby
<<Topic Subscriber>>
Customer_Service:Some_Ac:OrderEvents
<<Topic Publisher>>
Sales_Service:OrderEvents
65. @jeppec
Time
07:39
Time
07:40
Time
07:41
Time
07:45
Time
07:46
Time
07:50
Type Aggregate
Identifier
Sequence
Number
Timestamp Event
Identifier
EventType SerializedEvent
Order 14237 0 2014-01-06 7:39 {Guid-1} OrderCreated <serialized event>…
Order 14237 1 2014-01-06 7:40 {Guid-2} ProductAdded <serialized event>…
Order 14237 2 2014-01-06 7:41 {Guid-3} ProductAdded <serialized event>…
Order 14237 3 2014-01-06 7:45 {Guid-4} ProductRemoved <serialized event>…
Order 14237 4 2014-01-06 7:46 {Guid-5} ProductAdded <serialized event>…
Order 14237 5 2014-01-06 7:50 {Guid-6} OrderAccepted <serialized event>…
Order 14238 0 2014-01-07 9:10 {Guid-X} OrderCreated <serialized event>…
DomainEvents Table
66. @jeppec
Event Replaying
Type Aggregate
Identifier
Sequence
Number
Timestamp Event
Identifier
EventType SerializedEvent
Order 14237 0 2014-01-06 7:39 {Guid-1} OrderCreated <serialized event>…
Order 14237 1 2014-01-06 7:40 {Guid-2} ProductAdded <serialized event>…
Order 14237 2 2014-01-06 7:41 {Guid-3} ProductAdded <serialized event>…
Order 14237 3 2014-01-06 7:45 {Guid-4} ProductRemoved <serialized event>…
Order 14237 4 2014-01-06 7:46 {Guid-5} ProductAdded <serialized event>…
Order 14237 5 2014-01-06 7:50 {Guid-6} OrderAccepted <serialized event>…
Order
Accepted: true
Orderline
Orderline
67. @jeppec
Topics
Bus features
AC - 4
AC - 1
AC - 3
bus.registerReplayableTopicPublisher(InternalPricingEvents.TOPIC_NAME,
replayFromAggregate(Pricing.class)
.dispatchAggregateEventsOfType(
InternalPricingEvents.class
)
);
bus.subscribeTopic(SERVICE_AC_ID.topicSubscriber(”Pricing"),
InternalPricingEvents.TOPIC_NAME,
new PricingTopicSubscription(bus));
69. @jeppec
Topics
Bus features
Features:
• The Bus provides automatic and durable handling of Redelivery in case of message handling failure
through a Redelivery Policy
• Exponential Back-off
• Max retries
• Dead letter/Error Queue
• Support for resubscription at any point in the timeline of an Event Stream
• Automatically tracking of resubscription points - aka. resubscribe at last tracked point
AC - 4
AC - 1
AC - 3
72. @jeppec
Bus features
Features:
• Support for sending a message to a single consumer
• Default pattern is Competing Consumers
• The bus provides durability for all messages sent on a Queue
• The Bus provider automatic and durable handling of Redelivery in case of message handling failure
through a Redelivery Policy
• Exponential Backoff
• Max retries
• Dead letter/Error Queue
Durable Queues
74. @jeppec
Bus features
Notifications:
• Durable notifications with failover support
• bus.notify(notificationQueue, notificationMessage)
Notifications
Distributed
Broadcast
Broadcast:
• Broadcast a Message to all AC’s in the cluster
• Broadcast a Message to a UI client (all, per user, per privilege)
75. @jeppec
Bus features
Single Instance Task:
• Ensures that only one Active instance of a Task is active in the cluster at one time
• Other tasks of the same type are in hot standby
• Used to e.g. group multiple subscribers, to ensure that all subscribers are either all
active or standby.
• Used by our ViewRepositories
Distributed
SingleInstanceTask
bus.createClusterSingleInstanceTask(”MyTask",
new MyTask ()); // Where MyTask implements Lifecycle
76. @jeppec
Bus features
Process Manager
• Defines durable business processes
as a flow of Events
Sagas
Process Manager
Sales Service
Order
Accepted
Billing Service
Order Fulfilment
(Saga/
Process-Manager)
Shipping Service
MessageChannel(e.g.aTopic)
Order
Accepted
Order
Accepted
Customer
Billed
Customer
Billed
Order
Approved
Order
Approved
77. @jeppec
Application features
Workflow Manager:
• Defines the Tasks that required human intervention, such as:
• Approvals (e.g. Contract approval)
• Assistance/Help with a business problem
• Incident handling (e.g. a technical problem identified by a developer)
• Authorization (e.g. request manager approval)
• Reminders
• Common tasks supported: Claiming Tasks, Escalating Tasks, Completing Tasks
Workflow Manager
78. @jeppec
Correlation logging
Our infrastructure automatically captures information about an API call,
a Message delivery in the form of a CallContext:
• Message Id - the Id of the message/call being handled.
• Correlation Id – an Id that binds API calls and message handlings across
multiple services/AC’s together
• When – time of the “call”
• Who – which user performed the “call”
• Meta Data: Which Event or Command caused this “call”
The layers are technically de-coupled but not in terms of data or temporal.
Each layer has a technical integration layer that makes it harder to understand the correlations
Debugging is made more difficult
Single point of failure
Latency is high and scalability is low
Locking and scaling: If it takes 200 ms to carry out an operation that uses scaling, the system can maximum handle 5 concurrent users
Fragility: 2,3,4 .... Phase Commit - 2PC theory concludes by saying "and this does not work in reality" - in case of error (eg. Due to timeout while waiting for a resource commit phase) you always end up with having to decide what two do "Halt and manual recovery" or guess whether it was good or bad!
There are besides timeouts a big problem. At worst, timeouts last very long!
If both System A, B and C share the same technical infrastructure (e.g. database) and reside in the same memory space they can share a local transaction. In this case we’re not bound by the laws of distributed computing and everything is easy because the transaction manager will solve all our problems.
Synchronous RPC is the crack cocaine of distributed programming
When ever you connect 2 Nouns using a Verb you create COUPLING!
The service name Customer have a clear bias (have a name, and age, and address)
However in many domains Age will be more related to risk for e.g. Bank/Creditcard/Insurance