This document summarizes a talk about moving from a monolithic architecture to microservices. It discusses what microservices are, examples of large companies that adopted microservices like Amazon and Netflix, and the monolithic problems at Lendingkart. It then describes how Lendingkart broke up its monolith into multiple microservices for different functions. Some challenges of microservices like distributed tracing and increased operations overhead are also outlined. Best practices for adopting microservices like incremental adoption and clear interfaces are also provided.
Microservices architecture is an architectural style that structures an application as a collection of services that are. Highly maintainable and testable.
Credit goes to Christopher B Ferris @christo4ferris who put together this presentation which covers the latest developments of Hyperledger Fabric made available in Fabric 1.1 and 1.2 and what can be expected next.
Blockchains and Smart Contracts: Architecture Design and Model-Driven Develop...Ingo Weber
Software Architecture and Business Process Management research by the AAP team, Data61, CSIRO (Sydney Australia):
1. Architectural concerns of developing applications around Blockchain
2. Model-driven development for blockchain smart contracts
3. Business process execution on blockchain (details on one case of 2.)
Microservices architecture is an architectural style that structures an application as a collection of services that are. Highly maintainable and testable.
Credit goes to Christopher B Ferris @christo4ferris who put together this presentation which covers the latest developments of Hyperledger Fabric made available in Fabric 1.1 and 1.2 and what can be expected next.
Blockchains and Smart Contracts: Architecture Design and Model-Driven Develop...Ingo Weber
Software Architecture and Business Process Management research by the AAP team, Data61, CSIRO (Sydney Australia):
1. Architectural concerns of developing applications around Blockchain
2. Model-driven development for blockchain smart contracts
3. Business process execution on blockchain (details on one case of 2.)
Demystify blockchain development with hyperledger fabricBenjamin Fuentes
The World has been following blockchain technology last year with the raise of the public blockchains, the Bitcoin value overpassing Gold and now private blockchains for Business.
Why so many interest on permissioned blockchain ? Which industries are impacted ? What is new for 2017 ? You will have a short presentation on blockchain and a demo on the latest Hyperledger Fabric V1 around an original use case for airline industry
Blockchain Application Design and Development, and the Case of Programmable M...Ingo Weber
Slides from my CLOSER 2021 keynote ( https://www.insticc.org/node/TechnicalProgram/closer/2021/presentationDetails/1390 )
Blockchain has emerged as a decentralized platform for managing digital assets and executing 'smart contracts', i.e., user-defined code. While blockchain's suitability for a given use case should always be scrutinized, it does have the potential to disrupt many of the connection points between individuals, companies, and government entities. In this keynote talk, I will provide an overview of what architects and developers need to know in order to build blockchain-based applications, and how it relates to the cloud and software services. Among others, I will cover blockchain-as-a-service concepts, as well as architectural concerns and model-driven engineering for blockchain applications, the latter also in relation to collaborative business processes. To highlight some of the challenges, I will discuss insights from a project on "programmable money", i.e., blockchain-based money for conditional payments where the money itself checks whether it can be spent in a certain way at the point of payment. Finally, I will touch on insights into current adoption of blockchain.
Presentation from Grace Hopper Celebration 2016. Topic: Blockchain and Internet of Things (IoT) in the IBM Bluemix platform includes Demo. Speakers: Valerie Lampkin, Sumabala Nair and Carole Corley
Blockchain technology is increasingly being considered for applications in business contexts due to its key properties. It is also very much hyped for its potential to transform existing industries and business models. In Part 1, we will introduce the key properties of blockchain, its limitations, the field and the relevance for SAP and enterprises in general. In Part 2, we will focus on one of the prominent suites available today and provide an demonstration of the POC we’ve developed.
Bitcoin: A Peer-to-Peer Electronic Cash System
Satoshi Nakamoto
satoshin@gmx.com
www.bitcoin.org
Abstract.
A purely peer-to-peer version of electronic cash would allow online
payments to be sent directly from one party to another without going through a
financial institution. Digital signatures provide part of the solution, but the main
benefits are lost if a trusted third party is still required to prevent double-spending.
We propose a solution to the double-spending problem using a peer-to-peer network.
The network timestamps transactions by hashing them into an ongoing chain of
hash-based proof-of-work, forming a record that cannot be changed without redoing
the proof-of-work. The longest chain not only serves as proof of the sequence of
events witnessed, but proof that it came from the largest pool of CPU power. As
long as a majority of CPU power is controlled by nodes that are not cooperating to
attack the network, they'll generate the longest chain and outpace attackers. The
network itself requires minimal structure. Messages are broadcast on a best effort
basis, and nodes can leave and rejoin the network at will, accepting the longest
proof-of-work chain as proof of what happened while they were gone.
An introduction to the technology underlying blockchains. If you want to hear a voice over, you can find it at http://presentationtube.com/watch?v=tCZDIEezrpbhttp://presentationtube.com/watch?v=tCZDIEezrpb
Analysing Data from Blockchains - Keynote @ SOCCA 2020Ingo Weber
Keynote at the Symposium on Cryptocurrency Analysis (SOCCA 2020). Content:
In order to analyse how concrete blockchain systems as well as blockchain applications are used, data must be extracted from these systems. Due to various complexities inherent in blockchain, the question how to interpret such data is non-trivial. Such interpretation should often be shared among parties, e.g., if they collaborate via a blockchain. To this end, we devised an approach codify the interpretation of blockchain data, to extract data from blockchains accordingly, and to output it in suitable formats -- see https://arxiv.org/abs/2001.10281.
In addition, application developers and users of blockchain applications may want to estimate the cost of using or op- erating a blockchain application. In the keynote, I will also discuss our cost estimation method.
An introduction to blockchains, a session handled by Adri Jovin J J, Department of Information Technology, Sri Ramakrishna Institute of Technology. This gives a basic outline of the blockchains.
1. How the Royal Bank of Scotland is approaching research and innovation
2. What we have looked at in the distributed ledger space
3. Example: Building a digital wallet on the Hyperledger Fabric
4. What we've learnt about Hyperledger and other DLs
5. The challenges we face as a bank and as blockchain developers
6. Thoughts for the future
Introduction to Microservices Architecture - SECCOMP 2020Rodrigo Antonialli
This presentation gives an high-level overview of what is a Microservices Architecture, as a summary from well-known sources about the topic regarding it's characteristics, advantages and challenges, along with some enabling technologies.
[WSO2 API Day Dallas 2019] Extending Service Mesh with API ManagementWSO2
In this deck, we discuss how to augment service mesh functionality with API management capabilities, so you can create an end-to-end solution for your entire business functionality — from microservices to APIs, to end-user applications.
Demystify blockchain development with hyperledger fabricBenjamin Fuentes
The World has been following blockchain technology last year with the raise of the public blockchains, the Bitcoin value overpassing Gold and now private blockchains for Business.
Why so many interest on permissioned blockchain ? Which industries are impacted ? What is new for 2017 ? You will have a short presentation on blockchain and a demo on the latest Hyperledger Fabric V1 around an original use case for airline industry
Blockchain Application Design and Development, and the Case of Programmable M...Ingo Weber
Slides from my CLOSER 2021 keynote ( https://www.insticc.org/node/TechnicalProgram/closer/2021/presentationDetails/1390 )
Blockchain has emerged as a decentralized platform for managing digital assets and executing 'smart contracts', i.e., user-defined code. While blockchain's suitability for a given use case should always be scrutinized, it does have the potential to disrupt many of the connection points between individuals, companies, and government entities. In this keynote talk, I will provide an overview of what architects and developers need to know in order to build blockchain-based applications, and how it relates to the cloud and software services. Among others, I will cover blockchain-as-a-service concepts, as well as architectural concerns and model-driven engineering for blockchain applications, the latter also in relation to collaborative business processes. To highlight some of the challenges, I will discuss insights from a project on "programmable money", i.e., blockchain-based money for conditional payments where the money itself checks whether it can be spent in a certain way at the point of payment. Finally, I will touch on insights into current adoption of blockchain.
Presentation from Grace Hopper Celebration 2016. Topic: Blockchain and Internet of Things (IoT) in the IBM Bluemix platform includes Demo. Speakers: Valerie Lampkin, Sumabala Nair and Carole Corley
Blockchain technology is increasingly being considered for applications in business contexts due to its key properties. It is also very much hyped for its potential to transform existing industries and business models. In Part 1, we will introduce the key properties of blockchain, its limitations, the field and the relevance for SAP and enterprises in general. In Part 2, we will focus on one of the prominent suites available today and provide an demonstration of the POC we’ve developed.
Bitcoin: A Peer-to-Peer Electronic Cash System
Satoshi Nakamoto
satoshin@gmx.com
www.bitcoin.org
Abstract.
A purely peer-to-peer version of electronic cash would allow online
payments to be sent directly from one party to another without going through a
financial institution. Digital signatures provide part of the solution, but the main
benefits are lost if a trusted third party is still required to prevent double-spending.
We propose a solution to the double-spending problem using a peer-to-peer network.
The network timestamps transactions by hashing them into an ongoing chain of
hash-based proof-of-work, forming a record that cannot be changed without redoing
the proof-of-work. The longest chain not only serves as proof of the sequence of
events witnessed, but proof that it came from the largest pool of CPU power. As
long as a majority of CPU power is controlled by nodes that are not cooperating to
attack the network, they'll generate the longest chain and outpace attackers. The
network itself requires minimal structure. Messages are broadcast on a best effort
basis, and nodes can leave and rejoin the network at will, accepting the longest
proof-of-work chain as proof of what happened while they were gone.
An introduction to the technology underlying blockchains. If you want to hear a voice over, you can find it at http://presentationtube.com/watch?v=tCZDIEezrpbhttp://presentationtube.com/watch?v=tCZDIEezrpb
Analysing Data from Blockchains - Keynote @ SOCCA 2020Ingo Weber
Keynote at the Symposium on Cryptocurrency Analysis (SOCCA 2020). Content:
In order to analyse how concrete blockchain systems as well as blockchain applications are used, data must be extracted from these systems. Due to various complexities inherent in blockchain, the question how to interpret such data is non-trivial. Such interpretation should often be shared among parties, e.g., if they collaborate via a blockchain. To this end, we devised an approach codify the interpretation of blockchain data, to extract data from blockchains accordingly, and to output it in suitable formats -- see https://arxiv.org/abs/2001.10281.
In addition, application developers and users of blockchain applications may want to estimate the cost of using or op- erating a blockchain application. In the keynote, I will also discuss our cost estimation method.
An introduction to blockchains, a session handled by Adri Jovin J J, Department of Information Technology, Sri Ramakrishna Institute of Technology. This gives a basic outline of the blockchains.
1. How the Royal Bank of Scotland is approaching research and innovation
2. What we have looked at in the distributed ledger space
3. Example: Building a digital wallet on the Hyperledger Fabric
4. What we've learnt about Hyperledger and other DLs
5. The challenges we face as a bank and as blockchain developers
6. Thoughts for the future
Introduction to Microservices Architecture - SECCOMP 2020Rodrigo Antonialli
This presentation gives an high-level overview of what is a Microservices Architecture, as a summary from well-known sources about the topic regarding it's characteristics, advantages and challenges, along with some enabling technologies.
[WSO2 API Day Dallas 2019] Extending Service Mesh with API ManagementWSO2
In this deck, we discuss how to augment service mesh functionality with API management capabilities, so you can create an end-to-end solution for your entire business functionality — from microservices to APIs, to end-user applications.
An eventful tour from enterprise integration to serverless and functionsChristian Posta
In this talk, Marius (@mariusbogoevici) and I (@christianposta) discuss the value of event-driven architectures (both business and technical merits) and how the landscape of integration, streaming, and messaging and now functions/lambdas have evolved to implement EDA while balancing agility, utilization, and simplicity.
Docker microservices and the service meshDocker, Inc.
The nature of containerized, cloud-native applications is rapidly advancing with a fundamentally different architecture that will rely on service meshes with smarter proxies, traffic management, and enhanced observability for cooperating microservices, serverless functions, and complex workflows. In this session we will highlight the features that characterize this architectural transformation in the Docker cloud-native ecosystem.
NewsCred Dhaka hosted an interactive session on MircroServices. The main focus of the event was to provide a platform for people to share their experiences, understand the architecture and hear about the challenges and benefits of continuous deployment.
Presenters: Asif Rahman (CTO), Brian Schmitz (Director of Engineering), Rana Khandakar (Lead Software Engineer), Ashrafuzzaman Jitu (Engineering Manager), and Zahiduzzaman Setu (Senior Software Engineer), as they share their experiences with MicroServices and in the process find out if it is right for you.
[APIdays Paris 2019] API Management in Service Mesh Using Istio and WSO2 API ...WSO2
Stefano discusses how to augment service mesh functionality with API management capabilities, so you can create an end-to-end solution for your entire business functionality — from microservices, to APIs, to end-user applications.
With microservices gone mainstream a few years ago, many organizations have now adopted them; even though all are paying the price in terms of training, solution complexity and operational costs, few are reaping the promised benefits.
Lower velocity, quality and performance issues, along with an overall lack of visibility are what we hear about most often.
In this session, working from our experience as advisors to software development teams, we’ll walk you through some of the symptoms you might experience, their possible causes and some potential solutions.
JS Fest 2019/Autumn. Anton Cherednikov. Choreographic or orchestral architect...JSFestUA
When we developing a loosely coupled and reusable application, often arises the question: how to arrange to communicate between services or applications? To a large extent, it depends on the nature of the request and the granularity of your applications or services.
We will discuss the two classic microservice integration patterns: service choreography and orchestration.
What is the difference between these two modes of communication? Which one we should use? How to ensure data consistency? How to implement disturbed transactions?
We will discuss these issues, consider an example of implementing orchestration on nodejs, and of course we will not forget about logging, monitoring and alerting.
SkopjeTechMeetup is an initiative by Tricode for supporting and strengthening the Macedonian IT community. The meetups have the goal of establishing a networking platform for the IT crowd where they can share their know-how, best practices, as well as mutual inspiration.
The 6th STM installment took place at Piazza Liberta, Skopje last Thursday, the 29th of September. This meetup hosted 3 seasoned speakers, each accomplished in their own way.
Here's the presentation of Lazo Apostolovski.
The Microservices Architecture pattern is getting a lot of attention lately, even at the beginning of its adoption lifecycle. It has significant benefits when it comes to enabling agile development and delivering complex enterprise applications. Adopting Microservices can be a tricky and dangerous process. Making bad decisions early can lead to serious complications, expences and maybe even failure.
Commit Conf 2018 - Hotelbeds' journey to a microservice cloud-based architectureJordi Puigsegur Figueras
Hotelbeds is one of the leading bedbanks in the world, a global distributor of accommodation based in Palma de Mallorca, with an offering of more than 170.000 hotels and up to 2.500 Million request per day.
In the last two years Hotelbeds Group has undergone a profound technical transformation from an old monolythic on premise architecture to a microservices cloud-native architecture. Nowadays a big chunk of our monolyth has been transformed into java microservices deployed on cloud and we are already processing all our bookings on cloud. We have also moved to the cloud our availability engine, distributed to several locations accross the globe to be closer to the clients and reduce latencies.
This talk is about the path we followed on this fascinating journey, the decisions taken and lessons learned, as well as some insight on our future challenges.
Video available at https://www.youtube.com/watch?v=5H0X65QVF34
* All images used in this presentation are for illustration purposes only. They have been created specifically or obtained from publicly accessible sources. If any image is incorrectly used please contact me and it will be immediately removed and not used in the future.
Ato Z of Microservices Architecture by SystangoSystango
In the world of software engineering, Microservices has emerged as a big buzzword. The simplest way to understand a microservice is that it’s an application that does only one thing, and does it really well. We have put together a presentation for you explaining the same in detail. Know more at - https://bit.ly/2Il5WRv
[WSO2 API Day Toronto 2019] Extending Service Mesh with API ManagementWSO2
In this deck, we discuss how to augment service mesh functionality with API management capabilities, so you can create an end-to-end solution for your entire business functionality — from microservices to APIs, to end-user applications.
These are my summarized notes from all the microservices session I attended at QCon 2015. These sessions had tons of learning around how to scale microservices and avoid common pitfalls
Similar to Ledingkart Meetup #1: Monolithic to microservices in action (20)
How to Get CNIC Information System with Paksim Ga.pptxdanishmna97
Pakdata Cf is a groundbreaking system designed to streamline and facilitate access to CNIC information. This innovative platform leverages advanced technology to provide users with efficient and secure access to their CNIC details.
A tale of scale & speed: How the US Navy is enabling software delivery from l...sonjaschweigert1
Rapid and secure feature delivery is a goal across every application team and every branch of the DoD. The Navy’s DevSecOps platform, Party Barge, has achieved:
- Reduction in onboarding time from 5 weeks to 1 day
- Improved developer experience and productivity through actionable findings and reduction of false positives
- Maintenance of superior security standards and inherent policy enforcement with Authorization to Operate (ATO)
Development teams can ship efficiently and ensure applications are cyber ready for Navy Authorizing Officials (AOs). In this webinar, Sigma Defense and Anchore will give attendees a look behind the scenes and demo secure pipeline automation and security artifacts that speed up application ATO and time to production.
We will cover:
- How to remove silos in DevSecOps
- How to build efficient development pipeline roles and component templates
- How to deliver security artifacts that matter for ATO’s (SBOMs, vulnerability reports, and policy evidence)
- How to streamline operations with automated policy checks on container images
The Art of the Pitch: WordPress Relationships and SalesLaura Byrne
Clients don’t know what they don’t know. What web solutions are right for them? How does WordPress come into the picture? How do you make sure you understand scope and timeline? What do you do if sometime changes?
All these questions and more will be explored as we talk about matching clients’ needs with what your agency offers without pulling teeth or pulling your hair out. Practical tips, and strategies for successful relationship building that leads to closing the deal.
zkStudyClub - Reef: Fast Succinct Non-Interactive Zero-Knowledge Regex ProofsAlex Pruden
This paper presents Reef, a system for generating publicly verifiable succinct non-interactive zero-knowledge proofs that a committed document matches or does not match a regular expression. We describe applications such as proving the strength of passwords, the provenance of email despite redactions, the validity of oblivious DNS queries, and the existence of mutations in DNA. Reef supports the Perl Compatible Regular Expression syntax, including wildcards, alternation, ranges, capture groups, Kleene star, negations, and lookarounds. Reef introduces a new type of automata, Skipping Alternating Finite Automata (SAFA), that skips irrelevant parts of a document when producing proofs without undermining soundness, and instantiates SAFA with a lookup argument. Our experimental evaluation confirms that Reef can generate proofs for documents with 32M characters; the proofs are small and cheap to verify (under a second).
Paper: https://eprint.iacr.org/2023/1886
Unlocking Productivity: Leveraging the Potential of Copilot in Microsoft 365, a presentation by Christoforos Vlachos, Senior Solutions Manager – Modern Workplace, Uni Systems
Encryption in Microsoft 365 - ExpertsLive Netherlands 2024Albert Hoitingh
In this session I delve into the encryption technology used in Microsoft 365 and Microsoft Purview. Including the concepts of Customer Key and Double Key Encryption.
Communications Mining Series - Zero to Hero - Session 1DianaGray10
This session provides introduction to UiPath Communication Mining, importance and platform overview. You will acquire a good understand of the phases in Communication Mining as we go over the platform with you. Topics covered:
• Communication Mining Overview
• Why is it important?
• How can it help today’s business and the benefits
• Phases in Communication Mining
• Demo on Platform overview
• Q/A
Securing your Kubernetes cluster_ a step-by-step guide to success !KatiaHIMEUR1
Today, after several years of existence, an extremely active community and an ultra-dynamic ecosystem, Kubernetes has established itself as the de facto standard in container orchestration. Thanks to a wide range of managed services, it has never been so easy to set up a ready-to-use Kubernetes cluster.
However, this ease of use means that the subject of security in Kubernetes is often left for later, or even neglected. This exposes companies to significant risks.
In this talk, I'll show you step-by-step how to secure your Kubernetes cluster for greater peace of mind and reliability.
Alt. GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using ...James Anderson
Effective Application Security in Software Delivery lifecycle using Deployment Firewall and DBOM
The modern software delivery process (or the CI/CD process) includes many tools, distributed teams, open-source code, and cloud platforms. Constant focus on speed to release software to market, along with the traditional slow and manual security checks has caused gaps in continuous security as an important piece in the software supply chain. Today organizations feel more susceptible to external and internal cyber threats due to the vast attack surface in their applications supply chain and the lack of end-to-end governance and risk management.
The software team must secure its software delivery process to avoid vulnerability and security breaches. This needs to be achieved with existing tool chains and without extensive rework of the delivery processes. This talk will present strategies and techniques for providing visibility into the true risk of the existing vulnerabilities, preventing the introduction of security issues in the software, resolving vulnerabilities in production environments quickly, and capturing the deployment bill of materials (DBOM).
Speakers:
Bob Boule
Robert Boule is a technology enthusiast with PASSION for technology and making things work along with a knack for helping others understand how things work. He comes with around 20 years of solution engineering experience in application security, software continuous delivery, and SaaS platforms. He is known for his dynamic presentations in CI/CD and application security integrated in software delivery lifecycle.
Gopinath Rebala
Gopinath Rebala is the CTO of OpsMx, where he has overall responsibility for the machine learning and data processing architectures for Secure Software Delivery. Gopi also has a strong connection with our customers, leading design and architecture for strategic implementations. Gopi is a frequent speaker and well-known leader in continuous delivery and integrating security into software delivery.
Essentials of Automations: The Art of Triggers and Actions in FMESafe Software
In this second installment of our Essentials of Automations webinar series, we’ll explore the landscape of triggers and actions, guiding you through the nuances of authoring and adapting workspaces for seamless automations. Gain an understanding of the full spectrum of triggers and actions available in FME, empowering you to enhance your workspaces for efficient automation.
We’ll kick things off by showcasing the most commonly used event-based triggers, introducing you to various automation workflows like manual triggers, schedules, directory watchers, and more. Plus, see how these elements play out in real scenarios.
Whether you’re tweaking your current setup or building from the ground up, this session will arm you with the tools and insights needed to transform your FME usage into a powerhouse of productivity. Join us to discover effective strategies that simplify complex processes, enhancing your productivity and transforming your data management practices with FME. Let’s turn complexity into clarity and make your workspaces work wonders!
LF Energy Webinar: Electrical Grid Modelling and Simulation Through PowSyBl -...DanBrown980551
Do you want to learn how to model and simulate an electrical network from scratch in under an hour?
Then welcome to this PowSyBl workshop, hosted by Rte, the French Transmission System Operator (TSO)!
During the webinar, you will discover the PowSyBl ecosystem as well as handle and study an electrical network through an interactive Python notebook.
PowSyBl is an open source project hosted by LF Energy, which offers a comprehensive set of features for electrical grid modelling and simulation. Among other advanced features, PowSyBl provides:
- A fully editable and extendable library for grid component modelling;
- Visualization tools to display your network;
- Grid simulation tools, such as power flows, security analyses (with or without remedial actions) and sensitivity analyses;
The framework is mostly written in Java, with a Python binding so that Python developers can access PowSyBl functionalities as well.
What you will learn during the webinar:
- For beginners: discover PowSyBl's functionalities through a quick general presentation and the notebook, without needing any expert coding skills;
- For advanced developers: master the skills to efficiently apply PowSyBl functionalities to your real-world scenarios.
Threats to mobile devices are more prevalent and increasing in scope and complexity. Users of mobile devices desire to take full advantage of the features
available on those devices, but many of the features provide convenience and capability but sacrifice security. This best practices guide outlines steps the users can take to better protect personal devices and information.
Dr. Sean Tan, Head of Data Science, Changi Airport Group
Discover how Changi Airport Group (CAG) leverages graph technologies and generative AI to revolutionize their search capabilities. This session delves into the unique search needs of CAG’s diverse passengers and customers, showcasing how graph data structures enhance the accuracy and relevance of AI-generated search results, mitigating the risk of “hallucinations” and improving the overall customer journey.
GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using Deplo...James Anderson
Effective Application Security in Software Delivery lifecycle using Deployment Firewall and DBOM
The modern software delivery process (or the CI/CD process) includes many tools, distributed teams, open-source code, and cloud platforms. Constant focus on speed to release software to market, along with the traditional slow and manual security checks has caused gaps in continuous security as an important piece in the software supply chain. Today organizations feel more susceptible to external and internal cyber threats due to the vast attack surface in their applications supply chain and the lack of end-to-end governance and risk management.
The software team must secure its software delivery process to avoid vulnerability and security breaches. This needs to be achieved with existing tool chains and without extensive rework of the delivery processes. This talk will present strategies and techniques for providing visibility into the true risk of the existing vulnerabilities, preventing the introduction of security issues in the software, resolving vulnerabilities in production environments quickly, and capturing the deployment bill of materials (DBOM).
Speakers:
Bob Boule
Robert Boule is a technology enthusiast with PASSION for technology and making things work along with a knack for helping others understand how things work. He comes with around 20 years of solution engineering experience in application security, software continuous delivery, and SaaS platforms. He is known for his dynamic presentations in CI/CD and application security integrated in software delivery lifecycle.
Gopinath Rebala
Gopinath Rebala is the CTO of OpsMx, where he has overall responsibility for the machine learning and data processing architectures for Secure Software Delivery. Gopi also has a strong connection with our customers, leading design and architecture for strategic implementations. Gopi is a frequent speaker and well-known leader in continuous delivery and integrating security into software delivery.
GraphRAG is All You need? LLM & Knowledge GraphGuy Korland
Guy Korland, CEO and Co-founder of FalkorDB, will review two articles on the integration of language models with knowledge graphs.
1. Unifying Large Language Models and Knowledge Graphs: A Roadmap.
https://arxiv.org/abs/2306.08302
2. Microsoft Research's GraphRAG paper and a review paper on various uses of knowledge graphs:
https://www.microsoft.com/en-us/research/blog/graphrag-unlocking-llm-discovery-on-narrative-private-data/
2. What are Microservices?
● While there is no formal definition of microservices, there are certain
characteristics that help us identify the style.
● Microservices is a software development technique
● A variant of the service-oriented architecture (SOA) architectural style that
structures an application as a collection of loosely coupled services
● In a microservices architecture, services are fine-grained, deployable
● The communication protocols are lightweight (HTTP/REST with JSON)
● Each service runs a unique process
6. History of microservices
As Martin Fowler (UML, Design expert) points out, Netflix, eBay, Amazon,
Twitter, PayPal, Soundcloud and many other large-scale websites and
applications have all evolved from monolithic to microservices architecture.
Amazon: “In 2001 the Amazon.com retail website was a large architectural
monolith.” It was architected in multiple tiers, and those tiers had many
components in them. However, they were all very tightly coupled together,
where they behaved like one big monolith. Now they are completely using
microservices architecture and for starters, they use 120 services to generate
one product page in their ecom website!
8. Contd...
Netflix: Netflix has a widespread architecture that had evolved from monolithic
to SOA. It received more than one billion calls every day, from more than 800
different types of devices, to its streaming-video API. Each API call then prompts
around five additional calls to the backend service. They offer 500+ services in
total! Netflix took ten years to fully transition from monolith to microservices. In
2008, a database corruption due to a semicolon cost them huge headaches.
AirBnB: The home-sharing service operates in 65,000 cities located in 191
countries. 2017 saw their four millionth listing. Last summer, 45 million guests
stayed in an Airbnb listing, making a total of 200 million guest stays since 2008.
In order to make this possible, they deploy 3500 microservices per week, with a
total of 75 thousand production deploys per year.
14. Why microservices?
● High Availability (Decoupled)
● Scalability
● Reliability
● Service Management
● Clean Code
● More Testable
● Clear Ownership
● Release Management
● Data Centric Development.
● Ability to scale/integrate individual services written in different
frameworks/languages.
● Encourages Agile Methodologies
15. Initial Lendingkart Platform (monolithic)…
● Started with 2 Backend, 2 UI Developers, 1 QA
● As new developers joined the team, it was difficult to understand and
maintain the monolithic
● Initial monolithic catered to customer & ops with minimal features.
● Lot of experiments carried out on Customer to improve self-serve and on
Ops to reduce Turn around Time
● Stability was a major worry as we had to deploy features and bugs on a
daily basis
● Vertical scaling of our database server on a regular basis
16. Contd...
● Lending involves 3 major pieces: Origination, Credit analysis, Loan
management
● Our initial monolithic involved Customer, Origination and Credit analysis
modules (Spring MVC) structured on the lines of SOA
● For Loan management we used a third party service to avoid re-inventing
the wheel
17. Contd...
● First microservice was in the form of lead service (Dropwizard), followed by
notification service
● Subsequent break of our monolithic to microservice happened in the form
of customer, credit analysis, audit, reports, scheduler, webhooks, etc
● The greatest advantage was that since our requirements were changing
fast, we could make the changes quickly on the microservice and make it
more mature over time
19. How to decide if you need microservices?
● If distributed systems are the way to go, do I use microservices or
modularized applications?
● Do you have need to grow rapidly ?
● Do you have certain part of applications which needs special handling in
terms of load or feature rollout or hardware specifications ?
● Do you have too many merge conflicts and repo issues ? Is it taking
significant time and bandwidth of your developers?
● Is my business suitable for more monolithic? Do I have more batching and
scheduling to do ?
20. How to decide if you need microservices?
● Do I have enough resources to build, scale and deploy dozens of services ?
● Is there an integration expert/architect that can lead the project and follow
through?
● What if one of the service endpoints fail? How do I recover? How do I
monitor all these effectively?
● How do I manage security settings and authentication across all the
services?
● Would you need different technology stack for different applications or
different features ?
21. How we moved to microservices architecture
Monolithic to Microservice architecture
UI
MySQL DB
Business logic
Data Access Layer
MySQL DB MongoDB MySQL
Microservice
Customer
Microservice
Ops
Microservice
Notification
Microservice
Credit Analysis
Nginx
UI UI UI UI
22. How we broke our monolithic (Lendingkart Platform as an Example)
● Leads
○ Developed using Dropwizard as it had some great out of the box
features compared to Spring MVC
○ Worked well except since we didn’t have a system-wide monitoring &
alerting it was tough to manage the same
○ MongoDB as lead data from partners is sparse and reduces load on
MySQL DB
23. Contd...
● Audit
○ Use MongoDB to store audit data through CDC (mysql binlog &
mongodb oplog reader)
○ Data available through REST APIs
● Customer
○ Customer service was the first to come out of the main monolithic as it
had fewer APIs compared to Ops
○ Made it easier to identify issues
○ Increased fault tolerance and performance
24. Contd...
● Fixed Scheduler (Spring)
○ All scheduler tasks where moved to a separate service
○ Not a complete solution as it wasn’t distributed
● Scheduler API based (Quartz)
○ Ease of configuring scheduled jobs dynamically through REST APIs
○ Distributed
● Webhooks
○ APIs for third-party like Mixpanel, Survey Monkey, etc
○ Improved fault-tolerance as third-party API failure means difficulty in
resolving the same
25. Contd...
● Notifications
○ SMS, Email
○ Capability to throttle email/sms between multiple third-party providers
● Search Service
○ Elasticsearch for faster querying
○ Reduces load on relational database
26. Contd...
● Authentication & Authorization
○ Central authority. Easier to track issues
○ Important service as security is central to any financial service
○ Using Shiro framework
○ Authorization is now better handled as each department had a
different requirement. Now it is under one umbrella and we are solving
it in a transparent manner with our Ops team.
○ Providing hierarchical roles with access to specific features along with
fine-grained permissions (RBAC)
27. Challenges in moving to microservices
● Distributed tracing and debugging
● Architect for failures
● Dependency and version management
● Reporting and analytics
● Operations overhead
28. Distributed tracing and debugging
● Difficulty in debugging problems when a feature spans across multiple
microservices
● Challenges in identifying latency issues
Solutions:
● Centralized logging mechanism i.e ELK
● Inject requestId at the source
● Zipkin, spring-cloud-sleuth
29. Architect for failures
● Avoid failures by implementing change management strategies and
automatic rollout
● Exception handling
● Timeouts
● Retries with limited attempts and exponential backoff
● Circuit breaker i.e Netflix Hystrix, Fallbacks
● Rate Limiting and client whitelisting
● Test for failures i.e Chaos-Monkey
30. Dependency & version management
● Avoid cyclic dependencies
● Ensure dependencies form DAG (Directed Acyclic Graph)
31. Dependency & version management
● Maintain backward compatibility
● Api versioning
32. Reporting and analytics
● Data spread across a number of isolated heterogeneous data stores
● Consolidation and aggregation becomes challenging
Solution:
● Pull data from all microservices and store in centralized data store i.e
Redshift.
33. Operations overhead
● Keep releases and deployment manageable
● Service monitoring and automation is required
● Containerization and running multiple services at single instance
● Devops is a must
37. Best Practices
● As Martin Fowler reportedly said, “the only thing a Big Bang rewrite
guarantees is a Big Bang!”
38. Best Practices
● Mark edge services like caching, authentication/authorizations, logging etc.,
and move it to microservices first.
● Decomposition: Business Use Case, Domain etc.,
● CI/CD
● Monitoring & Alerting System.
46. Best Practices contd...
● Don't Burn That Old Bridge Before Getting a New One in Place.
● Move any new features/product to microservices.
● Clear cut interface/contract to communicate between services.
● Service should be extracted in such a way that it can be easily tested and
shipped to production.
● Service should follow single responsibility like OOPS object.
● Club closely related services together to avoid too many network calls.
47. Frameworks used in microservices
● Spring Boot with netflix oss, Dropwizard, Lagom, Play
● GraphQL: A great complement to REST APIs allowing for customized
consumption of services
● CI/CD: Gitlab, Docker, Kubernetes
● Monitoring & Alerting: Telegraf, InfluxDB, Grafana, Kapacitor.
● Distributed Messaging Queue: Kafka
Primary: Gautam
SOA is an architectural pattern in which application components provide services to other components. However, in SOA those components can belong to the same application.
Primary: Gautam
Primary: Gautam
Amazon: In 2000, Amazon had the unenviable task of reconciling the in-process changes from hundreds of developers, resolving all the conflicts between them, merging them into a single version, and producing a master version that waits on the queue to be moved into production.
Primary: Gautam
Primary: Gautam
Primary: Gautam
Primary: Gautam
Primary: Gautam
Primary: Gautam
Primary: Gautam
Primary: Mihir
High Availability: It’s easy to deploy individual service and have lesser downtime and also add more nodes to it then one big monolith! Also, in case of failure one service would not impact other.
Reliability: a microservice fault affects that microservice alone and its consumers, whereas in the monolithic model a service fault may bring down the entire monolith.
Primary: Gautam
Lendingkart started as a monolithic. A monolithic application is built as a single unit. We had taken care of the traditional layers: database, a client-side user interface (HTML & Angular framework), and a server-side application. This server-side application was used to handle HTTP requests, execute some domain specific logic, retrieve and update data from the database, and populate the HTML views to be sent to the browser.
It wa a monolith – a single logical executable.
Primary: Gautam
Primary: Gautam
Primary: Gautam
Primary: Mihir
Appliance solutions
Non-multi-tenant
B2C: Micro-services
Large Financial institutions are reluctant to move to micro-services due to compliance, security, processes hindering them
Team larger than say 20, leads to merge conflict and causes more bugs as teams can’t work independently in a monolithic architecture
Modular approach works fine but have to be aware of dependencies all the time and cannot use different versions of modules in same application but in microservices you can do. Failure in one module will bring down entire application but in microservices it will only impact one service and it’s consumers at worst!
Primary: Mihir
Primary: Gautam
Primary: Gautam
Example (The micro-services in Lendingkart): Talk about Why, How, When, Challenges, Performance improvements, Productivity, etc
Primary: Gautam
Primary: Gautam
Primary: Gautam
Primary: Gautam
Primary: Dinesh
Primary: Dinesh
Centralized monitoring and logging tools are needed to debug issues. Every service should data to a centralized data store i.e ELK.
In order to debug root cause, we should be adding a request id and serviceid in logs to identify which service was faulty.
primary:Dinesh
you deploy new code, or you change some configuration, you should apply these changes to a subset of your instances gradually, monitor them and even automatically revert the deployment if you see that it has a negative effect on your key metrics. We can adopt strategies like rolling or blue green deployment.
Primary: Dinesh
We need to make sure service calls should not form a cycle else it might result into distributed stack overflow.
Primary: Dinesh
Primary: Dinesh
Primary: Dinesh
As we add all this infrastrure like monitoring, alerting and orchestration tools, we need a dedicated devops team to manage infra.
In its simplest form, Aggregator would be a simple web page that invokes multiple services to achieve the functionality required by the application. Since each service (Service A, Service B, and Service C) is exposed using a lightweight REST mechanism, the web page can retrieve the data and process/display it accordingly
Proxy microservice design pattern is a variation of Aggregator. In this case, no aggregation needs to happen on the client but a different microservice may be invoked based upon the business need or rule. EXAMPLE:BOOKMYSHOW
Chained microservice design pattern produce a single consolidated response to the request. In this case, the request from the client is received by Service A, which is then communicating with Service B, which in turn may be communicating with Service C. All the services are likely using a synchronous HTTP request/response messaging.
Important aspect to understand here is to not make the chain too long. EXAMPLE: payment gateway.
Branch microservice design pattern extends Aggregator design pattern and allows simultaneous response processing from two, likely mutually exclusive, chains of microservices. This pattern can also be used to call different chains, or a single chain, based upon the business needs
Service A, either a web page or a composite microservice, can invoke two different chains concurrently in which case this will resemble the Aggregator design pattern. Alternatively, Service A can invoke only one chain based upon the request received from the client
It’s anti-pattern and should be used sparingly only during transition phase wherein you need to provide certain data for certain time. For ex, you are in transition phase and microservice is not fully autonomous but business need, pushes you to store or share data with some other service.
In this design pattern, Service A may call Service C synchronously which is then communicating with Service B and D asynchronously using a shared message queue
Event Firehose: Due to IoT, social networks and real-time stream processing, event firehose use cases are becoming common