SlideShare a Scribd company logo
Confoo 08-03-2018
Moving from legacy to
Event Driven with Kafka
Jeroen van der Gulik
About me
❖ Software Architect/ Consultant
❖ Co-Founder Isset
(https://isset.nl)
❖ Señor Developer
❖ Organizer DDDNL
❖ Builder of Artificial Stupidity
❖ @n0x13
“The road to hell is paved
with good intentions”
Flexibility
Security
Dependancy
Complexity
Costs
The 5 sins of Legacy
❖ Flexibility
❖ Security
❖ Dependancy
❖ Complexity
❖ Costs
“Nothing is more permanent than a
temporary solution”
The Project
❖ Proprietary software
❖ Hacked on “solutions”
❖ Old hardware
❖ Business logic in SP: No version control
❖ Original Team members gone
❖ Unwilling supplier
The Assignment
❖ Rebuild the Web Portal from scratch
❖ Decouple from the main system
❖ Make the data queryable without causing extra load
The Constraints
❖ Cannot change proprietary software easily
❖ Load excessively high: no additional load
❖ Real Time querying not an option
❖ Slow release cycles (quarterly)
❖ Time constraint
❖ Do not jeopardise production
Rebuild
Refactor
Upgrade Hardware
Clone Database(s)
Strangler Pattern
Event-driven architecture (EDA), is a software
architecture pattern promoting the production,
detection, consumption of, and reaction to events.
Lessons learned
❖ Events give you a powerful way to decouple
❖ Events are very good contracts
❖ Event driven is a flexible architecture
❖ Making team think in terms of Events can be challenging
❖ Kafka is a very powerful tool but needs research
❖ Mistake are great learning moments
❖ Eventual Consistency is still a hard problem

More Related Content

Similar to 2018 03-08 moving from legacy to event driven with kafka - confoo

Deploying using deployer
Deploying using deployerDeploying using deployer
Deploying using deployer
Jeroen van der Gulik
 
iOS Architecture
iOS ArchitectureiOS Architecture
iOS ArchitectureJacky Lian
 
Programming for the Internet of Things
Programming for the Internet of ThingsProgramming for the Internet of Things
Programming for the Internet of Things
Kinoma
 
50 migrations in 24 hours
50 migrations in 24 hours50 migrations in 24 hours
50 migrations in 24 hours
Jason Dea
 
globus.pptx
globus.pptxglobus.pptx
globus.pptx
Sam Van Loon
 
Engineer Engineering Software
Engineer Engineering SoftwareEngineer Engineering Software
Engineer Engineering Software
Yung-Yu Chen
 
2022-Devnexus-StatefulMicroservices.pptx.pdf
2022-Devnexus-StatefulMicroservices.pptx.pdf2022-Devnexus-StatefulMicroservices.pptx.pdf
2022-Devnexus-StatefulMicroservices.pptx.pdf
Grace Jansen
 
OSDC 2016 - Continous Integration in Data Centers - Further 3 Years later by ...
OSDC 2016 - Continous Integration in Data Centers - Further 3 Years later by ...OSDC 2016 - Continous Integration in Data Centers - Further 3 Years later by ...
OSDC 2016 - Continous Integration in Data Centers - Further 3 Years later by ...
NETWAYS
 
Cloud Computing, an online storage house
Cloud Computing, an online storage houseCloud Computing, an online storage house
Cloud Computing, an online storage house
Manoj Khetan
 
Composing and Scaling Data Platforms-2015
Composing and Scaling Data Platforms-2015Composing and Scaling Data Platforms-2015
Composing and Scaling Data Platforms-2015Rahul Kumar
 
Composing and scaling data platforms
Composing and scaling data platformsComposing and scaling data platforms
Composing and scaling data platforms
Sigmoid
 
How MongoDB is Being Used in China - Case Studies
How MongoDB is Being Used in China - Case StudiesHow MongoDB is Being Used in China - Case Studies
How MongoDB is Being Used in China - Case Studies
MongoDB
 
Shillum "Building for the Future: Interoperability"
Shillum "Building for the Future: Interoperability"Shillum "Building for the Future: Interoperability"
Shillum "Building for the Future: Interoperability"
National Information Standards Organization (NISO)
 
Enabling Your Data Science Team with Modern Data Engineering
Enabling Your Data Science Team with Modern Data EngineeringEnabling Your Data Science Team with Modern Data Engineering
Enabling Your Data Science Team with Modern Data Engineering
James Densmore
 
Speed up your regression and reduce cost load with Selenoid + K8s + ReportPortal
Speed up your regression and reduce cost load with Selenoid + K8s + ReportPortalSpeed up your regression and reduce cost load with Selenoid + K8s + ReportPortal
Speed up your regression and reduce cost load with Selenoid + K8s + ReportPortal
Danylo Kuvshynov
 
Selkie Overview
Selkie OverviewSelkie Overview
Selkie Overview
rogerkirkland
 
How Will Going Virtual Impact Your Search Performance?
How Will Going Virtual Impact Your Search Performance?How Will Going Virtual Impact Your Search Performance?
How Will Going Virtual Impact Your Search Performance?IdeaEng
 
2016 NDC - 모바일 게임 서버 엔진 개발 후기
2016 NDC - 모바일 게임 서버 엔진 개발 후기2016 NDC - 모바일 게임 서버 엔진 개발 후기
2016 NDC - 모바일 게임 서버 엔진 개발 후기
iFunFactory Inc.
 
Spotlight on the petroleum and energy vertical
Spotlight on the petroleum and energy vertical Spotlight on the petroleum and energy vertical
Spotlight on the petroleum and energy vertical
FileCatalyst
 

Similar to 2018 03-08 moving from legacy to event driven with kafka - confoo (20)

Deploying using deployer
Deploying using deployerDeploying using deployer
Deploying using deployer
 
iOS Architecture
iOS ArchitectureiOS Architecture
iOS Architecture
 
Programming for the Internet of Things
Programming for the Internet of ThingsProgramming for the Internet of Things
Programming for the Internet of Things
 
50 migrations in 24 hours
50 migrations in 24 hours50 migrations in 24 hours
50 migrations in 24 hours
 
globus.pptx
globus.pptxglobus.pptx
globus.pptx
 
Engineer Engineering Software
Engineer Engineering SoftwareEngineer Engineering Software
Engineer Engineering Software
 
2022-Devnexus-StatefulMicroservices.pptx.pdf
2022-Devnexus-StatefulMicroservices.pptx.pdf2022-Devnexus-StatefulMicroservices.pptx.pdf
2022-Devnexus-StatefulMicroservices.pptx.pdf
 
OSDC 2016 - Continous Integration in Data Centers - Further 3 Years later by ...
OSDC 2016 - Continous Integration in Data Centers - Further 3 Years later by ...OSDC 2016 - Continous Integration in Data Centers - Further 3 Years later by ...
OSDC 2016 - Continous Integration in Data Centers - Further 3 Years later by ...
 
Cloud Computing, an online storage house
Cloud Computing, an online storage houseCloud Computing, an online storage house
Cloud Computing, an online storage house
 
Final
FinalFinal
Final
 
Composing and Scaling Data Platforms-2015
Composing and Scaling Data Platforms-2015Composing and Scaling Data Platforms-2015
Composing and Scaling Data Platforms-2015
 
Composing and scaling data platforms
Composing and scaling data platformsComposing and scaling data platforms
Composing and scaling data platforms
 
How MongoDB is Being Used in China - Case Studies
How MongoDB is Being Used in China - Case StudiesHow MongoDB is Being Used in China - Case Studies
How MongoDB is Being Used in China - Case Studies
 
Shillum "Building for the Future: Interoperability"
Shillum "Building for the Future: Interoperability"Shillum "Building for the Future: Interoperability"
Shillum "Building for the Future: Interoperability"
 
Enabling Your Data Science Team with Modern Data Engineering
Enabling Your Data Science Team with Modern Data EngineeringEnabling Your Data Science Team with Modern Data Engineering
Enabling Your Data Science Team with Modern Data Engineering
 
Speed up your regression and reduce cost load with Selenoid + K8s + ReportPortal
Speed up your regression and reduce cost load with Selenoid + K8s + ReportPortalSpeed up your regression and reduce cost load with Selenoid + K8s + ReportPortal
Speed up your regression and reduce cost load with Selenoid + K8s + ReportPortal
 
Selkie Overview
Selkie OverviewSelkie Overview
Selkie Overview
 
How Will Going Virtual Impact Your Search Performance?
How Will Going Virtual Impact Your Search Performance?How Will Going Virtual Impact Your Search Performance?
How Will Going Virtual Impact Your Search Performance?
 
2016 NDC - 모바일 게임 서버 엔진 개발 후기
2016 NDC - 모바일 게임 서버 엔진 개발 후기2016 NDC - 모바일 게임 서버 엔진 개발 후기
2016 NDC - 모바일 게임 서버 엔진 개발 후기
 
Spotlight on the petroleum and energy vertical
Spotlight on the petroleum and energy vertical Spotlight on the petroleum and energy vertical
Spotlight on the petroleum and energy vertical
 

Recently uploaded

Atelier - Innover avec l’IA Générative et les graphes de connaissances
Atelier - Innover avec l’IA Générative et les graphes de connaissancesAtelier - Innover avec l’IA Générative et les graphes de connaissances
Atelier - Innover avec l’IA Générative et les graphes de connaissances
Neo4j
 
Graspan: A Big Data System for Big Code Analysis
Graspan: A Big Data System for Big Code AnalysisGraspan: A Big Data System for Big Code Analysis
Graspan: A Big Data System for Big Code Analysis
Aftab Hussain
 
Cracking the code review at SpringIO 2024
Cracking the code review at SpringIO 2024Cracking the code review at SpringIO 2024
Cracking the code review at SpringIO 2024
Paco van Beckhoven
 
A Sighting of filterA in Typelevel Rite of Passage
A Sighting of filterA in Typelevel Rite of PassageA Sighting of filterA in Typelevel Rite of Passage
A Sighting of filterA in Typelevel Rite of Passage
Philip Schwarz
 
Essentials of Automations: The Art of Triggers and Actions in FME
Essentials of Automations: The Art of Triggers and Actions in FMEEssentials of Automations: The Art of Triggers and Actions in FME
Essentials of Automations: The Art of Triggers and Actions in FME
Safe Software
 
Artificia Intellicence and XPath Extension Functions
Artificia Intellicence and XPath Extension FunctionsArtificia Intellicence and XPath Extension Functions
Artificia Intellicence and XPath Extension Functions
Octavian Nadolu
 
2024 eCommerceDays Toulouse - Sylius 2.0.pdf
2024 eCommerceDays Toulouse - Sylius 2.0.pdf2024 eCommerceDays Toulouse - Sylius 2.0.pdf
2024 eCommerceDays Toulouse - Sylius 2.0.pdf
Łukasz Chruściel
 
GOING AOT WITH GRAALVM FOR SPRING BOOT (SPRING IO)
GOING AOT WITH GRAALVM FOR  SPRING BOOT (SPRING IO)GOING AOT WITH GRAALVM FOR  SPRING BOOT (SPRING IO)
GOING AOT WITH GRAALVM FOR SPRING BOOT (SPRING IO)
Alina Yurenko
 
Mobile App Development Company In Noida | Drona Infotech
Mobile App Development Company In Noida | Drona InfotechMobile App Development Company In Noida | Drona Infotech
Mobile App Development Company In Noida | Drona Infotech
Drona Infotech
 
Automated software refactoring with OpenRewrite and Generative AI.pptx.pdf
Automated software refactoring with OpenRewrite and Generative AI.pptx.pdfAutomated software refactoring with OpenRewrite and Generative AI.pptx.pdf
Automated software refactoring with OpenRewrite and Generative AI.pptx.pdf
timtebeek1
 
GraphSummit Paris - The art of the possible with Graph Technology
GraphSummit Paris - The art of the possible with Graph TechnologyGraphSummit Paris - The art of the possible with Graph Technology
GraphSummit Paris - The art of the possible with Graph Technology
Neo4j
 
Using Xen Hypervisor for Functional Safety
Using Xen Hypervisor for Functional SafetyUsing Xen Hypervisor for Functional Safety
Using Xen Hypervisor for Functional Safety
Ayan Halder
 
Need for Speed: Removing speed bumps from your Symfony projects ⚡️
Need for Speed: Removing speed bumps from your Symfony projects ⚡️Need for Speed: Removing speed bumps from your Symfony projects ⚡️
Need for Speed: Removing speed bumps from your Symfony projects ⚡️
Łukasz Chruściel
 
Transform Your Communication with Cloud-Based IVR Solutions
Transform Your Communication with Cloud-Based IVR SolutionsTransform Your Communication with Cloud-Based IVR Solutions
Transform Your Communication with Cloud-Based IVR Solutions
TheSMSPoint
 
Navigating the Metaverse: A Journey into Virtual Evolution"
Navigating the Metaverse: A Journey into Virtual Evolution"Navigating the Metaverse: A Journey into Virtual Evolution"
Navigating the Metaverse: A Journey into Virtual Evolution"
Donna Lenk
 
E-commerce Application Development Company.pdf
E-commerce Application Development Company.pdfE-commerce Application Development Company.pdf
E-commerce Application Development Company.pdf
Hornet Dynamics
 
A Study of Variable-Role-based Feature Enrichment in Neural Models of Code
A Study of Variable-Role-based Feature Enrichment in Neural Models of CodeA Study of Variable-Role-based Feature Enrichment in Neural Models of Code
A Study of Variable-Role-based Feature Enrichment in Neural Models of Code
Aftab Hussain
 
May Marketo Masterclass, London MUG May 22 2024.pdf
May Marketo Masterclass, London MUG May 22 2024.pdfMay Marketo Masterclass, London MUG May 22 2024.pdf
May Marketo Masterclass, London MUG May 22 2024.pdf
Adele Miller
 
Orion Context Broker introduction 20240604
Orion Context Broker introduction 20240604Orion Context Broker introduction 20240604
Orion Context Broker introduction 20240604
Fermin Galan
 
OpenMetadata Community Meeting - 5th June 2024
OpenMetadata Community Meeting - 5th June 2024OpenMetadata Community Meeting - 5th June 2024
OpenMetadata Community Meeting - 5th June 2024
OpenMetadata
 

Recently uploaded (20)

Atelier - Innover avec l’IA Générative et les graphes de connaissances
Atelier - Innover avec l’IA Générative et les graphes de connaissancesAtelier - Innover avec l’IA Générative et les graphes de connaissances
Atelier - Innover avec l’IA Générative et les graphes de connaissances
 
Graspan: A Big Data System for Big Code Analysis
Graspan: A Big Data System for Big Code AnalysisGraspan: A Big Data System for Big Code Analysis
Graspan: A Big Data System for Big Code Analysis
 
Cracking the code review at SpringIO 2024
Cracking the code review at SpringIO 2024Cracking the code review at SpringIO 2024
Cracking the code review at SpringIO 2024
 
A Sighting of filterA in Typelevel Rite of Passage
A Sighting of filterA in Typelevel Rite of PassageA Sighting of filterA in Typelevel Rite of Passage
A Sighting of filterA in Typelevel Rite of Passage
 
Essentials of Automations: The Art of Triggers and Actions in FME
Essentials of Automations: The Art of Triggers and Actions in FMEEssentials of Automations: The Art of Triggers and Actions in FME
Essentials of Automations: The Art of Triggers and Actions in FME
 
Artificia Intellicence and XPath Extension Functions
Artificia Intellicence and XPath Extension FunctionsArtificia Intellicence and XPath Extension Functions
Artificia Intellicence and XPath Extension Functions
 
2024 eCommerceDays Toulouse - Sylius 2.0.pdf
2024 eCommerceDays Toulouse - Sylius 2.0.pdf2024 eCommerceDays Toulouse - Sylius 2.0.pdf
2024 eCommerceDays Toulouse - Sylius 2.0.pdf
 
GOING AOT WITH GRAALVM FOR SPRING BOOT (SPRING IO)
GOING AOT WITH GRAALVM FOR  SPRING BOOT (SPRING IO)GOING AOT WITH GRAALVM FOR  SPRING BOOT (SPRING IO)
GOING AOT WITH GRAALVM FOR SPRING BOOT (SPRING IO)
 
Mobile App Development Company In Noida | Drona Infotech
Mobile App Development Company In Noida | Drona InfotechMobile App Development Company In Noida | Drona Infotech
Mobile App Development Company In Noida | Drona Infotech
 
Automated software refactoring with OpenRewrite and Generative AI.pptx.pdf
Automated software refactoring with OpenRewrite and Generative AI.pptx.pdfAutomated software refactoring with OpenRewrite and Generative AI.pptx.pdf
Automated software refactoring with OpenRewrite and Generative AI.pptx.pdf
 
GraphSummit Paris - The art of the possible with Graph Technology
GraphSummit Paris - The art of the possible with Graph TechnologyGraphSummit Paris - The art of the possible with Graph Technology
GraphSummit Paris - The art of the possible with Graph Technology
 
Using Xen Hypervisor for Functional Safety
Using Xen Hypervisor for Functional SafetyUsing Xen Hypervisor for Functional Safety
Using Xen Hypervisor for Functional Safety
 
Need for Speed: Removing speed bumps from your Symfony projects ⚡️
Need for Speed: Removing speed bumps from your Symfony projects ⚡️Need for Speed: Removing speed bumps from your Symfony projects ⚡️
Need for Speed: Removing speed bumps from your Symfony projects ⚡️
 
Transform Your Communication with Cloud-Based IVR Solutions
Transform Your Communication with Cloud-Based IVR SolutionsTransform Your Communication with Cloud-Based IVR Solutions
Transform Your Communication with Cloud-Based IVR Solutions
 
Navigating the Metaverse: A Journey into Virtual Evolution"
Navigating the Metaverse: A Journey into Virtual Evolution"Navigating the Metaverse: A Journey into Virtual Evolution"
Navigating the Metaverse: A Journey into Virtual Evolution"
 
E-commerce Application Development Company.pdf
E-commerce Application Development Company.pdfE-commerce Application Development Company.pdf
E-commerce Application Development Company.pdf
 
A Study of Variable-Role-based Feature Enrichment in Neural Models of Code
A Study of Variable-Role-based Feature Enrichment in Neural Models of CodeA Study of Variable-Role-based Feature Enrichment in Neural Models of Code
A Study of Variable-Role-based Feature Enrichment in Neural Models of Code
 
May Marketo Masterclass, London MUG May 22 2024.pdf
May Marketo Masterclass, London MUG May 22 2024.pdfMay Marketo Masterclass, London MUG May 22 2024.pdf
May Marketo Masterclass, London MUG May 22 2024.pdf
 
Orion Context Broker introduction 20240604
Orion Context Broker introduction 20240604Orion Context Broker introduction 20240604
Orion Context Broker introduction 20240604
 
OpenMetadata Community Meeting - 5th June 2024
OpenMetadata Community Meeting - 5th June 2024OpenMetadata Community Meeting - 5th June 2024
OpenMetadata Community Meeting - 5th June 2024
 

2018 03-08 moving from legacy to event driven with kafka - confoo

  • 1. Confoo 08-03-2018 Moving from legacy to Event Driven with Kafka Jeroen van der Gulik
  • 2. About me ❖ Software Architect/ Consultant ❖ Co-Founder Isset (https://isset.nl) ❖ Señor Developer ❖ Organizer DDDNL ❖ Builder of Artificial Stupidity ❖ @n0x13
  • 3.
  • 4.
  • 5.
  • 6.
  • 7. “The road to hell is paved with good intentions”
  • 12. Costs
  • 13.
  • 14. The 5 sins of Legacy ❖ Flexibility ❖ Security ❖ Dependancy ❖ Complexity ❖ Costs
  • 15.
  • 16.
  • 17. “Nothing is more permanent than a temporary solution”
  • 18.
  • 19.
  • 20.
  • 21.
  • 22. The Project ❖ Proprietary software ❖ Hacked on “solutions” ❖ Old hardware ❖ Business logic in SP: No version control ❖ Original Team members gone ❖ Unwilling supplier
  • 23.
  • 24.
  • 25.
  • 26. The Assignment ❖ Rebuild the Web Portal from scratch ❖ Decouple from the main system ❖ Make the data queryable without causing extra load
  • 27. The Constraints ❖ Cannot change proprietary software easily ❖ Load excessively high: no additional load ❖ Real Time querying not an option ❖ Slow release cycles (quarterly) ❖ Time constraint ❖ Do not jeopardise production
  • 28.
  • 34.
  • 35.
  • 36.
  • 37.
  • 38.
  • 39.
  • 40.
  • 41.
  • 42.
  • 43.
  • 44.
  • 45.
  • 46.
  • 47. Event-driven architecture (EDA), is a software architecture pattern promoting the production, detection, consumption of, and reaction to events.
  • 48.
  • 49.
  • 50.
  • 51.
  • 52.
  • 53.
  • 54.
  • 55.
  • 56.
  • 57.
  • 58.
  • 59.
  • 60.
  • 61.
  • 62.
  • 63.
  • 64.
  • 65.
  • 66.
  • 67.
  • 68.
  • 69. Lessons learned ❖ Events give you a powerful way to decouple ❖ Events are very good contracts ❖ Event driven is a flexible architecture ❖ Making team think in terms of Events can be challenging ❖ Kafka is a very powerful tool but needs research ❖ Mistake are great learning moments ❖ Eventual Consistency is still a hard problem