SlideShare a Scribd company logo
Event processing with Node.js and RabbitMQ
by Zlatko Duric
Disclaimer (AKA about me)
● Experience
– Node.js? Lots
– NoSQL? Lots.
– Messaging? Little
● Backend Developer at ThinxNet.com
(http://thinxnet.com/#jobs)
Connected cars?
● Hardware device
● Various sensors – lots of data
● “Cloud” processing
Ready to serve. How?
● Event collection and processing
● Webapp, mobile app
● Backoffice and support apps, notification services etc.
● Reporting, third party software integration etc
● REST approach:
– Lots of requests
– Parsing of data is blocking
– Avoid choking devices
– Communication overhead
– Error management
What service? Microservice!
● Redis PUBSUB
● ZeroMQ, MQTT, AMQP
● RabbitMQ, ActiveMQ
About RabbitMQ
● Messaging broker based on AMQP protocol
● Platform/language independent
● Extensible (Erlang)
● Persistent message queues
● Node.js drivers (node-amqp, rabbit.js...)
Hello world
Exchanges and topics
● Publisher and subscriber. Simple as that.
● Get message, ack or reject (nack) it.
● Subscribers listen on a queue
● Publishers talk to an exchange
● Exchange routes to queues based on topics
Example
Channels and connections
● 1 node process = 1 connection
● Many channels, share same connection
PUBSUB
● Simple PUBSUB, as used elsewhere
● All listeners get the same message
● Suboptimal
Example
PUSH-PULL/WORKER
● Round-robin distribution
● Worker socket.ack() is explicit (rabbit.js
driver)
● Prefetch sets the number of msgs received
● Problem: concurrent/sequential
modification?
Example
Content-based routing
● Erlang extensions? Not standard.
● App-side:
– Keep internal mini-queue
– Do not ack a msg if it's not first on local
queue
– Hold internal messages based on
internal “topic”, ie “user.<ID>.save” -
hold msgs for that user
– Item.update() instead of item.save()
● Gets nasty fast, should minimize
these scenarios
Example
REST API with MQ
● REQUEST/REPLY for immediate responses
● Other patterns for direct access
– REST API just forwards the message
– replies will go by other route
● Pub/sub is easy!
Example
Error handling
● Uncaught exceptions are covered
● Retry logic is simplified – push incomplete
messages to a 'fixer' queue, requeue failed
jobs or when no resources
● Dedicated error handling
Example
Error handling
● Uncaught exceptions are covered
● Retry logic is simplified – push incomplete
messages to a 'fixer' queue, requeue failed
jobs or when no resources
● Dedicated error handling
Testing
● No good known mocks, please help
● RabbitMQ always on on Jenkins machine
● Test handlers directly
Buzzwords index
● Cloud
● Connected car
● Microservice
● Patterns
:)
Info sources
● https://www.rabbitmq.com/
● http://www.squaremobius.net/rabbit.js/
● http://www.manning.com/videla/
Note to self
● Thank the audience for patience
● Invite audience for a discussion and questions

More Related Content

What's hot

Moodle performance and stress testing
Moodle performance and stress testingMoodle performance and stress testing
Moodle performance and stress testing
moorejon
 
Zingme practice for building scalable website with PHP
Zingme practice for building scalable website with PHPZingme practice for building scalable website with PHP
Zingme practice for building scalable website with PHP
Chau Thanh
 
NullMQ @ PDX
NullMQ @ PDXNullMQ @ PDX
NullMQ @ PDX
Jeff Lindsay
 
Services, dependencies, and you
Services, dependencies, and youServices, dependencies, and you
Services, dependencies, and you
Sean Kelly
 
JMeter performance and scalability in Moodle Montana Moot 2014
JMeter performance and scalability in Moodle Montana Moot 2014JMeter performance and scalability in Moodle Montana Moot 2014
JMeter performance and scalability in Moodle Montana Moot 2014
moorejon
 
Distributed Systems
Distributed SystemsDistributed Systems
Distributed Systems
Bruno Bossola
 
Wordlist Generation and Wifi Cracking
Wordlist Generation and Wifi CrackingWordlist Generation and Wifi Cracking
Wordlist Generation and Wifi Cracking
Shakar Bhattarai
 
React web development
React web developmentReact web development
React web development
Rully Ramanda
 
Welcome to NPM search 2.0
Welcome to NPM search 2.0Welcome to NPM search 2.0
Welcome to NPM search 2.0
Dejan Toteff
 
Cache hcm-topdev
Cache hcm-topdevCache hcm-topdev
Cache hcm-topdev
Thanh Chau
 
Tools to create a secure build pipeline
Tools to create a secure build pipelineTools to create a secure build pipeline
Tools to create a secure build pipeline
Bruno Bossola
 
RestMS Introduction
RestMS IntroductionRestMS Introduction
RestMS Introduction
pieterh
 
[DevDay 2019] Reactive Programming with JavaScript - By Pham Nguyen Duc, Web ...
[DevDay 2019] Reactive Programming with JavaScript - By Pham Nguyen Duc, Web ...[DevDay 2019] Reactive Programming with JavaScript - By Pham Nguyen Duc, Web ...
[DevDay 2019] Reactive Programming with JavaScript - By Pham Nguyen Duc, Web ...
DevDay.org
 
Microservices - the lean way
Microservices - the lean wayMicroservices - the lean way
Microservices - the lean way
Bruno Bossola
 
Fosdem 2009
Fosdem 2009Fosdem 2009
Fosdem 2009
pieterh
 
ZaloPay Merchant Platform on K8S on-premise
ZaloPay Merchant Platform on K8S on-premiseZaloPay Merchant Platform on K8S on-premise
ZaloPay Merchant Platform on K8S on-premise
Chau Thanh
 
Moodle performance optimizations
Moodle performance optimizationsMoodle performance optimizations
Moodle performance optimizations
Jan Meier
 
Алексей Злобин «Scala in Goozy»
Алексей Злобин «Scala in Goozy»Алексей Злобин «Scala in Goozy»
Алексей Злобин «Scala in Goozy»
e-Legion
 
Code in python
Code in pythonCode in python
Code in python
MD ALL RABBI
 

What's hot (19)

Moodle performance and stress testing
Moodle performance and stress testingMoodle performance and stress testing
Moodle performance and stress testing
 
Zingme practice for building scalable website with PHP
Zingme practice for building scalable website with PHPZingme practice for building scalable website with PHP
Zingme practice for building scalable website with PHP
 
NullMQ @ PDX
NullMQ @ PDXNullMQ @ PDX
NullMQ @ PDX
 
Services, dependencies, and you
Services, dependencies, and youServices, dependencies, and you
Services, dependencies, and you
 
JMeter performance and scalability in Moodle Montana Moot 2014
JMeter performance and scalability in Moodle Montana Moot 2014JMeter performance and scalability in Moodle Montana Moot 2014
JMeter performance and scalability in Moodle Montana Moot 2014
 
Distributed Systems
Distributed SystemsDistributed Systems
Distributed Systems
 
Wordlist Generation and Wifi Cracking
Wordlist Generation and Wifi CrackingWordlist Generation and Wifi Cracking
Wordlist Generation and Wifi Cracking
 
React web development
React web developmentReact web development
React web development
 
Welcome to NPM search 2.0
Welcome to NPM search 2.0Welcome to NPM search 2.0
Welcome to NPM search 2.0
 
Cache hcm-topdev
Cache hcm-topdevCache hcm-topdev
Cache hcm-topdev
 
Tools to create a secure build pipeline
Tools to create a secure build pipelineTools to create a secure build pipeline
Tools to create a secure build pipeline
 
RestMS Introduction
RestMS IntroductionRestMS Introduction
RestMS Introduction
 
[DevDay 2019] Reactive Programming with JavaScript - By Pham Nguyen Duc, Web ...
[DevDay 2019] Reactive Programming with JavaScript - By Pham Nguyen Duc, Web ...[DevDay 2019] Reactive Programming with JavaScript - By Pham Nguyen Duc, Web ...
[DevDay 2019] Reactive Programming with JavaScript - By Pham Nguyen Duc, Web ...
 
Microservices - the lean way
Microservices - the lean wayMicroservices - the lean way
Microservices - the lean way
 
Fosdem 2009
Fosdem 2009Fosdem 2009
Fosdem 2009
 
ZaloPay Merchant Platform on K8S on-premise
ZaloPay Merchant Platform on K8S on-premiseZaloPay Merchant Platform on K8S on-premise
ZaloPay Merchant Platform on K8S on-premise
 
Moodle performance optimizations
Moodle performance optimizationsMoodle performance optimizations
Moodle performance optimizations
 
Алексей Злобин «Scala in Goozy»
Алексей Злобин «Scala in Goozy»Алексей Злобин «Scala in Goozy»
Алексей Злобин «Scala in Goozy»
 
Code in python
Code in pythonCode in python
Code in python
 

Viewers also liked

La generación y el uso limpios y adecuados de energía
La generación y el uso limpios y adecuados de energíaLa generación y el uso limpios y adecuados de energía
La generación y el uso limpios y adecuados de energía
Green Drinks International
 
Αφιέρωμα στο Γιάννη Βεζιρτζή
Αφιέρωμα στο Γιάννη ΒεζιρτζήΑφιέρωμα στο Γιάννη Βεζιρτζή
Αφιέρωμα στο Γιάννη Βεζιρτζή
Lampros Nikolaras
 
Artist Mise-En-Scene
Artist Mise-En-SceneArtist Mise-En-Scene
Artist Mise-En-Scene
itsjanakan
 
Trailer Content Analysis
Trailer Content AnalysisTrailer Content Analysis
Trailer Content Analysis
DarcyB16
 
Интернет-магазина от а до я - Александр Васильев - AWG
Интернет-магазина от а до я - Александр Васильев - AWGИнтернет-магазина от а до я - Александр Васильев - AWG
Интернет-магазина от а до я - Александр Васильев - AWG
awgua
 
Risk and compliance is a Business Strength
Risk and compliance is a Business Strength Risk and compliance is a Business Strength
Risk and compliance is a Business Strength
Canon Belgium
 
Reciclar en las Aulas
 Reciclar en las Aulas Reciclar en las Aulas
Reciclar en las Aulas
Green Drinks International
 
Restaurant porter kpi
Restaurant porter kpiRestaurant porter kpi
Restaurant porter kpi
diretjom
 
Reciclar en cordoba
Reciclar en cordobaReciclar en cordoba
Reciclar en cordoba
Green Drinks International
 
Contabilidad de costos
Contabilidad de costosContabilidad de costos
Contabilidad de costos
Marlon Fabrizio
 
Q1 front cover
Q1 front coverQ1 front cover
Q1 front cover
tiatesfaye
 
Basics of accounting_level_ii
Basics of accounting_level_iiBasics of accounting_level_ii
Basics of accounting_level_ii
mohamed atia
 
NI Company Profile
NI Company ProfileNI Company Profile
NI Company Profile
Adrianto Januri
 
Señales
SeñalesSeñales
Señales
ildefonso casas
 
Deacon PowerPoint pptx
Deacon PowerPoint pptxDeacon PowerPoint pptx
Deacon PowerPoint pptx
Dana Lindsey
 
Narrative theories
Narrative theoriesNarrative theories
Narrative theories
feelgoodinc2024
 
А. Васильев "Создание интернет-магазина: первые шаги, сложности и подводные к...
А. Васильев "Создание интернет-магазина: первые шаги, сложности и подводные к...А. Васильев "Создание интернет-магазина: первые шаги, сложности и подводные к...
А. Васильев "Создание интернет-магазина: первые шаги, сложности и подводные к...
awgua
 
Hawaii conference2015
Hawaii conference2015Hawaii conference2015
Hawaii conference2015
J C
 
naturaleza
naturalezanaturaleza
naturaleza
ildefonso casas
 
Total learning: The Intersection of Formal, Social and Experiential Learning
Total learning:  The Intersection of Formal, Social and Experiential LearningTotal learning:  The Intersection of Formal, Social and Experiential Learning
Total learning: The Intersection of Formal, Social and Experiential Learning
John Leh
 

Viewers also liked (20)

La generación y el uso limpios y adecuados de energía
La generación y el uso limpios y adecuados de energíaLa generación y el uso limpios y adecuados de energía
La generación y el uso limpios y adecuados de energía
 
Αφιέρωμα στο Γιάννη Βεζιρτζή
Αφιέρωμα στο Γιάννη ΒεζιρτζήΑφιέρωμα στο Γιάννη Βεζιρτζή
Αφιέρωμα στο Γιάννη Βεζιρτζή
 
Artist Mise-En-Scene
Artist Mise-En-SceneArtist Mise-En-Scene
Artist Mise-En-Scene
 
Trailer Content Analysis
Trailer Content AnalysisTrailer Content Analysis
Trailer Content Analysis
 
Интернет-магазина от а до я - Александр Васильев - AWG
Интернет-магазина от а до я - Александр Васильев - AWGИнтернет-магазина от а до я - Александр Васильев - AWG
Интернет-магазина от а до я - Александр Васильев - AWG
 
Risk and compliance is a Business Strength
Risk and compliance is a Business Strength Risk and compliance is a Business Strength
Risk and compliance is a Business Strength
 
Reciclar en las Aulas
 Reciclar en las Aulas Reciclar en las Aulas
Reciclar en las Aulas
 
Restaurant porter kpi
Restaurant porter kpiRestaurant porter kpi
Restaurant porter kpi
 
Reciclar en cordoba
Reciclar en cordobaReciclar en cordoba
Reciclar en cordoba
 
Contabilidad de costos
Contabilidad de costosContabilidad de costos
Contabilidad de costos
 
Q1 front cover
Q1 front coverQ1 front cover
Q1 front cover
 
Basics of accounting_level_ii
Basics of accounting_level_iiBasics of accounting_level_ii
Basics of accounting_level_ii
 
NI Company Profile
NI Company ProfileNI Company Profile
NI Company Profile
 
Señales
SeñalesSeñales
Señales
 
Deacon PowerPoint pptx
Deacon PowerPoint pptxDeacon PowerPoint pptx
Deacon PowerPoint pptx
 
Narrative theories
Narrative theoriesNarrative theories
Narrative theories
 
А. Васильев "Создание интернет-магазина: первые шаги, сложности и подводные к...
А. Васильев "Создание интернет-магазина: первые шаги, сложности и подводные к...А. Васильев "Создание интернет-магазина: первые шаги, сложности и подводные к...
А. Васильев "Создание интернет-магазина: первые шаги, сложности и подводные к...
 
Hawaii conference2015
Hawaii conference2015Hawaii conference2015
Hawaii conference2015
 
naturaleza
naturalezanaturaleza
naturaleza
 
Total learning: The Intersection of Formal, Social and Experiential Learning
Total learning:  The Intersection of Formal, Social and Experiential LearningTotal learning:  The Intersection of Formal, Social and Experiential Learning
Total learning: The Intersection of Formal, Social and Experiential Learning
 

Similar to Presentation

Activity feeds (and more) at mate1
Activity feeds (and more) at mate1Activity feeds (and more) at mate1
Activity feeds (and more) at mate1
Hisham Mardam-Bey
 
Gatling - Bordeaux JUG
Gatling - Bordeaux JUGGatling - Bordeaux JUG
Gatling - Bordeaux JUG
slandelle
 
Node.js Web Apps @ ebay scale
Node.js Web Apps @ ebay scaleNode.js Web Apps @ ebay scale
Node.js Web Apps @ ebay scale
Dmytro Semenov
 
Proactive monitoring tools or services - Open Source
Proactive monitoring tools or services - Open Source Proactive monitoring tools or services - Open Source
Proactive monitoring tools or services - Open Source
B.A.
 
PHP At 5000 Requests Per Second: Hootsuite’s Scaling Story
PHP At 5000 Requests Per Second: Hootsuite’s Scaling StoryPHP At 5000 Requests Per Second: Hootsuite’s Scaling Story
PHP At 5000 Requests Per Second: Hootsuite’s Scaling Story
vanphp
 
Website Monitoring with Distributed Messages/Tasks Processing (AMQP & RabbitM...
Website Monitoring with Distributed Messages/Tasks Processing (AMQP & RabbitM...Website Monitoring with Distributed Messages/Tasks Processing (AMQP & RabbitM...
Website Monitoring with Distributed Messages/Tasks Processing (AMQP & RabbitM...
Jimmy DeadcOde
 
Backend & Frontend architecture scalability & websockets
Backend & Frontend architecture scalability & websocketsBackend & Frontend architecture scalability & websockets
Backend & Frontend architecture scalability & websockets
Anne Jan Brouwer
 
Logging : How much is too much? Network Security Monitoring Talk @ hasgeek
Logging : How much is too much? Network Security Monitoring Talk @ hasgeekLogging : How much is too much? Network Security Monitoring Talk @ hasgeek
Logging : How much is too much? Network Security Monitoring Talk @ hasgeek
vivekrajan
 
Monkey Server
Monkey ServerMonkey Server
Monkey Server
Eduardo Silva Pereira
 
Introduction to ZeroMQ - eSpace TechTalk
Introduction to ZeroMQ - eSpace TechTalkIntroduction to ZeroMQ - eSpace TechTalk
Introduction to ZeroMQ - eSpace TechTalk
Mahmoud Said
 
Node.js Course 2 of 2 - Advanced techniques
Node.js Course 2 of 2 - Advanced techniquesNode.js Course 2 of 2 - Advanced techniques
Node.js Course 2 of 2 - Advanced techniques
Manuel Eusebio de Paz Carmona
 
Microservice architecture
Microservice architectureMicroservice architecture
Microservice architecture
Žilvinas Kuusas
 
Full stack development
Full stack developmentFull stack development
Full stack development
Pavlo Iuriichuk
 
Not my problem - Delegating responsibility to infrastructure
Not my problem - Delegating responsibility to infrastructureNot my problem - Delegating responsibility to infrastructure
Not my problem - Delegating responsibility to infrastructure
Yshay Yaacobi
 
Javascript Update May 2013
Javascript Update May 2013Javascript Update May 2013
Javascript Update May 2013
Ramesh Nair
 
Event driven systems
Event driven systems Event driven systems
Event driven systems
Shatabda Majumdar
 
Real World AngularJS recipes: beyond TodoMVC
Real World AngularJS recipes: beyond TodoMVCReal World AngularJS recipes: beyond TodoMVC
Real World AngularJS recipes: beyond TodoMVC
Carlo Bonamico
 
Real World AngularJS recipes: beyond TodoMVC - Carlo Bonamico, Sonia Pini - C...
Real World AngularJS recipes: beyond TodoMVC - Carlo Bonamico, Sonia Pini - C...Real World AngularJS recipes: beyond TodoMVC - Carlo Bonamico, Sonia Pini - C...
Real World AngularJS recipes: beyond TodoMVC - Carlo Bonamico, Sonia Pini - C...
Codemotion
 
NS3 Overview
NS3 OverviewNS3 Overview
NS3 Overview
Rahul Hada
 
Nodejs
NodejsNodejs

Similar to Presentation (20)

Activity feeds (and more) at mate1
Activity feeds (and more) at mate1Activity feeds (and more) at mate1
Activity feeds (and more) at mate1
 
Gatling - Bordeaux JUG
Gatling - Bordeaux JUGGatling - Bordeaux JUG
Gatling - Bordeaux JUG
 
Node.js Web Apps @ ebay scale
Node.js Web Apps @ ebay scaleNode.js Web Apps @ ebay scale
Node.js Web Apps @ ebay scale
 
Proactive monitoring tools or services - Open Source
Proactive monitoring tools or services - Open Source Proactive monitoring tools or services - Open Source
Proactive monitoring tools or services - Open Source
 
PHP At 5000 Requests Per Second: Hootsuite’s Scaling Story
PHP At 5000 Requests Per Second: Hootsuite’s Scaling StoryPHP At 5000 Requests Per Second: Hootsuite’s Scaling Story
PHP At 5000 Requests Per Second: Hootsuite’s Scaling Story
 
Website Monitoring with Distributed Messages/Tasks Processing (AMQP & RabbitM...
Website Monitoring with Distributed Messages/Tasks Processing (AMQP & RabbitM...Website Monitoring with Distributed Messages/Tasks Processing (AMQP & RabbitM...
Website Monitoring with Distributed Messages/Tasks Processing (AMQP & RabbitM...
 
Backend & Frontend architecture scalability & websockets
Backend & Frontend architecture scalability & websocketsBackend & Frontend architecture scalability & websockets
Backend & Frontend architecture scalability & websockets
 
Logging : How much is too much? Network Security Monitoring Talk @ hasgeek
Logging : How much is too much? Network Security Monitoring Talk @ hasgeekLogging : How much is too much? Network Security Monitoring Talk @ hasgeek
Logging : How much is too much? Network Security Monitoring Talk @ hasgeek
 
Monkey Server
Monkey ServerMonkey Server
Monkey Server
 
Introduction to ZeroMQ - eSpace TechTalk
Introduction to ZeroMQ - eSpace TechTalkIntroduction to ZeroMQ - eSpace TechTalk
Introduction to ZeroMQ - eSpace TechTalk
 
Node.js Course 2 of 2 - Advanced techniques
Node.js Course 2 of 2 - Advanced techniquesNode.js Course 2 of 2 - Advanced techniques
Node.js Course 2 of 2 - Advanced techniques
 
Microservice architecture
Microservice architectureMicroservice architecture
Microservice architecture
 
Full stack development
Full stack developmentFull stack development
Full stack development
 
Not my problem - Delegating responsibility to infrastructure
Not my problem - Delegating responsibility to infrastructureNot my problem - Delegating responsibility to infrastructure
Not my problem - Delegating responsibility to infrastructure
 
Javascript Update May 2013
Javascript Update May 2013Javascript Update May 2013
Javascript Update May 2013
 
Event driven systems
Event driven systems Event driven systems
Event driven systems
 
Real World AngularJS recipes: beyond TodoMVC
Real World AngularJS recipes: beyond TodoMVCReal World AngularJS recipes: beyond TodoMVC
Real World AngularJS recipes: beyond TodoMVC
 
Real World AngularJS recipes: beyond TodoMVC - Carlo Bonamico, Sonia Pini - C...
Real World AngularJS recipes: beyond TodoMVC - Carlo Bonamico, Sonia Pini - C...Real World AngularJS recipes: beyond TodoMVC - Carlo Bonamico, Sonia Pini - C...
Real World AngularJS recipes: beyond TodoMVC - Carlo Bonamico, Sonia Pini - C...
 
NS3 Overview
NS3 OverviewNS3 Overview
NS3 Overview
 
Nodejs
NodejsNodejs
Nodejs
 

Recently uploaded

ALGIT - Assembly Line for Green IT - Numbers, Data, Facts
ALGIT - Assembly Line for Green IT - Numbers, Data, FactsALGIT - Assembly Line for Green IT - Numbers, Data, Facts
ALGIT - Assembly Line for Green IT - Numbers, Data, Facts
Green Software Development
 
E-commerce Development Services- Hornet Dynamics
E-commerce Development Services- Hornet DynamicsE-commerce Development Services- Hornet Dynamics
E-commerce Development Services- Hornet Dynamics
Hornet Dynamics
 
DDS-Security 1.2 - What's New? Stronger security for long-running systems
DDS-Security 1.2 - What's New? Stronger security for long-running systemsDDS-Security 1.2 - What's New? Stronger security for long-running systems
DDS-Security 1.2 - What's New? Stronger security for long-running systems
Gerardo Pardo-Castellote
 
What is Augmented Reality Image Tracking
What is Augmented Reality Image TrackingWhat is Augmented Reality Image Tracking
What is Augmented Reality Image Tracking
pavan998932
 
Neo4j - Product Vision and Knowledge Graphs - GraphSummit Paris
Neo4j - Product Vision and Knowledge Graphs - GraphSummit ParisNeo4j - Product Vision and Knowledge Graphs - GraphSummit Paris
Neo4j - Product Vision and Knowledge Graphs - GraphSummit Paris
Neo4j
 
Using Query Store in Azure PostgreSQL to Understand Query Performance
Using Query Store in Azure PostgreSQL to Understand Query PerformanceUsing Query Store in Azure PostgreSQL to Understand Query Performance
Using Query Store in Azure PostgreSQL to Understand Query Performance
Grant Fritchey
 
GraphSummit Paris - The art of the possible with Graph Technology
GraphSummit Paris - The art of the possible with Graph TechnologyGraphSummit Paris - The art of the possible with Graph Technology
GraphSummit Paris - The art of the possible with Graph Technology
Neo4j
 
SMS API Integration in Saudi Arabia| Best SMS API Service
SMS API Integration in Saudi Arabia| Best SMS API ServiceSMS API Integration in Saudi Arabia| Best SMS API Service
SMS API Integration in Saudi Arabia| Best SMS API Service
Yara Milbes
 
Need for Speed: Removing speed bumps from your Symfony projects ⚡️
Need for Speed: Removing speed bumps from your Symfony projects ⚡️Need for Speed: Removing speed bumps from your Symfony projects ⚡️
Need for Speed: Removing speed bumps from your Symfony projects ⚡️
Łukasz Chruściel
 
LORRAINE ANDREI_LEQUIGAN_HOW TO USE ZOOM
LORRAINE ANDREI_LEQUIGAN_HOW TO USE ZOOMLORRAINE ANDREI_LEQUIGAN_HOW TO USE ZOOM
LORRAINE ANDREI_LEQUIGAN_HOW TO USE ZOOM
lorraineandreiamcidl
 
Microservice Teams - How the cloud changes the way we work
Microservice Teams - How the cloud changes the way we workMicroservice Teams - How the cloud changes the way we work
Microservice Teams - How the cloud changes the way we work
Sven Peters
 
原版定制美国纽约州立大学奥尔巴尼分校毕业证学位证书原版一模一样
原版定制美国纽约州立大学奥尔巴尼分校毕业证学位证书原版一模一样原版定制美国纽约州立大学奥尔巴尼分校毕业证学位证书原版一模一样
原版定制美国纽约州立大学奥尔巴尼分校毕业证学位证书原版一模一样
mz5nrf0n
 
Hand Rolled Applicative User Validation Code Kata
Hand Rolled Applicative User ValidationCode KataHand Rolled Applicative User ValidationCode Kata
Hand Rolled Applicative User Validation Code Kata
Philip Schwarz
 
A Study of Variable-Role-based Feature Enrichment in Neural Models of Code
A Study of Variable-Role-based Feature Enrichment in Neural Models of CodeA Study of Variable-Role-based Feature Enrichment in Neural Models of Code
A Study of Variable-Role-based Feature Enrichment in Neural Models of Code
Aftab Hussain
 
Oracle Database 19c New Features for DBAs and Developers.pptx
Oracle Database 19c New Features for DBAs and Developers.pptxOracle Database 19c New Features for DBAs and Developers.pptx
Oracle Database 19c New Features for DBAs and Developers.pptx
Remote DBA Services
 
Energy consumption of Database Management - Florina Jonuzi
Energy consumption of Database Management - Florina JonuziEnergy consumption of Database Management - Florina Jonuzi
Energy consumption of Database Management - Florina Jonuzi
Green Software Development
 
Introducing Crescat - Event Management Software for Venues, Festivals and Eve...
Introducing Crescat - Event Management Software for Venues, Festivals and Eve...Introducing Crescat - Event Management Software for Venues, Festivals and Eve...
Introducing Crescat - Event Management Software for Venues, Festivals and Eve...
Crescat
 
Measures in SQL (SIGMOD 2024, Santiago, Chile)
Measures in SQL (SIGMOD 2024, Santiago, Chile)Measures in SQL (SIGMOD 2024, Santiago, Chile)
Measures in SQL (SIGMOD 2024, Santiago, Chile)
Julian Hyde
 
SWEBOK and Education at FUSE Okinawa 2024
SWEBOK and Education at FUSE Okinawa 2024SWEBOK and Education at FUSE Okinawa 2024
SWEBOK and Education at FUSE Okinawa 2024
Hironori Washizaki
 
KuberTENes Birthday Bash Guadalajara - Introducción a Argo CD
KuberTENes Birthday Bash Guadalajara - Introducción a Argo CDKuberTENes Birthday Bash Guadalajara - Introducción a Argo CD
KuberTENes Birthday Bash Guadalajara - Introducción a Argo CD
rodomar2
 

Recently uploaded (20)

ALGIT - Assembly Line for Green IT - Numbers, Data, Facts
ALGIT - Assembly Line for Green IT - Numbers, Data, FactsALGIT - Assembly Line for Green IT - Numbers, Data, Facts
ALGIT - Assembly Line for Green IT - Numbers, Data, Facts
 
E-commerce Development Services- Hornet Dynamics
E-commerce Development Services- Hornet DynamicsE-commerce Development Services- Hornet Dynamics
E-commerce Development Services- Hornet Dynamics
 
DDS-Security 1.2 - What's New? Stronger security for long-running systems
DDS-Security 1.2 - What's New? Stronger security for long-running systemsDDS-Security 1.2 - What's New? Stronger security for long-running systems
DDS-Security 1.2 - What's New? Stronger security for long-running systems
 
What is Augmented Reality Image Tracking
What is Augmented Reality Image TrackingWhat is Augmented Reality Image Tracking
What is Augmented Reality Image Tracking
 
Neo4j - Product Vision and Knowledge Graphs - GraphSummit Paris
Neo4j - Product Vision and Knowledge Graphs - GraphSummit ParisNeo4j - Product Vision and Knowledge Graphs - GraphSummit Paris
Neo4j - Product Vision and Knowledge Graphs - GraphSummit Paris
 
Using Query Store in Azure PostgreSQL to Understand Query Performance
Using Query Store in Azure PostgreSQL to Understand Query PerformanceUsing Query Store in Azure PostgreSQL to Understand Query Performance
Using Query Store in Azure PostgreSQL to Understand Query Performance
 
GraphSummit Paris - The art of the possible with Graph Technology
GraphSummit Paris - The art of the possible with Graph TechnologyGraphSummit Paris - The art of the possible with Graph Technology
GraphSummit Paris - The art of the possible with Graph Technology
 
SMS API Integration in Saudi Arabia| Best SMS API Service
SMS API Integration in Saudi Arabia| Best SMS API ServiceSMS API Integration in Saudi Arabia| Best SMS API Service
SMS API Integration in Saudi Arabia| Best SMS API Service
 
Need for Speed: Removing speed bumps from your Symfony projects ⚡️
Need for Speed: Removing speed bumps from your Symfony projects ⚡️Need for Speed: Removing speed bumps from your Symfony projects ⚡️
Need for Speed: Removing speed bumps from your Symfony projects ⚡️
 
LORRAINE ANDREI_LEQUIGAN_HOW TO USE ZOOM
LORRAINE ANDREI_LEQUIGAN_HOW TO USE ZOOMLORRAINE ANDREI_LEQUIGAN_HOW TO USE ZOOM
LORRAINE ANDREI_LEQUIGAN_HOW TO USE ZOOM
 
Microservice Teams - How the cloud changes the way we work
Microservice Teams - How the cloud changes the way we workMicroservice Teams - How the cloud changes the way we work
Microservice Teams - How the cloud changes the way we work
 
原版定制美国纽约州立大学奥尔巴尼分校毕业证学位证书原版一模一样
原版定制美国纽约州立大学奥尔巴尼分校毕业证学位证书原版一模一样原版定制美国纽约州立大学奥尔巴尼分校毕业证学位证书原版一模一样
原版定制美国纽约州立大学奥尔巴尼分校毕业证学位证书原版一模一样
 
Hand Rolled Applicative User Validation Code Kata
Hand Rolled Applicative User ValidationCode KataHand Rolled Applicative User ValidationCode Kata
Hand Rolled Applicative User Validation Code Kata
 
A Study of Variable-Role-based Feature Enrichment in Neural Models of Code
A Study of Variable-Role-based Feature Enrichment in Neural Models of CodeA Study of Variable-Role-based Feature Enrichment in Neural Models of Code
A Study of Variable-Role-based Feature Enrichment in Neural Models of Code
 
Oracle Database 19c New Features for DBAs and Developers.pptx
Oracle Database 19c New Features for DBAs and Developers.pptxOracle Database 19c New Features for DBAs and Developers.pptx
Oracle Database 19c New Features for DBAs and Developers.pptx
 
Energy consumption of Database Management - Florina Jonuzi
Energy consumption of Database Management - Florina JonuziEnergy consumption of Database Management - Florina Jonuzi
Energy consumption of Database Management - Florina Jonuzi
 
Introducing Crescat - Event Management Software for Venues, Festivals and Eve...
Introducing Crescat - Event Management Software for Venues, Festivals and Eve...Introducing Crescat - Event Management Software for Venues, Festivals and Eve...
Introducing Crescat - Event Management Software for Venues, Festivals and Eve...
 
Measures in SQL (SIGMOD 2024, Santiago, Chile)
Measures in SQL (SIGMOD 2024, Santiago, Chile)Measures in SQL (SIGMOD 2024, Santiago, Chile)
Measures in SQL (SIGMOD 2024, Santiago, Chile)
 
SWEBOK and Education at FUSE Okinawa 2024
SWEBOK and Education at FUSE Okinawa 2024SWEBOK and Education at FUSE Okinawa 2024
SWEBOK and Education at FUSE Okinawa 2024
 
KuberTENes Birthday Bash Guadalajara - Introducción a Argo CD
KuberTENes Birthday Bash Guadalajara - Introducción a Argo CDKuberTENes Birthday Bash Guadalajara - Introducción a Argo CD
KuberTENes Birthday Bash Guadalajara - Introducción a Argo CD
 

Presentation

  • 1. Event processing with Node.js and RabbitMQ by Zlatko Duric
  • 2. Disclaimer (AKA about me) ● Experience – Node.js? Lots – NoSQL? Lots. – Messaging? Little ● Backend Developer at ThinxNet.com (http://thinxnet.com/#jobs)
  • 3. Connected cars? ● Hardware device ● Various sensors – lots of data ● “Cloud” processing
  • 4. Ready to serve. How? ● Event collection and processing ● Webapp, mobile app ● Backoffice and support apps, notification services etc. ● Reporting, third party software integration etc ● REST approach: – Lots of requests – Parsing of data is blocking – Avoid choking devices – Communication overhead – Error management
  • 5. What service? Microservice! ● Redis PUBSUB ● ZeroMQ, MQTT, AMQP ● RabbitMQ, ActiveMQ
  • 6. About RabbitMQ ● Messaging broker based on AMQP protocol ● Platform/language independent ● Extensible (Erlang) ● Persistent message queues ● Node.js drivers (node-amqp, rabbit.js...)
  • 8. Exchanges and topics ● Publisher and subscriber. Simple as that. ● Get message, ack or reject (nack) it. ● Subscribers listen on a queue ● Publishers talk to an exchange ● Exchange routes to queues based on topics
  • 10. Channels and connections ● 1 node process = 1 connection ● Many channels, share same connection
  • 11. PUBSUB ● Simple PUBSUB, as used elsewhere ● All listeners get the same message ● Suboptimal
  • 13. PUSH-PULL/WORKER ● Round-robin distribution ● Worker socket.ack() is explicit (rabbit.js driver) ● Prefetch sets the number of msgs received ● Problem: concurrent/sequential modification?
  • 15. Content-based routing ● Erlang extensions? Not standard. ● App-side: – Keep internal mini-queue – Do not ack a msg if it's not first on local queue – Hold internal messages based on internal “topic”, ie “user.<ID>.save” - hold msgs for that user – Item.update() instead of item.save() ● Gets nasty fast, should minimize these scenarios
  • 17. REST API with MQ ● REQUEST/REPLY for immediate responses ● Other patterns for direct access – REST API just forwards the message – replies will go by other route ● Pub/sub is easy!
  • 19. Error handling ● Uncaught exceptions are covered ● Retry logic is simplified – push incomplete messages to a 'fixer' queue, requeue failed jobs or when no resources ● Dedicated error handling
  • 21. Error handling ● Uncaught exceptions are covered ● Retry logic is simplified – push incomplete messages to a 'fixer' queue, requeue failed jobs or when no resources ● Dedicated error handling
  • 22. Testing ● No good known mocks, please help ● RabbitMQ always on on Jenkins machine ● Test handlers directly
  • 23. Buzzwords index ● Cloud ● Connected car ● Microservice ● Patterns :)
  • 24. Info sources ● https://www.rabbitmq.com/ ● http://www.squaremobius.net/rabbit.js/ ● http://www.manning.com/videla/
  • 25. Note to self ● Thank the audience for patience ● Invite audience for a discussion and questions