Talk given at the PHP user group Frankfurt regarding asynchronous events in Symfony 2 and the RabbitMQ library. We demonstrate some ups and downs as well as best practices and lessons learned.
How to cook Rabbit on Production - Serhiy Nazarov | Ruby Meditation 28Ruby Meditation
Talk of Serhiy Nazarov, founder QPARD, system architect, at Ruby Meditation #28 Kyiv 26.10.2019
Next conference - http://www.rubymeditation.com/
This presentation is connected with presentation How to cook Rabbit on Production - Bohdan Parshentsev | Ruby Meditation 28
Why we need a Rabbit? Let's try to cook it tasty. And what else? - Live demo of several useful dishes. Be ready to use Telegram to have fun with us.
Announcements and conference materials https://www.fb.me/RubyMeditation
News https://twitter.com/RubyMeditation
Photos https://www.instagram.com/RubyMeditation
The stream of Ruby conferences (not just ours) https://t.me/RubyMeditation
A simple explanation of basic principles of Distributed Programming with NodeJS. The CAP Theorem is fully explained, with working code the you can try yourself!
Talk given at the PHP user group Frankfurt regarding asynchronous events in Symfony 2 and the RabbitMQ library. We demonstrate some ups and downs as well as best practices and lessons learned.
How to cook Rabbit on Production - Serhiy Nazarov | Ruby Meditation 28Ruby Meditation
Talk of Serhiy Nazarov, founder QPARD, system architect, at Ruby Meditation #28 Kyiv 26.10.2019
Next conference - http://www.rubymeditation.com/
This presentation is connected with presentation How to cook Rabbit on Production - Bohdan Parshentsev | Ruby Meditation 28
Why we need a Rabbit? Let's try to cook it tasty. And what else? - Live demo of several useful dishes. Be ready to use Telegram to have fun with us.
Announcements and conference materials https://www.fb.me/RubyMeditation
News https://twitter.com/RubyMeditation
Photos https://www.instagram.com/RubyMeditation
The stream of Ruby conferences (not just ours) https://t.me/RubyMeditation
A simple explanation of basic principles of Distributed Programming with NodeJS. The CAP Theorem is fully explained, with working code the you can try yourself!
Maximize information exchange in your enterprise with AMQPKenneth Peeples
Businesses need to efficiently exchange information inside the enterprise as well as with other enterprises. In order to reduce cost and enhance business agility, an open messaging standard is a necessity for interoperability and integration. Advanced Message Queueing Protocol (AMQP) is the open standard wire-level binary messaging protocol that describes how a message should be structured and sent across the network.
Join this webinar to learn more about:
-What AMQP is and it's applications.
-The features and benefits of AMQP.
-Why you should use AMQP in your enterprise.
-The differences between AMQP and other messaging standards, such as JMS.
-Topologies and architectures possible through the use of AMQP.
AWS Webcast - Live Streaming using Amazon CloudFront and Wowza Media ServerAmazon Web Services
Learn how you can easily configure a production quality live streaming setup using Amazon CloudFront and Wowza Media Server (available through AWS Marketplace). We will take you step by step through the process of using AWS CloudFormation to provision the AWS resources, using an encoder to capture the live stream and delivering the stream at scale to various devices using Amazon CloudFront.
What RabbitMQ Can Do For You (Nomad PHP May 2014)James Titcumb
RabbitMQ is a message broker – an application that allows communication between applications by way of a message queuing system. In this talk, we look at some of the basic concepts of RabbitMQ and how it can help effectively scale your applications.
These slides are for the lightning talk I gave at Nomad PHP on 22nd May 2014.
Asynchronous processing with PHP and Symfony2. Do it simpleKirill Chebunin
The presentation for Symfony Camp UA 2013 about asynchronous processing.
We will talk about a near real-time processing of a business logic which does not require synchronous actions.
How to do it fast.
How to do it simple.
And how to do not ask PHP to struggle through alone.
We will analyze our experience, bad decisions and will try to get Zen of simplicity.
Scaling Symfony2 apps with RabbitMQ - Symfony UK MeetupKacper Gunia
Slides from my talk at Symfony UK Meetup. London, 20 Aug 2014. http://twitter.com/cakper
Video: https://www.youtube.com/watch?v=cha92Og9M5A
More Domain-Driven Design related content at: https://domaincentric.net/
Introducing message queue system, and explain how message queue can be used for queuing tasks. This is especially useful for web application to perform tasks in an asynchronously manner.
Maximize information exchange in your enterprise with AMQPKenneth Peeples
Businesses need to efficiently exchange information inside the enterprise as well as with other enterprises. In order to reduce cost and enhance business agility, an open messaging standard is a necessity for interoperability and integration. Advanced Message Queueing Protocol (AMQP) is the open standard wire-level binary messaging protocol that describes how a message should be structured and sent across the network.
Join this webinar to learn more about:
-What AMQP is and it's applications.
-The features and benefits of AMQP.
-Why you should use AMQP in your enterprise.
-The differences between AMQP and other messaging standards, such as JMS.
-Topologies and architectures possible through the use of AMQP.
AWS Webcast - Live Streaming using Amazon CloudFront and Wowza Media ServerAmazon Web Services
Learn how you can easily configure a production quality live streaming setup using Amazon CloudFront and Wowza Media Server (available through AWS Marketplace). We will take you step by step through the process of using AWS CloudFormation to provision the AWS resources, using an encoder to capture the live stream and delivering the stream at scale to various devices using Amazon CloudFront.
What RabbitMQ Can Do For You (Nomad PHP May 2014)James Titcumb
RabbitMQ is a message broker – an application that allows communication between applications by way of a message queuing system. In this talk, we look at some of the basic concepts of RabbitMQ and how it can help effectively scale your applications.
These slides are for the lightning talk I gave at Nomad PHP on 22nd May 2014.
Asynchronous processing with PHP and Symfony2. Do it simpleKirill Chebunin
The presentation for Symfony Camp UA 2013 about asynchronous processing.
We will talk about a near real-time processing of a business logic which does not require synchronous actions.
How to do it fast.
How to do it simple.
And how to do not ask PHP to struggle through alone.
We will analyze our experience, bad decisions and will try to get Zen of simplicity.
Scaling Symfony2 apps with RabbitMQ - Symfony UK MeetupKacper Gunia
Slides from my talk at Symfony UK Meetup. London, 20 Aug 2014. http://twitter.com/cakper
Video: https://www.youtube.com/watch?v=cha92Og9M5A
More Domain-Driven Design related content at: https://domaincentric.net/
Introducing message queue system, and explain how message queue can be used for queuing tasks. This is especially useful for web application to perform tasks in an asynchronously manner.
In today’s data-driven world, messages play a vital role as they are heavily used to transfer data and communicate among various IT ecosystems. As these ecosystems become more business critical, people expect messages to process and respond in less than a second, regardless of the complexity and distance between ecosystems.
The concept of “Asynchronous Messaging” can be applied to fulfill this industry essential as it helps in different ways and means to communicate efficiently and efficaciously. This webinar will discuss
"Enterprise Messaging with RabbitMQ" by Luis Majano is a comprehensive presentation that explores the concept of enterprise messaging and demonstrates how RabbitMQ, a widely used message broker, can facilitate effective communication within complex software systems. This presentation aims to provide a deep understanding of messaging patterns, architectures, and the benefits of adopting RabbitMQ in enterprise-level applications.
The presentation begins by introducing the fundamental principles of enterprise messaging and its significance in modern software development. It emphasizes the need for scalable, reliable, and decoupled communication between components and systems to ensure seamless operation and flexibility.
Luis Majano, an expert in enterprise software development, then delves into the key features and capabilities of RabbitMQ. He explains how RabbitMQ enables the implementation of various messaging patterns such as publish/subscribe, request/reply, and message routing. The presentation highlights the versatility of RabbitMQ in handling different messaging scenarios and discusses its integration possibilities with different programming languages and frameworks.
Moreover, the presentation addresses crucial concepts related to RabbitMQ, including exchanges, queues, bindings, and message acknowledgment. It explains how these components work together to enable message routing, filtering, and delivery guarantees in complex distributed systems.
Luis Majano also demonstrates the practical implementation of RabbitMQ through code examples and architectural diagrams. He illustrates how to set up RabbitMQ clusters for high availability and scalability, and how to handle common messaging challenges such as message persistence, fault tolerance, and load balancing.
Throughout the presentation, Luis Majano shares best practices, tips, and strategies for designing and implementing robust messaging solutions using RabbitMQ. He discusses message serialization, error handling, monitoring, and performance optimization techniques to ensure optimal message throughput and system responsiveness.
What we've learned from running thousands of production RabbitMQ clusters - L...RabbitMQ Summit
Watch the full lecture on YouTube: https://www.youtube.com/watch?v=ez9kQEhHsnc&list=PLDUzG2yLXrU4Lz33ZzSdHyfqdHJ8Zum5A
Since 2012 CloudAMQP has been running dedicated and shared RabbitMQ clusters for customers around the world, in seven different clouds. In this talk Lovisa will address the most common misconception, misconfigurations and anti-patterns in RabbitMQ usage, and how they can be avoided. Lovisa will talk about how you can increase RabbitMQ reliability and performance and she will also mention common RabbitMQ use cases among CloudAMQP’s customers.
--
The first RabbitMQ Summit connected RabbitMQ users and developers from around the world in London on November 12, 2018. Learn what's happening in and around RabbitMQ, and how top companies utilize RabbitMQ to power their services.
https://www.rabbitmqsummit.com
RabbitMQ Summit was organized by:
- Erlang Solutions, offering world-leading RabbitMQ Consultancy, Support, Health Checks & Tuning solutions https://www.erlang-solutions.com/
- CloudAMQP, offering fully managed RabbitMQ clusters https://www.cloudamqp.com
RabbitMQ Summit 2018 was sponsored by the following companies.
Platinum sponsors:
Pivotal
LShift
Gold sponors:
Trifork
AWS
Silver sponsor:
Cogin Queue Explorer
Linkedin has multiple data-centers hosting tens of thousands of servers across them. A large percentage of these servers host our data infrastructure - our distributed data store called Espresso is sizeable amongst them. The fleet of servers contain various hardware components including, but not limited to, SSDs; and hardware has a tendency of failing from time to time. In case of hardware failures the servers need to undergo maintenance which can take a significant amount of time based on type of failure. This creates reduced capacity for that duration and throws an interesting problem of maintaining capacity in the face of multiple failures. This talk covers how LinkedIn uses Camunda wrapped around with several components to achieve hands-off capacity management via multiple workflows, with asynchronous pauses and synchronisation among them. It will also highlight how we achieved seamless integrations with various platforms and components within Linkedin's Infrastructure, and a few best practices that helped us achieve the final state.
WebCamp Ukraine 2016: Instant messenger with Python. Back-end developmentViach Kakovskyi
Lessons learnt by the development of Instant Messenger with Python 2 and Twisted.
About the project:
* 100 000+ connected users
* 100+ nodes
* REST API for integration
Expected audience: developers who are new to development of Instant Messenger or folks who develop a system from scratch.
All the stuff based on own production experience.
WebCamp 2016: Python. Вячеслав Каковский: Real-time мессенджер на Python. Осо...WebCamp
Доклад построен на опыте разработки платформы реал-тайм мессенджера с характеристиками:
* 100 000+ одновременно подключенных пользователей
* 100+ серверов
* REST API для ботов
Структура доклада:
* Зачем разрабатывать мессенджер?
* Актуальные протоколы обмена сообщениями
* Архитектурные подходы к разработке мессенджера
* Библиотеки и инструменты
* Проблемы и подводные камни
PHP At 5000 Requests Per Second: Hootsuite’s Scaling Storyvanphp
Bill Monkman, Lead Engineer at Hootsuite, presenting on how Hootsuite went from zero to hundreds of millions of requests per day with its PHP codebase, and how dealing with that growth has shaped its future direction. Tips, optimizations, and horror stories from a rapidly-scaling PHP startup.
Video: https://www.youtube.com/watch?v=TZGeBAIMPII
In 2015, I used to write extensions for Joomla, WordPress, phpBB3, etc and I ...Juraj Vysvader
In 2015, I used to write extensions for Joomla, WordPress, phpBB3, etc and I didn't get rich from it but it did have 63K downloads (powered possible tens of thousands of websites).
GraphSummit Paris - The art of the possible with Graph TechnologyNeo4j
Sudhir Hasbe, Chief Product Officer, Neo4j
Join us as we explore breakthrough innovations enabled by interconnected data and AI. Discover firsthand how organizations use relationships in data to uncover contextual insights and solve our most pressing challenges – from optimizing supply chains, detecting fraud, and improving customer experiences to accelerating drug discoveries.
Atelier - Innover avec l’IA Générative et les graphes de connaissancesNeo4j
Atelier - Innover avec l’IA Générative et les graphes de connaissances
Allez au-delà du battage médiatique autour de l’IA et découvrez des techniques pratiques pour utiliser l’IA de manière responsable à travers les données de votre organisation. Explorez comment utiliser les graphes de connaissances pour augmenter la précision, la transparence et la capacité d’explication dans les systèmes d’IA générative. Vous partirez avec une expérience pratique combinant les relations entre les données et les LLM pour apporter du contexte spécifique à votre domaine et améliorer votre raisonnement.
Amenez votre ordinateur portable et nous vous guiderons sur la mise en place de votre propre pile d’IA générative, en vous fournissant des exemples pratiques et codés pour démarrer en quelques minutes.
Enterprise Resource Planning System includes various modules that reduce any business's workload. Additionally, it organizes the workflows, which drives towards enhancing productivity. Here are a detailed explanation of the ERP modules. Going through the points will help you understand how the software is changing the work dynamics.
To know more details here: https://blogs.nyggs.com/nyggs/enterprise-resource-planning-erp-system-modules/
We describe the deployment and use of Globus Compute for remote computation. This content is aimed at researchers who wish to compute on remote resources using a unified programming interface, as well as system administrators who will deploy and operate Globus Compute services on their research computing infrastructure.
AI Pilot Review: The World’s First Virtual Assistant Marketing SuiteGoogle
AI Pilot Review: The World’s First Virtual Assistant Marketing Suite
👉👉 Click Here To Get More Info 👇👇
https://sumonreview.com/ai-pilot-review/
AI Pilot Review: Key Features
✅Deploy AI expert bots in Any Niche With Just A Click
✅With one keyword, generate complete funnels, websites, landing pages, and more.
✅More than 85 AI features are included in the AI pilot.
✅No setup or configuration; use your voice (like Siri) to do whatever you want.
✅You Can Use AI Pilot To Create your version of AI Pilot And Charge People For It…
✅ZERO Manual Work With AI Pilot. Never write, Design, Or Code Again.
✅ZERO Limits On Features Or Usages
✅Use Our AI-powered Traffic To Get Hundreds Of Customers
✅No Complicated Setup: Get Up And Running In 2 Minutes
✅99.99% Up-Time Guaranteed
✅30 Days Money-Back Guarantee
✅ZERO Upfront Cost
See My Other Reviews Article:
(1) TubeTrivia AI Review: https://sumonreview.com/tubetrivia-ai-review
(2) SocioWave Review: https://sumonreview.com/sociowave-review
(3) AI Partner & Profit Review: https://sumonreview.com/ai-partner-profit-review
(4) AI Ebook Suite Review: https://sumonreview.com/ai-ebook-suite-review
Innovating Inference - Remote Triggering of Large Language Models on HPC Clus...Globus
Large Language Models (LLMs) are currently the center of attention in the tech world, particularly for their potential to advance research. In this presentation, we'll explore a straightforward and effective method for quickly initiating inference runs on supercomputers using the vLLM tool with Globus Compute, specifically on the Polaris system at ALCF. We'll begin by briefly discussing the popularity and applications of LLMs in various fields. Following this, we will introduce the vLLM tool, and explain how it integrates with Globus Compute to efficiently manage LLM operations on Polaris. Attendees will learn the practical aspects of setting up and remotely triggering LLMs from local machines, focusing on ease of use and efficiency. This talk is ideal for researchers and practitioners looking to leverage the power of LLMs in their work, offering a clear guide to harnessing supercomputing resources for quick and effective LLM inference.
Large Language Models and the End of ProgrammingMatt Welsh
Talk by Matt Welsh at Craft Conference 2024 on the impact that Large Language Models will have on the future of software development. In this talk, I discuss the ways in which LLMs will impact the software industry, from replacing human software developers with AI, to replacing conventional software with models that perform reasoning, computation, and problem-solving.
Graspan: A Big Data System for Big Code AnalysisAftab Hussain
We built a disk-based parallel graph system, Graspan, that uses a novel edge-pair centric computation model to compute dynamic transitive closures on very large program graphs.
We implement context-sensitive pointer/alias and dataflow analyses on Graspan. An evaluation of these analyses on large codebases such as Linux shows that their Graspan implementations scale to millions of lines of code and are much simpler than their original implementations.
These analyses were used to augment the existing checkers; these augmented checkers found 132 new NULL pointer bugs and 1308 unnecessary NULL tests in Linux 4.4.0-rc5, PostgreSQL 8.3.9, and Apache httpd 2.2.18.
- Accepted in ASPLOS ‘17, Xi’an, China.
- Featured in the tutorial, Systemized Program Analyses: A Big Data Perspective on Static Analysis Scalability, ASPLOS ‘17.
- Invited for presentation at SoCal PLS ‘16.
- Invited for poster presentation at PLDI SRC ‘16.
OpenMetadata Community Meeting - 5th June 2024OpenMetadata
The OpenMetadata Community Meeting was held on June 5th, 2024. In this meeting, we discussed about the data quality capabilities that are integrated with the Incident Manager, providing a complete solution to handle your data observability needs. Watch the end-to-end demo of the data quality features.
* How to run your own data quality framework
* What is the performance impact of running data quality frameworks
* How to run the test cases in your own ETL pipelines
* How the Incident Manager is integrated
* Get notified with alerts when test cases fail
Watch the meeting recording here - https://www.youtube.com/watch?v=UbNOje0kf6E
In the ever-evolving landscape of technology, enterprise software development is undergoing a significant transformation. Traditional coding methods are being challenged by innovative no-code solutions, which promise to streamline and democratize the software development process.
This shift is particularly impactful for enterprises, which require robust, scalable, and efficient software to manage their operations. In this article, we will explore the various facets of enterprise software development with no-code solutions, examining their benefits, challenges, and the future potential they hold.
Climate Science Flows: Enabling Petabyte-Scale Climate Analysis with the Eart...Globus
The Earth System Grid Federation (ESGF) is a global network of data servers that archives and distributes the planet’s largest collection of Earth system model output for thousands of climate and environmental scientists worldwide. Many of these petabyte-scale data archives are located in proximity to large high-performance computing (HPC) or cloud computing resources, but the primary workflow for data users consists of transferring data, and applying computations on a different system. As a part of the ESGF 2.0 US project (funded by the United States Department of Energy Office of Science), we developed pre-defined data workflows, which can be run on-demand, capable of applying many data reduction and data analysis to the large ESGF data archives, transferring only the resultant analysis (ex. visualizations, smaller data files). In this talk, we will showcase a few of these workflows, highlighting how Globus Flows can be used for petabyte-scale climate analysis.
Understanding Nidhi Software Pricing: A Quick Guide 🌟
Choosing the right software is vital for Nidhi companies to streamline operations. Our latest presentation covers Nidhi software pricing, key factors, costs, and negotiation tips.
📊 What You’ll Learn:
Key factors influencing Nidhi software price
Understanding the true cost beyond the initial price
Tips for negotiating the best deal
Affordable and customizable pricing options with Vector Nidhi Software
🔗 Learn more at: www.vectornidhisoftware.com/software-for-nidhi-company/
#NidhiSoftwarePrice #NidhiSoftware #VectorNidhi
Code reviews are vital for ensuring good code quality. They serve as one of our last lines of defense against bugs and subpar code reaching production.
Yet, they often turn into annoying tasks riddled with frustration, hostility, unclear feedback and lack of standards. How can we improve this crucial process?
In this session we will cover:
- The Art of Effective Code Reviews
- Streamlining the Review Process
- Elevating Reviews with Automated Tools
By the end of this presentation, you'll have the knowledge on how to organize and improve your code review proces
AI Genie Review: World’s First Open AI WordPress Website CreatorGoogle
AI Genie Review: World’s First Open AI WordPress Website Creator
👉👉 Click Here To Get More Info 👇👇
https://sumonreview.com/ai-genie-review
AI Genie Review: Key Features
✅Creates Limitless Real-Time Unique Content, auto-publishing Posts, Pages & Images directly from Chat GPT & Open AI on WordPress in any Niche
✅First & Only Google Bard Approved Software That Publishes 100% Original, SEO Friendly Content using Open AI
✅Publish Automated Posts and Pages using AI Genie directly on Your website
✅50 DFY Websites Included Without Adding Any Images, Content Or Doing Anything Yourself
✅Integrated Chat GPT Bot gives Instant Answers on Your Website to Visitors
✅Just Enter the title, and your Content for Pages and Posts will be ready on your website
✅Automatically insert visually appealing images into posts based on keywords and titles.
✅Choose the temperature of the content and control its randomness.
✅Control the length of the content to be generated.
✅Never Worry About Paying Huge Money Monthly To Top Content Creation Platforms
✅100% Easy-to-Use, Newbie-Friendly Technology
✅30-Days Money-Back Guarantee
See My Other Reviews Article:
(1) TubeTrivia AI Review: https://sumonreview.com/tubetrivia-ai-review
(2) SocioWave Review: https://sumonreview.com/sociowave-review
(3) AI Partner & Profit Review: https://sumonreview.com/ai-partner-profit-review
(4) AI Ebook Suite Review: https://sumonreview.com/ai-ebook-suite-review
#AIGenieApp #AIGenieBonus #AIGenieBonuses #AIGenieDemo #AIGenieDownload #AIGenieLegit #AIGenieLiveDemo #AIGenieOTO #AIGeniePreview #AIGenieReview #AIGenieReviewandBonus #AIGenieScamorLegit #AIGenieSoftware #AIGenieUpgrades #AIGenieUpsells #HowDoesAlGenie #HowtoBuyAIGenie #HowtoMakeMoneywithAIGenie #MakeMoneyOnline #MakeMoneywithAIGenie
Need for Speed: Removing speed bumps from your Symfony projects ⚡️Łukasz Chruściel
No one wants their application to drag like a car stuck in the slow lane! Yet it’s all too common to encounter bumpy, pothole-filled solutions that slow the speed of any application. Symfony apps are not an exception.
In this talk, I will take you for a spin around the performance racetrack. We’ll explore common pitfalls - those hidden potholes on your application that can cause unexpected slowdowns. Learn how to spot these performance bumps early, and more importantly, how to navigate around them to keep your application running at top speed.
We will focus in particular on tuning your engine at the application level, making the right adjustments to ensure that your system responds like a well-oiled, high-performance race car.
2. I’m Povilas Balzaravičius
● PHP developer over 10 years
● Software Engineer at Uber
● Before: developer at Boozt.com (Estina)
● Organizer of Vilnius PHP UG
● Not professional speaker but professional
developer :-)
3. What will be this talk about?
● The Problem
● RabbitMQ fundamentals
● How Problem was solved
● How we use messaging in general
4.
5. What is Boozt.com?
● One of top clothes e-
store in Scandinavia
● Over 300 brands &
10000 styles online
● Over 700 categories
8. Many items in categories
Possibly products at the top of page are selling
better.
When customer navigates to category page,
how products must be ordered?
9. Many items in categories
Items can not be sorted:
● Manually
● Randomly (or how they are stored on db)
● By one of sorting options (price, arrival date,
items on sale, etc.)
11. What is Autosort?
● Splits page into segments by number of
items
● Each segment defined by rules
● Segments can be sorted and shuffled
● Collection of segments is called “Autosort”
● “Autosorts” are assigned to categories
12.
13. Content managers ♥ Autosort...
● Segment sizes shrinked from 200 to 4-20
products - more queries
● Implemented autosort inheritance for
categories - increased sorted categories
count 20-40 times
● Job time increased from 0.25-0.5 to 6-8
hours. Here was our problem :-)
14. Looking for bottlenecks
● Optimized few code parts - performance
increased ~10%
● Tried tune MySQL indexes - no significant
impact
15. Bottleneck found!
Saving products positions to database:
UPDATE `product_category` SET `position` = CASE
WHEN product_id = 123 THEN 1
WHEN product_id = 456 THEN 2
WHEN product_id = 789 THEN 3
...
END
WHERE category_id = 1001 AND product_id IN (123, 456, 789)
16. Getting rid of bottleneck
● Update performed on single category only
● Other categories can be sorted during
update
● Need parallelize sorting1
!
1
With RabbitMQ of course!
17.
18. RabbitMQ?
● Multi Protocol Messaging Server
● Open source
● Commercial support
● Messaging via AMQP (Advanced Message
Queuing Protocol)
● Supports many programming languages
19. Why we need messaging?
● Separates Job request from the Job
● Jobs can be performed on separate
processes and machines
● Workers can be added or removed easily
● Messaging is asynchronous
20. RabbitMQ Basics
● Producer - sends messages
● Consumer - Running process and waiting
for messages
● Queue - a buffer that stores messages
● Exchange - distributes messages
21. Simple queue example
1. Producer sends message
to named queue
2. Message is stored in
queue
3. Consumer takes message
from named queue
4. Message removed from
queue immediately
5. Consumer “consumes” the
message
22. Consumer becoming slow?
● Just run additional
consumer
● Round-robin
dispatching by
default
● Jobs are processed
in parallel!
23. What happens if consumer dies?
● Message is lost if worker dies
● The task should be delivered to another
worker
● RabbitMQ supports message ACKs
● Message is removed when consumer sends
ACK
● ACKs are disabled by default
24. What happens if server dies?
● Messages in queues will be lost
● RabbitMQ supports durable queues
● Queue needs to be re-created as durable if
was defined before
● Messages should be sent in “persistent”
mode
● RabbitMQ doesn't do fsync(2) for every
message - not 100% durable :-)
25. Fair dispatch
● If consumer C1 will get more complex jobs
than C2, C1 will be busier all the time.
● Set prefetch_count = 1 property to not to
give more than one message to a worker at
a time.
● Requires ACKing
26. Let’s think about logging
system
This will help us understand examples
28. RabbitMQ Exchanges
● RabbitMQ dogma - producer never sends
any messages directly to a queue
● Producer sends messages to an exchange
● A routing_key must be used with messages
● Exchange type describes behavior:
○ Append the message to one/few/all queues
○ Discard the message
○ Types: fanout, direct, topic, headers
29. Temporary Queues
● If no queue is bounded to exchange,
messages will be lost.
● Temporary queues are deleted after
consumer disconnects.
● Created by consumer when only new
messages should be retrieved.
● RabbitMQ supports temporary queues.
30. Listening all messages: Fanout
● Fanout exchange type -
send messages to every
connected queue
● Temporary queues
used
● Routing key ignored by
fanout
31. Listening messages subset: Direct
● Direct exchange type -
deliver msg to queues
with same routing key
● Routing key not ignored
anymore
● Multiple bindings can
be defined for same
queue
● Multiple queues can use
same binding key
32. Listening messages by Topic
● Topic exchange type - filter messages by
patterns
● Topic is a list of words separated.by.dots
● Subscribing topic patterns:
○ * (asterisk) substitute one word - cron.*.error
○ # (hash) substitute 0 or more words - cron.#
● Topic exchange can behave as fanout or
direct
34. ● Create and provide callback queue from
provider
● Response will be sent to callback queue
● If single callback queue is used per provider,
use correlation_id to match request with
response
● It is slow and blocking
Receiving results: RPC
38. Scaling Autosort
● Producer sends categories which needs to
be sorted
● Exchange type is direct
● Single named queue with ACK and fair
dispatch is used
● Consumers are calling CLI commands (can
be used for other tasks too)
39. Scaling achievements
BEFORE
● 6-8 hours
● Single machine
● Single table for all
stores
AFTER
● 2-3 hours
● Separate machines
● 10 separate stores!
~28 times faster!
40. RabbitMQ in Boozt.com
Services platform
& RabbitMQ
Boozt store
ECCO store
Other store
Warehouse
Accounting
Admin system
Customer
service
Messages DB Calls
MySQL&Redisclusters
Sendgrid
41. Messaging use case #1
Order completed on Store
1. Message sent to Services platform from
Store
2. Services sends message to:
a. Warehouse to mark products as reserved
b. Accounting to register an order
c. Sendgrid to send an email
42. Messaging use case #2
Order confirmed on Accounting system
1. Message sent to Services from Accounting
2. Services sends message to:
a. Store to mark order as confirmed
b. Sendgrid to send an email
43. RabbitMQ ♥ PHP = php-amqplib
● Created by RabbitMQ developer - Alvaro
Videla
● Install via composer videlalvaro/php-
amqplib or
https://github.com/videlalvaro/php-amqplib
44. Dark side of php-amqplib
<?php
// ...
$channel->exchange_declare(
'topic_logs', 'topic', false, false, false);
list($queue_name, ,) = $channel->queue_declare(
"", false, false, true, false);
// Any comments required?
46. Supervisor
● Supervisor is a client/server system that
allows its users to monitor and control a
number of processes on UNIX-like operating
systems.
● Observes if consumers are running and
reloads them if they died
48. Tips for messaging
● Think about using unified workers
● And use priority queues for important tasks
(available since v3.5.0)
● Workers as P2P clients
● Use correlation_id for messages
49. Tips for messaging
● Manage your workers with supervisor or
similar tool
● Set lifetime for workers (eg. 15 mins)
● Use separate exchanges for separate apps
50. Resources
● https://www.rabbitmq.com - official site and
great tutorial
● http://tryrabbitmq.com - RabbitMQ simulator
● https://github.com/rabbitmq/rabbitmq-
tutorials/ - Source code of examples
● http://supervisord.org - supervisor