Redis as a Message Bus - Brian Leonard, TaskRabbit

38 views

Published on

The secret to building scalable apps seems to be loosely-coupled
components. Whether it be service-oriented architecture, microservices, or just good old-fashioned namespacing, code minding it’s own business is a good idea. But often, there are still dependencies. Many have found that publishing and subscribing via a message bus to be an elegant solution to keep things under control. In this talk, I discuss how we
transformed the Redis-based background libraries we were already using into a message bus. This allowed our code to remain focused while still allowing asynchronous inter-component communication.

Published in: Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
38
On SlideShare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
2
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide
  • Hi, my name is Brian Leonard.
    I’m the chief architect at TaskRabbit.
    We love Redis and use it for lots of stuff. For example, we are using Resque to do all of our background processing.
    We’ve also gone one step further to use these tools to create an asynchronous message bus system that I’m going to talk about today.
  • Chief Architect & Technical Co founder

    TaskRabbit is…

    And people make a ton of money.
  • So who has used Redis? Ha.

    So I probably don’t have have to tell you guys about Redis, but for our purposes the main point is that we can store lists and the operations are atomic.
  • Resque is a background queue built on top of Redis. It uses arrays and the atomic properties of Redis to get this done.

    The ability to put things in order and then atomically pop an item from the list is all that is really needed for a great background queue and Redis fits perfectly.

    There is plenty of code, but it all comes down to inserting json into the queue, popping, and executing code with that as an input.
  • The pattern goes something like this: something happens on the main (request) thread. It queues up a worker in the background with a class name and arguments. In this case, we enqueue the CalculateAverageWorker with the new rating id. The background worker then does whatever it needs to do. In this case, we update the TaskRabbit’s average rating on her profile.
  • Looking at the Redis data workflow, it looks like this.
  • As we moved our code to a more service-oriented architecture, we wanted to move profiles to a new Rails app. Most of this app was concerned with uploading media and content and other ways for our TaskRabbits to market themselves.

    This kind of architecture leads to better code. Focused code is good code.
    A very minor point is this app is this number to display on a profile. It’s not the core of the situation, but it still needs to know it happened. How can we make that happen.

    So we could do the same thing.
  • Samething…. The rating is saved
  • We put it redis
  • Except this other app over here has to know to pop it
  • Now… is that good. Maybe. But it’s certainlynot the best.
    The failure here is that we separated out the code, but they are still tightly coupled.

    The Task App needs to know to enqueue something for the profile app. This is not good.

    Good code is focused code. So the ideal would be to do this communication without the Task App even knowing the Profile App exists and the Profile app not knowing exactly how and where the calculation need came from – just that it needs to happen.
  • So this code is almost exactly the same but represents the ideal concept a little better.
    The Task App publishes that a rating was created --- just in case anyone cares.

    The Profile app can declare that he cares and then execute the code to save the updated rating.
  • But how can we make this happen?
  • What we do is this thing called a bus driver.
    I can’t resist a metaphor. We’ll talk about the riders later.
  • So again, the rating is saved and it moves into a queue.
  • This time, though, its a new generic queue called incoming.
  • The bus driver knows to
  • So now a copy of the event is in profile queue as well as possible other ones from other subscribers
  • So the profile app does his thing
  • And this other one might be left in there. Who might want that.
  • Maybe there is some sort of fraud detection app that is looking for reviews that suggest the client thought they were cheated in some way.
  • So now that app can pop it and do whatever he needs to do.
  • And we’re done. So why is this a good thing?
  • The thing to note here is that it’s all still Resque workers; which, in turn, is all just moving things around in Redis. This allows us leverage our current infrastructure but with new-found capabilities. We derived a lot of value out of that. We know how to monitor it and troubleshoot it.
  • As I mentioned, good code is focused code. This pattern makes the apps more focused.
    Now the Task app minds his business and the Profile app reacts. Neither worries too much about the other. They do one thing and do it well.
    The main downside is some global complexity. There are a lots of events flying around and we have to learn how to teach new people about them and stuff like that.
    We’ve decided that it’s worth it.
  • So what are some examples to use this for.

    This is what we’ve been talking about most of the time here. You can build your service by focusing on the code at hand.
    The Task marketplace can take care of itself and the profile can listen.
    Your user management system can be it’s own thing.

    This pattern and use of Redis allows these to all be their own apps using their own technologies. We certainly went down that road.
    Or they can all be in the same app/repo and just be a better way to namespace. We eventually brought our services back together in this way (and that’s a whole other story) but the bus still added a ton of value.
  • The bus enables a whole new class of apps that are just on the bus and have no UI. An example of this is a communications app.

    We have an app called Switchboard that has subscriptions to any events that mention they want to send an email, sms, or push messages.
    This means you can have many subsytems of code but none of them need to know exactly how to do these thins. They just publish to the bus and, like magic, the communication occurs.
  • Another big win is to not muddy up your main business logic with external stuff like metrics tracking.
    If you need to tell your email marketing provider about all signups and email changes, code dedicated to that integration can subscribe and make those communications. Your account system doesn’t have to even know it exists.

  • A use case that we’ve found really helpful is an having one piece of code that subscribes to every single event. It takes that and writes it into elasticsearch.
    All of a sudden you have a complete queryable log of everything that has happened in your ecosystem.
    You could then use that store for debugging or creating dashboards or just to make the data science team really happy.
  • Ok, I’ve convinced you. You want to get on the bus. There are a few options.
  • Look at all those arrows!

    Mic drop?
  • If you just waned to get started, try out our library, Queue Bus. This is workflow I’ve described.

  • If you want to be a hipster, try Kafka from LinkedIn.
  • If you want to be enterprisey, try something with an MQ in it like RabbitMQ.
  • Or maybe you like to do things yourself. Queue Bus is not complicated. At the bottom of it all is just a few redis functions
  • So remember this guy?
  • To make this work successfully, the main idea is to Always be Publishing.
    We publish on just about every row insert, update, or delete. If that seems a little nuts, you should at least publish on major state changes of your objects. Like creations or a task goes from the scheduled state to the canceled state or something like that.

    In that same vein, you should make it easy to publish and have common code to do so. That makes it so that the events all follow a predictable pattern. This makes it easier for the consumers.

    Finally, start small. Publish just to see if it works. Then maybe do some logging. Add a few subscriptions. See if you like it. I think you will.

    Thanks.
  • Redis as a Message Bus - Brian Leonard, TaskRabbit

    1. 1. Redis as a Message Bus Brian Leonard TaskRabbit 05/10/2016
    2. 2. Redis • Can store a list (queue) • Atomic operations
    3. 3. Resque
    4. 4. class Rating < ActiveRecord::Base after_commit :enqueue_worker def enqueue_worker Resque.enqueue(CalculateAverageWorker, self.id) end End class CalculateAverageWorker def self.perform(rating_id) rating = Rating.find(rating_id) total = Rating.where(rabbit_id: rating.rabbit_id).count sum = Rating.where(rabbit_id: rating.rabbit_id).sum(:value) profile = RabbitProfile.find_by(user_id: rating.rabbit_id) profile.ratings_total = total profile.rating_average = sum.to_f / (5*total.to_f) profile.save! end end
    5. 5. Task App Redis Rating Model Rating Calculation Rating Saved
    6. 6. Task App Redis Rating Model Rating Calculation Class name and args in queue
    7. 7. Task App Redis Rating Model Rating CalculationPops from queue and executes
    8. 8. Task App Redis Rating Model Rating Calculation
    9. 9. Profile App Rating Calculation Task App Redis Rating Model
    10. 10. Profile App Rating Calculation Task App Redis Rating Model Rating Saved
    11. 11. Profile App Rating Calculation Task App Redis Rating Model Class name and args in queue
    12. 12. Profile App Rating Calculation Task App Redis Rating Model Pops from queue and executes
    13. 13. Profile App Rating Calculation Task App Redis Rating Model
    14. 14. # Task App: model class Rating < ActiveRecord::Base after_commit :enqueue_worker def enqueue_worker QueueBus.publish("rating_created", self.attributes) end end # Profile App: initializer ResqueBus.dispatch("profile") do subscribe "rating_created" do |attributes| total = Rating.where(rabbit_id: attributes['rabbit_id‘]).count sum = Rating.where(rabbit_id: attributes['rabbit_id‘]).sum(:value) profile = RabbitProfile.find_by(user_id: attributes['rabbit_id‘]) profile.ratings_total = total profile.rating_average = sum.to_f / (5*total.to_f) profile.save! end end
    15. 15. Profile App Rating Calculation Task App Redis Rating Model
    16. 16. Profile App Rating Calculation Task App Redis Rating Model Bus Driver
    17. 17. Profile App Rating Calculation Task App Redis Rating Model Bus Driver Rating Saved
    18. 18. Profile App Rating Calculation Task App Redis Rating Model Bus Driver Event in incoming queue
    19. 19. Profile App Rating Calculation Task App Redis Rating Model Bus Driver Pop event and give to subscribers
    20. 20. Profile App Rating Calculation Task App Redis Rating Model Bus Driver Event in profile queue Event in other queue(s)
    21. 21. Profile App Rating Calculation Task App Redis Rating Model Bus Driver Pops from queue and executes Event in other queue(s)
    22. 22. Profile App Rating Calculation Task App Redis Rating Model Bus Driver Event in other queue(s)
    23. 23. Profile App Rating Calculation Task App Redis Rating Model Bus Driver Fraud App Rating Reaction Event in other queue(s)
    24. 24. Profile App Rating Calculation Task App Redis Rating Model Bus Driver Fraud App Rating Reaction Pops from queue and executes
    25. 25. Profile App Rating Calculation Task App Redis Rating Model Bus Driver Fraud App Rating Reaction
    26. 26. It’s just Resque
    27. 27. Loosely Coupled +1 Local Simplicity +1 Local Focus -1 Global Complexity
    28. 28. Applications Subsystems Loosely couple your app through the bus • Marketplace • Profile • Accounts • Mobile • Microservices … or not
    29. 29. Communications Subscribe to the bus to send messages • Centralized sending email, text messages, push notifications, etc. • No shared knowledge or credentials • Asynchronous API for all apps to use and just focus on the content.
    30. 30. External Vendors Subscribe to the bus to sync events to providers • Conversion Tracking • Email Marketing • Referral Service • Mobile downloads
    31. 31. Logging Subscribe to everything and write to Elasticsearch • Tracing events through system • Simple centralized logging • Realtime business dashboards and metrics
    32. 32. Bus Systems
    33. 33. Starter
    34. 34. Hipster
    35. 35. Enterprise
    36. 36. Redis • RPUSH: add to a queue • LPOP: remove from a queue • SADD: add to subscribed apps • HSET: list apps subscriptions • HGETALL: read subscriptions
    37. 37. Profile App Rating Calculation Task App Redis Rating Model Bus Driver Rating Saved Pops from queue and executes Pop event and give to subscribers
    38. 38. Profile App Rating Calculation Task App Redis Rating Model Bus Driver Rating Saved Pops from queue and executes Pop event and give to subscribers RPUSH LPOP, HGETALL, RPUSH LPOP
    39. 39. Best Practices • Always be publishing – All data changes, especially state changes • Set up patterns • Start small
    40. 40. Thanks! tech.taskrabbit.com taskrabbit.com/careers twitter.com/bleonard $20 off your first task! http://bit.ly/BLTASK

    ×