Our services team creates, builds, and maintains the as a service offering for base platform services within our organization. Several thousand applications use these custom services daily generating more than 700 million requests per minute. One of these services was our publish / subscriber offering, BQ with custom SDK and custom metrics based on Apache Pulsar. BQ is the core communication service within our organization, having more 200M RPM. All the core processes of the organization depend on this service for operation: the CDC of any of our RDBMS or NoSQL offering, all the eventing efforts of the organization, async communication between apps, notification systems, etc. The backend of the solution was Apache Pulsar running on EC2 on AWS and on top of that we built several components as wrappers of the actual backend, creating our own SDKs and abstractions and in many ways extending the features provided by Pulsar. We had a multi-cluster setup 100% on AWS, with custom Pulsar Docker images running on large ASG setups, along with our own wrapping and admin APIs and DBs. All of this in turn transformed the solution into a volatile solution.

1. Pulsar Summit
San Francisco
Hotel Nikko
August 18 2022
Keynote
Pulsar @ Scale. 200M
RPM and 1K instances
Ignacio Alvarez
Sr Software Development Expert • Mercado Libre

2. Ignacio Leads messaging services team
in MercadoLibre, orchestrating 25k
microservices and 100K+ virtual
machines. Before MercadoLibre, he had
a 20 year career in software
development, 13 years of OOP and
Architecture Teaching. He is a Java,
Linux and Open Source fan and
practitioner. Scale and performance at
top of mind. Also CTO for
tupaseapp.com
Ignacio Alvarez
Sr Soft Development Expert
Mercado Libre

3. Pulsar @ Scale
200M RPM and 1k+ instances
MeLi’s Scale

4. 12K+
34K+
250M+
18
75M+
Meli’s Scale
What is MercadoLibre
Largest e-commerce, fintech & logistics
company in LATAM
75M+ Active Users
18 Countries
250M+ items sold per quarter
34k+ Employees
12k Engineers

5. Meli’s Scale
Large Scale. Big & fun tech challenges.

6. Pulsar @ Scale
200M RPM and 1k+ instances
Abstractions

7. Abstractions
Mary
Let’s talk about Mary. She is a Mercado
Libre backend developer and wants to
solve a MercadoPago’s issue

8. Abstractions
Platform
PLATFORM
Capability A
Capability B
Capability C
Capability D
Pulsar @ Scale

9. Abstractions
Introducing Fury
Ops
Services
Admin &
Support
Hot links
Related tools
A lot of unused space 😱
Traffic
Dependencies

10. Abstractions
Features
On top of our PAAS offering, we build
SAAS solutions, as abstraction layers
on top of backend services
These abstractions are key to give
us flexibility

11. Pulsar @ Scale
200M RPM and 1k+ instances
Why Pulsar?

12. Our company
grows 2x YOY
A transparent
design was key
to the choice
Single digit is
a WOW thing
Our core
business data
depends on this
Scalability Open Source Latency Consistency
Why Pulsar?
Requirements

13. Why Pulsar?
Scaling out our ecosystem
Default*
Shipping*
MP*
Notifications
ApiCore*
Fulfillment
Services
Fraud
SINGLE-CLOUD /
SINGLE-REGION
FAILOVER + BACKUP
> 30 Productive Pulsar Clusters
MULTI-AZ

14. Pulsar @ Scale
200M RPM and 1k+ instances
How Pulsar?

15. How Pulsar
How does the abstraction work

16. How Pulsar
How a cluster looks like for us

17. How Pulsar
Cluster Layout

18. How Pulsar
Failover design

19. How Pulsar
Team design

20. How Pulsar
Lots of operational tooling
Monitoring
Failover
Topic migration
Multi-architecture build to be able to run
on graviton

21. Pulsar @ Scale
200M RPM and 1k+ instances
How did it go?

22. 85M
80M+
35M
Pulsar and us
A whole event-driven ecosystem
35M Produced MPM
85M Delivered MPM
80M Filtered MPM

23. Pulsar and us
Pulsar’s WOW things
Latency
Horizontal Scalability (@Bookies)
Time to delivery

24. Pulsar and us
Major issues and design flaws
Namespaces / Fixed partitioning
Updates
New features (i.e. tiered storage)
High cardinality / High filtering
Funny use cases

25. Pulsar and us
Operational anecdotes

26. Pulsar and us
Major anti-patterns

27. Pulsar @ Scale
200M RPM and 1k+ instances
Key learnings & takeaways

28. Key Learnings & Takeaways
What the abstraction enabled us to do

29. Key Learnings & Takeaways
Most relevant stuff

30. Key Learnings & Takeaways
What’s Pulsar’s game?

31. Ignacio Alvarez
Thank you!
ignacio.alvarez@mercadolibre.com
@AlvarezDomec
linkedin.com/in/ignacioalvarez79
Pulsar Summit
San Francisco
Hotel Nikko
August 18 2022