We at MiQ bid for billions of impressions everyday. This comes at a cost, and a lot many technical challenges. A million+ requests per second with response time limited to less than 10ms, Reactive programming enabled us to keep the resource costs minimum; with highly scalable and efficient ecosystem. In this session we delve deeper into the multiple aspects of the challenges, alongside a brush with the architecture of the system that keeps delivering for us.

1. Augmentation of bids for
programmatic ad auctions @ real
time with the power of Reactive
Architecture
Sumit Srivastava
Engineering Lead - MiQ

2. 2
❑ Marketing intelligence and Analytics
partner to many of the world’s most
prominent brands and media agencies
American Express, Avis, Lenovo,
Unilever, Microsoft, GroupM, Publicis
and IPG.
❑ AiQ is our technology that provides
modular, API-based analytics services
to rapidly build data solutions for
successful real-time business outcomes.
MiQ - Activating Marketing Intelligence via AiQ
Data Scientists & Engineers
Data Scientists & Analysts
Solution Engineers & Traders

3. 3
Daily Scale @ MiQ
80 Billion Ad
Impressions
5000+
Strategies
10+TB
Data1000+
Campaigns
750
million
users
Insights
Data mutation &
copies
Security Scale
● #campaigns
● #no. of people
● varied experience
levels
Quality
Multiple
languages &
tools

4. 4
The system
35 million
IP range records that we store in
PostgreSQL, each consisting respective
Geo information
35+
Features (parameters) each bid request
is evaluated upon
100ms
Average lifetime of an ad request
including bidding, campaign
optimization and creative delivery
7+TB
Amount of compressed data generated
every day
90%
Gateway to the exchange for MiQ
campaigns
Marketing Intelligence
Source to the data powering the core of
data science at MiQ

5. 5
❑ 1.5 million requests per second
❑ Response time below 10ms
❑ IP to Geo resolution
The problem statement

6. 6
The solution

7. 7
The journey
Version One Version Two
Components ● In-house Netty based server
● Off-heap pagination driven custom
data structure
● Vert.x
● PostgreSQL
Advantages ● Customized to have extremely low
memory footprint
● Non-blocking
● Low maintenance
● Highly scalable
● Active community
Disadvantages ● High maintenance
● Blocking
● Not scalable
● Complex debug process
● Readability
● No community support
● Slightly higher memory footprint

8. 8
Stats
❑ Requests per server: (10 connections, 10 threads)
● 200k/sec with no lookup data
● 25+k/sec with all features loaded
❑ Average response time: 4ms (95%)
❑ PostgreSQL queries response: <=2ms (99%)

9. 9
Q&A

10. 10
Thank You