CleverDATA_HighLoadStrategy_presentation

Company Profile
Make your DMP process 100% of
incoming bid requests
Ad at the speed of light
Sergey Zhemzhitsky,
CTO, CleverDATA
January 30, 2015

cleverdata.ru | info@cleverdata.ru
International market
business development
since 2012
One of three leading IT companies in Russia
43 branches in Russia and abroad
+5500 employees
100K projects for 10K customers
Data management innovative
platform (Data Exchange Service)
Cloud Service
In-house development
Internet advertising solutions
Data Management Platforms
Customers Base Management
Web Analytics
Marketing automation
Big Data
Data Mining
Digital Intelligence
Operational Intelligence
Low Latency and NoSQL
Cloud Computing

Agenda
• RTB intro;
• Challenges;
• Choice difficulties;
• Results;
• Do’s and Don’ts.

RTB intro

Real Time Bidding (RTB)
publishers
AD NETWORK
AD NETWORK
AD NETWORK
AD NETWORK
AD NETWORK
AD NETWORK
advertisers

How RTB works
publishers
AD NETWORK
AD NETWORK
AD NETWORK
AD NETWORK
AD NETWORK
AD NETWORK
advertisers

Demand Side Platform (DSP)
advertiserspublishers
AD NETWORK
AD NETWORK
AD NETWORK
AD NETWORK
AD NETWORK
AD NETWORK
D
S
P

Supply Side Platform (SSP)
publishers
AD NETWORK
AD NETWORK
AD NETWORK
AD NETWORK
AD NETWORK
AD NETWORK
advertisers
S
S
P

TRACKING DATA
Data Management Platform (DMP)
publishers
COOKIE SYNCs
ACCESS LOGS
PARTNERS’ DATA
3rd PARTY DATA
CLICK STREAMS
advertisers
S
S
P
D
S
P
WHATEVER ELSE DATA

RTB challenges
visitors publishers
advertisers100ms 20 ms
10.000+ rps

TRACKING DATA
publishers
COOKIE SYNCs
ACCESS LOGS
PARTNER’S DATA
3rd PARTY DATA
CLICK STREAMS
advertisers
S
S
P
D
S
P
DMP in the RTB ecosystem
DMP

Wishes…

Choice criteria
• Linear scalability;
• Sharding “out of the box”;
• Distributed by design;
• Data redundancy & replication;
• Low latency.

Which NoSQL … again

Aerospike Cassandra Mongo Redis
Scalability
Sharding
Redundancy
Latency
SPoF
Maintainability
Monitoring tools
Features

Public results

nginx 1.2.x
ngx-http-redis
lua-nginx-module
lua-resty-mongol
ngx-aerospike
aerospike 2.x
redis 2.6.x
mongodb 2.4.x
wrk
iperf
nmon
• Intel® Core™ i7-920 Quad-Core
• 48 GB RAM
• 1 Gbit/s NICs (*)
• 20 * 106 msgs
• 512 bytes msg
Tools & Materials

• No logging;
• No plugins;
• CPU Affinity.
NGINX
WRK
Measuring Nginx

Nginx results. Base line
L., ms., 50% L., ms., 75% L., ms., 90% L., ms., 99% T., r/s
1.35 1.46 1.64 1.90 68851

Measuring Redis
NGINX
WRK
Node 1
Shard 1
Shard 2
Slave 3
Slave 4
Node 2
Shard 3
Shard 4
Slave 5
Slave 6
Node 3
Shard 5
Shard 6
Slave 2
Slave 1

Redis results
2.68 3.10 3.25 3.90 34769

Redis results

Measuring Mongo
NGINX :: MONGOS
WRK
Node 1
mongod :: cfg
mongod :: repl
mongod :: repl
mongod :: repl
Node 2
mongod :: cfg
mongod :: repl
mongod :: repl
mongod :: repl
Node 3
mongod :: cfg
mongod :: repl
mongod :: repl
mongod :: repl
Shard 1
Replica Set 1
Shard 2
Replica Set 2
Shard 3
Replica Set 3

Mongo results
6.70 8.22 10.22 15.46 14220

Mongo results

Node 1
Chunk 1
Replica 2
Node 2
Chunk 2
Replica 3
Node 3
Chunk 3
Replica 1
NGINX
WRK
Measuring Aerospike

8.93 14.99 26.83 106.48 3402
Aerospike results

Aerospike results

Wks L, 50% L, 75% L, 90% L, 99% T, r/s Ngx, CPU Ngx, p/s As, CPU As, p/s
4 8.22 10.22 15.46 97.3 3402 7 8 K 1 1 K
8 7.89 15.16 92.4 89.8 6028 10 13 K 1 2 K
12 4.76 8.74 103.1 121.6 10233 15 20 K 2 3 K
16 3.91 6.21 99.6 111.3 13178 22 26 K 3 6 K
24 2.13 2.87 4.68 76.81 25744 21 60 K 5 10 K
32 2.01 2.60 4.42 81.29 28925 25 70 K 6 11 K
64 2.54 3.66 112.4 118.1 26468 27 70 K 6 11 K
… more Aerospike results

nginx
+ blocking I/O
How to shoot yourself in the foot…

Libevent to the rescue (2nd attempt)
Node 1
Chunk 1
Replica 2
Node 2
Chunk 2
Replica 3
Node 3
Chunk 3
Replica 1
LIBEVENT :: LIBEVHTP
WRK

Finally…
2.64 3.09 3.27 3.95 35746

Finally…

Nginx Redis Mongo Aerospike, b Aerospike, nb
Latency, 50% 1.35 2.68 6.70 8.93 2.64
Latency, 75% 1.46 3.10 8.22 14.99 3.09
Latency, 90% 1.64 3.25 10.22 26.83 3.27
Latency, 99% 1.90 3.90 15.46 106.48 3.95
Throughput, m/s 68851 34769 14220 3402 35746
CPU, Http, % 29 20 70 7 33
Network, Http, p/s, K 71 71 37 8 71
CPU, Db, % - 6 25 1 6
Network, Http, p/s, K - 12 21 1 12
Unofficial results

Aerospike

Let’s start building…

• Do use non-blocking I/O;
• Do computations locally;
• Be lazy. Don’t do more than needed;
• Don’t afraid of using other’s good ideas;
• Don’t trust anyone.
Do’s & Don’ts

Thanks a lot for your questions!

+7 (495) 967-66-50
info@cleverdata.ru
cleverdata.ru
facebook.com/CleverData

CleverDATA_HighLoadStrategy_presentation

Recommended

Recommended

More Related Content

Similar to CleverDATA_HighLoadStrategy_presentation

Similar to CleverDATA_HighLoadStrategy_presentation (20)

More from CleverDATA

More from CleverDATA (20)

Recently uploaded

Recently uploaded (20)

CleverDATA_HighLoadStrategy_presentation