Virtual Meetup Agenda and RTB Stack Overview

Virtual Meetup – January 2015

Agenda
• Developer Spotlight – Nicolas Emiliani – RTB Dev Lead
• RTBkit Behind the Scenes
• More than a Bidder – The RTBkit Stack
• Open Q&A
Copyright © 2015 Datacratic Inc. All rights reserved

Getting an RTBKit installation running.
More info here : http://datacratic.com/site/blog/tales-rtbkit-adventure
Nicolas Emiliani, RTB dev. team Tech Leader at Motrixi.

A little about our traffic :
 We are a mobile centered DSP
 We are connected to Nexage, Rubicon, Mopub and more recently to ADX
 We take up to 40k qps in peak hours from US and Canada

Life on the Amazon (not the rain forest)
 Create a VPC (or you will regret it, I promise)
 Create all your production nodes inside that VPC
 Be smart about security : do not expose your nodes
unnecessarily to the internet.
 Use a VPN software (ie: OpenVPN) to access the
nodes safely inside your VPC.

Isolating the RTBKit stack (BR loop)
 Chose an HTTP reverse proxy (we use NGINX)
 Set up the proxy as a gateway to your RTBKit installation, we use elastic
IP s on the NGINX nodes and allow incoming traffic only for the used
HTTP ports. Elastic IPS are cool and they will let you do all sort of handy
things.
 You can use the proxy as a load balancer if you have more than one
node
 You can use the proxy to automatically reply with 204s to the exchanges
in case you need to work on your installation
 You can use the proxy to clone the traffic and send it to a staging or
development environment (we recently cracked how to do this with
NGINX)

Isolating the RTBKit stack (BR loop)

Isolating the RTBKit PAL
 As with the BR loop use a reverse proxy with an Elastic IP

Type of nodes
 Different responsibilities → Different nodes → Easier scaling

Kernel parameters
 fs.file-max=300000 : RTBKit likes to chew up your file descriptors.
 net.ipv4.tcp_tw_reuse=1 : allow reusing sockets in TIME_WAIT state for
new connections.
 net.ipv4.tcp_fin_timeout=15 Close connections in the TCP FIN timeout
state in 15 seconds ( default 60 )
Tweaking this parameters will prevent crashes due to exceptions related to
too many open file descriptors.

Use the APIs, Luke
 Banker API : comes in handy when implementing pacing strategies thus
keeping logic simple in agents.
 Agent Configuration Service API : is useful to implement agent
configuration related APIs as the Agent Gateway
(https://github.com/nemiliani/agent_gateway).

Things that will help you out
 All our processes run under a watchdog, currently we are using monit
 Zabbix is your friend, particularly when things go wrong.
 Don't ever forget about Graphite. RTBKit is a huge black box otherwise.
Always keep an eye on your metrics. Yous should be doing it right now!,
no, really!

What’s taking so long?
● Ancient Node.js version in platform-deps
o Upgrading requires fixing the bindings
o Lots of work, very little pay-off

Alternative?
● Transition to HTTP interfaces
o Support for multiple languages
o Cleaner well defined OpenRTB protocol
o Start of RTBkit 2.0 implementation

How will it work?
● Abstracted agent communication
o HTTP Bidder Interface class
o Agent config specifies bidder URL

Details
● Added two new HTTP bidder examples:
o Python bidder based on tornado
o Node.js bidder
● Detailed walkthrough on the wiki

Backwards compatibility?
● Node.js support will be dropped for 14.04
● ZMQ C++ interface will remain for now.
➔ Will require an upgrade.
➔ Should be fairly painless.

Platform-deps
● Fork of platform-deps for RTBkit
o Upgrade dependencies to 14.04
o Cleanup of pointless dependencies

How long?
● Platform-deps fork is almost completed
o Should be released in a week or two
o Work done by Michael
● Currently re-writing our agents in golang
o Will iron-out kinks in the bidder interface
o Should be done in a month or two

More Than a Bidder
A Brief Introduction to an RTB Stack

But there is more to it than that ...
Campaign
Management
UI
Campaign
Reporting UI
ETL / Log
Processing
Offline
Optimization /
Bid Strategy
Logs / Log
Management
Ad Server
Online
Audience
Data Store
Offline Data
Prep
Monitoring Alerting 24/7 Ops
Conversion
Collection
Data
Collection /
DMP

You need an RTB Solution
Requirement Provided by RTBkit?
Reliable, high-performance core bidder system Yes
Online Impression / Click / Conversion matching Yes
Pacing No
Logging Customer has to set up and manage logs
Monitoring and Alerting Some support for Carbon and Graphite
24-7 operations and support No
Capacity planning and system evolution No
Campaign management UI integrated with the bidder No
Seats on exchanges / integration with exchanges No
ETL for campaign reporting and bidding agents No

Other Issues …
Keep it Simple
● There is an abundance of inventory. Your goal is to process enough bids to
achieve your goals. Over-engineering is expensive.
● Unreliability is probably OK
o Campaigns run for a long time and buy millions of impressions. Short outages
probably don't matter
o Most partners fine with noticeable error rates (~2%)

Other Issues …
Pacing and Strategy / Optimization
● Hard to do in real time, at high volume of transactions, in a distributed
system
● Hard to understand what data and how much data to use
● Risky
o blown daily caps
o uneven or poor performance

Other Issues … Ad Ops
Existing Ad Ops
● Used to feature set of DSP UI
● Comfortable delivering campaigns with current tool set
● May be resistant to using a new system
● May be resistant to trusting a new system
● You may spend a lot of time on UI features and reports, not on your real-
time platform

Host an RTBkit Meetup!

Q & A

Virtual Meetup Agenda and RTB Stack Overview

Recommended

Recommended

More Related Content

Similar to Virtual Meetup Agenda and RTB Stack Overview

Similar to Virtual Meetup Agenda and RTB Stack Overview (20)

Recently uploaded

Recently uploaded (20)

Virtual Meetup Agenda and RTB Stack Overview