More Related Content
Similar to Big data@accordantmedia - oanyc summit
Similar to Big data@accordantmedia - oanyc summit (20)
More from Open Analytics (20)
Big data@accordantmedia - oanyc summit
- 1. ©Accordant Media 2013. All Rights Reserved
BigData@Accordant Media
June 17 2013
Balaji Rao
Director, Technology
- 2. ©Accordant Media 2013. All Rights Reserved. 2
Accordant Media is an independent
programmatic media buying specialist,
focused on digital media buying and
optimization.
- 3. ©Accordant Media 2013. All Rights Reserved. 3
Bidding Mechanics
Exchanges Bidders
Bid Request
Asset Information
Bid Response
Max Offer
40 Billion Events Daily | 200ms per Auction|70 TB a day
Winning Bid
Sent Out
Ad inventory from
500k+ domains sent in
- 4. ©Accordant Media 2013. All Rights Reserved. 4
Amassing Actionable Data
All Media-Driven Activity
(53 fields)
Ad Verification Data
(35 fields)
• User ID
• Date/Time
• Creative
• Size
• CPM
• Bid Info
• Est. Pricing
• Clear Pricing
• IP
• Geo
• OS
• Browser
• Language
• Inv. Source
• Site
• Currency
• Advertiser
• Campaign
• Recency
• Frequency
• Imp/Click/Con
v
• Fees
• Revenue
• Order ID
Bid-Stream (won & lost)
(40 fields)
• User ID
• Date/Time
• Size
• Est. Pricing
• IP
• Geo
• OS
• Browser
• Language
• Inv. Source
• Site
• Age
• Gender
• Timezone
• Within iFrame
• Segments
(30,000+)
• User ID
• Date/Time
• Campaign
• Creative
• Inv. Source
• Site
• Site category
• Geo
• In View?
• Time in view
• # Views
• Engaged?
• Engage Time
• Hovers?
• Hover Time
• X/Y coord. of
Hovers
• Alerts
Up to 30,000+ segments for each bid request
- 6. ©Accordant Media 2013. All Rights Reserved. 6
Hosted on leading Cloud Hosting Providers
Listening 40~70 TB a day, EMR 2TB a day(on an avg.)
31TB compressed as of May @ ~3 TB per month
Polyglot persistence
AWS- EMR (Hive/Hadoop), S3, EC2, etc., Redshift, NoSQL,
Columnar DB, SQLServer
Google- BigQuery, GCE, Google Storage
Anything new, cheap, fast ….
• S3, Google Storage (Compressed)
• Partitioned, Year/Month logs at 1 per day Year/Month/Day for logs > 1 per day
• ETL- Talend, Elastic Mapreduce Hadoop/HIVE
• Monthly Jobs- Forecasting for partners, Weekly- Audience Audits
• Daily- Reporting Summary Daily, Hourly- e.g. Reporting Summary, Segmentation
Technology at a glance
- 7. ©Accordant Media 2013. All Rights Reserved 7
Date Impressions Clicks Conversions Total Cost Client Revenue eCPM eCPC eCPA eRPA CTR Conv Rate
Broad Reach 18283318 5041 848 14,952.00 611.73 0.82 2.97 17.63 0.72 0.028% 0.005%
8/10/2010 800254 224 18 533.97 0 0.67 2.38 29.66 - 0.028% 0.002%
8/11/2010 2318422 642 140 1,680.68 0 0.72 2.62 12.00 - 0.028% 0.006%
8/12/2010 1945529 527 126 1,516.63 159.7 0.78 2.88 12.04 1.27 0.027% 0.006%
8/13/2010 2122654 596 68 1,701.33 85.36 0.80 2.85 25.02 1.26 0.028% 0.003%
8/14/2010 1893693 576 40 1,497.68 0 0.79 2.60 37.44 - 0.030% 0.002%
8/15/2010 1907657 612 67 1,518.79 0 0.80 2.48 22.67 - 0.032% 0.004%
8/16/2010 1899805 523 106 1,459.14 185.91 0.77 2.79 13.77 1.75 0.028% 0.006%
8/17/2010 1475451 344 82 1,342.23 79.85 0.91 3.90 16.37 0.97 0.023% 0.006%
8/18/2010 1770367 437 82 1,737.09 100.91 0.98 3.98 21.18 1.23 0.025% 0.005%
8/19/2010 2149486 560 119 1,964.45 0 0.91 3.51 16.51 - 0.026% 0.006%
Broad Reach Driven Remarketing 234269 84 609 409.09 271.34 1.75 4.87 0.67 0.45 0.036% 0.260%
8/10/2010 1711 1 0 1.46 0 0.85 1.46 #DIV/0! #DIV/0! 0.058% 0.000%
8/11/2010 6591 3 22 5.35 0 0.81 1.78 0.24 - 0.046% 0.334%
8/12/2010 10138 5 20 8.29 16.54 0.82 1.66 0.41 0.83 0.049% 0.197%
8/13/2010 12059 1 32 10.13 26.6 0.84 10.13 0.32 0.83 0.008% 0.265%
8/14/2010 11669 2 38 10.15 0 0.87 5.07 0.27 - 0.017% 0.326%
8/15/2010 15157 7 31 13.26 0 0.88 1.89 0.43 - 0.046% 0.205%
8/16/2010 24550 13 39 36.15 82.25 1.47 2.78 0.93 2.11 0.053% 0.159%
8/17/2010 46890 16 113 100.10 32.78 2.13 6.26 0.89 0.29 0.034% 0.241%
8/18/2010 44564 18 145 95.72 113.17 2.15 5.32 0.66 0.78 0.040% 0.325%
8/19/2010 60940 18 169 128.48 0 2.11 7.14 0.76 - 0.030% 0.277%
Standard Remarketing 2848262 1723 8012 5,347.48 5962.25 1.88 3.10 0.67 0.74 0.060% 0.281%
8/10/2010 127528 60 274 110.17 0 0.86 1.84 0.40 - 0.047% 0.215%
8/11/2010 183476 83 520 168.08 0 0.92 2.03 0.32 - 0.045% 0.283%
8/12/2010 171385 79 447 159.33 923.25 0.93 2.02 0.36 2.07 0.046% 0.261%
8/13/2010 147360 78 349 137.16 569.47 0.93 1.76 0.39 1.63 0.053% 0.237%
8/14/2010 121316 65 240 114.55 0 0.94 1.76 0.48 - 0.054% 0.198%
8/15/2010 137568 74 429 130.28 0 0.95 1.76 0.30 - 0.054% 0.312%
8/16/2010 259435 141 543 459.98 1546.54 1.77 3.26 0.85 2.85 0.054% 0.209%
8/17/2010 527275 358 1663 1,268.31 878.39 2.41 3.54 0.76 0.53 0.068% 0.315%
8/18/2010 546179 359 1793 1,328.21 2044.6 2.43 3.70 0.74 1.14 0.066% 0.328%
8/19/2010 626740 426 1754 1,471.42 0 2.35 3.45 0.84 - 0.068% 0.280%
Grand Total 21365849 6848 9469 20,708.57 6845.32 0.97 3.02 2.19 0.72 0.032% 0.044%
Client Reporting: 24/7 Access
Client Client
Client Client
Client Client
Client
Client
Client
Client
- 8. ©Accordant Media 2013. All Rights Reserved 8
Proprietary Forecasting Tool
Accordant’s
proprietary tool
can forecast given
highly dynamic
targeting
parameters
- 10. ©Accordant Media 2013. All Rights Reserved 10
BigData Algorithm (Re)Model
Campaign Parameters Campaign Past Performance
Bid Stream
Exchange
1st Party
2nd Party
3rd Party (With Data Provider Agreement)
Domains
Lookalikes
Past Performance
Campaign (re)model
Algorithm models tell the bidders to bid (and at what price) or not to bid.
Campaigns are continuously re-modeled for optimum performance
- 11. ©Accordant Media 2013. All Rights Reserved 11
BigData –Lessons Learned
• Pay as you go vs Huge upfront cost
• Don’t get locked down
• Choose technology wisely
• Keep tech babble away
• Try new technologies
• Some will fail
• Keep analysis on BigData simple
• Deep analytics fail
- 12. ©Accordant Media 2013. All Rights Reserved 12
Q & A
• Thank You!
• (…And we’re hiring)!
• info@accordantmedia.com
