Having previously worked at both Millennial Media and AOL, Michael Conway brought his expertise to Bidtellect tasked with transforming the business to a self-service SaaS-based content distribution platform, enabling the company to grow 10-fold. Next DSS MIA Event - https://datascience.salon/miami/ During the 30-minute presentation, Michael will provide background information about Bidtellect and how data is an integral component of the business managing their premium native inventory across their supply ecosystem with over 5 billion native auctions per day. As Bidtellect embraces big data, Michael will share the challenges and successes he and his team have experienced along the way. In addition, Steve Sarsfield, Vertica Senior Product Marketing Manager, will be available to discuss how specific technologies (SQL, Python, R and embedded algorithms) can be combined in an MPP environment to achieve big data analytics success.

2. Founded in 2011, a Delray Beach, Florida company that helps other businesses develop and tell
their optimal story to share it with as many people possible and maximize sales through enabling
technologies in Content Distribution and Native Advertising.
Founded By Industry Veterans from AdTech and Media
Built from the ground up for Content Distribution and Native Advertising
Think of us as the Facebook of the Open Web !!!
Bidtellect Background

3. Tasked with transforming Bidtellect from a Managed service
business with limited focus on data to a data-focused self-service
SaaS-based content distribution platform - enabling the company
to grow 10-fold
The “Mike” Goal

5. Advertisers need to quantify, in real time, which ads are converting and which are not—and they must
constantly revise ads, delivery, and placement among surrounding content on the fly to improve results.
Big Data analytics and insights to empower people and machines to instantaneously make optimal
advertising decisions, helping brands achieve tangible results in their businesses.
Continual behavioral analysis of user behavior with learnings supporting the creation of behavioral
audience segmentation and retargeting pools for scale.
Self-service forecasting to support optimal campaign trafficking and near-real time adjustments to
campaign configuration optimizing performance and KPI objectives.
Understanding the
Need

6. Challenge(s)
Technology
● Complexity
● Fragmentation
● Scale
● Fraud
● Brand Safety
● Transparency
● Viewability
Creative
● Personalization
● Creative Optimization
● Content Creation
● Resources
● Catering Content
Distribution
● Targeting
● Campaign Measurement
● Content Measurement
● Insights
● Optimization
Content May Be King, But Context is Queen &
She Holds the Purse Strings

7. ● Increased number of supply partners
● Tremendous growth in daily data throughput
● Poor Data Strategy
● Poor Data Infrastructure
Challenge(s)
5BN 1MM 1BN
Transactions per Day Decisions per Second Possible Permutations
8BN 1.5MM 1.5BN
Transactions per Day Decisions per Second Possible Permutations

9. ● Put Data at the Center
○ Flipped roadmap on its head to focus on data, reach and optimization
● Fix the infrastructure
○ Replaced existing analytics and reporting infrastructure to mitigate current issues and support
projected growth
● Optimization
○ Change optimization approach to allow for multiple KPI
○ SaaS enablement
● Audience Targeting
○ Creation of internal Data Management Platform
■ Audience creation, management
■ Cross-device delivery
● Engagement Score 2.0
● Forecasting
○ SaaS enablement

12. Infrastructure
Situation
● nDSP Performance Issues within production, even for simple non-analytics tasks
○ Saving campaign changes
○ Logging in
○ Pulling reports
● Reporting data in multiple data stores
● Perception of instability
● Operational productivity hindered

13. Infrastructure
Root Cause
● Reporting data was in same database as nDSP production data (SQL & Greenplum)
● Generation, querying, and updating of these large data sets:
○ Blocking non-reporting/analytics queries
○ Ripple effect impacting the ability for platform to bid at optimal levels
■ Scheduled processes required by the platform that need to access timed out
● Performance and scalability of Greenplum was not sufficient to support our growth
● Performance would only degrade as we add users to the system.

14. Infrastructure
Solution
Replaced existing analytics and reporting infrastructure
● Conducted PoC and runoff against multiple vendors
● Selected Vertica
○ Estimated 2 months upon hardware installation to migrate all current analytics jobs and reporting to
Vertica.
■ Took less than a month
○ Met our query SLA times
■ Significantly reduced query times compared to SQL or Greenplum < 4ms on average vs 30s+
■ Provided ability to tune to achieve query result times under multiple query loads/multi
tenancy
○ Reduced the complexity of the data architecture
○ Enabled to deliver hourly granularity vs daily granularity

15. Vertica
Vertica essentials supporting our mission
Focus Real-Time Requirements
Storage and Data Management ● Column store enables sorting, compression and organization of data more
efficiently
● Ability to apply different compression and encoding algorithm varying across
data set
Queries ● No indexes and self-indexes data by sorting and encoding
Speed & Performance ● 50-1,000 times faster than legacy Greenplum instance
● Ability to process queries in parallel over multiple processors
● Linear scalability and high availability
Built in analytics ● Supports functions including pattern matching, approximate count distinct and
approximate count distinct synopsis
● Efficient design requires less coding

17. Bidtellect Optimization
● Our optimization technology, Intellibid™ leverages big data, predictive models and machine
learning to make the smartest buying decisions in real-time for Native Advertising and Content
Distribution campaigns.
● Our campaigns are getting smarter even as we speak!
● Ability to optimize against a single or multiple campaign KPIs

18. Optimization Breadth
Leverages Big Data and machine learning to make smarter buying decision in real-time for all Native
Advertising campaigns.
Algorithms make over 1MM decisions per second across the following parameters:
● Creative
● Creative Rendering
● Device
● Placement
● Recency
● Dynamic Bid Price
● Goal Type Target
● Ad Format Type
● User Frequency
● User Characteristics

19. Optimization
Objectives
Allows Advertisers to optimize their Native Advertising and Content Distribution campaigns against a
variety of objectives, including:
● CTR
● Conversions
● View Rate
● Bounce Rate
● Play Rate (video)
● Page Views per Visit
● Average Time on Site
● Engagement Score ** (Proprietary measurement and algorithm tool for post-click engagement
metrics.

20. SaaS Enablement
Developed and enabled agency self-service SaaS solution through Bidtellect nDSP allowing users to
fully control adjustments to optimization goals supporting their KPIs
Objectives
Measurement Goal

21. 30-days from impressions,
15-days from matched-conversions,
1-day from rollups/cache/optimization/…/672
Simple Approach -
(15 Day Easy CVR)
Fitted Functions in R

22. Simple Approach -
(15 Day Easy CVR)
Fitted Functions in R
15 Day attribution window for $30
click-eCPA cutoff
Click-eCPA = revenue/conversions

23. Audience Targeting
Distributes content to specific target audience through sophisticated audience targeting algorithm and
behavioral measurement.
Target across devices and products against multiple parameters, Contextual, Behavioral, Demographic
and Location
01 | PRODUCT TYPE
02 | DEVICE TYPE
03 | LOCATION/ ZIP CODE
04 | OPERATING SYSTEM
05 | AGE & GENDER
06 | DEMOGRAPHICS
07 | LANGUAGE & CURRENCY
08 | DAY PARTING
09 | FREQUENCY CAPPING
10 | BEHAVIORAL / RETARGETING
11 | FIRST / THIRD PARTY
12 | CATEGORY AND
KEYWORD CONTEXTUAL
13 | KPI SLIDER
14 | BROWSER
15 | SUPPLY SOURCE
16 | SUPPLY TIER
17 | IAS VERIFIED
FRAUD PROTECTION
18 | CONTEXTUALLY
BRAND SAFETY
19 | CARRIER TARGETING

25. Engagement Score is the ultimate measure of content marketing success, designed to capture and
analyze post-click consumer activity.
● Sessions
● Pageviews (content consumed)
● Bounce Rate
● Time on Site
Measured from the behavior of users once they land on the advertiser’s landing page.
The advanced formula is a linear combination of three logistic functions.
Behavioral Analytics
(Descriptive/Predictive)

26. Page Views Per Visit
We wish to construct a function so that when page
views per visit is 1, the score is low and when it reaches
2 the score is much higher. This behavior is
encapsulated in Equation
The contribution of page views per visit to
engagement score. It reaches a maximum as page
views per visit reaches > 3.
Behavioral Analytics
(Descriptive/Predictive)

27. The engagement score is a number ranging from 0 to 10 where 0 corresponds to the least customer
engagement and 10 represents the highest level of engagement. It is derived from four variables
measured from the behavior of users once they land on the advertiser’s site.
Advertisers can optimize their campaigns toward Engagement Score as a whole or the individual
post-click metrics.
Behavioral Analytics
(Descriptive/Predictive)

29. We are in the process of implementing a Self-service forecasting to support optimal campaign trafficking
and near-real time adjustments to campaign configuration optimizing performance and KPI objectives.
A forecasting tool where a user can enter all or most of the targeting, allowability, and capability options
and the forecasting tool produces a bid price vs (impressions, clicks, spend, viewable impressions,
plays, completes) graph.
It is critical to understand and plan a campaign for success before it starts. We often have campaigns
that begin and then we realize we don't have the right price/inventory to support the success of the
campaign.
The Forecasting tool allows our users to understand our landscape and how making changes to a
campaign affect the scale and will provide them will multiple alternative solutions for driving their
campaigns.
Forecasting Analytics
(Prescriptive)

30. Create graphs using fit functions to cut down on real data noise
● Derive the appropriate other metric from the combination of
win rate and what that metric needs applied to against
total available auctions
a. impressions (just win rate)
b. clicks (CTR and win rate)
c. viewable imps (viewability rate and win rate)
d. Spend (bid price * (impressions, clicks, plays,
completes–depending on selection of bid) and win
rate)
e. plays (play rate and win rate)
f. completes (completion rate and win rate)
● Filter feasible targeting, allowability and others
● Campaign product type
Forecasting Analytics
(Prescriptive) Bid Price Variable
CPM ● Impressions
● viewable impressions
● Clicks
● Spend $
● Plays (video)
● Completes (video)
CPC ● Clicks
● Spend $
vCPM ● Viewable Imps
● Spend $
CPP ● Plays (video)
● Spend $
CPCV ● Completes
● Spend $

32. Nothing new to anyone in this room
Data is an integral component of the my Bidtellect’s business, managing our premium native inventory across
their supply ecosystem with over 8 billion native auctions per day entering back into our systems
The ability to analyze and act on data is increasingly important to all businesses.
● Pace of change requires quick reaction to changing demands
● Increasingly more complex decisions required, but with faster action
● Greater and greater amounts of data
Data is The Key

33. ● Start with the ensuring that you have the infrastructure in place to support your objectives for scale and
performance
● Start analyzing the data
○ Condense large amounts of data into smaller pieces of information.
■ Use descriptive analytics ("the simplest class of analytics") to summarize what happened
○ Use a variety of statistical, modeling, data mining, and machine learning techniques to study recent and
historical data
■ Allowing staff to make predictions about the future
■ Forecast what “might” happen (probabilistic)
■ Should provide a sentiment score (positive, negative between +1 or -1)
● The emerging technology of prescriptive analytics goes beyond descriptive and predictive models
by recommending one or more courses of action -- and showing the likely outcome of each
decision.
How to Get There?

34. ● Provide greater insight and offer alternatives based on data learnings
○ Like predictive analytics prescriptive analytics provides the ability to prescribe a possible
action
○ Prescriptive model is able to predict the possible consequences based on different choice of action
○ Should also recommend the best course of action for any pre-specified outcome
Where to Start?