Data-Driven Reserve Prices for Social Advertising Auctions at LinkedIn
•
1 like•694 views
1) The document describes LinkedIn's use of data-driven reserve prices in their social advertising auctions to optimize revenue and auction health. 2) A scalable regression model is used to predict bidder valuations and derive individualized reserve prices, while also establishing campaign-level reserve prices based on percentiles. 3) Experiments show the new system increased demand, reduced prices in emerging markets, and modestly improved metrics like bids and revenue in developed markets while decreasing advertiser churn.
More Related Content
What's hot
What's hot (20)
Similar to Data-Driven Reserve Prices for Social Advertising Auctions at LinkedIn
Similar to Data-Driven Reserve Prices for Social Advertising Auctions at LinkedIn (20)
Recently uploaded
Recently uploaded (20)
Data-Driven Reserve Prices for Social Advertising Auctions at LinkedIn
- 1. Data-Driven Reserve Prices for Social Advertising Auctions at LinkedIn Tingting Cui Lijun Peng David Pardoe Kun Liu Deepak Kumar Deepak Agarwal Relevance @ LinkedIn KDD 2017
- 2. Introduction
- 3. LinkedIn Sponsored Content (SC) • LinkedIn news feeds consist of both organic updates and sponsored content (SC) • The number of SC LinkedIn can show to members is limited • Different positions have different desirability • Auctions: allocating positions
- 4. How Sponsored Content Auction Works Advertisers
- 5. How Sponsored Content Auction Works Advertisers Geo: US Title: SWE Skill: Java … QWB(1) QWB(2) QWB(3) … B1 B2 … R User Advertiser — Ad GSP Auction Targeting Serving
- 6. What & Why Reserve Price • What is reserve price • The minimum bid to enter auction • The minimum price to pay • Why reserve price • Protect valuable inventory and optimize revenue • Too high - advertisers are discouraged from participating in auctions, resulting in low sell-through rate and revenue • Too low - poor price support in lack of competition
- 7. Why Reserve Price for Sponsored Content • Goal: scalable data-driven reserve price system • Data-driven – Rate card based reserve prices were used when LinkedIn first launched SC, which does not reflect market dynamics now • Pricing support – Protect valuable LinkedIn inventory, especially in regional markets with low liquidity • Scalable - The scale of LinkedIn’s social advertising imposes significant challenges in designing an effective system to compute & serve reserve prices 500+M Members 200+M Monthly Active Members 100+M Daily Ad Requests
- 8. This Talk • A scalable regression model which predicts the distribution of bidders’ valuations to derive revenue maximizing reserve price at the user level • A novel mechanism that produces the segment-level reserve price considering the trade-off between our revenue and advertisers’ satisfaction • Field experiments from emerging and developed markets show that reserve prices improve revenue metrics and auction health
- 10. Two Stage Reserve Prices User-level Prices Campaign Target Quantile Campaign-level Prices
- 11. Reserve Price Optimization • Assumptions 1. Advertiser valuation distribution is known to LinkedIn and advertisers 2. Advertiser valuation distribution is log normal 3. Advertisers bid their true valuation • Click probability declines more dramatically by position • Advertisers have an incentive to bid their true valuation • The revenue optimizing reserve price 𝑟∗ (Myerson 1981) • 𝑟∗ = 1 − 𝐹(𝑟∗) /𝑓(𝑟∗) , where 𝐹 and 𝑓 are CDF/PDF of valuation distribution
- 12. Fitting Valuation Distribution • Fit valuation distribution for a user via linear regression • Fit log of valuations (𝑉) against users’ profile attributes (𝑋) via linear regression log 𝑉 = 𝑋 𝑇 𝛽 + 𝜀, 𝜀~𝑁(0, Σ). 𝑋: a user-by-attribute binary matrix indicating the absence/presence of profile attributes for a user. • Following the assumption that bids (𝐵) are asymptotically equal to valuations 𝛽 = 𝑋 𝑇 𝑋 + 𝜆𝐼 −1 𝑋 𝑇 log 𝐵 . • Fit separate regression models for different geographic markets to reflect different market dynamics
- 13. User-Level Reserve Prices • Run optimization at user level • Indivisible and mutually exclusive unit • Linear regression model to predict valuation distribution for each • Numerically solve 𝑟∗ = 1 − 𝐹(𝑟∗ ) /𝑓(𝑟∗ ) for each user with fitted 𝐹 and 𝑓 to find the optimal reserve price
- 14. Campaign-Level Reserve Prices • Serve at campaign level • Easy to communicate with advertisers • Regulate bidding behavior • Discourage cherry-picking • Campaign-level reserve price: quantile of member-level reserve prices • The reserve price for a campaign targeting a user segment 𝑆 : 𝑟𝑠 = sup 𝑟 > 0|Pr 𝑅 𝑆 ≤ 𝑟 ≤ 𝑝 , 0 < 𝑝 < 1 is the quantile of choice
- 15. Implementation
- 16. Engineering Implementation • Challenge – Scale of LinkedIn’s user base and ads business • Component - Offline Hadoop pipeline + online web service Offline Hadoop Pipeline Online Web Service • Read the latest member profile and ad auction logs • Fit the bidder valuation distributions & compute user-level reserve prices • Store the optimal reserve price for each user in Pinot, a realtime distributed OLAP datastore, which is used at LinkedIn to deliver scalable real time analytics with low latency • Ad server calls Pinot store to retrieve campaign level floor price at serving time • Campaigns with bid below the reserve price for the visiting member are removed from the auction • The remaining campaigns are charged by Max(second price cost, campaign level floor price)
- 17. Architecture of Reserve Price System Auc%on Log Linear Regression User Level Reserve Price User Dimension Aggregator Real-%me distributed OLAP data store Campaign Level Reserve Price Adver%ser Campaign Create/Update Requests Campaign Reserve Price Online Campaign ServiceOffline Data Pipeline User Profile
- 18. Results
- 19. Experiment in Emerging Markets • Emerging markets where sell-through-rates are relatively low • Compared against the legacy rate-card based approach • Results • Lower reserve prices: 20-60% drop depending on geographic market • Significant increase in demand: the percent of auctions with at least one participant increased by 30-60% • Positive revenue impact: the increased demand quickly made up for the lower price
- 20. Results from Emerging Markets Figure 1: Percentage of auctions with at least one participant in emerging markets, normalized so the starting value is 1.0.
- 21. Experiment in Developed Markets • Developed markets where sell-through-rates are relatively high • Report results from CPC campaigns targeting the US market only • Stratified sampling to balance advertiser’s type and remove outlier campaigns • Revenue-related metrics - direct revenue impact • +1.7% lift in median bid, +2.2% lift in median CPC for campaigns bidding above reserve prices • Advertiser-centric metrics – advertiser experience • +17% reduction in churn rate, mainly attributed to campaigns bidding at the reserve price, as they now tend to submit more realistic bids => more likely to win in auctions and stay active
- 22. Results from Developed Markets Campaign group Increase in median bid Increase in median revenue per click Bid at reserve price 36.0% 36.0% Bid above reserve price 1.7% 2.2% Campaign group Abandonment rate Churn rate New campaigns per advertiser Treatment 1.03 0.83 1.07 Control 1.0 1.0 1.0 Table 1: Changes in median bid and revenue per click, treatment v.s. control. Table 2: Advertiser-Centric Metrics, normalized so that the control group always have values of 1.0.
- 23. Future Work • Address Overestimation • Valuation is overestimated, as valuation below existing floors are not observed • Overestimation is more severe if auction is thinner • Current heuristic approach - Apply a discount factor depending on sell-through rates of different regional markets given the trade-off between revenue and efficiency • Future - Improve the estimation of bidders’ valuations
- 24. Thank you
- 25. Online Social Advertising • Distinct features of social advertising • Rich user profile – work experience, industry, skill, interests, education… • More effective targeting – users are usually required to log in, “I know what you did last night”
- 26. Generalized Second Price Auction • Auction Mechanism • Generalized first price (GFP) • Vickrey–Clarke–Groves (VCG) • Generalized second price (GSP) • GSP: widely used in industry & less susceptible to gaming • Ads are ranked by their quality-weighted bids • The price that an advertiser pays for a click is determined by the next highest bid (the minimum necessary to retain its position) • If there are fewer advertisers than slots, the last advertiser pays a reserve price 𝑟
- 27. Implementation Auction log Model Member database Member floorAdvertisers Offline Online Pinot data store
- 28. Results from Developed Markets
Editor's Notes
- The update stream of a user contains content generated by the user’s network connections, the groups the user belongs to, the companies the user follows, etc.
- Advertisers specify member segments of target Member profile attributes (geo, title, skill, …)
- Advertisers submit bids in their own choice of cost types Ads are rank ordered by quality-weighted bids Price is determined by the next highest bid and quality scores (generalized second price auction)
- When LinkedIn launched its social advertising product Sponsored Content (SC) in 2013, segment-specific reserve prices were introduced in the auction. This rate card determined reserve prices for different geographic regions with additional markups for other profile attributes such as seniority, title, and industry. The SC market dynamics then evolved significantly, and the rate-card-based reserve prices became obsolete. Also, the scale of LinkedIn’s social advertising imposes significant challenges in designing an effective system to compute and serve reserve prices. This motivated our development of a scalable data-driven approach to set reserve prices.
- We report on field experiments showing substantial increase in the sell-through rate from the emerging markets (e.g., Latin America), significantly higher bids and revenue per click (+36% lift for bids at floor and +2% lift for bids above floor) from the developed markets (e.g., United States), and great reduction in advertiser churn rates.
- (1) Necessary for Myerson’s reserve price to hold (2) We follow Ostrovsky and Schwarz’s assumption that the valuation distribution is log-normal. We observe for most campaigns, log-normal distributions are a reasonable fit (3) Click probability declines more dramatically by position since LinkedIn only allows one ad per page in the update stream. Advertisers bid their true valuation. The observed bid distribution is a good representation of advertiser valuation. The observed bid distribution for a user is a good representation of advertiser valuation, which we use directly to derive the optimal reserve price. The problem of optimizing reserve prices boils down to fitting the valuation distribution
- Modeling parameters such as the shrinkage parameters and the sampling rates 𝜆 is the shrinkage parameter of the Ridge regression Build a custom regression solver running on Hadoop clusters given the scale R-square values of different geographic models range between 0.7 and 0.85
- Easy to communicate: a social advertising campaign usually targets a segment consisting of thousands or millions of users, making it difficult to publish reserve prices for all targeted users 𝑅 𝑆 : a random variable that denotes the reserve price for a randomly selected user in segment 𝑆,
- More than 500 million users, each having up to several million profile features Pinot’s storage scalability and query optimization mechanism allows us to compute campaign level reserve prices very quickly (within 10 ms on average), ensuring a smooth advertiser experience.
- e.g., Asia and Latin America
- e.g., US and Canada Median revenue metrics: since the campaign bid distributions are heavy-tailed. Significant lift in bids and revenue per click Abandonment rate - measuring how many campaign creation attempts are abandoned out of all attempts; New campaigns created per advertiser, the average number of campaigns created by an advertiser in a given period; Churn rate, which measures how many campaigns become inactive out of all previously active campaigns. We notice that advertisers did NOT abandon the campaign creation process much more often (a 3% increase in the treatment group) despite the substantial increase in reserve prices. We consider the reduction in churn rate to be a great improvement in advertiser experience, especially for small and inexperienced advertisers.
- Cox model & probit model
- improve brand awareness and drive quality leads