Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Dynamic Pricing in Mobile Games

2,557 views

Published on

These are the slides from Bill Grosso's Keynote at the Conference on Artificial Intelligence in Creative Industries (nucl.ai).

The thesis of the talk is that "analytics" has gone from

-- "Reporting" which gives information about past events
-- to "Business Intelligence" which reframes the reporting in terms of business diagnostics (but is still about past events)
-- to "Predictive Analytics" which attempts to predict future events.
-- to ... frameworks that can actually alter system behavior in response to predicted outcomes.

Scientific Revenue is one of the first of the frameworks. It marries BI and Predictive Analytics to an adaptive system that can alter pricing in order to maximize lifetime revenue.

In this talk, which is somewhat technical, Bill discusses the design and architecture of adapative systems, using Scientific Revenue as an example.

Published in: Data & Analytics
  • I think you need a perfect and 100% unique academic essays papers have a look once this site i hope you will get valuable papers, ⇒ www.HelpWriting.net ⇐
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here

Dynamic Pricing in Mobile Games

  1. 1. © 2015. Company Confidential and Not for Redistribution. info@scientificrevenue.com 07/22/15 1 Dynamic Pricing In Mobile Games July 22, 2015
  2. 2. © 2015. Company Confidential and Not for Redistribution. info@scientificrevenue.com 07/22/15 2 About Me • CEO and Founder, Scientific Revenue • Previously CTO / SVP Product for Live Gamer (Payments Aggregator focused on F2P Gaming ) • Long history in data and artificial intelligence
  3. 3. © 2015. Company Confidential and Not for Redistribution. info@scientificrevenue.com 07/22/15 3 The Last AI Conference I Spoke At ….
  4. 4. © 2015. Company Confidential and Not for Redistribution. info@scientificrevenue.com 07/22/15 4 About Scientific Revenue Scientific Revenue provides a dynamic price management solution for mobile games that boosts in-app purchase revenue. We match the right prices with the right players at the right times, to keep players engaged, increase conversion, and grow profits for game publishers.
  5. 5. © 2015. Company Confidential and Not for Redistribution. info@scientificrevenue.com 07/22/15 5 What Heather Said
  6. 6. © 2015. Company Confidential and Not for Redistribution. info@scientificrevenue.com 07/22/15 6 Earlier Today …. • The heavy lifting going on around knowing what players are doing has to do with prediction and classification • Classic territory • Systems today are more strongly on detection and diagnosis, not action side • We’re starting to get solid predictive analytics • Scientific Revenue is about a control framework • Well, that and some pretty nice machine learning. • This talk is mainly about control frameworks.
  7. 7. © 2015. Company Confidential and Not for Redistribution. info@scientificrevenue.com 07/22/15 7 What is a Price
  8. 8. © 2015. Company Confidential and Not for Redistribution. info@scientificrevenue.com 07/22/15 8 An Offer To Sell a Good or Service for “Real Money”
  9. 9. © 2015. Company Confidential and Not for Redistribution. info@scientificrevenue.com 07/22/15 9 An Offer To Sell a Good or Service for “Real Money”
  10. 10. © 2015. Company Confidential and Not for Redistribution. info@scientificrevenue.com 07/22/15 10 Lots of Decisions Different: • Prices • Coin amounts • (volume discounts) • Framing text and cues • Default selection • Different bonus types Same: • Call to action
  11. 11. © 2015. Company Confidential and Not for Redistribution. info@scientificrevenue.com 07/22/15 11 These are Also “Pricing Decisions”
  12. 12. © 2015. Company Confidential and Not for Redistribution. info@scientificrevenue.com 07/22/15 12 Not Just “How Much for How Much” • Pricing decisions are also • What additional inducements do you offer? • When do you make the offer? • What channels you make the offer in? • What messages accompany the offer? • How long the user has to act on the offer? • ….
  13. 13. © 2015. Company Confidential and Not for Redistribution. info@scientificrevenue.com 07/22/15 13 The Problem
  14. 14. © 2015. Company Confidential and Not for Redistribution. info@scientificrevenue.com 07/22/15 14 Increase LTR (“R” = “Revenue”) • Pricing optimization is a tool to revenue maximization • Without causing adverse reactions • NOT Looking at things at the “individual transaction level” • For games with very high churn and very short retention times, these approaches overlap • But if you’re keeping your users around and hoping for more revenue later (second purchases, advertising, …) then there are other aspects to consider
  15. 15. © 2015. Company Confidential and Not for Redistribution. info@scientificrevenue.com 07/22/15 15 LTR • 20% of all purchases occurred on day 1 • All spending was done by day 40 • 27% of all first purchases occur on day 1 • 80% of all first purchases occur in week 1 • 49% of purchasers bought a second time • Half of all purchases occurred in week 1 • Second purchases were the same size as first purchases
  16. 16. © 2015. Company Confidential and Not for Redistribution. info@scientificrevenue.com 07/22/15 16 The Settled Science isn’t Very Useful • Classical Economics involves pricing to the demand curve (and, maybe, estimating demand curves using multi-armed bandits)
  17. 17. © 2015. Company Confidential and Not for Redistribution. info@scientificrevenue.com 07/22/15 17 Reality is Actually …. Kind of Unsettled • Training effects? • Framing Effects? • Volume discounts? • Churn Impacts? • Community Impact? • Moral Hazard?
  18. 18. © 2015. Company Confidential and Not for Redistribution. info@scientificrevenue.com 07/22/15 18 Moral Hazard This is absolutely disgusting. I'll be sure to tell everyone about this creepy, exploitive tracking of players.
  19. 19. © 2015. Company Confidential and Not for Redistribution. info@scientificrevenue.com 07/22/15 19 Our Predecessors Made Many Simplifying Assumptions …. • Non-negotiated pricing • Flexible return policy • Segmentable market demand • Highly competitive markets / little or no vendor loyalty • Publically available ratecards • Pre-existing anchoring on pricing and rates • Infrequent, large-dollar amount purchases • Customers return months or years later • Low variable costs • Fixed capacity • Inventory can be changed from one product to another • Perishable inventory
  20. 20. © 2015. Company Confidential and Not for Redistribution. info@scientificrevenue.com 07/22/15 20 The Architecture
  21. 21. © 2015. Company Confidential and Not for Redistribution. info@scientificrevenue.com 07/22/15 21 In Block Form
  22. 22. © 2015. Company Confidential and Not for Redistribution. info@scientificrevenue.com 07/22/15 22 Three Distinct Requirements • Data Collection • This isn’t an algorithm problem. It’s a modeling and feature problem and it requires a well designed data set informed by the machine learning goals • Control Framework • The point is to change prices. By itself, that’s actually pretty hard to do (in the “lot of code” sense) • Asynchronous Evaluation Framework • Most of our model building and training is done asynchronously
  23. 23. © 2015. Company Confidential and Not for Redistribution. info@scientificrevenue.com 07/22/15 23 Evaluation Framework: Global Cycle Calibrate Measure Diagnose Propose Promote Test
  24. 24. © 2015. Company Confidential and Not for Redistribution. info@scientificrevenue.com 07/22/15 24 Evaluation Framework: Calibration • Solving “Cold Start” • Have a canned set of 70+ segments (that are “known” to exhibit pricing and behavioral differences). • Have a predefined set of 250+ additional features • Have a diagnostic framework that can exhaustively measure a large number of metrics and which can evaluate features for predictive power
  25. 25. © 2015. Company Confidential and Not for Redistribution. info@scientificrevenue.com 07/22/15 25 Example: Part of Day / Day of Week • Left: Number of new users by date and hour, lighter = more • Right: Number of purchasers by date and hour, lighter = more • People who join at noon are 4 times more likely to spend than people who join at 5 AM (in this game)
  26. 26. © 2015. Company Confidential and Not for Redistribution. info@scientificrevenue.com 07/22/15 26 Evaluation Framework: Calibration • Run for three weeks to train the models against the initial segments and get baseline performance data • Compare the initial segments to each other to get an idea of variation and benchmark good performance • Spot “underperformers” and “overperformers” • Automated diagnostics to explore reasons why
  27. 27. © 2015. Company Confidential and Not for Redistribution. info@scientificrevenue.com 07/22/15 27 Directional Metrics • Traditional KPIs can indicate issues, but don’t help much with action • ARPU dropping? What is the automated, or partially automated, outcome? • Part of the power of our approach comes from putting features against finer-grained behavioral metrics • And then automating the sifting • Example: • Purchase Index • Default Acceptance Ratio • Upsell % • Downsell %
  28. 28. © 2015. Company Confidential and Not for Redistribution. info@scientificrevenue.com 07/22/15 28 Evaluation Framework: Segmentation • Predictive Analytics • Churn Prediction, Likely To Purchase, Potential Whale, … • Custom Models • Disposable Income, Affluence, Gamerness, Mobile Native Ness, … • “Possibly Important” Features • Lots of these • Propose Segments and Pricing Policies • Based on important features, create segments and compare
  29. 29. © 2015. Company Confidential and Not for Redistribution. info@scientificrevenue.com 07/22/15 29 Evaluation Framework: User Lifecycle (start) Join Initial Profile Baseline Modeling Baseline Prediction Observe Adjust Initial Pricing Set Reprice
  30. 30. © 2015. Company Confidential and Not for Redistribution. info@scientificrevenue.com 07/22/15 30 Dealing With Intuitive Ideas
  31. 31. © 2015. Company Confidential and Not for Redistribution. info@scientificrevenue.com 07/22/15 31 Intuitive Physics
  32. 32. © 2015. Company Confidential and Not for Redistribution. info@scientificrevenue.com 07/22/15 32 Intuitive Economics
  33. 33. © 2015. Company Confidential and Not for Redistribution. info@scientificrevenue.com 07/22/15 33 What is Stickershock A feeling of surprise and disappointment caused by learning that something you want to buy is very expensive. Astonishment and dismay experienced on being informed of a product’s unexpectedly high price.
  34. 34. © 2015. Company Confidential and Not for Redistribution. info@scientificrevenue.com 07/22/15 34 Formalizing Stickershock Before: • D0 to D3 timeframe • A user visits a payment wall (or purchase opportunity) early in their lifecycle (and unusually early) • We have other signals that they are likely to buy (usually behavior-oriented) During: • They don’t buy • They abandon relatively quickly After: • They don’t come back to the payment wall • They either grind or leave the game entirely Supporting Evidence: • Low affluence signals
  35. 35. © 2015. Company Confidential and Not for Redistribution. info@scientificrevenue.com 07/22/15 35 Options for Dealing with Stickershock • Give out more currency early • Initial Framing Offer • Targeted Intervention • Reorder baseline prices and reset default • Different set of Baseline Prices • Different set of Baseline Prices with windowing

×