Mobile game A/B testing and optimization


Published on

How do you optimize your mobile game? Find out some cool ideas and best practices on increasing conversions, engagement and retention for mobile games.

Appiterate is the world's first visual A/B testing platform for native mobile apps on iOS and Android. Appiterate helps app publishers optimize mobile apps. It lets you A/B test, manage your app content, analyze your app variations, deploy content in real-time and segment your user base.

  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Mobile game A/B testing and optimization

  1. 1. How, what & why of mobile game A/B testing and optimization Based on
  2. 2. Game Developer Myth #253: • A/B testing is change • CHANGE IS DANGEROUS!! • Thus, A/B testing is DANGEROUS!!!
  3. 3. But…
  4. 4. Morpheus is right…
  5. 5. Game developers have it easier than app developers
  6. 6. Game developers have it easier than app developers Why? You asked?
  7. 7. Game developers have it easier than app developers Why? You asked? Stat #1: 93% of paid app downloaders pay for a mobile gaming app
  8. 8. This is because a game is the CORE product and not a supplement to another product as might be the case for other apps
  9. 9. This is because a game is the CORE product and not a supplement to another product as might be the case for other apps Also, you have to give it your all BECAUSE it is your core product!
  10. 10. Stat #2: 80% of all revenue generated by mobile applications in 2012 was from games
  11. 11. Stat #2: 80% of all revenue generated by mobile applications in 2012 was from games But, this stat is skewed by the Zyngas and Rovios of the world (The top 25 developers alone took in half of mobile development revenue in 2012)
  12. 12. So, what do you do if you are a small developer working out of your garage?
  13. 13. This perfectly portrays game developers!
  14. 14. • Acquiring the user is just getting off the blocks • In fact, you did well to get another download But… you still haven’t finished the race...
  15. 15. So, what do you need to make some sweet moolah??
  16. 16. So, what do you need to make some sweet moolah?? You need to be a NINJA. A R-E-M ninja! R-Retention E-Engagement M-Monetization Let’s have a look at what you can do to be one…
  17. 17. ENGAGEMENT Create targeted experiences for gamer segments • Tailor your game’s event flow based on previous interactions • If a gamer is unable to complete certain levels, you can dynamically change the difficulty level from our dashboard • Use preset conditions over a user group to achieve this!
  18. 18. ENGAGEMENT Game features • Common sense says - higher the number of unlocked levels, more would be the amount of time spent in a freemium game • But, what if you are keeping money on the table by giving your gamers too many free levels? • Possible in case of an engaging game • Developers shouldn’t lose out on what they deserve. • A/B test how many free levels should be there • Treat advanced gamers differently from a novice gamer • Different feature launches - another way to provide a more tailored gaming experience
  19. 19. ENGAGEMENT In-app messaging • Powerful medium of • Cross selling other games in your kitty • Selling your wares in an in-game store • Test which levels are the sweetest to send an in-app message and for what category of messages • Use user segments (both behavioural and demographic) here to create an instant win
  20. 20. RETENTION Start position • How many free credits do you give out at the start? • Too many means you lose out on later credit purchases • Too few means a higher chance of an uninstall • Can only be found out from rigorous experimentation and putting your numerous hypotheses to test • Segment the results using analytics to find what works for which user base
  21. 21. RETENTION Registration screens • These are a way to hold user information in games where you build your profile and earn reputation • Once you a gamer has come to this screen after downloading your game (which already means a WIN for you), you should not piss him off with a badly designed form • Test different variations of fields, form layouts and designs • Experiment using social sign-up plugins as well (Facebook, Twitter and the like).
  22. 22. RETENTION Characters in Tutorial • First ask yourself - do you even need a tutorial at first launch of your game on a device? • Separate the advanced gamers from the novices • Determine your target audience for a tutorial • Show different tutorial characters depending on gender or age of the gamer
  23. 23. RETENTION Theme layouts • Change game screen depending on • • An approaching festival or event • • Time of the year Use different layouts by country Use a visual mobile editor platform to do all that and much more.
  24. 24. MONETIZATION In-game ads • Advertisements are the most common form of earning revenue for free games • Test when maximum Click Through Rates (CTR) happen • • Or is it the text size or a different way in which the ad was worded? • • Is it a certain size, layout, format, on-screen position of an ad? Why not create different ad texts for gamers speaking different languages? Although interstitial ads are hated, you can always test delay times to increase CTRs
  25. 25. MONETIZATION In game purchases (1/2) • These form the biggest revenue earner for a freemium game • • • Virtual goods alone make $12.92+ per user per month for such a game That’s a lot of money, considering majority of gamers do not bother buying Iteratively find the sweet spot in pricing • Test different pricing strategies for various goods to find which goods are elastic and inelastic for you* • Test the amount of free currency you want to give out to gamers • Give out free currency to re-engage a person who isn’t able to progress in a game because he/she doesn’t want to buy + *
  26. 26. MONETIZATION In game purchases (2/2) • Targeted pricing • Increase your prices before weekends/holidays • Give discounts to a light user Play around pricing a lot
  27. 27. MONETIZATION In-game store design • Store layouts can help a lot in increasing purchases in a game • Answers to questions like these can be found out from real gamer data: • Which goods come before others? • How big should the picture of an item be? • Should you use descriptive text or does it clutter your screen? • Test where the link to the store should be present in the game menu! All this is to improve your ARPAU (another fancy term for Average Revenue Per Active User) – the raison d’être of a game developer/marketer
  28. 28. Maybe the greatest tribute to mobile game A/B testing was paid by Mark Pincus, CEO, Zynga, when he said, “We no longer need to build it for three months and hope and pray.”
  29. 29. To sign off, don’t forget what your purpose as a game developer is!
  30. 30. SOME ADDITIONAL TIPS (1/3) Remove bottlenecks in the testing process: • Use Appiterate’s code-free visual platform to eliminate approvals from your coders and save precious developer time • Create and deploy tests on the fly • Use live user data to deploy winning variations Appiterate is so easy even marketers can do it!
  31. 31. SOME ADDITIONAL TIPS (2/3) Remove failures: • Too much experimenting gives rise to experiments that don’t work • Appiterate’s dashboard helps you eliminate failures easily by letting you disable live tests Don’t change everything: • Split an experiment into two tests so that you can pinpoint the reason of the change in your goal metric • Too many types of changes in a multivariate test can spoil the experiment and you will end up without any insight
  32. 32. SOME ADDITIONAL TIPS (3/3) Segment your tests • Run some types of tests only on first-time users of your app • They have not seen your layout before and the UI would be new for them • Resulting data would be clean, unbiased and insightful!
  33. 33. Visit for more