This document provides summaries of 14 recipes for optimizing mobile user experience through A/B testing using Leanplum. The recipes address goals like re-engaging inactive users, optimizing the first-time user experience, improving tutorial flows, announcements of new features, push notifications, monetization strategies, and more. Each recipe includes suggestions for what to test and measure and provides a difficulty level and potential impact. The document is intended to help mobile teams use A/B testing to improve activation, engagement, retention and monetization.

1. 14 Delicious Recipes
for Lean a/bs
leanplum
Mobile Gaming Edition

3. From re-activation to increasing mobile engagement and
monetization, we hope these delicious A/B testing recipes
will help your team optimize the mobile user experience.
The Leanplum Team
Leanplum unleashes the value of your user data by making it easy to A/B test and
personalize mobile content and messaging. Get immediate feedback on experiments
with metrics you can act on today.
leanplum

4. Table of Recipes
Re-Activate
First Time User Experience (FTUE)
HOLIDAY PROMOTION
TUTORIAL FLOW
SOCIAL SIGN IN
NEW FEATURE ANNOUNCEMENT
Push Notification
EARLY MONETIZATION
IN-APP PURCHASE
ABANDONMENT
LEVEL ABANDONMENT
3
9
11
1
13
5
7
Few opportunities exist that allow you
to re-engage your users with a push
notification without irritating them. These
recipes provide the right opportunities.
The cost to re-activate and keep an
existing user is much lower than the
cost to acquire a new user, so use this
channel strategically.
User flows are the lifeblood of your game.
If there are any arterial blockages, users
can churn after asking themselves “Now
What?” The most important user flow to
optimize is the first 10 seconds when a
user enters your game (the first time user
experience), because a majority of users
will leave within this time period and never
come back.

5. Engage
Monetize
NEW FEATURE ANNOUNCEMENT
In-App Message
OPTIMIZE FOR
SOCIAL BEHAVIOR
VIRTUAL ECONOMY BALANCE:
SOURCES AND SINKS
NEW FEATURE DEPLOYMENT
APP RATING SCREEN
LEVEL DIFFICULTY
PRICE SENSITIVITY
15
17
25
19
21
23
27
No mobile app is truly static. Being
lean means iterating, learning and
gaining insights through data and
generating new ideas to test. User
engagement encompasses a broad
swath of recipes, from individual
UI elements within screens (call
to action, copy, colors, images) to
funnel analysis and optimizing for
specific goals or achievements (social
sharing, posting content, uploading
photos, leveling up, etc).
Game monetization is critical for any
company that relies on ROI to keep the
lights on. Make sure you know who your
most avid and loyal users are. They are
the ones most likely to monetize, or keep
monetizing. On the flip side, there may
be users who may be willing to pay, or
pay more, given the right price point or
engaging content.

6. NEW FEATURE ANNOUNCEMENT
Push Notification
WHAT TO MEASURE:
As a Product Manager, I want to notify users of recently launched new
content and features via push notification to maximize engagement
and retention.
• Push notification:
- text
- deep link location
- in-app message (aka landing page)
Re-Activate
As you develop a cadence of announcements, measure and get
feedback on which features and content different user segments find
the most relevant and engaging. Be sure to measure and optimize
your campaigns by the most responsive user segments and A/B test
your messaging.
1
Difficulty level:
Novice
Impact:
3/5
WHAT TO A/B TEST:
GOAL:
leanplum
Pro-Tip:

7. WHAT TO MEASURE:
WHO TO TARGET:
• Push notification:
- total deliveries
- total opens
- open rate
• New feature or content clicks
per user
• DAU/MAU
• Day 1, 3, 7, 14, 30 retention
• ARPU, ARPPU
All users

8. HOLIDAY PROMOTION
As a Marketer, I want to re-engage inactive users with a seasonal
promotion push notification.
• Push notification:
- text
- discount coupon
- in-app message (aka landing page)
- deep link location
Re-Activate
Messaging users with special offers during seasonal events like
holidays is a good tactic to re-activate users who may be more
receptive during these times.
3
Difficulty level:
Basic
Impact:
2/5
WHAT TO A/B TEST:
leanplum
Pro-Tip:
GOAL:

9. WHAT TO MEASURE:
WHO TO TARGET:
• Push notification:
- total deliveries
- total opens
- open rate
• Promoted item clicks per user
• Time spent in app per daily
user
• DAU/MAU
• Day 1, 3, 7, 14, 30 retention
All users during the seasonal
event

10. IN-APP PURCHASE ABANDONMENT
As a Marketer, I want to re-engage users who clicked on an in-app
purchase but did not go through with the purchase by sending them
a reminder via local push notification.
• Push notification
- text
- discount coupon
Re-Activate
This recipe is a must-do for marketers. Use targeted local push
notifications as a best practice to re-engage with a user’s purchase
intent. Offering a discount or special offer may persuade them
to come back and complete their purchase. Pair this with a
corresponding in-app message to maintain consistency in offer
messaging and incentives.
5
Difficulty level:
Intermediate
Impact:
3/5
WHAT TO A/B TEST:
leanplum
Pro-Tip:
GOAL:

11. WHAT TO MEASURE:
WHO TO TARGET:
• Push notification:
- total deliveries
- total opens
- open rate
• Checkout occurrences per
user
• Time spent in app per daily
user
• DAU/MAU
• Day 1, 3, 7, 14, 30 retention
• ARPU / ARPPU
All users who select an in-app
purchase but don’t complete
the purchase within a certain
period of time

12. LEVEL ABANDONMENT
As a Marketer, I want to send hints to my stuck users using local push
notifications.
• Push notification:
- text
- in-app message (aka landing page)
- deep link location
Re-Activate
Not all users are created equal. Some love the ‘grind’ - others hate it.
Try sending them a tip or trick that will ease the friction and get them
over the hump.
7
Difficulty level:
Intermediate
Impact:
4/5
WHAT TO A/B TEST:
leanplum
Pro-Tip:
GOAL:

13. WHAT TO MEASURE:
WHO TO TARGET:
• Push notification:
- total deliveries
- total opens
- open rate
• Level completion rate
• Time spent in app per daily
user
• DAU/MAU
• Day 1, 3, 7, 14, 30 retention
• ARPU / ARPPU
All users who abandon a level
and don’t return after a certain
period

14. TUTORIAL FLOW
As a Product Manager, I want to experiment with a completely new
re-designed tutorial flow to optimize the completion rate.
• Tutorial flow
- Number of steps
- Completion incentive on / off
- New tutorial flow on / off
FirstTimeUserExperience(FTUE)
Adding a tutorial is a must to ensure you activate your users and get
them more engaged. Zynga, EA and other large game studios have
perfected the art of the tutorial to ensure there is no “now what?”
moment within the first 10 seconds of a user’s experience. A/B
testing is key to getting that right.
9
Difficulty level:
Intermediate
Impact:
5/5
WHAT TO A/B TEST:
leanplum
Pro-Tip:
GOAL:

15. WHAT TO MEASURE:
WHO TO TARGET:
• Funnel Analysis:
- onboarding completion rate
- step-to-step completion rate
- time spent in each step
- time spent from first to last
step
• Time spent in app per daily
user
• DAU/MAU
• Day 1, 3, 7, 14, 30 retention
• ARPU / ARPPU
New users

16. SOCIAL SIGN-IN
As a Product Manager, I want to maximize the number of new users
who sign in / create a profile with a social plug-in (FB / Twitter /
Google+).
• Number of registration form fields
• Position of social sign in button
• Sign in screen text, look & feel
FirstTimeUserExperience(FTUE)
One-tap social sign-ins reduce friction when compared to email
registration, and enable app developers to capture a richer set of
user data to better tailor the user experience. As a best practice, you
should always tell your users why you want them to login with a social
sign-in so they have context for the request.
11
Difficulty level:
Intermediate
Impact:
4/5
WHAT TO A/B TEST:
leanplum
Pro-Tip:
GOAL:

17. WHAT TO MEASURE:
WHO TO TARGET:
• Funnel Analysis:
- social sign in rate
- manual sign in rate
- step-to-step completion rate
- time spent in each step
- time spent from first to final
step
• Average first session length
• Time spent in app per daily
user
• DAU/MAU
• Day 1, 3, 7, 14, 30 retention
• ARPU / ARPPU
• New users

18. EARLY MONETIZATION
As a Product Manager, I want users to monetize early with a small
purchase to increase the probability they will become repeat
purchasers.
• Price point
• Discount %
• Look and Feel
- text
- offer image
- background image
- button copy / color / size /
placement
FirstTimeUserExperience(FTUE)
Studies show that if early users monetize, even with a small amount,
they are more likely to become recurring payers and much more
highly engaged. Showing them a special offer after an initial period of
engagement (tutorial and a milestone completion) can also increase
your LTV substantially earlier in the user lifecycle and lower the
carrying costs associated your media buys.
13
Difficulty level:
Basic
Impact:
5/5
WHAT TO A/B TEST:
leanplum
Pro-Tip:
GOAL:

19. WHAT TO MEASURE:
WHO TO TARGET:
• Offer:
- total views
- total clicks
- click-through rate
• Time spent in app per daily
user
• DAU/MAU
• Day 1, 3, 7, 14, 30 retention
• ARPU / ARPPU
All users

20. NEW FEATURE DEPLOYMENT
As a Product Manager, I want to test new content or features on a
subset of my users first to avoid hurting engagement or monetization.
• On-off switch for the new content or feature
Engage
Sophisticated game companies rollout new features and content
gradually first to a small percentage of their users, and then step-by-
step to 100% of their user base. Using this method mitigates the risk
of any new release negatively affecting your user experience.
15
Difficulty level:
Intermediate
Impact:
5/5
WHAT TO A/B TEST:
leanplum
Pro-Tip:
GOAL:

21. WHAT TO MEASURE:
WHO TO TARGET:
• New feature or content:
- total views
- total clicks
- click-through rate
• Time spent in app per daily
user
• DAU/MAU
• Day 1, 3, 7, 14, 30 retention
• ARPU / ARPPU
Smaller percent of all users,
potentially limited to tier 2
regions or markets first.

22. NEW FEATURE ANNOUNCEMENT
In-App Message
WHAT TO MEASURE:
As a Product Manager, I want to notify returning users of recently
launched new content and features via in-app messaging to maximize
engagement and retention.
• In-app message
- headline text
- body text
- background image
- call-to-action color, copy, placement
• In-app message on / off
Engage
Your app is never static. You want your users to know there is always
something new and exciting in store for them. Not everyone will
be interested in every new feature, so make sure you target your
announcement to the most relevant customer segment.
17
Difficulty level:
Novice
Impact:
2/5
WHAT TO A/B TEST:
leanplum
Pro-Tip:
GOAL:

23. WHAT TO MEASURE:
WHO TO TARGET:
• In-app message:
- total views
- total clicks
- click-through rate
• New content or feature clicks
per user
• Time spent in app per daily
user
• DAU / MAU
• Day 1, 3, 7, 14, 30 retention
• ARPU, ARPPU
Returning users when the
game starts, but only once per
user

24. OPTIMIZE FOR SOCIAL BEHAVIOR
As a Product Manager, I want to maximize social engagement
opportunities for my most viral users but limit them for my less social
users.
• Social sharing features on / off
Engage
You want to optimize virality as much as possible for highly social
users. However, non-social users will not respond to requests to
share/post achievements and should have limited exposure to social
requests in order to prevent them from churning.
19
Difficulty level:
Intermediate
Impact:
3/5
WHAT TO A/B TEST:
leanplum
Pro-Tip:
GOAL:

25. WHAT TO MEASURE:
WHO TO TARGET:
• Social sharing features:
- total views
- total clicks
- click-through rate
• Social vs. non-social user
funnel analysis
• Time spent in app per daily
user
• DAU/MAU
• Day 1, 3, 7, 14, 30 retention
Start with all users, then target
social / non-social personas

26. APP RATING SCREEN
As a Product Manager, I want to optimize for a 5 star rating by
requesting happy users to rate my game at the appropriate time.
• In-app message:
- headline text
- body text
- background image
- call-to-action color, copy,
placement
• In-app message on / off
Engage
High average app ratings are a driving force behind gaining more
organic and viral installs of your app. Low ratings can hurt the adoption
of your app. If you are running paid marketing campaigns, low ratings
can significantly increase your overall cost-per-acquisition.
21
Difficulty level:
Basic
Impact:
3/5
WHAT TO A/B TEST:
leanplum
Pro-Tip:
GOAL:

27. WHAT TO MEASURE:
WHO TO TARGET:
• Total 4 or 5 star ratings
• Time spent in app per daily
user
• DAU/MAU
• Day 1, 3, 7, 14, 30 retention
• ARPU, ARPPU
All users who pass an
emotional milestone event and
spent some time in the app.
Limit to once per user.

28. LEVEL DIFFICULTY
As a Product Manager, I want to tune game play mechanics for game
difficulty in order to test the optimal challenge level for certain user
personas (warriors, socializer, etc.)
• Flow
- number of steps to achievement
- completion incentive on/off
• New challenge mechanic on / off
• Achievement difficulty setting
• Look and feel
- icon graphic
- achievement image
- text
Engage
Make sure you are aware of where the greatest drop-off on level
completion is. Certain areas of your level or game could be too hard
(or easy!) and could be dissuading users from continuing. Looking at
the user flow and optimizing for milestone completion ensures users
get deeper and deeper into your game, increasing the likelihood
they’ll remain loyal and keep coming back.
23
Difficulty level:
Intermediate
Impact:
3/5
WHAT TO A/B TEST:
leanplum
Pro-Tip:
GOAL:

29. WHAT TO MEASURE:
WHO TO TARGET:
• Funnel Analysis:
- level completion rate
• Downstream behavior over
time by user persona
• Time spent in app per daily
user
• DAU/MAU
• Day 1, 3, 7, 14, 30 retention
• ARPU, ARPPU
Specific user behavioral
personas

30. VIRTUAL ECONOMY BALANCE: SOURCES
AND SINKS
As a Product Manager, I want to balance my in-app economy by
adjusting virtual currency amounts awarded (sources) versus virtual
currency amounts spent (sinks).
• Source
- amounts (virtual currency
awarded)
- frequency of new sources
• Sink
- amounts (virtual currency
depleted)
- frequency of sink opportunities
Monetize
An oldie but a goodie, this recipe is hard to setup correctly, but
vital for balancing your virtual economy. Make sure your users are
properly incentivized to progress through your game with the right
challenge and mastery level to provide peak engagement and
enough ‘grind’ to eventually monetize.
25
Difficulty level:
Advanced
Impact:
5/5
WHAT TO A/B TEST:
leanplum
Pro-Tip:
GOAL:

31. WHAT TO MEASURE:
WHO TO TARGET:
• Monetization rate at point of
virtual currency depletion
• Time to monetization
• # Sessions to monetization
• Source and sink amounts
stacked over time
• Time spent in app per daily
user
• DAU/MAU
• Day 1, 3, 7, 14, 30 retention
• ARPU, ARPPU
• LTV
All users

32. PRICE SENSITIVITY
As a Product Manager, I want to maximize total revenue by testing
price points for virtual currency bundles.
• Price point • Storefront
- look and feel
- text
- call-to-action button placement,
text, color, size
- background image
Monetize
Find the sweet spot for each virtual currency bundle’s price. Test
extremely high or low price points to anchor users to a mid-tier
bundle. Or try offering exponentially more virtual currency bundles
for higher price points.
27
Difficulty level:
Intermediate
Impact:
3/5
WHAT TO A/B TEST:
leanplum
Pro-Tip:
GOAL:

33. WHAT TO MEASURE:
WHO TO TARGET:
• Time spent in app per daily
user
• DAU/MAU
• Day 1, 3, 7, 14, 30 retention
• ARPU / ARPPU
All users

34. contact@leanplum.com
+1 415-230-0397