Soft Launch Planning and Management | Dylan Tredrea
•Download as PPTX, PDF•
1 like•313 views
1. Soft launches are different from regular live operations and require specific strategies to maximize learning. 2. It is important to take risks and make big changes during soft launches to learn as much as possible, as learnings accumulate over multiple tests and most soft launches fail. 3. Proper soft launch planning includes preparing server-side controls, operations plans, and qualitative feedback channels to support high-risk, high-reward experiments that could potentially "break" the game.
Recommended
More Related Content
What's hot
What's hot (20)
Similar to Soft Launch Planning and Management | Dylan Tredrea
Similar to Soft Launch Planning and Management | Dylan Tredrea (20)
More from Jessica Tams
More from Jessica Tams (20)
Recently uploaded
Recently uploaded (20)
Soft Launch Planning and Management | Dylan Tredrea
- 1. Running Effective Soft Launches Saving the games that can be saved Dylan Tredrea Head of Publishing Zeptolab
- 2. About Me •Internship at f2p platform in 2005(!) •Product @ Disney •BI @ Geewa •Product @ Rovio •Now Head of Publishing at Zeptolab
- 3. Too many free to play mobile games ‘die’ in soft launch because game teams don’t appreciate how different a soft launch is from regular live operations.
- 4. Talk Goals •Understand ‘rules’ of F2P •Best strategy for playing F2P ‘game’ in SL •How to prepare & plan for soft launch •How to make product/priority decisions in soft launch •How to incorporate qualitative feedback into decision making •How to manage risks (aka, respect players while still staying in business)
- 5. #1: Power Law Outcomes
- 6. #2: Risk & Reward are Related
- 7. Exceptions/Best Practices • IAP Prices: 50% & 150% • Difficulty: much easier & much more difficult
- 8. #3: You can break the game in SL
- 10. #5: A global launch ‘costs’ several Soft launches 200 Ideas 80 Prototypes 5 Soft Launches 1 Global
- 11. The Game of Games
- 12. So the rules of the game.. • Power law outcomes • High risk->higher rewards • Learnings accumulate • Near zero* downside • Most soft launches fail …. How do we play this game? What’s the dominate strategy for success?
- 13. Make as many big, deep, and DISTINCT changes in soft launch as possible GAMES
- 15. When to soft launch? NOW!!
- 16. Alpha Launch •Basically a polished prototype to measure CPI & D1 •Validate audience match, appeal of core gameplay
- 17. Territories & Platforms •Add territories liberally •Avoid territories used to localization •Android only at first
- 18. Prep Starts on Day 1 Keep track of: •Big, deep design options •Risks •Questions
- 19. Server Side Controls •Every number, parameter, string, etc. •Features have on/off flags
- 20. Never waste a user!
- 22. What if D1 >40% •Clear contender, but still a long way to go! •Still do some core mechanic tests (faster/slower, easier…) •But quickly move on to progression & long term features
- 23. What if D1 30-40% •Ok to spend a 1-2 months to fix •But high likelihood the game will not improve •Try some big changes (controls, difficulty, speed, even type of gameplay)
- 24. What if D1 >20% •Spend weeks trying to fix •Possible there is a tech issue, or game is way, way too difficult •But if not, should be killed immediately
- 26. Retention •Backbone of the business case •Really, really difficult to improve without big changes
- 27. SHIP MVPs •Don’t waste time with perfect art: ship MVPs! •20% of the design will have 80% of the impact!
- 28. High Risk/High Reward Ops •Don’t wait to measure things perfectly! •Instead of waiting to measure D30, look at the ‘fun factor’ (ratio of D3:D1; D7:D1) •Higher P Value for tests
- 29. Perhaps Most Importantly: Distinct Changes A proper product process is hypothesis, not data, driven Don’t spent too much time on something unless it’s a pillar or there is clear lift.
- 30. Don’t be Vestigial •C.A.T.S. had a sabotage feature •Wasn’t being used •So we cut it!
- 31. Nothing is for nothing •So few efforts having an impact is frustrating •But if well run, nothing will be for nothing
- 33. Support & Community •Highly valued team members •Regular, standardized reports •Must be put in context •Not always reliable data!
- 34. Risk Management & Respecting Players
- 35. Tool Set • Near zero risk doesn’t mean near zero responsibility! • MVP: Send all players X amount of hard currency with a custom text message “sorry our balance messed up the last event, here’s 100 gems!”
- 36. Operations Tips • Communicate whenever possible: ‘We’re trying X for a week, let us know what you think!’ • End successful soft launches with a big gift of hard currency • End unsuccessful soft launches gracefully with hard currency gifts & 1-2 months to use bought/gifted currency
- 38. Summary •Soft launch ASAP (if you’re not nervous- you waited too long) •Prepare operations, back end controls, and soft launch plans to max out your ability to learn •Don’t waste time with small changes, (almost) try to break the game with high risk experiments •Work closely w/ CS & CM •Always. Be. Learning. •Great soft launches only happen with full leadership support
Editor's Notes
- In my opinion, too many too many free to play mobile games ‘die’ in soft launch because game teams don’t appreciate how markedly different a soft launch is from regular live operations (after global release). Soft launches are very different business environments with unique opportunities, risks, and challenges. This is something I personally wish I had better understood at the start of my career. It’s also something I find myself talking a lot about in my current role in publishing where I see teams making what, in my opinions, are a lot of big mistakes that significantly reduce their chances of success when planning and running soft launches.
- The vast majority of changes will have no measurable impact, or more specifically, if you group together all the updates, ab tests, new features, etc. I’ve seen added to games and distributed by their impact on KPIs it looks like a sharp power law curve. A tiny percent have a huge impact, a 1/5th or so have some positive impact, but vast, vast majority have no meaningful impact. It’s a bit depressing sometimes, but also I think quite nice. It’s actually NOT that easy to manipulate people. It’s really, really hard to change people’s behavior with design. It’s not easy to throw something together and delight someone or give them a meaningful, engaging choice. So this is the first, key step in managing your soft launch: looking at how far away you are from your goal KPIs and being honest about how difficult it be to get there. If you’re quite close to the goals you need for global launch, great! But if you have a ways to go it’s irresponsible to go forward based on the assumption that this law doesn’t apply to you and to not admit to yourself and your team that you’re going to have to take a much more aggressive approach if you want your game to have a chance of reaching global release.
- By be more aggressive, I mean the team will have to take on much more risk with their decisions because... risk and reward are tighty related. There’s one thing everyone in finance knows that I don’t think everyone in game’s knows, but we need to understand and never forget: higher reward comes from higher risk. This is another law of life, we often forget. If you need to double your LTV, it’s not going to come from fixing the onboarding funnel or tweaking drop tables. You need to make a deep change to the experience. And I think this makes sense. If you want to see very different results, your players are going to need to play a very different game.
- Unless there is another top priority my default first test for a soft launch is difficulty. And once retention is looking solid my next step is IAP prices. But these are exceptions and tend to be ‘small’ +10-20% LTV wins They won’t turn around a game. For additional +20% LTV life you’re going to have to make big changes to the experience. Which is good news in soft launch because…
- Often as a PM I would suggest or advocate for a proposal that is, from the designers perspective, was very risky. Time and time again you hear ‘that’s going to break the game, our community will leave’ After global launch this is a real (though I think often overstated!) risk. Sometimes changes do break the game and do you have a negative, material impact on the business. In soft launch this risk is nearly zero, however. You can always just add a few more territories and spend some UA money to get new users back. It’s a huge and probably the most important difference between soft launch & global release: this is the one time when you can take huge risks. So make use of it!
- A well run soft launch- even if the majority of efforts have no measurable impact will yield a stack of learnings… And this is the only way to improve the tiny chance of your efforts having a positive impact Always be shipping experiments- district efforts with a clear purpose & experimental design- and always be learning If you do this well, you can capture big wins after a flurry of failure
- At Zeptolab, for our internal projects which are high risk, high reward, and very innovative we’ll go through 200 ideas to create around 80 prototypes for about 5 soft launches for 1 global launch. So what this means is be prepared for it to be really, really hard. It’s the entertainment industry. Most things fail. But just because it’s really tough, doesn’t mean you can’t do a lot to increase your chances of success
- So this is where we are… if we accept this foundation, then what’s the best way to go forward? What’s the best strategy to give your game the best chance for success?
- That’s it. When I’m reviewing soft launch plans this is what I always look for. I’m not interested in feature design, that’s why we have professionals on the game team, I want to make sure they are making as much of this limited time and the limited users of soft launch so we have the best chance of reaching global launch and if we do, it’s the strongest product possible. That happens by getting as many learnings out of your soft launch as possible, by shipping as many big, deep, and distinct changes as you can
- Ok so we’ve covered the strategy of soft launches and the operations- let’s now get into planning the soft launch for your game. How does one go about that?
- The biggest soft launch mistake is waiting too long to soft launch. Until the game is feature complete. In this approach the game will either be a hit or not and you’ll have almost no opportunity to save anything or learn anything. With some exceptions (e.g., idle games, RPG, and other meta heavy products) games should be soft launched as soon as you can get a read on early retention with just the core mechanic. Ugly UI, no sound, no tutorial… release it now! The first reason to do this is to validate your core mechanic. There is always the risk- and it’s not small- that your core mechanic just isn’t working and is not that engaging. By releasing with only the core mechanic you can measure d1 and find out if you’re really, really far away from the foundation for a game that can be sustainable. Personally, there are definitely a couple projects I wished we had soft launched much earlier because they were just not workable and it would have been great to know this much, much earlier. But even if you’re core mechanic is solid, you can still be learning. You can test the speed of the core mechanic, the difficulty curve of the first day or so of play- there are almost always some nice wins to get by just testing and tweaking the parameters of the core gameplay. And yes, if you soft launch early some of the numbers won’t be great, but that’s ok. The point here is learning and validating. By releasing with ugly UI, no progression features, etc… you can see exactly how how much each of these matter (Spoiler: most will not!) But now you know! When you’re struggling with what to prioritize late in rsoft launch and beyond you will have learned valuable lessons about what your players truly care about and what are few things that have a shot at changing their behavior I really can’t stress this enough, soft launch early- if you’re comfortable with when you’re soft launching, you’ve waited way too long. You should be nervous about it.
- Some projects can be greatly de-risked with an Alpha launch
- The next step in planning your soft launch is maxing out the number of users going through your game. Why? Because learning will cost users. You will need a baseline or control & a variant to compare that with and each will require enough users to give you confidence in the result. So it’s simple: the more users you have in soft launch, the more you will learn, the bigger your improvements will be, and the more profitable the product. (assuming you make good use of them of course!) The key considerations of territory planning: -How reliant you are on featuring of course if you’re really far away from your KPI goals this is the last thing you should be thinking about. Also if your game isn’t relying on on a mass market audience that featuring brings, the opportunity cost is much lower. -We recomend android only at first, so you can quickly update the game and also AB test your store front
- For the game team, the prep work for soft launch should start on day one. From the start of development the game team should keep track of big design options/opportunities, risks, and questions. Design options For example with CATS at Zeptolab, a big question was the monetization model so various approaches were tried in soft launch before setting on the current model. There were also concerns about machines with no driver, so this was tested at the end of soft launch which was successful in particular for marketing asset performance. Questions: Additionally, you'll have a running list of questions about how users will be interacting with the game? Do they like this mechanic, this block of a level, or this aspect of the meta? Are they engaging with this/that part of the experience? This list of questions should go to the analyst just before soft launch to design your analytics scheme so the data you’re getting from players will answer questions that will help the game team make decisions, validate theories, etc.
- Also as you’re approaching soft launch, that backlog of ideas and list of questions should inform how you set up your back end controls- or what stuff you can change in the game without a client update. MVP, every number, parameter, string should be server set and features should have on/off flags. The goal here is to give the team a lot of flexibility...
- Your scarcest resource isn’t money, time, or talent; it’s users. Never waste a user in soft launch, or in general ever. Always be generating learnings from them. The ultimate goal of soft launch- to learn as many meaningful things as possible as quickly as possible and that happens by making sure you're always learning. For example... say you update the game with a new feature, you AB test it for 1/2 of users, it has a positive impact, and you push it live for all users. Great! But now you have a couple weeks until you're next client update is live. If you have robust server side controls and a nice backlog of test ideas, you can make use of this time by running some tests with those server controls. This way instead of sitting and waiting, you're constantly generating learnings.
- OK so your team and your plan are set up… how you need to run the game… what product decisions do you prioritize?
- So this is where we are… if we accept this foundation, then what’s the best way to go forward? What’s the best strategy to give your game the best chance for success?
- So this is where we are… if we accept this foundation, then what’s the best way to go forward? What’s the best strategy to give your game the best chance for success?
- So this is where we are… if we accept this foundation, then what’s the best way to go forward? What’s the best strategy to give your game the best chance for success?
- Those are some general guidelines for a causal game. but frankly it always depends A business case is a lot bigger than just D1, but one thing that's clear is foundation is always retention- don't spend time improving monetization until retention is solid. And in my experience, retention is WAY WAY more difficult to change. I think a good PM can almost double ARPDAU in any game in long enough, well equipped soft launch- assuming retention is strong out of the gate or quickly. But retention is a stubborn beast and requires big changes to the experience.
- More high level, these D1 retention numbers are for games in competitive genres. It's certainly possible to build a game out of terrible retention numbers- it happens all the time, but of course they are wildly outperforming in other areas such as viral acquisition. So if you're fortunate enough to be in this situation, of course you should be weighing a focus on retention vs monetization and quickly getting to market very differently.
- When shipping updates in soft launch, only Ship MVPs Don’t waste time with perfect art Remember Pareto, 80% of the impact comes from 20% of the design. It's sometimes... or even often... hard to accept this as creators, but the fact is that a roughly done great feature will have uplift, but a perfectly done terrible feature will have no uplift. If you don't see an impact from your MVP, then you need to accept that this direction doesn't work and move onto other areas of the experience.
- Another way you can be more aggressive in soft launch, taking on more risk to increasing the chances of having a big return, is in your operations and how you make decisions. Instead of waiting for D30 numbers, you can look at what we call the 'fun factor' or the ratio of D3 or D7 to D1. If that ratio is improving, it's likely your D30 is improving as well. You can also have a lower requirement for significance, making decisions from a P value of .1 instead of .05... or even anytime you see a significant lift. It all depends how far away you are from your goals and how aggressive you need to me.
- One of the most important aspects of soft launch product decisions is ensuring you’re making distinct changes, especially in the start. Unless your positive a certain feature or experience is going to be a pillar of the business case for the product. For example in mid core events are going to need to drive big revenue spikes in order to for the game to be profitable, so in this case I spend a lot of soft launch time & energy on events. Without this specific example though, you want to make sure you’re testing very different parts of the experience… a kind of shot gun approach. Make big changes throughout the experience, following the old Sid Meier’s balancing advice: double or cut in half. If you see an impact, then drill down, but if you don’t see any movement then move on to another hypothesis and/or part of the experience.
- And if things don't work out, don't leave them in. If something isn't showing clear value then make sure to remove it from the game, for example in CATS there was originally sabatoge option in the battle, players never really used it so it was taken out to keep the game streamlined and focused. make sure to really absorb the learning from these things, if players were eagerly engaging with this feature it would have supported increasing the interaction during battles, but with players pretty much ignoring this feature it strongly supported the hypothesis that we should be focusing on the DIY nature of the game.
- Adding and then removing a feature like sabotage is frustrating. Indeed it can be really frustrating as so many things you do have no measurable impact and feel like a waste, but in a well run soft launch nothing is wasted and everything is generating a learning. Hopefully the point of this is clear that in every effort, you can slowly piece together a coherent… well corner of a picture of your audience. Learning from everything and constantly learning is really the only way to have a chance of saving your game now. Its the only way to have a chance of growing it further beyond global launch. Plus- hopefully it's super interesting to experiment, learn, and eventually discover what really motivates and engages your players.
- The ups & downs of high risk operations have a lot of opportunities for learning, but only if your team is properly set up & supported! Customer support leads & CMs needs to be highly valued, highly competent team members sitting with the team. These are not worker bees. They need to take a mass of messy data- qualitative feedback- and find standardized ways to communicate it to you and the game team. Additionally, when things do go wrong you need the proper support to get the most out of it. Just seeing a bunch of KPIs tank isn’t very helpful if you don’t have customer support and community management helping you put that into context. That said, the product lead needs to be able to put this into context. The fact is community uproar is often a leading indicator of success. And aren't always honest. In one game I worked on we ran surveys in game after seeing players complaining, all the time, about how difficult was to get special event characters. We worried this was driving churn, but even when we compared the behavior of people complaining it turns out they were spending a lot more time playing the event than before.
- Just because we should be taking huge risks doesn’t mean we shouldn’t be doing anything to manage and mitigate risk!
- Of course, just because you can break the game doesn’t mean you should be a jerk to your players! Teams needs tools & tech to help mitigate risks A full fancy tool set where you can send certain players a certain amount of a certain resources is expensive, not all developers have that, but being able to send all players X amount of hard currency with a custom message should be manageable for any professional dev team. You must build out this capability. This way you are comfortable taking big risks and making big changes, because you can at least compensate players when you mess up. Trust me, it makes it MUCH easier to make risky decisions when you know you can do more than say ‘oops’ on social media feeds that only a tiny % of players read.
- Embrace the iterative process WITH your players. Try something crazy, but tell them it’s temporary. This way they know you are trying different things (which is great, better than a stale experience), it encourages more players (not just the angry ones complaining about the fact that games aren’t 100% free) and if they don’t like it, it probably won’t last forever.
- Soft launches, for good reason, are really stressful The business pressure will always be there But this approach frees the team to try big things, take big risks At least for me personally, this is way more fun than simply releasing it and hoping the metrics are already good We can never get rid of the business stress, but at least we can empower teams to have a chance to save games that are failing. So personally the most frustrating experiences were on struggling games where we had no room to try anything. The most rewarding games were those where we had the support, time, and resources to really try and save the game. Even if it didn’t work out (it’s still the entertainment business, nothing is easy!) it was far more enjoyable, rewarding, and educational than just hoping that our FTUE fixes or polished UI is going to turn around a failing product. The team isn’t going to be taking any risks if they know that any kind of negative change will result in getting yelled at or losing key support for the project. The team needs to know it is their mission to spend soft launch making big, high risk bets- even if it sometimes has a negative impact or seems like a bad idea to those outside the game team. I even go so far to say, only half joking, that if we never break the game in soft launch I’m going to be a bit bummed… because that means we left some lift on the table. The team needs to be empowered and trusted. It’s the only way this works. As leaders, we have to expect that individuals are going to prioritize their family & financial security first. They won’t- and shouldn’t, in my opinion- put their own career and standing at risk to change the culture and strategy of a studio. So if leadership isn’t making it incredibly clear that their job is to take big risks and look for big wins, they are going to be playing it safe, which at the end of the day is going to hurt your shareholders and leaders just as much- if not more- than the game team.