Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
@HAStarkHeather StarkKinranStrata ConferenceLondonOctober 1 2012                    1
@HAStarkService access, delivery, and pricing                                        2
@HAStarkService access, delivery, and pricing                                        3
+    +    F2P4          @HAStark
5    @HAStark
@HAStark                                       $3.75 billion   http://venturebeat.com/2012/01/06/deanbeat-game-companies-...
@HAStark                                            IPO    Zynga                                                          ...
@HAStarkA lot of the development and the energy in the eco-system is notgoing towards building desktop stuff anymore, its ...
@HAStark     Facebook       nearly 1 bn Facebook users       25% play games regularly       virtual goods spend on FB ...
@HAStarkSource: Velo Partners aggregation of AppData datahttp://www.slideshare.net/evanvrs/end-of-the-bull-market         ...
@HAStarkis everywhere                11
@HAStark              PlayfishPlayfish                           Development really starts with launch                    ...
13     @HAStark
@HAStarkDesign = Change                  14
How?15            @HAStark
@HAStark                     • events (i.e. user actions & results)     Logging                     • context             ...
@HAStark                                                               http://www-conf.slac.stanford.edu/xldb2012/talks/xl...
@HAStark                • Troubleshooting  Reactive                                                            Superficial...
@HAStark(this slide intentionally left blank)                                        19
@HAStarkBI                         UX Acquisition cost            Funnel tracking(service delivery costs)      Cohort s...
@HAStark   Facebook resell       What is the right metric for    per app summary        ‘engagement’?    data about DAU ...
@HAStark Engage-                 Retention  ment                                            Monetis-                      ...
@HAStarkRetention                                        Play after day 1?                                                ...
@HAStarkContext                is about discovering                       these relationships Out-comes     Experience    ...
@HAStark                  Sequential     Filter                                 goal                                 lossC...
@HAStarkand techniques                 26
@HAStark                                   Activities               Logging   Report/query Exploratory   Model testingApp ...
@HAStarkGENERAL                                   ACTIVE EUROPEAN PLAYERS   dozens of app tracking solutions         Pin...
@HAStark                                              Activities                         Logging    Report/query Explorato...
@HAStarkServices focus on predicting behaviours with direct immediate revenue impact        e.g.   Sonamine’s             ...
@HAStarkDmitry Nozhnin, Head of analytics and monetisation, Innova:“We have tested over 60 individual and game-specific me...
@HAStark                                7%                                        %Volume                                0...
@HAStark  • apply           Playnomics/Naked Communications    unsupervised                                             Pl...
@HAStark   useful open source tools readily available       r + libraries       WEKA       Python + libraries   data ...
@HAStarkBoredomBafflement             35
36     @HAStark
@HAStark “When a company                                                              And that data eventually is filled w...
@HAStark                       "when people start building their                     games to a spreadsheet its like putti...
@HAStarkTadgh Kellyhttp://www.slideshare.net/tadhgk/the-four-lenses-of-game-making-and-social-gameshttp://www.whatgamesare...
?     ?40             @HAStark
@HAStark                                    We don’t understand what’s going on. All                                    we...
@HAStarkBEAN-COUNTER   DESIGNER?                           42
@HAStark                     As someone who wants to analyse our games,                    but with no detailed knowledge ...
@HAStark   4th producer                                                       Firm size distribution    globally         ...
@HAStarknot data-driven                  45
@HAStarkFOCUS ON QUESTIONS THAT           SEEK ANSWERS THAT   are important                    are actionable   are int...
@HAStark   Depth      Csikszentmihalyi Flow model   psychology                    Behavioural                    economics...
@HAStark  Game types                                                       Player types                                   ...
@HAStarkA question bigger than itself   Change                  Think                   Look                              ...
@HAStark...and it’s very engaging                            50
Upcoming SlideShare
Loading in …5
×

Discovery driven design - heather stark - strata london

1,214 views

Published on

The speakernotes are working OK - needed to version down from .pptx. Click on the slide number tab underneath the comments box to see the text for that slide.
If you want to download .pptx version with speakernotes:
https://docs.google.com/open?id=0B1_F67kwEpJ_VkQzZ2RZNUQzWVk

Published in: Technology
  • Be the first to comment

Discovery driven design - heather stark - strata london

  1. 1. @HAStarkHeather StarkKinranStrata ConferenceLondonOctober 1 2012 1
  2. 2. @HAStarkService access, delivery, and pricing 2
  3. 3. @HAStarkService access, delivery, and pricing 3
  4. 4. + + F2P4 @HAStark
  5. 5. 5 @HAStark
  6. 6. @HAStark $3.75 billion http://venturebeat.com/2012/01/06/deanbeat-game-companies-raised-a-record-breaking-1-55b-in-2011/ 6
  7. 7. @HAStark IPO Zynga Facebook IPO -40% -76%http://www.google.com/finance?cid=481720736332929, Sept 19 2012 7
  8. 8. @HAStarkA lot of the development and the energy in the eco-system is notgoing towards building desktop stuff anymore, its going towardsbuilding mobile stuff Mark ZuckerbergSept 11 2012http://www.reuters.com/article/video/idUSBRE88A1F220120912?videoId=237677561...in July, Facebook sent people to the Apple App Store and Google play more than170 million times...https://developers.facebook.com/blog/post/2012/09/14/facebook---gdc-europe--developer-day-recap/ 8
  9. 9. @HAStark  Facebook  nearly 1 bn Facebook users  25% play games regularly  virtual goods spend on FB estimated at $1.2 billion 2011  iOS and Android  games revenue 2011 = 2x traditional portable console (reverse of 2009)  in-app purchases 87% of revenue of top 25 grossing apps  total worldwide game revenue $1.6 bn 2011  68% of sessions from ‘indie’ developers (i.e. new market entrants)Sources: Facebook, Flurry, InsideNetworks 9
  10. 10. @HAStarkSource: Velo Partners aggregation of AppData datahttp://www.slideshare.net/evanvrs/end-of-the-bull-market 10
  11. 11. @HAStarkis everywhere 11
  12. 12. @HAStark PlayfishPlayfish Development really starts with launch hired Ocado analytics lead King.com Analytical creativity after exec search extends the product lifecycle 10-15% of staff on analytics IsCool one release every day (formerly WEKA) more daily transactions than the FTSE http://www.gamesbrief.com/2010/04/playfishs-advice-for-building-social-games-development-really-starts-with-launch/ Philip Reiseberger, Chief Games Officer, BigPoint, Evolve in London conference December 2011 http://www.renovatapartners.com/news/kingcom-appoints-director-bi http://www.facebookgarage.org.uk/talk/big-games-vs-big-data http://www.slideshare.net/IsCoolEnt/iscool-entertainment-big-data-paris 12
  13. 13. 13 @HAStark
  14. 14. @HAStarkDesign = Change 14
  15. 15. How?15 @HAStark
  16. 16. @HAStark • events (i.e. user actions & results) Logging • context • metrics on standard event typesReporting/querying • with a priori segmentation Monitoring • game state conditions • patterns Investigating • dependencies 16
  17. 17. @HAStark http://www-conf.slac.stanford.edu/xldb2012/talks/xldb2012_wed_1125_DanielMccaffrey.pdf Daniel McCaffrey, We do Lots of General Manager, hundreds and experiments... Platform and Analytics hundred of ~3-5K active at Engineering experiments any time... 2012 every quarter. ManyKen Rudin, Most fail.General Manager,Analytics fail.2010http://tdwi.org/videos/2010/08/actionable-analytics-at-zynga-leveraging-big-data-to-make-online-games-more-fun-and-social.aspx 17
  18. 18. @HAStark • Troubleshooting Reactive Superficial • Use emergent result to guide FundamentalOpportunistic features and fixes change plan (almost) Any design • Adopt best outcome of split test any design element... Planned • Control timing of new feature element introduction • Algorithmic/heuristic parameter Automated value variation And... adapting the design variations 18
  19. 19. @HAStark(this slide intentionally left blank) 19
  20. 20. @HAStarkBI UX Acquisition cost  Funnel tracking(service delivery costs)  Cohort segmentation Customer lifetime value Engagement Monetisation Retention 20
  21. 21. @HAStark Facebook resell  What is the right metric for per app summary ‘engagement’? data about DAU and MAU, and  Which is the ‘better’ pattern? leaderboard info is widely freely available people think DAU/MAU is about retention and engagement but... it ain’t 21
  22. 22. @HAStark Engage- Retention ment Monetis- ation Virality Customeracquisition LTCV cost 22
  23. 23. @HAStarkRetention Play after day 1? No Yes Recruited by friend OtherValue 10% of inviters responsible for 50% of successful invitationsDiffusion dynamics of games on online social networks, Wei, Yang, Adamic, de Araújo and Rehkihttp://www-personal.umich.edu/~ladamic/papers/FBgames/FBgameDiffusion.pdf 23
  24. 24. @HAStarkContext is about discovering these relationships Out-comes Experience types K LTCVActions P I Player typesOptions 24
  25. 25. @HAStark Sequential Filter goal lossConcurrentFactual risk Imputed action resource level friend present or unease momentum suspense event loss choice reward 25
  26. 26. @HAStarkand techniques 26
  27. 27. @HAStark Activities Logging Report/query Exploratory Model testingApp trackingBI frontendData science 27
  28. 28. @HAStarkGENERAL ACTIVE EUROPEAN PLAYERS dozens of app tracking solutions  Pingflux  some platform specific  Playful  some more gamey than others  lots of new market entrants  Honeytracks BI front-ends are generic  Geosophy  visual exploration important  Fireteam Data science tools  Qlikview  can be so heavily wrapped in  .. services you can’t see them or  come as a blinking cursor – DIY – Data-mining services open source  Game analytics - UK or  Games analytics – Danish  as a bolt-on to std stats tools  .. 28
  29. 29. @HAStark Activities Logging Report/query Exploratory Model testingApp trackingBI frontendData science predictives graph db services cloud Testing +Serving 29
  30. 30. @HAStarkServices focus on predicting behaviours with direct immediate revenue impact e.g. Sonamine’s Player Lifecycle Management ™ ConvertSoon™ ChurnSoon™ PurchaseMoreSoon™ InfluenceSoon™ Source: sonamine.com 30
  31. 31. @HAStarkDmitry Nozhnin, Head of analytics and monetisation, Innova:“We have tested over 60 individual and game-specific metrics.None of them are critical enough to cause churn. None ofthem! We havent found a silver bullet -- that magic barrierpreventing players from enjoying the game.”http://gamasutra.com/view/feature/170472/predicting_churn_datamining_your_.php 31
  32. 32. @HAStark 7% %Volume 0.55 %Paying % 6% 36% 25% 2.34% 7Day Ret Virality Potential 57% $0.75 1.30% $4.40 CAC 26% 31% $2.21 0.89% 22% Early Enthusiasts 12% $1.75 0.86 59% Confident Completers 14% $3.57 0.97 Social Involver 5% 21% 0.19 9% $1.94 Sporadic SemiEngaged $2.3 8 Losing Momentum Need Guidance Revenue Potential Borderline IncompetentSource: Games Analytics 32
  33. 33. @HAStark • apply Playnomics/Naked Communications unsupervised PlayRM™ Messaging, learning to id individualised based on patterns in the segmentation/scoring type, frequency and sequence of player actions SaaS • predict P(return, VAS engage, invite, PlayRM™ Marketplace, monetize) using target players by their supervised historical and predicted learning game behaviourSource: Playnomics.com 33
  34. 34. @HAStark useful open source tools readily available  r + libraries  WEKA  Python + libraries data volumes quickly lead practitioners to become interested in: sampling  but... ‘interesting’ events are usually long-tailed efficiency  e.g. simplex volume maximisation for convex hull constrained matrix factorisation in n ▪ Christian Thureau, IT University of Denmark, Data Mining in Games, http://vimeo.com/14390303 parallelisation  Hadoop map/reduce in particular  some ML algorithms more suitable for Map/Reduce style parallelisation than others ▪ KDD2011, Vijay Narayanan (Yahoo!) and Milind Bhandarkar (Greenplum Labs, EMC), Algorithms in modelling with Hadoop http://www.slideshare.net/hadoop/modeling-with-hadoop-kdd2011 34
  35. 35. @HAStarkBoredomBafflement 35
  36. 36. 36 @HAStark
  37. 37. @HAStark “When a company And that data eventually is filled with engineers, becomes a crutch it turns to engineering to for every decision, solve problems. paralyzing the company Reduce each decision to and preventing it a simple logic problem. from making Remove all subjectivity any daring design and just look at the data.. decisions.”http://stopdesign.com/archive/2009/03/20/goodbye-google.htmlFor a more recent and enthusiastic exposition of Google’s split testing approach, see ex-Googler Josh Wills’ take on it:Experimenting at Scale, http://www.stanford.edu/group/mmds/slides2012/s-wills.pdfand http://berkeleydatascience.files.wordpress.com/2011/03/20110301berkeley.pdf 37
  38. 38. @HAStark "when people start building their games to a spreadsheet its like putting cars through a wind tunnel-they all come out looking like bullets"Matias Myllyrinne, CEO, Remedy Entertainmenthttp://www.gamesindustry.biz/articles/2012-09-05-remedy-entertainment-real-artists-ship 38
  39. 39. @HAStarkTadgh Kellyhttp://www.slideshare.net/tadhgk/the-four-lenses-of-game-making-and-social-gameshttp://www.whatgamesare.com/2011/12/the-four-lenses-of-game-making.html 39
  40. 40. ? ?40 @HAStark
  41. 41. @HAStark We don’t understand what’s going on. All we know is we’re going to keep running these experiments to try and understandeconomist in residence to look atJune 23 2012 Value hires Greek better what it is that our customers are telling us. And there are clearly things that we don’t understand because a simpleshared currency issues. analysis ...implies very contradictory yet reproducible results. So clearly there are things that we don’t understand, and we’re trying to develop theories for them. It’s... an exciting time but also a very troubling time. Gabe Newell, Founder, Valve Oct 23 2011 http://www.geekwire.com/2011/experiments-video-game-economics-valves-gabe-newell/ 41
  42. 42. @HAStarkBEAN-COUNTER DESIGNER? 42
  43. 43. @HAStark As someone who wants to analyse our games, but with no detailed knowledge of how to query databases or program, I need the easy-to-use solutions with the pretty presentation, and I need our web guys to set them up. I suspect a lot of companies who are new to analytics will be the same - the staff who need to analyse are not qualified to deep data mine themselves.Matt FalcusProduct ManagerTeam 17 Software 43
  44. 44. @HAStark 4th producer Firm size distribution globally over 200 Number of employees £1b to GDP 51 to 200 21 to 50 ~4000 people ~500 companies 11 to 20 5 to 10  Skillset (2008) Less than 5 9,000 developers 0% 5% 10% 15% 20% 25% 30% down 11% from Percentage of firms 2008  TIGA (2011)http://www.creativeskillset.org/uploads/pdf/asset_16891.pdf, 19/9/2012http://www.tiga.org/about-us-and-uk-games/uk-video-games-industry, 19/9/2012 44
  45. 45. @HAStarknot data-driven 45
  46. 46. @HAStarkFOCUS ON QUESTIONS THAT SEEK ANSWERS THAT are important  are actionable are interesting  but also are open, not closed  lead to more questions  although may need to be operationalised as yes/no connect to concerns and curiosities about  your design, and  how people use it, and  how it will evolve emerge from answers 46
  47. 47. @HAStark Depth Csikszentmihalyi Flow model psychology Behavioural economics + “personality”Image of Sigmund Freud, Library of Congress, Prints & Photographs Division, Sigmund Freud Collection, LC-DIG-ppmsca-23761.Image of Amos Tversky, Stanford News Network, http://psycnet.apa.org/journals/amp/58/9/images/amp_58_9_723_fig1a.jpgFogg Behaviour model reproduced with permission from B J Fogg http://www.behaviormodel.org/index_files/pasted-graphic.jpgCsikszentmihalyi Flow model, http://upload.wikimedia.org/wikipedia/commons/thumb/f/f6/Challenge_vs_skill.svg/300px-Challenge_vs_skill.svg.png 47
  48. 48. @HAStark Game types Player types Chris Bateman brainhex.comTadgh Kellyhttp://www.whatgamesare.com/2011/12/the-four-lenses-of-game-making.html 48
  49. 49. @HAStarkA question bigger than itself Change Think Look 49
  50. 50. @HAStark...and it’s very engaging 50

×