Topic: What is the role of analytics in running live games? Game studios generally know that they need to instrument and use analytics, but often don't know how to use the data they are presented with to improve their games. Allison presents a "playbook" with detailed instructions on how to analyze the health of a game, what data to be focusing on to get insights and direction, and a framework by which everyone can be involved in the analysis process. This presentation pulls on Allison's experience building the game analytics function at PopCap Games, as well as bring insights from other leaders in the game industry and analytics space. [These were the supporting slides for Allison's talk at Nordic Game 2014].
Allison Bilas Bio: Hailing from Seattle, Washington, Allison earned her MBA at the University of Washington and then joined mobile games giant PopCap in 2011, where she built the game analytics team from scratch and assisted on hit titles Bejeweled Blitz and Plants vs. Zombies 2, amongst others. Now with several years of high-level user behavior analysis under her belt, Bilas joined the GameAnalytics team as VP of Product to manage the product design team and align efforts in making the best game analytics tool for game developers of all sizes.
Predictive analytics refers to a variety of statistical techniques that analyze current and historical facts to make predictions about future events. Using such techniques as predictive modeling, machine learning, data mining and game theory, predictive analytics can build models that exploit patterns found in historical and transactional data, patterns that can identify business risks and potential opportunities.
Apple iOS 7.1 new options for In-App Purchase RestrictionsYaniv Nizan
Apple introduced new options in iOS 7.1 regarding In-App Purchase restrictions for iPhone and iPad:
- Users are notified about 15 minute rule
- Users can change settings to require immediate passwords
5 Mistakes with Virtual Economies in Mobile GamesYaniv Nizan
The SOOMLA Project's mission is to help mobile game developers build better economies more quickly. We have gathered 5 mistakes that most 1st time mobile game developers do. We are hoping to help people avoid them.
Viral Installs - getting your users to share and rate your gameYaniv Nizan
Why some games have what we call "the k factor" while others don't? What makes users want to share some games. Design a narrative that entice users to socially engage with the game, improve the ease of sharing game acheivements and identify the best timing to ask users to connect, share and rate.
Blue print - framework for worlds, levels, missions and records in mobile gamesYaniv Nizan
New open source framework by SOOMLA. Building worlds, levels, quests, missions, challenges, gates, records, badges and rewards is now a lot easier for mobile game developers.
Predictive analytics refers to a variety of statistical techniques that analyze current and historical facts to make predictions about future events. Using such techniques as predictive modeling, machine learning, data mining and game theory, predictive analytics can build models that exploit patterns found in historical and transactional data, patterns that can identify business risks and potential opportunities.
Apple iOS 7.1 new options for In-App Purchase RestrictionsYaniv Nizan
Apple introduced new options in iOS 7.1 regarding In-App Purchase restrictions for iPhone and iPad:
- Users are notified about 15 minute rule
- Users can change settings to require immediate passwords
5 Mistakes with Virtual Economies in Mobile GamesYaniv Nizan
The SOOMLA Project's mission is to help mobile game developers build better economies more quickly. We have gathered 5 mistakes that most 1st time mobile game developers do. We are hoping to help people avoid them.
Viral Installs - getting your users to share and rate your gameYaniv Nizan
Why some games have what we call "the k factor" while others don't? What makes users want to share some games. Design a narrative that entice users to socially engage with the game, improve the ease of sharing game acheivements and identify the best timing to ask users to connect, share and rate.
Blue print - framework for worlds, levels, missions and records in mobile gamesYaniv Nizan
New open source framework by SOOMLA. Building worlds, levels, quests, missions, challenges, gates, records, badges and rewards is now a lot easier for mobile game developers.
App developers needs to know the LTV - user lifetime value. This is mostly needed for financing and marketing purposes. This slideshare shows how to extract the needed data from Flurry analytics and leverage a free online calculator to get the result.
Why is a player willing to spend 30 minutes collecting virtual coins with no real value in the real world? Because they see genuine, tangible value the virtual goods and benefits in your game! Creating demand for your virtual goods, and engaging players with your virtual money translates to retention, conversion, and…yes…monetization.
At every job fair, there's a company missing: yours. Here's a quick look at the types of companies you might start (pursuing scale, reliability, or freedom) and how to get started.
Book coming soon at http://startupcareerguide.com
Google Analytics is a popular choice among app developers. Getting LTV using GA is hard and this slideshare explains how to use retention and DAU data along side an online lifetime value calculator to get the result.
Jelly Splash: Puzzling your way to the top of the App Stores - GDC 2014Wooga
The match 3 puzzle genre is almost as old as it gets. Scour the App Store and you'll find hundreds of different varieties out there. Very few of these succeed however, and even less manage to hit the number one spot on the U.S. Apple App Store top download chart. Wooga's Jelly Splash managed to do just that, and in this session Florian Steinhoff, the creator of Jelly Splash, will give a detailed account on how his team managed that and what he learned throughout the development process.
Get ready to lock and load through this quick overview of some of the newest most innovative, tools around. Source: http://www.takipiblog.com/7-new-tools-java-developers-should-know/
All new features, expected features and speculations regarding the upcoming Java 9 release: the Jigsaw project, performance improvements and long awaited APIs:
www.takipiblog.com/java-9-the-ultimate-feature-list/
Create Tabletop Games to Foster Organizational LearningKarl Kapp
How can a simple game transform your learning efforts?
The CIA uses tabletop games to teach intelligence gathering, overcoming collection obstacles, and collaboration. The Harvard Business Review describes board games as a microcosm of business training that can help leaders and managers build the skills needed to operate effectively in the real world. In fact, board games have been used formally for teaching business concepts since at least the 1960s with the introduction of the MIT Beer Distribution game.
Many instructional designers, course developers, and training managers struggle to create engaging learning programs that get results. At the ATD LearnNow: Game Design workshop, you’ll learn how to design a tabletop game that can help transform your live instruction into a powerful, memorable learning experience.
User Experience 6: Qualitative Methods, Playtesting and InterviewsMarc Miquel
This presentation introduces the most fundamental qualitative methods: the playtesting and the interview. It discusses when to use it and the possible bias the researcher may incur.
These slides were prepared by Dr. Marc Miquel. All the materials used in them are referenced to their authors.
How to Design Effective Learning Games: Sharon Boller and Karl KappSharon Boller
Slides used during September 2017 ATD Learn workshop facilitated by Sharon Boller & Karl Kapp: "Play to Learn: Effective Learning Game Design"
Includes numerous slides identifying DIY game creation resources, templates, tools for creating learning games.
App developers needs to know the LTV - user lifetime value. This is mostly needed for financing and marketing purposes. This slideshare shows how to extract the needed data from Flurry analytics and leverage a free online calculator to get the result.
Why is a player willing to spend 30 minutes collecting virtual coins with no real value in the real world? Because they see genuine, tangible value the virtual goods and benefits in your game! Creating demand for your virtual goods, and engaging players with your virtual money translates to retention, conversion, and…yes…monetization.
At every job fair, there's a company missing: yours. Here's a quick look at the types of companies you might start (pursuing scale, reliability, or freedom) and how to get started.
Book coming soon at http://startupcareerguide.com
Google Analytics is a popular choice among app developers. Getting LTV using GA is hard and this slideshare explains how to use retention and DAU data along side an online lifetime value calculator to get the result.
Jelly Splash: Puzzling your way to the top of the App Stores - GDC 2014Wooga
The match 3 puzzle genre is almost as old as it gets. Scour the App Store and you'll find hundreds of different varieties out there. Very few of these succeed however, and even less manage to hit the number one spot on the U.S. Apple App Store top download chart. Wooga's Jelly Splash managed to do just that, and in this session Florian Steinhoff, the creator of Jelly Splash, will give a detailed account on how his team managed that and what he learned throughout the development process.
Get ready to lock and load through this quick overview of some of the newest most innovative, tools around. Source: http://www.takipiblog.com/7-new-tools-java-developers-should-know/
All new features, expected features and speculations regarding the upcoming Java 9 release: the Jigsaw project, performance improvements and long awaited APIs:
www.takipiblog.com/java-9-the-ultimate-feature-list/
Create Tabletop Games to Foster Organizational LearningKarl Kapp
How can a simple game transform your learning efforts?
The CIA uses tabletop games to teach intelligence gathering, overcoming collection obstacles, and collaboration. The Harvard Business Review describes board games as a microcosm of business training that can help leaders and managers build the skills needed to operate effectively in the real world. In fact, board games have been used formally for teaching business concepts since at least the 1960s with the introduction of the MIT Beer Distribution game.
Many instructional designers, course developers, and training managers struggle to create engaging learning programs that get results. At the ATD LearnNow: Game Design workshop, you’ll learn how to design a tabletop game that can help transform your live instruction into a powerful, memorable learning experience.
User Experience 6: Qualitative Methods, Playtesting and InterviewsMarc Miquel
This presentation introduces the most fundamental qualitative methods: the playtesting and the interview. It discusses when to use it and the possible bias the researcher may incur.
These slides were prepared by Dr. Marc Miquel. All the materials used in them are referenced to their authors.
How to Design Effective Learning Games: Sharon Boller and Karl KappSharon Boller
Slides used during September 2017 ATD Learn workshop facilitated by Sharon Boller & Karl Kapp: "Play to Learn: Effective Learning Game Design"
Includes numerous slides identifying DIY game creation resources, templates, tools for creating learning games.
Learning is driven by core skill set of kids. We have focused on identifying and improving core skill set by casual gaming. The process of skill set enhancement is very important at very young age.
For more information
info@twistmobile.in
Not WHEN Games but WHICH Learning GamesSharon Boller
L&D people think games are useful in a subset of situations. This session showcases numerous games to show how vast the landscape of learning games can be - from games involving only people to tabletop games to asynchronous digital games
Informing SoTL using playtesting techniquesKatrin Becker
Typical research methodologies in SoTL include many of the traditional educational research methodologies, yet there are methodologies in other disciplines and industries that can provide fresh perspectives. One of these industries is the videogame industry. While there are fundamental differences such as the fact that one seeks primarily to entertain and the other to enlighten or educate, the practices, processes, and theories behind playtesting games can be used to inform aspects of pedagogy that relate to engagement. The primary focus in game design is on the experience and there has been considerable research into ways to assess and measure the player experience through playtesting. Playtesting is concerned with such things as whether or not the game is fun, which parts are too easy or hard, whether and when people become bored, and with the player’s progression through the to the end. All of these properties have relevance to teaching and learning even though they may not appear to be directly connected with whether or not the learning objectives were met. Rather, playtesting is concerned with the motivation of the player/student to continue the particular course of learning, which speaks to the success of the methodology in capturing the student's imagination. Sometimes, simply taking a novel approach to evaluation can yield insights that were not uncovered by more common approaches. This presentation will provide a brief overview of formal playtesting procedures and highlight ways these approaches could be used in the classroom as well as how this could inform pedagogy.
PHP World DC 2015 - What Can Go Wrong with Agile Development and How to Fix ItMatt Toigo
A talk I gave at the 2015 PHPWorld Conference. PDF Version of the slides at www.matt-toigo.com/files/phpworld_2015_presentation.pdf
Agile and Scrum are often pitched together as the definitive silver bullet for eliminating pain from software development, but they include their own sets of problems that commonly drag down development teams. Whether an agile team is executing an internal project or doing work for a client, a very similar set of problems begins to afflict all the members of such teams, regardless of their roles. The common root causes of these problems can be quickly identified, and complementary solutions can be easily implemented to ensure a happy team that continues to deliver high-quality work.
06-04-2024 - NYC Tech Week - Discussion on Vector Databases, Unstructured Data and AI
Discussion on Vector Databases, Unstructured Data and AI
https://www.meetup.com/unstructured-data-meetup-new-york/
This meetup is for people working in unstructured data. Speakers will come present about related topics such as vector databases, LLMs, and managing data at scale. The intended audience of this group includes roles like machine learning engineers, data scientists, data engineers, software engineers, and PMs.This meetup was formerly Milvus Meetup, and is sponsored by Zilliz maintainers of Milvus.
06-04-2024 - NYC Tech Week - Discussion on Vector Databases, Unstructured Data and AI
Round table discussion of vector databases, unstructured data, ai, big data, real-time, robots and Milvus.
A lively discussion with NJ Gen AI Meetup Lead, Prasad and Procure.FYI's Co-Found
Adjusting primitives for graph : SHORT REPORT / NOTESSubhajit Sahu
Graph algorithms, like PageRank Compressed Sparse Row (CSR) is an adjacency-list based graph representation that is
Multiply with different modes (map)
1. Performance of sequential execution based vs OpenMP based vector multiply.
2. Comparing various launch configs for CUDA based vector multiply.
Sum with different storage types (reduce)
1. Performance of vector element sum using float vs bfloat16 as the storage type.
Sum with different modes (reduce)
1. Performance of sequential execution based vs OpenMP based vector element sum.
2. Performance of memcpy vs in-place based CUDA based vector element sum.
3. Comparing various launch configs for CUDA based vector element sum (memcpy).
4. Comparing various launch configs for CUDA based vector element sum (in-place).
Sum with in-place strategies of CUDA mode (reduce)
1. Comparing various launch configs for CUDA based vector element sum (in-place).
Levelwise PageRank with Loop-Based Dead End Handling Strategy : SHORT REPORT ...Subhajit Sahu
Abstract — Levelwise PageRank is an alternative method of PageRank computation which decomposes the input graph into a directed acyclic block-graph of strongly connected components, and processes them in topological order, one level at a time. This enables calculation for ranks in a distributed fashion without per-iteration communication, unlike the standard method where all vertices are processed in each iteration. It however comes with a precondition of the absence of dead ends in the input graph. Here, the native non-distributed performance of Levelwise PageRank was compared against Monolithic PageRank on a CPU as well as a GPU. To ensure a fair comparison, Monolithic PageRank was also performed on a graph where vertices were split by components. Results indicate that Levelwise PageRank is about as fast as Monolithic PageRank on the CPU, but quite a bit slower on the GPU. Slowdown on the GPU is likely caused by a large submission of small workloads, and expected to be non-issue when the computation is performed on massive graphs.
Unleashing the Power of Data_ Choosing a Trusted Analytics Platform.pdfEnterprise Wired
In this guide, we'll explore the key considerations and features to look for when choosing a Trusted analytics platform that meets your organization's needs and delivers actionable intelligence you can trust.
Ch03-Managing the Object-Oriented Information Systems Project a.pdf
Your Game Analytics Playbook
1.
2. Your Game Analytics
Playbook
Allison Bilas
@allisonbilas / allison@gameanalytics.com
1. Introduction
1. Currently VP of Product @ GameAnalytics
2. Just moved to Denmark, loving hygge.
2. PopCap for last 3.5 years.
1. Hired when they didn’t know what to do with data.
2. Built game analytics team from scratch.
3. Learned a LOT about how to infuse data into the decision making process behind developing and operating games.
4. Hope to share some of my experience with you.
3. Agenda
‣ What is a game analytics playbook?
‣ Why is it important?
‣ Components of a “play”
‣ Examples: pre-season to end season
This presentation is about:
!
1. What a game analytics playbook is
2. Why it’s important to have one
3. I’ll go into the component of a game analytics “play”
4. And I’ll give you some concrete examples from the pre-season to the end season
!
Let’s get started…
4. What is a game analytics
playbook?
1. A playbook comes from American football.
1. I’m not actually that into the sport.
2. But it’s a powerful tool.
2. I want you to imagine a football team on the field. The QB calls his team together.
1. Competitors position
2. Health of his team
3. Time left on the clock
2. Makes a call for a specific play > team takes action > success/fail
3. If the game isn’t over, they start again.
!
!
5. What allows the QB to make that decision is their playbook.
1. Put together at the beginning of season.
2. Anticipates every situation the team is likely to encounter.
!
A game analytics playbook is your plan for using data to inform the development and operations of your game.
!
We had one at PopCap that was explicit, but it doesn’t have to be.
5. Information resolves
uncertainty.
1. So, why is this important?
2. Assuming you’re here because you understand the power of data.
1. Playbook can be used to inform AEM and live ops.
2. In short, it helps to resolve uncertainty and speed up your decision making and development.
3. Caution that analytics doesn’t solve everything. Game still needs to be a great experience for your players.
6. Components of a Play
Context Question
1. Let’s get into the components of a “play”.
1. This is essentially the analysis process.
2. Most studios start with thinking about what to track, but there are a couple of steps that should happen before that…
2. Like the QB on the football field, first step is CONTEXT.
1. Where your game is at in its lifecycle. (We’ll go over these in the next sections.)
2. Competitive context and industry trends
3. Goals for your game in terms of KPIs
3. By assessing the context, you’ll start to come up with analysis QUESTIONS.
1. Typically focused on AEM, but also can be more technical in nature around things like bugs and frame rates.
2. Likely to have a LOT of questions.
3. Prioritize based on: measureable / actionable / most impact.
7. Components of a Play
Context Question Data / Tools Action
1. DATA AND ANALYSIS TOOLS
1. Here is where you need to start thinking about what to track in order to answer the questions you have…
2. In deciding what to track you should also consider the type of analysis you’re going to do, for example:
1. Segmentation, benchmarking, A/B testing and trend analysis.
2. Last component is ACTION. Here’s is where the playbook is most powerful.
1. If you’ve done everything right: CONTEXT > QUESTION > DATA/TOOLS
2. Deciding what action should be very easy. Typically implied in the questions you’re asking.
8. Pre-season
(Dev)
Early Season
(Beta)
Mid Season
(Launch)
Late Season
(Ongoing Ops)
Seasons
1. Next I’m going to give examples that follow the life cycle of your game from:
1. Pre-season or development
2. Early season or beta period
3. It’s not until the mid season that you’ve finally launched
4. And then the late season, or ongoing ops
9. Pre-Season (Development)
‣ Instrument tracking EARLY and TEST
‣ Be judicious in what you track
‣ Think through event hierarchies
‣ Consider multiple tracking systems
1. Pre-season is straight forward - SET UP TRACKING.
2. I recommend you start doing this very early.
1. This means you’ll track things initially that you don’t need in the end. This is okay.
2. It lets you start to refine the questions you have and ensure that you have the data needed to answer them.
3. Also gives you time to test. Every game I’ve every launched has found tracking bugs in the first day after launch.
3. Let’s you be smart about what you track. E.g. Game I was working with wanted to track every time a player turn on/off the sound.
4. Also in the early season, you should be thinking very closely about your event hierarchies.
1. These can dictate the type of analysis you can do, and it’s very nuanced.
2. It helps to start tracking and trying out analysis, and then making changes.
5. Multiple tracking systems — can add some complexity, but allows validation of data.
10. Early Season (Beta)
1. Early season is BETA period.
1. Select markets, and you have data! YAY!
2. Exciting period. Easy to loose focus.
2. Most important = look for unexpected behaviors in gameplay and monetization.
3. HIDDEN AGENDA
1. Hidden object game launched on FB a few years ago. Few hundred Australians playing it every day.
2. One player was spending a lot of money in the game. WHALE!
3. What was she spending money on > gotcha boxes of consumables.
4. How was she progressing in the game > noticed she was stuck on Level 18.
5. Needed a “finger print duster” to move forward. But that item wasn’t available until Level 20. BLESS HER!
11. Early Season (Beta)
1. Because it was the beta period, this wasn’t terrible. We were able to identify the weird behavior, find the problem and fix it.
2. In terms of analysis in the beta period:
1. Segmentation is very important. In addition to the few hundred Australians, we had thousands of Philipine installs coming and going in the
game in order to pressure test.
2. Important to filter and look at them separately.
3. I also want to mention that A/B testing isn’t necessary. Lots of info just by observing.
12. Mid Season (Launch)
1. Finally at mid-season the game launches! This is when things start moving really quickly.
2. Analysis should focus on high level metrics > dive deeper when something isn’t working.
3. One example from my experience at PopCap was when I was working with PVZ2.
1. Immediately after launch, millions of installs. Playing for a few days and then leaving. Retention was horrible. Everything else was good!
2. Digging in to engagement… players spending less than 5 minutes per session in the game. BUT, time to complete a level took 15 to 20
minutes.
3. Players had to return 3 or 4 times to the game before they had a reward moment. Felt like a slog.
4. Team identified this and set about finding ways to shorten the game lengths.
13. Mid Season (Launch)
1. Quite honestly, too late! Millions came and left and it was hard to get them to come back.
2. In launch analysis, you’ll be looking at data all the time. HOURLY, DAILY.
3. Intense period lasts probably 1 month… THEN…
14. Late Season (Ongoing Ops)
1. At some point you need to slow down and go deeper with your analysis. This is your end game, but it should never actually end!
1. Focus on refinement and longer term game design.
2. Robust content pipeline and promotional program.
2. Start looking at trends in the data.
3. Bejeweled Blitz Rare Gems
1. Super interesting game. FB for 5yrs, mobile for 3yrs.
2. Success because of content and promotions.
3. Main content is Rare Gems, extra powerful boosts.
4. Had to launch a few of them look at trends in order to understand what made them successful.
1. First one was a big hit, just because it was first.
2. 2nd and 3rd were also good…
3. Wasn’t until 6th and 7th that we started to get it.
5. Rare Gem experience is made up of:
1. In game visuals and explosions
2. Last hurrah with explosions
3. Slot machine mechanic at the end
6. Most successful had ALL of them, but with trend analysis we could understand the tradeoffs between including each of these components or
not.
7. This informed our development and let us make explicit decisions around the tradeoffs in the content design.
15. Summary
A game analytics playbook is your plan for using data
to inform the decisions you make in operating your
game.
Context Question Data / Tools Action
1. I’ve told you what a playbook is and how it relates to building and operating you game.
2. Your next step is to consider where your game is at in its lifecycle, and start developing your plays.
!
3. I would love to talk more about this with you! I love thinking through this process and how to make it both accessible and actionable.