Playtesting for Indie Studios

Pejman Mirza-Babaei; Naeem Moosajee; Brandon Drenikow, University of Ontario Institute of Technology

Playtesting for Indie Studios
Pejman Mirza-Babaei, Ph.D.,
Brandon Drenikow
Naeem Moosajee, M.Sc.
@naeemmoose
naeem.moosajee@gmail.com

Me
o  Focus on taking methods and techniques learned in
industry and developing techniques indies can u7lize
o  Research focused on Games User Research
2

Agenda
Intro
Methods &
Approach
Games
Tested
Takeaways
Next Steps

Execution Labs 4
o  XL is a hands-on early stage investor in game studios

Games User Research 5
o  Games User Research focuses on players’ psychology
and their behaviour via techniques such as
playtes7ng, analy7cs, expert analysis, and others.
o  Game User Researchers aim to help game developers
deliver players the best gaming experience possible.
h"p://gamesuserresearchsig.org/what-is-gur/

Summary
o  6 diﬀerent commercial games
o  11 playtest case studies
o  118 par7cipants in total
7

Five Key Factors - Tips
o  Inﬂuencing playtes7ng for indie studios
1.  Playtes7ng process
2.  Playtest focus and methods
3.  Playtest lab setup
4.  Building par7cipants database
5.  Developers playtes7ng training
8

(1) Playtesting Process
o  Process should deﬁne
•  Design of playtes7ng stages
•  Number of par7cipants, 7me playing, number of
researchers needed etc.
•  Analysis and repor7ng
9

(2) Playtest Focus & Methods
o  What to evaluate?
o  What ques7ons do you want me to answer?
o  What is your design intent?
o  FTUE and RITE
o  First hour experience and onboarding phase
10

(3) Playtest Lab Setup
o  Dedicated lab vs ad hoc
setup
o  Observa7onal data
•  Video game capture
•  Microphone recording
•  Webcam recording of faces or
input interac7on
11

(4) Building Participants Database
o  Large diverse par7cipant pool
o  Online form with demographic data
•  On which pla^orm do you usually play?
•  How many hours per week do you spend playing video
games on consoles or PC?
•  What are your favorite gaming genres and themes?
•  At which frequency do you play video games on tablets or
smartphones?
•  What are your favorite mobile gaming genres and themes?
12

(5) Developer Training
o  Workshops to train developers
o  Sessions on how to:
•  Conduct an interview / focus group
•  Create a survey
•  What telemetry hooks are important to track
•  Selec7ng par7cipants
•  Colour, shape and signs for Usability
13

Next Steps
o  Playtes7ng can help
•  Resources are not always available
o  Focus on how to address indie needs
•  More explora7on needed
•  Indie development 7meframe
14

This document outlines a presentation on using emergent behavior AI in horror games. It discusses the project background, methodology used, development technologies of Unity3D and Visual Studio, design and implementation, a demo, evaluation methods and results, and conclusions. The project aims to create replay value and a horror aesthetic by developing an AI system where ghouls exhibit unpredictable behaviors. Evaluation surveys found that users perceived the ghoul's behaviors to be evenly unpredictable.

DevOpsDaysRiga 2017 ignite: Kārlis Broders - Breaking barriers between busine...

DevOpsDays Riga

Creating a fast flow in the value delivery stream and enabling feedback loops are at the core of DevOps ideas and principles. The talk discusses the issue and presents a solution for automating the flow and monitoring capability of information artefacts that represent the state of an ongoing project. "To build the right product it is critical for the developer to receive timely feedback from the end-user or the business or product owner. Feedback is needed because writing perfect requirements beforehand and then delivering the needed software has proved to be extremely difficult. In the words of Martin Fowler, the co-author of the original Agile Manifesto: “…it's very hard to specify what you want in advance and be correct; people find it much easier to see something that's not quite right and say how it needs to be changed.” Not only feedback itself is important, but also its qualities like regularity, speed, context, precision and also the ease of tracking it. The speech focuses on enabling regular, fast and precise feedback by integrating and synchronizing work environments of separated product/business and development teams. In particular, we look at a government case study where separation is more acute, because the developer is outsourced through a contracted development company. "

Design for results2

Mirjam Eladhari

The document discusses experimental game design through iterative prototyping and playtesting. It outlines how prototypes can demonstrate early design ideas, playtests can provide feedback, and data collection helps analyze and improve subsequent iterations. The goal is to explore new design spaces through a production cycle of prototyping, testing, analyzing data, and redesigning in order to imagine potential new game features.

Design for Results - Considerations for experimental prototyping and play tes...

Mirjam Eladhari

How to bring creativity in testing

Tabăra de Testare

This document discusses how to bring more creativity and critical thinking to software testing. It notes that typical problems with testing like being boring, repetitive, or thoughtless can be addressed by automating routine tasks, differentiating infrastructure from user experience testing, focusing on testing strategy, and innovating new testing approaches like games. The document encourages testing teams to teach others, think critically about their work, and find ways to make testing more engaging and thoughtful.

Monkeytalk - Agnes Stenlund, Marvel app - Inside Marvel: How to build a great...

Monkeyshot

This document outlines 5 key lessons the author learned while interning at Marvel about building great products: 1. Keep everyone aligned on goals and priorities. 2. Thoroughly understand your users through various research methods. 3. Plan product roadmaps by evaluating options based on user experience, value, costs, and risks. 4. Use an adaptive design process that prototypes solutions to different types of problems. 5. Foster an enjoyable work culture where teams have fun while building reliable, usable products.

Getting Good UX Into Mobile

Steven Hoober

Ben Weedon from Serco Games Research presented on the challenge of measuring gameplay experience. He discussed how traditional usability metrics do not fully capture the experiential dimensions of games like fun and playability. Weedon suggested ethnographic research and diary studies adapted for mobile/casual games to provide quick feedback during development on issues related to context of use. The goal is to better understand general requirements for these new gaming areas and locate key issues through playtesting titles directly on users' devices.

Digital Learning Game Design: Lessons from the Trenches

Sharon Boller

User Experience 6: Qualitative Methods, Playtesting and Interviews

Marc Miquel

IGDA NL Game Jam at Sci Fi on the Rock 2012

LAFS Game Design 8 - Playtesting

The document provides guidance on playtesting games during development. It discusses that playtesting involves gaining feedback from players to improve a game's experience. Playtesting is not the same as internal design reviews, focus groups, usability testing, or quality assurance testing. The goal of playtesting is to ensure a game is functioning as intended, complete, balanced and fun. It recommends conducting iterative playtesting as a game develops to catch design issues. Key aspects of playtesting covered include finding suitable playtesters, running testing sessions, collecting both qualitative and quantitative feedback, and dealing with feedback received.

How Neuroscience revolutionizes User Research / Philipp Zent (Brainamics GmbH...

DevGAMM Conference

User research is an often overlooked but absolutely crucial part of developing a successful game. Although there has been a strong increase of popularity of user research in games, the methodologies that are used to gain player insights are highly questionable from a neuropsychological perspective. The core issues with traditional user research methodologies are that they only measure conscious data, although 90% of decision-making and emotions are subconscious, and that it fully relies on self-reported and thereby subjective data that is collected post-exposure. To fully understand the deep and often overlooked reservoir of subconscious human motivations and emotions that influence human behavior, an objective second-by-second depiction of the true emotional responses to stimuli are necessary. This can only be achieved by measuring the emotions directly where they happen, namely in the brain. How that can work in practice is demonstrated in this talk.

Unit 3: Research - Introduction

chriswalton

This document discusses research and pre-production techniques for game development. It introduces Japanese word "irashimasu" and outlines aims for a class on initial research methods, including having students ask questions about gameplay, establish preferences, and explain why pre-production is important. Various research techniques like focus testing, analyzing development processes, and considering localization are presented. Students are assigned tasks to document their own gameplay and compare it to an online gamelog site to apply these research concepts.

Game Development as A Business

Samuel Henry

Summerschool p6, user of playtesting

Louis Klomp

The document discusses user playtesting for games. It recommends asking five questions before each playtest: Why is the test being conducted, who should be recruited as playtesters, where should testing take place, what is being looked for from the test, and how should the test be structured and data collected. Effective playtesting involves finding the right testers to evaluate specific aspects of a game, like whether different groups play differently or if certain levels are too long. The goals of each test and how the results will be gathered and analyzed should also be considered carefully to improve the game based on user feedback.

Informing SoTL using playtesting techniques

Katrin Becker

Typical research methodologies in SoTL include many of the traditional educational research methodologies, yet there are methodologies in other disciplines and industries that can provide fresh perspectives. One of these industries is the videogame industry. While there are fundamental differences such as the fact that one seeks primarily to entertain and the other to enlighten or educate, the practices, processes, and theories behind playtesting games can be used to inform aspects of pedagogy that relate to engagement. The primary focus in game design is on the experience and there has been considerable research into ways to assess and measure the player experience through playtesting. Playtesting is concerned with such things as whether or not the game is fun, which parts are too easy or hard, whether and when people become bored, and with the player’s progression through the to the end. All of these properties have relevance to teaching and learning even though they may not appear to be directly connected with whether or not the learning objectives were met. Rather, playtesting is concerned with the motivation of the player/student to continue the particular course of learning, which speaks to the success of the methodology in capturing the student's imagination. Sometimes, simply taking a novel approach to evaluation can yield insights that were not uncovered by more common approaches. This presentation will provide a brief overview of formal playtesting procedures and highlight ways these approaches could be used in the classroom as well as how this could inform pedagogy.

Make Your Games Play, Teach, and Socialize Better: Usability & Playability Te...

Jason Schklar Consults!

The document summarizes techniques for improving games through user testing and feedback. It discusses common barriers to user testing like cost and process integration. It then presents case studies of techniques used on different projects that addressed these barriers, such as conducting remote usability tests with developers or testing low-fidelity prototypes with interns. General principles are to collaborate with developers to address issues in real-time and iterate testing to validate fixes. The document argues these techniques can help improve serious games by reducing errors and costs while broadening engagement.

IGDA NL Sci-Fi on the Rock Game Developer's Jam 2013

Research methodology

Hetika31

1) Online gaming is becoming more popular than outdoor gaming due to its accessibility and ease of play. Online games can be played by anyone regardless of skill level and require minimal physical movement. 2) Traditional outdoor games often require specialized skills and training as well as provide physical exercise. However, online games provide mental exercise by challenging players to think strategically and solve problems quickly. 3) Both online and outdoor gaming have benefits, but online options are more inclusive and accessible. Maintaining a balance of both is ideal for well-rounded entertainment and development.

Game Design 2: UI in Games - Revision Lecture

David Farrell

The document provides an overview of the topics and content that will be covered in the Game Design 2 exam. It includes: 1) A reminder of the key topics and content from each lecture that could appear in the exam, such as menus, text, UI components, prototyping techniques, and usability principles. 2) Information about the format of the exam, including that it will contain 6 questions worth 10 marks each, and students must answer 4 out of the 6 questions. 3) Advice for students on how to focus their studying, including reviewing the content and examples from each lecture, as the exam will test understanding of both technical and conceptual aspects of the topics.

GAMIFIN 2019 Conference Keynote: How to fail at #gamification research

Lennart Nacke

Lennart Nacke describes the many ways that failure is important and necessary for iterative design and development of gamification research. He outlines several ways that current gamification research can improve on experiments, execution, and publication of gamification studies. He touches on areas of game thinking, user experience, and design to tie all the examples of failure together into a call for honest design and research in gamification.

LAFS SVI Level 3 - Game Design and Analysis

The document discusses the core elements of game design, including the roles of players and designers. It outlines the iterative design process that designers go through, from initial ideas and prototyping to playtesting and refinement. Key aspects covered include brainstorming ideas, creating game documentation, pitching concepts, and defining core elements like objectives, rules, and resources. The document provides an overview of different genres and explains how genres can be combined.

Playtestix, Services Overview

Maria Kokovina

Playtestix offers professional team services for organizing and carrying out playtests, and provides access to wide base of game players (casual, mid-core and hardcore gamers) and frequent app users. Playtestix closely cooperates with leading developers and publishers of games and applications in the CIS region and Western Europe, such as FDG Entertainment, Wyse Games, Zeprolab, Creative Mobile, Bumper Apps, Room 8 studio, SMI games, Bytex, Digital Worlds, Vostok Games, Sly Lamb, Pixonic, etc.

New Microsoft PowerPoint Presentation.pptx

nilesh405711

The document discusses core principles of game design, including that games should be simple, unique, represent real-life environments, involve social factors, and be fun. It outlines the game design process of building a concept, including getting an idea, creating goals and emotional experiences for players. It also discusses differences between games and movies, and creating game specifications by identifying players, genres, environments, and success criteria.

Exam resource (new)

JCA

This document provides information and instructions to prepare students for an exam on the promotion and marketing of video games. It outlines the exam format, introduces exam topics like platforms for playing games and reasons for playing them. It includes activities for students to identify games they play and why. Students are also prompted to consider personality types and target audiences for games. The document guides students to work in groups to develop a promotional campaign for a new game called Typhoon 3D.

Designing a Studio For Game Designers | Luna Javier

Jessica Tams

This document outlines strategies for setting up an effective game design studio. It discusses screening designers to filter out those with poor teamwork or communication skills. Training involves building communication systems, giving designers independence, and teaching quick decision making. Regular design training is recommended to improve skills in areas like free-to-play monetization, writing, and pitching. The goal is to address common problem areas like teamwork, motivation, and decision making through screening, training, and development processes.

What the AI revolution means for Open Source, Open Tech and Open Societies

This document discusses Nextcloud and its founder Frank Karlitschek. Nextcloud aims to provide open source productivity, security and on-premises solutions as an alternative to proprietary services. It touches on Nextcloud's file sharing, collaboration and office tools. The document also discusses Nextcloud's vision for ethical artificial intelligence using fully open source and self-hosted models to avoid data leakage while reducing bias and carbon footprint.

Data balance sheets laying foundations for sustainable and ethical use of data

Similar to Playtesting for Indie Studios

Serco Usability Research, Ben Weedon, The challenge of measuring game play ex...

Use8.net

Digital Learning Game Design: Lessons from the Trenches

Sharon Boller

User Experience 6: Qualitative Methods, Playtesting and Interviews

Marc Miquel

IGDA NL Game Jam at Sci Fi on the Rock 2012

LAFS Game Design 8 - Playtesting

How Neuroscience revolutionizes User Research / Philipp Zent (Brainamics GmbH...

DevGAMM Conference

Unit 3: Research - Introduction

chriswalton

Game Development as A Business

Samuel Henry

Summerschool p6, user of playtesting

Louis Klomp

Informing SoTL using playtesting techniques

Katrin Becker

Make Your Games Play, Teach, and Socialize Better: Usability & Playability Te...

Jason Schklar Consults!

IGDA NL Sci-Fi on the Rock Game Developer's Jam 2013

Research methodology

Hetika31

Game Design 2: UI in Games - Revision Lecture

David Farrell

GAMIFIN 2019 Conference Keynote: How to fail at #gamification research

Lennart Nacke

LAFS SVI Level 3 - Game Design and Analysis

Playtestix, Services Overview

Maria Kokovina

New Microsoft PowerPoint Presentation.pptx

nilesh405711

Exam resource (new)

JCA

Designing a Studio For Game Designers | Luna Javier

Jessica Tams

Similar to Playtesting for Indie Studios (20)

Serco Usability Research, Ben Weedon, The challenge of measuring game play ex...

Digital Learning Game Design: Lessons from the Trenches

User Experience 6: Qualitative Methods, Playtesting and Interviews

IGDA NL Game Jam at Sci Fi on the Rock 2012

LAFS Game Design 8 - Playtesting

How Neuroscience revolutionizes User Research / Philipp Zent (Brainamics GmbH...

Unit 3: Research - Introduction

Game Development as A Business

Summerschool p6, user of playtesting

Informing SoTL using playtesting techniques

Make Your Games Play, Teach, and Socialize Better: Usability & Playability Te...

IGDA NL Sci-Fi on the Rock Game Developer's Jam 2013

Research methodology

Game Design 2: UI in Games - Revision Lecture

GAMIFIN 2019 Conference Keynote: How to fail at #gamification research

LAFS SVI Level 3 - Game Design and Analysis

Playtestix, Services Overview

New Microsoft PowerPoint Presentation.pptx

Exam resource (new)

Designing a Studio For Game Designers | Luna Javier

More from Mindtrek

What the AI revolution means for Open Source, Open Tech and Open Societies

Data balance sheets laying foundations for sustainable and ethical use of data

Towards data responsibility - how to put ideals into action

Täytä velvollisuudet ja hyödynnä mahdollisuudet – käytännön työkaluja regulaa...

Datatalouden ja tekoälyn regulaatio – missä mennään?

Green ICT Tools for Sustainable Digitalization

Future-proof digitalization is on full speed – are you on board?

How to (Help to) Save Our Planet with Green Coding

The document discusses how software and coding can help reduce greenhouse gas emissions and energy consumption. It notes that the ICT sector accounts for 4-10% of global energy use and growing. Practical solutions proposed include reducing the amount of data transferred by using caching and compression, removing unnecessary code, and improving coding efficiency by focusing on optimizing hotspots. The key message is that reducing energy use in software is an ongoing journey that requires incremental improvements over time.

National Library of Finland - open source solutions in the development of nat...

The Case for Open Source in the Public Sector

KEYNOTE: From Lutece to CiteLibre, City of Paris' commitment to open source

Freedom & Functionality – A Startup Approach to Open Source & Innovation for ...

Open source and open innovation are good for business by allowing freedom and functionality. Freedom means the ability to explore, experiment, and choose the best technology and business models without artificial limits. Functionality means technical quality, features, and design that provide tangible business benefits. An open source approach balances products that are standard and cost-efficient with services that uniquely create value. This startup-like approach leverages open innovation, open source development, and fair business models to efficiently co-create value through design thinking and an ambidextrous organization.

How open source empowers startups to start big, with case Double Open Oy

Sustainable Open Source; Balancing Business and Community

Empowering Employment: The Swedish Public Employment Service’s digital transf...

KEYNOTE: How to automate the world the open source way

"Perspectives from the EU level" by Henna Virkkunen

"Sand battery and other new energy concepts by Vatajankoski" by Pekka Passi

"Finnish National Rural Network: Support framework for Smart Villages" by Sal...

"Smart Villages in Finland" by Marianne Selkäinaho