Games User Research is for Game Design!

Games User Researchis forGame Design!Marina Kobayashi, Electronic ArtsSarah Romoslawski, Sifteo

Who are we?SarahRomoslawski Marina Kobayashi Game Designer Lead Games User Researcher

Who are we?SarahRomoslawski Marina Kobayashi

OverviewIntrosWhat is games user research(GUR)?How does GUR turn game design into GREAT gamesApplying games user research to game design Platforms Genres Target users (age, lifestyle)Our conclusionsQ&A

From two perspectives: vs. What a mess!

Who are you?It is ok to raise your hand more than once! • Game Designer • Artist • UX/UI Designer • Programmer • Producer • Wearer of many hats • I’m just a unicorn

There is always a player in a game. Gamesare made for people by people.

How do gamesbecome greatgames?Answer:by doing GamesUser Research!

Games user research is a feedback loop forgame designers to understand how usersexperience their games.

GUR + Game Designer = CollabGenius Lather Rinse Repeat

When Should I do GUR? Concept Early Full Alpha Post Phase Production Production Phase ReleaseCollaging/ White Box Vertical Slice Level Tests E3/XBL Demo Full Play Benchmark DLC Tests Tests MP Tests Tests Through Tests Tests Tests All stages of the dev cycle!

From early stages…

Combat Mechanics Test Test EARLY and OFTEN: White Box Testing

To full playthroughs…

Full Playthrough Test Average Player Experience (Via Survey Stops) 10.0 9.0 8.0 7.0 6.0 5.0 4.0 3.0 2.0 1.0 0.0 Fun Difficulty Frustration Start of End of Game Game

What do we learn about? FUNImpact Difficulty factor Ramping Usability in context slate of Puzzle of the new platform Content Early Character and Story Development

Corporate PerspectivePlatformGenreTarget Audience

Platforms,Genres,Gamersand GUR at EA Game Lab“Why do we fall sir? So we might learn to pick ourselves up. ” - Alfred, Batman Begins

Know Your Platform

All Platforms Have Affordances

All Platforms Have Affordances The quality of the device that lets you perform an action

All Platforms Have Constraints

All Platforms Have Constraints Small screens, dual screens, touch screens

All Platforms Have Standards

Testing Different Platformsat EAConcrete Examples: Kinect Voice for Mass Effect 3 and EA Sports Active: NFLTraining Camp

GUR on Kinect Voice inMass Effect 3+ 1:1 think-aloud (command aloud) + exploration & experimentation + usability

GUR on Kinect Voice for Mass Effect 3 Kinect voice ≠ natural language

GUR on Kinect Voice in Mass Effect 3 What would you say to control the game?

Video clips

Player 1 Ok, Mordin I’ll see what I can do… “…” Kill kill kill… Die die die…

Player 2 “Xbox Inventory?” Can she “Liara?” hear me? Hmm… I forgot the names of the powers “Liara, Lift Orb?”

GUR on Kinect Voice in Mass Effect 3 What would you say to control the game? ExpectedCommandsUsability of VoiceCommands

GUR on Sports Active Games

“I have to wear what?” Usability Ergonomics

So how do I use this thing?

So how do I use this thing? 8 Tutorials Opinion 8 8 7 7 7 7 7 6 6 6 6 5 5 5 4 4 4 3 3 3 2 2 2 2 2 1 1 1 1 0 0 0 0 0 0 0 0 0 0 0 Liked tutorial video Tutorial video needs improvement Players commented that they would like to see There were two mentions that holding the more of a step-by-step breakdown of the exercises. Wii-motes felt awkward, that they were not sure One player also mentioned not knowing they were how to hold them or that the exercise required supposed to observe the trainer doing the exercises. using them at all.

How’d you do?

GUR and Genre at EAConcrete Examples: different methods for different games

Why we used different methods forBF3 Multiplayer vs. Warp?

Why we did different methods for BF3Multiplayer vs. Warp?First-Person Shooter Puzzle Platformer

GUR for theFirst-Person Shooter+ 1:1 think-aloud usability tests for Single Player and Multiplayer+ Dyad usability tests for Co-op+ Multiplayer playtests (over 32 players in-game simultaneously)+ Full playthrough tests+ PPT screen walkthroughs+ Flash Prototype tests

Some observations are best made with live gameplay and think-aloud.

GUR for thePuzzle Platformer+ 1:1 think-aloud usability+ Full playthrough

Warp [Puzzle Platformer]

GUR for Puzzle PlatformersWarp

GUR for RPGsHow to test a game that has 160 hours of content?+ 1:1 think-aloud usability+ Extended Playthrough+ Combat Encounter Tests+ Diary Studies

Dragon Age II

GUR for Facebook RPGsHow to test a game that might be played in 5-10 minuteincrements?+ 1:1 think-aloud usability+ Diary Studies

Dragon Age Legends

Dragon Age Legends Dragon Age Legends

Diary Study

GUR on targetaudience~OR~Figuring out who the right gamers are and how to get themin the door.

But who is Mr. Right - for this playtest?

Does Mr. Right check offall these boxes?

Wait you’re giving out incentives?

Phone Screening“Good answer.”

Why getting the right players matters?

Franchise Fans vs. Newcomers Shooting Target Sliding Shooting Shooting Weak Time #Deaths 1 (max # cycles ) (cyc) Spots (min)New PlayersPair 1 8 4 4cyc. 3rd cyc. 3New PlayersPair 2 10 2 5cyc. 1st cyc. 6Fan Pair 1 10 3 3cyc. 1st cyc. <1Fan Pair 2 5 1 4cyc. 1st cyc. <1Average 8.25 2.5 max 4cyc.

Why getting the right players matters?4-player co-op campaign

Why getting the right players matters?When testing a 4-player co-op campaign: “Be the best you can be!” “I will pwn you!”

Why getting the right players matters? Melee DPS Tank Healer / Tech Ranged DPS4-player co-op campaign

Why getting the right players matters? Melee DPS Tank Healer / Tech Ranged DPS “I really like the idea behind the mag-shield. I usually play tanks/defenders aside from assault classes and this “I really enjoyed allowed me to police the playing Izzy. I front lines while my frequently choose teammates flanked and healers in MMOs or sniped from cover. Definitely group/co-op games fits my play style.” and thought she was a great character to start off with.”

Grassroots PerspectivePlatformGenreTarget Audience

Wait, What is Sifteo?

Platforms,Game Design,and GUR at Sifteo(or in other words) “neighbor” isn’t a verb

The Need for the Sifteo PlayLab Original Sifteo Cubes Ok so now let’s find some people to play…

The Need for the Sifteo PlayLabLess Off Site to More On Site Testing! Ok so now let’s find some people to play…

Design in Parallel Wait, you want me to drive on that?

Creating Standard Visual Language Neighbor isn’t a verb

What we do at the PlayLab:MeasureFunthroughPlaytesting

How does Sifteo Measure Fun? Fun = Learning (Koster, A Theory of Fun) tinkering FUN of MASTERING new skill Frustration, the game doesn’t teach me!

All Platforms Have Affordances The quality of the device that lets you perform an action

Affordances – Spatial reasoning and CubeBuddiesSifteo Cubes AFFORD for spatial reasoning puzzles, but spatial skills arehoned throughout life.

Game Play

Zorg’s EYE Problem

Really Mixed Up Faces

Almost Done, But Eyes Wrong

The player would get to here andSTOP

What Skill Were We Forgetting to teach?

We didn’t teach aneye for an eye!

Solution: Teach How to Swap UpsideDown Eyes

All of the Above -- Sifteo Example Dealing with platform CONSTRAINTS, AFFORDANCES and STANDARDS on Word Caravan

WC Game Play

Why WC?• Must be fun for 7-12, the platform requires this as a STANDARD for all games

Why WC?• Must be fun for 7-12, the platform requires this as a STANDARD for all games• Must improve our original word game, Word Play, yet still be playable on 3 or more cubes, a design CONSTRAINT of the platform

Why WC?• Must be fun for 7-12, the platform requires this as a STANDARD for all games• Must improve our original word game, Word Play, yet still be playable on 3 or more cubes, a design CONSTRAINT of the platform• Shuffling letters to make words exploits the AFFORDANCES of the platform

Why WC?• Must be fun for 7-12, the platform requires this as a STANDARD for all games• Must improve our original word game, Word Play, yet still be playable on 3 or more cubes, a design CONSTRAINT of the platform• Shuffling letters to make words exploits the AFFORDANCES of the platform• ALL OF THE ABOVE due to the platform resulted in the creation of an appropriate hint mechanism imited inputs and screen space

Pre-existing Models ofWord Games

Problem(s)• Game gets hard too quickly• Players get stuck too early in the game• Players naturally want to apply models of other word games•BUT we didn’t want to make it too easy for the older age groups Everyone I know has a big but…

Too much frustration, too little fun FUN of MASTERING tinkering new skill Frustration, the game doesn’t teach me!

Will pattern recognition help?•Block, lock•Lock, unlock•Date, update•Make remake•Act, fact, facts, factor, artifact

We simplified the puzzles BAT FIX WORD

What do kids do when they can’t reada word?

Solution: Hint MechanismHave the game be the parent! SCAFFOLD the player.

Iterate, test, iterate, test, iterate, test…• Earned hint, tracked with icon• Automatic hint• Increase the size of the hint icon• Delayed hint Older version, Not noticeable

Hypotheses vs. Actual …well I guess not this time…

Final Implementation – InfiniteDelayed Hints, Large Icons

Developer On Site! Children often do not report seeing animated elements that appear on one or more of the screens, Even if the elements are high contrast and cover 1/64 of the total screen area. Increasing the size to 1/16 total screen area seemed to solve this problem. Chris March, Lead on Word Caravan

GUR and Genre at SifteoDesigning task oriented 1:1 sessions for the Sandwich Kingdom RPG

Sandwich Kingdom RPG

Task Oriented, 1 or 1:2, Think Aloud

Designing a Script: Quest or TaskFocused I am… vs. Pearl is… Yes, yes with help, or no

PLAY THE BUILD

WHERE DID I GET CONFUSED?

Early Feature Test: Dangerous Pits

Feature Test: Dangerous PitsNo NPCsNo Final Art

So we use similar methods over allgames

But we ask different questions in theend

Acquiring Actionable Data fromKids~OR~when the chocolate covered pretzels are more interesting thanyour game, you might have a design problem

Informed Consent, NDA and PhotoRelease

The Researcher’s Role

Facilitate Play We want you to help us find problems! We want you to tinker!

Facilitate Communication Teach each other how to play.

Play with Parent

Everyone wants to feel smart, evenkids!

Sifteo: Longitudinal StudiesConcrete Example -- Collecting Data With Kids on the ~Entire Puzzle Game~

No Stats Saved on HW at this Point.• So how do we know how long it takes?• How many tries to solve the puzzle?• Where is the player successful?• Hrmmm…• We can do this!

Solution: Pizza and Games•IT’S SUMMER!•2-3 x 3hr sessions•Lunch included (pizza)•Gift card for the parent•Certificate of completion•Did I mention these were 3 - 4 hr sessions with kids 14 and under?

Testing Chroma Splash Puzzle Mode

Game Play

Goal:Puzzle Difficulty Ramp Up

A Chroma Splash Puzzle

What about adults?Sometimes you run out of kids…BUT you still need data!

Kids and Adults are Different My biggest surprise was how reticent older adults seemed to be to experiment with the game. That resulted in certain adults taking far longer to complete puzzles than kids did. Scott Kim, Chroma Splash

Actionable Data can be Tricky wanted a Some testers perfect solver that would fail the user in any situation where they could not proceed. This seemed dangerous and very difficult, so I did not attempt it. Eric Liao, Chroma Splash

Key Takeaways

GUR is your hydrant to feedback…enjoy it!

Thank you!Marina Kobayashi Sarah Romoslawskimkobayashi@ea.com sarah@sifteo.com

http://www.linkedin.com	63
https://twitter.com	5
http://pinterest.com	2

Games User Research is for Game Design!

by Marina Kobayashi on Sep 19, 2012

Accessibility

Categories

Upload Details

Usage Rights

3 Embeds 70

Statistics

Games User Research is for Game Design! Presentation Transcript