Evaluation And Usability for Games


Published on

An overview of ways to integrate playtesting into game design, from concept to beta, based on theories from Human-Computer Interaction.

  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide
  • The rest of the presentation will detail each of these methods.
  • “ Mind what requires explanation” - the more you have to explain a feature, the more important it is to really focus on that area of UI. You won’t be there to explain to people who go through your game. Think about how you can tell them in the game, through the tutorial, backstory, or, preferably, the mechanics themselves.
  • A heuristic evaluation is a way to look at specific types of problems or to organize problems you already have. These problems should be graded on severity based on how frequently they appear, how big their impact is, how persistent the issue is, and how much of an aesthetic issue it is.
  • These are the 10 Usability Heuristics by Jakon Nielsen http://www.useit.com/papers/heuristic/heuristic_list.html These are the “categories” into which almost all problems fall. Visibility of system status The system should always keep users informed about what is going on, through appropriate feedback within reasonable time. Match between system and the real world The system should speak the users' language, with words, phrases and concepts familiar to the user, rather than system-oriented terms. Follow real-world conventions, making information appear in a natural and logical order. User control and freedom Users often choose system functions by mistake and will need a clearly marked "emergency exit" to leave the unwanted state without having to go through an extended dialogue. Support undo and redo. Consistency and standards Users should not have to wonder whether different words, situations, or actions mean the same thing. Follow platform conventions. Error prevention Even better than good error messages is a careful design which prevents a problem from occurring in the first place. Either eliminate error-prone conditions or check for them and present users with a confirmation option before they commit to the action. Recognition rather than recall Minimize the user's memory load by making objects, actions, and options visible. The user should not have to remember information from one part of the dialogue to another. Instructions for use of the system should be visible or easily retrievable whenever appropriate. Flexibility and efficiency of use Accelerators -- unseen by the novice user -- may often speed up the interaction for the expert user such that the system can cater to both inexperienced and experienced users. Allow users to tailor frequent actions. Aesthetic and minimalist design Dialogues should not contain information which is irrelevant or rarely needed. Every extra unit of information in a dialogue competes with the relevant units of information and diminishes their relative visibility. Help users recognize, diagnose, and recover from errors Error messages should be expressed in plain language (no codes), precisely indicate the problem, and constructively suggest a solution. Help and documentation Even though it is better if the system can be used without documentation, it may be necessary to provide help and documentation. Any such information should be easy to search, focused on the user's task, list concrete steps to be carried out, and not be too large.
  • If the Heuristic Evaluation is too much, here is a simpler way to categorize your tasks and the resulting problems.
  • Many of us have experienced the lackluster feedback from friends, family, and choice users. Surveys, like tasks, give testers specific areas to focus on.
  • For the presentation, we asked the class to do a heuristic evaluation of our game, Influence. Half the class simply played the game, the other half was given specific tasks to accomplish, which are shown on the next page.
  • Evaluation And Usability for Games

    1. 1. Usability and Evaluation How to get the most out of walkthroughs and playtesting Chelsea Howe ActionXL 11/03/09 huge thanks to Dan Cosley, HCI 3450
    2. 2. Why evaluate? <ul><li>Find bugs earlier </li></ul><ul><li>Address design issues </li></ul><ul><li>Increase user enjoyment (more fun) </li></ul><ul><li>Objective look at features and failures </li></ul><ul><li>Prioritize the big To Do list </li></ul><ul><li>Makes your game BETTER </li></ul>
    3. 3. How to evaluate? <ul><li>Depends on where you are </li></ul><ul><li>Different stages have different requisites </li></ul><ul><li>What you’re looking for changes </li></ul><ul><li>Stage </li></ul><ul><ul><li>Concepting </li></ul></ul><ul><ul><li>Prototyping </li></ul></ul><ul><ul><li>Internal Playtest </li></ul></ul><ul><ul><li>External Playtest </li></ul></ul><ul><ul><li>BETA+ </li></ul></ul><ul><li>Method </li></ul><ul><ul><li>Cognitive Walkthrough </li></ul></ul><ul><ul><li>Cog/Phys Walkthrough </li></ul></ul><ul><ul><li>Heuristic Eval/Nielsen </li></ul></ul><ul><ul><li>HE, Surveys, Observation </li></ul></ul><ul><ul><li>Surveys, Observation, Forum </li></ul></ul>
    4. 4. Tasks vs. Free play <ul><li>Giving evaluators “tasks” is useful in forcing people to explore an interface </li></ul><ul><li>Guided evaluation can focus people’s attention where you want it </li></ul><ul><li>Careful, tasks can be biased </li></ul><ul><li>Hardest issue is choosing right tasks </li></ul><ul><ul><li>Easier in games: Beat level 2, save, pause, defeat two droids, check high scores </li></ul></ul>
    5. 5. Cognitive Walkthrough <ul><li>You click the game icon on your PC and see a page with three buttons: play, settings, exit… </li></ul><ul><ul><li>State diagrams </li></ul></ul><ul><ul><li>Flowcharts </li></ul></ul><ul><ul><li>Control schematics </li></ul></ul><ul><li>Figure out UI functionality </li></ul><ul><li>Plan overall flow of experience </li></ul>
    6. 6. Physical Walkthrough <ul><li>Paper prototype </li></ul><ul><ul><li>Loose menu/UI layout </li></ul></ul><ul><ul><li>What fits on a page? </li></ul></ul><ul><ul><li>What’s not allowed? (Error messages?) </li></ul></ul><ul><li>Mind what requires explanation </li></ul><ul><ul><li>Not at stand alone point, but insightful </li></ul></ul><ul><ul><li>Can warn about future problem areas </li></ul></ul>
    7. 7. Heuristic Evaluation <ul><li>Many problems are universal and fall into consistent categories </li></ul><ul><ul><li>Approach 1 </li></ul></ul><ul><ul><ul><li>Look at categories, seek problems </li></ul></ul></ul><ul><ul><li>Approach 2 </li></ul></ul><ul><ul><ul><li>Find problems, organize with categories </li></ul></ul></ul><ul><li>Hone in on problem areas </li></ul><ul><li>Focus on most severe issues </li></ul><ul><ul><li>Frequency, impact, persistence, aesthetic </li></ul></ul>
    8. 8. Heuristic Evaluation Cont’d <ul><li>Simple and natural dialog </li></ul><ul><li>Speak the user’s language </li></ul><ul><li>Reduce user memory load </li></ul><ul><li>Be consistent </li></ul><ul><li>Provide feedback </li></ul><ul><li>User control and freedom </li></ul><ul><li>Flexibility and efficiency of use </li></ul><ul><li>Help users recognize, diagnose, and recover from errors </li></ul><ul><li>Prevent errors </li></ul><ul><li>Help and documentation </li></ul>Jakob Nielsen’s 10 Usability Heuristics http://www.useit.com/papers/heuristic/heuristic_list.html
    9. 9. Notes on HE <ul><li>Can be done by creators or observers </li></ul><ul><li>Each person should go through twice </li></ul><ul><ul><li>1 - figure out the general layout, understand the overall concept (“get” the mechanic) </li></ul></ul><ul><ul><li>2 - go through again checking for the heuristics </li></ul></ul><ul><li>For each problem, indicate location, type, severity, and possible fixes </li></ul><ul><li>Use this method to organize your To Do. </li></ul><ul><li>Finds many errors, types of errors, good! </li></ul>
    10. 10. Nielsen’s Guidelines <ul><li>Learnability : How quickly can users understand your main mechanics? </li></ul><ul><li>Efficiency : How effectively can users perform the tasks they want to do? (Make sure the challenge is in the right place) </li></ul><ul><li>Memorability : How easy is it to pick up the game again after an hour, day, month off? </li></ul><ul><li>Errors : How often do users make errors (is death an error?), how severe, and how easily can they recover? </li></ul><ul><li>Satisfaction : Do people like your game? </li></ul>
    11. 11. A note on evaluation <ul><li>Most of HCI is driven by EFFICIENCY </li></ul><ul><li>Games are not classic HCI targets </li></ul><ul><ul><li>Emotion, enchantment, flow </li></ul></ul><ul><ul><li>Experience vs. product </li></ul></ul><ul><ul><li>“ Intentional Inefficiencies” </li></ul></ul><ul><li>Watch reactions as much as actions. </li></ul><ul><li>Face vs. Screen </li></ul>
    12. 12. Observation <ul><li>Screen </li></ul><ul><ul><li>Unexpected behavior </li></ul></ul><ul><ul><li>Glitches/bugs </li></ul></ul><ul><ul><li>“ Think aloud” > “Why did you do that?” </li></ul></ul><ul><li>Face </li></ul><ul><ul><li>Frustration </li></ul></ul><ul><ul><li>Focus </li></ul></ul><ul><ul><li>Fiero! </li></ul></ul><ul><ul><li>Enchantment </li></ul></ul>
    13. 13. Surveys <ul><li>Why?! </li></ul><ul><ul><li>“ Cool game” </li></ul></ul><ul><ul><li>“ Yeah it was great” </li></ul></ul><ul><ul><li>“ I really liked it” </li></ul></ul><ul><li>Focus attention </li></ul><ul><li>Increase feedback </li></ul><ul><li>Open-ended questions </li></ul>
    14. 14. Forums <ul><li>Games benefit from dev/user discussion </li></ul><ul><li>Unity3D Free  iPhone  Playhaven </li></ul><ul><li>Get friends involved in the iterations </li></ul><ul><li>Find/fix bugs as they appear </li></ul>
    15. 15. Summary <ul><li>Different methods, different information </li></ul><ul><li>Tasks vs. Freeplay </li></ul><ul><li>Walkthroughs, cognitive and physical </li></ul><ul><li>Heuristics and design sensibilities </li></ul><ul><li>Efficiency vs. Experience </li></ul><ul><li>Observation, Surveys, Forums </li></ul>
    16. 16. Example Game: Influence <ul><li>Half the Class on Tasks </li></ul><ul><li>Half the Class on Freeplay </li></ul><ul><li>Who finds the most heuristics? </li></ul>
    17. 17. Influence - Tasks <ul><li>Pick an allegiance </li></ul><ul><li>Max the # boids in a game </li></ul><ul><li>Play as the violin </li></ul><ul><li>Have 6 adversaries </li></ul><ul><li>Play a multiplayer game with at least three other players </li></ul><ul><li>Record the maximum amount of time you survived a multiplayer game </li></ul><ul><li>Play a game in respawn mode </li></ul>