More Actions than Inputs: A Game Usability Study


Published on

How easy is it to adopt a control scheme of a game that has 74 available actions and only 14 inputs?

  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

More Actions than Inputs: A Game Usability Study

  1. 1. Design of Experiment – Don W. Lail, Jr. & Chris Roe Data Collection – Don W. Lail, Jr. (Based on on-campus availability.) Data Analysis – Don W. Lail, Jr. (Quantitative), Chris Roe (Qualitative) Presentation & Review – Don W. Lail, Jr. & Chris Roe
  2. 2. Out of 25 respondents to the original survey, 15 made appointments to set up a trial and actually play the game. Subjects were all given the same introduction and were allowed the same five minutes of free-play time before their ten-minute trial. The context of the trial included an area that encompassed a village and hill that were both guarded by enemy forces. In order to complete the objectives, the subject had to move through the environment and deal with/hide from enemies.
  3. 3. Practice Time (Subjects were given 5 minutes to free-play in this environment. They were allowed to move through the environment in any way and experiment with the controls/actions of the game. Should they reach the end of the test area, the practice time would end.) Diagnostic Objectives (achieved during practice time): 1. Lie down and change camouflage. 2. Equip the Solid Eye and change to NV and then back to normal. 3. Change between Mk2 pistol and stun knife (and back).
  4. 4. Play-Test (During this time, subjects were asked to complete 9 objectives within 10 minutes. They were given the opportunity to see someone else play through once before doing it themselves.) Test Objectives (achieved during the play-test): 1. Shoot an enemy soldier at range. 2. Take down an enemy soldier in close-quarters-combat (fist or knife). 3. Purchase a special bullet for the Mk2 in the shop and shoot someone with it. 4. Reach a difficult to get item by grabbing a ledge while rolling (two choices). 5. Move a dead, sleeping, or knocked-out body to an inconspicuous location. 6. Throw an empty magazine to make a guard look the other way. 7. Sneak around/past an active enemy. 8. Move to the next area without causing an alert. 9. Make it to the end of the level or time limit without dying.
  5. 5. All subjects were required to use the same hardware/software to complete the trial. A Playstation 3 with a Dualshock 3 wireless controller were used in conjunction with a 50-inch HDTV which the subjects used to play the introduction to the second chapter of the game Metal Gear Solid 4: Guns of the Patriots. Subjects were screened during the survey process to ensure that they were currently a Purdue Calumet student.
  6. 6. • There are more actions available in Metal Gear Solid than buttons/inputs available on the controller (input device). • Metal Gear Solid 4 utilizes context and button/input combinations to provide 70+ actions on a 17-button controller. • Is this effective? • Can players be reasonably expected to adapt to and understand this solution? • Does this solution apply well to the target audience? • Who is the target audience?
  7. 7. Genre: Metal Gear Solid 4 is commonly considered an action/adventure game. Target Market: Action Gamers Adventure Gamers Shooter Gamers Role-Playing Gamers *Metal Gear Solid defines itself as “Stealth Action”, or more specifically “Tactical Espionage Action”. So, we are comparing usability between: (Source: Genre: Action Games Perspective: 1st-Person Perspective, 3rd-Person Perspective Themes: Stealth, Shooter, Spy/Espionage So, First-and-Third-Person, Non-Shooter Action Games and First-Person, Shooter Action Games. Category 1: First/Third Person, Non-Shooter Action Games Category 2: First-Person, Shooter Action Games
  8. 8. All subjects must identify themselves by their experience with action and FPS games. • Subjects affiliated strongly with FPS (more than two-thirds of Category 1, less than one third of Category 2) will be grouped into the FPS trials. • Subjects affiliated strongly with Action (more than two-thirds of Category 2, less than one third of Category 1) will be grouped into the Action trials. • Subjects that do not affiliate with either category (less than one third of both categories) will be grouped into the Inexperienced trials. Subjects were surveyed before the trial to determine their past experiences. The full trial consisted of: An outline of trial objectives for the subject’s review A live demonstration by another player A five-minute free-play time before their ten-minute trial
  9. 9. The definitions of action-game-player and FPS-player changed due to the fact that none of the 25 survey respondents or the 15 participants in the trial fit in the action or FPS categories as defined. When looking at the data in the review of the experiment, we looked at these factors as ratios/percentages instead of distinct categories.
  10. 10. In order to prove the effectiveness of Metal Gear Solid 4’s solution to the action/input problem, we will make the following hypotheses: • • • HO : Previous game experience does not affect performance on MGS4 trials. HA1 : Previous game experience negatively affects performance on MGS4 trials. HA2 : Previous game experience positively affects performance on MGS4 trials. More specifically, we could define variables as: • • • SP : Shooter Performance (Average number of objectives accomplished per FPS trial) AP : Action Performance (Average number of objectives accomplished per action trial) UP : Un-Experienced Performance (AVG # of objectives accomplished per other trial)
  11. 11. Unfortunately, it is impossible to directly compare these values in our hypotheses. Therefore, it is necessary to compare these to a pre-established, arbitrary number. It would be preferred to use the same value across hypotheses in order to make comparisons easier in recommending further study, or deducing implications of the data. • • HO1 : UP ≤ 60% (Un-experienced players will accomplish less than 60% of the objectives.) HA1 : UP > 60% (It would be interesting to find that un-experienced players can accomplish over 60% of the objectives.) • • HO2 : SP ≥ 60% (FPS players will accomplish more than 60% of the objectives.) HA2 : SP < 60% (It would be interesting to find that FPS players can accomplish under 60% of the objectives.) • • HO3 : AP ≥ 60% (Action game players will accomplish more than 60% of the objectives.) HA3 : AP < 60% (It would be interesting to find that action game players can accomplish under 60% of the objectives.)
  12. 12. The look and placement afford behaviors other than those intended. Does Metal Gear Solid 4 afford behaviors other than those intended? Absolutely. This is ultimately the point of the game. Let’s look at an example:
  13. 13. The look and placement afford behaviors other than those intended. You are approaching a guard from behind. If he spots you, he will raise an alarm. He hasn’t noticed you yet, but as you get closer your footsteps have a higher chance of giving you away. He stands about five feet in front of you and acts as if he is going to turn around. Which of these actions do you perform?
  14. 14. The look and placement afford behaviors other than those intended. • • • • • • • • • • • • • • • • • • • • • • • • • • Shooting Punching Accessing an item menu Accessing a weapon menu Calling for advice over a radio Kneeling Crawling Climbing over/onto objects Crawling into/through small spaces in first-person view A "punch, punch, kick" style melee system Grabbing a guard to throw them Grabbing a guard from behind to choke them Looking around in first person view Shooting while kneeling, crawling, or running Knocking on walls (used to just be punching toward a wall) Leaning on walls (while standing or crouching) Moving along walls (while standing or crouching) Seeing around corners while pressed against a wall Accessing elevators, computers, or other devices Shooting while riding in a vehicle Hanging over a ledge Doing a chin-up while hanging on a ledge Shooting in first person view Leaning in first person view Leaning out from a corner while pressed against a wall Jumping out from a corner to shoot • • • • • • • • • • • • • • • • • • • • • • • Executing a forward roll Holding up a guard Grabbing a guard from behind and to drag them Grabbing a guard from the front Grabbing a guard and disarming them Grabbing a guard and using them as a shield Grabbing a guard and interrogating them Grabbing a guard and cutting their throat with a knife Throwing a guard on the ground and holding them up as they lay down Shooting while hanging from a ledge (in first or third person mode) A system for selecting up to 8 weapons at a time to use in a sub-menu A system for selecting up to 8 items at a time to use in a sub-menu A camouflage system that requires the use of face paint and clothing to blend in to the environment Slow "stealth" walking and crawling Using a knife Moving while aiming in first-person view Rolling in a drum/barrel Tapping someone laying down on the back to retrieve items or wake them up Controlling a small robot that can move and attack Crawling on your stomach or back (at multiple speeds) Shooting while laying on your back Stabbing an enemy with a sensor to track their movements Walking while crouched
  15. 15. Does not fit to what was expected. (Example: Proximity to similar functions.) Players consistently found it surprising that they could not climb over waist-high obstacles. Players had difficulty understanding that an additional button needed to be pressed to grab onto a ledge while falling. Players had difficulty understanding why equipping an item required three separate actions. (Much confusion over which menu did what.)
  16. 16. What you do and what you get look/feel similar. Everyone achieved the objective to shoot a soldier at range. The L1+R1 button combination felt appropriate enough for everyone to figure it out easily. Because of the low difficulty level, a soldier would fall asleep after being hit by a sleeping dart at any spot on the body. Most subjects felt satisfied and that the response was appropriate, but some high-level players found it odd. All subjects were able to tell immediately when they had been spotted, though most of the time they didn’t seem to be sure why.
  17. 17. Planning and decision-making under stress. Subjects found the looming time limit to be difficult, and were consistently forgetting objectives altogether because of the pressure. After attempting an objective once or twice, players would sometimes skip to the next objective on the list and never return (even if they had time left or had much better opportunity than before).
  18. 18. Ability to perceive color, detail, distance. Subjects were shown how to use a night-vision item to see in low-light areas. Several locations in the game were so dark that no detail could be seen. Players often made mistakes in these locations because they couldn’t clearly tell the distance between the player character and an enemy, or the direction either was facing. A single color scheme was used for the whole environment, making everything appear to be a shade of blue/green. Some subjects had a hard time spotting enemies moving at a distance because they blended in to the background.
  19. 19. Alarms, false alarms, occupational noise. The alert noise did an excellent job of letting subjects know when they had been spotted. False alarms: Subjects often heard enemies saying things like “enemy spotted” in reaction to non-player characters on the map. Sometimes subjects took this as a sign that they had been spotted. Gunshots from an ongoing battle between enemies and other non-player characters often caused inexperienced subjects to immediately take cover and wait for the fighting to end, even though it did not have anything to do with them.
  20. 20. Top-down processing vs. bottom-up processing. Depth cues. Top-down processing – Subjects familiar with first-person shooter games felt accomplished and closer to completing objectives when they shot everyone in the area. Only after doing so did they realize that some of the objectives could no longer be completed. Bottom-up processing – It was easy for most subjects to see the relation between weapon and item menus because the item menu (on the left) was accessed with the left trigger, and the weapon menu (on the right) was accessed with the right trigger. Once one was learned, both were known and remembered.
  21. 21. Pop-out effect. Gestalt organizing principles: proximity, closure, continuity, symmetry, similarity. Pop-out effect – Soldiers were sometimes difficult to differentiate from the background. Subjects who preferred shooting often left night-vision active to help distinguish soldiers from the environment. Symmetry – The weapon/item menus were easy to understand. Proximity – The process of equipping an item was difficult because the buttons and menus were disconnected/discontinuous. Similarity – The pause menu offered an item menu, weapon menu, and shop which all looked remarkably similar. This confused subjects to the point where many gave up using them to achieve an objective. Also, enemy and friendly soldiers were nearly impossible for subjects to differentiate due to their similar appearance.
  22. 22. Storage of information; short (30-90 seconds) and long (nearly permanent) term. Many subjects forgot objectives along the way even after having reviewed them on paper and seen them performed. Some even forgot after having attempted an objective and failing. Many subjects forgot how to use the shop and weapon selection inside the pause menu, while they did not experience difficulty using the on-screen item/weapon menus. In the second half of the level, subjects could never remember the appropriate path to scale the hill at the end. Some subjects experimented with the same incorrect route multiple times.
  23. 23. Selecting one choice from a number of choices involving some level of uncertainty. Most subjects attempted the objectives in order. Almost all subjects attempted to grab a ledge in the same location, even though there were two choices. Subjects had the opportunity to select from four different special bullets to achieve the 3rd objective. The bullets accomplished generally the same thing, though in different ways.
  24. 24. Shortcuts that aren’t guaranteed to lead to the best answer, but are more efficient. Subjects often chose to start shooting all the enemy soldiers they could find because they felt it would make their objectives easier (despite the fact that some objectives required the soldiers to be alive). None of the subjects attempted to use any of the other items or weapons provided.
  25. 25. Skilled behavior which can extract, integrate, assess, and act upon stimuli. One piece of equipment—the Solid Eye—which was shown to subjects during their free-play time showed a motion-radar in the top corner of the screen which helps players identify the locations of nearby enemies. No one showed that they noticed it at all. During each trial in which the subject was spotted and caused an alert, there was an instance where an enemy that the subject saw before had now moved to a location that the subject did not expect. No one seemed to be keeping track of enemy movements. Many subjects did not notice when enemy soldiers woke each other up (after being incapacitated), even when there was dialogue heard and displayed onscreen.
  26. 26. The following is the most relevant data collected during our project.