X-Construction Lite: HCI evaluation                                               Title                                   ...
Title Text                                                Screenshot of page from                                         ...
Overview of gameplay           First screen of instructions (animates)       Second screen of instructions                ...
Evaluation approach 1. Expert review   1.1. Cognitive walkthrough, written up   1.2. Evaluation vs Nielsen heuristics 2. U...
User evaluation 8 test subjects recruited  (all Android owners, familiar with UI)Tom Hume   Human-Computer Interaction, as...
Results of cognitive walkthrough                                   Issue                                  # reports      Z...
Results of heuristic evaluation                       Agreement                                  Disagreement       Match ...
Results of quantitative data                                     300    Seconds to complete first level                    ...
Results of questionnaire data                Age of test subjects                               Female         Male   40  ...
Results of user testing                                   Issue                      # users      Misunderstood girder pla...
Key issue: misunderstood girder placementTom Hume   Human-Computer Interaction, assignment 1
Key issue: mixed up replaying/restarting levelTom Hume   Human-Computer Interaction, assignment 1
Conclusions Good news • Most players complete the first level, first time • Most players enjoy the game more than they thoug...
Recommendations 1. Warn players on the first level when they place a girder which doesn’t finish attached to an end-point; 2...
Critique of methods  Hard to record video in sunlight  Users were self-selecting, to a degree  Slight inconsistencies in q...
Questions?Tom Hume   Human-Computer Interaction, assignment 1
Upcoming SlideShare
Loading in …5
×

X-Construction Lite: HCI Evaluation

3,145 views

Published on

Presentation of a usability study a group of MSc students at Sussex University (including myself) carried out on the Android game, X-Construction Lite

0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
3,145
On SlideShare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
6
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

X-Construction Lite: HCI Evaluation

  1. 1. X-Construction Lite: HCI evaluation Title Andy Keavey, Mariana de Rojas-Morao, Merve Yildirim, Queen Ajifa Ododo, Tom Hume TextTom Hume Human-Computer Interaction, assignment 1
  2. 2. Title Text Screenshot of page from Android MarketplaceTom Hume Human-Computer Interaction, assignment 1
  3. 3. Overview of gameplay First screen of instructions (animates) Second screen of instructions Game-play screen Results of play screenTom Hume Human-Computer Interaction, assignment 1
  4. 4. Evaluation approach 1. Expert review 1.1. Cognitive walkthrough, written up 1.2. Evaluation vs Nielsen heuristics 2. User testing 2.1. Pre-test questionnaire 2.2. User test of first level (video recorded) 2.3. Interview with user whilst reviewing recording 2.4. Post-test questionnaireTom Hume Human-Computer Interaction, assignment 1
  5. 5. User evaluation 8 test subjects recruited (all Android owners, familiar with UI)Tom Hume Human-Computer Interaction, assignment 1
  6. 6. Results of cognitive walkthrough Issue # reports Zoom functions misused ✓✓✓ Errors placing girders ✓✓✓ Misunderstanding help messages ✓✓✓ Unsure how to proceed ✓✓ Game crashed on start ✓ Problem starting play ✓ Confused by pop-up adverts ✓ “I successfully drew some girders, but I struggled a bit because they were going “Some of my girders went out of view at the wrong direction most the top of the screen. I used the zoom out of the time” gesture consistent with Google Maps and could see a much better view of the task”Tom Hume Human-Computer Interaction, assignment 1
  7. 7. Results of heuristic evaluation Agreement Disagreement Match between system & real world User control & freedom Consistency & standards Error prevention Flexibility & efficiency Recognition over recall Aesthetics & minimalismTom Hume Human-Computer Interaction, assignment 1
  8. 8. Results of quantitative data 300 Seconds to complete first level 225 150 75 0 0 1 2 3 4 # attempts to complete first levelTom Hume Human-Computer Interaction, assignment 1
  9. 9. Results of questionnaire data Age of test subjects Female Male 40 30 25% 20 10 75% 0 8 7 5 4 6 1 3 2 Estimated vs actual fun Android gamer Not android gamer 8 6 4 38% 2 63% 0 1 2 3 4 5 6 7 8 Estimated ActualTom Hume Human-Computer Interaction, assignment 1
  10. 10. Results of user testing Issue # users Misunderstood girder placement ✓✓✓✓✓✓ Didn’t understand help/instructions ✓✓✓✓✓ Mixed up replaying/restarting level ✓✓✓✓ Couldn’t get the train started ✓✓✓ Panned unintentionally ✓✓ Couldn’t tell they were complete ✓ Couldn’t find girder removal tool ✓ Confused by an advert ✓ Misunderstood tool instructions ✓ Couldn’t find instructions again ✓Tom Hume Human-Computer Interaction, assignment 1
  11. 11. Key issue: misunderstood girder placementTom Hume Human-Computer Interaction, assignment 1
  12. 12. Key issue: mixed up replaying/restarting levelTom Hume Human-Computer Interaction, assignment 1
  13. 13. Conclusions Good news • Most players complete the first level, first time • Most players enjoy the game more than they thought • Many expert evaluators are now regular players Bad news • 75% players, 3/5 experts had issues with placing girders • 62% players don’t read help, 3/5 experts misunderstand it • 50% players mix up replaying and restarting the level • 37% players, 2/5 experts have trouble starting their train • 20% players pan unintentionally, 3/5 experts had zoom issuesTom Hume Human-Computer Interaction, assignment 1
  14. 14. Recommendations 1. Warn players on the first level when they place a girder which doesn’t finish attached to an end-point; 2. Clarify language in help messages and screens; 3. Improve the iconography, labelling or placement of the “replay failure” and “restart level” icons to reduce confusion between them; 4. Teach players to use the “play” button by animating it on the first level when they have a workable bridge; 5. Allow pinch-and-zoom to zoom into the bridge during play.Tom Hume Human-Computer Interaction, assignment 1
  15. 15. Critique of methods Hard to record video in sunlight Users were self-selecting, to a degree Slight inconsistencies in questionnaire usageTom Hume Human-Computer Interaction, assignment 1
  16. 16. Questions?Tom Hume Human-Computer Interaction, assignment 1

×