Cheap & Quick Internal User Testing
                    - Flirtomatic’s Recent Challenge -




                           ...
Quick Introduction


- BA in Education from SNU
- MSc in Information Technology from QMUL
- PgDip in HCI from UCLIC
- MSc ...
Cheap & Quick Internal User Testing
                           – Flirtomatic’s Recent Challenge


                   1.   ...
1. Why Cheap & Quick?
                                                                                       1.1. Evaluati...
1. Why Cheap & Quick?
                                                                         1.2. Usability Firms


 - H...
1. Why Cheap & Quick?
                                                                     1.3. Self-Evaluation


 - Then ...
7
Mobile Design UK   © 2008 Handmade Mobile. All Rights Reserved.
Cheap & Quick Internal User Testing
                           – Flirtomatic’s Recent Challenge


                   1.   ...
2. Experiment Design
                                                                                                2.1. ...
2. Experiment Design
                                                                       2.2. User Testing


 - Normal ...
2. Experiment Design
                                                                       2.3. GUI Preference


 - Asked...
2. Experiment Design
                                                                       2.4. Focus Group Discussion


...
Cheap & Quick Internal User Testing
                           – Flirtomatic’s Recent Challenge


                   1.   ...
3. Good Lessons – Dos and Donts
                                                                       3.1. Outcomes


 - ...
3. Good Lessons – Dos and Donts
                                                                        3.2. Difficulties
...
3. Good Lessons – Dos and Donts
                                                                                   3.3. Do...
3. Good Lessons – Dos and Donts
                                                                                   3.4. Do...
3. Good Lessons – Dos and Donts
                                                                         3.5. Resources


...
Thank you

                          Jaeseung Chang
                   (jaeseung@flirtomatic.com)




                    ...
Upcoming SlideShare
Loading in...5
×

Cheap& Quick Internal user testing

809

Published on

Hwo soes one do user-testing on a shoe-string when incrementally improving a live service? Here's some pointers

Published in: Technology
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
809
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
15
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

Cheap& Quick Internal user testing

  1. 1. Cheap & Quick Internal User Testing - Flirtomatic’s Recent Challenge - 3 Dec 2008 Jaeseung Chang (jaeseung@flirtomatic.com) Usability Architect @ Handmade Mobile Entertainment Ltd. 1 Mobile Design UK © 2008 Handmade Mobile. All Rights Reserved.
  2. 2. Quick Introduction - BA in Education from SNU - MSc in Information Technology from QMUL - PgDip in HCI from UCLIC - MSc by Research in Computer Science from QMUL - Open Education (Korea) / Usability Designer - Samsung Electronics (Headquarter in Korea) / Mobile Usability Designer + Researcher - Handmade Mobile (Flirtomatic) / Mobile Usability Architect 2 Mobile Design UK © 2008 Handmade Mobile. All Rights Reserved.
  3. 3. Cheap & Quick Internal User Testing – Flirtomatic’s Recent Challenge 1. Why Cheap & Quick? 2. Experiment Design Good Lessons – Dos and Donts 3. 3 Mobile Design UK © 2008 Handmade Mobile. All Rights Reserved.
  4. 4. 1. Why Cheap & Quick? 1.1. Evaluation Needs - We all know the necessity of usability evaluation - At least 2 times per 1 iteration cycle - But the reality sometimes doesn’t help Requirement Fast Analysis Prototyping Evaluation Evaluation Specification Implementation 4 Mobile Design UK © 2008 Handmade Mobile. All Rights Reserved.
  5. 5. 1. Why Cheap & Quick? 1.2. Usability Firms - Hiring professional research firms? - Excellent facilities, professional experiment design, reliable recruitment, & nice report (maybe?) - But costs a fortune! 5 Mobile Design UK © 2008 Handmade Mobile. All Rights Reserved.
  6. 6. 1. Why Cheap & Quick? 1.3. Self-Evaluation - Then we should go DIY - Venue: How about using our meeting room? - Experiment design: We can do that maybe? - Recruitment: Find some volunteers from our users? - Observation and analysis?: 15 day trial version MORAE! 6 Mobile Design UK © 2008 Handmade Mobile. All Rights Reserved.
  7. 7. 7 Mobile Design UK © 2008 Handmade Mobile. All Rights Reserved.
  8. 8. Cheap & Quick Internal User Testing – Flirtomatic’s Recent Challenge 1. Why Cheap & Quick? 2. Experiment Design Good Lessons – Dos and Donts 3. 8 Mobile Design UK © 2008 Handmade Mobile. All Rights Reserved.
  9. 9. 2. Experiment Design 2.1. Overview - The 1st iteration of our ‘Redesign Project’ - Based on wireframes (workflows) produced so far - Evaluation on our first redesign UI/IA works - To discover good/bad points for more future works - Included three parts: UT + GUI Preference + FGD + + GUI Focus Group User Testing Preference Discussion 9 Mobile Design UK © 2008 Handmade Mobile. All Rights Reserved.
  10. 10. 2. Experiment Design 2.2. User Testing - Normal user testing techniques (think-aloud) - Went through 6 sample tasks with 8 participants - Used a hi-fi prototype to simulate our new UI - Controlled experiment with video recording (MORAE) - 1 moderator and multiple observers CONFIDENTIAL CONFIDENTIAL 10 Mobile Design UK © 2008 Handmade Mobile. All Rights Reserved.
  11. 11. 2. Experiment Design 2.3. GUI Preference - Asked the same participants to choose our new GUI designs - A sort of multiple-choice survey questions - Asked to choose the best 2 & the worst 1 design CONFIDENTIAL 11 Mobile Design UK © 2008 Handmade Mobile. All Rights Reserved.
  12. 12. 2. Experiment Design 2.4. Focus Group Discussion - To collect real users’ preferences and behaviours - In terms of using the current version of Flirtomatic - Focused on gathering additional information - To help redesign the details of our WAP service - Went through 5 topics with 3 groups of users (m, f, & gay) 12 Mobile Design UK © 2008 Handmade Mobile. All Rights Reserved.
  13. 13. Cheap & Quick Internal User Testing – Flirtomatic’s Recent Challenge 1. Why Cheap & Quick? 2. Experiment Design Good Lessons – Dos and Donts 3. 13 Mobile Design UK © 2008 Handmade Mobile. All Rights Reserved.
  14. 14. 3. Good Lessons – Dos and Donts 3.1. Outcomes - Were able to conduct evaluation ourselves – DIY! - Captured some important findings to validate our prototype - Less than 2 wks (Experiment design + experiment + analysis) - Less than 500.00 GBP - Much quicker and cheaper than hiring research firms - Were able to share the importance of usability evaluation with the management level and all other departments - Self-learn about the process of usability evaluation 14 Mobile Design UK © 2008 Handmade Mobile. All Rights Reserved.
  15. 15. 3. Good Lessons – Dos and Donts 3.2. Difficulties - Recruitment error: No show up, recruited friends, etc. - MORAE error (due to the inexperienced operation): No video recording, low quality audio recording, etc. - Some mistakes of our colleagues: intruding into the UT room to connect to wireless network, interrupting UT sessions to ask questions to the moderator, giving money to participants during the session, etc. 15 Mobile Design UK © 2008 Handmade Mobile. All Rights Reserved.
  16. 16. 3. Good Lessons – Dos and Donts 3.3. Dos - Good preparation in every way: 1. Documentation on task scenario, forms, analysis 2. Train yourself to master any SW tools (like MORAE) 3. Clean up the venue in advance - Share the importance of usability evaluation with everyone 1. With the management level 2. Especially with colleagues nearby the UT room (should politely ask silence during each session) - Share the formal process of evaluation with team members - Educate yourself how to run each session professionally 16 Mobile Design UK © 2008 Handmade Mobile. All Rights Reserved.
  17. 17. 3. Good Lessons – Dos and Donts 3.4. Donts - Do not rely on the quick list of recruited users (importance of having some back-up participants) - Do not expect everyone else in office will behave as you do 1. They don’t know how important it is 2. Actually they don’t care what you do - Do not expect your team members understand everything 1. They don’t know how to formally run each session 2. Importance of pre-training for usability evaluation 17 Mobile Design UK © 2008 Handmade Mobile. All Rights Reserved.
  18. 18. 3. Good Lessons – Dos and Donts 3.5. Resources - MORAE: http://www.techsmith.com/morae.asp - The Usability Methods Toolbox by James Hom: http://jthom.best.vwh.net/usability - Usability First: http://www.usabilityfirst.com/ - Jakob Nielsen: http://www.useit.com/ 18 Mobile Design UK © 2008 Handmade Mobile. All Rights Reserved.
  19. 19. Thank you Jaeseung Chang (jaeseung@flirtomatic.com) 19 Mobile Design UK © 2008 Handmade Mobile. All Rights Reserved.
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×