XWiki Usability Testing Sessions

1. Usability Testing 2014 Highlights & Findings Ecaterina Moraru — 26 July 2014 —

2. Why is good to do usability sessions? You confirm issues you've suspected Get amazed by new things you discover “ ” · ·

3. SCENARIO Scenario #4 PERIOD 27 May - 12 June 2014 VERSION XE 6.0 SKIN Colibri DURATION 9:45:45 h FEATURES Profile, Navigation, Pages, Wikis, Applications Usability Sessions 3/44

4. Tasks

6. Task 1

7. INSTRUCTIONS HYPOTHESES What are your first impressions of XWiki? What do you think you can do with it? You can scroll, hover, but please don't click on anything yet. Introduction Testing: Homepage Status: Not-logged-in Status: Logged-in Task 1 · · · · 7/44

8. Task 2

9. INSTRUCTIONS HYPOTHESES Imagine this is your first day at work and XWiki is used internally. You want to customise your profile and add some information about yourself. How can you do that? UC1 : Measure the performance of 'menu-button' vs. 'icon-link' (#tmEdit vs. .editProfileCategory) 9/9 'icon-link' usage Testing: Profile 9/9 Success Task 2 · · · 9/44

10. (video) Task 2 - Profile Editing Safari Power Saver Click to Start Flash Plug-in · Add hints to Profile Preferences · Specify default values 10/44

11. Task 3

12. INSTRUCTIONS SUBTASKS You've been given the assignment to add some content (text+image) for the Marketing team. Task 3.1 Task 3.2 Task 3.3 INSTRUCTIONS HYPOTHESES First let's browse a bit the product and locate where the Marketing material is found. UC2 : Measure the performance of Search, Search Suggest vs. Wiki Index 7/9 Search Suggest 2/9 Wiki Index Testing: Navigation 3/9 couldn't find Marketing without help Task 3 Task 3.1 · · · · 12/44

13. (playlist) No suggest for partial words No Wiki results in Search Pressing Enter too fast 3/9 Suggest worked in first try 4/9 Suggest failed in first try Task 3.1 - Navigation - Search Suggest Safari Power Saver Click to Start Flash Plug-in · · · · · 13/44

14. (video) Harder to find under menu No Wikis gadget on Homepage No Applications panel entry 1/9 Used only Wiki Index 2/9 Found it alone after Suggest fail Task 3.1 - Navigation - Wiki Index Safari Power Saver Click to Start Flash Plug-in · · · · · 14/44

15. Task 3.2

16. INSTRUCTIONS HYPOTHESES You need to create a page called 'XWiki Enterprise' (in the Marketing location). As a result, the page should look similar to: UC3 : In order to edit, you need to be a member of the wiki. Measure task success. UC4 : Measure 'Add image' dialog naming. UC5 : Measure table performances. Testing: Page Creation Testing: Page Editing Extended: wiki syntax Extended: macro usage Task 3.2 · · · · 16/44

17. Task 3.2 - Page Creation EXPECTED RESULT 0/9 expected result 3/9 created Wiki 2/9 edited Sandbox ACTUAL RESULTS · · · x2 x2 17/44

18. (video) Task 3.2 - Page Creation EXPECTED RESULT ACTUAL RESULTS Safari Power Saver Click to Start Flash Plug-in 18/44

21. (video) Task 3.2 - Page Creation ACTUAL RESULTS ACTUAL RESULTS Safari Power Saver Click to Start Flash Plug-in 21/44

22. Task 3.2 - Page Creation Challenges: Finding the Add button Creating a Page and not something else Creating the page in the right location Issues: Problems finding the Add button Missing Add and Edit actions from Indexes Difficulties in finding Wiki Index and understand current location Confusion between Page and Wiki concepts Depending on the current location, having automatically Parent set Missing permissions by not completing the Join this wiki step · ··· · ······ 22/44

23. (video) 4/9 used 'Join this wiki' Confusing between Move - Copy - Rename - Export Missing 'Add - Page' option Missing Rights to move pages Task 3.2 - Join this wiki Safari Power Saver Click to Start Flash Plug-in · · · · 23/44

24. (playlist) Preference for 'External image' Save confirmation needed Improvements: Single click, Pre-filled label, etc. Task 3.2 - Page Editing Safari Power Saver Click to Start Flash Plug-in · · · 24/44

25. Task 3.3

26. INSTRUCTIONS HYPOTHESES Create another page called 'Products'. Link the 2 pages one from another in order to be able to navigate between them. Is it possible also to make an hierarchy? (where 'Products' page is on an upper level than 'XWiki Enterprise' page) UC6 : Measure discoverability for 'Page Parent' functionality. UC7 : Asses the clearness of hierarchy display (breadcrumb, document index tree, document information, etc.). Testing: Page Linking 7/9 WYSIWYG Page Linking Testing: Page Parent 1/9 Page Parent Task 3.3 · · · · 26/44

27. (video) Missing visibility for Parent Cross wikis issues Tree is hidden and ambiguous Task 3.3 - Page Parent Safari Power Saver Click to Start Flash Plug-in · · · 27/44

28. Task 4

29. INSTRUCTIONS HYPOTHESES You would like to let the manager know that the content has been added and the page is ready. How would you do that? UC8 : Evaluate alternative communication methods. 8/9 Send Message 1/9 Send Page by mail Testing: Send Message Testing: Page Comments Testing: Send Page by e-mail Task 4 · · · · · 29/44

30. Task 6

31. INSTRUCTIONS HYPOTHESES Go to the Home page and remove the first paragraph from the 'Welcome to your wiki' section. UC9 : As simple users the ability to edit inline is not available. Since the default editor is WYSIWYG, the user will not be able to remove the paragraph without removing the whole Dashboard macro. 8/8 failed to edit the text 2/8 changed the title 2/8 deleted gadget Testing: Dashboard Testing: Inline Editing Task 6 · · · · · 31/44

32. Task 6 - Welcome Editing - Main (playlist) Safari Power Saver Click to Start Flash Plug-in 32/44

33. Task 6 - Welcome Editing - Dashboard (playlist) Safari Power Saver Click to Start Flash Plug-in 33/44

34. Task 7.1

35. INSTRUCTIONS HYPOTHESES You would like to create a dedicated place for Development material, just like the one existing for the Marketing team. Create such a location. UC10 : Evaluate the preference for wiki or space solution. 3/9 Space instead of Wiki UC11 : Evaluate Distribution Wizard's impact and friendliness. 6/9 Later 1/9 Never 2/9 Install Testing: Wiki Creation Testing: Space Creation Testing: Distribution Wizard Prereq: X user has 'Add Wiki' right Task 7.1 · · · · · · · · 35/44

36. Task 7.1 - Wiki Creation (video) Safari Power Saver Click to Start Flash Plug-in 36/44

37. Task 7.2

38. INSTRUCTIONS HYPOTHESES The Development team could use a Calendar application. Find out how to install this application on your wiki. UC12 : Evaluate multiple Calendar applications naming ambiguity. 6/8 Full Calendar Macro 1/8 Calendar Panel Application 1/8 Mocca Calendar Application Testing: Administration Testing: Application Panel Testing: Extension Manager Task 7.2 · · · · · · 38/44

39. (playlist) Display only Applications Add a Search button Provide entries in Suggest Task 7.2 - Application Install · · · 39/44

40. Task 5

41. INSTRUCTIONS HYPOTHESES Thinking about your overall Conclusions experience with XWiki, what did you like about the product? What did you not like? Would you use it in the future? Why or why not? Task 5 · 41/44

42. Questions?

43. Thank you to all the participants and for your attention Ecaterina Moraru — 26 July 2014 —