1.
Usability Testing
2014 Highlights & Findings
Ecaterina Moraru — 26 July 2014 —

2.
Why is good to do usability
sessions?
You confirm issues you've suspected
Get amazed by new things you discover
“
”
·
·

3.
SCENARIO
Scenario #4
PERIOD
27 May - 12 June 2014
VERSION
XE 6.0
SKIN
Colibri
DURATION
9:45:45 h
FEATURES
Profile, Navigation, Pages, Wikis,
Applications
Usability Sessions
3/44

4.
Tasks

5.
Tasks
1 | First Impressions
2 | Profile Editing
3.1 | Wiki Navigation
3.2 | Page Editing
3.3 | Page Hierarchy
4 | Sending Message
6 | Welcome Editing
7.1 | Wiki Creation
7.2 | Application Install
5 | Conclusions
5/44

6.
Task 1

7.
INSTRUCTIONS HYPOTHESES
What are your first
impressions of XWiki?
What do you think you can
do with it?
You can scroll, hover, but
please don't click on
anything yet.
Introduction
Testing: Homepage
Status: Not-logged-in
Status: Logged-in
Task 1
·
·
·
·
7/44

8.
Task 2

9.
INSTRUCTIONS HYPOTHESES
Imagine this is your first
day at work and XWiki is
used internally.
You want to customise
your profile and add some
information about yourself.
How can you do that?
UC1 : Measure the performance of 'menu-button'
vs. 'icon-link' (#tmEdit vs.
.editProfileCategory)
9/9 'icon-link' usage
Testing: Profile
9/9 Success
Task 2
·
·
·
9/44

10.
(video)
Task 2 - Profile Editing
Safari Power Saver
Click to Start Flash Plug-in
· Add hints to Profile Preferences · Specify default values
10/44

11.
Task 3

12.
INSTRUCTIONS SUBTASKS
You've been given the assignment to add some
content (text+image) for the Marketing team.
Task 3.1
Task 3.2
Task 3.3
INSTRUCTIONS HYPOTHESES
First let's browse a bit the
product and locate where
the Marketing material is
found.
UC2 : Measure the performance of Search,
Search Suggest vs. Wiki Index
7/9 Search Suggest
2/9 Wiki Index
Testing: Navigation
3/9 couldn't find
Marketing without
help
Task 3
Task 3.1
·
·
·
·
12/44

13.
(playlist)
No suggest for partial words
No Wiki results in Search
Pressing Enter too fast
3/9 Suggest worked in
first try
4/9 Suggest failed in
first try
Task 3.1 - Navigation - Search Suggest
Safari Power Saver
Click to Start Flash Plug-in
·
·
·
·
·
13/44

14.
(video)
Harder to find under menu
No Wikis gadget on Homepage
No Applications panel entry
1/9 Used only Wiki Index
2/9 Found it alone after
Suggest fail
Task 3.1 - Navigation - Wiki Index
Safari Power Saver
Click to Start Flash Plug-in
·
·
·
·
·
14/44

15.
Task 3.2

16.
INSTRUCTIONS HYPOTHESES
You need to create a page
called 'XWiki Enterprise' (in
the Marketing location).
As a result, the page should
look similar to:
UC3 : In order to edit, you need to be a
member of the wiki. Measure task
success.
UC4 : Measure 'Add image' dialog naming.
UC5 : Measure table performances.
Testing: Page Creation
Testing: Page Editing
Extended: wiki syntax
Extended: macro usage
Task 3.2
·
·
·
·
16/44

17.
Task 3.2 - Page Creation
EXPECTED RESULT
0/9 expected result
3/9 created Wiki
2/9 edited Sandbox
ACTUAL RESULTS
·
·
·
x2 x2
17/44

18.
(video)
Task 3.2 - Page Creation
EXPECTED RESULT ACTUAL RESULTS
Safari Power Saver
Click to Start Flash Plug-in
18/44

19.
(video)
Task 3.2 - Page Creation
EXPECTED RESULT ACTUAL RESULTS
Safari Power Saver
Click to Start Flash Plug-in
19/44

20.
(video)
Task 3.2 - Page Creation
EXPECTED RESULT ACTUAL RESULTS
Safari Power Saver
Click to Start Flash Plug-in
20/44

21.
(video)
Task 3.2 - Page Creation
ACTUAL RESULTS ACTUAL RESULTS
Safari Power Saver
Click to Start Flash Plug-in
21/44

22.
Task 3.2 - Page Creation
Challenges:
Finding the Add button
Creating a Page and not something else
Creating the page in the right location
Issues:
Problems finding the Add button
Missing Add and Edit actions from Indexes
Difficulties in finding Wiki Index and understand current location
Confusion between Page and Wiki concepts
Depending on the current location, having automatically Parent set
Missing permissions by not completing the Join this wiki step
·
···
·
······ 22/44

23.
(video)
4/9 used 'Join this wiki'
Confusing between Move - Copy -
Rename - Export
Missing 'Add - Page' option
Missing Rights to move pages
Task 3.2 - Join this wiki
Safari Power Saver
Click to Start Flash Plug-in
·
·
·
·
23/44

24.
(playlist)
Preference for 'External image'
Save confirmation needed
Improvements: Single click, Pre-filled
label, etc.
Task 3.2 - Page Editing
Safari Power Saver
Click to Start Flash Plug-in
·
·
·
24/44

25.
Task 3.3

26.
INSTRUCTIONS HYPOTHESES
Create another page called
'Products'.
Link the 2 pages one from
another in order to be able
to navigate between them.
Is it possible also to make
an hierarchy?
(where 'Products' page is
on an upper level than
'XWiki Enterprise' page)
UC6 : Measure discoverability for 'Page
Parent' functionality.
UC7 : Asses the clearness of hierarchy
display (breadcrumb, document
index tree, document information,
etc.).
Testing: Page Linking
7/9 WYSIWYG Page
Linking
Testing: Page Parent
1/9 Page Parent
Task 3.3
·
·
·
·
26/44

27.
(video)
Missing visibility for Parent
Cross wikis issues
Tree is hidden and ambiguous
Task 3.3 - Page Parent
Safari Power Saver
Click to Start Flash Plug-in
·
·
·
27/44

28.
Task 4

29.
INSTRUCTIONS HYPOTHESES
You would like to let the
manager know that the
content has been added
and the page is ready.
How would you do that?
UC8 : Evaluate alternative communication
methods.
8/9 Send Message
1/9 Send Page by mail
Testing: Send Message
Testing: Page
Comments
Testing: Send Page by e-mail
Task 4
·
·
·
·
·
29/44

30.
Task 6

31.
INSTRUCTIONS HYPOTHESES
Go to the Home page and
remove the first paragraph
from the 'Welcome to your
wiki' section.
UC9 : As simple users the ability to edit
inline is not available. Since the
default editor is WYSIWYG, the user
will not be able to remove the
paragraph without removing the
whole Dashboard macro.
8/8 failed to edit the text
2/8 changed the title
2/8 deleted gadget
Testing: Dashboard
Testing: Inline Editing
Task 6
·
·
·
·
·
31/44

32.
Task 6 - Welcome Editing - Main (playlist)
Safari Power Saver
Click to Start Flash Plug-in
32/44

33.
Task 6 - Welcome Editing - Dashboard (playlist)
Safari Power Saver
Click to Start Flash Plug-in
33/44

34.
Task 7.1

35.
INSTRUCTIONS HYPOTHESES
You would like to create a
dedicated place for
Development material, just
like the one existing for the
Marketing team.
Create such a location.
UC10 : Evaluate the preference for wiki or
space solution.
3/9 Space instead of Wiki
UC11 : Evaluate Distribution Wizard's
impact and friendliness.
6/9 Later
1/9 Never
2/9 Install
Testing: Wiki Creation
Testing: Space Creation
Testing: Distribution
Wizard
Prereq: X user has 'Add
Wiki' right
Task 7.1
·
·
·
·
·
·
·
·
35/44

36.
Task 7.1 - Wiki Creation (video)
Safari Power Saver
Click to Start Flash Plug-in
36/44

37.
Task 7.2

38.
INSTRUCTIONS HYPOTHESES
The Development team
could use a Calendar
application.
Find out how to install this
application on your wiki.
UC12 : Evaluate multiple Calendar
applications naming ambiguity.
6/8 Full Calendar Macro
1/8 Calendar Panel
Application
1/8 Mocca Calendar
Application
Testing: Administration
Testing: Application
Panel
Testing: Extension
Manager
Task 7.2
·
·
·
·
·
·
38/44

39.
(playlist)
Display only Applications
Add a Search button
Provide entries in Suggest
Task 7.2 - Application Install
·
·
·
39/44

40.
Task 5

41.
INSTRUCTIONS HYPOTHESES
Thinking about your overall
Conclusions
experience with XWiki,
what did you like about the
product?
What did you not like?
Would you use it in the
future? Why or why not?
Task 5
·
41/44

42.
Questions?

43.
Thank you to all the participants
and for your attention
Ecaterina Moraru — 26 July 2014 —