We used TechSmithMoraeto conduct usability testing of the new WVU Libraries RoomMeweb application on desktop, tablet, and mobile devices. This round of usability testing was internal, utilizing undergraduate student employees, graduate student employees, and staff. This is the results of the first round of usability testing.

1. USABILITY TESTING
WVU Libraries RoomMe

2. WVU LIBRARIES WEBSITE STUDY
Problem:
We used TechSmith Morae to conduct usability testing of the new WVU Libraries RoomMe web application on desktop, tablet,
and mobile devices. This round of usability testing was internal, utilizing undergraduate student employees, graduate student
employees, and staff. This is the results of the first round of usability testing.
  

3. 1ST ROUND OF TESTING
Task 1: Go to WVU Libraries' room reservation.
Task 2: Reserve a study room at the Downtown Library for a group that has 10 people.
Task 3: At the Downtown Library, what is the maximum number of hours for one reservation?
Task 4: At the Evansdale Library, is Classroom 130 available at 9:00AM on August 18th? If so, reserve it.
Task 5: How far ahead are you able to make a reservation at the Downtown Library? At Evansdale?
  

4. TEST PARTICIPANTS
Evan
Undergraduate Student
Jessica
Librarian
Rae
Undergraduate Student
Autumn
Graduate Student
David
Developer
    

5. OVERALL TASK:
AVERAGE TIME
* Average time includes delivery of tasks.

6. OVERALL
TASK:
SUCCESS
Completed
Completed with difficulty
Completed with ease
Almost Didn’t Completed

7. TASK 2: Reserve a study room at the Downtown Library for a group that
has 10 people.
User Observations:
- 60% of users expect that the ‘Find A Room Button’ will change
the contents of the first loaded results, and don’t understand that
the information is dynamically loading. A grayed-out button that
turns blue once a different selection is made could alleviate this.
- http://community.wvu.edu/~twb004/usability3/t21.wmv
- http://community.wvu.edu/~twb004/usability3/t22.wmv
- http://community.wvu.edu/~twb004/usability3/t23.wmv
- 80% of people don’t see a way to search specifically for room
capacities, or are just selecting rooms one-by-one until they find a
listed room capacity, despite the room policies information. Listing
a room capacity or suggested room capacity on each room page
could fix this issue. An enhancement could also be to add room
capacity as a search criteria.
- http://community.wvu.edu/~twb004/usability3/t24.wmv
- http://community.wvu.edu/~twb004/usability3/t25.wmv
- Form/input buttons on iPad are too thin vertically. Increase height
for iOS devices.    

8. TASK 3: At the Downtown Library, what is the maximum number of hours
for one reservation?
User Observations:
- 80% of the users are overlooking ‘Room Policies’, especially on
mobile devices. Room Policies should be more visually apparent
on mobile devices, and should be better integrated into the
individual rooms’ information with better visibility.
- http://community.wvu.edu/~twb004/usability3/t31.wmv
- 60% of users are expecting a NUI experience in reserving the
rooms and are confused when met with errors, complications, or
policy information differences; users are always going to use the
system before reading the policies. If a user should be limited in
scheduling to six hour intervals instead of being allowed to
schedule 23 hours, or just allowed to schedule six hours from the
start time if that is the policy.
- http://community.wvu.edu/~twb004/usability3/t32.wmv
    

9. TASK 4: At the Evansdale Library, is Classroom 130 available at 9:00AM
on August 18th? If so, reserve it, then Cancel it.
User Observations:
- Once the user is on room 130 in the Evansdale Library they are not
seeing a way to reserve the room, so they think it is off-limits or broken
as opposed to busy. An error message speaking to specifically to a room
being reserved should be implemented, and reservation results should
appear at the top of the page in list format, rather than at the bottom in a
table format.
- http://community.wvu.edu/~twb004/usability3/t41.wmv
- Users can see all of the rooms via the building view, but aren’t able to
reserve those buildings do to being disabled for the summer. If rooms
are disabled for renting during the summer, they shouldn’t populate on
the building list at all, or she be marked ‘unavailable’.
- http://community.wvu.edu/~twb004/usability3/gen1.wmv
- Users are confused when not being able to find the room in the mobile
list even if it is not available. List unavailable rooms on mobile in red and
parenthetically cite ‘reserved’ or ‘unavailable’ afterwards.
- http://community.wvu.edu/~twb004/usability3/t42.wmv
- Users are confused when search results don’t populate at the top of the
page before search options, and wonder if the search occurred. Make
search results always populate at the top of the page.
- Users don’t see changes occurring with dynamic data. Add a loading
wheel for queries that dynamically load content in-page.
- http://community.wvu.edu/~twb004/usability3/t43.wmv
    

10. TASK 5: How far ahead are you able to make a reservation at the
Downtown Library? At Evansdale?
User Observations:
- Again, 80% of users are expecting a NUI experience in reserving
the rooms and are confused when met with errors, complications,
or policy information differences; users are always going to use
the system before reading the policies. If a user should be limited
in scheduling only so many weeks or hours into the future, they
should not be presented options if policies are limiting, or
scheduling is not a possibility.
- http://community.wvu.edu/~twb004/usability3/t51.wmv
- http://community.wvu.edu/~twb004/usability3/gen2.wmv
- Users can’t delete reservations on Mobile. Change reservations
from displaying in a table format, especially on mobile.
- http://community.wvu.edu/~twb004/usability3/t53.wmv
    

11. SYSTEM USABILITY SCALE (SUS)
QUESTIONS
1. I think that I would like to use this system frequently
2. I found the system unnecessarily complex
3. I thought that the system was easy to use
4. I think that I would need the support of a technical person to be able to use this system
5. I found the various functions in this system were well integrated
6. I thought there was too much inconsistency in this system
7. I would imagine that most people would learn to use this system very quickly
8. I found the system very cumbersome to use
9. I felt very confident using the system
10. I needed to learn a lot of things before I could get going with this system

12. SUS: RESULTS
1
3
3
1
2
3
2
2
1
1
1
1
1
3
2
1
1
1
1
1
2
1
1
2
1
1
1
2
1
1
3
2
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
I think that I would like to use this system frequently
I found the system unnecessarily complex
I thought that the system was easy to use
I think that I would need the support of a technical person to be able to use this system
I found the various functions in this system were well integrated
I thought there was too much inconsistency in this system
I would imagine that most people would learn to use this system very quickly
I found the system very cumbersome to use
I felt very confident using the system
I needed to learn a lot of things before I could get going with this system
Strongly Disagree Disagree N/A Agree Strongly Agree

13. USABILITY SUGGESTIONS
Users by far have a more favorable desktop and tablet experience - which requires less time and understanding - than a mobile
experience. The mobile experience seems to frustrate users and causes task completion and functionality confusion.
UI and UX suggestions to follow…

14. UX ENHANCEMENTS
- Implement a grayed-out button that turns blue once a different selection
is made.
- List a room capacity or suggested room capacity on each room page.
- Add room capacity as a search criteria.
- Increase input heights for iOS devices.
- Room Policies should be more visually apparent on mobile devices and
within individual rooms.
- Users should be limited in scheduling to six hour/policy intervals in
dropdowns instead of being allowed to schedule the default 23 hours.
- Implement error messaging that speaks to specific room policies within
the room being reserved.
- Reservation results should appear at the top of the page in list format
rather than at the bottom in a table format.
- If rooms are disabled for renting during the summer, they shouldn’t
populate on the building list at all.
- Unavailable rooms should be marked as such.
- List unavailable rooms on mobile in red and parenthetically cite
‘reserved’ or ‘unavailable’ afterwards.
- Make search results always populate at the top of the page.
- Add a loading wheel for queries that dynamically load content in-page.
- Limit users to scheduling only so many weeks or hours into the future
based on policies; don’t present options or scheduling if it’s not possible
to accomplish.
- Change ‘My Reservations’ from displaying in a table format, especially
on mobile.

15. UI ENHANCEMENTS
I would suggest possibly implementing the following user interface
changes in the next upgrade or version of RoomMe:
- Remove or minimize website integration on mobile devices
This is mainly the header and footer and website navigation, which
should be secondary to the web application’s core functionality, and
takes up too much space on a mobile device.
This could be accomplished in the branding upgrade we do this fall,
which will have a simplified navigation and minimal branding.
- Implementing a tab bar
Implement a sticky tab bar that positions essential and repetitive tasks
– searching for a room, login, policies, etc. – through universal
iconography. This would afford users the screen space to accomplish
the core functions that are integral and necessary, while providing a
ease to finding navigation and repetitive tasks.
- Optimize use of interactive content
Use accordions and modals to subdivide content, which will conserve
in-page space and group together like content. Examples of this are
forms, room information, and in-page policies.
Forms need to be completely standardized and re-worked to be more
mobile friendly, as well as work with notification and completion
signage to better indicate failure, progress, and success.