Designing usable websites becomes a subject of major importance with the growth of e-commerce applications and the wide range of offers the web suggests. To reach potential customers and maintain their loyalty under the conditions of high competition, organizations need to constantly control usability of the websites representing them. In this paper we present a systematic approach to usability, based on a roadmap, that aims to overcome quality gaps appearing in website lifecycle. To address these gaps it is possible to adopt different techniques. Our solution consists in identifying the quality gaps and suggesting the techniques most adequate to overcome a given gap. The result is a multi-step process for determining and realizing the improvements and re-engineering of the website. We illustrate our approach on the website of a tourist destination.

1. A quality gaps roadmap
for a website usability
process
Luisa Mich
Mariangela Franch
Nadzeya Kiyavitskaya
University of Trento, Italy

2. gennaio 11, 2018 ISE'2008
Contents
1. Problem Description
2. Approach: Quality Gaps Roadmap
3. Case study
4. Lessons learned
2

3. Problem Description
v Organizations need to constantly control
usability of their websites
v Existing approaches
gennaio 11, 2018 ISE'20083
§ Bottom-up
• focus on basic aspects
and features, often
independent from goals
and domain
• a linear “waterfallstyle”
usability process
§ Top-Down
• usability is the central
point for website
development
• difficult to contextualize
in individual projects

4. Problem Description
v Our contribution
§ A usability evaluation process that helps to
identify the sequence of techniques to be
applied in a specific application
§ The process is designed so as to be understood
by both the management and the web
development teams
gennaio 11, 2018 ISE'20084

5. Approach: Quality Gaps Roadmap
v Different types of Quality
§ Unexpected (latent) quality: site satisfies
needs that are new to the users
§ Expected quality: the quality the users expect
from website to satisfy their needs
§ Quality in use: perceived by the user in a
specific use context
§ Perceived quality: the quality that the users
believe they have received as compared to
their expectations
gennaio 11, 2018 ISE'20085

6. Approach: Quality Gaps Roadmap
v Different types of Quality (cont’d)
§ Designed quality: the quality the website
owner has planned to offer to the user
§ Delivered (issued) quality: the result of the
transposition of the design into a website
v Question
§ How to balance among the different qualities?
v Our solution
§ Use the quality gaps as a roadmap
gennaio 11, 2018 ISE'20086

7. Approach: Quality Gaps Roadmap
gennaio 11, 2018 ISE'20087

8. gennaio 11, 2018 ISE'2008
Approach: Quality Gaps Roadmap
Gap 0
Step 1 concerns the role of
the website for the organization business models.
Creativity and forecasting techniques can be used
to generate new ideas
Gap 1
Step 2.1 is to compare website quality to
requirements
Step 2.2 is to compare the website with the
competitors’ sites
8
Overcoming usability gaps

9. Approach: Quality Gaps Roadmap
gennaio 11, 2018 ISE'20089
Gap 2
Step 3 is evaluation of quality in use with tests
and experiments with users.
Elicitation of eventual difficulties met when using
the website to fulfill predefined tasks.
Gap 3
Step 4 is evaluation of the perceived quality by
means of questionnaires, interviews, or indexes
related to loyal customers and conversion rate.
Overcoming usability gaps (cont’d)

10. Case Study
v Usability evaluation project
§ Application of the roadmap to the website of
the autonomous province Alto Adige - South
Tyrol: www.suedtirol.info
v Specific difficulties
§ Multiple owners and sponsors having
conflicting interests
§ Integration with large back-end information
systems
§ Presence of large number of sections or micro-
sites
§ International audience, etc.
gennaio 11, 2018 ISE'200810

11. Case Study
Step 1: web business model
§ Applying Delphi method to explore innovative
choices that together with an analysis of their
impact on the usability can add a considerable
added value, increasing the company
competitive advantage
§ Work in progress…
gennaio 11, 2018 ISE'200811

12. Case Study
Step 2: inspective evaluation
1. A standard evaluation schema based on 7Loci
model to identify strong and weak points
2. A detailed evaluation: identify specific aspects
that have to be improved to increase overall
quality
gennaio 11, 2018 ISE'200812

13. Case Study
Step 2.1: Main inspective evaluation results
according to the standard schema
gennaio 11, 2018 ISE'200813
Identity dimension Score Observations
Brand: does the
website have a
strong identity?
2 Inconsistency in the use and interpretation of the logo
Image: Does the
website transmit a
sound image of the
company?
1 The nature of the company behind the site is not clear
Design: Is the
website’s graphics
attractive to the users?
2 Original design, but not straightforwardly intuitive. The left
section is not always used properly to indicate current
location and what are other functions available, while the
right section has certain “random” data in some pages
Personalization: Does
the website adapt to
different user
categories?
1 The three fields of the site are not easily recognizable as
different modes of adapting to three types of tourists

14. Case Study
Final result Step 2.1 - quality diagram for
www.suedtirol.info
Note that Identification and Identity dimensions are
the weakest ones
gennaio 11, 2018 ISE'200814

15. Case Study
v Step 2.1: The detailed analysis of the problems
noted by the experts using the standard schema
v The most problematic points identified
1. weak website identity
2. the three areas of the site – “familiarmente”,
“piacevolmente”, “attivamente”, i.e., family, wellbeing,
and active package respectively – not understood as
directed to three different segments
3. a non intuitive navigation logic of the website
gennaio 11, 2018 ISE'200815

16. Case Study
Step 2.2: Comparison of the quality of
www.suedtirol.info to other websites
gennaio 11, 2018 ISE'200816
Dimension South Tyrol Trentino Tyrol Carinthia
Content 0.78 0.81 0.81 0.63
Services 0.83 0.92 0.83 0.58
Identity 0.80 1.00 1.00 0.87
Identification 0.42 0.75 0.92 0.83
Management 0.88 0.88 0.88 0.63
Usability 0.89 0.89 0.83 0.78
Weakest dimension

17. Case Study
v Step 3: evaluation with a panel of users
§ “Youth” and “Family” groups
§ 6 people in each group
§ Groups are given 4 tasks with different
search parameters
1. Make a hotel reservation
2. Request information for a vacation package
3. Find out how to get to the destination
4. Find a means of asking illustrative material
gennaio 11, 2018 ISE'200817

18. v Results of Step 3: difficult issues identified
§ the need to specify a locality even where the
tourist could not still make a decision
§ the lack of any result for the specified
conditions of accommodation
§ insufficient flexibility of the reservation system
with respect to the topic chosen by the tourist
(sport, cultural, or other)
§ unclear navigation structure of the website
gennaio 11, 2018 ISE'200818

19. Case Study
Step 4: Answers to the questionnaire
v Focus of the questionnaire
§ Complaints about the time constraints when
fulfilling the 4 tasks
§ User satisfaction problems
v Critical issue identified
§ Misinterpretation of 3 thematic areas
gennaio 11, 2018 ISE'200819

20. Lessons Learned
v The gaps-driven method allows to collect
information in systematic way
v The data obtained from Steps 1 and 2
helps in planning of Step 3
v Perception of the website has to be
compared to the one designed
gennaio 11, 2018 ISE'200820

21. Questions and feedback:
luisa.mich@unitn.it