DFAIT Usability Final 6 Apr08

Usability testing of DFAIT website
Prepared for: Barry Nesbitt, DFAIT
Contract number: 08170-070584/001/CY
Version 1.2 – April 16, 2008
Une version française du sommaire exécutif est disponible sur demande
Authors: Gord Hopkins, Mike Atyeo, Scott Smith & Paula Preston

Neo Insight usability evaluation - 2Copyright 2008 Neo Insight Inc.
Table of contents
 Executive summary 3
 Task performance summary 19
 Usability evaluation results 31
 Consular 33
 eDiscussion and START 56
 Trade negotiations and agreements 74
 International youth program 93
 Russian mission site 113
 Contact information 129

Introduction
 Background
 The Department of Foreign Affairs and International Trade (DFAIT)
continues to make changes to its websites with the goal of creating a
unified web presence to better address the needs of both domestic and
international clients
 This work includes implementation of new design, navigation and
content; plus development of a new information architecture
 Objectives
 Test the usability of the current design and information architecture plus
some proposed wire frames
 Gather user feedback to provide direction for the future evolution of the
websites tested
 Test the ease with which users can navigate the websites to find
information, the suitability of the link labels and terminology, and the
overall appeal of the website designs
Executive Summary

Method
 Recruiting
 26 participants in each of 5 audiences were recruited and tested on
different DFAIT content:
 General: Adult, international traveller (Consular website)
 University: Policy students/professors (eDiscussion and START)
 Business: Involved in international trade (Trade Negotiations)
 Youth: Between 18-25 years (International Youth Program)
 Russian: New arrivals in Canada (Russia mission site)
 Remote usability testing
 All participants were tested in their own environments (home or
office) using remote testing software (UserVue from TechSmith).
 Screen interactions and participant audio was captured for later analysis
 Participants were asked to try and complete 5 to 7 typical tasks
 Measured ability to complete tasks (with or without hints) and their level
of satisfaction with the way the website supported the task
 Testing took place from March 10th
-14th
, 2008
 Participants were paid an honorarium for their participation
Executive Summary
For details, see DFAIT Facilitator Guide v1.0 Master 09Mar08.doc

Strengths – Across all areas tested
Area of DFAIT Description of Strength
Consular Crisp, clear layout
Good search engine
Good coverage of travel warnings and alerts
eDiscussion and
START
Simple layout with clear headings for most sections
START offers good imagery
Good coverage of Canada’s activities abroad
Trade
Negotiations and
Agreements
Good access to wide range of international trade info
Regularly updated content
Ability to get notifications of updates
International
Youth Program
Good collection of programs and associated countries
Provides access to opportunities of value to youth
Helps expand understanding of international issues
Russian Mission
site
Good info on working/studying/visiting/moving to Canada
Expands awareness of Canada’s culture
Helps people understand English terminology
Executive Summary

Scorecard
 Comparison of task completion, satisfaction and usability issues
Executive Summary
Area Critical Major Minor
Youth 2 4 9
Consular 1 6 10
Russia 4 0 4
Trade 3 5 9
Policy 1 6 9
Task Completion Rate by Area Tested
0% 20% 40% 60% 80% 100%
Policy
Trade
Russian
Consular
Youth
Complete Complete with Hints Incomplete
Satisfaction Rating by Area Tested
0% 20% 40% 60% 80% 100%
Policy
Trade
Russian
Consular
Youth
Satisfied Marginal Dissatisfied

Consular summary
Executive Summary
 Task performance overview
 7 tasks tested – 1 Critical and 1 Major task problem which included:
 Difficulty finding potential issues associated with travelling with dual citizenship
 Not finding warning information in Travel Report because some people did not
scroll
 Usability overview
 1 Critical, 6 Major and 10 Minor issues – Critical and Major included:
 Content missed because it was not where expected
 Travel Report information missed by some users
 Country list not visible without scrolling
 Table of contents overlooked
 Too many competing and redundant links on the Home page
 Difficult to discriminate some sections
 Key Information vs. Essential Resources vs. Quick Links
 Verbose and unstructured text in some areas

Consular key recommendations
Executive Summary
 Key recommendations
 Re-examine information architecture in a few key areas to ensure content
is where it is expected to be
 Change layout of the Travel Reports country listing to ensure people see
the start of country listings without having to scroll
 Eliminate redundant links from the Home page and organize it into clearly
differentiated sections
 Build on the existing categories of before, during or returning from travel
 Help people focus on the country of interest and the stage in their travel
 Provide links to Offices Abroad or Embassies on the Home page
 Visually integrate content on the right hand side to make it more
noticeable

eDiscussion and START summary
Executive Summary
 7 tasks tested – 1 Critical and 5 Major task problems which included:
 Not able to find START program from main DFAIT Home page
 Not able to find specific funding details for Global Peace and Security Fund
 Difficulty locating information on how to submit a Policy Position Paper
 Difficulty finding the current eDiscussion and using the email alert form
 Difficulty finding video about Children at War
 Not able to access specific contact information about Sudan activities
 No links to funding eligibility information
 Title of current eDiscussion difficult to find
 Policy Paper information not expected under eDiscussions
 No links from summary of Policy Position Papers to actual papers
 Annotation of Video Library link is misleading
 eNewsletter subscription form confusing to use
 Global Issues not an obvious link to look for the START program

eDiscussion and START key recommendations
 Support key tasks associated with looking for funding information
 Provide easy access to Eligibility and How to Apply information
 Highlight the current eDiscussion title more prominently
 Clarify the relationship between eDiscussions and Policy Position Papers
 Change the Policy eDiscussions section to Policy Discussions to better
accommodate the range of content found there
 Policy Position Papers, Online Discussions, Video Interviews, etc.
 Provide cross-links to related information
 e.g. – Link to How to Submit a Policy Position Paper from the summary listing
of papers
 Make link labels match user expectations regarding destination content
 Simplify the eNewsletter subscription form
 Re-evaluate where the START program is located or change the Global
Issues link label to something which would better convey the notion of
Programs or Canada’s Involvement abroad
 Ensure navigation breadcrumbs are used consistently throughout the site
Executive Summary

Trade negotiations summary
Executive Summary
 5 tasks tested – 2 Critical task problems which included:
 Unable to associate finding information about business travel with Trade
Negotiations and Agreements
 Difficulty finding information about trade barriers
 Business travel regulations not expected under Trade Negotiations link
 Business travel information inconsistent in different parts of website
 Trade barriers not associated with the link label of Market Access Report
 ExportSource and InfoExport terms do not communicate well
 Valuable right hand content and links often overlooked
 Links often not arranged for ease of scanning, nor in the user’s language
 Search was not working during testing

Trade negotiations key recommendations
 Provide cross-links to business travel regulations on the Services for
Travellers pages
 Ensure consistency of information by taking a more database approach
– maintain only one valid source for Business Travel Requirements
which may be accessed from several locations
 Consider changing the Market Access Report label to Trade and
Investment Barriers
 Replace internal terms, like ExportSource and InfoExport, with more
meaningful terms
 Visually integrate information in right hand column with central content
 Arrange links into bulleted lists so they can be scanned easily
 Ensure links are provided at the point where the content or resources
are described – e.g. Trade Barrier descriptions should link to database
 Reduce the amount of text and structure content for easy scanning
Executive Summary

International youth program summary
Executive Summary
 5 tasks tested – 1 Critical and 0 Major task problems which included:
 Unable to find link to the International Youth Program from DFAIT home page
 International Youth Program not expected under Foreign Policy section
 Program descriptions provide no links to more details or country opportunities
 Link to International Job Opportunities in Engineering goes to a generic
University website Home page
 Content being described is often missing links to actually get to the described
information
 Key information missing from French version of How Our Programs Work page
 Right side navigation is often overlooked

International youth key recommendations
 Provide cross-links from the Travel and Business sections and consider
changing the name to something like Work or Study Abroad
 Link program titles on Program Descriptions page to more specific
information about the program and participating countries
 Allow people to choose a program of interest and then see what
countries support it OR choose a country of interest and then see what
programs are supported
 Ensure links to resources are provided at the point when the resources
are being described
 Ensure the same information is available in both official languages
 Visually integrate right hand links so it is clear that they are related to the
central content
 Relocate the Country drop-down menu to a more noticeable position and
change the prompt to Select Country
 Place related links at relevant points throughout the sub-site rather than
grouping them under the vague label Other Links
Executive Summary

Russian mission site summary
Executive Summary
 5 tasks tested – 2 Critical task problems which included:
 Difficulty getting to the Trade area of the website
 Difficulty determining what documentation is required for moving from
Armenia to Canada
 4 Critical, 0 Major and 4 Minor issues – Critical included:
 Link to Trade area of website is missing from main menu
 Key links to the Trade area of the site are not visible because they are
unexpectedly on the right hand side
 Not all English menu items are on the Russian-language version of menu
 Links on the right hand side are not evident to users

Russian mission site key recommendations
 Put a link to the Trade area on the Russian-language version
 Map menu items exactly between the Russian and English versions to
facilitate people learning the corresponding English terminology to use
 Move key links from the right hand side into the central area and
visually tie in the content in the right hand column
 Put links to the Trade Commissioner service in the central area
 Ensure all links are evident to the users – underline if necessary
 Highlight the currently active link on the left hand menu when
appropriate
 Ensure the link labels correspond to the headings on the destination
pages
Executive Summary

Future directions
 Continue to develop a task-based structure to the websites
 Structure text and lists of links so they are easy to scan
 Directly link to the resources being described
 Examine user’s steps in achieving their goals and help them complete
each step towards their goal
 Remove verbose descriptions and allow people to get to the actions as
quickly as possible
 Test all terminology for it’s ability to communicate as intended, especially
link labels – consider doing a Customer Carewords analysis
 www.CustomerCarewords.com
 Further improve navigation and user experience
 Explore opportunities to integrate and simplify information, maintaining a
consistent look and feel across the multiple sub-sites
 Continue to refine the effectiveness of search by managing search based
on analysis of search logs and ongoing user testing
Executive Summary

Next steps
1. Address the critical and major usability issues from this testing
2. Address typical user experience issues
1. Put the users’ most frequent and important tasks front and centre
2. Move departmental content out of the way
3. Reduce clutter by eliminating redundant links
4. Adopt common style sheets to be used across all sub-sites
5. Visually integrate the right hand content better with central content
6. Provide context-sensitive and specific contact information
7. Provide a search box on every page
3. Consider conducting a full Customer Carewords analysis
www.customercarewords.com to optimize terminology, information
architecture, and users’ task flow management
Executive Summary

Task
performance
summary

Task performance summary
 The following tables and charts show participant performance and
satisfaction ratings for each of the audiences
 These tasks were chosen in consultation with the stakeholders to
represent the range of tasks the websites should support
 Critical, major or minor performance issues are highlighted by colour
 Out of the 30 tasks investigated in this project, 18 exhibited some level
of performance issue
 7 were critical
 6 were major
 5 were minor
 The recommendations found throughout the rest of this document are
designed to address the usability issues observed
MINOR <80 % completed without hints
MAJOR <60 % completed without hints
CRITICAL <40 % completed without hints

Consular – Task list
Task Label Task Description
5 Greek citizenship
CRITICAL ***
[Start from Consular home page] http://www.voyage.gc.ca/consular_home-en.asp
You are an18 year old Canadian but your are eligible for dual citizenship because your parents
are Greek citizens. Find out if there are any issues you should be aware of before visiting Greece.
1 Kenya travel
MAJOR **
[Start from DFAIT home page] http://www.international.gc.ca/index.aspx
You have heard about riots in Kenya and are worried about your friend who is currently travelling
there. Find out what the current recommendations are for travelling in this area.
3 Guide for women
MINOR *
A woman you know is planning to travel abroad alone. Find a guide specifically designed to help
women prepare for travelling abroad.
2 Poland office
MINOR *
You are travelling in Poland and have had your passport stolen. Find out how to contact the
Canadian authorities in Poland.
4 Romanian embassy [Start from Consular home page] http://www.voyage.gc.ca/consular_home-en.asp
You need to quickly contact the Canadian Embassy in Romania. Find the contact details.
7 Travel with children [Start from Consular home page] http://www.voyage.gc.ca/consular_home-en.asp
You are planning to travel outside Canada with your child, but without your spouse. Find out if
there are any special arrangements which need to be made.
6 Drugs and travel [Start from Consular home page] http://www.voyage.gc.ca/consular_home-en.asp
Your doctor recently prescribed some medications for you. Find out what you need to do to make
sure you don’t run into any problems when travelling with them.

Consular – Task summary
Mean Time on Task (min) - Consular
0 1 2 3 4
OVERALL
Drugs and travel
Travel with children
Romanian embassy
Poland office
Guide for women
Kenya travel
Greek citizenship
Task Completion by Task - Consular
0% 20% 40% 60% 80% 100%
OVERALL
Drugs and travel
Romanian embassy
Poland office
Guide for women
Kenya travel
Greek citizenship
Satisfaction by Task - Consular
0% 20% 40% 60% 80% 100%
OVERALL
Drugs and travel
Romanian embassy
Poland office
Guide for women
Kenya travel
Greek citizenship

eDiscussion/START – Task list
7 START funding
CRITICAL ***
[Start from START home page] http://www.international.gc.ca/fac/START-GTSR/index.aspx?lang=en
You are working for an international organization focused on stopping human trafficking. Find
out how much funding is available to support global peace and security this year and if you are
eligible for any of this funding.
3 Email alerts
MAJOR **
You’re interested in keeping up to date on discussions about international affairs. Find out what
topics are currently being discussed and arrange to be notified whenever new information is
added to the discussion.
4 Child war videos
MAJOR **
You’ve heard about interesting videos on children in war. Find these videos to view.
5 Sudan info
MAJOR **
You would like to know more about Canada's reconstruction and peacekeeping efforts in Sudan.
Find out who you would contact to get more specific information.
2 Policy paper
MAJOR **
You’ve heard it’s possible to submit a policy position paper. Find out what format is acceptable
and to whom you would submit it.
6 START achieve
MAJOR **
You have been asked to write a report on Canada's Stabilization and Reconstruction Task Force
(START) to better understand Canada's capacity to respond to international crises. Find a
document describing this task force’s achievements over the past year.
1 Share ideas
MINOR *
You’re interested in sharing your thoughts on Canada’s foreign policies. Find out how to
participate in any ongoing exchange of ideas and how to register so you can start contributing.

eDiscussion/START – Task summary
Mean Time on Task (minutes) - Policy
0 1 2 3 4 5 6
OVERALL
Share ideas
START achieve
Policy paper
Sudan info
Child war videos
Email alerts
START funding
Task Completion by Task - Policy
0% 20% 40% 60% 80% 100%
OVERALL
Share ideas
START achieve
Policy paper
Sudan info
Child war videos
Email alerts
START funding
Satisfaction by Task - Policy
0% 20% 40% 60% 80% 100%
OVERALL
Share ideas
START achieve
Policy paper
Sudan info
Child war videos
Email alerts
START funding

Trade negotiations – Task list
1 Mexico documents
CRITICAL ***
You know Canada has a business agreement with Mexico and you are flying to Mexico do some
business. You know you need a passport, but what other formal documentation is required?
5 Trade barriers
CRITICAL ***
[Start from TNA main page] http://www.international.gc.ca/trade-agreements-accords-
commerciaux/index.aspx?lang=en&menu_id=9&menu=
You are a Canadian producer of Canola Oil and would like to know which countries currently
have trade barriers which would impact your ability to export to those countries.
2 WTO panel [Start from TNA main page]
http://www.international.gc.ca/trade-agreements-accords-commerciaux/index.aspx?lang=en&menu_id=9&menu
=
You’ve heard that a World Trade Organization panel was set up to settle some of the disputes
over US agricultural subsidies. Find out when this panel was established.
4 Update alerts [Start from TNA main page] http://www.international.gc.ca/trade-agreements-accords-
commerciaux/index.aspx?lang=en&menu_id=9&menu=
How could you arrange to be notified by email of any updates to the Trade Negotiations and
Agreements website.
3 Korea free trade [Start from TNA main page]
http://www.international.gc.ca/trade-agreements-accords-commerciaux/index.aspx?lang=en&menu_id=9&menu
=
You are interested in doing business with Korea. Determine the status of a free trade agreement
with Korea and who to contact for more information.

Trade negotiations – Task summary
Mean Time on Task (minutes) - Trade
0 2 4 6 8 10
OVERALL
Korea free trade
Update alerts
WTO panel
Trade barriers
Mexico docs
Task Completion by Task - Trade
0% 20% 40% 60% 80% 100%
OVERALL
Korea free trade
Update alerts
WTO panel
Trade barriers
Mexico docs
Satisfaction by Task - Trade
0% 20% 40% 60% 80% 100%
OVERALL
Korea free trade
Update alerts
WTO panel
Trade barriers
Mexico docs

Int’l youth program – Task list
1 Travel in Holland
CRITICAL ***
[Start from DFAIT home page]
http://www.international.gc.ca/index.aspx
You are interested in traveling and doing a bit of work in Holland. Find out what opportunities are
offered in that country.
3 IYP int’l jobs
MINOR *
[Start from IYP home page]
http://www.international.gc.ca/culture/123go/menu-en.asp
You are an engineering student looking for work abroad. Find out how many opportunities the
International Youth Program has for you.
2 IYP eligibility
MINOR *
[Start from IYP home page]
You are a permanent resident in Canada. Find out if you are eligible to participate in the
International Youth Program.
4 Work in Belgium [Start from IYP home page]
You are 31 years old. Are you eligible for a Working Holiday visa in Belgium?
5 SWAP comparison [Start from IYP home page]
What is the difference between SWAP and the Working Holiday Program?

Int’l youth program – Task summary
Mean Time on Task (minutes) - Youth
0 1 2 3 4
OVERALL
SWAP comparison
Work in Belgium
IYP eligibility
IYP int'l jobs
Travel in Holland
Task Completion by Task - Youth
0% 20% 40% 60% 80% 100%
OVERALL
SWAP comparison
Work in Belgium
IYP eligibility
IYP int'l jobs
Travel in Holland
Satisfaction by Task - Youth
0% 20% 40% 60% 80% 100%
OVERALL
SWAP comparison
Work in Belgium
IYP eligibility
IYP int'l jobs
Travel in Holland

Russian mission site – Task list
4 Trade section
CRITICAL ***
[Start from Russia site home page]
http://www.dfait-maeci.gc.ca/missions/russia-russie/menu-rus.asp
Find out how to contact the trade section of the embassy.
1 Immigration docs
CRITICAL ***
[Start from preferred search engine]
I’m from Armenia and I want to know if there is any special documentation/visa requirement for
moving to Canada.
5 Art exhibit [Start from Russia site home page]
Find out if there is a Canadian Art Exhibit currently showing in Russia.
3 Info sessions [Start from Russia site home page]
Find out if there are any information sessions you could attend to find out more about moving to
Canada to live.
2 Embassy hours [Start from Russia site home page]
You are in Russia, find the hours of operation of the Embassy in Moscow.
6 Russian embassy [Start from DFAIT home page]
http://www.international.gc.ca/index.aspx
From this page, you would like to get to the Canadian Embassy in Russia web site.

Russian mission site – Task summary
Mean Time on Task (minutes) - Russia
0 2 4 6 8 10
OVERALL
Russian embassy
Embassy hours
Info sessions
Art exhibit
Immigration docs
Trade section
Task Completion by Task - Russia
0% 20% 40% 60% 80% 100%
OVERALL
Russian embassy
Embassy hours
Info sessions
Art exhibit
Immigration docs
Trade section
Satisfaction by Task - Russia
0% 20% 40% 60% 80% 100%
OVERALL
Russian embassy
Embassy hours
Info sessions
Art exhibit
Immigration docs
Trade section

Usability
evaluation
results

Ratings guide and prioritization
 Guide to ratings
 Usability issues are prioritized and given a rating as either Critical, Major or Minor.
 Rating system and colour coding of annotations
Usability issue either prevents the user from completing their task or causes
significant confusion as to the operation of the user interface.
Usability issue results from significant inefficiency in completing common tasks (e.g.
navigation, layout, terminology, inconsistencies, etc.
Usability issue does not prevent completion but does cause some inefficiency or
annoyance with prolonged use.
Positive usability element which helps people accomplish their tasks efficiently,
effectively, or with great satisfaction.
Other comment, suggestion, or opportunity for helping to make the website, product
or service more useful and usable.
Critical
Major
Minor
Strength
Comment

Usability evaluation results
Consular

Section overview – Consular (1 of 4)
 Usability evaluation
 1 Critical, 6 Major and 10 Minor issues
 Summary of observations
 Strengths
 Layout of page is clear and crisp
 Search engine worked well in most instances
 Clear, salient and visible warning indications
 Critical usability issue
 Critical content is in different sections from that expected by users
 Recommend: Move information to the sections expected by users or provide
clear cross-reference links

 Summary of observations (continued)
 Major usability issues
 Too many competing links on the Home page
 Recommend: Minimize the number of links by creating clear areas of focus for
different types of information
 Difficult to discriminate several similar looking links on the Home page
 Recommend: Provide only one salient link on the Home page to Drugs and
Travel and one to Children and Travel
 Essential Resources section difficult for users to see in bottom right
 Recommend: Remove redundancies and move Essential Resources to a more
prominent location – possibly integrate into other sections
 Travel Reports are being missed because nothing is visible above the fold
 Recommend: Reduce the amount of text before listing country names
 People are not aware of Travel Report information because the Table of
Contents is often not recognized
 Recommend: Integrate Table of Contents into the central content area
 Redundant content can often hide valuable resources
 Recommend: Make descriptive text more concise, structured and bulleted to
facilitate scanning

 Minor usability issues
 The Map and Travel Report drop-down menu on Home page were not well
utilized – makes it difficult to justify the value of the map given the large
amount of screen real-estate it takes up
 Recommend: Eliminate the map and make the Travel Report drop-down menu
more salient, including possibly changing the name
 No link to Embassies or Offices Abroad from the Home page
 Recommend: Provide a link on the Home page to Offices Abroad
 Popular Travel Reports section not used
 Recommend: Reduce the amount of competing information on the Home page
 Obscure titles makes finding useful guides difficult
 Recommend: Use descriptive titles for link labels – e.g. Travel Tips for Women
rather than Her Own Way
 Logos do not link to Home pages
 Recommend: Link logos to their respective Home pages

 Minor usability issues (continued)
 Passport page is not task oriented
 Recommend: Provide a short list of annotated links which take people directly to
the content pertinent to their task
 Long drop-down menus are difficult to use
 Recommend: List countries in a fashion similar to that used for the Country
Reports but arranged into 3 or 4 columns
 Colour mapping of recommendations does not follow standard convention
 Recommend: Use conventional colour mappings for Cautions and Warnings
 Unstructured text difficult to scan for desired information
 Recommend: Provide suitable sub-headings and provide easy-to-see links to
additional detail
 Drop-down menus not accessible via keyboard
 Recommend: Use a submit button to make drop-down menus more accessible

User comments – Consular
 Information of most interest to participants
 Documentation requirements, country information, travel planning and
being able to check requirements with a local embassy
 Would come to DFAIT website if concerned about travelling to countries
which may be in conflict or which adjoin countries at war but wouldn’t
think to go to the website if travelling to Europe
 Travel agent currently uses phone book to get contact information for
local embassies
 Other information needs
 Travel agent wanted to know more about transferring “odd items”, like
those needed for handicapped travel – may be more of an airline issue

Poor use of Home page real estate
Minor: Map not being used. The
map provides good access to
countries in each section but it is
not being used. Only two people
clicked on the map. It takes up a
lot of valuable screen real estate
for the amount of value it
provides.
Recommend: Eliminate the map
and provide easier access to the
more commonly accessed
countries. If the map is kept,
allow people to zoom into a
specific region to continue their
selection of a given country using
the map.
Minor: Travel Report drop-down
menu not used. This valuable
short-cut was not seen or used by
any of the participants.
Recommend: Make this feature
much more salient. The phrase
Travel Report may not be
communicating the best.
Minor: No link to embassies
abroad. Several people expected
to find a link to Canada’s offices
abroad on the left navigation bar.
Recommend: Provide a link on
the Home page to Offices Abroad,
including their contact information.

Too many competing links on Home page
Major: Too many
competing links on the
home page. People often
had difficulty deciding on
where to look for information
on the Home page because
very similar links exist in
several locations.
Recommend: Minimize the
number of links by creating
clear areas of focus for
different types of information.
Comment: Why are
travel reports and
updates spread over
a wide area of the
Home page.
Strength: Dual coding of
Travel Warnings using both
colour and an asterisk helps
all users, including those with
colour perception anomalies.

Similar links vie for attention
Major: Difficult to
discriminate what look like
redundant links. People
found multiple links to what
they perceived to be the
same information.
These multiple links clutter
the home page and make it
difficult for people to choose
which link to follow.
Recommend: Provide only
one link on the home page to
information about Drugs and
Travel and one about
Children and Travel.

Essential Resources often not seen
Minor: Popular Travel Reports
not used. No one made use of
the link to the Kenya travel report
under Popular Travel Reports.
Recommend: Reduce the
amount of competing information
on the Home page.
Major: Essential resources lost on bottom
right. People had difficulty finding links
within Essential Resources. Many people
dismiss right-hand content as being
unimportant.
Recommend: Remove redundancies and
move Essential Resources to a more
prominent location.
Minor: Obscure titles do not help people
find information. No one was clear what the
link Her Own Way would lead to.
Recommend: Use more descriptive link
labels – e.g. Travel tips for women. Make
people more aware of valuable guides.

Search engine works well
Strength: Search engine
worked well, returning a
manageable number of hits.
Strength: Desired document
was the first hit and other hits
were relevant to issues with
women travelling in general.

Country listing for Travel Reports not visible
Major: No Travel Report countries
visible without scrolling. A couple
of people thought this was just
descriptive text and did not notice the
country listings which were below the
fold (not visible without scrolling). Two
of them clicked on the Travel Reports
and Warnings link in the left-hand
navigation bar from this page,
thinking they were in the wrong place.
Recommend: Reduce the amount of
text before the listing of country
names. Also ensure that people are
aware of what page they are on so
they don’t click on a link to the same
page. The link should not be active
and it should be highlighted in some
way to show it is the current page.
Minor: Logos do not link to
home page. People expect to
be able to click the logos to get
to Home pages.
Recommend: Links logos to the
respective Home pages.

Passport page not task oriented
Minor: Passport page is not
task oriented. People did not see
link buried at the bottom of this
page for dealing with a lost
passport. The text shown does
not support common tasks of
determining eligibility, acquiring a
passport, renewing a passport,
reporting a lost passport, etc.
Recommend: Provide a short list
of annotated links which take the
user directly to the information
relevant to their task – in this
case, reporting a lost passport.

Drop-down menu difficult to use
Minor: Long drop-down
menus are difficult to use.
People were able to use the
long drop-down menu of
country names, but with
difficulty.
Recommend: Provide
access to Government
Offices Abroad using a listing
more similar to that used for
Country Reports (see below).
One improvement would be
to list the countries in 3 or 4
columns, rather than in rows
with vertical separators.

Table of Contents not seen or used
Major: People not aware of
breadth of information in
Travel Report. Several
people did not notice the
Table of Contents. They
started scrolling and, because
it was so long, they left before
they found the information
they were looking for – e.g.
Embassy contacts or Local
Laws.
Recommend: Put the Table
of Contents in the middle with
two columns. The left column
should contain the most often
sought after information.
Combine Warnings and
Recommendations with Safety
and Security as people can’t
discriminate them.
Strength: Good to have a very
clear, salient and visible warning.
Minor: Colour of status
message does not
support the message.
Blue was not associated
with Caution.
Recommend: Use
conventional colours to
emphasize the
recommendations – like
the red used in the lower
example. In this case,
make Caution an amber
colour.

Redundant content hides valuable resources
Major: Redundant content can
hide valuable resources. No one
found this document on Dual
Citizenship. It does an excellent job
of explaining the Risks and
Problems.
Recommend: Make the descriptive
text on the main page more concise
and bulleted. Move the Dual
Citizenship link from the bottom to
the top of this section.
Comment: People expected this
page to contain a list of countries in
which Dual Citizenship can be a
problem and provide links to the
relevant information.

Text not structured for ease of scanning
Comment: It would be good
to list some of the key
conditions of which people
need to be aware.
Critical: Information not where expected. Most of the people tested
expected the information regarding dual citizenship issues to be under the
heading of Entry/Exit Requirements.
Recommend: Move this information to the Entry/Exit Requirements section.
Minor: Text difficult to scan
for desired information.
People often had trouble
finding the relevant
information in this section
because there is so much
unstructured text.
Recommend: Provide
suitable sub-headings
regarding things like Legal
Rights, Drug and Alcohol
Laws, Compulsory Military
Service, etc. Instead of
simply stating “You are
subject to local laws”, it would
be better to provide a listing
of some of the major
differences and a link to more
detail.

Drop-down menus not keyboard accessible
Minor: Drop-down menus not
keyboard accessible. People trying
to one of these drop-down menus
using just the keyboard can only
access the first entry – Afghanistan.
Recommend: This is not a major
issue because there are alternate
ways of getting to the same
information. These drop-down
menus could be made accessible by
requiring the use of a Go or Submit
button.

Highlight videos – Consular
 Highlight video descriptions
 Difficulties with finding the link to Canadian offices abroad
 First person (travel agent) goes to Passports and misses the link to
Canadian offices abroad. She then tries While Abroad and Problems.
She finds Passport Services Abroad and then clicks on Canadian
offices abroad the second time around.
 Second person goes to the Travel Report for Poland and clicks on
Canadian offices abroad. He does not notice that the Travel Report
has the information he’s seeking.
 Third person reads right over the Canadian offices abroad link but is
unsure where to go next. She looks at Loss of Belongings and
Passports and then wanders aimlessly. After a prompt she finds the
Canadian offices abroad link.
 View video (Consular-PassportStolen.wmv – 5:56)

 Problems interpreting and using Travel Reports
 First person (travel agent) goes to Travel Reports and then clicks on
Travel Reports again, not aware she is already there.
 Second person goes to Travel Reports and doesn’t notice any country
listings (they are just off-screen). When asked to return to the page,
she sees the country listings but doesn’t think they will provide the
information about potential dangers in travelling in a particular country.
She starts searching a variety of other locations but is very unsure of
where to look.
 View video (Consular-TravelReports.wmv – 3:46)

 Difficulties finding Contact Information
 First person misses Contact Info link on the main page even though it
is there.
 Second person misses the Contact Info link on the main page of the
Romanian Embassy and then can’t find it on any other page.
 Third person has difficulty finding Contact Info for the Romanian
Embassy. He doesn’t see the Contact Info in the Travel Report.
Instead he clicks on Canadian offices abroad and then has to get back
to Romania.
 View video (Consular-ContactInfo.wmv – 3:18)

 Difficulty find publication for women travelling alone
 First person thinks publication should be renamed using more
meaningful terms.
 Second person starts scanning a variety of links. She goes to Before
You Go but is surprised she can’t find anything. She tries FAQ and
Quick Links, then starts to get frustrated.
 The third person, even after a hint, cannot find Essential Resources or
the Her Own Way publication.
 View video (Consular-HerOwnWay.wmv – 5:22)

 Difficulties trying to find any issues with citizens travelling to Greece
 Person looks under Dual Nationality but encounters difficulties once she
starts looking under Foreign Offices in Canada.
 View video (Consular-GreekCitizens.wmv – 1:34)

eDiscussion and
START

Section overview - eDiscussion (1 of 4)
 Strengths
 eDiscussions page has simple layout with clear headings for most sections
 START offers good imagery and coverage of Canada’s activities abroad
 Critical usability issue
 No links to information about eligibility or how to participate in funding programs
 Recommend: Provide readily accessible links to Eligibility and How to Apply
information
 Title of current eDiscussion difficult to find
 Recommend: Move current eDiscussion to the top of page and give it a sub-heading
under Policy eDiscussions
 Policy Paper details difficult to find – not expected under eDiscussions
 Recommend: Consider changing the section name to Policy Discussions with sub-
sections for Online Discussions, Position Papers, Videos, etc.

Section overview - eDiscussion (2 of 4)
 Major usability issues (continued)
 No links from summary of Policy Position Papers to actual papers
 Recommend: Provide links to relevant resources, like a list of Policy Position Papers,
how to submit a Policy Position Paper, etc
 Annotation associated with Video Library is misleading
 Recommend: Change link label to something more generic, like Video clips on a
variety of international issues
 eNewsletter subscription form is confusing
 Recommend: Clarify and simplify form to make it clear which fields and buttons to
use
 Global Issues not an obvious link to find the START program
 Recommend: Consider a label change to Foreign Policy Programs or Canada’s
Involvement or something similar

Section overview – eDiscussion (3 of 4)
 Relationship between eDiscussions and Policy Position Papers is unclear
 Recommend: Clarify the relationship between Policy Position Papers and
eDiscussions
 Key words for some video titles are not in the primary scanning area
 Recommend: Place key words at the beginning of the title – e.g. Children in War –
Monitoring and Reporting on Violations
 Difference between Foreign Policy and Canada in the World is unclear
 Recommend: Consider changing sub-link labels to help differentiate the two – e.g.
Activities by Country and Region
 Contact information is non-specific to content areas
 Recommend: Provide specific, context-sensitive contact information rather than
general enquiry contact information
 Active Home links on the Home page creates confusion
 Recommend: Have only one Home link and make it inactive when the user is on
the Home page

Section overview – eDiscussion (4 of 4)
 Sub-sites change look and feel plus link to different Home pages
 Recommend: Maintain a consistent look and feel through the website and sub-
sites
 eNewsletter link leads to an unexpected page regarding RSS and Travel
 Recommend: Ensure eNewsletter subscription leads to simple subscription page
and provide extra links to RSS or Travel information
 Navigation breadcrumbs disappear on some pages
 Recommend: Provide consistent breadcrumbs throughout the website
 Multimedia link is associated with more than just Podcasts
 Recommend: Change link label to Podcasts and provide cross-links to other
multimedia content

User comments – eDiscussion
 eDiscussion landing page comments
 Clear headings segment sections well
 Good to have University-specific link
 Would like to see more University and NGO content
 Don’t like that visited links fade and are no longer obvious
 START landing page comments
 Not so interested in news as in links to content
 News content makes sense
 Didn’t notice that red blocks on right were links, thought they were titles
 Would like information on how to get access to funding
 Assumed there would be brochure describing the Peace and Security
Funding Program

Position Paper not expected with eDiscussion
Major: Current eDiscussion
not visible enough. A few
people had difficulty finding, or
did not find, this information
about the Current eDiscussion.
Recommend: Move this
information to the top and give it
a sub-heading under Policy
eDiscussions. Put the subscribe
links under the Participate
header.
Major: Policy paper not thought of as
part of an eDiscussion. People had
difficulty finding how to submit a Position
Paper because they did not associate a
physical paper with an eDiscussion.
They thought of the latter was only
online
Recommend: Consider changing the
section name to Policy Discussions with
sub-sections for online discussions,
position papers, videos, etc. Also
highlight the phrase Policy Position
Paper and make it a link, similar to the
way Participation Guidelines are handled
under Participate.

Missing link from summary to specific papers
Major: No links from
summary of policy position
papers. People were
surprised to find no links from
this summary page to the
specific Policy Papers or to
how to submit a Policy Paper.
Recommend: Provide links to
relevant resources – e.g. list of
Policy Papers, how to submit
a Policy Paper, etc.
Minor: Relation between
eDiscussions and Policy
Position papers unclear.
People did not understand
how eDiscussions (thought to
be an online discussion)
related to Policy Position
Papers. They did not think the
two were related.
Recommend: Clarify the
relationship between Policy
Position papers and
eDiscussions.

Video annotation misleading
Major: Annotation
confusing to some users.
Half the people did not
expect to find videos
regarding Violations against
Children in War in this
section because they did not
associate this type of content
with foreign policy thinkers.
Recommend: Change
annotation to something
more generic, like Video clips
on a variety of international
issues.
Strength: Most people saw
this link quite quickly.

Key words not in primary scan area
Minor: Key words not in
scanning area. People often
missed finding these videos
on Children in War because
the title is quite long and the
key words are at the end of
the sentence. People often
scan only the first 3 or 4
words on the left.
Recommend: Place key
words at the beginning of the
title – e.g. Children in War –
Monitoring and Reporting on
Violations.
Strength: It is good to group
a number of common
interviews under a smaller set
of themes.
Comment: If this list
continues to grow it might be
worthwhile providing an index
to the themes or providing a
simple find feature, just for
the videos.

Contact information not specific
Minor: Contact information
is non-specific. People were
expecting to find someone to
contact regarding Canada in
Sudan but instead got a
generic list of contacts (see
below).
Recommend: Provide
context-sensitive contact
information to help people
find the appropriate
resources for a given topic.
Minor: Change of website
unexpected. People were
confused by this different
looking website which still
seemed to be dealing with
Foreign Affairs.
Recommend: Maintain a
consistent look and feel
throughout the website and
sub-sites.
Minor: Home links to
current page. People were
confused when they clicked
on Home and nothing
happened. They did not
expect this to be the Home
page.
Recommend: Have only
one Home link and make it
inactive when the user is
on the Home page.

Subscription form confusing
Minor: Destination page does not match user
expectation. People expected to see a form to sign
up for an eNewsletter but encountered RSS feeds
and Travel information instead.
Recommend: Ensure the title and content of the
destination page match the link label.
Major: Unclear as to which form
fields to use. Several people had
difficulty interpreting this form.
They often didn’t see the button at
the bottom of the page. Two
people thought the second email
field was for Unsubscribing
because of the text above it.
Recommend: Clarify the form to
make it more evident which fields
and buttons to use.
Minor: No breadcrumbs
available. People expected
consistent breadcrumbs.
Recommend: Provide
consistent breadcrumbs.

Multimedia link implies more than podcasts
Major: Global Issues not an obvious
link to get to START program. Only
two people found the START program
under Global Issues without a hint. The
others thought it might be under
Canada in the World. They did not
associate Global Issues with programs.
Recommend: Consider providing a link
to Foreign Policy Programs or changing
the link label to something like
Canada’s Involvement.
Minor: Multimedia link
associated with more than
just Podcasts. Some people
looked under the Multimedia
link to find videos and were
surprised there were none,
nor links to video resources.
Recommend: Change the
link label to Podcasts if that is
all it contains and provide
cross-links to other
multimedia.

No links to participation information
Strength: Details about the
program and how to apply
are readily available.
Critical: No links to how to
participate in program. People
could not find any information on
program eligibility or how to apply
for funding for the GPSP or GPOP
programs. There is a lot of
description but no links for people
to take action.
Recommend: Provide readily
accessible links to help people
determine their eligibility for the
program and how to apply for
funding.

Distinction between major sections unclear
Minor: Difference between sections
unclear. A few people were confused
as to the difference between the
section on Foreign Policy and that of
Canada in the World.
Recommend: The distinction may be
made clearer by changing some of
the sub-link labels – e.g. Activities by
Country and Region.

Highlight videos – eDiscussions and START
 Can’t find current eDiscussion
 Person scans the page but still cannot find the current topic of
discussion.
 View video (Policy-CurrentDiscussion.wmv – 1:15)
 Difficulty using the subscription page for the email alerts
 Several people have difficulty interpreting the page for email alerts.
They are unsure which fields they need to fill in.
 View video (Policy-eNewsSubscription.wmv – 1:15)
 Policy Position Papers not expected with eDiscussions
 People comment on not expecting Policy Position Papers being
associated with eDiscussions. They think the later is strictly online and
does not apply to submission of a paper.
 View video (Policy-PositionPaper.wmv – 2:12)

 Video annotation is misleading
 Several people have difficulty finding videos of Children in War because
the annotation misleads them. It talks about interviews from foreign
policy thinkers which they do not associate with Children in War.
 View video (Policy-VideoAnnotation.wmv – 2:36)
 Media Room confused with multimedia content
 Person goes to Media Room looking for multimedia content. She was
expecting audio/visual content.
 View video (Policy-VideoLocation.wmv – 1:20)

 Difficult to find START program
 People are not expecting the START program to be under a link called
Global Issues. They associate Global Issues more with news and current
affairs.
 View video (Policy-FindingSTART.wmv – 1:18)
 New windows and different look unexpected
 Person is not expecting a new window to open with something that looks
like a whole new web-site. She is trying to find out about Canada’s
involvement in Sudan.
 View video (Policy-SudanSubsite.wmv – 0:26)
 Multiple Home pages can be confusing
 Person is confused by the Home link taking her to the same page she is
already on. She expects it to take her to the DFAIT home.
 View video (Policy-HomeConfusion – 3:08)

Trade
negotiations

Section overview – Trade (1 of 4)
 Strengths
 Provides access to a wide range of information affecting international trade
 Regularly updated with ability to get notified of updates
 Critical usability issues
 Business travel regulations not expected under Trade Negotiations link
 Recommend: Provide a cross-link to business travel information from the Services for
Travellers pages
 Business travel information inconsistent in different parts of website
 Recommend: Ensure consistency of information found in different parts of website
and put information where people expect to find it
 Trade barriers not associated with the Market Access Report
 Recommend: Consider changing name to Trade and Investment Barriers

Summary of observations (continued)
 Link labels like ExportSource and InfoExport do not communicate well
 Recommend: Create more meaningful link labels which communicate the destination
content better
 Valuable right hand content and links are often overlooked (x2)
 Recommend: Explore ways to integrate the right side content with the central content
 Links often difficult to scan and not in user language
 Recommend: Arrange key links in a bulleted list in the central scan area of the page
 Search was not working during the test sessions
 Recommend: Fix search engine and provide more helpful error messages which
clearly state the reason for a problem and how to correct the situation

 Difficult to read and discriminate multi-line link labels in left navigation
 Recommend: Increase spacing between link labels or use a fine rule line
 Confusion about Export and Import Controls link on left and right
 Recommend: Make the relationship clear between links which are repeated on a
page
 Several pages have active links back to the page the user is on
 Recommend: Do not provide an active link to the current page
 Logo does not link to Home page as expected
 Recommend: Link the logo to the Home page
 Descriptive text often does not provide links to content being described
 Recommend: Provide active links at the point the user is reading about the
content or resource provided

 Key words are buried in prose and neither highlighted nor linked
 Recommend: Highlight key words and structure text for ease of scanning
 Fields required for Trade Barriers search are unclear
 Recommend: Clarify instructions so people start with the minimum input
required
 Some pages are quite verbose with little structure to support quick scans
 Recommend: Make text more concise and use sub-titles and bulleted lists to
structure the text for easier scanning
 Table of Content links in the right column are often not seen
 Recommend: Move the On this Page links into a more central area

User comments – Trade
 Comments on Trade Negotiation and Agreements landing page
 Perception that home page is cluttered and contains of a lot of text
 “Le site est lourd”, “La page est chargée”
 Would like to see a search box
 Navigation breadcrumbs are good to have
 Would like to see Archives to see how trade and negotiations have
changed over time with different countries
 Right hand topics are useful but graphics take away from importance
 News in central area is good to see how things are changing
 Titles are often not related to what you find
 Surprised to see Guide for Canadian Municipalities under International
Trade Agreements
 “Links on left is what I choose from, not sure of what is on the right”
 Assumes content in centre changes rapidly, content on right less so
 Thinks elements on the right are “flavour of the month” things
 Trade Matters is a very vague phrase

Link labels ambiguous
Major: Right hand
content often
overlooked. Several
people did not see the
content on the right
hand side. They did not
see Dispute Settlement
even when asked to
scan the page for
anything related to
Dispute Settlement.
They commented that
they typically avoid the
right hand column as it
is usually advertising or
peripheral information.
Recommend: Explore
ways to integrate the
right side with the
central content more.
Reduce the use of
images on the right as
they often lead to the
impression that this is
just advertising for
other sites.
Minor: Difficult to read link
labels. It was unclear to users
where a link label started and
ended without rolling the mouse
over the link.
Recommend: Increase the
spacing between link labels a bit
or use a fine rule line between
them.
Major: Link labels are
ambiguous. These link labels
often caused confusion. People
were unsure what they would find
under ExportSource or InfoExport.
They also thought that Export and
Import Controls covered more than
just Canada.
Recommend: Create more
meaningful link labels which give
people an accurate expectation of
the underlying content.

Multiple links with same name confusing
Minor: Confusion about
difference between Export and
Import Controls on left and
right. Several people queried why
there was something about Export
and Import Controls on both the
left and right. They were unsure if
they were the same.
Recommend: If the intent is to
elaborate on the content behind a
link label, then create a direct
mapping to the left navigation bar
and explain each of the links in
order. This way people will see the
relationship.

Business travel info not expected under Trade
Critical: Unexpected to look under Trade
Negotiations and Agreements for business travel
information. Not one person thought to look under
Trade Negotiations and Agreements for information
about business travel to Mexico. Even after a hint,
people felt they were in the wrong area when they got to
NAFTA. All people initially wanted to look under Services
for Travellers (see next page).
Recommend: Provide a link to this information from the
Services for Travellers content pages.

Travel information inconsistent
Critical: Information is inconsistent with respect to
business travel. Information from the NAFTA section
(top left) provides a listing of documentation required for
a business traveller. However, it appears to differ from
that provided on the voyage.gc.ca site (bottom right).
The latter seems to imply the need for business, work
and visitor visas plus a tourist card.
Recommend: Clarify exactly what is required for
business travel in both places and ensure the
information is consistent. Provide a clear separation of
business and pleasure travel requirements. People
expect to find this information in the Entry/Exit
Requirements section of the Mexico Travel Report.

Right hand links difficult to scan
Major: Links on right difficult to scan and not in user language. Some
people had difficulty finding the link to Cross Border Movement of Business
Persons. This is not the first area they scan (they scan the centre first) and
they had to interpret that this link was relevant to business travel regulations.
Recommend: Replace the centre description of the topics with the links on
the right. Each link can then be put on a single line, making them easier to
scan. Remove all right side links. Reword Cross Border Movement of
Business Persons to something like Business Travel Regulations.
Minor: Active link to
current page. People get
confused when they click on
an active link and nothing
seems to happen.
Recommend: Remove this
as an active link when the
user is already on this page.

Search not working
Major: Search not working. Several people
wanted to use search but it always returned an
error condition during testing. The error message
provided no help to the user as to what had gone
wrong or how to correct the situation.
Recommend: Fix search engine. If an error does
occur, ensure the person understands what kind
of error has occurred and what they can do about
it, if anything.
Minor: Logo does not link to home page.
People would often click the logo and expect it to
link to the DFAIT home page, but it does not.
Recommend: Link the logo to the Home page.

Trade barriers & Market Access not associated
Critical: Trade barriers not associated with
the link International Market Access Report.
Four out of five people could not find
information on trade barriers without a hint.
Recommend: Consider changing the name to
Trade and Investment Barriers. Most of the text
references this and it is closer to the language
of the user.
Major: Right menu items
only seen as last resort.
People typically scanned the
central content before
noticing the links on the right.
Recommend: Integrate
access to the database and
reports into the central
content.

Descriptions with no links to action
Minor: Description of trade barrier
database but no link. People usually
scanned the central content and found the
tool they were looking for but could not
access it. There were no active links in the
central area. They had to look elsewhere.
Recommend: Provide a clear sub-title for
the database and make the title an active
link to the database.
Minor: Key words are not
highlighted or linked. People are
scanning for trade or investment
barriers, not a database.
Recommend: Highlight the key
words and the action – e.g. Search
Trade & Investment Barriers. Also
make this the link to the database
so people don’t have to look
elsewhere to find the link. The
same recommendation applies to
the modifications which were made
after testing (see below).
Comment: More… is
ambiguous. Make the title the
link and eliminate the need
for this more… link.

Required form fields unclear
Minor: Required fields unclear.
Half the people tested tried to fill in a
Sector and Product Code when they
were not needed.
Recommend: Clarify the instructions
so people start with the minimum
information first. This is to prevent
them from overly constraining the
search and not finding anything
because of a mis-categorization of
the Sector or Product Code.

Verbose text makes it difficult to scan
Minor: Right side links not
used. People did not see the
links on the right which would
have got them directly to the
contact information. Instead
they had to scroll 5 or 6
pages to find the information.
Two people gave up before
they reached that point.
Recommend: Move the On
this page links into a more
central location.
Minor: Page is quite verbose with little structure to
help scanning. People want to be able to quickly scan
the page and pick up the key information but they found
this page to be too long and dense.
Recommend: Tighten up the text and make more use of
sub-titles and bulleted lists to organize the key points.

Opportunity to elaborate on link labels
Comment: There is an opportunity to use this
page real estate to annotate the links which are
also found on the left. This would help to
disambiguate the content associated with each
link. There is also an opportunity to provide sub-
links. These would help to define the parent link
content and provide direct access to some of
key information in each area. This is especially
important for some of the more generic links,
like Goods, Services and Further Opportunities.

Highlight videos – Trade negotiations
 Difficulty finding business travel document requirements
 Several people do not expect to find information about business travel
requirements under Trade Negotiations and Agreements. One person
thinks he’s in the wrong place when he gets into the NAFTA content
on the site. Another person has difficulty navigating the information on
the destination page for Cross Border Movement of Business Persons.
 View video (Trade-BusinessTravelRegs.wmv – 6:53)
 Difficulties finding Trade Barrier information
 People scan the central content but cannot find links to the content
they want. They comment on Market Access Report not being an
obvious choice for getting at Trade Barriers. They suggest using the
term Trade Barriers instead of Market Access. The last person
expresses frustration at not being able to find anything to do with
Trade Barriers.
 View video (Trade-Barriers.wmv – 3:08)

Highlight videos – Trade negotiations
 Export/Import label confusions
 People comment on not knowing the differences between
ExportSource, InfoExport, and Export and Import Controls. People
also question the difference between Export and Import Controls links
on the left versus the right.
 View video (Trade-ExportImportConfusion.wmv – 1:51)
 Right hand content often missed
 People have difficulty finding link to information on Dispute Settlement
even though it is in the right column. One person contends a dispute
should not be under a section about agreements.
 View video (Trade-RightSideNotSeen.wmv – 3:08)
 Circular links back to the current page
 Person explains his confusion about clicking on a link which seems to
do nothing. He finally realizes he has gone back to the same page but
can’t figure out why the link would be active.
 View video (Trade-LinkLooping.wmv – 1:33)

International
youth program

Section overview – IYP (1 of 4)
 Strengths
 Simple collection of programs and participating countries
 List of opportunities around the world which appeal to youth
 International Youth Program was not expected under Foreign Policy
 Recommend: Provide cross-links from Travel and Business sections and possibly
change the name to Work or Study Abroad
 No links from the Program descriptions to details or participating countries
 Recommend: Link titles of program descriptions to specific details regarding the
program and which countries participate

Summary of observations (continued)
 Link related to specific content goes to a generic Home page instead
 Recommend: Provide a link directly to the listing of International Jobs on University
website
 Textual descriptions of actions to take but no links to help user take action
 Recommend: Provide “deep links” to the detailed information or appropriate
resources at the point when the information is first presented
 Key information missing from French version of How Our Programs Work page
 Recommend: Ensure the same information is presented in both official languages
 Right side navigation is being missed by some people
 Recommend: Visually integrate right hand links with central content to make the
relationship to the current content clear

 Country drop-down menu not seen or used
 Recommend: Position the menu below the IYP header and change the prompt
to read Select Country
 No direct links provided to the described content (x3)
 Recommend: Provide direct links from the descriptive text to the associated
content and/or actions
 Active link to current page makes it look like link does nothing
 Recommend: Do not have an active link to the current page and provide an
indication of which link corresponds to the active page
 Link labelled Other Links is not very meaningful
 Recommend: Place related links at relevant points throughout the sub-site rather
than grouped under a generic title – e.g. link to Passport Office when talking
about the requirement for a Passport

 Difficult to compare program descriptions – not consistently formatted
 Recommend: Provide a matrix so people can compare programs across a
number of common dimensions
 Common synonyms for countries are not provided in country list
 Recommend: Provide common synonyms for country names in list
 FAQ list is long and unstructured
 Recommend: Group related questions together and label the grouping

User comments – IYP
 Appropriateness for youth
 Image gives good idea of lifestyle
 A couple of people felt more imagery of youth having fun in typical
situations abroad would be more engaging
 Current picture shows older people and looks too serious
 Not sure what Choose in drop-down menu means
 Prefer websites which are not too flashy – buttons are easy to read
 Would like menu across top with drop-down menus
 Not sure why print ads are on the website
 Would like information in a more straightforward manner – have to
rely too much on ride side menu bar
 Would like search box on each page

International Youth Program difficult to find
Critical: Location for International Youth Program
unclear. 80% of people could not find where to look
for information about working holidays. No one
expected it to be under Foreign Policy. They tended
to want to look under Services for Travellers.
Recommend: No one expected to find this kind of
information on the DFAIT website. We have no clear
recommendation on how to address this issue other
than through media promotion and changing the link
name to Work or Study Abroad. Cross links could
also be provided from the voyage.gc.ca and Services
for Business sections.

Right side navigation often missed
Major: Right side navigation is missed by
some people. About half the people ignored
the right hand navigation and focused on the
left hand navigation and centre. This led
some of them to leave the IYP pages and go
to sections like Employment and Training,
which is further down on the left.
Recommend: Links on right need to be more
visually integrated with central content. It
must also be clear that these links are the
local navigation for the current topic area.

No direct links to described content
Minor : Drop-down
menu not seen or
used. Not one person
noticed or used the
drop-down menu
shortcut to countries.
Even if someone
noticed it, the word
Choose did not
communicate it’s use.
Recommend: Position
the drop-down menu
below the header for
the International Youth
Program on the right
or move it to the
centre. Change the
prompt from Choose
to Select Country.
Minor : No direct links provided to described
content. People read the centre area but could not
directly link to the information of interest. They had to
look elsewhere to find the associated links.
Recommend: Provide direct links from the descriptive
text shown here to the associated content.
Strength: The
image worked
well at conveying
the kind of
experiences
provided by the
programs.

Active link to current page
Minor : Active link to
current page. People did
not think this was the
Home page for the
International Youth
program. They clicked on
this link even though they
were already on the
page.
Recommend: Do not
create multiple Home
pages. If this is a main
page, then name it
accordingly in the page
heading in the centre.
Ensure the link label uses
the same wording.
Ensure there are no
active links to the current
page. Provide an
indicator in the navigation
bar to show which link is
currently active.
Minor : Other Links not very
meaningful. People had no
expectations about what they would
find under Other Links.
Recommend: Place related links at the
appropriate point within the pages of
this sub-site. For example, link to
Passport office when talking about the
requirement to have a passport.

Link to generic home page rather than specific
Major: Link goes to generic home
page instead of specific content.
People had difficulty following up on the
link to Engineering positions abroad
because the link goes to the generic
UBC Engineering Home page, not the
ICE relevant content.
Recommend: Provide a direct link to the
listing of International Jobs rather than
making people figure out how to
navigate a new website.

No links from program titles to details
Minor : Difficult to compare working
holiday programs. People had
difficulty determining the essential
differences between the Working
Holiday Programs versus the SWAP
Working Holiday program. They were
not together, nor were the descriptions
consistent in terms of describing the
what, who and why of the programs.
Recommend: Consider providing a
matrix so people can see how the
various programs compare on a
number of different dimensions.
Critical: No links to programs. People
expected to be able to link to the
program to find more details about
eligibility, how to apply, etc.
Recommend: Make the titles links to
the details related to each of the
programs.

Description of process but no action links
Major : Text description of what to do
but no links to take action. People
could read about how the programs
work but there were no embedded links
to help them take the actions described,
such as finding a job in the country,
applying for a visa or contacting the
country’s embassy.
Recommend: Help people not only
understand what process is involved but
help them take action by providing “deep
links” directly to the appropriate
resources, forms, etc.
Comment: Most of the people coming to
this page will not care about any of this
first paragraph. They will want to find out
what programs are available, if they are
eligible, and how to proceed.

Key information missing from French page
Major: Key information
missing from French
version of page. French
participant was not able to
complete the task regarding
eligibility requirements
because the French version
of the page was missing
some critical information
presented on the English
version.
Recommend: Ensure the
same information is
conveyed in both languages.

No active links from program/country listing
Minor : People looking for other common names for
country. A couple of people missed Netherlands because
they were scanning for Holland.
Recommend: Provide common alternate names for
countries, when they exist – e.g. Netherlands (Holland).
Minor : No active links in listing of
programs and participating countries.
When people found the combination they
wanted in the description – e.g. working
holiday in Netherlands – they could not
directly link to that country’s program.
Recommend: Make the country names at the
top link directly to the information about the
specific program and country combination.
List the countries in 2 or 3 easy-to-scan
columns under each type of program.

FAQ list long and unstructured
Minor : List is long and
unstructured. People found
it inefficient to scan the FAQ.
Recommend: Group related
questions together, provide
sub-headings for these
groups, eliminate the Q: in
front of each one and make it
a bulleted list. For example,
group all the questions into
categories of participation,
applying, costs, programs,
etc.

List of programs not linked to details
Minor : Listing of programs
not linked to details. This
was not tested but has the
same problem as in other
areas – a list of programs
which are not linked to the
descriptions. People should
not have to look elsewhere
for the links.
Recommend: Link these list
items to their respective
program descriptions.

Opportunity for cross-links
Comment: Opportunity to put
cross-links in to the
International Youth Program
from here, since it applies to
this country.

Highlight videos – Int’l Youth Program
 Difficulty finding the path to the International Youth Program
 Several people have difficulty finding their way to the International
Youth Program from the DFAIT home page. They don’t associate a
working holiday with a link labelled as Education and Youth. The
tendency is to look under Travel or Business services.
 View video (IYP-LocNotObvious.wmv – 3:29)
 Expectation that program titles would link to more information
 Person expects program descriptions to link to more details. She is not
aware of the links on the right hand side.
 View video (IYP-ProgramsNotLinked.wmv – 0:55)

Highlight videos – Int’l Youth Program
 Difficulty finding eligibility information
 Person expects to find eligibility information under the FAQ link. She
has no clue where to find it elsewhere. She tries to venture off-site.
 View video (IYP-EligibilityCan'tFind.wmv – 1:13)
 Active link to current page
 Person clicks on link to International Youth Program – Home even
though she is already on that page.
 View video (IYP-ClickHomeFromHome.wmv – 0:16)

Russian mission
site

Section overview – Russia (1 of 2)
 Strengths
 Good info on working/studying/visiting/moving to Canada
 Expands awareness of Canada’s culture
 Helps Russian immigrants understand English terminology
 The link to the Trade area of the site is missing from the main menu
 Recommend: If access to the Trade area is important, put a link to it on the Russian-
language page as is available on the English version
 Not all English menu items are on the Russian-language version of the menu
 Recommend: Match the menu items exactly in the Russian-language version; if no
translation is available, divert users to the English page
 Links on the right hand side are not evident to users
 Recommend: Move links to key tasks into the centre area, or visually tie links on the
right to the centre content; underlining links may help increase the visibility of clickable
words

Section overview – Russia (2 of 2)
 Critical usability issues (continued)
 Key links to the Trade area of the site are not visible because they are on the
right hand side
 Recommend: Put links to the Trade Commissioner service in centre content OR
visually tie the links to centre content so users can notice them
 Trying to get to the Russia site from a search engine can be difficult
 Recommend: Offer links to search Mission pages from the DFAIT main search
page. Determine users’ key search phrases and use them on Mission pages so
Google-users can reach the correct page directly
 Links on right hand items do not appear clickable
 Recommend: Make links to key content underlined
 There is no indication of where the user is in the site
 Recommend: Provide a visual distinction for menu items at left when the user is on
the page to which the link corresponds; extend breadcrumbs throughout the site
 Link labels do not match the heading on the page to which they lead
 Recommend: Make sure link wording matches the headings on the destination page

Closing question: Translation? – Russia
 At the end we asked users – “does the translation make
sense?” These are their responses:
 Match the English menus
 Russian menu items are not complete (match English menus)
 Russian is missing links About Canada, Doing Business With Canada,
Armenia and Uzbekistan
 Match the English content
 Put the English information in Russian – people need to clarify
 Provide more in the Russian section of the website
 If people don’t know English it is handy
 Check the wording
 The Russian words are borrowed from English
 Some of the endings of words seemed odd
 Some spelling mistakes

Strengths of the pages we tested
Comment: This was the
Russian-language version we
tested.
Strength: People could easily
find their way back to the
home page. Breadcrumbs
helped.
Strength: Main links are often
in the centre content.
Participants primarily scanned
the centre area.Strength: People
appreciate having the
same words in English
and Russian, in the
same order, as they flip
back and forth between
languages.
Strength: The left-hand menu
is used consistently in
Russian anyway. People
quickly got used to it.
Strength: The picture helped
one user remember the
cultural link.

Finding the Trade area in Russian
Critical: The link to the Trade area of the site is
missing from the main menu. Several users noted that
they expect the main menu to be the same in Russian as
it is in English. The Doing Business With Canada menu
item is not on the Russian-language menu.
Recommend: If access to the Trade area is important,
put a link to it on the Russian-language page as done on
the English version.
Comment: Perhaps if
enough users suggest the
CLF-translation of Contact
Us then Google Translate
will more accurately reflect
this translation.
Comment: The Contact Us page did not provide Trade-
related contacts. For this task, only one user tried to
click on the CLF button.

Armenia needs to be on the Russian page
Critical: Not all English menu items are on the
Russian-language version of the menu – in this case,
the menu items for Armenia and Uzbekistan. All
participants said it should be. As one said “there are
Russian-speaking people in Armenia”. Though one said
Uzbeks may be less likely to speak English than Russian.
One user said pages should be “synchronized with
English pages”.
Recommend: Match the menu items exactly in the
Russian-language version. If no translation is available,
divert users to the English page.

Finding the Trade area on the right side
Critical: Key links to Trade services are not
visible because they are on the right hand
side. Two people explained that they “do not
pay attention” to items on the right. Another
“was looking on other parts of the page”.
Another said they “always look on the centre”
because the right hand side “looks a little like
advertisements”. One said it might help if the
colour of the heading bar was red like other
heading bars.
Recommend: Put links to the Trade
Commissioner service in centre content. Or,
visually tie the links to centre content so users
can notice them.
Comment: Note what the page text looks like
when the user enlarges the text of the page
using their browser. Links span more than one
line, adding to the time it took to scan these
links. Note the CLF bar also spans more than
one line.

Right-hand links are not evident
Critical: Links on the right hand side are not
evident to users, in this case links to an
Information Session. All participants said they
do not look on the right, but scan the middle of
the page; “the left is for main links, the middle
for main links for this page, and the right I don’t
pay attention to”. Another was “dissatisfied” on
this task, but would have been “very satisfied” if
the menu item had been on the Russian pages.
Even the angle-bracket which normally draws
the eye, did not help in this case.
Recommend: Move links to key tasks into the
centre area, or visually tie links on the right to
the centre content. Underlining links may help
increase the visibility of clickable words.
Minor: Links on right hand items do not
appear clickable. Two participants were not
sure what text in the Trade box was clickable.
Recommend: Make links to key content
underlined so users can scan for them and
compare them easily with other key links.

An unfruitful search experience
Comment: This user expected to see a link
to a consulate at this point. She reached this
voyage.gc.ca page after clicking the
Consular site on the search engine.
Strength: DFAIT ranks high on a Google search for visa information.
Minor: Trying to get to the
Russia site from a search
engine can be difficult – in
this case when searching for
Armenia visa information. This
Google search about Armenia
points to a Youth page on the
main DFAIT site. After two
fruitless on-site searches, this
user clicked the “Consular
site”. Reaching voyage.gc.ca,
the user could not find the
Russia site.
Recommend: Offer links to
search Mission pages from the
DFAIT main search page.
Determine users’ key search
phrases and use them on
Mission pages so Google-
users can reach the correct
page directly.
But, it leads to a
404 error.

Users need to know where they are
Minor: There is no indication of where
the user is in the site. Breadcrumbs
helped in some circumstances. When
asked “do you know where you are in the
site”, users did not always know. In some
cases, users would browse deep in the
content, which they appreciated, but
moving to related pages in that area was
difficult.
Recommend: Provide a visual distinction
for menu items at left when the user is on
the page to which they link. Extend
breadcrumbs throughout the site.

Link doesn’t match target heading
Strength: The
link matches the
heading in
English.
Comment: It is fixed! The page content was updated after
the test – so the link now matches in Russian.
Minor: Links are not the same as
the heading to which they lead. In
this case, a user took a while to
figure out that the link and
destination page were related.
Recommend: Make sure link
wording matches the headings on
the page to which they lead the user.

Home page link to business cooperation
Comment: This page caused a little confusion. Clicking this home
page link leads to a page about a trip to Russia instead of business
development in Russia. Users recovered easily, following the
instruction to try the English site instead.
Comment: It may be linked to Hotels and
trip to Russia because on the English page
the link is for Canadians Abroad.
Note: This is a
translation of
the Russian.

Highlight videos – Russian mission site
 Difficulty getting to the Trade area of the site
 First user scans the embassy and consulate but says “there is
nothing about business or trade”. On the bilateral relations page, she
expects a link to the trade section – but there is none. To show how
much effort this adds to browsing, she says “I would probably –
without your help – click on every link to find the Trade portion”.
 Second user does not see the Trade links because he “was looking
on a different part of the page” and says the link “should be in the
middle” of the page. He says it would be “better if it was more clearly
visible … in the centre… or a better colour” or one of the links in the
centre.
 View video (Russia-HardToFindTrade.wmv – 3:08)

Highlight videos – Russian mission site
 Highlight video descriptions (continued)
 Not all English menu items are on the Russian-language version
 First user explains that “only half the links were present” making it difficult
to find the Trade section of the site, and comments “I’m always
disappointed when I find some information in one language that is missing
in the other one”.
 Second user is very dissatisfied because it is hard to find information
related to Armenia; “If Armenia were listed on the left, like in English, it
would be easier, I would get there”.
 Third user can not find information for people from Armenia, because “in
Russian version there is no such link”, but he expects there should be.
 View video (Russia-MenuMismatch.wmv – 2:08)

DFAIT Usability Final 6 Apr08

Recommended

Recommended

More Related Content

Viewers also liked

Viewers also liked (20)

Similar to DFAIT Usability Final 6 Apr08

Similar to DFAIT Usability Final 6 Apr08 (20)

DFAIT Usability Final 6 Apr08