Slides used during talk on focus groups and other audience techniques, with Shelley Boden at Univ of Brighton.

1. Usability and evaluation –
focus groups and other
techniques
Martin Bazley
University of Brighton
25 Nov 10

2. Martin Bazley
Previously
• Teaching (7 yrs)
• Science Museum, London,
Internet Projects (7yrs)
• E-Learning Officer, MLA South East (3yrs)
Currently
• Vice Chair, DLNet (was E-Learning
Group for Museums, Lib, Archives)
• Consultancy, websites, training, user
testing, evaluation …
Martin Bazley & Associates
www.martinbazley.com
•
Slides and notes available afterwards

4. www.martinbazley.com

5. Users don’t always ‘get’ what we
are offering:
a real–world analogy

6. Even a slight difference in viewpoints…
…can cause real problems for users

13. In a conflict between visual affordance…
…and written instructions
visual affordance almost always wins

14. Another example

15. So what is the point of all this?

16. People interact with digital material
differently…

17. … from the way they use books, object
labels, magazines, newspapers, etc

18. For most people the web is a predominantly
visualmedium

19. Website usersWebsite users

20. Website users
• Who uses your website?
• Why would they want to use it?
• How would they find it?
• What do they get out of it?
• What do they dislike about it?

21. How do you get it right for
everyone?
• Answer:
• You can’t get it right for everyone.
• You have to make choices, and stick to
them:
• Who is it for?
• What..
• How…

22. Who for…?Who for…?
What for?What for?
How will they use it?How will they use it?

23. Who for what for ...
• Who for? (audience)
Need to be clear from start
• mum + 2 children looking for something to do this
weekend
• teachers of yr5/6 in local area with whiteboards
• men interested in gadgets

24. Who for what for ...
• What ‘real-world’ outcomes?
What will they do as a result of using the site?
• make a donation
• plan a visit to a museum
• buy a train ticket
• think differently about learning disability

25. Who for what for ...
• How will they use it? (user experience)
What do they actually do on the site?
• browse and read articles
• working alone or in pairs? (learning resources)
• lean forward or sit back?
• Browsing, following, searching…
• Also Where, When and Why?

26. Who for what for ...
• Website appraisal
– For each example note first impressions
• Who is it for?
• What does it offer them?
• How will they use it?

27. Websites for differentWebsites for different
audiencesaudiences

28. Websites for different audiences
The following tips are based on
• numerous evaluation sessions
• numerous user testing sessions
• talking to other people who use websites
• talking to other people who make websites

29. ‘General users’
There is no such thing as a general user
Are you a general user?

30. Adults/families with general
interest
• What does the site tell me at a
glance?
• genuine enthusiasts = interested whatever
the website looks like, and will spend
some time looking around it or phone up
for more information if required.

31. Adults/families with general
interest
• But most will not bother unless something
engages them within a few seconds
• The questions people might like answered
within a few seconds of arriving on a
museum site probably include:

32. Adults/families with general
interest
• Where is it?
Further down the home page it says
‘alongside Middle Wallop airfield’ but I
have no idea where that is.
• a schematic map on every page, or at
least on the home page and visit info,
would really help in attracting visitors who
don’t know the area

33. Adults/families with general
interest
• What’s the rough cost and roughly
how long might I/we want to spend
there?
This would give me an idea of whether to
view it as a place to pop into on the way
somewhere or combine it with another
attraction; or whether it requires more
serious investment of time or money

34. Adults/families with general
interest
• What kind of experience will I get?
I know there will be ‘displays’ – it is a
museum!
• but will there be people around to help
bring the place alive for me, my spouse,
my children or friends?
• – or are there events, or things to do like
dressing up in a pilot’s uniform, or games
to play, etc?

35. Websites for schools
‘Serve the National Curriculum’ or ‘extend or
enhance’?
‘enhance’ sounds good but most teachers want:
1. curriculum specific – by all means cross-
curricular but with one scheme of work or topic
as ‘headline’ (think ‘product byline’)
2. ready-to-use – teachers may want to adapt to
their own situation (esp second time round), but
most will not have time – offer at least one ready
to use version
3. minimal preparation and with time commitment
(preparation time and class time) clearly
specified
4. flexible/adaptable/extensible where possible

36. Foundation and KS1 (3-7yrs)
Production of materials for this age range is
particularly tricky:
• aim at teachers not children, so…
• good bank of images, videos or other mainly
visual assets
• think of interactive whiteboards

37. Key Stage 2 (7 to 11 years
old)
• keep no of words on each page to a minimum, say
50 in total
• illustrate key ideas visually as well as verbally and
use audio if possible
• do not assume that the teacher can be over their
shoulder at all points – so keep the instructions
and processes simple
• try to use language, images, ideas, and settings
that will appeal to the target audience

38. Key Stage 3 (11 to 14 years
old)
• For KS2 a cross-curricular approach is OK
(for example they may use the same site for
Geography and Science) but:
• At KS3 cater for a single subject (and
scheme of work)
• (Can offer suggestions for cross-curricular
working, but remember generally each
teacher teachers only one subject each.)

39. ‘Lifelong learners’
• for (non-specialist) interest level
think of 12 yr olds
• Identify a particular audience with
specific interests/motivations for using
your site
• then focus on constraints to allow
successful design to proceed.
(In a formal learning setting constraints
often implicit in the course, physical set up
etc.)

40. Specialist researchers
• Fact-oriented, less graphics and design,
more text and specifically relevant images,
with good search facility
• Examples of specialist researchers:
– HE students and staff
– experts or enthusiasts in this subject area

41. Martin Bazley
Website evaluation and testing
Need to think ahead a bit:
– what are you trying to find out?
– how do you intend to test it?
– why? what will do you do as a result?
The Why?Why? should drive this process

42. Martin Bazley
Test early
Testing one user early on in the project…
…is better than testing 50 near the end

43. Martin Bazley
When to evaluate or test and why
• Before funding approval – project planning
• Post-funding - project development
• Post-project – summative evaluation

44. Martin Bazley
Testing is an iterative process
Testing isn’t something you do once
Make somethingMake something
=> test it=> test it
=> refine it=> refine it
=> test it again=> test it again

45. Martin Bazley
Before funding – project planning
• *Evaluation of other websites
– Who for? What for? How use it? etc
– awareness raising: issues, opportunities
– contributes to market research
– possible elements, graphic feel etc
• *Concept testing
– check idea makes sense with audience
– reshape project based on user feedback
Focus group
Research

46. Martin Bazley

47. Martin Bazley
Post-funding - project development
• *Concept testing
– refine project outcomes based on
feedback from intended users
• Refine website structure
– does it work for users?
• *Evaluate initial look and feel
– graphics,navigation etc
Focus group
Focus group
One-to-one tasks

48. Martin Bazley

49. Martin Bazley

50. Martin Bazley

51. Martin Bazley

52. Martin Bazley
Post-funding - project development 2
• *Full evaluation of a draft working
version
– usability AND content: do activities work, how
engaging is it, what else could be offered, etc
Observation of actual use of website
by intended users,
using it for intended purpose,
in intended context – classroom, workplace, library, home, etc

53. Martin Bazley

54. Martin Bazley

55. Martin Bazley

56. Martin Bazley

57. Martin Bazley

58. Martin Bazley

59. Martin Bazley

60. Martin Bazley

61. Martin Bazley

62. Martin Bazley
Post-funding - project development 3
• Acceptance testing of ‘finished’
website
– last minute check, minor corrections only
– often offered by web developers
• Summative evaluation
– report for funders, etc
– learn lessons at project level for next time

63. Martin Bazley
Two usability testing techniques
“Get it” testing
- do they understand the purpose, how it
works, etc
Key task testing
- ask the user to do something, watch how
well they do
Ideally, do a bit of each, in that order

64. Martin Bazley

65. Martin Bazley
User testing – who should do it?
• The worst person to conduct (or interpret)
user testing of your own site is…
– you!you!
• Beware of hearing what you want to
hear…
• Useful to have an external viewpoint
• First 5mins in a genuine setting tells you
80% of what’s wrong with the site
• etc

66. Creating resources forCreating resources for
use on whiteboardsuse on whiteboards

67. Using whiteboards
• http://www.teachers.tv/ict/whiteboardtips
• http://smarttech.com/trainingcenter/tutorials.as
• http://www.prometheanplanet.com

68. Roles of IWB
… at different points in the lesson /
learning cycle
– Starter
– Main
– Plenary

69. Interactive” means
• “lots of things moving on screen, clickable,
automatic response, quizzes etc
• interaction between students, teacher and
screen – activities, conversation, cognitive
engagement, etc
first meaning used mainly by companies trying to
market whiteboards, software etc
as ‘interactive’
second used mainly by educators

70. Resources for use on whiteboards -
examples
• Ford Madox Brown MAG
• Tate Tools
• Museum Network Artworks
• Museum Network Myths
• National Portrait Gallery Mary Seacole
• National Gallery
• Museum of London Fire of London

71. Resources for use on whiteboards -
examples
• Britons at War
• Wartime in Bedford
• http://www.movinghere.org.uk/schools/def
ault.htm
• www.mylearning.org/overview.asp?
journeyid=409
• www.mylearning.org/overview.asp?
journeyid=441

72. Some examples
– http://www.nhm.ac.uk/nature-online/british-
natural-history/index.html
– http://www.manchestergalleries.org/the-
collections/highlights-of-the-
collection/narrativeobject.php?irn=876
– www.seayourhistory.org.uk/content/view/39/7
7/
– http://www.portsmouth.gov.uk/business/2781.
html

73. Whiteboard resource exercise
• We are using Powerpoint (or Word if you
prefer) just to summarise your ideas on
the board.
• Don’t spend too long formatting /
designing – just focus on
– what items, text and links would be on screen,
and
– What teacher / pupils would do with them

74. ‘templates’
• The following slide (Britons at War) is a
sample web page – think about how
people would get to your whiteboard
page(s)
• The next slide is an outline template for a
whiteboard page – edit or ignore this
completely.
• Make sure you have at least something
to show on screen

75. 75
Search
Adv search
Britons at War banner
Home
Topics
Resources
Schools area
Ways to use this
site
About this site
Search
BaW
logo
Home | About | Links | Contact | Sitemap
an online resource for schools
NMPFT
logo
link to home
YFA
logo
link to home
Topic: Bombing Thumbnails of photos +
films, etc for this topic
Link to Photo pages
Bombing
Rationing
Local life
Home Guard
Women and children
Evacuation
Guy Fawkes
Day 1944
Prisoners of
War
Whit Sunday
in Hyde Park
VE Day
Brief introduction to Topic: Bombing
Asfd
Asdf
sadf

76. About this image (caption, copyright, etc) Also key question?
Title / heading if needed – otherwise delete this box.

77. More information / advice /
ideas
Martin Bazley
0780 3580 737
www.martinbazley.com