1. Leonardo da Vinci Partnerships Project
GUI USABILITY AND ACCESSIBILITY:
EXCHANGING KNOWLEDGE AND EXPERIENCES
Introduction to User-Centred Analysis (UCA)
Data Analysis Techniques
Cristina Cachero
This project has been funded with support from the European
Commission under the Lifelong Learning Programme
2. Leonardo da Vinci Partnerships Project
GUI USABILITY AND ACCESSIBILITY:
EXCHANGING KNOWLEDGE AND EXPERIENCES
This project has been funded with support from the European
Commission under the Lifelong Learning Programme
2
User-Centred Data Analysis Techniques
Quantitative data (out of the scope of this course)
Statistical methods (closed-questions in surveys)
Clustering or other machine learning methods (web analytics,
card sorting)
Qualitative data (comments, observations, etc.)
Data Exploration
Term Analysis
Affinity Diagramming
3. Leonardo da Vinci Partnerships Project
GUI USABILITY AND ACCESSIBILITY:
EXCHANGING KNOWLEDGE AND EXPERIENCES
This project has been funded with support from the European
Commission under the Lifelong Learning Programme
3
Qualitative data: Data Exploration
4. Leonardo da Vinci Partnerships Project
GUI USABILITY AND ACCESSIBILITY:
EXCHANGING KNOWLEDGE AND EXPERIENCES
This project has been funded with support from the European
Commission under the Lifelong Learning Programme
4
Qualitative data: Term Analysis
synonyms (words with the same meaning)
antonyms (words with the opposite meaning)
related concepts
broader and narrower terms
concepts frequently mentioned together
5. Leonardo da Vinci Partnerships Project
GUI USABILITY AND ACCESSIBILITY:
EXCHANGING KNOWLEDGE AND EXPERIENCES
This project has been funded with support from the European
Commission under the Lifelong Learning Programme
5
Qualitative data: Affinity Diagramming
6. Leonardo da Vinci Partnerships Project
GUI USABILITY AND ACCESSIBILITY:
EXCHANGING KNOWLEDGE AND EXPERIENCES
This project has been funded with support from the European
Commission under the Lifelong Learning Programme
6
Affinity Diagramming: 2*2 Matrices
Dimensions:
How often the participant uses the site
How experienced the participant is with the
product
How familiar the participant is with the content
How they looked for information – searching or
browsing
Type of task:Were they getting a quick fact or
exploring in detail?
Is the issue about a positive or negative
experience?
How much content would they need to answer
their question?
Any sort of demographic of the participant?
…
7. Leonardo da Vinci Partnerships Project
GUI USABILITY AND ACCESSIBILITY:
EXCHANGING KNOWLEDGE AND EXPERIENCES
This project has been funded with support from the European
Commission under the Lifelong Learning Programme
7
Affinity Diagramming: Multidimensional
Dimensions:
How often the participant uses the site
How experienced the participant is with the product
How familiar the participant is with the content
How they looked for information – searching or browsing
Type of task:Were they getting a quick fact or exploring in detail?
Is the issue about a positive or negative experience?
How much content would they need to answer their question?
Any sort of demographic of the participant?
…
Discuss each dimension with your team
8. Leonardo da Vinci Partnerships Project
GUI USABILITY AND ACCESSIBILITY:
EXCHANGING KNOWLEDGE AND EXPERIENCES
These slides are made available under the license Creative Commons
Attribution-NonCommercial-NoDerivs CC BY-NC-ND. More
information about license:
http://creativecommons.org/licenses/by-nc-nd/3.0/.
These slides were created under Leonardo daVinci Partnerships
Project 2012-1-PL1-LEO04-28181 GUI USABILITY AND
ACCESSIBILITY: EXCHANGING KNOWLEDGE AND EXPERIENCES (
http://usability-accessibility.org/).
This project has been funded with support from the European
Commission under the Lifelong Learning Programme
8
Attributions
Editor's Notes
Hello, my name is Cristina Cachero, and I am associate professor at the University of Alicante.
In this video I am going to introduce you into the main data analysis techniques proposed as part of a User-Centered Development process.
Whatever the data gathering technique chosen (interview, observation, survey, etc.), you will probably end up with a lot of data that needs to be analyzed. Depending on the nature of the data, you can opt for quantitative or qualitative techniques.
The use of quantitative techniques (hypothesis refutation methods, machine learning methods, etc.) is far too complex to be tackled in this course and requires a good background in statistics.
Qualitative techniques are much easier to grasp. Also, they are most widely used in usability studies, which tend to be qualitative in nature and to rely on open questions and lots of non-structured data.
Next, I am explaining to you three very easy and broadly used data analysis techniques: data exploration, term analysis and affinity diagramming.
The favourite way of analyzing data for many usability analysts is to drop everything into a spreadsheet with columns labelled things such as 'source', 'tag' and 'comment'.
First they go through and record lots of stuff in the comments column. This is the start of deconstructing the data – identifying the individual pieces. Each row represents one thing from the research. ‘Things’ can be individual sentences from an interview transcript, search terms from internal search, comments made during a card sort, answers from a survey response, etc. Each is an independent idea the usability analyst may want to look at individually. You don’t have to record every single sentence from every single interview – a lot of it will be irrelevant – but record anything you think could be useful for your project.
After you have recorded a pile of stuff into a spreadsheet, you need to start coding the data with simple tags (keywords). Tag each line with whatever comes to mind. Don’t worry about creating a complete or consistent set of tags – you can go back later and revise them. Usability experts usually go through the set of tags twice and fix up some tags in the earlier data to match how they’ve tagged data later in the spreadsheet. You can use two columns of tags – one to record the topic of the line item, and the other to record something like how people use the data. This may vary depending on your data.
The last step is to start exploring the data. Sort it by tags so similar ideas are grouped together. Look at each one for basic patterns and interesting issues. You’ll be able to identify key information needs, the issues mentioned most frequently, how people described similar needs (they may have done it in different ways) and whether there was consistency between sources.
Another useful technique is Term analysis. Term analysis is a technique used to learn about terminology and understand how people describe ideas.
The first step is to choose something you want to know about, and use your research notes to see how they describe it. Look for:
- words used to describe the concept
- synonyms (words with the same meaning)
- antonyms (words with the opposite meaning)
- related concepts
- broader and narrower terms
- concepts frequently mentioned together
This technique is very useful to organize content and choose the language of your application.
This technique is very useful to organize content and choose the language of your application.
The third technique we are discussing in this course is affinity diagramming.
Affinity diagramming is a great team analysis activity. To start, each person in the team goes through the research (transcripts, spreadsheets etc). As they read the different sources, they identify the issues that they find interesting (e.g. comments of things that work well or poorly on the current application) and write them on sticky notes. Here a question arises: what is an interesting issue? Well, it will depend on your data and the objectives of your research.
Once everybody has come up with their own interesting issues, it is time to put them together, refund some of the issues, split others and give meaningful titles to each set of related issues. This is how the team comes together to a common understanding of the problem context. It is very important to understand that the diagram isn’t the point. It’s the team discussion and the ability to look at things in different ways that’s important.
Another way to work with the data gathered is, once you have identified the main issues, define a set of dimensions and categorize them according to those dimensions.
One very easy way to do so is using a 2 by 2 matrix and chosing pairs of dimensions. Imagine for example that we are categorizing comments according to their positive or negative nature (dimension 1) and according to whether they refer to searching or browsing (dimension 2). How would you classify the following comments?
1.- 'The A-Z index works really well': it clearly is a positive comment related to browsing.
2.- 'this navigation, the titles are just confusing', 'I can never find stuff', 'you have to know who does what to find things' or 'last week I wanted the policy and I had to look for ages': they are negative comments related to the same browsing behaviour. In fact, look how all these comments are pointing at a poorly designed information architecture.
3.- 'search is ok when I want something quickly' or 'I usually have luck with the search': they all are positive comments related to searching.
Summarizing, this brief classification exercise suggests that, for the application under study, the search works just fine, and that the A-Z index is well appreciated. However, the information architecture needs to be wholy redefined. According to the comments, such redesign probably implies getting away from an organizational-driven information architecture, which seems to be one of the main causes for people not finding the information (look at the comment 'you have to know who does what to find things')
As with affinity diagramming, the diagram isn’t the point. It’s the discussion and the ability to look at things in different ways that’s important.
____________________________________________________________
Also, in order to review your data you can perform a multi-dimensional analysis. As with 2x2 matrixes, you need to identify a set of dimensions to examine. Then, instead of plotting sticky notes on a 2x2 matrix, start a discussion on each dimension with your team.
For example if you identify a range of familiarity with the topic (from novice users to experts), look at other aspects and see if there are differences across people. Do people unfamiliar with the domain use different terms to describe concepts or content? Do they need different types of content?
When you’ve examined all you can for one dimension, start over with another.
For example, you may have also identified that users differ on their frequency of use of the application you could try to answer questions such as What do very frequent users need (are they returning frequently to keep up-to-date?) compared to medium frequency users (who may be after the same content over and over) and infrequent users (who may just need a fast answer)? In brief, you need to think about how people’s needs and experiences differ.
The discussion about people’s experiences and needs for each dimension may also suggest new dimensions to examine.
This is less of a diagramming activity and more of a team discussion. It can feel a bit directionless for a while, but as you discuss each idea, you’ll gain a deeper understanding of what’s going on.
So, this is all for now. Hope you have enjoyed this video. See you in next section!
