Full paper in: http://www.cs.man.ac.uk/~apaolaza/research.html Abstract: Laboratory studies are a well established practice that present disadvantages in terms of data collection. One of these disadvantages is that laboratories are controlled environments that do not account for unpredicted factors from the real world. Laboratory studies are also obtrusive and therefore possibly biased. The Human-Computer Interaction (HCI) community has acknowledged these problems and has started exploring in-situ observation techniques. These observation techniques allow for bigger participant pools and their environments can conform to the real world. Such real-world observations are particularly important to the accessibility community who has coined the concept accessibility-in-use to differentiate real world from laboratory studies. Real-world observations provide low-level interaction data therefore making a bottom-up analysis possible. This way behaviours emerge from the obtained data instead of looking for predefined models. Some in-situ techniques employ Web logs in which the data is too coarse to infer meaningful user interaction. In some other cases an exhaustive manual modification is required to capture interaction data from a Web application. We describe a tool which is easily deployable in any Web application and captures longitudinal interaction data unobtrusively. It enables the observation of accessibility-in-use and guides the detection of emerging tasks.

1. Understanding Users in
the Wild
Aitor Apaolaza, Simon Harper, Caroline Jay
University of Manchester
W4A 13th May 2013

2. ● Real users in the real world trying to
achieve real goals
● Differentiates real world and laboratory
studies
● In situ and unobtrusive
● Low-level
Accessibility-in-use
Understanding Users in the Wild
W4A 13th May 2013 1

3. ● Laboratory studies
○ Controlled environments
○ Obtrusive
○ Short term
○ Use of predefined tasks
Observation methods
Understanding Users in the Wild
W4A 13th May 2013 2
● Our proposal
○ Remote observation
○ Ecological and naturalistic
○ Longitudinal
○ Low-level, tasks emerge

4. Our Approach
Understanding Users in the Wild
W4A 13th May 2013 3
Visualization
tool
Data
Processing
Behaviour
model library
Capture module
Data
Query and
update
Explore
information

5. ● Requires identifying unique users
● Enables learnability studies
● Study the effect of changes to the interface
Longitudinal
Understanding Users in the Wild
W4A 13th May 2013 4

6. ● In the Mood to Click? Q. Guo et al.
■ Single purpose website study
● WebInSitu Bigham et al.
■ Differences between blind and sighted users' usage
● Support for remote usability evaluation of
web mobile applications Carta et al.
■ Comparison of single sessions
Similar work
Understanding Users in the Wild
W4A 13th May 2013 5

7. Our Approach
Understanding Users in the Wild
W4A 13th May 2013 6
Interaction
events
HTML
+ JS
Client
Visualization
tool
Data
Processing
Behaviour
model library
Capture module
Capture server
Data
Web Server
Query and
update
Explore
information

8. Our Approach
Understanding Users in the Wild
W4A 13th May 2013 7
Interaction
events
HTML
+ JS
Client
Visualization
tool
Data
Processing
Behaviour
model library
Capture module
Capture server
Data
Web Server
Query and
update
Explore
information

9. Our Approach
Client Capture serverWeb Server
Understanding Users in the Wild
W4A 13th May 2013 8

10. Our Approach
HTML
+ JS
<script type="text/javascript">
//Configure and import capture solution
</script>+
● JavaScript is added to all Web pages
Web Server Client Capture server
Understanding Users in the Wild
W4A 13th May 2013 9

11. Understanding Users in the Wild
W4A 13th May 2013 10
Our Approach
HTML
+ JS
Web Server Client Capture server
● Listeners for events are registered
● Mouse
● Keyboard
● Form input
● Window
Interaction
events

12. Web Server Client
Our Approach
Interaction
events
HTML
+ JS
● Events get processed and stored
Understanding Users in the Wild
W4A 13th May 2013 11
Capture server

13. Our Approach (examples)
"nodeInfo" : {
"nodeDom" : "id("selectquicklinks")",
"nodeType" : "SELECT",
},
"value" : "http://manc.ac.uk/software/",
"selected" : "8",
"event" : "change",
What user sees What we see
Understanding Users in the Wild
W4A 13th May 2013 12

14. ● Extendible and easy to deploy in the wild
● Provide low-level interaction data
● Longitudinal in-situ studies are possible
● Allows the recreation of the interaction
● Sensitive elements can be blacklisted
Advantages
Understanding Users in the Wild
W4A 13th May 2013 13

15. Emerging behaviours
● Users' tasks emerge from low-level
interaction data
● Ecologically valid behaviour models
● Accessibility problems arise
● Similar users have similar problems
Understanding Users in the Wild
W4A 13th May 2013 14

16. Understanding Users in the Wild
Aitor Apaolaza, Simon Harper and Caroline Jay
http://www.cs.man.ac.uk/~apaolaza/
Thank you!
Understanding Users in the Wild
W4A 13th May 2013 15