9B_1_Trust in web gis a preliminary investigation of the environment agencys wiyby website with non-expert users Presentation Transcript
Trust in Web GIS: A Preliminary Investigation of the Environment Agency’s WIYBY website with non-expert users Artemis Skarlatidou, Muki Haklay, Tao Cheng and Nicola Francis Department of Civil, Environmental and Geomatic Engineering, University College London GIS Research UK 2010 - 16 April 2010 -
Contents of Presentation
Case Study and Experimental Design
Conclusions and Future Experiments
Why trust is important in Web GIS?
Existing Web GIS instruct, advise users and provide information and analysis, which according to Fogg (2003) are amongst these situations where computers’ credibility/trustworthiness matters.
Online trust studies - mainly for e-commerce
- influence the intentions to engage
- the use and acceptance of these systems
- enhance cooperative behaviours
- influence the perceived User Experience (Shneiderman, 2000; Fogg, 2001…)
Trust Concepts/Trustee Attributes
Different Trustee Attributes in Literature
Not a systematic Approach is taken for trust investigations
Aim : To understand what influence trust perceptions of non-expert users in Web GIS – what attributes are important?
How: Investigation of different interfaces with non-expert users using HCI methodology
Cooperative Approach followed discussing trust issues while using the website
What’s In Your Back Yard (WIYBY)
Web GIS application provides environmental spatial and non-spatial data and maps
Risk and uncertainty
Previous studies (Alsop, 2008; Francis, 2009) found several usability problems
Preliminary Expert Evaluations found several trust-related problems associated with the system’s functionality
e.g. external links, vocabulary, information about map accuracy, unpleasant maps, problematic menu – 102 prob.
(perceived usability, aesthetics, perceived trustworthiness, User Experience and Trust Cues/trust expectations)
Think Aloud Data
Additional Quantitative Data for Usability Metrics
Actual Usability – Success Rate Formula (Nielsen, 2001)
Time spent on each task and input events (clicks) - (mental effort)
Experimental Design/ Recruitment
10 participants were recruited
additional experiments with another 10 showed the same results – but also the same results were found with the second application
Condition: no GIS training or education (non-experts)
Trust perceptions of experts and non-experts can differ significantly
Familiarity with web-mapping (not constrained)
No Gender or other conditions as there are no studies which report any trust implications
Experimental Design / Tasks
Scenario-based to help people realise the risk
The Post-Questionnaire open questions showed that all people were aware of the risks
After providing a scenario users were asked to find the following information for two areas:
SW19 3BJ Fulham & Hammersmith
Flood Risk Air pollution – pollutants most dangerous
How risk of flood can be reduced Quality of Drinking Water
Distance from nearest landfill site
Experimental Design Post-Test Questionnaires
Open Questions ( user problems, suggestions for improvement and trust perceptions)
Perceptual and Functional Attributes according to previous studies
Likert Scale inter-related Items (1:S. Disagree to 5:S. Agree) – including duplicates and negatives
Usability and Aesthetics / Examples
Built based on:
Previous Expert Evaluations
Well-established heuristics and previous studies (e.g. Flavian, 2005; Nivala et al. 2008)
General Usability Items
- I found the website easy to use
- The website worked as it was expected
GIS Usability Items
- Map was of an appropriate size
- Legend was not easy to understand
- Map legend was problematic and affected my trust towards the maps
- It was easy to understand how to select/deselect layers
- The colours of the website were easy on the eye
- The maps were not aesthetically pleasant
Trust Cues & User Experience
Trust Cues (mainly based on literature e.g. Cheskin Report, 1999)
- Branding: There were sufficient logos and branding to increase my trust of the website..
- Expectations: A blog or forum to connect people from similar areas would be an improvement in the website and would increase trust.
- I will use the website in the future
- Not Stressful
Previous studies focus mainly on e-commerce so irrelevant items
Fogg & Tseng (1999) for credibility assessment
- I trust all the information provided
- EA is a credible source
- The website is credible
- The website is transparent
3 males and 7 females
Most were students 18-24
Computer Skills (Intermediate 5 or Advanced 4)
Frequency of using web-mapping apps (Frequently 6, Occasionally 3, Never 1)
The users failed in 21/60 tasks
Less than 30 minutes
Tasks UT Success Rate Table – Participants & Tasks 1 2 3 4 5 6 7 8 9 10 1a)(i) S S S S S S S S S S 1a)(ii) F F F F F F F F S S 1b)(i) S S S S PS S S S S S 1b)(ii) F F F F F F F F F F 2a) F F F S S S S S S S 2b) S S S S PS PS S PS PS S
Perceived Usability Low (7/10 think website was not easy to use and 8/10 that needs improvement!) – Recall that actual usability was relatively high, 61% success rate
The majority of users agreed with problems found by experts
All users think that the map visualisation needs improvement!
9/10 trust the information provided although they think the website was not transparent!!
“ I would trust information coming from a Government Agency on principle”
Measuring the Perceived Usability
Influence User Experience (e.g. future engagement)
“ Not particularly easy to use, would probably rather use an alternative information source”
Users recognised and mentioned almost all trust-related problems!
Thought the website not transparent!
Yet they trusted it! (no comments of suspicion for maps!!)
Most comments about maps focus on aesthetics
Trust perceptions and functional attributes
“ Halo effect” (Fogg,2003)? (Academic and Governmental sources)
People blame themselves?
Are finally perceptual attributes stronger in Web GIS context?
We can’t say that whatever you design people will trust it
But what is most important to them in terms of a trust-based interface design?
Trust-oriented Interface Design will be examined in a controlled environment
Different Interfaces are currently under development for the case of the site selection of a nuclear waste repository
Testing different conditions and ask users to rate which interface they trust more