HOW VGI Influences Online Usability

561 views

Published on

How should designers of mashups and neogeographic systems think about VGI (volunteered geographic information) if they are looking to create a unique and exciting user experience? Presented a the 4th workshop on GIS Usability (

Published in: Design, Technology, Business
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
561
On SlideShare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
Downloads
10
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide
  • Our new home at the Loughborough Design School
  • Brief overview of the background research in my PhD so you understand why the research I will be talking about is important
  • VGI = Volunteered Geographic Information, PGI = Professional Geographic InformationThese are the main user groups that emerged from the interviewsEach user group perceives the other one differently, unique relationshipsEach one perceived VGI and PGI differentlyThis is based on user requirements AND existing beliefs.HAPPYNESS: We can’t please all the people all the time
  • Some photos of my data gathering with focus groups, participatory observations and diary studies.I didn’t capsize but I did manage to sink my kayak in the middle of the river!
  • Focus Group Graph = Impact of Information on Outcomes; #refs madeEarliest stages of kayaking activity rely on external informationLater stages rely on internal information (water gauges)Lots of VGI used in planning, therefore sharing experiences importantNo mention about volunteering the info – GAP!!!!VGI best for fast changing, subjective areasPGI best for static, objective features.HAPPYNESS: The info only helps make you happy when it helps you achieve your activity
  • Current Research
  • What we are trying to find out
  • These are the real influences we are measuring in the study
  • Two theories used as the backbone to this researchDID NOT mix them!
  • But we can’t just research any user group, we need on which fits our requirementsCLICKSo as you might have guessed by now, wheelchair users were selected!Compatible to kayakers, but different enough to understand the wider issues of use of VGI
  • Study in two parts…
  • Does adding VGI to PGI increase the quality, accuracy and usability perceptions of the mashup?Does telling people there is VGI in the PGI increase quality, accuracy and usability perceptions?
  • So we analyses the data, compiled it and presented it through a website FREE TRAVELLER
  • After using the website, participants given a questionnaire derived from the two base theories of this research4 questions (2 positive, 2 negative) per dependable variable. Wording of questions based on suggested key works as laid out by the frameworks.Then we run Multivariate Analysis of Variance to understand the influence of the fixed variables on the groups of judgements. (MANOVA)
  • That was the route we took over 6 hours!
  • So I went out in London with a group of wheelchair users and created my own VGI data set.Each wheelchair user had a diary and we recorded the good and the bad points about public transport in London
  • Done by literature review of all current transport data about London and disabled access. Gaining ‘best case scenario’ of what is currently presented professionally.Along with VGI, two data sets were created of a demonstrable quality above that commonly found online today.
  • Remember that website that we built?
  • And here is an example of the data with presented the participants withAt the end of using the maps a 32 question Likert scale (5 point) was presented to test the perceptions of Quality, Authority and Usability
  • Before MANOVA could be applied to survey data, assumptions needed to be tested.Initially three categories of data were presumed, in line with the theories
  • Confirmatory Factor Analysis demonstrated these two groups, not three
  • Multicollinearity and singularity testing (Spearman rho) demonstrated high correlation between some of the factors. These were removed from the data set to allow MANOVA to meet all of its assumptions. Removed these as we are interested in not just if variables are significant, but also their effect size.From this the frameworks of assessing the user reaction were reached.All other assumptions of the data required to run MANOVA were met.
  • All those demonstrated here are positively influenced by the VGI stimuliBare in mind, this graph is designed as an easily understandable overview of the data. Each dependable variable is demonstrating the Partial Eta Squared for the lowest N value to achieve significance. Realistic would be more skewed and misleading in one graph. Done this was due to the use of Sample Size Estimation which biases the data when taken to the Nth degree, so this graph is more realistic to a researchable sample size.0.01 is a small effect, 0.09 a medium effect and 0.25 a large effect
  • Graph to show currencyThe most dramatic. Shown to demonstrate the kind of relations were working with.
  • 0.01 is a small effect, 0.09 a medium effect and 0.25 a large effectBare in mind that the effect sizes for usefulness on the Q&A and SA are similar.
  • While the extent to which this study can represent the exact nature of VGI is debatable, this study has demonstrated some important underlying phenomenon
  • While the extent to which this study can represent the exact nature of VGI is debatable, this study has demonstrated some important underlying phenomenon
  • HOW VGI Influences Online Usability

    1. 1. Past Research How We Got To Where We Are Now
    2. 2. Usability of VGI THE IMPACT OF VGI ON USER JUDGEMENT
    3. 3. Study AimsWhat We Are Trying To Work Out• The extent to which the content of the information within the mashup affects the users’ judgements on the mashup.• The extent to which the user judgements on the information influence the overall usability and system acceptance of the mashup.• The extent to which the facets of the users’ judgements which may be harnessed to optimise the design of future mashups combining both VGI and PGI information.
    4. 4. Study RationalWhat We Are Really Getting At Presenting VGI  Telling Participants Alongside PGI in a Their Mashup mashup Contains VGI  The Utility of VGI,  The degree to which what it may offer to current perceptions of users that PGI does information from not amateur volunteers  The Style of VGI, a influences perception different way of of neogeography presenting information 16/01/2012
    5. 5. Study RationalDependable Variables User Judgements of Online Information System Acceptance Model Rieh (2002) Maguire (1998) Quality Authority Usability Good Trustworthy Usefulness Accurate Credible Clear Current Reliable Efficient Useful Official Satisfaction Important Authority
    6. 6. Methodology THE IMPACT OF VGI ON USER JUDGEMENT
    7. 7. MethodologySelection of a Participant Community• Need to critically evaluate information• Can be bounded in as a simple user group• Make use of PGI as well as generating and utilising VGI• Critically assess information for risk management• BUT be normal people doing normal things. • Selection Criteria • Age 18-65 • Full time wheelchair user • Non cognitive or Sensory Disabled • Non-house bound • Other (regular ethics)
    8. 8. MethodologyTwo Parts of the Study• Part 1 • Observation (6 wheelchair users) • VGI: Generate Data Set • Literature Review • PGI: Pool data• Part 2 • Mono Methods • Experimental • Online (101 participants) • Four Groups – with a unique combination of contents of the map and information about the map.
    9. 9. MethodologyFixed Variables and Groups Information Presented In Map PGI PGI + VGI Participant Told PGI Group 1 Group 3 What Map PGI + VGI Group 2 Group 4 Contained
    10. 10. Part 2Experiment Website: Free Traveller
    11. 11. Study RationalDependable Variables User Judgements of Online Information System Acceptance Model Rieh (2002) Maguire (1998) Quality Authority Usability Good Trustworthy Usefulness Accurate Credible Clear Current Reliable Efficient Useful Official Satisfaction Important Authority
    12. 12. Part 1Collecting Information
    13. 13. Part 1Data Collection Routes
    14. 14. Part 1VGI Data Collection
    15. 15. Part 1PGI Data Collection
    16. 16. Part 2 The Experiment
    17. 17. Part 2www.FreeTraveller.co.uk
    18. 18. Part 2Assumption Testing User Judgements of Online Information System Acceptance Model Rieh (2002) Maguire (1998) Quality Authority Usability Good Trustworthy Usefulness Accurate Credible Clear Current Reliable Efficient Useful Official Satisfaction Important Authority
    19. 19. Part 2Assumption Testing: Factor Analysis User Judgements of Online Information System Acceptance Model Rieh (2002) Maguire (1998) Quality & Authority Usability Good Trustworthy Usefulness Accurate Credible Clear Current Reliable Efficient Useful Official Satisfaction Important Authority
    20. 20. Part 2Assumption Testing: Collinearity User Judgements of Online Information System Acceptance Model Rieh (2002) Maguire (1998) Quality & Authority Usability Good Trustworthy Usefulness Accurate Credible Clear Current Reliable Efficient Useful Official Satisfaction Important Authority
    21. 21. Results Two-Way MANOVA
    22. 22. ResultsQuality & Authority• Information Presented in Mashup • 101 Participants Significant • F (9, 89) = 2.99 • p = .004 • Wilks’ Lambda = .77 • ηp2 = .232• Information Told to Participants • 303 Participants (Estimation) • F (9, 392) = 3.48 • p = 3.5E-7 > .00268 • Wilks’ Lambda = .93 • ηp2 = .074• No Statistically Significant Interaction 16/01/2012
    23. 23. Results Quality & Authority0.11 0.10.090.080.070.06 Information Presented0.05 Information Told0.040.030.020.01 0 Currency Importance Usefulness Credibility Authority Goodness Reliable Accurate Official 16/01/2012
    24. 24. ResultsQuality & Authority 16/01/2012
    25. 25. ResultsQuality and Authority• Presenting VGI alongside PGI • Most notable influence is on currency, as perceived by the inclusion of VGI within the data set and should be considered • Importance, Credibility, Usefulness and Authority are all positively influenced, but how much consideration needed is debatable• Telling People their mashup contains VGI alongside PGI • Most notable influence is on Authority • Usefulness and Credibility worth considering • The importance of there variables and effect of being told is limited
    26. 26. ResultsSystem Acceptance• Information Presented in Mashup • 303 Participants (Estimation) • F (3, 297) = 4.67 • p = .003 > .00851 • Wilks’ Lambda = .96 • ηp2 = .045• Information Told to Participants • 404 Participants (Estimation) • F (3, 398) = 4.38 • p = .005 > .00851 • Wilks’ Lambda = .97 • ηp2 = .032• No Statistically Significant Interaction 16/01/2012
    27. 27. Results Quality & Authority0.11 0.10.090.080.070.06 Information Presented0.05 Information Told0.040.030.020.01 0 Usefulness Efficiency Satisfaction 16/01/2012
    28. 28. ResultsSystem Acceptance• Presenting VGI alongside PGI • Most notable influence is on usefulness • May result from more info = more utility• Telling People their mashup contains VGI alongside PGI • Most notable influence is on slightly increased levels of satisfaction • May be due to idea of social interaction?
    29. 29. Conclusions What does this mean?
    30. 30. ConclusionsFrom Research• Presenting VGI Alongside PGI • Reinforces the mantra of right data for right task • Does VGI cover more ground than PGI? • Judgements of Quality & Authority increased along with Usability • No negative Influence!• Telling Users They Are Using PGI • Personally held biases have a measurable effect on Quality & Authority and overall Usability • Effects are relatively minimal and attention may be best given to other factors • No negative influence!
    31. 31. ConclusionsFor Design• VGI did not change the game dramatically • Though currency is a very interesting outcome• In every case VGI enhanced the user perceptions and usability• VGI is not necessarily a game changer in normal situations, but instead, if applied correctly is polish. • Wont make a bad system good • But has potential to enhance what is already good, making it more satisfying. • Little point polishing low end mashups, but high end a small amount of polish is VERY important

    ×