Wevquery: Testing Hypotheses about Web Interaction Patterns
•
0 likes•494 views
WevQuery, a scalable system to query user interaction logs in order to allow designers to test their hypotheses about users’ behaviour.
Recommended
Recommended
More Related Content
Similar to Wevquery: Testing Hypotheses about Web Interaction Patterns
Similar to Wevquery: Testing Hypotheses about Web Interaction Patterns (20)
More from MOVING Project
More from MOVING Project (20)
Recently uploaded
Recently uploaded (20)
Wevquery: Testing Hypotheses about Web Interaction Patterns
- 1. WevQuery: Testing Hypotheses about Web Interaction Patterns Aitor Apaolaza Markel Vigo EICS 2017 June 28, 2017 - Lisbon, Portugal
- 2. WevQuery EICS 2017 Motivation ● We don’t know what users do ● We can get the data, but we don’t have the expertise to handle it ● Data is big, confusing and difficult to handle 2
- 4. WevQuery EICS 2017 Lab settings ● Strengths ○ Provide a deep understanding ○ Allows for controlling variables ○ Good for internal validity ● Weaknesses ○ Resource demands: time for data analysis ○ Guinea Pig effect and response bias ○ Task driven ○ Risks to external validity ○ Recruitment problem 4
- 5. WevQuery EICS 2017 Remote settings ● Strengths ○ Good for external validity (if naturalistic) ○ Ecologically valid ○ Users are easily recruited ○ Easy to implement ● Weaknesses ○ Resource demands: have the right infrastructure ○ Cannot control the confounding variables 5
- 6. WevQuery EICS 2017 Remote log file analysis ● Common technique for HCI and IR ● Large data: millions of datapoints from thousands of users ● Current approaches have one or more drawbacks: ○ Not informative enough ○ Not straightforward for non-specialists ○ Not flexible ○ Not scalable ○ Lack of support for decision-making ● WevQuery addresses the above issues 6
- 7. WevQuery EICS 2017 What’s WevQuery ● Web Event Query ● Bridge the gap between designers and data analysts ● Guided, flexible and visual query system ● Users without data analysis experience ● Highly scalable 7
- 20. WevQuery EICS 2017 Proof of concept ● Defined two hypothetical scenarios where designers argue about how their website is being used ● Hypotheses tested on real data ○ 5.7m datapoints consisting of events ○ 2,445 users on 3,287 web pages ○ 2 months 20
- 21. WevQuery EICS 2017 Proof of concept ● Defined two hypothetical scenarios where designers argue about how their website is being used ● Hypotheses tested on real data ○ 5.7m datapoints consisting of events ○ 2,445 users on 3,287 web pages ○ 2 months 21
- 24. WevQuery EICS 2017 Scenario 1 24 Users will spend too much time hovering over every element Designer A Designer B I think is a nice way to remove clutter from the Web page
- 25. WevQuery EICS 2017 Scenario 1 25 Users will spend too much time hovering over every element Designer A Designer B Hypothesis There will be unnecessarily long hovering actionI think is a nice way to remove clutter from the Web page
- 26. WevQuery EICS 2017 I think is a nice way to remove clutter from the Web page Scenario 1 26 Hypothesis There will be unnecessarily long hovering action WevQuery Users will spend too much time hovering over every element Designer A Designer B
- 27. WevQuery EICS 2017 Scenario 1 ● Results ○ 29,770 episodes containing mouseover and mouseout ○ Half of them (14,850 cases) contains >3 seconds hover actions ● Refinement ○ Increased threshold to 10 seconds (down to 8,277 cases) ○ Homepage is the source of most 27
- 29. WevQuery EICS 2017 Scenario 1 ● Results ○ 29,770 episodes containing mouseover and mouseout ○ Half of them (14,850 cases) contains >3 seconds hover actions ● Refinement ○ Increased threshold to 10 seconds (down to 8,277 cases) ○ Homepage is the source of most 29
- 30. WevQuery EICS 2017 Scenario 1 ● Results ○ 29,770 episodes containing mouseover and mouseout ○ Half of them (14,850 cases) contains >3 seconds hover actions ● Refinement ○ Increased threshold to 10 seconds (down to 8,277 cases) ○ Homepage is the source of most ● Findings ○ Identified an <a> element that when hovered discloses up to 45 links ○ They decide to change the hover to a toggle ○ They will use A/B testing to determine the effectiveness of the change 30
- 31. WevQuery EICS 2017 ● Results ○ 29,770 episodes containing mouseover and mouseout ○ Half of them (14,850 cases) contains >3 seconds hover actions ● Refinement ○ Increased threshold to 10 seconds (down to 8,277 cases) ○ Homepage is the source of most ● Findings ○ Identified an <a> element that when hovered discloses up to 45 links ○ They decide to change the hover to a toggle ○ They will use A/B testing to determine the effectiveness of the change Scenario 1 31
- 32. WevQuery EICS 2017 Scenario 1 32 ● Results ○ 29,770 episodes containing mouseover and mouseout ○ Half of them (14,850 cases) contains >3 seconds hover actions ● Refinement ○ Increased threshold to 10 seconds (down to 8,277 cases) ○ Homepage is the source of most ● Findings ○ Identified an <a> element that when hovered discloses up to 45 links ○ They decide to change the hover to a toggle ○ They will use A/B testing to determine the effectiveness of the change
- 35. WevQuery EICS 2017 Users rarely use the information at the bottom Fitting the entire Web page in the screen is important 35 Designer A Designer B Scenario 2
- 36. WevQuery EICS 2017 36 Scenario 2 Designer A Designer B Hypothesis Users scroll down immediately after loading the page Users rarely use the information at the bottom Fitting the entire Web page in the screen is important
- 37. WevQuery EICS 2017 37 Hypothesis Users scroll down immediately after loading the page WevQuery Scenario 2 Designer A Designer B Users rarely use the information at the bottom Fitting the entire Web page in the screen is important
- 38. WevQuery EICS 2017 Scenario 2 ● Results ○ 33,444 episodes containing a page load and either a mousewheel or a scroll ○ 18,729 contains >10 seconds load to scroll actions ● Refinement ○ Decreased threshold to 3, 1, and 0.5 seconds (down to 16,129, 11,885, and 8,903 cases) 38
- 39. WevQuery EICS 2017 Scenario 2 ● Results ○ 33,444 episodes containing a page load and either a mousewheel or a scroll ○ 18,729 contains >10 seconds load to scroll actions ● Refinement ○ Decreased threshold to 3, 1, and 0.5 seconds (down to 16,129, 11,885, and 8,903 cases) ● Findings ○ Highly frequent ○ Further refinement is decided 39
- 40. WevQuery EICS 2017 40 Scenario 2 ● Further refinement
- 41. WevQuery EICS 2017 41 Scenario 2 (refined) ● Further refinement
- 42. WevQuery EICS 2017 Scenario 2 (refined) ● Results ○ 371 occurrences ○ No clicks on elements at the bottom of the page ● Findings ○ Information at the bottom is not sought ○ Making pages fit the screen is beneficial 42
- 43. WevQuery EICS 2017 Proof of concept ● Designers decide to take a compromise solution ● On the performance ○ Run on a laptop ○ Mouseover and mouseout accounts for 2.8m events (out of 5.7m) ○ Execution time depends on event frequency (~50 seconds to ~2 minutes) 43
- 44. WevQuery EICS 2017 Conclusions ● Post-hoc ● Flexible ● Reusability of queries ● Scalable ● Hypotheses easy to refine and test iteratively ● Connects designers with data 44
- 45. WevQuery EICS 2017 45 Work in progress
- 46. WevQuery EICS 2017 Future work ● Sequence pattern mining ● Further support for non-experts ● User evaluation 46
- 47. https://github.com/aapaolaza/wevquery aitor.apaolaza@manchester.ac.uk Aitor Apaolaza and Markel Vigo EICS 2017 June 28, 2017 - Lisbon, Portugal Questions? The MOVING project has received funding from the European Union’s Horizon 2020 research and innovation programme under grant agreement No 693092.