Is Your User Hunting or Gathering Insights? Identifying Insight Drivers Across Domains.


Published on

Presenter: Michael Smuc, Eva Mayr, Hanna Risku
BELIV 2010 Workshop

1 Like
  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Is Your User Hunting or Gathering Insights? Identifying Insight Drivers Across Domains.

  1. 1. Is Your User Hunting or Gathering Insights? Identifying Insight Drivers Across Domains Michael Smuc, Eva Mayr, Hanna Risku
  2. 2. Motivation <ul><li>Goal: Evaluate a visualization tool for temporal pattern analysis </li></ul><ul><li>Expert-User Dilemma </li></ul><ul><ul><li>Real domain experts are rare , hard to find (and to motivate) </li></ul></ul><ul><ul><li>Sometimes they don‘t even exist </li></ul></ul><ul><li>Across-domain development </li></ul><ul><li>What if you wanted to develop a tool suitable for different domains? </li></ul>
  3. 3. Solutions <ul><li>Use heuristics instead of insight analysis </li></ul><ul><ul><li>In case you find suitable ones for your tool, you can‘t keep in touch with the users </li></ul></ul><ul><li>Educate non-experts </li></ul><ul><ul><li>time consuming, „they will tell you what you taught them to tell” </li></ul></ul><ul><ul><li>even more time-consuming for multiple domains </li></ul></ul><ul><li>=> we would like to propose another solution </li></ul>
  4. 4. Our approach <ul><li>Across-domain testing: Experts have to solve tasks in data visualizations of their own domain and from other domains </li></ul><ul><li>Additional aid story about the data </li></ul><ul><li>Research questions </li></ul><ul><ul><li>How do experts differ in different domains? More insights? </li></ul></ul><ul><ul><li>Results only useful when experts work in own domain? </li></ul></ul><ul><ul><li>… and what makes an domain expert? </li></ul></ul>
  5. 5. Specific knowledge of domain experts
  6. 6. Common ground of domain experts in our case „temporal data explorers“
  7. 8. Insight Study <ul><li>9 experts in temporal data analysis from 4 different domains </li></ul><ul><li>think-aloud </li></ul><ul><li>Insights: quantitative & qualitative analysis </li></ul><ul><li>Q: Effect of domain expertise? </li></ul>
  8. 9. => Across domain testing works! Effect of domain expertise on insights t = -0.29, df = 24, p > .05
  9. 10. Different kinds of insights in a catering business dataset <ul><li>Type 1: &quot;At noon there is a red belt.&quot; </li></ul><ul><li>Type 2: &quot;There is quite some breakfast business.&quot; </li></ul>
  10. 11. Type 1 Type 2
  11. 12. Typology <ul><li>Type 1: insight-gatherers: simple description </li></ul><ul><li>Type 2: insight-hunters: active search for insights, driven by prior knowledge </li></ul><ul><ul><li>“… . even the smallest domain information is used to create novel interpretations and make as much sense of the data as possible ” </li></ul></ul><ul><li>no sign. differences for the number of insights </li></ul><ul><li>but significant differences for use of prior knowledge & hypotheses </li></ul>
  12. 13. <ul><li>Is insight hunting driven by domain-expertise? </li></ul><ul><li>NO , 50% of the domain experts, but 25% of the non-domain experts showed this behavior </li></ul><ul><li>(no sign., but small dataset) </li></ul><ul><li>only every second domain expert hunted for insights </li></ul>
  13. 14. Discussion <ul><li>Across-domain-testing works, but domain expertise needs a differentiated approach? (redefinition?) </li></ul><ul><li>experts common ground & the story are insight-drivers </li></ul><ul><li>Even shallow insights are useful, insights interrelate </li></ul><ul><li>Typology iHunters | iGatherers </li></ul><ul><ul><li>allows selective sampling </li></ul></ul><ul><ul><li>“ hunters’ insights are the best argument to sell“ </li></ul></ul><ul><ul><li>sampling is easier </li></ul></ul><ul><ul><li>“ sometimes it is sufficient to gather the second choice experts ” </li></ul></ul>
  14. 15. Questions ? what is expertise? across domain testing shallow insights Relational Insight Organizer RIO what is a domain? insight hunting insight gathering experimental setting story about the data sampling what makes an expert? prior knowledge compensation by experts common ground applicability future research generalizability
  15. 16. additional
  16. 17. t = 0.16, df = 14, p > .05 t = -3.80, df = 7, p < .01 t = -4.87, df = 14, p < .001
  17. 18. Definition of insights “… the understanding gained by an individual using a visualization tool (or parts thereof) for the purpose of data analysis, which is a gradual process toward s discovering new knowledge “