Presentation slides of the paper "Evaluating the Effect of Style in Information Visualization" during the IEEE Infovis conference (2012) in Seattle, WA, USA.

1. Evaluating the Effect of
Style in Information
Visualization
Andrew Vande Moere KU Leuven, Belgium
Martin Tomitsch The University of Sydney, Australia
Christoph Wimmer T.U.Wien, Austria
Christoph Boesch T.U.Wien, Austria
Thomas Grechenig T.U.Wien, Austria

2. Goal
•visualizationimpact of style in information
to measure
• by comparing 3 different ‘design alternatives’
• in terms of visual and interactive style
• style demonstrators based on real-world
examples
• then contrasted resulting insights against each
other

3. Dataset
•must be ‘agnostic’ to stylistic approach
• e.g. dance music vs. cancer statistics
•The New York Times news articles
• containing terms ‘hope’ or ‘fear’ (4,644)
• title, abstract, date, page number, news desk
• extra 24 descriptive keywords

4. “Reversible”
“Factual”
Gapminder (2007)
Many Eyes (2007)
OECD eXplorer (2009)

5. “Irreversible”
“Meaningful”
Bitalizer (2008) Poetry on the Road (2004) Texone (2005)

6. Partly “Reversible”
Partly “Factual”
Digg Swarm (2007)
ReMap (2009)
We Feel Fine (2006)

7. “Analytical” Style (ANA)

8. “Magazine” Style (MAG)

9. “Artistic” Style (ART)

10. Study
•1. style validation study
• did our 3 demonstrators correspond to the
according style examples?
•2. online evaluation study
• between-subject design
• recruitment through mailing lists on information
visualization, HCI, blogs, social media, etc.

11. Participation
44
persons
50
persons
44
persons
3.05 level
average expertise
Analytical Magazine Artistic

12. Interaction
18m09s
(average)
12m49s
(average)
11m55s
(average)
181.0
interactions
87.7
interactions
88.9
interactions
(average) (average) (average)
Analytical Magazine Artistic

13. No Reported Insights
1
participant
11
participants
9
participants
Analytical Magazine Artistic

14. Interface “Insights’
6%
6 insights
12%
13 insights
29%
27 insights
Analytical Magazine Artistic

15. Insight Typology
•insight classification (Chen et al.*)
• 2 independent coders (34% agreement)
• revisited classification (89% agreement)
• then decided together (100% agreement)
•“meaning”: added new insight class
• all connotations to ‘content’
(*)
Y. Chen, J. Yang and W. Ribarsky, “Toward Effective Insight Management in Visual Analytics Systems,” IEEE
Pacific Visualization Symposium (PacificVis'09), IEEE, 2009, pp. 49-56.

16. Insight Analysis
ANA MAG ART
Difference 24% (24) 26% (26) 17% (11)
Cluster 22% (22) 15% (15) 9% (6)
Distribution 11% (11) 12% (12) 17% (11)
Compound 9% (9) 14% (14) 11% (7)
Trend 8% (8) 4% (4) 8% (5)
Outliers 6% (6) 10% (10) 15% (10)
Value 6% (6) 1% (1) 0% (0)
Association 5% (5) 3% (3) 6% (4)
Meaning (*) 3% (3) 4% (4) 14% (9)
Extreme 4% (4) 6% (6) 0% (0)
Categories 2% (2) 1% (1) 0% (0)
Rank 1% (1) 2% (2) 3% (2)

17. Insight Analysis
ANA MAG ART
Difference 24% (24) 26% (26) 17% (11)
Cluster 22% (22) 15% (15) 9% (6)
Distribution 11% (11) 12% (12) 17% (11)
Compound 9% (9) 14% (14) 11% (7)
Trend 8% (8) 4% (4) 8% (5)
Outliers 6% (6) 10% (10) 15% (10)
Value 6% (6) 1% (1) 0% (0)
Association 5% (5) 3% (3) 6% (4)
Meaning (*) 3% (3) 4% (4) 14% (9)
Extreme 4% (4) 6% (6) 0% (0)
Categories 2% (2) 1% (1) 0% (0)
Rank 1% (1) 2% (2) 3% (2)

18. Insight Analysis
Rating (1 - 5) ANA MAG ART
uncertain - confident 4.10 (1.11) 4.21 (0.87) 4.17 (0.95)

19. Insight Analysis
Rating (1 - 5) ANA MAG ART
uncertain - confident 4.10 (1.11) 4.21 (0.87) 4.17 (0.95)
difficult - easy 3.78 (1.17) 3.63 (1.29) 4.00 (1.24)

20. Insight Analysis
Rating (1 - 5) ANA MAG ART
uncertain - confident 4.10 (1.11) 4.21 (0.87) 4.17 (0.95)
difficult - easy 3.78 (1.17) 3.63 (1.29) 4.00 (1.24)
shallow - deep 3.18 (1.10) 2.93 (1.08) 2.54 (1.17)

21. Insight Analysis
Rating (1 - 5) ANA MAG ART
uncertain - confident 4.10 (1.11) 4.21 (0.87) 4.17 (0.95)
difficult - easy 3.78 (1.17) 3.63 (1.29) 4.00 (1.24)
shallow - deep 3.18 (1.10) 2.93 (1.08) 2.54 (1.17)
shallow – deep (expert rating) 2.44 (0.78) 2.36 (0.70) 2.28 (0.64)

22. Insight Analysis
Rating (1 - 5) ANA MAG ART
ugly - beautiful 3.48 (0.85) 3.08 (1.03) 3.11 (1.02)
obtrusive - fluid 3.27 (0.95) 3.08 (1.01) 2.80 (1.00)

23. Insight Analysis
Rating (1 - 5) ANA MAG ART
ugly - beautiful 3.48 (0.85) 3.08 (1.03) 3.11 (1.02)
obtrusive - fluid 3.27 (0.95) 3.08 (1.01) 2.80 (1.00)
ambiguous - clear 3.39 (1.17) 1.98 (0.89) 2.00 (0.86)
difficult - easy to understand 3.55 (1.04) 2.08 (1.07) 2.14 (1.07)
intended inform – express 2.80 (1.15) 3.54 (1.18) 3.66 (1.06)
useless - useful 3.61 (0.95) 2.70 (1.09) 2.45 (0.90)
frustrating - enjoyable 3.43 (1.00) 2.54 (1.16) 2.34 (1.06)
unusable - usable 3.77 (0.91) 2.78 (1.13) 2.64 (1.12)
boring - engaging 3.43 (0.93) 3.10 (0.95) 2.80 (1.00)
non-functional - functional 3.93 (0.82) 2.80 (1.18) 2.50 (1.13)
tool - art 2.30 (1.07) 3.32 (1.19) 3.68 (0.93)

24. Discussion
•insight classification
• based on very short descriptions (M=17.86)
• methodology missing to benchmark insights
against each other
•‘controlling’ style
• are the 3 conditions representative?
• e.g. similarities MAG / ART

25. Conclusions
•style impacts perception of usability
• in particular for embellished versus non-
embellished styles
• analytical style was perceived as more
understandable, clear, enjoyable, engaging,
useful, functional, ...

26. Conclusions
•style does not impact insight depth
• participants were able to overcome huge
incomprehensibility issues of ART
• and in a minimum amount of time

27. Conclusions
•style has impact on ‘kind’ of insights
• analytical focus of facts versus meaning of
content, explanation of reasoning, ...
• driven by e.g. graphic incorporation of content,
fluidity of interface, ...

28. Guidelines
•to accurately benchmark insights...
• make distinction between analytical
characteristics of an insight and its meaning
• motivate participants to report insights in a more
expansive way
• e.g. insight categorization,...
• allow participants to report usability issues in
parallel with insights
• consider alternative ways of insight analysis
• e.g. card sorting, affinity diagramming,...

29. Thank you!
Andrew Vande Moere
andrew.vandemoere@asro.kuleuven.be
@infosthetics

30. Introduction Stage
continue >

31. Insight Recording Stage

32. Survey Stage