VII Congrés ACCID - Presentació de la ponència: Myths and truth of excellence in reporting design

1. PAGE I
Myths and Truth of Excellence in Reporting Design
Heimo Losbichler

2. PAGE I 2
Agenda
 About Us
 Eye Tracking - Methodology
 Use Cases
 Selected Findings
 Current Research and Outlook

3. PAGE I
Today, we are an international recognized competence center for
reporting design - based on six years of applied and basic research
− 6 Full Professors
− 1 Post-doc Professor
− 2 PdD Students
− 5 Research Assistents
Team
To identify criteria that
make the visualization of
reports objectively good
Objective
− Internal and external
reporting/dashboards/...
− Single visualizations
− Single report pages
− Full reports
Scope
− +1,000 Eye-tracking tests
− Unbiased findings
(general, specific, non)
− Comprehensive Models
− Leading scientific
conferences and journals
Research
Media
− Print
− Large screen interactive
displays
− Mobile devices
− Multiple devices
− Projects with leading
companies
− We do not push a
standard
− Company specific
Practice
Research that
matters!

4. PAGE I 4
Agenda
 About Us
 Eye Tracking - Methodology
 Use Cases
 Selected Findings
 Current Research and Outlook

5. PAGE I 5
Eye-tracking tests are easy to conduct but require a sound
methodology

6. PAGE I 6
Eye-Tracking provides new insights and unveils how people
read reports and process information
 Scanpaths  Focusmaps
 AOIs (Areas of Interest)

7. PAGE I 7
Can report readers draw the
right conclusion from the
presentation?
Measured by task accuracy
(% of correct answers)
How long do report readers need to draw their conclusion?
Measured by task completion time (in seconds)
Is the visualization appealing to the
reader?
Qualitative feedback of the
report reader
We use three criteria for evaluating the “excellence” of visualizations
Effectiveness
Efficiency
Attractiveness
Effectiveness
Efficiency
1
2
3

8. PAGE I
How much is the difference between PREVIOUS YEAR and ACTUAL for
the product group SHOES?
Try yourself and answer the following question:
EBIT structure
Hycom Group
Dec. 2014
in MEUR
Key:
Actual
Previous Year
3,4
2,5
3,0
1,6
clothing / adults
clothing / children
shoes / adults
shoes / children
3,9
3,1
2,2
1,3
8

9. PAGE I 9
How much is the difference between PREVIOUS YEAR and ACTUAL for
the product group SHOES?
Try again with a different design:
EBIT structure
Hycom Group
Dec. 2014
in MEUR
Key:
Actual
Previous Year
positive deviation
negative deviation
3,4
2,5
3,0
1,6
3,4
2,5
3,0
1,6
0,8
0,3
-0,5
-0,6
3,9
3,1
2,2
1,3
clothing / adults
clothing / children
shoes / adults
shoes / children

10. PAGE I 10
Effectiveness (task
accuracy): 85% vs. 100%
Efficiency (task time):
24,7 sec. vs. 19,0 sec..

11. PAGE I
The heat map gives an overview of all report readers
11

12. PAGE I
Alternative designs can be objectively compared ...
12
Comparison of two different designs:
n original design
n alternative design
Error rate
in %
respone time
in sec.
Report 4
Report 3
Report 2
Report 1
50%
100%
0 13 25 38 50

13. PAGE I
... and differences among user groups can be identified
13
Comparison of two different user groups:
n group 1
n group 2
Error rate
in %
respone time
in sec.
Report 4
Report 3
Report 1
Report 2
50%
100%
0 13 25 38 50

14. PAGE I 14
Agenda
 About Us
 Eye Tracking - Methodology
 Use Cases
 Selected Findings
 Current Research and Outlook

15. PAGE I
We apply a dual fact-based approach
Industry projects
Analysis of
current reports
Selection, customer
prefernces
Design of alternatives
Selection
Test setup, test
Evaluation
Presentation
Facts based final design
Guideline
Research
Theories, research
question
Test setup, test
Evaluation
Findings
Statistical models
15

16. PAGE I
Design and selection of alternatives
J
Y
Y
JY
N
16
Analysis of
current reports
Selection, customer
prefernces
Design of alternatives
Selection

17. PAGE I 17
Test and evaluation: six readers – original design

18. PAGE I 18
Test and evaluation: six readers - alternative design

19. PAGE I 19
Agenda
 About Us
 Eye Tracking - Methodology
 Use Cases
 Selected Findings
 Current Research and Outlook

20. PAGE I 20
General findings: use appropriate visualization
Sales Volume 4 5 6 7 8 9 10 11 12 13 14
Profit 2,30 4,47 6,13 7,26 8,14 8,77 9,14 9,26 9,13 8,74 8,10
0
5
10
3 4 5 6 7 8 9 10 11 12 13 14
Profit
Sales Volume

21. PAGE I
General findings: Avoid „non-data ink“ and use data labels if
possible
21
Direkte Beschriftung und das
Vermeiden von Dekoration
erhöhen die Geschwindigkeit
Fehlerquote: 0% vs. 26,9%
Ø Antwortzeit: 11,9 vs. 18,8

22. PAGE I
Specific findings (Europe): Use column charts or line charts
for time series from left to right and avoid bar charts
22
.
Error rate: 0.0% vs. 7.7%
Ø Task time: 10.3 vs. 12.6

23. PAGE I 23
Findings: standardization works at the individual level and can
improve effectiveness and efficiency
time in millisecond
repetition in weeks
32.734,0
26.072,8
24.629,8
20.905,1 21.580,1
1 2 3 4 5
-34,1%
Efficiency

24. PAGE I 24
Findings: standardization works at the individual level. Eye-tracking
shows why!
Week 1: efficiency 32.7 seconds

25. PAGE I 25
Findings: standardization works at the individual level. Eye-tracking
shows why!
Reader is familiar with
the design and does
not read the legend
anymore!
Week 5: efficiency 21.6 seconds (-30%)

26. PAGE I
Findings: standardization ignores differences between individuals such
as cognitive stiles or cultural differences and specific situations
26

27. PAGE I
Findings: standardization ignores differences between individuals such
as cognitive stiles or cultural differences and specific situations
27

28. PAGE I
Findings: standardization ignores differences between individuals such
as cognitive stiles or cultural differences and specific situations
28

29. PAGE I
29
Differences need to be considered between companies,
groups, individuals, tasks, etc.
Task type
Visualization
(type and design)
Data type Cognitive load
Intrinsic
Extrinsic
Recipient
 Working memory capacity
 Experience/domain expertise
 Knowledge
Efficient and effective
information processing
and decision making

30. PAGE I 30
Agenda
 About Us
 Eye Tracking - Methodology
 Use Cases
 Selected Findings
 Current Research and Outlook

31. PAGE I 31
Current research: visualization and interaction for collaborative
exploration of big data and group decision making
Treemaps Network-Charts
Parallel-Coordinates Sankey-Charts

32. PAGE I 32
Prototype “Interactive Explorer”: MS BI supplier quality
analysis
Sankey-Diagramm
Key questions:
Which suppliers
provide the best
and worst quality?
Which plants
recognize/reject
defects on time
and minimize
downtime?

33. PAGE I 33
Prototype “Interactive Explorer”: MS BI supplier quality
analysis
Sankey-Diagramm
Key questions:
Which suppliers
provide the best
and worst quality?
Which plants
recognize/reject
defects on time
and minimize
downtime?

34. PAGE I 34
Collaborative sense making and group decision making -
MS Surface Hubs & Tablets

35. PAGE I
Video
35
Collaborative sense making and group decision making
Huddle Lamp

36. PAGE I 36
Collaborative sense making and group decision making -
Multi-tablet-prototype

37. PAGE I
37
More information (German): www.top-reports.com

38. PAGE I 38
Contact
Prof. (FH) Dipl.-Ing. Dr. Heimo Losbichler
FH Oberösterreich F&E GmbH - Controlling,
Rechnungswesen und Finanzmanagement (CRF)
Wehrgrabengasse 1-3
4400 Steyr
Mobil: +43 (0) 7252 884 33700
heimo.losbichler@fh-steyr.at
Prof. (FH) DI Mag. Peter Hofer
FH Oberösterreich - Controlling, Rechnungswesen und
Finanzmanagement (CRF)
Wehrgrabengasse 1-3
4400 Steyr
Tel.: +43 (0) 50804-33911
peter.hofer@fh-steyr.at