This document discusses visualizing data quality and uncertainty in the context of the Wiisard project. Wiisard is a joint project between the VA and UC San Diego to aid in mass casualty triage using PDAs and tablet PCs. The problem is that if the wireless network becomes partitioned, the displayed patient information may become outdated, leading to poor medical resource deployment decisions. The document outlines research in visualizing uncertainty in data to help humans make better judgments. It provides examples of visualizing uncertainty in data using techniques like contour widths, transparency, and color hue. It concludes with research questions on determining relevant data quality metrics and developing visualizations suited to different users and contexts.

1. Data Quality and
Uncertainty Visualization
UC San Diego
COGS 220
Winter Quarter 2006
Barry Demchak

2. Immediate Motivation: Wiisard
 A joint project of Veterans Administration and
UC San Diego, funded by the National Library
of Medicine
 Mass casualty triage and treatment
 Enter patient information via PDAs
 Patient information summarized on tablet PCs
 Command/control for supervisors and incident
comment personnel
 Tied together using 802.11b and store-and-
forward database access

3. Wiisard – Explosion with Pesticides

4. Wiisard – Network Deployment

5. Wiisard – Tablet Display

6. Wiisard – Command/Control

7. Wiisard – The Problem
 What if the network becomes partitioned?
 Tablet display shows out-of-date patient
information
 Summary displays are out of date, too
 How does this lead to bad decisions?
 Supervisors may mis-deploy doctors
 Incident command may mis-deploy resources
People may die

8. DOD Example
 Sensor-to-shooter (STS) Networks – Patrick
Driscoll (USMA), June 2002

9. DOD Example

10. DOD Example
 “… our first attempt to get the military
community to realize that there is a degree of
uncertainty involved in (digital) information
systems that cannot be engineered out of the
system.”
 “Ultimately, our concern was an awareness
issue (for the decision maker) …”
 “… woman at MITRE had proposed a system
of tagging intelligence starting at the source
in a way that would reflect the uncertainty of
the data being put into the intel database.”

11. The Problem
 How to visualize the uncertainty in data so
that humans can exercise judgment in
making the best decision
 Accounting for uncertainty is not the same
thing as visualizing uncertainty

12. What Labs are Involved
 MIT Sloan School of Management
 Richard Wang (Data Quality)
 Penn State University
 Alan MacEachren (GIS)
 University of Maine
 Kate Beard-Tisdale (GIS)
 University of California, Santa Cruz
 Alex Pang (Scientific Visualization)
 University of Arkansas, Little Rock
 Master of Sciences in Information Quality

13. What Conferences are There?
 MIT Information Quality (IQatMIT)
 ACM SIGMOD Workshop on Information Quality in
 ACM SIGKDD (Knowledge Discovery and Data M
 MIT International Conference on Information Qual

14. Semiotic Interpretation

15. Definition of Data Quality
 From Wand & Wang:

16. Metrics
 Timeliness
 How up to date relative to intended purpose
 Ballou et al:
 Timeliness = Max(0, 1-(currency/volatility)
 Currency = delivery_time – input_time
 Volatility = length of time data remains valid
 Apply sensitivity factor “s”: Timeliness ^ s

17. Interplay with Uncertainty
 Metrics are application dependent
 Metrics are data dependent
 Metrics are user dependent
 Question: If a metric describes an individual
data element, what is the effect of
aggregating data elements having
uncertainty??

18. GIS Examples – NCGIA
 Sample point locations
as overlay
 Sample points and
corresponding contours
using naïve shading

19. GIS Examples – NCGIA
 Gray shading
uncertainty surface
captures distance
function used by
interpolation method
 Uncertainty encoded in
contour line widths

20.  Fill Clarity
 Resolution
GIS Techniques
 Contour Crispness
 Fog

21. Merging Data and Uncertainty
 Risk and
uncertainty
separately
 Risk and
uncertainty
combined

22. Basic Data Examples
 Errors

23. Basic Data Examples
 Errors

24. Basic Data Examples
 Ambiguation

25. Basic Data Examples
 Ambiguation

26. Photo Realistic

27. Uncertainty Vector Glyphs

28. Uncertainty Vector Glyphs

29. Hue as Uncertainty
 With
out
 With

30. Texture as Uncertainty
Raw
Trans-
parent
Points
Cer-
tain-
ty
Opaque
Lines

31.  Data Confidence
 x is a device, α is decay constant, R(x) is a
weighting for device x in the calculation
Back to Wiisard
∑












•




 −
+
=
x
xpingtime
xposttimecurtime
xRC
α
)(
)(
1
1
)(

32. Back to Wiisard
 Individual data (annotation)
 Aggregate data (annotated/integrated)

33. Back to Wiisard
 Annotated

34. Back to Wiisard
 Integrated

35. Research Questions
 What are the dimensions of metrics relevant
for determining data quality for medical
providers in a mass casualty context?
 What kind of visualization best conveys the
use suitability for various kinds of data?
 Single data points
 Streaming bioinformation
 Aggregated information

36. Research Questions
 What kinds of visualizations are best suited to
field personnel?
 Non-IS frenzied technicians
 High glare, small footprint screens
 Low processing power
 What kinds of visualizations are best suited to
incident command?
 Seasoned experts
 Large, high density displays
 Highly connected with high data processing

37. Conclusion
 Data Quality and Uncertainty Visualization
are like the weather …
… everyone’s talks about it, but no one
does anything about it