You don’t have to be a detective for your job to involve detective work. Various kinds of data visualizations can help solve mysteries using partial clues that have complex relationships. Using examples from medicine, science, and security, this presentation shows how visualizations can enhance the investigative process.

1. Data
Detec'ves
Enhancing
the
inves'ga've
process
with
visualiza'ons
Hunter Whitney
San Francisco
November 13, 2013

2. Think
of
a
detec.ve
trying
to
solve
a
case
where
the
answer
may
lie
hidden
within
stacks
of
evidence,
.me
is
limited,
and
there
are
serious
consequences
for
any
errors.

3. From
healthcare
workers
to
security
professionals,
this
same
basic
scenario
applies
to
many
people
who
work
with
data.

4. While
there’s
no
single
approach
to
finding
answers
in
an
array
of
data,
UX
design
can
help
support
various
inves.ga.ve
processes.

5. • PuCng
the
pieces
together
• Weighing
the
evidence
• Evidence
handling
• Collabora.ng
Data
as
evidence

6. Pu8ng
the
Pieces
Together
Users
oFen
have
to
work
with
evidence
that
is:
• Fragmentary,
mixed,
and
incomplete
• Delivered
in
an
asynchronous,
non-­‐sequen.al
manner

7. Case
1:
Where
and
when
did
an
outbreak
of
a
foodborne
illness
begin?
Visual
Forensics

8. Visually
connec'ng
data
about
'me,
place,
and
rela'onship
Palan.r
Technologies,
Inc.

9. Case
2:
Who
is
the
ringleader
of
a
recent
interna'onal
bank
fraud
scheme?
Visual
Forensics

10. Visually
connec'ng
data
about
'me,
place,
and
rela'onship
Palan.r
Technologies,
Inc.

11. Case
3:
Why
are
a
soldier’s
post-­‐trauma'c
symptoms
not
responding
to
treatment?
Visual
Forensics

12. Visual
juxtaposi'on
of
different
kinds
of
data
EEG
MRI
MRI
(DTI)
Electrical
ac.vity
data
Anatomical
Features
Fiber
Tracts
(Connec.vity)
Hunter
Whitney
and
Jeff
Chang,
MD
Please
see
hSp://vimeo.com/album/2489932

13. Visual
juxtaposi'on
of
different
kinds
of
data
EEG
MRI
MRI
(DTI)
Electrical
ac.vity
data
Anatomical
Features
Fiber
Tracts
(Connec.vity)
Hunter
Whitney
and
Jeff
Chang,
ER
Radiologist
Please
see
hSp://vimeo.com/album/2489932

14. MRI +
DTI
DTI
Visual
transi'ons
of
different
kinds
of
data
Hunter
Whitney
and
Jeff
Chang,
ER
Radiologist
Please
see
hSp://vimeo.com/album/2489932

15. Weighing
the
evidence
A
piece
of
evidence
does
not
exist
in
isola'on.
It
should
be
considered
in
context
(but
the
context
can
rapidly
change):
• The
rela.ve
meaning
and
significance
of
any
piece
of
data
may
change
as
other
available
informa.on
comes
in
• A
piece
of
evidence
may
be
true
but
irrelevant
or
poten.ally
misleading
if
not
well
framed

16. Case
4:
What
is
causing
a
child’s
persistent
fever,
rash,
and
joint
pain?
Visual
Forensics

17. THE
CASE
A
previously
healthy
12-­‐year-­‐old
boy
presents
to
an
emergency
department
in
southern
Ohio
in
October
with
a
chief
complaint
of
fever
and
rash.
The
pa'ent
has
had
subjec've
fever
for
3
days
with
weakness,
malaise,
anorexia,
and
decreased
ac'vity.
He
had
emesis
followed
by
diarrhea
on
the
second
day.
He
complains
of
diffuse
myalgias
and
migratory
arthralgias,
which
began
in
the
right
shoulder
and
have
progressed
to
involve
the
cervical
spine
and
all
joints
of
the
extremi'es
bilaterally,
but
sparing
the
hips.
A
rash
developed
bilaterally
on
his
inner
thighs
at
the
onset
of
illness,
then
spread
to
involve
his
forearms,
abdomen,
trunk,
and
hands,
sequen'ally.
In
the
ED,
his
temperature
is
37.3C,
blood
pressure
is
136/70
mmHg,
pulse
is
104
beats
per
minute,
and
respiratory
rate
is
24
breaths
per
minute.
The
pa.ent's
past
medical
history
is
significant
only
for
aSen.on-­‐deficit
disorder
(ADD)
without
hyperac.vity,
and
his
medica.ons
include
acetaminophen,
melatonin,
and
methylphenidate.
He
has
no
known
drug
allergies,
and
his
immuniza.ons
are
current.
Physical
examina.on
reveals
erythematous
lesions
on
the
palate
and
pain
with
ac.ve
and
passive
range
of
mo.on
in
all
extremi.es.
There
is
no
joint
swelling,
erythema,
or
warmth.
A
red
maculopapular
rash
with
scaSered
vesicular
lesions
is
present
on
the
thighs,
forearms,
trunk,
and
palms.
A
rapid
group
A
streptococcal
an.gen
test
on
a
throat
swab
is
nega.ve.
The
pa.ent
is
Tanner
stage
4
for
genital
development.
He
has
no
discharge
or
lesions.
The
pa.ent
is
prescribed
ibuprofen
and
discharged
from
the
ED
with
a
diagnosis
of
hand,
foot,
and
mouth
disease.
Worsening
symptoms
The
pa.ent
returns
to
the
ED
2
days
later
with
persistent
fever,
worsening
arthralgia,
and
new
swelling
in
the
leF
knee.
He
is
unable
to
ambulate
and
has
decreased
oral
intake.
Both
wrists
and
mul.ple
finger
joints
have
become
painful.
The
pain
is
greatest
in
the
right
wrist
and
leF
knee.
Further
ques.oning
reveals
that
the
pa.ent
traveled
to
southern
Kentucky
about
4
weeks
ago
for
a
Labor
Day
camping
trip
with
his
family.
He
recalls
having
found
.cks
on
his
clothing
but
not
on
his
skin.
He
denies
alcohol
or
drug
use.
The
family
has
numerous
pets
including
1
hermit
crab,
1
African
clawed
frog,
2
lizards,
1
domes.c
white
rat,
1
python,
and
a
tank
of
tropical
fish.
They
had
2
dogs,
but
both
recently
died
of
unknown
causes.
The
pa.ent
denies
any
animal
bites.
He
reports
that
he
has
a
girlfriend,
but
he
denies
any
sexual
ac.vity.
His
temperature
is
38.9C,
blood
pressure
is
100/72
mmHg,
pulse
is
110
beats
per
minute,
and
respiratory
rate
is
30
breaths
per
minute.
The
rash
has
progressed,
with
an
increased
number
of
well-­‐demarcated,
macular,
papular,
and
vesicular
lesions
involving
the
extremi.es,
especially
the
palms
and
soles.
The
lesions
are
blanching
and
nonpruri.c.
The
leF
knee
is
warm,
nonerythematous,
and
swollen
with
ballotable
fluid.

18. Further
ques.oning
reveals
that
the
pa.ent
traveled
to
southern
Kentucky
about
4
weeks
ago
for
a
Labor
Day
camping
trip
with
his
family.
He
recalls
having
found
.cks
on
his
clothing
but
not
on
his
skin.
He
denies
alcohol
or
drug
use.
He
reports
that
he
has
a
girlfriend,
but
denies
any
sexual
ac.vity.
The
family
has
numerous
pets
including
1
hermit
crab,
1
African
clawed
frog,
2
lizards,
1
domes.c
white
rat,
1
python,
and
a
tank
of
tropical
fish.
They
had
2
dogs,
but
both
recently
died
of
unknown
causes.
The
pa.ent
denies
any
animal
bites.
Many
clues,
but
which
are
relevant?

19. Accelera'ng
the
diagnos'c
process
with
visualiza'ons
• Sugges.ng
possibili.es
• Process
of
elimina.on
• Visual
emphasis
of
promising
direc.ons
• Dynamic
weigh.ng
• Transparency
of
the
process

20. Differen'al
Diagnosis:
the
full
range
of
poten.al
causes,
or
combina.ons
of
causes,
for
a
set
of
symptoms
Reaves,
L;
Hamilton,
S;
Vossmeyer,
M;
and
Brady,
R.
Fever,
rash
and
migratory
polyarthralgia.
Contemporary
Pediatrics.
Mar
2010;27(3):24-­‐38.

21. Juvenile Idiopathic Arthritis
Postinfectious arthritis
Differen'al
Diagnosis
(Alternate
layout
1)
Henoch-Schönlein purpura
Kawasaki disease
Infectious
Rheumatologic
Familial Mediterranean Fever
Genetics
Septic arthritis
Lyme disease
Bacterial
Enteroviruses
Parvovirus
Viral
+5
-­‐4
-­‐6
History and Physical
Labs
Radiology
Fever,
rash,
joint
pain
-­‐3
+3
-­‐3
more
Rock Mountain Spotted Fever
Hunter
Whitney
and
Veena
R
Kumar,
Pediatric
Emergency
physician

22. Diagnos'c
Dilemma:
when
the
most
likely
causes
of
a
pa.ent’s
symptoms
have
been
ruled
out
and
a
defini.ve
diagnosis
has
not
been
established

23. Differen'al
Diagnosis
(Alternate
layout
1)
Infectious
Septic arthritis
Lyme disease
Bacterial
Viral
+5
-­‐4
History and Physical
Labs
Radiology
Fever,
rash,
joint
pain,
handles/sleeps
with
pet
rat
oFen,
rat
bit
both
dogs
-­‐3
+3
Rock Mountain Spotted Fever
Meningococal meningitis
Gonorrhea
Secondary syphilis
Leptospirosis
Rat-bite fever
+2
CBC
normal,
Rapid
strep
neg.,
blood
culture
neg.,
joint
fluid
culture
posi.ve
for
Strep.
moniliformis
Chest
X-­‐ray
nega.ve
Hunter
Whitney
and
Veena
R
Kumar,
Pediatric
Emergency
physician

24. Further
ques.oning
reveals
that
the
pa.ent
traveled
to
southern
Kentucky
about
4
weeks
ago
for
a
Labor
Day
camping
trip
with
his
family.
He
recalls
having
found
.cks
on
his
clothing
but
not
on
his
skin.
He
denies
alcohol
or
drug
use.
He
reports
that
he
has
a
girlfriend,
but
denies
any
sexual
ac.vity.
The
family
has
numerous
pets
including
1
hermit
crab,
1
African
clawed
frog,
2
lizards,
1
domes'c
white
rat,
1
python,
and
a
tank
of
tropical
fish.
They
had
2
dogs,
but
both
recently
died
of
unknown
causes.
The
pa.ent
denies
any
animal
bites.
Many
clues,
but
which
were
relevant?

25. Evidence
Handling
• Visualiza.ons
are
just
one
part
of
inves.ga.on
and
analysis
• All
the
decisions
star.ng
from
data
collec.on,
storage,
organiza.on,
sta.s.cal
summariza.ons,
modeling,
etc.
will
influence
the
analysis
and
interpreta.on
When
possible:
• Enable
easy
annota.ons
and
access
to
source
material
• Provide
mul.ple
views/ways
of
looking
at
the
data

26. No
ma^er
how
good
the
visualiza'on,
the
outcome
of
the
inves'ga'on
depends
on
the
integrity
of
every
step
of
the
process
From
Data
Insights:
New
Ways
to
Visualize
and
Make
Sense
of
Data
by
Hunter
Whitney
(Morgan
Kaufmann
Publishers,
November
2012)

27. Collabora'ng
• Willing
to
acknowledge
the
limits
of
your
own
exper.se
and
to
ask
for
help
when
necessary
• Visualiza.ons
can
enhance:
• Partnering
with
machines:
detec.ves
have
their
Watson
• Collabora.ng
with
other
people

28.
• There’s
a
lot
of
experimenta.on
and
a
growing
awareness
of
poten.al
uses
• Visualizing
data
is
nothing
new,
but
is
now
evolving
rapidly
• Many
different
kinds
of
creators
and
users
are
coming
to
the
table
with
varying
exper.se,
perspec.ves,
and
goals
• Approaches
to
data
visualiza.ons
are
definitely
not
one
size
fits
all.
That’s
why
the
UX
design
dimension
is
so
important!
Problem
Solving
with
Emerging
Tools

29. Thank
You

30. Slide
1:
hSp://i.huffpost.com/gen/1375606/thumbs/o-­‐DETECTIVE-­‐facebook.jpg
Slide
2:
Slide
3:
hSp://www.theguardian.com/tv-­‐and-­‐radio/tvandradioblog+medical-­‐drama
Slide
4:
http://images4.wikia.nocookie.net/__cb20091004222608/flashforward/images/1/1e/MI_wall.jpg
Slide 5: http://www.cafepress.com/+forensic_evidence_large_poster,698750731
Slides 6 and 14: http://www.cs.rit.edu/~rlc/Courses/IS/Projects/20131/Proj2/Clue_rules.html
Slide 26: http://www.gonemovies.com/WWW/Hoofd/A/PhotoLarge.php?Keuze=BrettJeremy, http://
www.digitaltrends.com/computing/ibm-watson-supercomputer-goes-to-college/
Supplementary Image Credits: