Complex Health Data Visualization

MAKING SENSE OF RISK BY VISUALIZING
COMPLEX HEALTH DATA
Nicholas Tenhue
MSc ICT Innovation
University College London, 2014

COMPLEX HEALTH DATA
Nicholas Tenhue
MSc ICT Innovation & Data Vis & Design Intern, Intel Health & Life Sciences
Author and Principal Investigator
Ann Blandford
Professor of Human-Computer Interaction
Thesis Supervisor
Chiara Garattini
Anthropology & UX Research, Intel Health & Life Sciences
Industry Supervisor
This report is submitted as part requirement for the MSc Degree in ICT Innovation,
at University College London. It is substantially the result of my own work except
where explicitly indicated in the text.
The report may be freely copied and distributed provided the source is
explicitly acknowledged.
September 2014
University College London, 2014

1
COMPLEX HEALTH DATA
Methods for visualising patient data in a way that supports sensemaking
may help clinicians to understand risk factors at the individual patient level.
This thesis uses sensemaking theory and visualisation techniques to de-
velop a tool and test it with clinicians in the healthcare domain. This is an
exploratory study into how information visualisation techniques can help cli-
nicians make sense of risk in a patient. We present a chronological account
of the approach taken to build and assess a visual tool for sensemaking.
We present two main findings (i) making sense of risk is a multifaceted pro-
cess that entails complexity beyond just using research evidence and clini-
cal expertise (ii) we have preliminary evidence that the visual tool supports
by creating externalisations that facilitate to make the implicit processes
that they use frequently in their work, explicit.
Keywords: information visualisation, sensemaking, design

3
I owe this journey of discovery and learning to a number of people.
Academic Supervisor, Ann Blandford, who gave sound advice throughout.
Industry Supervisor, Chiara Garattini, for advising me with her deep
knowledge of anthropology and UX.
Mentors, Mario Romero and Connor Upton, who shared their expertise in
information visualisation.
My peers Misha Patel, Hanna Schneider and David Pribil for their constant
encouragement.
This thesis is written and reported, in entirety, by the Author. However,
employees at the Author’s internship company provided invaluable
assistance:
 Chiara Garattini – risk calculator research & persona creation; partici-
pants’ recruitment;
 Marisa Parker – assisting in design of Stage 2 & 3 prototypes;
 Reese Bowes – final screenshots for use in external publication.

5
TABLE OF CONTENTS
1 Introduction..........................................................................................7
2 Background .......................................................................................11
2.1 Sensemaking..............................................................................11
2.2 Visualisation for sensemaking ....................................................14
3 Methods.............................................................................................21
3.1 Participants.................................................................................21
3.2 Apparatus & Materials ................................................................22
3.3 Ethical considerations.................................................................25
4 Design, study & analysis....................................................................27
4.1 Patient personas.........................................................................27
4.2 Stage 1.......................................................................................29
4.3 Stage 2.......................................................................................33
4.4 Stage 3.......................................................................................48
5 Results ..............................................................................................61
5.1 Data complexity & the implicit.....................................................61
5.2 Clinical workflow.........................................................................64
5.3 Thinking about risk .....................................................................66
5.4 Dealing with data........................................................................68
5.5 Visualising risk............................................................................74
6 Limitations .........................................................................................83
6.1 Design failures in the tool ...........................................................83

6
6.2 Limitations of the study...............................................................85
7 Conclusions.......................................................................................87
8 Bibliography.......................................................................................89
9 Appendices........................................................................................97
9.1 Appendix A – Participant Information..........................................97
9.2 Appendix B – Consent Form.....................................................100
9.3 Appendix C – Interview Plan (Stage 2) .....................................101
9.4 Appendix D – Interview Plan (Stage 3) .....................................104
9.5 Appendix E – Requirements Statement ....................................105

7
1 INTRODUCTION
When dealing with risk the clinician must search for and decide what data is rele-
vant to the individual patient. This complex task involves judging the integrity of
the data, the relevance of that data to the individual patient, the particulars of the
circumstances, patient wishes, and a host of other variables. Few tools support
this process because they downplay the role of clinical expertise for judging par-
ticular circumstances, instead they rely only on empirical population studies that
may or may not apply to the individual in question and leave the rest to the clini-
cian.
Clinicians are forced to deal with an excess of data in their work (Feblowitz,
Wright, Singh, Samal, & Sittig, 2011). This issue will continue to grow as a sur-
plus of noisy, multivariate, homogeneous data is generated from a number of dis-
similar sources. Clinicians already deal with patient reported data and hospital
generated data. Also, the proliferation of wearables and mobile devices lead us to
believe that doctors are increasingly exposed to self-generated health data (My-
natt, 2011), providing new insight into patients’ lives that population studies. Most
of the existing risk algorithms and evidence-based research do not presently take
data generated from these new types of eHealth self-monitoring devices into ac-
count. In addition, recent developments in whole genome sequencing and ge-
nomic science are making personalised healthcare possible. There is potential to
integrate this wealth of data with traditional health data, to present a tailored rep-
resentation of patient. Nevertheless, the question arises of how the clinician
makes sense of all of these factors to form an overarching understanding of risk
in an individual patient.
Patient information is highly complex with data intervals ranging from minutes to
decades (Shneiderman, Plaisant, & Hesse, 2013). The best possible care is de-
livered when clinicians can, without difficulty, consolidate and make sense of this
patient information in a way that matches their mental model (Johnson-Laird,
1983). Clinicians must seek out data and organize it internally to form a unified

8
understanding of the patient’s current condition (Faiola & Hillier, 2006), this in-
creases cognitive load and the time spent foraging for relevant information.
Possibly more problematic is the lack of externalisations that show relational or
context-based data (Faiola & Hillier, 2006) that would allow clinicians to recog-
nise trends and relationships between co-variables. This leads to the question
‘How can we inform, rather than overwhelm clinicians when they are faced with
these problems?’
One way to tackle this issue is through the use of digital, interactive, visual repre-
sentations of data (Card, Mackinlay, & Shneiderman, 1999). Through the effec-
tive use of visualisations it is possible to deliver what Spence (2007) calls an ‘A
Ha!’ moment, providing insight to task-specific problems that clinicians face. Yet,
information visualisation alone is not the answer. The solution must also support
clinicians’ sensemaking whilst focusing on the quality of the fit between the user
and system models.
This thesis expands on existing knowledge within information visualization and
sensemaking literature and attempts to apply it to creating a tool to support clini-
cians who deal with complex health data when assessing risk.
The aims of this thesis are twofold:
 Firstly, we aim to provide a domain specific account of the way in which clini-
cians make sense of complex health data when assessing risk in a patient.
 Secondly, by understanding clinicians’ needs and practices we aim to itera-
tively develop a visualisation tool with representations that reduce the gap
between the data and the clinicians’ mental model.
The goal of this thesis is to stimulate discussion in the human-computer interac-
tion (HCI) and information visualisation communities by contributing to the under-
standing of how visualization of complex health data can support clinicians in
making sense of a patient’s risk. To further this goal, this thesis presents findings
from an exploratory study where a tool was designed to support clinicians’ needs
during the assessment of risk.

9
Chapter 2 begins with a review of relevant literature in sensemaking and infor-
mation visualisation. We discuss how visualisations can be powerful tools for
making sense of a domain, and then look at literature on how to design visualisa-
tions. In Chapter 3, the methods used in the study are presented along with the
rationale behind using them. Chapter 4 is a chronological account of the project.
The design process of the tool created for the study, study procedure, and ap-
proach to analysis of each stage are described. The HCI and information visuali-
sation methods that were used are reflected upon. Findings and discussion are
combined in Chapter 5, where we first talk about how clinicians’ understand and
think about the domain, and then give examples of where the tool supported this.
We discuss the limitations of the study and the tool in Chapter 6, along with pos-
sible areas for further investigation. Finally, we draw conclusions in Chapter 7.

11
2 BACKGROUND
This chapter provides an outline of the existing work relevant to clinicians making
sense of risk and visualising complex health data. Also, the motivations behind
using that work are given. Firstly, literature on sensemaking and existing models
are explored. Secondly, we look at how visualisation can play a role in making
sense of a domain. Finally, information visualisation design considerations and
techniques required to create the tool in this thesis are discussed.
Sensemaking is regarded as the process of information seeking and interpreta-
tion, it is about how people make sense of and understand a domain or topic, in
this case clinicians making sense of risk in a patient. Sensemaking research is
employed in a number of disciplines (e.g. decision-making, organisational re-
search), here we use it specifically to look at how the individual clinician makes
sense of risk. Klein, Moon, & Hoffman (2006) describe sensemaking in modern
research as a continuous effort to comprehend connections between individuals,
places, and events in order to anticipate their trajectories and act accordingly.
There have been a number of efforts to
formalise this process in sensemaking re-
search. In this section we will delve into
some of the leading theories in sense-
making and discuss their relevance to this
thesis as well as how looking through the
sensemaking lens might be beneficial in
understanding the problem domain.
Russell, Stefik, Pirolli, & Card (1993)
describe the sensemaking process
through what they call the Learning
Loop Complex model (Figure 1).
Figure 1 the Learning Loop Complex.
Image taken from Russell et al.
(1993).

12
This model follows a pattern where information in encoded in representations to
reduce the cost of operations. This involves four sensemaking phases:
 Search for good representations – representations are created to track
regularities that are significant to the sense maker. This is the generation loop.
 Instantiate representations – significant information is identified and encoded
in a suitable representation. Encodons are created in the data coverage loop.
 Shift representations – data (residue) that does not fit the existing schema
force a change in representations by moving up through the representational
shift loop, leading to merging, division and generation of schema.
 Consume encodons – encodons are used in task-specific information
processing.
The Learning Loop Complex model shows how sense makers use a top-down
(representation instantiation) and bottom-up (representation search) process to
form a mental model of a domain.
Pirolli & Card (2005) identify sensemaking in terms of two loops; the foraging and
sensemaking loops. The foraging loop includes three processes; exploring, en-
riching and exploiting. Exploring is about searching a space to gain new infor-
mation. In the setting of healthcare, this could mean retrieving doctor’s notes and
health record data. Through enriching, a clinician might order new investigations
or drill deeper into information to come up with a higher precision account of a
patient. Exploiting items in a set could mean going through patient information
and making inferences and detecting patterns. The sensemaking loop involves a
recurring process in which a mental model that matches the evidence is created.
However, these models do not take into account interaction effects that can occur
in sensemaking. Weick (1996) notes that schemas do not shift easily as residue
goes unnoticed by the sense maker. People to people interactions can cause
representational shift through exchange of ideas. In the context of this study, this
refers to how clinician-clinician and clinician-patient communication affect the un-
derstanding of risk in the patient. Healthcare is very much a human-centred do-
main where interactions affect the outcome. Sharma (2006) shows how theories

13
can be reconciled to provide a richer understanding of sensemaking. Interper-
sonal interactions help the sense maker notice residue, and consequently change
schemas. In addition, patients themselves can be the information source and col-
laborate with the clinician in the data coverage loop. This suggests that rather
than simply interpreting newly discovered data, sensemaking is about creation
and invention between the various actors.
Klein, Phillips,
Rall, & Peluso
(2007) propose
a sensemaking
theory, which
successfully
condenses the
characteristics of
the previously
discussed models.
Data-frame theory (Figure 2)
states that the sense maker
places data into a frame about
what that data represents; pre-existing
frames (results of previous experiences) influence how the new data is framed.
Three cycles make up the process of sensemaking; elaborating, preserving and
reframing. There are a total of seven steps in the data-frame model:
 Data and frame connection –data set is connected to a frame.
 Questioning the frame – unexpected or surprising data is encountered and a
frame is questioned.
 Elaborating a frame – a frame is elaborated but not changed due to new data.
 Preserving the frame – data is disregarded or ignored, preserving the frame.
 Seeking a frame – recalling or constructing a fitting frame.
 Comparing multiple frames – numerous frames are compared.
 Reframing – a frame is either replaced or combined with another.
Figure 2 Data-frame theory of sensemaking.
Redrawn from Klein, Phillips, Rall,
& Peluso (2007).

14
Regardless of whether we talk about schemas, or frames, they both refer to the
way that individuals subjectively look at, filter, and sort the data that they encoun-
ter. A number of questions can be raised when making sense of how clinicians
think about risk in a patient. How do clinicians deal with inconsistencies and
anomalies in data? How do they judge the plausibility and quality of data? When
do clinicians seek and infer or disregard data? How do they seek and infer new
relationships in data? In later chapters, we will attempt to explain our findings
through the sensemaking lens by using the data-frame theory as a framework for
understanding how clinicians think about complex data and risk.
As we can see from exploring these models, sensemaking is a cyclical and itera-
tive process where data is collected and assimilated into pre-existing frames, or
frames are modified based on previous experiences. Sensemaking is about gen-
erating new internal frameworks based on new data. In the context of this thesis,
these sensemaking models provide a way to explain how clinicians deal with the
complex data that they are presented with. The visualisations created for this the-
sis are informed by sensemaking literature and focus on supporting the pro-
cesses clinicians go through when making sense of risk.
There are a number of existing applications of visualisation techniques in
healthcare. Rind et al. (2013) explore effective ways of visualising electronic
health records. Bui & Hsu (2010) discuss systems for adaptive visual interfaces
that integrate clinical information necessary to users’ aims. Faiola & Hillier (2006)
show how complex clinical datasets can be transformed into contextual
knowledge using visualisations, improving the quality of clinical decision-making
and decreasing the time wasted foraging for information by organising it in a con-
text-related format in a single location. Others have looked at the applications of
visualisation for classification and assessment of risk in chronic heart disease
(Harle, Neill, & Padman, 2012) and diabetes (Harle, Neill, & Padman, 2008).
However, when it comes to research into visualisation for healthcare, few studies
look through the sensemaking lens.

15
Faisal, Blandford, & Potts (2013) identify potential ways that information visuali-
sation can assist both clinicians and patients in making sense of health data, but
conclude that more work needs to be done in order to incorporate the sensemak-
ing processes into the design of these tools.
Each person makes sense in his or her own way; sensemaking does not occur
externally, but by definition, inside the mind of the user. As users engage with vis-
ual representations, they also interact with the interface itself, in order to do so
they rely on the mental model that they develop (Sarah Faisal, Cairns, & Bland-
ford, 2007). This internal creation of concepts happens through interaction with
the external world.
Kirsh (2010) identifies a number of ways external representations help sense-
making and allow us to ‘think more powerfully’. As described in the cyclical pro-
cesses of sensemaking theory, when one experiences externalisations, the inter-
nal conceptualisations of a domain are generated, updated and used (Russell et
al., 1993). These externalisations can be in the form of visualisations. Spence
(2007) recognises visualisation as a cognitive activity; when designed well, visu-
alisations can amplify cognition and in turn amplify the sensemaking process
(Card et al., 1999).
Card et al. (1999) state that the purpose of visualisation is insight, as opposed to
just being ‘pictures’ to look at. Insights can be gained when data is represented in
a visual manner, thus supporting the user through visual sense making. Pirolli &
Card (2005) talk about insight being engrained in the sensemaking tasks; infor-
mation gathering, re-representation of data in schema, creation of insight through
manipulating representations, creating a knowledge product or direct action. In-
sight is but a single step in the sensemaking process, but sensemaking may not
be the only way to gain insight (Yi, Kang, Stasko, & Jacko, 2008).

16
Yi, Kang, Stasko, & Jacko (2008) propose four ways in which users gain insight
through information visualisation. Firstly, provide overview is about understanding
the big picture; it informs the user of what is known and what is not known about
a data set. Although it is not directly related to gaining insight, it leads to an un-
derstanding of what parts need further investigation. Adjust is about changing the
level of abstraction or range of selection, this can be done by filtering or grouping.
Detect pattern is about finding trends, relationships, outliers etc. During this pro-
cess users may not only discover what they were looking for but also discover the
unexpected. Match mental model is about decreasing the gap between the data
and the mental model (Johnson-Laird, 1983) of the user, thus reducing cognitive
load.
These processes are not separate and can be used together to gain insight, they
are cyclical and iterative, much like sensemaking. These processes are relevant
to the design of visualisations in this thesis; however, we found no concrete
guidelines in the literature for designing visualisations to provide users with in-
sight. Insight is a qualitative process (Saraiya, North, Lam, & Duca, 2006) making
it well suited to exploration with the methods used in this study.
The previous sections in this chapter focused on sensemaking theory and the
way in which visualisations can move away from simply communicating known
insights in the data toward an exploratory process of iterative understanding that
supports the sensemaking process.
This section will give an overview of information visualisation principles and ap-
propriate concepts from pedagogues of information visualisation (e.g. Mazza,
2004; Spence, 2014) that were used in the design of visualisation in this thesis.

17
The data that clinicians deal with comes from plethora of sources; verbally re-
ported data, sensors, health monitors, clinical tests etc. Card et al. (1999) men-
tion a number of points to consider before information visualisation visual repre-
sentations of data can be made:
 Data measurements – Nominal data is categorically discrete data such as
(e.g. behavioural, genetic, social, demographic). Ordinal data has a natural
ordering but the intervals between values are not the same (e.g. high, me-
dium, low risk). Interval data is numerical data (e.g. integers or real numbers).
 Data dimensions – univariate (1 dimension), bivariate (2 dimensions), trivari-
ate (3 dimensions), and multivariate (4 or more dimensions).
 Data structure – linear (made up of arrays, tables, lists etc.) temporal, spatial
or geographic (maps), hierarchical (taxonomies, genealogies etc.), network
(graph structures).
 Interaction type – static (print), transformable (user can manipulate)
Visualisations can be an effective way of representing information if designed
well. People assimilate information much more rapidly through visualisations than
they do through text (Ware, 2013). This section will cover the ways in which vis-
ual elements can be used to facilitate this.
Visual variables create mappings and structures; these should pull out interesting
features from the data. It is possible to take advantage of pre-attentive pro-
cessing to design effective visualisations. In this case, defined as the term as-
signed to objects that are processed faster than 10ms (Treisman, 1998):
 Form – line direction, size, curvature, grouping, marking, and luminosity
 Colour – hue and intensity
 Motion – flicker and direction of motion
 Spatial position – position, stereo-depth, convexity and concavity

18
Figure 3 Visual types. Image by
Krygier & Wood (2005).
Bertin (2010) identified attributes
that he called ‘retinal variables’
in his 1967 work, Sémiologie
Graphique. Each of these
variables were identified as best
used to show either or both
quantitative and qualitative data.
Krygier & Wood (2005) expanded
on these ‘retinal variables’ – size,
colour value, texture, orientation,
and shape – by representing
them in points, lines, and
areas (Figure 3).
Visual properties refer to the way in which we are able to create differentiation in
the visualisation and effectively show representations. Fry (2004) identifies con-
trast as the most fundamental visual property. Gestalt principles (Wagemans et
al., 2012) explain how we notice visual elements as being contrasting. Pre-atten-
tive features are all ways to differentiate or contrast visual elements. Hierarchy is
about the order of importance of elements; visualisations should emphasise ele-
ments important to the task and de-emphasise those that are not, this can be
achieved through creating a hierarchy. Grouping is about clustering elements to
imply a relationship or shared meaning. Grouping creates patterns; dissimilar ele-
ments that are grouped together can also highlight differences or contrast.
Weight of elements such as the size or thickness of lines can show relative im-
portance or differentiation. Prominence should always be on ‘showing the data’
(Tufte, 1995), distracting with design runs the risk of data representations being
missed. Use of borders must be carefully thought out as not to increase the
amount of ‘non-data ink’.

19
Figure 4 Hue,
value, and
saturation.
On a screen, colour is represented by a combination of red, green, and blue.
When referring to colour, the model of hue, brightness, and value
is better understood by the human mind (Figure 4). The hue
is what would usually be meant when colour
is mentioned (for example green or
magenta) value is the range
of black to white, and
saturation is the intensity
of the colour. Colour is useful
for contrast and mapping data.
Placement conveys hierarchy by ordering elements. Contrast can be shown
when an outlier is placed away from a group of similar elements. Grouping is the
principal use of placement.
A problem with displaying complex data is that it cannot be easily displayed in
one view. The user must be able to transform the view in order to use externali-
sations to forage for information.
A number of taxonomies have been proposed by researchers such as ‘overview,
zoom, filter, details-on-demand, relate, history and extract’ from Shneiderman
(1996), and ‘zoom, pan, scroll, focus+context and magic lens’ by Spence (2014).
These taxonomies describe low-level interaction techniques. We refer to the in-
teraction techniques proposed by Shneiderman (1996) when we talk about con-
crete operations in the visualisation that do not imply the cognitive aspect of user
intent.

20
Yi, ah Kang, Stasko, & Jacko (2007) present a taxonomy based on user intent, or
what the user aims to do by interacting with the system, thus adding a cognitive
dimension to interaction. In this thesis, we adopt this taxonomy to refer to user in-
tent and the user tasks the visualisations were intended to support. The following
are the seven user intent interaction techniques proposed by Yi, ah Kang,
Stasko, & Jacko (2007):
 Select – mark something as interesting
 Explore – show me something else
 Reconfigure – show me a different arrangement
 Encode – show me a different representation
 Abstract/Elaborate – show me more or less detail
 Filter – show me something conditionally
 Connect – show me related items’
Armed with a toolbox of sensemaking and information visualisation knowledge
we can move toward creating a tool that supports the clinician in understanding
risk in a patient.

21
3 METHODS
This chapter describes the participants and recruitment, apparatus and materials,
data gathering methods, and ethical considerations of the study.
The study was comprised of a total sample of 10 participants. Eight of whom
were male and two were female. All participants were doctors from primary or
secondary care. Participants were either general practitioners or specialists. Nine
participants were from the UK and one participant was from the USA.
The study was divided into three stages; six participants took part in Stage 2 and
ten participants took part in Stage 3 (including the six from Stage 2). Sessions
were performed face-to-face (F2F) where possible, but some sessions had to be
performed remotely for pragmatic reasons.
Participant summary:
 Number of participants: 10 (8 male, 2 female)
 Inclusion criteria: clinicians in primary or secondary healthcare who need
to about ‘risk’ of developing diseases when dealing with patients
 Demographic: 9 United Kingdom, 1 United States of America
Purposive sampling (Jupp, 2006), a form of non-probability sampling, was the
main method for recruitment. A variety of specialists and general practitioners
were selected as the sample of they matched the inclusion criteria. A range of
specialists and general practitioners were chosen in order to gain an understand-
ing of the similarities and differences in the way various clinicians think about risk.
Recruitment was carried out through email and word of mouth using industry and
academic connections.

22
Figure 5 (A) Lenovo Thinkpad
with Stage 1 Visualisation 1 on
screen. (B) Samsung 700T
with Stage 2 visualisation
on screen.
The following software, hardware, data gathering tools, printouts & documents
were used in the study:
 Adobe Creative Cloud for creating designs; Stage 1 & Stage 2
 Axure RP for creating the Stage 3 visualisation
 Camtasia Studio® for capturing audio and screen activity
 Voice recorder as a back-up for capturing audio
 Lenovo Thinkpad 3680K16 laptop, Windows 8 64-bit, Intel® Core™ i5 CPU
M540 2.53GHz, 4096MB RAM, 1280 x 800 resolution, seen in Figure 5(A)
 Samsung 700T tablet, 1366 x 768, seen in Figure 5(B)
 10 x Participant information (Appendix A)
 10 x Consent form (Appendix B)
 2 x Interview plan: 1 x Stage 2 (Appendix C) & 1 x Stage 3 (Appendix D)
 Notebook for taking notes during interviews and think-aloud
 Pink and yellow Post-it® notes for creating the affinity diagram
 Coloured pens for colour-coding the affinity diagram
A B

23
A semi-structured interview (Gillham, 2005) format was used – rather than struc-
tured or open interviews – in order to strike a balance between structure and
openness. This enabled a wide scope of questioning whilst remaining on topic.
This method was useful for both gathering requirements for re-design of the visu-
alisations and understanding how clinicians think about risk using complex data.
Semi-structured interviews made it possible to cover important questions while
also allowing for the pursuit of unanticipated themes as they arose. Interview
guides were used to guide the researcher’s line of questioning. Audio recording
of the semi-structured interviews was used to transcribe the interviews.
The purpose of the initial part of the interview was to make the participant com-
fortable, and learn about the clinical work they are involved in. This was useful to
understand the context of the problems they face, since participants came from a
variety of specialisations and medical fields. The middle of the interview was ded-
icated to constructing a deep understanding of participants’ work and sensemak-
ing activities. The aim of this part of the interview was to understand how clini-
cians make sense of complex health data in order to assess risk in a patient,
leading to a better informed understanding of how it might be possible to design
tools to support this. The end of the interview was used to bring up any lingering
points that the participants felt had not been covered.
Focus was always on framing the questions in real-life incidents that the partici-
pants had encountered, they were encouraged to talk about specific incidents ra-
ther than the general. The interviews were also a good way of identifying possible
biases within the sample (e.g. differences between medical fields), helping to mit-
igate those biases when analysing the data.
Nevertheless, interviews do not always elicit all interesting information from par-
ticipants; things that are obvious to the participant but not to the researcher may
be overlooked and remain unmentioned, therefore it was also beneficial to use

24
think-aloud protocol. Conversely, participants will not mention every part of their
thinking during the think-aloud either because they do not think it is important, or
because they are not consciously aware of the particulars of their thought pro-
cesses. Interviews are a good way to extract the information that might be omit-
ted in the think-aloud session.
Think-aloud protocol (Boren & Ramey, 2000) was used in Stages 2 and 3 of the
study, whereby participants verbalised their thoughts as they completed a task.
Verbalising their thoughts helped to highlight differences in the user’s mental
model and the system image. Think-aloud data contributed to the iterative devel-
opment of the visualisations used in the study by fixing the limitations of the pre-
ceding designs (Ericsson & Simon, 1993).
Before the session started, participants were provided with detailed instructions
of how to think aloud and were encouraged to speak freely as they noticed things
in the visualisation. Whenever participants stopped thinking aloud, due to becom-
ing too involved in the task or forgetting to keep reporting verbally, they were
prompted (by the moderator) to continue. Care was taken to remain sensitive to
when the participant needed prompting, to prevent interruptions in the partici-
pants’ thought processes.
Unnecessary questioning was avoided, as users who are asked for information
about something they are not attending to in the think-aloud are forced to infer ra-
ther than recall their mental processes (Anders & Simon, 1980), leading to inac-
curate reporting.
The advantage of having participants verbalise their thoughts over merely ob-
serving their activity, was that it enabled the articulation of their understanding of
the activity.

25
A risk assessment was completed and ethical approval for this study was gained
through University College London Interaction Centre.
Informed consent was given by all participants after they read the participant
information and signed the consent form. The participant kept one signed copy of
the consent form and the researcher kept another. Participants were all healthy
adults and did not belong to vulnerable or dependent groups.
The study adheres to the Data Protection Act 1998. Data was gathered with
consent, kept confidentially and securely. All participant data was anonymised
and made unidentifiable in reports and other shared materials.

27
4 DESIGN, STUDY & ANALYSIS
This chapter presents a chronological account of the steps taken to
generate the results of this thesis. We describe the details of the ob-
jectives, study design used with participants, visualisation design,
analysis, and outcomes of each stage. The motivations behind the
approach taken, as well as reflections on the strengths and weak-
nesses of those approaches are also presented throughout.
To begin with, we cover the development of the patient personas for
use in the visualisations. Then, the three stages of the study are pre-
sented. Firstly, the iterative design and evaluation of the two visuali-
sations in Stage 1 are described. Secondly, we explain the parallel
design approach to the two Stage 2 visualisations, the interview and
think-aloud procedure with participants 1-6, and then Stage 2 analysis
& requirements generation. Thirdly, an account of the final design of
the single Stage 3 visualisation, the study protocol with participants 1-
10, and Stage 3 analysis is given.
Three patient personas were created for the purposes of the study.
Unique patient information was generated for each of them. These
personas were not based on real patients or real patient data, the pa-
tient persona information was synthesised from research into particu-
lar conditions.
The personas were created with characteristics that would not clearly
place them at exceptionally high or low risk of a condition, this was
done in the hope that it would tease out the way in which participants
thought about risk.

28
Figure 6 - Patient
personas for use
in the think aloud
scenario and to
populate the tool
with data.
In order to verify the personas, we consulted with a nurse in the
healthcare industry to review them; the feedback was used to update
and improve the persona data to be more representative of a ‘typical
patient’.
Persona A was Diego Blanco, a 35-year-old male with potential type
2 diabetes risk. Persona B was Deirdre Maguire, a 64-year-old female
with potential melanoma risk. Persona C was John Smith, a 75-year-
old male with potential lung cancer risk. These personas had thirty or
more pieces of patient information each. Each piece of patient infor-
mation (ethnicity, BMI, diet etc.) was placed into a category (social
history, clinical stats, behavioural etc.). The information was then as-
signed a value (Hispanic, 29, high calorie etc.), metadata (eats out
with clients, low activity due to family life etc.), and risk severity (gen-
eral patient information, reduced risk, low risk, high risk etc.). The pro-
file photographs and information for patient personas can be seen in
Figure 6.
There were two uses for the personas; first, the data set of each per-
sona was used to populate the visualisations with data, and second,
the personas were used in the think-aloud sessions to introduce the
scenario and the task where participants evaluated a patient persona
for risk of developing a specific condition in the near or distant future.

29
The aim of Stage 1 was to use the persona data to produce a concept
design that informed the visualisations in Stage 2. An iterative ap-
proach (Nielsen, 1993) was adopted to improve the Stage 1 visualisa-
tions. Two visualisations were created in Stage 1. Multipage PDF
documents were created as the artefacts for each of the two visuali-
sations in Stage 1. Each page of the PDF had one screen of the inter-
face on. Both Stage 1 visualisations relied on existing information vis-
ualisation literature (Section 2.2.2) as a reference for design. Expert
evaluation with an information visualisation professional was used to
evaluate and provide recommendations for the next iteration.
The first Stage 1 visualisation consisted of two views; Compound
View and Category View. A risk severity number on an interval scale
of 1-8 was assigned to each risk factor; the designer assigned this ar-
bitrarily, but it can assumed that real software would use relative risk
from population studies to assign severity. Patient information was
plotted in circles along the x-axis according to their risk severity. A
search bar was present in both views to perform a query on the da-
taset to reduce the amount of data that is visible on screen. For ex-
ample, filtering by category or individual risk factor. Clicking on an in-
dividual circle would bring up information associated with that piece of
patient data, thus providing details-on-demand to the user upon re-
quest.
The Compound View, illustrated in Figure 7(A), gave an overview of
the whole dataset. In this view, the circles were pushed outward on
the y-axis subject to the amount of other circles already in that area,
making the concentration of circles larger with the intention of making

30
Figure 7 Stage 1 iteration 1.
(A) Compound View.
(B) Category View.
the distribution of risk more apparent pre-attentively. An overall risk
value was assigned based on the average distribution of circles. The
circle colour was mapped to patient information categories. A different
set of colours was also assigned to risk factors that were changeable
(can change) and risk factors that that were not changeable (can not
change) through intervention; a key was placed in the bottom right
hand side to act as a reference.
In the Category View, illustrated in Figure 7(B), the categories (ge-
netic, medical history etc.) were divided along the y-axis and as-
signed to lines along the x-axis. The average risk for categories was
displayed, showing the distribution of risk within individual categories.

31
The second Stage 1 visualisation (Figure 8) mapped changeable/non-
changeable to shapes in order to decrease the amount of colours
used; circles showed ‘can change’ whilst triangles showed ‘can not
change’. The 1-8 interval scale was divided into an ordinal scale of
protective, neutral, and low, medium, med/high and high risk. Filters
were added to show what the patient could change and what the clini-
cian could change through intervention, allowing the user to adjust.
Figure 8 Stage 1 iteration 2.
Category names and colours
updated from previous version.
Filters added to search types
of data. Triangles and circles
differentiate between ‘can’ and
‘can not change’. Risk has
been split into protective,
neutral & low-high relative risk
for each data point.
(A) Compound View.
(B) Category View.

32
By filtering, the user was able to see something conditionally, for ex-
ample ‘show me things that the patient ‘can change’ that have been
proven to influence the risk of type 2 diabetes’.
Expert evaluation was practical because it could be done at any time
and with minimal resources, providing a satisfactory cost-benefit ratio
(Nielsen, 1994). In contrast, issues can be missed (validity) and differ-
ent experts can find different issues (reliability). A data visualisation
expert was consulted and a ‘simplified think-aloud’ was carried out to
identify and provide suggestions for re-designs. This type of evalua-
tion is no replacement for real users; this method was used in order to
resolve basic usability and design issues before sessions with real
participants, so that the focus of testing would be on how the visuali-
sation supports clinicians’ sensemaking. Below are some examples of
the issues that were identified through the expert evaluation:
Categories are a nominal type of data. In the visualisations, the mis-
take of representing the data in a way that implicitly suggested an or-
der to it was made. The colour choices in both Stage 1 visualisations
were sequential in nature, using diverging schemes of colour led to
confusion. Using progressive variations transitioning between hues
suggested continuity, something that is not present in the categorical
or nominal data (Silva, Sousa Santos, & Madeira, 2011).
Since we assumed that actionable data is what makes clinical inter-
vention possible, we wanted to see if visualising the factors that could
be changed through intervention would support clinicians thinking
about risk. The first visualisation used colour to differentiate changea-
ble factors, while the second visualisation used shape to represent
the same thing. Although the latter was more effective, it was not use-
ful for clearly seeing how the risk is weighted among changeable and
non-changeable factors.

33
Table 1 Visualisations and patient
personas matrix showing the six
variations.
Horsky et al. (2012) state that ‘poor usability is one of the core barri-
ers to adoption and a deterrent’ to use of clinical decision support
systems. Upon reflection, using the iterations in Stage 1 to identify
and fix usability and design problems provided a good foundation for
designing better visualisations in the sessions with real participants.
Following Stage 1 visualisations, work begun on Stage 2 visualisa-
tions where a parallel design approach (Nielsen & Faber, 1996) was
used. The rationale behind using parallel design was that less time
was required to explore designs than if they were produced sequen-
tially. Parallel design was useful for testing and comparing visualisa-
tion types, presenting the same data set in different visual structures.
Two designers worked simultaneously in this stage; each worked in-
dependently on different visualisations. The Principal Investigator
worked on Visualisation 1 and a different designer worked on Visuali-
sation 2. Two separate designs were created for Stage 2; the three
personas and their data were used to populate each of the designs.
As before, the designs were exported to multipage PDFs, this time
the PDF documents were created with interactivity in the form of click-
able parts of the interface that were hyperlinked to other parts of the
document. A total of six variations were created; these can be seen in
Table 1. The artefacts created in Stage 2 were used on a laptop with
participants during the think-aloud part of the study protocol.
Persona A Persona B Persona C
Visualisation 1 1A 1B 1C
Visualisation 2 2A 2B 2C

34
In Visualisation 1, illustrated in Figure 9, the colour scheme for cate-
gories was changed in an attempt to avoid implying magnitude differ-
ences between categories. Differences in hue with only slight differ-
ences in the lightness were used to differentiate categories, but using
nine colours to represent the categorical data made it hard to discrim-
inate between categories. As a consequence, the ability for the user
to memorise the meaning of each block in the visualisation was di-
minished, MacDonald (1999) suggests using seven or less colours to
show data of this kind. In Stage 2 Visualisations, the name of the per-
sona was added to the title, along with the condition they were sus-
pected of being at risk of. In addition, a frame at the bottom-centre of
the screen was added with the patient name, photograph and dummy
text for notes on the patient.
Figure 9 Stage 2
Visualisation 1 with
Compound View
selected.

35
Figure 10 Compound View
with ‘early osteoporosis’
selected in the main
visualisation, thus
changing the bottom-
centre frame content to an
image of the patient’s
DEXA scan with notes.
The content of this frame changed as the visualisation was interacted
with, as can be seen in the Compound View (Figure 10). The selected
block was highlighted with an orange outline and a line connected the
selected block to the bottom-centre frame, inferring a relationship. We
also included a maximise button in the top right hand of the frame to
expand the window and zoom into the data subset. This interactivity
was also present in in the Category View, illustrated in Figure 11.
In the Compound View, each item of patient data was represented by
a solid block of a fixed size. These blocks stacked upon each other
additively in columns out from the central line that divided factors that
‘can’ and ‘can not change’. This allowed spatial grouping to be used,
instead of colour or shape, to differentiate between the entities ‘can
change’ and ‘can not change’. This made it possible to distinguish the

36
Figure 11 Category
View with ‘nevi checked’
selected, notes and an
image of the patient’s
nevi is displayed in the
bottom-centre frame.
weighting of risk in each risk severity group through the height of the
block stacks from the distance the stacks protruded from the
mid-section.
In the Category view, ‘can change’ and ‘cannot change’ were still rep-
resented with shape. However, instead of using triangles and circles,
as was the case in the second Stage 1 visualisation, squares repre-
sented ‘cannot change’ and squares with rounded edges represented
‘can change’. The user is able to encode the data in a different repre-
sentation by shifting views from Compound view to Category view
and vice versa.

37
Figure 12 Visualisation 2.
Three columns (left to
right) show factors that
reduce risk, patient
information and factors
that increase risk.
In Stage 2 Visualisation 2, illustrated in Figure 12, three separate col-
umns were used to visualise data. The central column contained the
total information that was available for that patient. The left column
represented factors that reduce risk and the right column represented
factors that implied increased risk for the condition being assessed.
In Visualisation 1, the severity of risk of an individual data point
relative to a condition was shown by separating data into columns;
low, medium and high risk. In Visualisation 2, the severity of risk was
represented by the size of a block in the ‘Increase Risk’ column. Lines
were drawn from the central ‘Patient Information column to show and
infer relationships between factors that reduced, increased, or had no
direct correlation to risk.

38
Figure 13 Individual data point (A) Exposure to
radon. Map from ukradon.org (B) Lung X-rays.
Clicking on a data point would enlarge the
patient information bar and move it to the
left hand side of the screen and let the
user look closer at the data subset for that
patient data (Figure 13); what Tufte (1995)
would refer to as a micro view or what
Shneiderman (1996) refers to as zoom.
Zooming added deeper contextual text to
the enlarged patient information column
for each data point. A red line on the right
side of a data point in the patient
information column meant that it was a
risk factor, while a green line on the left of
the data point meant it was a risk reducing
factor.
The additional properties related to a data
point were displayed on the right hand
side of the zoomed in patient information
bar. Some of the screens displaying
zoomed in patient information, such as in
Figure 13(A), had the design error of
heavy use of thick lines as borders in the
design placing less prominence on
‘showing the data’ (Tufte, 1995). This led
to users becoming distracted with the
design rather than with the actual data the
design was trying to display; excluding
this from the design would have reduced
the amount of ‘non-data ink’.
A
B

39
In advance of the actual study, the protocol was piloted with a USA
based doctor in the healthcare industry in order to modify and im-
prove interview questions and think-aloud procedures. The feedback
from the pilot session has not been included in the sample. Prior to
conducting the sessions, participant information and consent form
documents were sent in an email to participants 1-6 (Table 2). The
body of the email confirmed the time and place of the interview (con-
ference call details if session was remote).
Upon commencement of the session, the Principal Investigator – ac-
companied by his Industry Supervisor – asked participants if they had
any questions about the study and if they understood everything in
the participant information. Upon confirmation that the terms of the
study were understood, consent forms were signed and collected (a
signed and scanned version of consent form for remote sessions).
Participants were given a short introduction to the procedure that
would follow; firstly, an interview involving current workflow, decision
making processes, the actionability of data, how they make sense of
the information they deal with, the various sources of data, trustwor-
thiness of data and communicating risk to others. Secondly, a think-
aloud session that would involve looking at two different visualisa-
tions, where the participants would explore a patients risk of develop-
ing a condition.
The researcher then started audio recording on the voice recorder,
and audio & screen recording on the laptop using Camtasia Studio®.
The researcher also took notes using a notebook and pen during both
interview and think-aloud.
The semi-structured interview, described in Section 3.2.1.1, took
place. The researcher asked the main questions and followed up with

40
Table 2 The column ‘Vis’ shows the visualisation order and patient persona used during the think-aloud. For
example, participant 3 saw visualisation 1 with patient persona A followed by visualisation 2 with persona C.
additional questions to probe further. Clarifying questions were asked
when a point was unclear or confirm of what a participant meant want
needed. The interview took around 30 minutes per participant.
Once the interview part of the session was over, the researcher ex-
plained think-aloud protocol (Section 3.2.1.2). Participants were told
that they would explore two visualisations representing complex
health data about a patient, and were asked to talk about what they
understood from the interface and how it might relate to their task of
assessing risk. The scenario was explained; the hypothetical patient
in question was sitting outside in the waiting room and that this was
the first time that the participant was viewing the patient’s data. Their
task was to discern whether the patient was at risk of developing a
condition based on what was understood from the interface. The
think-aloud session took around 30 minutes per participant. Around
15 minutes was spent thinking aloud about each visualisation.
Upon completing the interview and think-aloud the session was con-
cluded; participants were de-briefed and asked if they would be will-
ing to be re-contacted for Stage 3 of the study. Once all six Stage 2
sessions had been completed, analysis took place.
Participant Gender Medical field Location Vis F2F
P1 M Cardiology UK V1A, V2A Yes
P2 M Cancer genomics USA V2B, V1A No
P3 M Clinical Pharmacology & General Medicine UK V1A, V2C Yes
P4 M Psychiatry UK V2B, V1B No
P5 M General Practitioner UK V1B, V2C Yes
P6 M Paediatric Pathology UK V2C, V1C Yes

41
This section describes the approach to analysis, findings from the
Stage 2 study with participants 1-6, and requirements statements.
The main purpose of analysing data after the Stage 2 study was to in-
form the design of the Stage 3 visualisation. However, findings from
Stage 2 were also integrated into results in Sections 5.1-5.4.
To begin Stage 2 analysis, all interview and think-aloud data was
transcribed word-for-word using the audio from the voice recorder.
When it was not clear which part of the interface a participant was
talking about in the think-aloud recordings, the screen recording was
used as a reference.
Interviews were annotated in a word processor with approximate
codes. Recurring patterns in the way clinicians think about risk, prob-
lems they face, work practices, work environment, and attitudes to-
ward data helped to form initial interpretations.
The majority of analysis in Stage 2 was on the think-aloud data. The
think-aloud transcripts were printed onto A4 paper and a highlighter
was used to mark substantive statements for each participant.
Post-it® notes were then attached to these substantive statements,
the theme of the statement was summarised and an identifying code
was written on the Post-it® to recognise where the data came from in
the transcript. The letters and numbers in square brackets in Section
4.3.5.1 follow the same order as these codes; participant number, vis-
ualisation number, patient persona. For ease of visual differentiation,
Pink Post-it® notes were used for visualisation 1 and yellow Post-it®
notes were used for yellow visualisation 2.

42
The following pattern was used to identify where data came from:
Once all of the Post-it® noted has been added to the transcript high-
lights, an affinity diagram (Hartson & Pyla, 2012) was used to aid
analysis (Figure 14).
Large sheets of A1 paper were taped together and the Post-it® notes
were taken from the transcripts, placed in clusters with similar
themes, and given topical labels on the A1 paper. These clusters
soon formed groups within hierarchies; these were all labelled. Even-
tually a structure began to emerge.
Figure 14 Affinity
diagram created
during analysis.
42

43
Through the creation of the affinity diagram, both interface specific
observations and abstract findings about clinicians needs began to
emerge:
Although participants said Visualisation 1 was a tool with a structured
presentation [P1V1A], they had to deal with various data at the same
time [P1V1A].
There were also usability issues; for example, a participant was con-
fused about the horizontal relationship of data points in the Com-
pound View, even though there was no meaningful relationship in-
tended in the design of the visualisation [P4V1B].
It was noted that this tool might be good for a specialist consultation
[P1V1A] rather than for a busy general practitioner, since it took a
while to digest all of the information on the screen.
43

44
Visualisation 1 had too much data to process at once, leading to par-
ticipants asking for more ‘black and white’ and ‘yes and no’ [P1V1A].
In Visualisation 2, participants said that the interface was putting all
variables for assessment in one place [P6V2C], which gave a nice
sense of the data collected and how it influences risk up or down
[P2V1B]. The simpler information structure of the overview in Visuali-
sation 2 helped participants to understand the three columns
[P2V2B]. However, it was not always apparent that the columns were
not equivalent; the size of the data points relating to severity of risk
was not a connection that was easily made [P4B2B], also having a
smaller pixel area for low risk factors made interaction a challenge
[P1V2A].
In both visualisations, users generally had trouble interpreting the sig-
nificance or meaning of category colours [P6V1C] [P6V2C] [P1V2A]
[P4V2B]. This led us to believe that using colour to differentiate be-
tween categories is not essential to the task of assessing risk. Using
colour that does not provide further insight for the user can be per-
plexing as they try to understand its meaning and, hence, should be
avoided (MacDonald, 1999).
The filters in both visualisations were not visually apparent to all users
[P4V2B] and the terminology was not well understood [P3V2C]. The
readability of Visualisation 2 was generally better than Visualisation 1;
only one participant noted the readability of the text was poor in Visu-
alisation 2 [P4V2B], in Visualisation 1 participants complained about
small boxes, end-of-line hyphenation, differentiation between square
and rounded edge squares & small text size [P1V1A] [P2V1A].
Although both visualisations succeeded in showing complex data
from disparate sources in one interface, an understanding of whether
the patient was at risk or not was missing. Information about how a
single data point correlates to risk (why it is placed in high, medium,

45
low or protective) [P6V2C] was not present. Adding context about
where a patient fits into a risk population would have helped.
Participants pointed out that an overall risk score was missing from
both visualisations, something the majority of participants noted
[P2V1B&2A] [P3V1A&2C] [P1V1A&2A] [P6V1C&2C]. Due to the vari-
ety of data represented in the visualisations, finding an existing risk
calculator or algorithm that considered all factors was not possible.
Nevertheless, including existing risk calculators to be applied on a
subset of data emerged as an option. In the end, risk is complex, but
in a clinician’s daily work, a summary is needed [P1V2A].
The think-aloud protocol used with the two Stage 2 visualisations
highlighted both strengths and weaknesses in their respective de-
signs. The interviews also revealed the way in which clinicians think
about trusting data, actionability, workflow, attitudes, understanding,
and communicating risk.
Following the completion of the affinity diagram, the designers dis-
cussed possible design ideas and any unanswered questions or
holes in the data that required further investigation in Stage 3. The
learnings were merged from both the think-aloud and semi-structured
interviews into a requirements statement document (Appendix E).
When moving from codes in the affinity diagram toward requirements,
focus was on matching the internal mental model of clinician through
externalisations. The rationale behind the requirements were dis-
cussed and design recommendations were made. Care was taken by
the designers to avoid bias toward their own competing design ideas
by looking objectively at the findings in the data when creating the
requirements document.

46
Table 3 Requirement
Statement structure.
The requirements statements follow the structure shown in Table 3. A
traffic light metaphor was used to show priority, red (top) being high-
est priority and green (bottom) being lowest. Textures have been re-
dundantly mapped to the colours for colour-blind readers.
Priority was assigned requirements according to the severity of the
problem, which was derived from, to how frequently the topic or con-
cept in question came up in interviews and think-aloud data and the
amount of design work required to implement the change.
An assortment of requirements statements have been taken out of
the ‘Requirements Statement’ document (Appendix E) to show
as examples. These can be found in Table 4.
Requirement Statement Priority
#: Name of feature/category
Second-level feature/category
Requirement statement [place in Affinity Diagram]
Rationale (if useful): Rationale behind requirement
Design recommendation (optional): Commentary about requirement

47
Table 4 Selected examples from
Requirement Statements
document
5. User input
Editing risk category
If there is no/low evidence for a data point, allow the user to assign a risk category. But, do not al-
low the strongly evidence based data points to be moved. Track all changes and show if an item
has been moved through the interface [information/supporting interpretation &
‘objective’ vs. ‘subjective’ data]
Rationale: ’Solid’ data has its limitations [P1V1A]. For example, occupational history is useful
[P5C1B], but interpretation is subjective. In clinical work evidence & subjective opinions are mixed,
this tool gives an objective view of both that can be reviewed [P6V1&2]
Design recommendation: The data points that aren’t used in validated scales can be re-assigned
to another risk category, but these changes must be tracked for later review
9. Individual data point view
Relative risk
Let the user know how a single data point correlates to risk [P6V2C] [risk type>relative risk]
Rationale: Relative risk is not known for all of the data points, but it would be helpful is it was avail-
able for those points that are known [P2V2B]
Design recommendation: Display relevant patient data on classic graphs & scales within the indi-
vidual data point view. Let user know the high/med/low in the overview is based on the relative risk
of that specific measurement in order to ground the perspective [P2V1A]
17. Overview
Overall risk calculation
The user needs an overall risk calculation that gives a quantitative measure [interface specific]
Rationale: Overall risk was missing from both visualisations, something the majority of participants
noted [P2V1&2] [P3V1&2] [P1V1&2] [P6V1&2]
Design recommendation: Individual conditions have their own risk calculators (i.e. risk of diabetes
in 5 years is X [P3V1A]). Include a risk calculator to display overall risk

48
After defining the requirements, the design and implementation of the
Stage 3 Visualisation (also referred to as ‘the tool’) started. This in-
volved merging the best parts of the two versions produced in Stage
2 into one design.
In order to do this, the two designers used the requirements state-
ment document created from Stage 2 learnings. In Stage 2, the de-
signers worked separately, but in Stage 3, they worked collaboratively
on the visualisation.
The merged design aimed to apply findings about how clinicians think
about and make sense of risk in a patient by creating tool that re-
duced the gap between the clinicians’ mental model and the data in
the externalisations that they currently work with.
Before the design work begun, personas ‘A’ and ‘B’ were improved
based on feedback from participants in Stage 2. Persona ‘C’ was not
included in Stage 3 design due to temporal constraints in the develop-
ment period; this persona was discarded because it produced the
least interesting data during the think-aloud sessions.
Two artefacts were produced; one visualisation populated with per-
sona ‘A’ data and another with persona ‘B’ data. The Stage 3 visuali-
sation was created in the rapid prototyping tool, Axure RP, and then
exported to HTML and JavaScript for use on a Samsung 700T tablet
in Stage 3 of the study.

49
Figure 15 The overview shows
the title bar, tool bar, patient
information, risk factors.
This section will discuss the design of each part of the tool and the ra-
tionale behind the design. The interface was interactive but some
parts such as ‘Go To Investigation’ were not functional. Data showed
that they were important to clinicians’ needs and understanding, but
beyond the scope of what could be implemented. They were included
in the interface to hint at what their functionality might be.
49

50
The overview screen (Figure 15) was made up of a number of parts;
the header, patient information and risk factors. In Shneiderman's
(1996) mantra, ‘overview first, zoom and filter, then details-on-de-
mand’, overview refers to the act of looking at ‘the big picture’. The
overview screen in the tool does this by presenting all of the patient
data in a macro view (Tufte, 1995). Spence (2007) speaks of over-
view as the ‘qualitative aspect of some data’ – in this case the pa-
tient’s risk of developing a condition – that is ideally ‘acquired rapidly
and, even better, pre-attentively’. This screen attempted to achieve
this by showing the clinician factors that are known to contribute to
risk, how severe they are, and whether those factors can be changed.
Since user attention is first attracted to visually strong (big, colourful,
prominently placed) objects, the design attempted to lead users’ gaze
towards high risk factors. The user can then search for details among
less prominent elements in patient information. This was key because
if information is not organised in an optimal manner overview (Yi et
al., 2008) clinicians could potentially be stuck in the foraging loop (Pi-
rolli & Card, 2005) for longer than necessary.
The header, illustrated in (Figure 16), contained the title bar, serving
as a reminder of the purpose of the tool. Under that, basic patient in-
formation and photograph helping identification of the patient, useful
when a clinician deals with many patients. On the bottom, the condi-
tion the patient was being assessed as being at risk for, risk calcula-
tor selection, search bar, and buttons to lead onto next steps (which
were beyond the scope of this thesis) within the clinical workflow.
Figure 16 Header at the top of
the tool. Different conditions
could be selected from ‘Risk of
developing’ and risk calculators
can be applied from the
dropdown ‘Apply calculator’.

51
Figure 18 Risk factors.
Figure 17 Patient Information.
Hierarchies and structures refer to elements within elements. They
also refer to an element that has a pointer to another element. In the
‘Patient Information’ (Figure 17) section of the overview the patient in-
formation was ordered into categories using lines to divide them. This
was designed in a structure that follows the clinical workflow that par-
ticipants described in Stage 2 interviews and think-aloud.
The coloured strips on the right hand side of the boxes relate to the
level of risk that piece of patient data is thought to have relative to an
empirical population study. All information available about a patient is
displayed here.
There is a danger that, if a part of the interface that contains less im-
portant data draws attention, features that are more important might
be overlooked. We tried to make sure that the most prominent fea-
tures of the interface were also the most significant parts of the data.
By placing all factors that are known to contribute directly to the risk
of a condition in one place (Figure 18) the clinician is able to conserve
mental resources that would otherwise be spent searching for risk
factors.
The location of elements also affects how a screen is viewed. For this
reason we grouped changeable factors separately from non-changea-
ble, indicating to the clinician which data is actionable. High risk fac-
tors are represented with highest saturation red and low risk factors
with the lowest saturation red.

52
Figure 19 If the risk
calculator runs when
information is missing,
a prompt appears
requesting missing
information.
Selecting a risk calculator from ‘Apply Calculator’ would bring up an
overlay that superimposed a yellow hue on the factors that the calcu-
lator took into account. As a reference, the NHS (National Health Ser-
vice, 2013) calculator was used with the persona information to cre-
ate a more realistic outcome. If patient information that the calculator
algorithm needed was missing, as shown in (Figure 19), the tool dis-
played an alert prompting the clinician to gather that information.
52

53
Figure 20 The completed
risk score calculation.
NHS Diabetes Risk
Calculator available from:
www.nhs.uk/Tools/Pages
/Diabetes.aspx.
The factors that were present and contributed to risk had a line drawn
across from ‘Patient Information’ to ‘Risk Factors’. By connecting two
elements, it is possible for a relationship between the two to be
shown. This feature was meant to allow the user to differentiate be-
tween factors taken into account by the calculator that did contribute
to risk and those that did not contribute to risk in the patient.
53

54
Figure 21 Individual
data point view.
The advantage to overlaying the cal-
culator on top of the interface is that it
is plain to see which factors are iden-
tified as contributing to risk by other
evidence sources, but not taken into
account by that particular calculator.
Providing all the required information
was present, the risk calculator ran
and displayed an overall risk score
(Figure 20). This was the absolute
risk of developing a condition within a
timeframe. A segmented coloured
scale as well as descriptive text rep-
resented this.
Clicking on a piece of patient
information on the overview screen
would lead the user to an individual
data point view (Figure 21). This view
effectively zoomed into the patient
information bar, showing more detail
about each data point on the left and
providing extensive properties about
the patient information that was
selected. The individual data point
view displayed various types of
information depending on the type of
data; risk severity, relative risk (with
embedded visualisations to show it),
and contextual information.

55
Figure 22 Examples of the
embedded visualisations
within the tool.
Individual data points had embedded visualisations in order to repre-
sent the patient data in a way that was easy to make sense of it. Only
data that was better represented visually was represented this way,
for example natural language notes were better displayed as textual
information. A number of the embedded visualisations are shown in
Figure 22. Examples of these representations included the following:
 Tables – these were used to show two-dimensional data. How-
ever tables are only useful when there are a limited number of en-
tries for the columns and rows, otherwise they get too crowded.
 Line graphs – these are a number of data points connected by
lines, showing continuity across the values. Line graphs were
used to show data such as hemoglobin A1c levels over time.
 Bar graph – were used to display series data where there
was no continuity between values. Bar graphs were used to show
data such as steps taken per day.
 Geographical map – these were used to show environmental ex-
posure and post code information.
 Matrix – wo dimensional sets such as the measures for BMI were
plotted in a matrix.
 Tree – used when hierarchically ordered data is used. This was
useful for showing family heart disease history.

56
The bottom of the individual data point view had a series of buttons:
Notes – these were added to allow easy access
to notes when searching for information about a
data point or allowing the clinician to add notes
when they gain insight.
Guidelines – these were guidelines that link
directly to the National Institute for Health and
Care Excellence. This was added to support
clinicians through guidelines, advice and
appraisals.
Evidence – these were added for the times that
guidelines do not apply. Clinicians are able to
search for available evidence and latest literature
concerning a risk factor.

57
More – there were three buttons in this submenu.
‘Go to Investigations’ and ‘Go to Diagnosis’ were
placeholder to hint at following phases in the clini-
cal workflow. As they were beyond the scope of
this work, they were not made interactive. Change
risk classification opened a pop-up window when
tapped.
Change risk classification – allowed the user to
flag a piece of patient information with a risk factor
(e.g. if the clinician finds out that the patient
spends too much time in the sun that contextual
factor can be flagged). Marking helps users make
sense of a domain on their own terms and track
their own developing understanding (Huang &
Eades, 2013).
Confirming the action – The risk classification
change is flagged for review if the user continues,
this acts as a safeguard for rogue actions. The
user is prompted before the change is submitted
for review; adding an extra step to confirm in an at-
tempt to minimise error.

58
Following the completion of the tool, Stage 3 sessions with partici-
pants 1-10 were scheduled. Participants 7-10 were sent participant
information and consent form documents attached to an email, since
they had not taken part in Stage 2. Participants 1-6 simply confirmed
that they were still willing to take part in the study. Table 5 shows the
participants, order in which they saw the personas, and whether the
session was F2F or remote.
The protocol for Stage 3 sessions followed the same structure as
Stage 2 sessions, but with one main difference; the think-aloud was
carried out before the semi-structured interview. Sessions lasted
around 60 minutes, however this time the think-aloud sessions took
approximately 45 minutes and the interviews took approximately 15
minutes.
This time, when the think-aloud was explained, the participants were
asked to perform a number of tasks that were designed to test the as-
sumptions made about the visualisation being able to support clini-
cians thinking about risk:
 Use risk calculator to see a risk summary for this patient.
 Look through patient information and explore some of the data.
 Flag a piece of social information as a risk factor because you
know (persona specific scenario) might be putting them at risk.
Participants thought aloud with persona A and persona B. About half
of the 45 minutes was spent viewing the visualisation with each pa-
tient persona.
The interview plan (Appendix D) in Stage 3 sessions was also fo-
cused on evaluating the tool and the way complex health data was
visualised, as well as confirming preliminary evidence from Stage 2
about how clinicians make sense of risk in the patients they deal with.

59
Participant Gender Medical field Location Vis F2F
P1 M Cardiology UK A, B Yes
P2 M Cancer genomics USA B, A No
P3 M Clinical Pharmacology & General Medicine UK B, A Yes
P4 M Psychiatry UK A, B Yes
P5 M General Practitioner UK A, B Yes
P6 M Paediatric Pathology UK A, B Yes
P7 F Ophthalmology (specializing in genetics) UK A, B Yes
P8 F Radiology UK B, A Yes
P9 M General Practitioner UK B, A Yes
P10 M General Practitioner UK B, A Yes
Table 5 Participants for Stage 3 study. ‘Vis’ shows the order that the patient personas were presented
to each participant during the think-aloud. For example, participant 10 saw the visualisation with
persona B followed by persona A.

60
A thematic analysis (Braun & Clarke, 2006) was used to identify, ana-
lyse and report themes in the data. This approach was used to give a
rich description, reflecting the predominant themes that arose within
the data set. The multi-stage approach to the study meant that Stage
2 was focused on understanding users’ needs and practices to yield
requirements for Stage 3 design, but the overall aims were to under-
stand how clinicians make sense of risk. As was the case with Stage
2 analysis, interviews and think-aloud sessions from Stage 3 were
transcribed word-for-word by the researcher. This helped with becom-
ing familiar with the data and forming initial interpretations (Riessman,
1993).
Learnings from Stage 2 analysis were carried over to Stage 3. The in-
itial themes and approximate codes were reviewed as new data was
incorporated from the ten Stage 3 sessions. Notes made by the Prin-
cipal Researcher during the sessions was also used to support the
other sources of data. Data from research notes, interviews and think-
aloud sessions was triangulated (Guion, Diehl, & McDonald, 2011),
increasing validity of results.
At first, notebooks and paper were used when the focus was more on
the exploration of the data. Preliminary codes were first added to the
transcriptions, these codes were sorted by recurrent patterns in the
data. This process was done in an iterative manner, themes were cre-
ated for similar codes, and themes were merged and adapted in light
of new data. The main tool for analysis then shifted to a word proces-
sor, making it easier to organise data in a proper structure. This was
done systematically whilst building up a narrative about each theme,
reviewing themes when inconsistences in the data emerged. This
was done iteratively until a clear narrative emerged within the data.
The findings from this process are presented in the next chapter.

61
5 RESULTS
This section contains the findings that arose from the data. Firstly, the main find-
ings are stated and discussed in Section 5.1. Sections 5.2 to 5.5 present and dis-
cuss secondary findings, which include the way in which clinicians systematically
organise their work, think about risk, arrive at a decision, and communicate risk.
Finally, Section 5.6 contains findings related to the tool in terms of its efficacy in
using visualisation to support clinicians dealing with complex health data.
The data collected from think-aloud and interviews and subsequent analysis
revealed two main findings:
I. Clinicians use a number of disparate sources of information to make sense of
risk in a patient – this entails complexity. There is a discrepancy between how
clinicians talk about their work, and what they actually do when assessing risk
in a patient.
II. We have promising preliminary evidence from this exploratory study that
tools, such as the one described in this thesis, can support clinicians by
creating externalisations that facilitate the implicit processes that they use
frequently in their work.
Clinicians routinely use complex data from a number of different places to try
to form a unified understanding of a patient’s risk. This diverse data is usually
presented in a way that makes the relationships between potentially significant
information difficult to perceive.
Our findings show that clinicians speak of the data that they use when assessing
risk as being ‘limited to validated clinical data’. They routinely use studies based
on specific populations and tools such as risk calculators. However, clinicians ac-
cept that research evidence is an abstracted generalisation that does not always
represent the actual risk of the individual patients they deal with. Patients do not

62
always fit into the population of the epidemiological studies that they are com-
pared to. In reality, the patient may be found anywhere within the vast landscape
of risk. Similarly, guidelines are a common point of reference; however, they are
not always sufficient, as there are always outliers that do not fit the typical case.
The difficulty arises when it comes to making sense of what all of this actually
means for the patient that the clinician is attempting to assess.
Our participants described how they use studies, calculators, and guidelines in
conjunction with patient data related to risk of a certain condition, in order to as-
certain whether or not an individual is at risk and what – if any – intervention
should be taken.
Current health policy and evidence based medicine (Straus, 2011) state that clini-
cal practice is a scientific discipline where the science base is derived from ra-
tional, universal, and objective evidence. The rationalistic attitude of only utilising
validated empirical evidence to assess risk is not completely reflected in the way
that clinicians seek and use information in practice. We found that clinicians use
empirical tools and validated data alongside contextual factors specific to the pa-
tient they are dealing with when assessing risk.
In terms of data-frame theory (Klein et al., 2007), the empirical tools present data
that cause a connection to an existing frame about what the risk score or sum-
mary means. The clinician questions the frame by asking whether these
measures apply to the patient in front of them. The frame may be preserved if the
evidence is adequate enough for the clinician to confidently say that the risk data
about the patient matches their existing frame. If there are remaining questions
about whether the patient is at risk, the frame may elaborated as clinicians seek
and infer data in the current context (specific to the individual patient), through
adding and filling slots to build up a more comprehensive frame of the current sit-
uation. These contextual factors are not currently accommodated for by existing
externalisations that come in the form of risk calculator algorithms, guidelines and
population studies.

63
Clinicians apply their internal knowledge, the unstated, that when applied usually
presents itself through skilful performance in order to gain insight. This is what
Schön (1983) refers to as reflection in action. As part of the decision-making pro-
cess, clinicians apply expertise, reflect on their old model, and change their think-
ing to fit the new task. This process is internalised and unique to the individual. In
the same way, Polanyi (1983) speaks of this phenomenon as tacit knowledge.
This leads us to believe that the sensemaking process is more complex than
practitioners are able to verbalise. Henry (2010) supports this finding by saying
appreciation of the tacit dimension of knowledge ‘will help clinicians to build a
more accurate critical framework for evaluating what kinds of information are im-
portant for particular clinical decisions’ (p.296).
When elaborating a frame, clinicians do not simply amass data and elaborate
frames based on best fit, for this would result in erroneous frames being pre-
served. Instead, past experience, clinical expertise, and critical thinking are used
to inform this process. Novices can end up relying on anchors that are not correct
leading distortions or flawed interpretations of what risk data means, leading to
preservation of incorrect frames (Sieck et al., 2007). More experienced clinicians
are able to rely on their larger knowledge base to avoid such pitfalls:
P6 That pattern [some aspect of patient data] can either be caused by only
one thing or it can be caused by 100 things, and that's what comes from
basically learning and doing the job.
Understanding the epistemological aspects of how clinicians think about the data
they use for decision-making and assessment of risk, as well as the environment
and workflow that these processes are embedded in was crucial to the develop-
ment of the tool.

64
Through testing the tool on participants, we have found preliminary evidence that
it supports clinicians by making the implicit (knowledge that they utilise but do not
directly express), explicit. This allowed participants to gain insight and infer rela-
tionships between data through use of the tool.
One participant verbalised the way in which the tool attempts to support the way
in which clinicians apply general rules to specific cases using their internal
knowledge:
P4 I suppose experienced clinicians will have these gestalts, you know. You
see someone and you recognize a certain sort of configuration of features,
and if you then focus on bringing out important risk factors in this patient…
and here's the evidence base behind it... You are facilitating that process.
Risk assessment sits deeply in the context of clinical workflow. This process is
shown in Figure 23. Of course, this is an oversimplification of what actually hap-
pens in clinical practice; however, we found that the general process remained
the same regardless of whether the participant was talking about primary/second-
ary preventative/reactive care. This may have resonated with opinion throughout
partly because it reflects the way in which medicine is taught.
The evidence based medicine process (Gronseth, Woodroffe, & Getchius, 2011)
is aligned with the desired clinical workflow that clinicians expressed. As we can
see, the process is comparable. However, as highlighted by Mynatt (2011), fac-
tors such as diet, activity levels, social conduct other types of intervention, are
generally not viewed in their relation to disease until it presents itself.
In this model, stages may be skipped depending on the availability and confi-
dence that the clinician has in the information they have to work with. For exam-
ple, a clinician may move from Assess Info to Diagnosis if there is strong enough
evidence, or may go from Investigation to Patient monitoring if a test comes back
negative.

65
Figure 23 Model of
desired clinical
workflow expressed
by participants which
aligns with the
evidence based
medicine process
adapted from
(Gronseth,
Woodroffe, &
Getchius, 2011)
The workflow described by participants is as follows:
 Assess info – reviewing existing information establishes a mental model of
the patient, there can be varying levels of data available to the clinician at this
point. This is where the tool developed for this study attempted to target the
needs of clinicians.
 Examination – gathering medical history and taking an account of the pre-
senting complaint through physical examination of the patient. The tool only
emulated this part of workflow where the user had to enter missing blood
pressure information for the patient during the think-aloud.
 Investigation – ordering or performing tests that are not part of the examina-
tion in order to find out something that the current information does not offer.
 Diagnosis – establishing that the patient is suffering from the condition
 Intervention – taking measures to improve decrease risk in a patient.
 Patient monitoring – looking for changes in patient condition. A change may
result in the retrieval of schemas due to a new combination of data elements.
 Patient-clinician communication and consultation – the constant exchange
between patient and clinician that may occur at all times during the clinical
workflow. Patients can be sources of data and collaborate with the clinician in
the data coverage loop ((Russell et al., 1993).
 Clinician-clinician communication – consulting another clinician in order to
gain further information. Communication can help the sense maker become
aware of residue and change schemas (Sharma, 2006). Multidisciplinary
team meetings and second opinions about a patient can lead to reframing.

66
This section contains findings of the way in which clinicians think about risk.
The participants expressed a need for a quantitative measure when they think
about risk. Regardless of whether it was for an overall summary of risk or the risk
related to a single data point that contributes to the risk of a condition, the partici-
pants wanted the numbers. The probabilistic view of risk has the clinician antici-
pating an empirical measure.
One way that clinicians think about risk is absolute risk. This refers to the risk of
being affected by a condition over a period. For example, a 1 in 5, 20%, or 0.2
risk of developing of developing diabetes in the next 10 years.
Relative risk is the ratio of the probability of the patient who is part of a risk group
that is affected by a condition compared to the probability of somebody outside
the risk group being affected. This allows the clinician to put the risk in context
based on the two groups within the population.
The way in which clinicians think about risk is a complex process. The risk of de-
veloping a disease or condition is not the only one that affects the decision-mak-
ing process. Aspects of risk include shared risk factors of related diseases, tem-
poral risk, risk from intervention or medications, the risk of failure to act, risk from
co-morbidities, and economic risk etc. These factors, as well as others, are taken
into account when thinking about risk.
Discussing an exhaustive list of these factors is beyond the scope of this thesis,
so the risk versus benefit of intervention is used as an example. Lifestyle
changes, clinical tests, medication, surgery and other interventions all carry risks
in themselves. When the risk of an intervention gets closer to outweighing the

67
benefits of reducing the risk of developing a condition, clinicians become less
likely to go ahead with that intervention. In primary preventative care, such as in
the scenario used in this study, the tolerance to risk of developing a condition di-
minishes. For example, if the patient has a 1 in 5 chance of developing diabetes
II within 10 years and evidence has shown that an intervention reduces the rela-
tive risk by 50%, the absolute risk goes down from 1 in 5 to 1 in 10. However, a
hypothetical intervention might carry an increased risk of heart attack that out-
weighs the benefits of this risk reduction and therefore the clinician would not tol-
erate the risk of prescribing that intervention. This obviously changes in a situa-
tion where the benefits on intervention are higher than risks of doing nothing. One
participant expanded on this point:
P1 The person who is well, but sort of just in a prevention strategy, we are
not going to give them very major things that carry high risks. Whereas the
individual who I mentioned before whose heart disease condition is so
severe that he's going to be dead in the next three to four months, an
operation with a 25% mortality is balanced by a 100% mortality without it.
Individuals make sense in their own way; sensemaking is inherently an activity
that occurs in the mind of the individual. The way a person makes sense is influ-
enced by the history they have had in medicine, the patients they have seen, and
any other experiences affect a clinician’s understanding of a situation. Therefore,
it should not come as a surprise that participants talked about clinicians having
varying interpretations of risk from the same data:
P3 Different individuals and different clinicians would have different
interpretations of qualitative risks. A small risk or a tiny risk or a moderate
risk... and that's why clinical practice is not consistent between different
doctors, they all have slightly different interpretations of what's going on
and the risks of different things.

68
P6 What I often find is that different clinicians seem to remember risks a bit
differently. Sometimes one will say ‘oh, this [a risk factor] is more important’
and the other would say ‘this [the same risk factor] is less important’.
This inconsistency between clinicians is due in part to the nature of clinical prac-
tice, which is a subjective activity, despite the aspiration to remain objective when
assessing risk. Our findings echo Sutherland & Dawson (2002) who state, ‘in the
doctors’ worlds, new information is received and interpreted on the basis of past
experiences, cognitive structures, and social context’.
In some cases the threat of accountability, may create a lower tolerance toward
risk. It is possible for lawsuits to occur due to ‘greed, or simply because the pa-
tient or family members did not like the outcome or the doctors involved’ (Noland
& Carl, 2006, p.88). As a participant said, underestimating risk can result in nega-
tive consequences for the clinician, but overestimating may cause negative con-
sequences to the patients:
P1 I think for medical legal reasons, the world gets a little more defensive
and therefore we don't want to underestimate risk of a treatment procedure
and risk the patient or colleagues saying 'you only quoted 10%, it was
clearly 25%'. […] Maybe then there's a worry that we overestimate the risks
to be defensive. Much better to have been conservative than provide
someone with a higher risk, but if you start doing that [i.e. overestimating]
we might get more people saying no and declining important care.
Findings show that the participants deal with an excess of data from multiple of
sources. Extracting the most important information from seemingly irrelevant ma-
terial remains a challenge. The presentation of data, availability of information,
and trust placed in information all have severe implications on clinicians’ deci-
sion-making.

69
Data from interviews and think-aloud sessions revealed that trust is assigned to
data in different ways. The weighting attached to data in the decision-making pro-
cess surrounding risk is connected to how much they trust it.
Trust in self-reported data from patients tends to be lower than other types of
data because of the expectation of a higher margin of mistake in them: patients
may lie for their own reason, misreport, forget, or lack the appropriate knowledge.
P1 People lie and in consultations, not everyone tells the truth the whole
time, but then it's possible that you get an incorrect value based on an
incorrect piece of information […] You get a lot of people that say 'ah,
everything's fine' and then you scratch below the surface and it’s not. Trust
is a judgement, but anything that involves a person […] what I'm saying
there is a higher risk versus sending a blood sample to a laboratory and
getting a number back.
There is also a chance of misinterpretation during communication between pa-
tient and clinician. This happens when the mental model of the patient does not
match that of the clinician. Coping strategies to mitigate these issues include ask-
ing the patient in a number of different ways and approaching witnesses (e.g.
asking friends, family and co-workers) to confirm what the patient reports. In our
study, clinicians regarded expertise as the key factor in being able to know how
much weight to place on patient reported data.
The quality of the tools used to gather data is another factor clinician’s take into
account when assigning trust. For example, a participant was describing a patient
using their personal sphygmomanometer to measure their own blood pressure
(BP) from home. The sphygmomanometer is not quality assessed like the ones in
the hospital; therefore, trust in the patient’s equipment is lower. With the increase
of consumer eHealth devices, trust in the quality measurement tools will likely be-
come a prevailing issue. One possible way to mitigate the uncertainty created by
this lack of trust is by triangulating data from a number of sources.

70
Clinicians do not only assign trust to the tool itself, but also the way in which it is
used. The quality of the tool may be fine, but the measurement method may be
flawed:
P5 The issue that some patients take the BP at the wrist as opposed to
taking it on their arm. The risk reading is going to be a different reading to
the arm. The data that the patients sometimes present might not be as
accurate as the data that we have.
Other ways that clinicians assign trust are:
 Volume of data – more usually means more confidence as trends over time
are revealed. For example, P5 said that BP readings from a patient might be
preferable if there are many data points making anomalies easier to track.
 Age of data – depending on the type of data, older data can be less trusted
than newer data, for example blood chemistry tests from years ago have little
bearing on the patients current condition.
The findings suggest that clinicians say that they assign trust to data is partly due
to the amount of uncertainty surrounding a data point. Since the limitations are
known for clinically generated quantitative data, clinicians assign higher trust:
P1 [the highest trust is assigned to] hospital generated data, which is hard
data. Results from the laboratory, results from a scan, results from a
procedure where we will review that, we will know […] some of the
limitations of that.
Although there was a consensus that all data should be viewed with scepticism, a
number of participants spoke of quantifiable hard data as the basis of clinical
knowledge, and therefore overall more trustworthy. At the opposite spectrum, we
find ‘soft’ qualitative data. The tendency (due in part to medical training) is to as-
sign to numerical, clinical and quantified data (i.e. ‘objective’ data) a higher
amount of trust than qualitative, textual and contextual ones (i.e. patient reported
and ‘subjective’ data).

Complex Health Data Visualization

Complex Health Data Visualization

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Viewers also liked

Viewers also liked (17)

Similar to Complex Health Data Visualization

Similar to Complex Health Data Visualization (19)

Recently uploaded

Recently uploaded (20)

Complex Health Data Visualization