The document discusses using theory-based research to improve health informatics (HI). It provides examples of testing theories from fields like communication, decision-making, and behavior change to optimize eHealth interventions before randomized controlled trials. Specific theories and studies testing things like how alert formatting impacts prescribing are summarized. The document argues this approach can help establish HI as a professional discipline by building a scientific evidence base for more reliable eHealth tools.

1. How can research reveal the science underlying
health informatics ?
So we can make HI more professional - like building bridges
Prof Jeremy Wyatt DM FRCP ACMI Fellow
Leadership chair in eHealth research, University of Leeds, UK
Clinical adviser on new technologies, Royal College of Physicians
From 1/1/16: Director, Wessex Institute of Health, University of
Southampton
j.c.wyatt@leeds.ac.uk

2. Some tough questions
1. Why are usable EPRs so hard to engineer ?
2. Why do one third of CDSS trials fail (Garg 2005) – when
those CDSS must be very well engineered for an RCT ?
3. How can an ePrescribing system cause so much harm ?
(Koppel, JAMA 2005)
A clue: why don’t bridges fall down nowadays:
a) There is a science of materials and construction methods
b) Engineers are professionals: they learn the science &
keep up to date
Tay Bridge disaster, 1879

3. Is HI / eHealth a “professional”
discipline yet ?
Evolution of professionalism:
• Intuition – a craft
• Mapping, taxonomy – a trade
• Testing of predictive theories - research
• Reliable engineering based on this – a profession
Heathfield H, Wyatt JC. Methods Inf Med, 1995
For HI:
1960-70s
1980-1990s
2000-2020
2020 on ?

4. What kinds of theories are relevant
in eH / HI ?
User 2
Health information
system
Decision
Improved behaviour
& outcome
User 1
Theories of communication
Theories of information retrieval
Theories of decision making
Behaviour change theories
(personal / organisational)
Consider a simple eHealth system: an internet forum to support
smoking cessation

5. How to carry out theory-based
eHealth research
Identify a promising theory
Identify a common, important
eHealth problem
Version of information system
that ignores the theory
Incorporate this theory
into an information system
Measure
usage & impact
of both systems
Analyse problem characteristics
and possible solutions
New knowledge about the
problem - and the theory
Literature review,
systematic review

6. Example 1: Does Fogg’s theory help website
persuade people to donate organs for transplant?
Persuasive features:
1. URL includes https, dundee.ac.uk
2. University Logo
3. No advertising
4. References
5. Address & contact details
6. Privacy Statement
7. Articles all dated
8. Site certified (W3C / Health on Net)
Work of Thomas Nind,
PhD Student, Dundee

7. Example 2: does feedback on group
performance increase exercise ?
RCT with 32 students: all sent us daily txt msg of step count
Half (“Team B”) got weekly feedback on total step count of
“their” group vs control group
Modest support for “group obligation” theory
Control (team A)
Intervention (team B)
Work of Sam Dhesi,
Medical Student, Leeds

8. Intervention modelling
experiments
Aim: to optimise the intervention before an RCT
Example methods:
• Attitude surveys
• Focus groups
• Formal usability studies
• Log file analysis
• Eye tracking studies
• Neuromarketing methods
• Simulated decision studies

9. Example 3: How to improve the
acceptability of prescribing alerts?
DSS are effective tools to improve prescribing (Garg
2005)
However, GPs usually turn off their prescribing alerts,
because:
• Too many alerts – no grading by severity
• False positives: poor knowledge base, poorly coded data
Question:
• Can we improve acceptability of alerts while still
reducing prescribing errors ?
Work of Greg Scott, ACF, London funded by NPfIT

10. Potential ways to improve
clinical alerts
Alert content:
• Wording – signal words (“Warning !”)
• Other material: symbols – alert triangles etc.
• Clickable list of actions to perform
Alert accuracy:
• Improve completeness, quality of coded patient data
• Improve completeness, quality of drug knowledge
• Improve underlying alert logic eg. calculate event probability
How the alert appears on screen:
• Location, size
• Persistence

11. Interruptive alert

12. Non-interruptive alert

13. Summary of results
Modal alerts: participants 12X (95% CI 6.0 to 22.3) less likely to make prescribing
error than when not shown any alert
Non-modal alert: 3 times (CI 1.9 to 5.3) less likely to make prescribing error
Non-modal alert error rate 4 times higher (CI 1.9 to 7.0) than with modal alerts
“Safe” Dr = 0 or 1 error out of 24 scenarios

14. Some participant comments
“When you are in a rush, the one that pops up is better –
forces you to click on OK”
“Pop-ups make you think more as you do it”
“[I prefer] interruptive – likely to miss otherwise. But
recognise the problems, irritating in daily use.”
“Interruptive tend to be annoying. But if it’s something you
don’t want to miss…”
“Difficult to say what deserves one type or other of alert”
“Didn’t notice it”
Published as: Scott et al JAMIA 2011

15. The MOST SMART approach
MOST: multiphase optimisation (of complex
interventions):
1. Screen intervention components for effectiveness (lab expts
on simulated decisions, RCTs, full / fractional ANOVA…)
2. Fine tune the combination of intervention components using
SMART, qv.
3. Standard RCT to confirm effectiveness
SMART: sequential multiple assignment randomised
trial (of time-varying interventions):
• Randomise participants at each stage to competing
interventions, as suggested by theory
• Collins et al. Am J rev Med 2007

16. SMART: example for an
exercise SMS programme
Assess stage
of change
(Prochaska)
-ve / +ve framed
msgs
Positive framed
msgs better for
relapsers ?
Own name
or not
Own name
annoying after
a while ?
Individual /
aggregate team
feedback
Risk of
everyone
matching lowest
performer in
group ?
Theories tested:

17. What is eHealth research
really for ?
Relevant theory
Rigorous research
Generic, reliable,
actionable knowledge
Safer, more reliable
eHealth tools
Publication,
dissemination
Health
problem

18. Benefits of building the
eHealth “theory base”
• No more trial and error or re-invention of ad hoc systems
that seemed sensible at the time
• eHealth will evolve from an intuitive craft (reliant on
experts and apprenticeship) into a professional discipline,
making its decisions based on tested theories
• Systems will be safe, efficient & predictable (like bridges)
• No need to evaluate every version of every app / website
/ serious game...

19. Conclusions
1. Professionalism requires sound theories
2. eHealth research should test theories from
information, cognitive, organisational and
computer science
3. Suggested procedure:
• Define a question of generic importance to our field
• Identify a candidate theory, relevant eHealth case
study & potential biases
• Select the best evaluation method to test the theory
• Carry out the study
4. Promote the results to students and eH
practitioners

21. Even a tablet is a complex
intervention
Doctor / nurse /
pharmacist instructions
Leaflet insert
Packaging
Colour of the pills
Monitoring of drug levels, response to therapy
Pt expectations
Clinician
expectations
Experience of others

22. eHealth mechanism of action
22/39
Clinical eHealth
system
eHealth system
Clinician
Outcome
Patientt1
action Disease
activityt1
Patientt2
Disease
activityt2
Patient eHealth
system
Decision
interval
t2-t1
ii
data collection bias
placebo effectcontamination,
checklist effect

23. TIDieR intervention reporting checklist
Hoffmann et al. Template for Intervention Description and Replication
(TIDieR) checklist and guide. BMJ 2014
23/39
BRIEF NAME - name or a phrase that describes the intervention.
WHY Describe any rationale, theory, or goal of the elements essential to the intervention.
WHAT: Materials: Describe any physical or information materials used, including those provided to participants or
used in intervention delivery or in training of intervention providers. Provide information on where the materials can
be accessed (e.g. online appendix, URL).
Procedures: Describe each procedure activity, and/or process used in the intervention, including any enabling or
support activities.
WHO PROVIDED For each category of intervention provider (e.g. psychologist, nursing assistant), describe their
expertise, background and any specific training given.
HOW Describe the modes of delivery (e.g. face-to-face or by some other mechanism, such as internet or
telephone) of the intervention and whether it was provided individually or in a group.
WHERE Describe the type(s) of location(s) where the intervention occurred, including any necessary infrastructure
or relevant features.
WHEN and HOW MUCH Describe the number of times the intervention was delivered and over what period of time
including the number of sessions, their schedule, and their duration, intensity or dose.
TAILORING If intervention was planned to be personalised / adapted, then describe what, why, when, and how.
MODIFICATIONS If the intervention was modified during the course of the study, describe the changes (what, why,
when, and how).
HOW WELL:
Planned: If intervention adherence or fidelity was assessed, describe how and by whom, and if any strategies were
used to maintain or improve fidelity, describe them.
Actual: If intervention adherence or fidelity was assessed, describe the extent to which the intervention was
delivered as planned.

24. Cross disciplinary research
Chindogu device for restarting your PC

25. Neuromarketing – a food industry
example
Theory: for behaviour, emotion > information (Kahneman’s System 1)
Methods: FMRI; EDA; facial EMG; web-cam facial expression recognition

26. Study aim & methods
Aim: to help develop more effective SMS msgs for health
promotion, by:
• Developing a reliable methods to capture EDA, facial EMG
• Validate it against words & phrases of known emotional import
• Use it to test & improve new phrases and txt msgs before an RCT
Methods - 40 volunteers:
• Measure EDA and facial EMG
• Exposed to 20 words of known emotional import, 5 words about
exercise, 5 nonsense words & their own name in random order
Work of Gabriel Mata, Leeds PhD student funded by CONACYT, Mexico

27. Methods

28. Results
hysineral
moofthrist
fim
napsate
retrating
kiss
sexy
explosion
hysterical
killer
nightmare
music
family
clown
news
infection
funeralpillow
relax
table
nunbored
pale
you today
activity
exercisephysical
[NAME]
-0.04000000
-0.02000000
0.00000000
0.02000000
0.04000000
0.06000000
0.08000000
1 6 11 16 21 26
reactivityinµS
word
EDA reactivity