SlideShare a Scribd company logo
1 of 8
Download to read offline
 
How	
  we	
  are	
  evaluating	
  the	
  impact	
  of	
  new	
  care	
  
models	
  on	
  how	
  people	
  feel	
  in	
  Wessex	
  
	
  
Introduction	
  
When	
  the	
  New	
  Care	
  Models	
  (NCM)1	
  
evaluation	
  
team	
  set	
  out	
  their	
  priorities	
  and	
  expectations	
  for	
  
vanguard	
  evaluation	
  in	
  2017/18,	
  they	
  described	
  
a	
  need	
  to	
  strengthen	
  the	
  focus	
  on	
  evaluating	
  
patients’	
  experience	
  of	
  transformed	
  services	
  
(New	
  Care	
  Models	
  June	
  2017).	
  
Wessex	
  AHSN	
  are	
  working	
  in	
  partnership	
  with	
  
the	
  Centre	
  for	
  Implementation	
  Science,	
  
University	
  of	
  Southampton	
  (CIS)	
  and	
  R-­‐
Outcomes	
  Ltd	
  to	
  evaluate	
  two	
  PACS	
  vanguards	
  
in	
  North	
  East	
  Hampshire	
  and	
  Farnham	
  (NEHF)	
  
and	
  the	
  Isle	
  of	
  Wight	
  (IoW)	
  and	
  a	
  number	
  of	
  
other	
  new	
  services	
  and	
  care	
  models	
  across	
  
Wessex.	
  	
  We	
  have	
  completed	
  a	
  number	
  of	
  
detailed	
  evaluations,	
  including	
  social	
  prescribing	
  
and	
  integrated	
  care	
  team	
  models,	
  with	
  a	
  strong	
  
focus	
  on	
  the	
  impact	
  on	
  patients,	
  people,	
  carers	
  
and	
  staff.	
  	
  	
  
This	
  short	
  report	
  describes	
  the	
  mixed-­‐methods	
  
approach	
  to	
  evaluation	
  that	
  we	
  have	
  used,	
  what	
  
we	
  have	
  found	
  and	
  how	
  this	
  has	
  helped	
  develop	
  
new	
  care	
  models.	
  	
  Our	
  approach	
  has	
  been	
  to	
  
consistently	
  do	
  five	
  things	
  to	
  understand	
  how	
  
people	
  feel	
  about	
  new	
  care	
  models	
  (NCMs)2
:	
  
• Collect	
  and	
  analyse	
  large	
  numbers	
  of	
  R-­‐
Outcomes	
  from	
  people,	
  patients	
  and	
  staff	
  
• Undertake	
  qualitative	
  interviews	
  with	
  
patients,	
  carers	
  and	
  staff	
  
• Thematic	
  analysis	
  of	
  case	
  studies	
  
• Observe	
  teams	
  at	
  work	
  using	
  Normalisation	
  
Process	
  Theory	
  
• Synthesise	
  this	
  data	
  to	
  develop	
  triangulated,	
  
rounded	
  findings	
  and	
  evidence	
  of	
  impact	
  
	
  
	
  
	
  
	
  
Methods	
  
1. R-­‐Outcomes	
  
We	
  knew	
  we	
  wanted	
  self-­‐reported	
  outcomes	
  to	
  
play	
  a	
  major	
  role	
  in	
  our	
  evaluation	
  of	
  NCMs.	
  We	
  
understood	
  that	
  an	
  intrinsic	
  part	
  of	
  new	
  care	
  
models	
  is	
  changing	
  how	
  patients,	
  carers	
  and	
  
staff	
  feel	
  about	
  the	
  care	
  they	
  receive	
  and	
  
deliver.	
  	
  The	
  clue	
  was	
  in	
  the	
  title	
  -­‐	
  in	
  NEHF	
  the	
  
vanguard	
  is	
  called	
  Happy,	
  Healthy	
  at	
  Home	
  and	
  
on	
  the	
  Isle	
  of	
  Wight	
  My	
  life	
  a	
  full	
  life.	
  	
  Logic	
  
models	
  listed	
  desired	
  outcomes	
  like	
  improved	
  
wellbeing,	
  confidence	
  to	
  self-­‐care	
  and	
  staff	
  
satisfaction.	
  	
  We	
  wanted	
  to	
  find	
  a	
  way	
  to	
  reliably	
  
measure	
  and	
  understand	
  if	
  this	
  was	
  happening.	
  
A	
  small	
  team	
  reviewed	
  the	
  self	
  reported	
  
outcome	
  measures	
  available	
  and	
  selected	
  R-­‐
Outcomes	
  (R-­‐Os).	
  	
  R-­‐Os	
  are	
  a	
  family	
  of	
  short,	
  
generic,	
  validated3,4,5
,	
  self	
  reported	
  outcome	
  
measures	
  for	
  patients/	
  client,	
  carers	
  and	
  staff.	
  	
  	
  
They	
  have	
  been	
  carefully	
  designed	
  to	
  be	
  quick	
  
and	
  easy	
  to	
  complete	
  -­‐	
  which	
  means	
  that	
  more	
  
people	
  complete	
  them.	
  	
  Being	
  generic,	
  they	
  can	
  
be	
  applied	
  to	
  almost	
  any	
  health	
  or	
  care	
  setting,	
  
from	
  support	
  at	
  home	
  to	
  acute	
  inpatient	
  care.	
  	
  
They	
  can	
  be	
  collected	
  through	
  the	
  mechanism	
  
that	
  best	
  suits	
  the	
  service	
  –	
  including	
  on	
  paper,	
  
on-­‐line,	
  smart	
  phones	
  and	
  tablets,	
  or	
  the	
  
telephone.	
  	
  They	
  work	
  as	
  a	
  family	
  of	
  measures,	
  
with	
  the	
  ability	
  to	
  select	
  combinations	
  from	
  a	
  
current	
  menu	
  of	
  5	
  patient/	
  client	
  measures,	
  4	
  
measures	
  for	
  staff	
  and	
  3	
  for	
  carers.	
  	
  Further	
  
details	
  are	
  attached	
  as	
  an	
  appendix	
  and	
  at	
  
www.r-­‐outcomes.com.	
  
At	
  the	
  time	
  of	
  writing	
  we	
  have	
  collected	
  
outcomes	
  from	
  around	
  6000	
  patients/	
  people	
  
and	
  500	
  staff	
  in	
  Wessex	
  over	
  the	
  past	
  two	
  years.	
  	
  
Typically	
  we	
  collect	
  people’s	
  outcomes	
  on	
  
referral	
  and	
  once	
  they	
  have	
  been	
  supported	
  or	
  
cared	
  for	
  to	
  look	
  for	
  evidence	
  of	
  a	
  change.	
  	
  Free	
  
text	
  provides	
  additional	
  feedback	
  which	
  is	
  often	
  
presented	
  graphically	
  as	
  a	
  word	
  cloud.	
  
 
Evaluating	
  the	
  impact	
  of	
  new	
  care	
  models	
  on	
  people	
  
2	
  
Example	
  1:	
  Health	
  Confidence	
  in	
  patients	
  
supported	
  by	
  new	
  Integrated	
  Care	
  Teams.	
  
Patients	
  report	
  large	
  and	
  statistically	
  significant	
  
improvements	
  in	
  their	
  health	
  confidence	
  
following	
  the	
  support	
  from	
  the	
  five	
  Integrated	
  
Care	
  Teams.	
  	
  The	
  biggest	
  improvements	
  are	
  in	
  
people	
  reporting	
  that	
  they	
  can	
  get	
  the	
  right	
  help	
  
when	
  they	
  need	
  it	
  and	
  being	
  involved	
  in	
  
decisions	
  –	
  which	
  is	
  the	
  heart	
  of	
  this	
  focused	
  
and	
  personalised	
  care	
  plan.	
  	
  Local	
  reporting	
  drills	
  
down	
  into	
  the	
  differences	
  between	
  the	
  
outcomes	
  from	
  different	
  teams.	
  
	
  
Example	
  2:	
  Health	
  Status	
  of	
  patients	
  supported	
  
by	
  eight	
  different	
  social	
  prescribing	
  services.	
  
Patients	
  reported	
  significant	
  improvements	
  in	
  
their	
  health	
  status	
  –	
  particularly	
  feeling	
  less	
  low	
  
or	
  worried.	
  	
  Similar	
  improvements	
  were	
  found	
  in	
  
their	
  health	
  confidence,	
  wellbeing	
  and	
  
experience	
  of	
  care.	
  
	
  
Example	
  3:	
  A	
  team	
  of	
  Care	
  Navigators.	
  
The	
  team	
  reported	
  a	
  high	
  level	
  of	
  wellbeing	
  at	
  
work,	
  are	
  very	
  satisfied	
  with	
  their	
  job	
  and	
  give	
  
top	
  marks	
  for	
  feeling	
  that	
  their	
  job	
  is	
  
worthwhile.	
  	
  	
  	
  
They	
  scored	
  their	
  confidence	
  in	
  delivering	
  their	
  
roles	
  less	
  highly.	
  They	
  can	
  get	
  help	
  when	
  they	
  
need	
  it.	
  	
  Very	
  high	
  scores	
  were	
  reported	
  for	
  the	
  
level	
  of	
  personal	
  service	
  and	
  empathy	
  they	
  
provide	
  but	
  less	
  positive	
  for	
  the	
  organisation	
  of	
  
the	
  service.	
  	
  	
  
These	
  surveys	
  have	
  been	
  repeated	
  to	
  analyse	
  
changes	
  over	
  time.	
  
	
  
Example	
  4:	
  	
  Comparing	
  the	
  before	
  and	
  after	
  
patient	
  scores	
  for	
  two	
  different	
  services.	
  
R-­‐Outcomes	
  can	
  be	
  used	
  to	
  look	
  for	
  changes	
  
over	
  time	
  and	
  to	
  compare	
  services.	
  	
  Reference	
  
groups	
  have	
  been	
  developed	
  to	
  help	
  this.	
  
	
  
	
  
	
  
0
20
40
60
80
100
Mean	Sore
Mean	scores	for	Localities	Pre	and	Post	Intervention
ESP	Pre	(n=51) ESP	Post	(n=99) MHH	Pre	(n=270) MHH	Post	(n=371)
 
Evaluating	
  the	
  impact	
  of	
  new	
  care	
  models	
  on	
  people	
  
3	
  
2.	
  	
  Qualitative	
  interviews	
  with	
  
patients,	
  carers	
  and	
  staff	
  	
  
To	
  explore	
  the	
  nature	
  and	
  extent	
  of	
  a	
  change,	
  
it’s	
  important	
  to	
  use	
  qualitative	
  methods	
  to	
  
ensure	
  a	
  good	
  depth	
  of	
  investigation	
  is	
  achieved.	
  	
  
In	
  our	
  case,	
  we’ve	
  used	
  qualitative	
  methods	
  to	
  
explore	
  whether	
  new	
  roles	
  and	
  new	
  innovations	
  
have	
  improved	
  patients’	
  health	
  and	
  wellbeing,	
  
and	
  health	
  service	
  delivery	
  by	
  health	
  care	
  
professionals.	
  Using	
  semi-­‐structured	
  
interviews	
  with	
  patients,	
  carers	
  and	
  staff,	
  we	
  
have	
  sought	
  the	
  narrative	
  of	
  the	
  innovation	
  
from	
  those	
  receiving	
  and	
  delivering	
  the	
  new	
  
role	
  or	
  innovation.	
  Obtaining	
  this	
  narrative	
  has	
  
helped	
  us	
  explain	
  why	
  change	
  has	
  or	
  has	
  not	
  
occurred,	
  what	
  impacts	
  have	
  been	
  observed	
  
by	
  patients	
  and	
  staff,	
  and	
  if	
  any	
  unintended	
  
consequences/impacts	
  were	
  observed.	
  
Questions	
  for	
  each	
  interview	
  have	
  been	
  
developed	
  using	
  the	
  logic	
  model	
  for	
  each	
  new	
  
role/innovation,	
  in	
  order	
  to	
  keep	
  the	
  interviews	
  
focused	
  on	
  the	
  topics	
  of	
  interest	
  and	
  to	
  allow	
  
synthesis	
  with	
  the	
  quantitative	
  evidence	
  
gathered.	
  
	
  
3.	
  	
  Themed	
  analysis	
  of	
  case	
  studies	
  
In	
  most	
  of	
  our	
  evaluations,	
  case	
  studies	
  have	
  
been	
  provided	
  by	
  staff	
  who	
  know	
  the	
  
individual’s	
  situation	
  and	
  journey,	
  and	
  are	
  
completed	
  on	
  a	
  simple	
  template	
  that	
  includes	
  
the	
  situation,	
  what	
  happened	
  and	
  how	
  the	
  
service	
  worked	
  with	
  the	
  individual.	
  	
  We	
  
undertake	
  thematic	
  analysis	
  of	
  these	
  case	
  
studies	
  (see	
  below)	
  as	
  part	
  of	
  our	
  evaluation.	
  
For	
  example,	
  eight	
  case	
  studies	
  of	
  patients	
  
receiving	
  input	
  from	
  an	
  Integrated	
  Care	
  Team	
  
revealed	
  four	
  key	
  themes:	
  a	
  quick	
  response,	
  a	
  
joined	
  up	
  response,	
  problem	
  solving	
  and	
  the	
  
impact	
  on	
  patients.	
  This	
  evidence	
  provided	
  
important	
  indications	
  that	
  the	
  new	
  care	
  model	
  
was	
  achieving	
  its	
  desired	
  outcomes.	
  When	
  
triangulated	
  with	
  the	
  findings	
  from	
  the	
  R-­‐
Outcomes	
  analysis,	
  the	
  evidence	
  of	
  impact	
  was	
  
further	
  strengthened.	
  
It	
  is	
  also	
  possible	
  to	
  use	
  case	
  notes	
  to	
  develop	
  
case	
  studies.	
  	
  For	
  an	
  evaluation	
  of	
  a	
  Recovery	
  
College,	
  a	
  random	
  set	
  of	
  five	
  case	
  notes	
  were	
  
reviewed	
  and	
  used	
  to	
  chart	
  each	
  student’s	
  
journey	
  of	
  enrolling	
  and	
  participating	
  in	
  recovery	
  
courses	
  and	
  their	
  wider	
  use	
  of	
  mental	
  health	
  
services.	
  	
  These	
  were	
  displayed	
  graphically	
  as	
  
follows:	
  	
  
Thematic	
  analysis	
  of	
  what	
  is	
  heard	
  in	
  qualitative	
  
interviews	
  and	
  written	
  in	
  case	
  studies	
  is	
  an	
  
important	
  part	
  of	
  our	
  evaluation	
  and	
  we	
  use	
  a	
  
well	
  defined	
  and	
  widely	
  used	
  approach.	
  This	
  
method	
  explores	
  the	
  semantic	
  (explicit)	
  themes	
  
and	
  latent	
  (implicit)	
  themes	
  within	
  the	
  
qualitative	
  data	
  via	
  a	
  five	
  stage	
  process:	
  
familiarise	
  yourself	
  with	
  the	
  data,	
  generate	
  
initial	
  codes,	
  search	
  for	
  themes,	
  develop	
  and	
  
refine	
  themes,	
  and	
  define	
  and	
  name	
  the	
  themes.	
  
It	
  	
  aims	
  to	
  find	
  repeated	
  patterns	
  of	
  meaning	
  
whilst	
  also	
  focusing	
  on	
  the	
  relevance	
  of	
  issues	
  in	
  
relation	
  to	
  the	
  aims	
  of	
  the	
  evaluation	
  of	
  the	
  new	
  
care	
  model.	
  Importantly,	
  as	
  with	
  many	
  forms	
  of	
  
qualitative	
  analysis,	
  data	
  saturation	
  is	
  the	
  goal.	
  
Once	
  individual	
  narratives	
  begin	
  to	
  repeat,	
  and	
  
the	
  same	
  themes	
  and	
  sub-­‐themes	
  are	
  evident	
  in	
  
the	
  interviews,	
  data	
  collection	
  is	
  stopped6
.	
  
4.	
  	
  Team	
  observation	
  using	
  
normalisation	
  process	
  theory	
  
As	
   many	
   of	
   the	
   NCMs	
   being	
   evaluated	
   are	
  
formed	
   of	
   new	
   teams	
   of	
   staff,	
   we	
   were	
  
particularly	
   interested	
   to	
   find	
   a	
   pragmatic	
   and	
  
validated	
   evaluation	
   tool	
   to	
   understand	
   the	
  
extent	
  to	
  which	
  the	
  team	
  was	
  able	
  to	
  embed	
  the	
  
0	
1	
2	
3	
4	
5	
6	
7	
8	
Jan-15	Feb-15	M
ar-15	Apr-15	M
ay-15	Jun-15	Jul-15	Aug-15	Sep-15	Oct-15	Nov-15	Dec-15	Jan-16	Feb-16	M
ar-16	Apr-16	M
ay-16	Jun-16	
Number	of	contacts	
Recovery	College	case	study	–	dates	of	course	enrolment	and	
number	of	mental	health	service	contacts	
Enrolled	in	RC	
Wellness	for	life	
Managing	intense	emo4ons	
Confidence	&		
self	esteem
 
Evaluating	
  the	
  impact	
  of	
  new	
  care	
  models	
  on	
  people	
  
4	
  
implementation	
  NCMs	
  on	
  a	
  day	
  to	
  day	
  basis	
  in	
  a	
  
way	
  that	
  was	
  sustainable	
  in	
  the	
  long	
  term.  
Understanding	
  if	
  and	
  how	
  new	
  roles,	
  techniques	
  
and	
  innovations	
  for	
  clinical	
  and	
  professional	
  
practice	
  are	
  effectively	
  translated	
  into	
  practice	
  is	
  
vital	
  for	
  the	
  teams	
  delivering	
  them	
  and	
  the	
  
programmes	
  commissioning	
  them.	
  	
  
Normalisation	
  Process	
  Theory	
  (NPT]7
	
  was	
  
developed	
  to	
  better	
  understand	
  	
  how	
  
innovations	
  are	
  implemented	
  in	
  practice	
  and	
  in	
  
particular	
  how	
  activities	
  associated	
  with	
  
introducing	
  new	
  ways	
  of	
  working	
  are	
  both	
  
enabled	
  and	
  constrained	
  with	
  in	
  a	
  particular	
  
setting.	
  It	
  is	
  a	
  theoretically	
  developed	
  and	
  widely	
  
used	
  sociological	
  theory	
  of	
  how	
  innovations	
  are	
  
understood,	
  engaged	
  with,	
  enacted	
  and	
  
reflected	
  upon.	
  NPT	
  has	
  widely	
  been	
  used	
  in	
  the	
  
dynamic	
  setting	
  of	
  health	
  care.	
  	
  
The	
  starting	
  point	
  of	
  NPT	
  is	
  to	
  understand	
  the	
  
embedding	
  of	
  a	
  practice	
  i.e.	
  what	
  people	
  
actually	
  do	
  and	
  how	
  they	
  work	
  together.	
  NPT	
  
provides	
  an	
  explanatory	
  framework	
  to	
  better	
  
understand	
  the	
  routine	
  embedding	
  of	
  
healthcare	
  interventions	
  in	
  their	
  social	
  contexts,	
  
in	
  particular	
  why	
  some	
  processes	
  seem	
  to	
  lead	
  
to	
  a	
  practice	
  becoming	
  sustained	
  over	
  a	
  long	
  
term	
  while	
  others	
  do	
  not	
  (May	
  and	
  Finch,	
  2009).	
  
Although	
  the	
  authors	
  make	
  no	
  claim	
  of	
  absolute	
  
predictive	
  power,	
  they	
  argue	
  that	
  within	
  certain	
  
limits	
  the	
  extent	
  of	
  sustainable	
  long	
  term	
  
embedding	
  can	
  be	
  anticipated.	
  This	
  means	
  that	
  
NPT	
  can	
  help	
  ascertain	
  the	
  likelihood	
  of	
  the	
  
routine	
  embedding	
  of	
  an	
  intervention	
  within	
  
certain	
  limits.	
  
We	
  have	
  conducted	
  observations	
  of	
  team	
  
meetings	
  and	
  key	
  interactions	
  between	
  staff	
  
using	
  the	
  four	
  key	
  components	
  (Coherence,	
  
Cognitive	
  Participation,	
  Collective	
  Action,	
  
Reflexive	
  Monitoring).	
  In	
  addition,	
  the	
  NPT	
  
NoMAD	
  questions8
	
  have	
  been	
  used	
  for	
  a	
  survey	
  
undertaken	
  during	
  the	
  structured	
  focus	
  groups.	
  
The	
  survey	
  also	
  included	
  questions	
  on	
  feeling	
  
valued	
  as	
  part	
  of	
  the	
  team	
  and	
  questions	
  about	
  
the	
  extent	
  to	
  which	
  the	
  expected	
  outcomes	
  of	
  
the	
  service	
  (as	
  articulated	
  in	
  its	
  logic	
  model)	
  
were	
  felt	
  to	
  have	
  been	
  achieved.	
  
We	
  	
  have	
  now	
  conducted	
  four	
  evaluations	
  of	
  
different	
  teams	
  that	
  are	
  working	
  in	
  an	
  
integrated	
  way	
  to	
  implement	
  NCMs.	
  
Table	
  1:	
  Results	
  for	
  coherence	
  
Table	
  1	
  shows	
  team	
  results	
  for	
  coherence	
  or	
  
how	
  a	
  NCM	
  is	
  conceptualised	
  and	
  held	
  together	
  
in	
  action.	
  In	
  this	
  example	
  coherence	
  was	
  rated	
  
positively.	
  The	
  team	
  members	
  had	
  a	
  shared	
  
understanding	
  of	
  the	
  new	
  practice	
  and	
  
understood	
  how	
  it	
  affected	
  the	
  nature	
  of	
  their	
  
work	
  and	
  could	
  see	
  potential	
  value	
  in	
  it	
  for	
  their	
  
work.	
  Should	
  the	
  results	
  have	
  been	
  negative,	
  
this	
  may	
  have	
  required	
  some	
  adjustments	
  in	
  
relation	
  to	
  the	
  implementation	
  of	
  the	
  new	
  
practice.	
  	
  
	
  
5.	
  	
  Synthesising	
  findings	
  
As	
  we	
  have	
  consistently	
  applied	
  these	
  methods	
  
for	
  evaluating	
  how	
  people	
  experience	
  and	
  feel	
  
about	
  new	
  care	
  models,	
  we	
  have	
  also	
  developed	
  
our	
  method	
  for	
  bringing	
  together	
  and	
  
synthesising	
  these	
  findings.	
  
The	
  focus	
  is	
  a	
  synthesis	
  meeting	
  that	
  brings	
  
together	
  all	
  of	
  the	
  people	
  involved	
  in	
  gathering	
  
the	
  data	
  and	
  evidence	
  described	
  in	
  this	
  paper,	
  as	
  
well	
  as	
  the	
  quantitative	
  elements	
  of	
  the	
  
evaluation,	
  including	
  changes	
  in	
  activity	
  (e.g.	
  
A&E	
  attendances)	
  and	
  economic	
  evaluation.	
  
Before	
  the	
  synthesis	
  meeting	
  takes	
  place,	
  we	
  
	
Coherence	(n-9)		
Av.	score	
1.		The	NCM	is	distinct	from	previous	
ways	of	working	
7.8	
2.		Team	members	have	a	shared	
understanding	of	the	purpose	of	the	
NCM	and	of	the	specific	responsibilities	
required.	
8.4	
3.		Team	members	understand	how	the	
NCM	affects	the	nature	of	their	work	
8.4	
4.		Team	members	can	see	potential	
value	of	the	NCM	for	their	work	
8.2	
Total	 8.2
 
Evaluating	
  the	
  impact	
  of	
  new	
  care	
  models	
  on	
  people	
  
5	
  
aim	
  to	
  meet	
  with	
  representatives	
  of	
  the	
  new	
  
care	
  model	
  to	
  take	
  them	
  through	
  it,	
  check	
  for	
  
accuracy	
  and	
  get	
  their	
  initial	
  feedback	
  or	
  
reflections.	
  	
  	
  At	
  the	
  synthesis	
  meeting	
  all	
  of	
  the	
  
material	
  is	
  pooled	
  and	
  worked	
  through	
  together	
  
to	
  triangulate	
  the	
  evidence	
  and	
  identify	
  and	
  
agree	
  the	
  findings,	
  lessons	
  and	
  
recommendations.	
  	
  An	
  important	
  part	
  of	
  this	
  
synthesis	
  is	
  to	
  identify	
  the	
  active	
  ingredients	
  
that	
  have	
  contributed	
  to	
  (or	
  hindered)	
  the	
  
outcomes	
  delivered	
  by	
  the	
  new	
  model	
  of	
  care.	
  	
  	
  
	
  
6.	
  	
  How	
  the	
  findings	
  have	
  been	
  used	
  
by	
  our	
  clients	
  
We	
  understood	
  that	
  NCMs,	
  by	
  their	
  nature,	
  
would	
  be	
  likely	
  to	
  adapt	
  and	
  evolve	
  during	
  the	
  
Vanguard’s	
  journey.	
  We	
  have	
  tried	
  to	
  support	
  
teams	
  in	
  this	
  creative	
  process	
  by	
  providing	
  early	
  
headlines	
  from	
  the	
  evaluation	
  findings	
  and	
  
exploring	
  with	
  them	
  the	
  possible	
  explanations	
  
for	
  the	
  emerging	
  evidence.	
  This	
  feedback	
  helps	
  
the	
  team	
  and	
  the	
  programme	
  to	
  understand	
  
whether	
  an	
  NCM	
  is	
  having	
  the	
  desired	
  impact	
  
and	
  if	
  not	
  why	
  not,	
  and	
  helps	
  the	
  evaluation	
  
team	
  to	
  decide	
  where	
  to	
  focus	
  the	
  inquiry	
  if	
  the	
  
results	
  are	
  unexpected	
  (e.g.	
  to	
  examine	
  one	
  area	
  
in	
  more	
  detail,	
  or	
  to	
  differentiate	
  the	
  data	
  set	
  in	
  
different	
  ways).	
  	
  It	
  has	
  been	
  valuable	
  to	
  have	
  
contemporaneous	
  access	
  to	
  outcomes	
  data	
  that	
  
can	
  enable	
  timely	
  reporting	
  to	
  staff	
  with	
  a	
  keen	
  
interest	
  in	
  how	
  their	
  NCM	
  is	
  performing.	
  	
  
The	
  following	
  examples	
  describe	
  some	
  of	
  the	
  
beneficial	
  outcomes	
  of	
  ‘formative’	
  evaluation	
  for	
  
those	
  involved	
  in	
  implementing	
  NCMs:	
  	
  
Example	
  1:	
  The	
  evaluation	
  of	
  teams	
  through	
  
semi-­‐structured	
  observation	
  has	
  influenced	
  the	
  
design	
  of	
  organisational	
  development	
  
programmes,	
  by	
  identifying	
  priority	
  areas	
  to	
  
focus	
  on	
  e.g.	
  barriers	
  or	
  enablers	
  to	
  the	
  NCM.	
  	
  	
  
Example	
  2:	
  Evidence	
  on	
  the	
  impact	
  of	
  a	
  Referral	
  
Management	
  Service	
  has	
  informed	
  the	
  business	
  
case	
  for	
  the	
  development	
  of	
  a	
  locality	
  based	
  
community	
  dermatology	
  service.	
  
Example	
  3:	
  Data	
  on	
  patient	
  outcomes	
  (R-­‐
Outcomes)	
  from	
  a	
  number	
  of	
  different	
  
integrated	
  care	
  teams	
  across	
  Wessex	
  has	
  
initiated	
  a	
  new	
  line	
  of	
  inquiry	
  –	
  do	
  differences	
  in	
  
casemix	
  account	
  for	
  differences	
  in	
  patient	
  
reported	
  outcomes?	
  
Example	
  4:	
  	
  Evidence	
  of	
  the	
  different	
  impact	
  
from	
  different	
  forms	
  of	
  social	
  prescribing	
  service	
  
has	
  informed	
  a	
  new	
  commissioning	
  service	
  
specification	
  –	
  against	
  which	
  the	
  services	
  were	
  
re-­‐commissioned.	
  	
  Patients	
  and	
  carers	
  
emphasised	
  the	
  importance	
  of	
  support	
  at	
  the	
  
weekend	
  and	
  this	
  was	
  included.	
  	
  The	
  new	
  
service	
  level	
  agreements	
  include	
  quality	
  
outcomes	
  (R-­‐Outcomes)	
  KPIs.	
  
Example	
  5:	
  	
  Community	
  ambassadors	
  have	
  been	
  
engaged	
  in	
  collecting	
  R-­‐Outcomes	
  from	
  people	
  
in	
  waiting	
  rooms	
  using	
  iPads	
  and	
  have	
  
participated	
  in	
  the	
  review	
  and	
  discussion	
  of	
  all	
  of	
  
the	
  qualitative	
  findings	
  and	
  symposium	
  events.	
  
Example	
  6:	
  Bespoke	
  measures	
  have	
  been	
  
incorporated	
  into	
  R-­‐Outcomes	
  to	
  cover	
  issues	
  of	
  
particular	
  interest	
  to	
  the	
  service	
  or	
  service	
  users	
  
e.g.	
  to	
  understand	
  how	
  people	
  perceive	
  
‘integration.’	
  	
  	
  
	
  
	
   	
  
 
Evaluating	
  the	
  impact	
  of	
  new	
  care	
  models	
  on	
  people	
  
6	
  
7.	
  	
  Our	
  reflections	
  on	
  how	
  to	
  
evaluate	
  the	
  impact	
  of	
  new	
  care	
  
models	
  on	
  people	
  
Our	
  evaluation	
  vital	
  statistics	
  at	
  the	
  time	
  of	
  
writing	
  are:	
  
NCM	
  evaluations	
  completed	
  and	
  reported	
   15	
  
R-­‐Outcomes	
  recorded	
  by	
  patients	
   6,000	
  
R-­‐Outcomes	
  completed	
  by	
  staff	
   500	
  
Qualitative	
  interviews	
  undertaken	
   85	
  
Case	
  studies	
  analysed	
   93	
  
Evaluations	
  planned	
  for	
  17/18	
   21	
  
	
  
R-­‐Outcomes	
  have	
  provided	
  a	
  reliable	
  way	
  of	
  
measuring	
  the	
  impact	
  on	
  patients	
  and	
  staff	
  at	
  
scale	
  and	
  in	
  a	
  short	
  period	
  of	
  time.	
  	
  They	
  enable	
  
us	
  to	
  look	
  for	
  evidence	
  of	
  change	
  during	
  
implementation	
  of	
  a	
  NCM	
  and	
  increasingly	
  we	
  
are	
  using	
  them	
  to	
  benchmark	
  the	
  impacts	
  on	
  
different	
  cohorts	
  of	
  patients	
  and	
  services.	
  
Qualitative	
  interviews	
  are	
  important	
  as	
  they	
  
provide	
  the	
  most	
  in-­‐depth	
  method	
  for	
  
understanding	
  the	
  lived	
  experience	
  of	
  patients	
  
and	
  staff.	
  However,	
  whilst	
  every	
  lived	
  
experience	
  reported	
  by	
  a	
  patient	
  or	
  team	
  
member	
  is	
  valid	
  in	
  understanding	
  the	
  impact	
  of	
  
NCMs,	
  qualitative	
  data	
  collection	
  can	
  be	
  very	
  
resource	
  intensive	
  for	
  evaluators	
  and	
  create	
  
additional	
  work	
  for	
  staff	
  (e.g.	
  consenting	
  
individuals	
  for	
  interview,	
  scheduling	
  interviews,	
  
Information	
  Governance	
  and	
  Ethics	
  approvals).	
  	
  
The	
  number	
  of	
  participants	
  that	
  are	
  suitable	
  for	
  
interview	
  can	
  be	
  small.	
  In	
  our	
  experience,	
  we	
  
have	
  typically	
  identified	
  around	
  6	
  patients	
  and	
  6	
  
staff	
  per	
  NCM	
  with	
  interview	
  times	
  of	
  up	
  to	
  an	
  
hour	
  each.	
  In	
  order	
  to	
  ensure	
  we	
  obtain	
  an	
  in-­‐
depth	
  picture	
  of	
  patient	
  and	
  staff	
  experience	
  
and	
  also	
  adhere	
  to	
  data	
  collection	
  timeframes,	
  
we	
  are	
  looking	
  at	
  how	
  we	
  gather	
  and	
  analyse	
  
this	
  data	
  to	
  be	
  sufficiently	
  rigorous	
  but	
  more	
  
time	
  and	
  resource	
  efficient.	
  
Case	
  studies	
  are	
  also	
  a	
  powerful	
  way	
  of	
  
capturing	
  the	
  in-­‐depth	
  impact	
  for	
  individuals.	
  
Teams	
  that	
  have	
  been	
  motivated	
  to	
  collect	
  these	
  
stories	
  have	
  produced	
  around	
  20-­‐30	
  case	
  studies	
  
for	
  analysis.	
  A	
  simple	
  template	
  has	
  been	
  helpful	
  
in	
  some	
  instances,	
  but	
  the	
  most	
  significant	
  
enabler	
  has	
  been	
  a	
  team	
  leader	
  who	
  is	
  
convinced	
  of	
  the	
  value	
  of	
  collecting	
  this	
  data.	
  	
  	
  
NPT	
  has	
  proved	
  helpful	
  insights	
  into	
  the	
  extent	
  
to	
  which	
  integrated	
  ways	
  of	
  working	
  have	
  
become	
  embedded	
  by	
  new	
  care	
  model	
  teams.	
  It	
  
has	
  been	
  used	
  successfully	
  with	
  focus	
  groups	
  
from	
  4	
  to	
  24.	
  The	
  findings	
  have	
  been	
  recognised	
  
by	
  those	
  participating	
  and	
  informed	
  team	
  
development.	
  	
  
	
  
8.	
  	
  Looking	
  forward	
  
Three	
  new	
  measures	
  have	
  recently	
  been	
  added	
  
to	
  the	
  R-­‐Outcomes	
  suite	
  and	
  will	
  be	
  trialled	
  in	
  
Vanguard	
  evaluations	
  during	
  the	
  coming	
  year,	
  
measuring:	
  
• Patient	
  and	
  staff	
  perception	
  of	
  service	
  
integration	
  
• A	
  short	
  staff	
  measure	
  based	
  on	
  
Normalisation	
  Process	
  Theory	
  
• People	
  and	
  organisations’	
  readiness	
  to	
  
innovate	
  
• People’s	
  confidence	
  in	
  using	
  digital	
  health	
  
care	
  solutions	
  
	
  
	
  
November	
  2017	
  
	
  
	
  
  	
  
For	
  more	
  information,	
  contact	
  the	
  authors:	
  
	
  
Andrew	
  Liles,	
  Director,	
  R-­‐Outcomes,	
  andrew.liles@r-­‐outcomes.com	
  
Phlippa	
  Darnton,	
  Evaluation	
  Lead,	
  Wessex	
  AHSN,	
  
philippa.darnton@wessexahsn.net	
  
Dr	
  Andrew	
  Sibley,	
  Evaluation	
  Programme	
  Manager,	
  Wessex	
  AHSN,	
  
Andrew.sibley@wessexahsn.net	
  
Dr	
  Catherine	
  Matheson-­‐Monnet,	
  Senior	
  Research	
  Fellow,	
  Centre	
  for	
  
Implementation	
  Science,	
  University	
  of	
  Southampton,	
  	
  
c.b.matheson@soton.ac.uk	
  
References:	
  
1.	
  New	
  Care	
  Models.	
  (June	
  2017)	
  The	
  local	
  evaluation	
  of	
  the	
  new	
  care	
  models	
  vanguards:	
  our	
  
expectations	
  and	
  offer	
  of	
  support	
  for	
  2017/18.	
  http://www.england.nhs.uk/vanguards	
  
2.	
  Liles	
  A,	
  Darnton	
  P	
  (2017)	
  Social	
  Prescribing	
  in	
  Wessex;	
  Understanding	
  its	
  impact	
  and	
  supporting	
  its	
  
spread.	
  	
  Wessex	
  AHSN.	
  https//www.r-­‐outcomes.com/publications	
  
3.	
  Benson	
  T,	
  Potts	
  HWW.	
  A	
  short	
  generic	
  patient	
  experience	
  questionnaire:	
  howRwe	
  development	
  
and	
  validation	
  (2014).	
  BMC	
  Health	
  Services	
  Research	
  2014,	
  14:499	
  .	
  
http://www.biomedcentral.com/1472-­‐6963/14/499	
  
4.	
  Benson	
  T,	
  Whatling	
  J,	
  Arikan	
  S,	
  Sizmur	
  S,	
  McDonald	
  D,	
  Ingram	
  D:	
  Evaluation	
  of	
  a	
  new	
  short	
  generic	
  
measure	
  of	
  HRQoL:	
  howRu.	
  Informatics	
  in	
  Primary	
  Care	
  2010;	
  18:89-­‐101.	
  https://hijournal.b	
  
cs.org/index.php/jhi/article/view/758/770	
  
5.	
  Benson	
  T,	
  Bowman	
  C,	
  Sladen	
  J,	
  Liles	
  A,	
  Potts	
  HWW.	
  Health	
  Confidence	
  Score	
  (HCS)	
  –	
  Development	
  
and	
  Validation.	
  Research	
  Paper	
  17/02	
  September	
  2017.	
  https://www.r-­‐outcomes.com/publications	
  
6.	
  Braun,	
  V.	
  and	
  Clarke,	
  V.	
  (2006)	
  Using	
  thematic	
  analysis	
  in	
  psychology.	
  Qualitative	
  Research	
  in	
  
Psychology,	
  3	
  (2).	
  pp.	
  77-­‐101.	
  www.eprints.uwe.ac.uk/11735/2/thematic_analysis_revised.	
  	
  
7.	
  May,	
  C.	
  and	
  Finch,	
  T.	
  (2009)	
  Implementation,	
  embedding,	
  and	
  integration:	
  an	
  outline	
  of	
  
Normalization	
  Process	
  Theory.	
  Sociology	
  43	
  (3):	
  535-­‐55http://www.normalizationprocess.org	
  
8.	
  Finch	
  et	
  al.	
  (2013)	
  Improving	
  the	
  normalization	
  of	
  complex	
  interventions:	
  measure	
  development	
  
based	
  on	
  normalization	
  process	
  theory	
  (NoMAD):	
  study	
  protocol.	
  Implementation	
  
Science,	
  8,	
  1,	
  43.	
  doi:10.1186/1748-­‐5908-­‐8-­‐43	
  
9.	
  Completed	
  vanguard	
  evaluation	
  reports	
  for	
  Farnham	
  Integrated	
  Care	
  Team,	
  Referral	
  Management	
  
Service	
  and	
  Pre-­‐Diabetes	
  Education	
  Programme	
  and	
  for	
  a	
  Recovery	
  College;	
  www.wahsn.org.uk	
  
	
  
	
  
	
  
	
  
	
  
	
  
	
  
	
  
	
  
	
  
Acknowledging	
  the	
  evaluation	
  team	
  
Dr	
  David	
  Kryl,	
  Director	
  of	
  Insight,	
  Centre	
  for	
  Implementation	
  Science	
  (CIS)/	
  Wessex	
  AHSN	
  
Joe	
  Sladen,	
  Evaluation	
  Programme	
  Manager,	
  Wessex	
  AHSN	
  
Sydney	
  Anstee,	
  Research	
  Consultant,	
  Wessex	
  AHSN	
  
Cindy	
  Brooks,	
  Research	
  Fellow,	
  CIS,	
  University	
  of	
  Southampton	
  
Tim	
  Benson,	
  Director,	
  R-­‐Outcomes	
  
Jess	
  Done,	
  Project	
  Manager,	
  R-­‐Outcomes	
  
 
Evaluating	
  the	
  impact	
  of	
  new	
  care	
  models	
  on	
  people	
  
8	
  
Appendix	
  –	
  R-­‐Outcome	
  patient	
  measures	
  
	
  
Patient-Reported Outcome Measures
R-Outcomes’ family of short generic patient-reported measures cover
health status, patient experience, personal wellbeing and health
confidence. They can be used at the point of care or between visits.
These share a common framework with 4 items and 4 responses,
suitable for use on a patient’s own smart-phone, tablet, PC
or on paper.
These tools are research-based and are short, quick and easy to use.
They are generic and suitable for almost all patients irrespective of
conditions across health and social care. The results measure trends,
changes and comparisons.
Results are easy to interpret, giving feedback to patients, clinicians,
managers and commissioners, tracking changes and differences
between units.
These validated tools are short and quick to use with simple
unambiguous wording, understood by those whose first language is
not English.
PATIENT
© R-Outcomes Ltd 2017 www.r-outcomes.com info@r-outcomes.com
Health Status
HowRu is a short generic patient-reported
outcome measure (PROM), to track and
compare patients’ perceptions of how they
feel physically and mentally and what they
can do (disability and dependence).
Experience
HowRwe is a short generic patient-reported
experience measure (PREM), which
measures patients perceptions of the care
and service provided. It is suitable for all
types of patient and care setting.
Confidence
HCS captures people’s confidence in their
knowledge, self-management, access to
help and shared decision-making.
Wellbeing
The Personal Wellbeing Score (PWS),
based on National Statistics ONS4, covers
life evaluation, worthwhileness, positive
and negative experience.
Service Integration
Integration across service boundaries is
a challenge for all health services. It is a
priority for new models of care.
Length
Name Items Words
Reading
age
Outcome
howRu 4 37 7
EQ-5d (inc. visual analogue scale) 6 230 11
SF-12 12 474 11
NHS PROMS (Hip pre-op) 27 1,485 11
Experience
howRwe 4 29 7
NHS Friends and Family Test 1 44 12
HCAHPS 32 1,156 13
GP Patient Survey 62 2,922 12
NHS Adult Inpatient Survey 76 3,353 12
Engagement
Health Confidence Score 4 50 8
Patient Activation Measure (PAM) 13 293 12
Health Literacy Questionnaire 44 1,001 12
Wellbeing
Personal Wellbeing Score 4 43 9
ONS Personal Well-being 4 95 11
Short Warwick-Edinburgh 7 89 8
ICECAP-A 5 273 10
Tel: 0785 568 2037
Require help from others
How are you today? (past 24 hours)
Limited in what you can do
Feeling low or worried
Pain or discomfort
None A little
Quite
a lot
Extreme
How do you feel and how much can you do?
Choose one answer on each line
How are we doing?
See you promptly
Well organised
Listen and explain
Treat you kindly
Excellent Good Fair Poor
What do you think about our service?
Choose one answer on each line
Health Confidence
I can get the right help if I need it
I am involved in decisions about me
I can look after my health
I know enough about my health
Strongly
agree
Agree Neutral Disagree
How do you feel about caring for your health?
How much do you agree?
Personal Wellbeing
I was happy yesterday
I was NOT anxious yesterday
What I do in my life is worthwhile
I am satisfied with my life
Strongly
agree
Agree Neutral Disagree
How are you feeling in general?
How much do you agree?
Service Integration (patient view)
I don’t have to repeat my story
Different services work well together
Staff know what other services do
Services talk to each other
Strongly
agree
Agree Neutral Disagree
How well do services work together?
How much do you agree?
Length and reading age affect respondent
burden and response rates.

More Related Content

What's hot

nursing process Evaluation
nursing process Evaluationnursing process Evaluation
nursing process Evaluation
Shaells Joshi
 
Nursing process (Assessment)
Nursing process (Assessment)Nursing process (Assessment)
Nursing process (Assessment)
Malek Hroub
 
Nursing diagnosis
Nursing diagnosisNursing diagnosis
Nursing diagnosis
Arul Valan
 

What's hot (20)

nursing process Evaluation
nursing process Evaluationnursing process Evaluation
nursing process Evaluation
 
Nursing care
Nursing care Nursing care
Nursing care
 
Operationalizing Clinical Excellence: Lessons Learned
Operationalizing Clinical Excellence: Lessons LearnedOperationalizing Clinical Excellence: Lessons Learned
Operationalizing Clinical Excellence: Lessons Learned
 
The Strategic Plan for Children's National Health System: ICU/ER Satellite La...
The Strategic Plan for Children's National Health System: ICU/ER Satellite La...The Strategic Plan for Children's National Health System: ICU/ER Satellite La...
The Strategic Plan for Children's National Health System: ICU/ER Satellite La...
 
Nursing-process
 Nursing-process Nursing-process
Nursing-process
 
Nursing process, deepani nanayakkara, nursing tutor, srilanka
Nursing process,  deepani nanayakkara, nursing tutor, srilanka Nursing process,  deepani nanayakkara, nursing tutor, srilanka
Nursing process, deepani nanayakkara, nursing tutor, srilanka
 
Basics of Nursing Care Plan
Basics of Nursing Care PlanBasics of Nursing Care Plan
Basics of Nursing Care Plan
 
Nursing process
Nursing processNursing process
Nursing process
 
Nursing Process in psychiatric nursing
Nursing Process in psychiatric nursingNursing Process in psychiatric nursing
Nursing Process in psychiatric nursing
 
The Saskatchewan Surgical Initiative: Lessons Learned
The Saskatchewan Surgical Initiative: Lessons LearnedThe Saskatchewan Surgical Initiative: Lessons Learned
The Saskatchewan Surgical Initiative: Lessons Learned
 
Nursing Process
Nursing ProcessNursing Process
Nursing Process
 
Organizing nursing services and patient care
Organizing nursing services and patient careOrganizing nursing services and patient care
Organizing nursing services and patient care
 
Annette Bartley: Making it happen - Intentional Rounding
Annette Bartley: Making it happen - Intentional RoundingAnnette Bartley: Making it happen - Intentional Rounding
Annette Bartley: Making it happen - Intentional Rounding
 
Nursing process
Nursing processNursing process
Nursing process
 
Unit 5 nursing process
Unit   5 nursing processUnit   5 nursing process
Unit 5 nursing process
 
Nursing process (Assessment)
Nursing process (Assessment)Nursing process (Assessment)
Nursing process (Assessment)
 
Nursing diagnosis
Nursing diagnosisNursing diagnosis
Nursing diagnosis
 
Nursing diagnosis
Nursing diagnosisNursing diagnosis
Nursing diagnosis
 
Evaluation nsg. process
Evaluation nsg. processEvaluation nsg. process
Evaluation nsg. process
 
Nursing process
Nursing process Nursing process
Nursing process
 

Similar to Wessex AHSN: Evaluating patient outcomes in Wessex

Innovation in Care Delivery: The Patient Journey
Innovation in Care Delivery: The Patient JourneyInnovation in Care Delivery: The Patient Journey
Innovation in Care Delivery: The Patient Journey
Jane Chiang
 
PED 138 – Cardio Variety – Home Assignment For the Week of 31620.docx
PED 138 – Cardio Variety – Home Assignment For the Week of 31620.docxPED 138 – Cardio Variety – Home Assignment For the Week of 31620.docx
PED 138 – Cardio Variety – Home Assignment For the Week of 31620.docx
karlhennesey
 
DrSteeleWhitePaper_v7
DrSteeleWhitePaper_v7DrSteeleWhitePaper_v7
DrSteeleWhitePaper_v7
Kate Williams
 
Comment 1Development of an evidence-based practice project mus
Comment 1Development of an evidence-based practice project musComment 1Development of an evidence-based practice project mus
Comment 1Development of an evidence-based practice project mus
JeniceStuckeyoo
 
Job Redesign For Expanded HIM Functions Case Study...
Job Redesign For Expanded HIM Functions Case Study...Job Redesign For Expanded HIM Functions Case Study...
Job Redesign For Expanded HIM Functions Case Study...
Amanda Hengel
 
Prudent healthcare and patient activation (1)
Prudent healthcare and patient activation (1)Prudent healthcare and patient activation (1)
Prudent healthcare and patient activation (1)
Andrew Rix
 
NURSING ADMINISTRATION Care Redesign Discussion.pdf
NURSING ADMINISTRATION Care Redesign Discussion.pdfNURSING ADMINISTRATION Care Redesign Discussion.pdf
NURSING ADMINISTRATION Care Redesign Discussion.pdf
bkbk37
 
Building_the_case_of_clinical_care_in_the_Home
Building_the_case_of_clinical_care_in_the_HomeBuilding_the_case_of_clinical_care_in_the_Home
Building_the_case_of_clinical_care_in_the_Home
Sara Cain
 
Care Redesign Article and Answer the following questions.pdf
Care Redesign Article and Answer the following questions.pdfCare Redesign Article and Answer the following questions.pdf
Care Redesign Article and Answer the following questions.pdf
bkbk37
 
Measuring job satisfaction and impact of demographic characteristics among Do...
Measuring job satisfaction and impact of demographic characteristics among Do...Measuring job satisfaction and impact of demographic characteristics among Do...
Measuring job satisfaction and impact of demographic characteristics among Do...
International Multispeciality Journal of Health
 
Data Collection Plan .docx
Data Collection Plan                                            .docxData Collection Plan                                            .docx
Data Collection Plan .docx
theodorelove43763
 
Vital Clinical Role in The US Healthcare System Article Analysis.pdf
Vital Clinical Role in The US Healthcare System Article Analysis.pdfVital Clinical Role in The US Healthcare System Article Analysis.pdf
Vital Clinical Role in The US Healthcare System Article Analysis.pdf
sdfghj21
 

Similar to Wessex AHSN: Evaluating patient outcomes in Wessex (20)

Ware DEFINING AND MEASURING PATIENT SATISFACTION 1983.pdf
Ware DEFINING AND MEASURING PATIENT SATISFACTION 1983.pdfWare DEFINING AND MEASURING PATIENT SATISFACTION 1983.pdf
Ware DEFINING AND MEASURING PATIENT SATISFACTION 1983.pdf
 
Innovation in Care Delivery: The Patient Journey
Innovation in Care Delivery: The Patient JourneyInnovation in Care Delivery: The Patient Journey
Innovation in Care Delivery: The Patient Journey
 
PED 138 – Cardio Variety – Home Assignment For the Week of 31620.docx
PED 138 – Cardio Variety – Home Assignment For the Week of 31620.docxPED 138 – Cardio Variety – Home Assignment For the Week of 31620.docx
PED 138 – Cardio Variety – Home Assignment For the Week of 31620.docx
 
DrSteeleWhitePaper_v7
DrSteeleWhitePaper_v7DrSteeleWhitePaper_v7
DrSteeleWhitePaper_v7
 
Innovation & Recent trends in Nursing
Innovation & Recent trends in NursingInnovation & Recent trends in Nursing
Innovation & Recent trends in Nursing
 
Patient Experience June 2015
Patient Experience June 2015Patient Experience June 2015
Patient Experience June 2015
 
Comment 1Development of an evidence-based practice project mus
Comment 1Development of an evidence-based practice project musComment 1Development of an evidence-based practice project mus
Comment 1Development of an evidence-based practice project mus
 
Importance Of Evidence Based Practice In Nursing
Importance Of Evidence Based Practice In NursingImportance Of Evidence Based Practice In Nursing
Importance Of Evidence Based Practice In Nursing
 
Streatham High Surgery - Outstanding CQC Inspection Nov 2014
Streatham High Surgery - Outstanding CQC Inspection Nov 2014Streatham High Surgery - Outstanding CQC Inspection Nov 2014
Streatham High Surgery - Outstanding CQC Inspection Nov 2014
 
Job Redesign For Expanded HIM Functions Case Study...
Job Redesign For Expanded HIM Functions Case Study...Job Redesign For Expanded HIM Functions Case Study...
Job Redesign For Expanded HIM Functions Case Study...
 
This week will see the launch of Health Outcomes Insights (Formerly DHP Resea...
This week will see the launch of Health Outcomes Insights (Formerly DHP Resea...This week will see the launch of Health Outcomes Insights (Formerly DHP Resea...
This week will see the launch of Health Outcomes Insights (Formerly DHP Resea...
 
Prudent healthcare and patient activation (1)
Prudent healthcare and patient activation (1)Prudent healthcare and patient activation (1)
Prudent healthcare and patient activation (1)
 
Research.docx
Research.docxResearch.docx
Research.docx
 
NURSING ADMINISTRATION Care Redesign Discussion.pdf
NURSING ADMINISTRATION Care Redesign Discussion.pdfNURSING ADMINISTRATION Care Redesign Discussion.pdf
NURSING ADMINISTRATION Care Redesign Discussion.pdf
 
Initial post week 12
Initial post week 12 Initial post week 12
Initial post week 12
 
Building_the_case_of_clinical_care_in_the_Home
Building_the_case_of_clinical_care_in_the_HomeBuilding_the_case_of_clinical_care_in_the_Home
Building_the_case_of_clinical_care_in_the_Home
 
Care Redesign Article and Answer the following questions.pdf
Care Redesign Article and Answer the following questions.pdfCare Redesign Article and Answer the following questions.pdf
Care Redesign Article and Answer the following questions.pdf
 
Measuring job satisfaction and impact of demographic characteristics among Do...
Measuring job satisfaction and impact of demographic characteristics among Do...Measuring job satisfaction and impact of demographic characteristics among Do...
Measuring job satisfaction and impact of demographic characteristics among Do...
 
Data Collection Plan .docx
Data Collection Plan                                            .docxData Collection Plan                                            .docx
Data Collection Plan .docx
 
Vital Clinical Role in The US Healthcare System Article Analysis.pdf
Vital Clinical Role in The US Healthcare System Article Analysis.pdfVital Clinical Role in The US Healthcare System Article Analysis.pdf
Vital Clinical Role in The US Healthcare System Article Analysis.pdf
 

More from Health Innovation Wessex

More from Health Innovation Wessex (20)

Reducing opioid prescribing (in general practice)
Reducing opioid prescribing (in general practice)Reducing opioid prescribing (in general practice)
Reducing opioid prescribing (in general practice)
 
Pharmacist Interventions and Medication Reviews at Care Homes - Improving Med...
Pharmacist Interventions and Medication Reviews at Care Homes - Improving Med...Pharmacist Interventions and Medication Reviews at Care Homes - Improving Med...
Pharmacist Interventions and Medication Reviews at Care Homes - Improving Med...
 
The SBAR Patient Engagement Pathway Tool
The SBAR Patient Engagement Pathway ToolThe SBAR Patient Engagement Pathway Tool
The SBAR Patient Engagement Pathway Tool
 
Reducing medication related falls risk in patients with severe frailty
Reducing medication related falls risk in patients with severe frailtyReducing medication related falls risk in patients with severe frailty
Reducing medication related falls risk in patients with severe frailty
 
Assessing the outcomes of structured medication reviews
Assessing the outcomes of structured medication reviewsAssessing the outcomes of structured medication reviews
Assessing the outcomes of structured medication reviews
 
Polypharmacy SMR reviews in outpatient bone health clinics
Polypharmacy SMR reviews in outpatient bone health clinicsPolypharmacy SMR reviews in outpatient bone health clinics
Polypharmacy SMR reviews in outpatient bone health clinics
 
Polypharmacy reviews of asthma and COPD patients over 65 and 10 or more medic...
Polypharmacy reviews of asthma and COPD patients over 65 and 10 or more medic...Polypharmacy reviews of asthma and COPD patients over 65 and 10 or more medic...
Polypharmacy reviews of asthma and COPD patients over 65 and 10 or more medic...
 
Evaluating the impact of a specialist frailty multidisciplinary team pathway ...
Evaluating the impact of a specialist frailty multidisciplinary team pathway ...Evaluating the impact of a specialist frailty multidisciplinary team pathway ...
Evaluating the impact of a specialist frailty multidisciplinary team pathway ...
 
Genomics strategy and implementation in the NHS in England
Genomics strategy and implementation in the NHS in EnglandGenomics strategy and implementation in the NHS in England
Genomics strategy and implementation in the NHS in England
 
Pharmacogenomics into practice - stroke services and a systems approach
Pharmacogenomics into practice - stroke services and a systems approachPharmacogenomics into practice - stroke services and a systems approach
Pharmacogenomics into practice - stroke services and a systems approach
 
Pharmacogenomics in Practice - Dr Hayley Wickens
Pharmacogenomics in Practice - Dr Hayley WickensPharmacogenomics in Practice - Dr Hayley Wickens
Pharmacogenomics in Practice - Dr Hayley Wickens
 
Working with patients to support transformational healthcare
Working with patients to support transformational healthcareWorking with patients to support transformational healthcare
Working with patients to support transformational healthcare
 
The Generation study by Alice Tuff-Lacey
The Generation study by Alice Tuff-LaceyThe Generation study by Alice Tuff-Lacey
The Generation study by Alice Tuff-Lacey
 
To evaluate the benefits of Structured Medication Reviews in elderly Chinese ...
To evaluate the benefits of Structured Medication Reviews in elderly Chinese ...To evaluate the benefits of Structured Medication Reviews in elderly Chinese ...
To evaluate the benefits of Structured Medication Reviews in elderly Chinese ...
 
Review of patients on high dose opioids at Living Well PCN.pdf
Review of patients on high dose opioids at Living Well PCN.pdfReview of patients on high dose opioids at Living Well PCN.pdf
Review of patients on high dose opioids at Living Well PCN.pdf
 
Re-establishing autonomy in elderly frail patients.pdf
Re-establishing autonomy in elderly frail patients.pdfRe-establishing autonomy in elderly frail patients.pdf
Re-establishing autonomy in elderly frail patients.pdf
 
Improving Medication Reviews using the NO TEARS Tool.pdf
Improving Medication Reviews using the NO TEARS Tool.pdfImproving Medication Reviews using the NO TEARS Tool.pdf
Improving Medication Reviews using the NO TEARS Tool.pdf
 
Improving care in County Durham under the STOMP agenda - A 5 year review.pdf
Improving care in County Durham under the STOMP agenda - A 5 year review.pdfImproving care in County Durham under the STOMP agenda - A 5 year review.pdf
Improving care in County Durham under the STOMP agenda - A 5 year review.pdf
 
Impact of an EMIS search to prioritise care home residents for a pharmacist l...
Impact of an EMIS search to prioritise care home residents for a pharmacist l...Impact of an EMIS search to prioritise care home residents for a pharmacist l...
Impact of an EMIS search to prioritise care home residents for a pharmacist l...
 
Identifying Orthostatic Hypotension caused by Medication.pdf
Identifying Orthostatic Hypotension caused by Medication.pdfIdentifying Orthostatic Hypotension caused by Medication.pdf
Identifying Orthostatic Hypotension caused by Medication.pdf
 

Recently uploaded

Call Girl Service In Mumbai ❤️🍑 9xx000xx09 👄🫦Independent Escort Service Mumba...
Call Girl Service In Mumbai ❤️🍑 9xx000xx09 👄🫦Independent Escort Service Mumba...Call Girl Service In Mumbai ❤️🍑 9xx000xx09 👄🫦Independent Escort Service Mumba...
Call Girl Service In Mumbai ❤️🍑 9xx000xx09 👄🫦Independent Escort Service Mumba...
Sheetaleventcompany
 
❤️ Zirakpur Call Girl Service ☎️9878799926☎️ Call Girl service in Zirakpur ☎...
❤️ Zirakpur Call Girl Service  ☎️9878799926☎️ Call Girl service in Zirakpur ☎...❤️ Zirakpur Call Girl Service  ☎️9878799926☎️ Call Girl service in Zirakpur ☎...
❤️ Zirakpur Call Girl Service ☎️9878799926☎️ Call Girl service in Zirakpur ☎...
daljeetkaur2026
 
Low Rate Call Girls Jaipur {9521753030} ❤️VVIP NISHA CCall Girls in Jaipur Es...
Low Rate Call Girls Jaipur {9521753030} ❤️VVIP NISHA CCall Girls in Jaipur Es...Low Rate Call Girls Jaipur {9521753030} ❤️VVIP NISHA CCall Girls in Jaipur Es...
Low Rate Call Girls Jaipur {9521753030} ❤️VVIP NISHA CCall Girls in Jaipur Es...
Sheetaleventcompany
 
💚 Low Rate Call Girls In Chandigarh 💯Lucky 📲🔝8868886958🔝Call Girl In Chandig...
💚 Low Rate  Call Girls In Chandigarh 💯Lucky 📲🔝8868886958🔝Call Girl In Chandig...💚 Low Rate  Call Girls In Chandigarh 💯Lucky 📲🔝8868886958🔝Call Girl In Chandig...
💚 Low Rate Call Girls In Chandigarh 💯Lucky 📲🔝8868886958🔝Call Girl In Chandig...
Sheetaleventcompany
 
Lucknow Call Girls Service ❤️🍑 9xx000xx09 👄🫦 Independent Escort Service Luckn...
Lucknow Call Girls Service ❤️🍑 9xx000xx09 👄🫦 Independent Escort Service Luckn...Lucknow Call Girls Service ❤️🍑 9xx000xx09 👄🫦 Independent Escort Service Luckn...
Lucknow Call Girls Service ❤️🍑 9xx000xx09 👄🫦 Independent Escort Service Luckn...
Sheetaleventcompany
 
Premium Call Girls Bangalore {9179660964} ❤️VVIP POOJA Call Girls in Bangalor...
Premium Call Girls Bangalore {9179660964} ❤️VVIP POOJA Call Girls in Bangalor...Premium Call Girls Bangalore {9179660964} ❤️VVIP POOJA Call Girls in Bangalor...
Premium Call Girls Bangalore {9179660964} ❤️VVIP POOJA Call Girls in Bangalor...
Sheetaleventcompany
 
Indore Call Girl Service 📞9235973566📞Just Call Inaaya📲 Call Girls In Indore N...
Indore Call Girl Service 📞9235973566📞Just Call Inaaya📲 Call Girls In Indore N...Indore Call Girl Service 📞9235973566📞Just Call Inaaya📲 Call Girls In Indore N...
Indore Call Girl Service 📞9235973566📞Just Call Inaaya📲 Call Girls In Indore N...
Sheetaleventcompany
 
Delhi Call Girl Service 📞8650700400📞Just Call Divya📲 Call Girl In Delhi No💰Ad...
Delhi Call Girl Service 📞8650700400📞Just Call Divya📲 Call Girl In Delhi No💰Ad...Delhi Call Girl Service 📞8650700400📞Just Call Divya📲 Call Girl In Delhi No💰Ad...
Delhi Call Girl Service 📞8650700400📞Just Call Divya📲 Call Girl In Delhi No💰Ad...
Sheetaleventcompany
 
Low Rate Call Girls Udaipur {9xx000xx09} ❤️VVIP NISHA CCall Girls in Udaipur ...
Low Rate Call Girls Udaipur {9xx000xx09} ❤️VVIP NISHA CCall Girls in Udaipur ...Low Rate Call Girls Udaipur {9xx000xx09} ❤️VVIP NISHA CCall Girls in Udaipur ...
Low Rate Call Girls Udaipur {9xx000xx09} ❤️VVIP NISHA CCall Girls in Udaipur ...
Sheetaleventcompany
 

Recently uploaded (20)

Call Girl Service In Mumbai ❤️🍑 9xx000xx09 👄🫦Independent Escort Service Mumba...
Call Girl Service In Mumbai ❤️🍑 9xx000xx09 👄🫦Independent Escort Service Mumba...Call Girl Service In Mumbai ❤️🍑 9xx000xx09 👄🫦Independent Escort Service Mumba...
Call Girl Service In Mumbai ❤️🍑 9xx000xx09 👄🫦Independent Escort Service Mumba...
 
💸Cash Payment No Advance Call Girls Nagpur 🧿 9332606886 🧿 High Class Call Gir...
💸Cash Payment No Advance Call Girls Nagpur 🧿 9332606886 🧿 High Class Call Gir...💸Cash Payment No Advance Call Girls Nagpur 🧿 9332606886 🧿 High Class Call Gir...
💸Cash Payment No Advance Call Girls Nagpur 🧿 9332606886 🧿 High Class Call Gir...
 
❤️ Zirakpur Call Girl Service ☎️9878799926☎️ Call Girl service in Zirakpur ☎...
❤️ Zirakpur Call Girl Service  ☎️9878799926☎️ Call Girl service in Zirakpur ☎...❤️ Zirakpur Call Girl Service  ☎️9878799926☎️ Call Girl service in Zirakpur ☎...
❤️ Zirakpur Call Girl Service ☎️9878799926☎️ Call Girl service in Zirakpur ☎...
 
Low Rate Call Girls Jaipur {9521753030} ❤️VVIP NISHA CCall Girls in Jaipur Es...
Low Rate Call Girls Jaipur {9521753030} ❤️VVIP NISHA CCall Girls in Jaipur Es...Low Rate Call Girls Jaipur {9521753030} ❤️VVIP NISHA CCall Girls in Jaipur Es...
Low Rate Call Girls Jaipur {9521753030} ❤️VVIP NISHA CCall Girls in Jaipur Es...
 
💚 Low Rate Call Girls In Chandigarh 💯Lucky 📲🔝8868886958🔝Call Girl In Chandig...
💚 Low Rate  Call Girls In Chandigarh 💯Lucky 📲🔝8868886958🔝Call Girl In Chandig...💚 Low Rate  Call Girls In Chandigarh 💯Lucky 📲🔝8868886958🔝Call Girl In Chandig...
💚 Low Rate Call Girls In Chandigarh 💯Lucky 📲🔝8868886958🔝Call Girl In Chandig...
 
❤️ Call Girls service In Panchkula☎️9815457724☎️ Call Girl service in Panchku...
❤️ Call Girls service In Panchkula☎️9815457724☎️ Call Girl service in Panchku...❤️ Call Girls service In Panchkula☎️9815457724☎️ Call Girl service in Panchku...
❤️ Call Girls service In Panchkula☎️9815457724☎️ Call Girl service in Panchku...
 
💞 Safe And Secure Call Girls Mysore 🧿 9332606886 🧿 High Class Call Girl Servi...
💞 Safe And Secure Call Girls Mysore 🧿 9332606886 🧿 High Class Call Girl Servi...💞 Safe And Secure Call Girls Mysore 🧿 9332606886 🧿 High Class Call Girl Servi...
💞 Safe And Secure Call Girls Mysore 🧿 9332606886 🧿 High Class Call Girl Servi...
 
💞 Safe And Secure Call Girls gaya 🧿 9332606886 🧿 High Class Call Girl Service...
💞 Safe And Secure Call Girls gaya 🧿 9332606886 🧿 High Class Call Girl Service...💞 Safe And Secure Call Girls gaya 🧿 9332606886 🧿 High Class Call Girl Service...
💞 Safe And Secure Call Girls gaya 🧿 9332606886 🧿 High Class Call Girl Service...
 
Lucknow Call Girls Service ❤️🍑 9xx000xx09 👄🫦 Independent Escort Service Luckn...
Lucknow Call Girls Service ❤️🍑 9xx000xx09 👄🫦 Independent Escort Service Luckn...Lucknow Call Girls Service ❤️🍑 9xx000xx09 👄🫦 Independent Escort Service Luckn...
Lucknow Call Girls Service ❤️🍑 9xx000xx09 👄🫦 Independent Escort Service Luckn...
 
❤️Chandigarh Escort Service☎️9814379184☎️ Call Girl service in Chandigarh☎️ C...
❤️Chandigarh Escort Service☎️9814379184☎️ Call Girl service in Chandigarh☎️ C...❤️Chandigarh Escort Service☎️9814379184☎️ Call Girl service in Chandigarh☎️ C...
❤️Chandigarh Escort Service☎️9814379184☎️ Call Girl service in Chandigarh☎️ C...
 
❤️Chandigarh Escorts Service☎️9815457724☎️ Call Girl service in Chandigarh☎️ ...
❤️Chandigarh Escorts Service☎️9815457724☎️ Call Girl service in Chandigarh☎️ ...❤️Chandigarh Escorts Service☎️9815457724☎️ Call Girl service in Chandigarh☎️ ...
❤️Chandigarh Escorts Service☎️9815457724☎️ Call Girl service in Chandigarh☎️ ...
 
❤️Chandigarh Escorts☎️9814379184☎️ Call Girl service in Chandigarh☎️ Chandiga...
❤️Chandigarh Escorts☎️9814379184☎️ Call Girl service in Chandigarh☎️ Chandiga...❤️Chandigarh Escorts☎️9814379184☎️ Call Girl service in Chandigarh☎️ Chandiga...
❤️Chandigarh Escorts☎️9814379184☎️ Call Girl service in Chandigarh☎️ Chandiga...
 
💸Cash Payment No Advance Call Girls Pune 🧿 9332606886 🧿 High Class Call Girl ...
💸Cash Payment No Advance Call Girls Pune 🧿 9332606886 🧿 High Class Call Girl ...💸Cash Payment No Advance Call Girls Pune 🧿 9332606886 🧿 High Class Call Girl ...
💸Cash Payment No Advance Call Girls Pune 🧿 9332606886 🧿 High Class Call Girl ...
 
Premium Call Girls Bangalore {9179660964} ❤️VVIP POOJA Call Girls in Bangalor...
Premium Call Girls Bangalore {9179660964} ❤️VVIP POOJA Call Girls in Bangalor...Premium Call Girls Bangalore {9179660964} ❤️VVIP POOJA Call Girls in Bangalor...
Premium Call Girls Bangalore {9179660964} ❤️VVIP POOJA Call Girls in Bangalor...
 
❤️Chandigarh Escort Service☎️9815457724☎️ Call Girl service in Chandigarh☎️ C...
❤️Chandigarh Escort Service☎️9815457724☎️ Call Girl service in Chandigarh☎️ C...❤️Chandigarh Escort Service☎️9815457724☎️ Call Girl service in Chandigarh☎️ C...
❤️Chandigarh Escort Service☎️9815457724☎️ Call Girl service in Chandigarh☎️ C...
 
Indore Call Girl Service 📞9235973566📞Just Call Inaaya📲 Call Girls In Indore N...
Indore Call Girl Service 📞9235973566📞Just Call Inaaya📲 Call Girls In Indore N...Indore Call Girl Service 📞9235973566📞Just Call Inaaya📲 Call Girls In Indore N...
Indore Call Girl Service 📞9235973566📞Just Call Inaaya📲 Call Girls In Indore N...
 
💸Cash Payment No Advance Call Girls Hyderabad 🧿 9332606886 🧿 High Class Call ...
💸Cash Payment No Advance Call Girls Hyderabad 🧿 9332606886 🧿 High Class Call ...💸Cash Payment No Advance Call Girls Hyderabad 🧿 9332606886 🧿 High Class Call ...
💸Cash Payment No Advance Call Girls Hyderabad 🧿 9332606886 🧿 High Class Call ...
 
Delhi Call Girl Service 📞8650700400📞Just Call Divya📲 Call Girl In Delhi No💰Ad...
Delhi Call Girl Service 📞8650700400📞Just Call Divya📲 Call Girl In Delhi No💰Ad...Delhi Call Girl Service 📞8650700400📞Just Call Divya📲 Call Girl In Delhi No💰Ad...
Delhi Call Girl Service 📞8650700400📞Just Call Divya📲 Call Girl In Delhi No💰Ad...
 
Low Rate Call Girls Udaipur {9xx000xx09} ❤️VVIP NISHA CCall Girls in Udaipur ...
Low Rate Call Girls Udaipur {9xx000xx09} ❤️VVIP NISHA CCall Girls in Udaipur ...Low Rate Call Girls Udaipur {9xx000xx09} ❤️VVIP NISHA CCall Girls in Udaipur ...
Low Rate Call Girls Udaipur {9xx000xx09} ❤️VVIP NISHA CCall Girls in Udaipur ...
 
💸Cash Payment No Advance Call Girls Kolkata 🧿 9332606886 🧿 High Class Call Gi...
💸Cash Payment No Advance Call Girls Kolkata 🧿 9332606886 🧿 High Class Call Gi...💸Cash Payment No Advance Call Girls Kolkata 🧿 9332606886 🧿 High Class Call Gi...
💸Cash Payment No Advance Call Girls Kolkata 🧿 9332606886 🧿 High Class Call Gi...
 

Wessex AHSN: Evaluating patient outcomes in Wessex

  • 1.   How  we  are  evaluating  the  impact  of  new  care   models  on  how  people  feel  in  Wessex     Introduction   When  the  New  Care  Models  (NCM)1   evaluation   team  set  out  their  priorities  and  expectations  for   vanguard  evaluation  in  2017/18,  they  described   a  need  to  strengthen  the  focus  on  evaluating   patients’  experience  of  transformed  services   (New  Care  Models  June  2017).   Wessex  AHSN  are  working  in  partnership  with   the  Centre  for  Implementation  Science,   University  of  Southampton  (CIS)  and  R-­‐ Outcomes  Ltd  to  evaluate  two  PACS  vanguards   in  North  East  Hampshire  and  Farnham  (NEHF)   and  the  Isle  of  Wight  (IoW)  and  a  number  of   other  new  services  and  care  models  across   Wessex.    We  have  completed  a  number  of   detailed  evaluations,  including  social  prescribing   and  integrated  care  team  models,  with  a  strong   focus  on  the  impact  on  patients,  people,  carers   and  staff.       This  short  report  describes  the  mixed-­‐methods   approach  to  evaluation  that  we  have  used,  what   we  have  found  and  how  this  has  helped  develop   new  care  models.    Our  approach  has  been  to   consistently  do  five  things  to  understand  how   people  feel  about  new  care  models  (NCMs)2 :   • Collect  and  analyse  large  numbers  of  R-­‐ Outcomes  from  people,  patients  and  staff   • Undertake  qualitative  interviews  with   patients,  carers  and  staff   • Thematic  analysis  of  case  studies   • Observe  teams  at  work  using  Normalisation   Process  Theory   • Synthesise  this  data  to  develop  triangulated,   rounded  findings  and  evidence  of  impact           Methods   1. R-­‐Outcomes   We  knew  we  wanted  self-­‐reported  outcomes  to   play  a  major  role  in  our  evaluation  of  NCMs.  We   understood  that  an  intrinsic  part  of  new  care   models  is  changing  how  patients,  carers  and   staff  feel  about  the  care  they  receive  and   deliver.    The  clue  was  in  the  title  -­‐  in  NEHF  the   vanguard  is  called  Happy,  Healthy  at  Home  and   on  the  Isle  of  Wight  My  life  a  full  life.    Logic   models  listed  desired  outcomes  like  improved   wellbeing,  confidence  to  self-­‐care  and  staff   satisfaction.    We  wanted  to  find  a  way  to  reliably   measure  and  understand  if  this  was  happening.   A  small  team  reviewed  the  self  reported   outcome  measures  available  and  selected  R-­‐ Outcomes  (R-­‐Os).    R-­‐Os  are  a  family  of  short,   generic,  validated3,4,5 ,  self  reported  outcome   measures  for  patients/  client,  carers  and  staff.       They  have  been  carefully  designed  to  be  quick   and  easy  to  complete  -­‐  which  means  that  more   people  complete  them.    Being  generic,  they  can   be  applied  to  almost  any  health  or  care  setting,   from  support  at  home  to  acute  inpatient  care.     They  can  be  collected  through  the  mechanism   that  best  suits  the  service  –  including  on  paper,   on-­‐line,  smart  phones  and  tablets,  or  the   telephone.    They  work  as  a  family  of  measures,   with  the  ability  to  select  combinations  from  a   current  menu  of  5  patient/  client  measures,  4   measures  for  staff  and  3  for  carers.    Further   details  are  attached  as  an  appendix  and  at   www.r-­‐outcomes.com.   At  the  time  of  writing  we  have  collected   outcomes  from  around  6000  patients/  people   and  500  staff  in  Wessex  over  the  past  two  years.     Typically  we  collect  people’s  outcomes  on   referral  and  once  they  have  been  supported  or   cared  for  to  look  for  evidence  of  a  change.    Free   text  provides  additional  feedback  which  is  often   presented  graphically  as  a  word  cloud.  
  • 2.   Evaluating  the  impact  of  new  care  models  on  people   2   Example  1:  Health  Confidence  in  patients   supported  by  new  Integrated  Care  Teams.   Patients  report  large  and  statistically  significant   improvements  in  their  health  confidence   following  the  support  from  the  five  Integrated   Care  Teams.    The  biggest  improvements  are  in   people  reporting  that  they  can  get  the  right  help   when  they  need  it  and  being  involved  in   decisions  –  which  is  the  heart  of  this  focused   and  personalised  care  plan.    Local  reporting  drills   down  into  the  differences  between  the   outcomes  from  different  teams.     Example  2:  Health  Status  of  patients  supported   by  eight  different  social  prescribing  services.   Patients  reported  significant  improvements  in   their  health  status  –  particularly  feeling  less  low   or  worried.    Similar  improvements  were  found  in   their  health  confidence,  wellbeing  and   experience  of  care.     Example  3:  A  team  of  Care  Navigators.   The  team  reported  a  high  level  of  wellbeing  at   work,  are  very  satisfied  with  their  job  and  give   top  marks  for  feeling  that  their  job  is   worthwhile.         They  scored  their  confidence  in  delivering  their   roles  less  highly.  They  can  get  help  when  they   need  it.    Very  high  scores  were  reported  for  the   level  of  personal  service  and  empathy  they   provide  but  less  positive  for  the  organisation  of   the  service.       These  surveys  have  been  repeated  to  analyse   changes  over  time.     Example  4:    Comparing  the  before  and  after   patient  scores  for  two  different  services.   R-­‐Outcomes  can  be  used  to  look  for  changes   over  time  and  to  compare  services.    Reference   groups  have  been  developed  to  help  this.         0 20 40 60 80 100 Mean Sore Mean scores for Localities Pre and Post Intervention ESP Pre (n=51) ESP Post (n=99) MHH Pre (n=270) MHH Post (n=371)
  • 3.   Evaluating  the  impact  of  new  care  models  on  people   3   2.    Qualitative  interviews  with   patients,  carers  and  staff     To  explore  the  nature  and  extent  of  a  change,   it’s  important  to  use  qualitative  methods  to   ensure  a  good  depth  of  investigation  is  achieved.     In  our  case,  we’ve  used  qualitative  methods  to   explore  whether  new  roles  and  new  innovations   have  improved  patients’  health  and  wellbeing,   and  health  service  delivery  by  health  care   professionals.  Using  semi-­‐structured   interviews  with  patients,  carers  and  staff,  we   have  sought  the  narrative  of  the  innovation   from  those  receiving  and  delivering  the  new   role  or  innovation.  Obtaining  this  narrative  has   helped  us  explain  why  change  has  or  has  not   occurred,  what  impacts  have  been  observed   by  patients  and  staff,  and  if  any  unintended   consequences/impacts  were  observed.   Questions  for  each  interview  have  been   developed  using  the  logic  model  for  each  new   role/innovation,  in  order  to  keep  the  interviews   focused  on  the  topics  of  interest  and  to  allow   synthesis  with  the  quantitative  evidence   gathered.     3.    Themed  analysis  of  case  studies   In  most  of  our  evaluations,  case  studies  have   been  provided  by  staff  who  know  the   individual’s  situation  and  journey,  and  are   completed  on  a  simple  template  that  includes   the  situation,  what  happened  and  how  the   service  worked  with  the  individual.    We   undertake  thematic  analysis  of  these  case   studies  (see  below)  as  part  of  our  evaluation.   For  example,  eight  case  studies  of  patients   receiving  input  from  an  Integrated  Care  Team   revealed  four  key  themes:  a  quick  response,  a   joined  up  response,  problem  solving  and  the   impact  on  patients.  This  evidence  provided   important  indications  that  the  new  care  model   was  achieving  its  desired  outcomes.  When   triangulated  with  the  findings  from  the  R-­‐ Outcomes  analysis,  the  evidence  of  impact  was   further  strengthened.   It  is  also  possible  to  use  case  notes  to  develop   case  studies.    For  an  evaluation  of  a  Recovery   College,  a  random  set  of  five  case  notes  were   reviewed  and  used  to  chart  each  student’s   journey  of  enrolling  and  participating  in  recovery   courses  and  their  wider  use  of  mental  health   services.    These  were  displayed  graphically  as   follows:     Thematic  analysis  of  what  is  heard  in  qualitative   interviews  and  written  in  case  studies  is  an   important  part  of  our  evaluation  and  we  use  a   well  defined  and  widely  used  approach.  This   method  explores  the  semantic  (explicit)  themes   and  latent  (implicit)  themes  within  the   qualitative  data  via  a  five  stage  process:   familiarise  yourself  with  the  data,  generate   initial  codes,  search  for  themes,  develop  and   refine  themes,  and  define  and  name  the  themes.   It    aims  to  find  repeated  patterns  of  meaning   whilst  also  focusing  on  the  relevance  of  issues  in   relation  to  the  aims  of  the  evaluation  of  the  new   care  model.  Importantly,  as  with  many  forms  of   qualitative  analysis,  data  saturation  is  the  goal.   Once  individual  narratives  begin  to  repeat,  and   the  same  themes  and  sub-­‐themes  are  evident  in   the  interviews,  data  collection  is  stopped6 .   4.    Team  observation  using   normalisation  process  theory   As   many   of   the   NCMs   being   evaluated   are   formed   of   new   teams   of   staff,   we   were   particularly   interested   to   find   a   pragmatic   and   validated   evaluation   tool   to   understand   the   extent  to  which  the  team  was  able  to  embed  the   0 1 2 3 4 5 6 7 8 Jan-15 Feb-15 M ar-15 Apr-15 M ay-15 Jun-15 Jul-15 Aug-15 Sep-15 Oct-15 Nov-15 Dec-15 Jan-16 Feb-16 M ar-16 Apr-16 M ay-16 Jun-16 Number of contacts Recovery College case study – dates of course enrolment and number of mental health service contacts Enrolled in RC Wellness for life Managing intense emo4ons Confidence & self esteem
  • 4.   Evaluating  the  impact  of  new  care  models  on  people   4   implementation  NCMs  on  a  day  to  day  basis  in  a   way  that  was  sustainable  in  the  long  term.   Understanding  if  and  how  new  roles,  techniques   and  innovations  for  clinical  and  professional   practice  are  effectively  translated  into  practice  is   vital  for  the  teams  delivering  them  and  the   programmes  commissioning  them.     Normalisation  Process  Theory  (NPT]7  was   developed  to  better  understand    how   innovations  are  implemented  in  practice  and  in   particular  how  activities  associated  with   introducing  new  ways  of  working  are  both   enabled  and  constrained  with  in  a  particular   setting.  It  is  a  theoretically  developed  and  widely   used  sociological  theory  of  how  innovations  are   understood,  engaged  with,  enacted  and   reflected  upon.  NPT  has  widely  been  used  in  the   dynamic  setting  of  health  care.     The  starting  point  of  NPT  is  to  understand  the   embedding  of  a  practice  i.e.  what  people   actually  do  and  how  they  work  together.  NPT   provides  an  explanatory  framework  to  better   understand  the  routine  embedding  of   healthcare  interventions  in  their  social  contexts,   in  particular  why  some  processes  seem  to  lead   to  a  practice  becoming  sustained  over  a  long   term  while  others  do  not  (May  and  Finch,  2009).   Although  the  authors  make  no  claim  of  absolute   predictive  power,  they  argue  that  within  certain   limits  the  extent  of  sustainable  long  term   embedding  can  be  anticipated.  This  means  that   NPT  can  help  ascertain  the  likelihood  of  the   routine  embedding  of  an  intervention  within   certain  limits.   We  have  conducted  observations  of  team   meetings  and  key  interactions  between  staff   using  the  four  key  components  (Coherence,   Cognitive  Participation,  Collective  Action,   Reflexive  Monitoring).  In  addition,  the  NPT   NoMAD  questions8  have  been  used  for  a  survey   undertaken  during  the  structured  focus  groups.   The  survey  also  included  questions  on  feeling   valued  as  part  of  the  team  and  questions  about   the  extent  to  which  the  expected  outcomes  of   the  service  (as  articulated  in  its  logic  model)   were  felt  to  have  been  achieved.   We    have  now  conducted  four  evaluations  of   different  teams  that  are  working  in  an   integrated  way  to  implement  NCMs.   Table  1:  Results  for  coherence   Table  1  shows  team  results  for  coherence  or   how  a  NCM  is  conceptualised  and  held  together   in  action.  In  this  example  coherence  was  rated   positively.  The  team  members  had  a  shared   understanding  of  the  new  practice  and   understood  how  it  affected  the  nature  of  their   work  and  could  see  potential  value  in  it  for  their   work.  Should  the  results  have  been  negative,   this  may  have  required  some  adjustments  in   relation  to  the  implementation  of  the  new   practice.       5.    Synthesising  findings   As  we  have  consistently  applied  these  methods   for  evaluating  how  people  experience  and  feel   about  new  care  models,  we  have  also  developed   our  method  for  bringing  together  and   synthesising  these  findings.   The  focus  is  a  synthesis  meeting  that  brings   together  all  of  the  people  involved  in  gathering   the  data  and  evidence  described  in  this  paper,  as   well  as  the  quantitative  elements  of  the   evaluation,  including  changes  in  activity  (e.g.   A&E  attendances)  and  economic  evaluation.   Before  the  synthesis  meeting  takes  place,  we   Coherence (n-9) Av. score 1. The NCM is distinct from previous ways of working 7.8 2. Team members have a shared understanding of the purpose of the NCM and of the specific responsibilities required. 8.4 3. Team members understand how the NCM affects the nature of their work 8.4 4. Team members can see potential value of the NCM for their work 8.2 Total 8.2
  • 5.   Evaluating  the  impact  of  new  care  models  on  people   5   aim  to  meet  with  representatives  of  the  new   care  model  to  take  them  through  it,  check  for   accuracy  and  get  their  initial  feedback  or   reflections.      At  the  synthesis  meeting  all  of  the   material  is  pooled  and  worked  through  together   to  triangulate  the  evidence  and  identify  and   agree  the  findings,  lessons  and   recommendations.    An  important  part  of  this   synthesis  is  to  identify  the  active  ingredients   that  have  contributed  to  (or  hindered)  the   outcomes  delivered  by  the  new  model  of  care.         6.    How  the  findings  have  been  used   by  our  clients   We  understood  that  NCMs,  by  their  nature,   would  be  likely  to  adapt  and  evolve  during  the   Vanguard’s  journey.  We  have  tried  to  support   teams  in  this  creative  process  by  providing  early   headlines  from  the  evaluation  findings  and   exploring  with  them  the  possible  explanations   for  the  emerging  evidence.  This  feedback  helps   the  team  and  the  programme  to  understand   whether  an  NCM  is  having  the  desired  impact   and  if  not  why  not,  and  helps  the  evaluation   team  to  decide  where  to  focus  the  inquiry  if  the   results  are  unexpected  (e.g.  to  examine  one  area   in  more  detail,  or  to  differentiate  the  data  set  in   different  ways).    It  has  been  valuable  to  have   contemporaneous  access  to  outcomes  data  that   can  enable  timely  reporting  to  staff  with  a  keen   interest  in  how  their  NCM  is  performing.     The  following  examples  describe  some  of  the   beneficial  outcomes  of  ‘formative’  evaluation  for   those  involved  in  implementing  NCMs:     Example  1:  The  evaluation  of  teams  through   semi-­‐structured  observation  has  influenced  the   design  of  organisational  development   programmes,  by  identifying  priority  areas  to   focus  on  e.g.  barriers  or  enablers  to  the  NCM.       Example  2:  Evidence  on  the  impact  of  a  Referral   Management  Service  has  informed  the  business   case  for  the  development  of  a  locality  based   community  dermatology  service.   Example  3:  Data  on  patient  outcomes  (R-­‐ Outcomes)  from  a  number  of  different   integrated  care  teams  across  Wessex  has   initiated  a  new  line  of  inquiry  –  do  differences  in   casemix  account  for  differences  in  patient   reported  outcomes?   Example  4:    Evidence  of  the  different  impact   from  different  forms  of  social  prescribing  service   has  informed  a  new  commissioning  service   specification  –  against  which  the  services  were   re-­‐commissioned.    Patients  and  carers   emphasised  the  importance  of  support  at  the   weekend  and  this  was  included.    The  new   service  level  agreements  include  quality   outcomes  (R-­‐Outcomes)  KPIs.   Example  5:    Community  ambassadors  have  been   engaged  in  collecting  R-­‐Outcomes  from  people   in  waiting  rooms  using  iPads  and  have   participated  in  the  review  and  discussion  of  all  of   the  qualitative  findings  and  symposium  events.   Example  6:  Bespoke  measures  have  been   incorporated  into  R-­‐Outcomes  to  cover  issues  of   particular  interest  to  the  service  or  service  users   e.g.  to  understand  how  people  perceive   ‘integration.’            
  • 6.   Evaluating  the  impact  of  new  care  models  on  people   6   7.    Our  reflections  on  how  to   evaluate  the  impact  of  new  care   models  on  people   Our  evaluation  vital  statistics  at  the  time  of   writing  are:   NCM  evaluations  completed  and  reported   15   R-­‐Outcomes  recorded  by  patients   6,000   R-­‐Outcomes  completed  by  staff   500   Qualitative  interviews  undertaken   85   Case  studies  analysed   93   Evaluations  planned  for  17/18   21     R-­‐Outcomes  have  provided  a  reliable  way  of   measuring  the  impact  on  patients  and  staff  at   scale  and  in  a  short  period  of  time.    They  enable   us  to  look  for  evidence  of  change  during   implementation  of  a  NCM  and  increasingly  we   are  using  them  to  benchmark  the  impacts  on   different  cohorts  of  patients  and  services.   Qualitative  interviews  are  important  as  they   provide  the  most  in-­‐depth  method  for   understanding  the  lived  experience  of  patients   and  staff.  However,  whilst  every  lived   experience  reported  by  a  patient  or  team   member  is  valid  in  understanding  the  impact  of   NCMs,  qualitative  data  collection  can  be  very   resource  intensive  for  evaluators  and  create   additional  work  for  staff  (e.g.  consenting   individuals  for  interview,  scheduling  interviews,   Information  Governance  and  Ethics  approvals).     The  number  of  participants  that  are  suitable  for   interview  can  be  small.  In  our  experience,  we   have  typically  identified  around  6  patients  and  6   staff  per  NCM  with  interview  times  of  up  to  an   hour  each.  In  order  to  ensure  we  obtain  an  in-­‐ depth  picture  of  patient  and  staff  experience   and  also  adhere  to  data  collection  timeframes,   we  are  looking  at  how  we  gather  and  analyse   this  data  to  be  sufficiently  rigorous  but  more   time  and  resource  efficient.   Case  studies  are  also  a  powerful  way  of   capturing  the  in-­‐depth  impact  for  individuals.   Teams  that  have  been  motivated  to  collect  these   stories  have  produced  around  20-­‐30  case  studies   for  analysis.  A  simple  template  has  been  helpful   in  some  instances,  but  the  most  significant   enabler  has  been  a  team  leader  who  is   convinced  of  the  value  of  collecting  this  data.       NPT  has  proved  helpful  insights  into  the  extent   to  which  integrated  ways  of  working  have   become  embedded  by  new  care  model  teams.  It   has  been  used  successfully  with  focus  groups   from  4  to  24.  The  findings  have  been  recognised   by  those  participating  and  informed  team   development.       8.    Looking  forward   Three  new  measures  have  recently  been  added   to  the  R-­‐Outcomes  suite  and  will  be  trialled  in   Vanguard  evaluations  during  the  coming  year,   measuring:   • Patient  and  staff  perception  of  service   integration   • A  short  staff  measure  based  on   Normalisation  Process  Theory   • People  and  organisations’  readiness  to   innovate   • People’s  confidence  in  using  digital  health   care  solutions       November  2017      
  • 7.     For  more  information,  contact  the  authors:     Andrew  Liles,  Director,  R-­‐Outcomes,  andrew.liles@r-­‐outcomes.com   Phlippa  Darnton,  Evaluation  Lead,  Wessex  AHSN,   philippa.darnton@wessexahsn.net   Dr  Andrew  Sibley,  Evaluation  Programme  Manager,  Wessex  AHSN,   Andrew.sibley@wessexahsn.net   Dr  Catherine  Matheson-­‐Monnet,  Senior  Research  Fellow,  Centre  for   Implementation  Science,  University  of  Southampton,     c.b.matheson@soton.ac.uk   References:   1.  New  Care  Models.  (June  2017)  The  local  evaluation  of  the  new  care  models  vanguards:  our   expectations  and  offer  of  support  for  2017/18.  http://www.england.nhs.uk/vanguards   2.  Liles  A,  Darnton  P  (2017)  Social  Prescribing  in  Wessex;  Understanding  its  impact  and  supporting  its   spread.    Wessex  AHSN.  https//www.r-­‐outcomes.com/publications   3.  Benson  T,  Potts  HWW.  A  short  generic  patient  experience  questionnaire:  howRwe  development   and  validation  (2014).  BMC  Health  Services  Research  2014,  14:499  .   http://www.biomedcentral.com/1472-­‐6963/14/499   4.  Benson  T,  Whatling  J,  Arikan  S,  Sizmur  S,  McDonald  D,  Ingram  D:  Evaluation  of  a  new  short  generic   measure  of  HRQoL:  howRu.  Informatics  in  Primary  Care  2010;  18:89-­‐101.  https://hijournal.b   cs.org/index.php/jhi/article/view/758/770   5.  Benson  T,  Bowman  C,  Sladen  J,  Liles  A,  Potts  HWW.  Health  Confidence  Score  (HCS)  –  Development   and  Validation.  Research  Paper  17/02  September  2017.  https://www.r-­‐outcomes.com/publications   6.  Braun,  V.  and  Clarke,  V.  (2006)  Using  thematic  analysis  in  psychology.  Qualitative  Research  in   Psychology,  3  (2).  pp.  77-­‐101.  www.eprints.uwe.ac.uk/11735/2/thematic_analysis_revised.     7.  May,  C.  and  Finch,  T.  (2009)  Implementation,  embedding,  and  integration:  an  outline  of   Normalization  Process  Theory.  Sociology  43  (3):  535-­‐55http://www.normalizationprocess.org   8.  Finch  et  al.  (2013)  Improving  the  normalization  of  complex  interventions:  measure  development   based  on  normalization  process  theory  (NoMAD):  study  protocol.  Implementation   Science,  8,  1,  43.  doi:10.1186/1748-­‐5908-­‐8-­‐43   9.  Completed  vanguard  evaluation  reports  for  Farnham  Integrated  Care  Team,  Referral  Management   Service  and  Pre-­‐Diabetes  Education  Programme  and  for  a  Recovery  College;  www.wahsn.org.uk                       Acknowledging  the  evaluation  team   Dr  David  Kryl,  Director  of  Insight,  Centre  for  Implementation  Science  (CIS)/  Wessex  AHSN   Joe  Sladen,  Evaluation  Programme  Manager,  Wessex  AHSN   Sydney  Anstee,  Research  Consultant,  Wessex  AHSN   Cindy  Brooks,  Research  Fellow,  CIS,  University  of  Southampton   Tim  Benson,  Director,  R-­‐Outcomes   Jess  Done,  Project  Manager,  R-­‐Outcomes  
  • 8.   Evaluating  the  impact  of  new  care  models  on  people   8   Appendix  –  R-­‐Outcome  patient  measures     Patient-Reported Outcome Measures R-Outcomes’ family of short generic patient-reported measures cover health status, patient experience, personal wellbeing and health confidence. They can be used at the point of care or between visits. These share a common framework with 4 items and 4 responses, suitable for use on a patient’s own smart-phone, tablet, PC or on paper. These tools are research-based and are short, quick and easy to use. They are generic and suitable for almost all patients irrespective of conditions across health and social care. The results measure trends, changes and comparisons. Results are easy to interpret, giving feedback to patients, clinicians, managers and commissioners, tracking changes and differences between units. These validated tools are short and quick to use with simple unambiguous wording, understood by those whose first language is not English. PATIENT © R-Outcomes Ltd 2017 www.r-outcomes.com info@r-outcomes.com Health Status HowRu is a short generic patient-reported outcome measure (PROM), to track and compare patients’ perceptions of how they feel physically and mentally and what they can do (disability and dependence). Experience HowRwe is a short generic patient-reported experience measure (PREM), which measures patients perceptions of the care and service provided. It is suitable for all types of patient and care setting. Confidence HCS captures people’s confidence in their knowledge, self-management, access to help and shared decision-making. Wellbeing The Personal Wellbeing Score (PWS), based on National Statistics ONS4, covers life evaluation, worthwhileness, positive and negative experience. Service Integration Integration across service boundaries is a challenge for all health services. It is a priority for new models of care. Length Name Items Words Reading age Outcome howRu 4 37 7 EQ-5d (inc. visual analogue scale) 6 230 11 SF-12 12 474 11 NHS PROMS (Hip pre-op) 27 1,485 11 Experience howRwe 4 29 7 NHS Friends and Family Test 1 44 12 HCAHPS 32 1,156 13 GP Patient Survey 62 2,922 12 NHS Adult Inpatient Survey 76 3,353 12 Engagement Health Confidence Score 4 50 8 Patient Activation Measure (PAM) 13 293 12 Health Literacy Questionnaire 44 1,001 12 Wellbeing Personal Wellbeing Score 4 43 9 ONS Personal Well-being 4 95 11 Short Warwick-Edinburgh 7 89 8 ICECAP-A 5 273 10 Tel: 0785 568 2037 Require help from others How are you today? (past 24 hours) Limited in what you can do Feeling low or worried Pain or discomfort None A little Quite a lot Extreme How do you feel and how much can you do? Choose one answer on each line How are we doing? See you promptly Well organised Listen and explain Treat you kindly Excellent Good Fair Poor What do you think about our service? Choose one answer on each line Health Confidence I can get the right help if I need it I am involved in decisions about me I can look after my health I know enough about my health Strongly agree Agree Neutral Disagree How do you feel about caring for your health? How much do you agree? Personal Wellbeing I was happy yesterday I was NOT anxious yesterday What I do in my life is worthwhile I am satisfied with my life Strongly agree Agree Neutral Disagree How are you feeling in general? How much do you agree? Service Integration (patient view) I don’t have to repeat my story Different services work well together Staff know what other services do Services talk to each other Strongly agree Agree Neutral Disagree How well do services work together? How much do you agree? Length and reading age affect respondent burden and response rates.