Mastery training in diagnostic radiology


Published on

Reviewing some of the educational literature on training for mastery applied to diagnostic radiology, and demonstrating how online radiology case repositories can be used to facilitate training for mastery

Published in: Education
  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Mastery training in diagnostic radiology

  1. 1. 6/12/12   Introduc3on   •  What  do  we  do  as  diagnos3c  radiologists?   –  One  of  our  key  tasks  is  to  provide  a  diagnosis,  and  jus3fy  it   àRadiology  prac3ce  –  See/detect,  recognize,  interpret   (Krupinski,  2010;  Modine  &  Mello-­‐Thomas,  2010)   Mastery  Training  in  Diagnos3c     •  Why  should  we  be  interested  in  this  topic  (of  mastery   Radiology   training)?   –  Competency  and  proficiency  is  not  enough,  we  should   Dr  Goh  Poh  Sun   challenge  ourselves  to  “master”  a  field   MB,BS(Melb),  FRCR,  FAMS,  MHPE(Maastricht)     Senior  Consultant     Department  of  Diagnos3c  Radiology   •  How  can  we  examine  this  topic,  from  the  perspec3ve  of  a   Na3onal  University  Hospital   radiology  resident,  and  radiology  teacher?   Na3onal  University  Health  System   Singapore   –  What  can  a  resident  do  on  a  day  to  day  basis?     –  What  are  some  strategies  a  radiology  teacher  can  use?   –  How  can  we  use  educa3onal  theory  to  improve  radiology   training   What  is  the  current  situa3on  for  a   What  occurs  from  a  learning   radiology  resident?   perspec3ve?  •  Daily  case  repor3ng  with  feedback   •  Encounter  new  diagnos3c  category  –  “I  have  never   –  Subspecialty  based   seen  this  before”   •  Encounter  more  examples  of  previously  encountered  •  Several  tutorials  per  week   categories  –  “I  didn’t  know  that  it  could  look  like  this”  •  Weekly  topic  based  lectures  (and  annual  courses/conf)   •  Encounter  look  alike  and  confounding  cases  –  “these  2  •  Weekly  study  3me   cases  really  look  similar  except  for  this  key  finding”  •  Resident  topic  based  presenta3ons  and  journal   •  Understand  “why”?  –  explain  the  radiological   reviews  (including  conferences  and  symposia)   appearance  •  Systema3c  reading  and  case  review  to  prepare  for   •  Progressively  develop  the  confidence  to  state  the  likely   fellowship  examina3ons   diagnosis  with  convic3on,  and  provide  arguments  for   and  against  a  par3cular  diagnosis   1  
  2. 2. 6/12/12   How  is  exper3se  and  diagnos3c  mastery  in  radiology  developed?   •  1.  exper3se  in  radiology  is  directly  correlated  with  experience   •  2.  case  experience  should  be  cumula3ve   •  3.  case  experience  should  include  full  spectrum  of  the  presenta3on  of  any  given  diagnosis   •  4.  case  experience  should  include  material  of  increasing  complexity   •  5.  increasing  complexity  involves  cases  showing  typical,  then  less  typical,  less  easily   detectable  features,  with  less  associated  features,  or  more  confounding  associated   features,  or  more  than  one  pathology,  and  progressively  less  or  contradictory  addi3onal   Lets  dis3ll  some  ideas  from  the   informa3on   educa3onal  research  literature   •  •  6.  radiology  task  is  simple  three  step  process  -­‐  see,  recognize,  diagnose   7.  this  requires  perceptual  learning  or  pagern  recogni3on,  categorical  learning  or  the   ability  to  list  key  features,  and  diagnos3c  reasoning  or  the  ability  to  correlate  with  the   clinical  sehng   8.  experts  use  rapid  wholis3c  percep3on  linked  to  experience,  which  are  mental  schemas   à  à  à   •  or  representa3ons  of  the  key  features  of  a  wide  range  of  pathology   •  9.  experts  use  rapid  200msec  big  picture  percep3on  to  generate  the  most  likely   hypothesis,  which  then  drives  deliberate  search  for  key  features   •  10.  experts  rapidly  decide  on  whether  radiograph  or  scan  is  normal,  or  has  a  likely   abnormality,  and  then  use  visual  search  and  reasoning  to  confirm  diagnosis   •  11.  online  case  repositories  can  accelerate  the  development  of  exper3se,  by  widening  the   case  experience    of  radiologists,  beyond  that  of  individual  day  to  day  direct  experience   •  12.  the  key  is  to  use  case  material  in  a  deliberate  way   Outline   What  are  the  indicators  of  mastery?  •  What  are  the  indicators  of  mastery?   •  Literature  •  How  can  this  be  assessed?   •  Survey  and  focus  group  discussion  •  How  can  we  train  for  mastery?   –  Brainstorm  •  What  is  the  role  of  key  images?   –  Rank  and  jus3fy  •  What  is  the  role  of  case  repositories?   2  
  3. 3. 6/12/12   What  are  the  indicators  of  mastery?   How  can  this  be  assessed?  •  knowledge  base   •  Plain  films  •  years  of  experience   •  Cross  sec3onal  imaging  •  ability  to  recognize  unusual  presenta3ons   –  How  many  are  required?  •  understanding  underlying  principles   –  What  spectrum/complexity  of  cases  is  required?  •  accuracy  (of  descrip3on)  •  completeness  (of  descrip3on)   •  Assessment  •  speed   –  Accurate/complete  selec3on  of  key  images/key   features  •  efficiency  of  search  strategy           –  Accurate  descrip3on  of  key  features  •  effec3veness  •  confidence   –  Speed  and  confidence  in  screening  all  images  and   iden3fying  key  images/screening  all  areas  of  plain  film  •  low  error  rates   and  iden3fying  key  features   Examples  •  Lets  look  at  plain  radiographs  à  CXR  •  and  •  Cross  sec3onal  imaging  à  CT  scans  of  the   brain   3  
  4. 4. 6/12/12   4  
  5. 5. 6/12/12   How  can  we  train  for  mastery?   •  Role  of  deliberate  prac3ce  (systema3c  case   review,  with  expert  guidance  and  feedback)   •  Role  of  background  reading  /  reflec3on  on  day   to  day  clinical  experience  (“ad  hoc”  case   review)  à  typical  case,  varia3ons,  look  alike   and  differen3al  diagnoses             What  is  deliberate  prac3ce?   What  is  the  role  of  key  images?  •  Deliberate  prac,ce  refers  to  a  consciously  planned  and   •  Key  images  exemplify  most  important  findings   executed  program  of  repeated  focused  exercise  of  the   skills  required  for  expert  performance  accompanied  by   •  Can  be  used  to  learn  key  features   appropriate  feedback  (Anders  Ericsson,  Krampe,  &   Tesch-­‐Romer,  1993).     •  Repeated  use  can  help  accelerate  mastery  •  Paired  and  mixed  prac,ce  involves  the  use  of  a  series   training   of  similar  and  contras3ng  examples  of  imaging   abnormali3es,  with  an  agempt  to  describe  and  reflect   –  hgp:// on  similari3es,  and  differences  between  the  similar   2012/11/deep-­‐learning.html   and  contras3ng  examples  (Anders-­‐Ericsson,  2004;   Hatala,  Brooks  &  Norman,  2003;  Norman,  2008;     Norman,  Young  &  Brooks,  2007).     5  
  6. 6. 6/12/12   How  long  does  it  take?  •  Literature   –  10  to  20  years  (chess  masters,  concert  level   musicians,  top  athletes  -­‐  Anders  Ericsson,  2004)   –  10,000  hours  to  25,000  hours  (lots  and  lots  of   prac3ce)   –  (for  paediatric  ankle  radiographs)  20  cases  per   diagnosis  à  threshold,  maximum  rate  of   improvement  21  to  50  cases,  and  no  upper  limit   (Pusic  et.  al.,  Academic  Medicine,  2011)   What  are  case  repositories?   Literature  Review   What  is  the  role  of  case  repositories?   Efficiency  and  effec3veness     Rumble,  1997   Compare  and  contrast  commonly  use  to  solve  individual  day  to  day  clinical  problems  •  Collec3ons  of  key  images,  both  plain  film  and   Berbaum,  Franken  &  Smith,  1985   cross  sec3onal  imaging  (hard  copy  à  digital   Radiology  prac3ce  –  See/detect,  recognize   Paired  prac3ce  promotes  ability  to  recognize     Interpret   key  diagnos3c  features,  and  ability  to  elaborate   form  +/-­‐  hyperlink)   Krupinski,  2010   on  these  features   Modine  &  Mello-­‐Thomas,  2010   Markman  &  Gentner,  1993  •  Use  in  the  con3nuum  of  training     Mixed  prac3ce  promotes  schema  acquisi3on     –  Undergraduate  à  exemplar  case   Compare  and  Contrast  Prac3ce   and  analogous  transfer   Hatala,  Brooks  &  Norman,  2003   Glick  &  Pagerson,  1992   –  Residency  and  fellowship  à  mastery  training   Norman,  Young  &  Brooks,  2007     Norman,  2008   Combina3on  of  paired  and  mixed  prac3ce   –  CPD/CME     or  compare  and  contrast,  helps  develop  and     Deliberate  Prac3ce     refine  categorical  knowledge   (Anders  Ericsson,  Krampe   Namy  &  Clepper,  2010    &  Tesch-­‐Romer,  1993)   6  
  7. 7. 6/12/12   Perceptual  Learning  vs  Cogni3ve  Reasoning  –   Addi3onal  Recommenda3ons  from  Literature   •  In  addi3on  to  gaining  exposure  to  the  whole  spectrum   of  the  presenta3on  of  any  given  condi3on,  go  deeper   to  try  to  understand  why   •  Underlying  anatomy,  pathology,  clinical  background   •  This  aids  percep3on  and  memory  retrieval,  and  also   helps  problem  solving  using  first  principles   •  Novices  start  using  linear  problem  solving  /  analy3cal   thinking,  experts  rely  first  on  big  picture  perceptual   reasoning  /  pagern  recogni3on  /  non-­‐analy3cal   thinking,  with  analy3cal  thinking  as  back-­‐up   –  Engel,  Medical  Teacher,  2008   How  is  exper3se  and  diagnos3c  mastery  in  radiology  developed?  •  1.  exper3se  in  radiology  is  directly  correlated  with  experience   (RLOs  /  Cases  in)  Digital  Repositories  Support  • •  2.  case  experience  should  be  cumula3ve   3.  case  experience  should  include  full  spectrum  of  the  presenta3on  of  any  given  diagnosis   the  Con3nuum  of  Medical  Educa3on  •  4.  case  experience  should  include  material  of  increasing  complexity  •  5.  increasing  complexity  involves  cases  showing  typical,  then  less  typical,  less  easily   •  Con3nuum  –  Undergraduate,  Postgraduate,   detectable  features,  with  less  associated  features,  or  more  confounding  associated   features,  or  more  than  one  pathology,  and  progressively  less  or  contradictory  addi3onal   Con3nuing  Professional  Development     informa3on   •  Undergraduate/Novice  Resident  –  Building  the  • •  6.  radiology  task  is  simple  three  step  process  -­‐  see,  recognize,  diagnose   7.  this  requires  perceptual  learning  or  pagern  recogni3on,  categorical  learning  or  the   founda3on,  basic  concepts,  fundamental   ability  to  list  key  features,  and  diagnos3c  reasoning  or  the  ability  to  correlate  with  the   clinical  sehng   knowledge  à  Exemplar  Cases  •  8.  experts  use  rapid  wholis3c  percep3on  linked  to  experience,  which  are  mental  schemas   or  representa3ons  of  the  key  features  of  a  wide  range  of  pathology   •  Postgraduate  –  Applying  knowledge,  building  •  9.  experts  use  rapid  200msec  big  picture  percep3on  to  generate  the  most  likely   skills,  drills  and  quizzes  à  Broaden  range   hypothesis,  which  then  drives  deliberate  search  for  key  features  •  10.  experts  rapidly  decide  on  whether  radiograph  or  scan  is  normal,  or  has  a  likely   •  CPD  –  Keeping  up  to  date,  learning  new  concepts   abnormality,  and  then  use  visual  search  and  reasoning  to  confirm  diagnosis   and  skills  à  Revision,  Deliberate  Prac3ce,  •  11.  online  case  repositories  can  accelerate  the  development  of  exper3se,  by  widening  the   case  experience    of  radiologists,  beyond  that  of  individual  day  to  day  direct  experience   Working  on  Differen3a3on  and  Discrimina3on  •  12.  the  key  is  to  use  case  material  in  a  deliberate  way   7  
  8. 8. 6/12/12   Typical  textbook  example(s)  à  Aim  to  gain  exposure  to  many,  many  more  (the  whole  spectrum)   How  can  we  use  these  ideas?   •  Residents  –  take  advantage  of  ad  hoc  case  encounters  to   undertake  deliberate  prac3ce;  and  also  systema3cally   schedule  sessions  of  deliberate  prac3ce  –  do  not  wait  3ll   revising  for  the  fellowship   •  Radiology  teachers  –  systema3cally  collect  as  many   examples  of  each  diagnos3c  category  on  an  ongoing  basis,   build  up  collec3on  of  full  spectrum  of  each  disease,   including  look-­‐alikes  and  differen3al  diagnosis   •  For  both  residents  and  teachers  –  consider  reviewing  and   presen3ng  cases  in  groups  of  similar  and  contras3ng  cases,   to  improve  discrimina3on  and  diagnos3c  ability   •  Plan  for  lots  and  lots  of  prac3ce  (“deliberate  prac3ce”)   Spaced  (distributed)  vs  Massed   What  do  we  do  with  case  collec3ons?   Prac3ce  Learning  is  more  effec3ve  when  spread  over  several  shorter  sessions,  over  a  long  period    3me,  compared  with  repeated  studying  over  a  short  period  of  3me  –  “spacing  effect”   •  From  teacher’s  perspec3ve,  in  what  form  can  Ebbinghaus,  in  Memory:  a  contribu3on  to  experimental  psychology,  1913   they  be  collected?   •  From  resident’s  perspec3ve,  how  can  I  use   this  case  material  in  an  efficient  and  effec3ve   way?  Bloom  &  Schuell,  The  Journal  of  Educa3onal  Research,  1981  hgp://     8  
  9. 9. 6/12/12   Outline   •  Personalised  learning  content  through  RLOs   Personalized  Learning  Content  Through  RLOs   •  Illustra3ve  examples  of  crea3ve  use  of  RLOs     •  Experience  of  posi3ve  educa3onal  aspects  for   Lets  look  from  teacher’s  perspec3ve,  then  student’s   students  and  tutors   •  Problems  using  this  approach  for  students  and   tutors   Personalized  Learning  Through  RLOs   Illustra3ve  Examples  •  What  are  RLOs?   •  How  can  RLOs  be  used?   –  Raw  asset,  e.g.,  an  image  (most  flexible,  reusable)   –  Learning  asset,  e.g.,  annotated  image.     –  Task  or  exercise,  e.g.,  interac3ve  case,  or  problem  solving   task.     –  Learning  design  with  content  (containing  some  or  all  of   classes  1–3).     –  Generic  learning  design,  e.g.,  a  generic  problem-­‐based   learning  design.     –  Koppi,  A.  J.  &  Hodgson,  L.  (2001)  Universitas  21  Learning  Resource   Catalogue  using  IMS  Meta-­‐  data  and  a  New  Classifica3on  of  Learning   Objects.  EDMEDIA  2001,  World  Conference  on  Educa7onal  Mul7media,   Hypermedia  and  Telecommunica7ons,  Tampere,  Finland,  June  25-­‐30,   998–1001.     9  
  10. 10. 6/12/12   Examples  of  Posi3ve  Educa3onal   Purpose   Experiences  •  Collec3on  of  RLOs   •  Case-­‐based  problem  solving  •  Sehng  up  Repository  of  Cases   •  Increasing  experience  •  Use  of  Cases   •  Mastery  learning   Raw  Asset  à  Learning  Asset  (annotated  image)  à  Task  or  Exercise   Raw  Asset  à  Learning  Asset  (annotated  image)  à  Task  or  Exercise   10  
  11. 11. 6/12/12   Building  Up  Online  Repositories   •  Over  2  years  (2010  to  2012),  2  online  repositories  set   up  containing  over  1000  case  examples  covering  CT  Lets  look  at  an  example  of  a  case   scans  of  the  brain;  and  over  1000  CXRs  (Chest   Radiographs)   repository   •  Radiology  images  are  hyperlinked  in  2  categories  (both   unannotated/not  labeled;  and  annotated  with   diagnosis/searchable  online)   •  Material  used  and  reused  in  different  formats/ platorms  –  including  websites,  blogs,  face  to  face  case   based  teaching  and  guided/self  directed  learning;  and   index  linked  to  other  repositories  of  similar  or   complementary  cases   Online  Case  Repository   11  
  12. 12. 6/12/12   12  
  13. 13. 6/12/12   Imagine  the  teaching  and  learning  possibili3es  with  1000s  of   cases  in  eBook  format  on  (internet  enabled)  tablets  /   smartphones    Imagine  if  the  cases  can  be  selected,  grouped  and  reordered  to   fit  any  learning  or  teaching  purpose   We  can  go  beyond  radiology  images   à  pathology  à  clinical  findings   13  
  14. 14. 6/12/12  Advantages  of  online  case  repositories   Problems  Using  RLO  Approach  •  For  Tutors   •  Formalizing  System  and  Procedures  for  Regular   –  Online  repository  facilitates  access   Systema3c  Collec3on  of  RLOs   –  Unannotated  case  images  give  greatest  flexibility  for   •  Quality  control  and  peer  review   repurposing  and  different  educa3onal  usage   –  URL  linkage  for  each  case  allows  easy  incorpora3on  of   •  Metadata,  tagging,  labeling  and  finding  resources,  and   case  material  on  websites,  blogs  etc.  (this  addresses  the   ensuring  pa3ent  confiden3ality/anonymity   issue  of  copyright  à  use  a  hyperlink,  similar  to  a  cita3on   •  Lack  of  awareness  in  academic  culture  –  u3liza3on  and   in  tradi3onal  academic  publica3on)   sharing  •  For  Residents   –  Online  case  repositories  can  be  an  efficient  method  for   •  Reluctance  to  make  unpublished  teaching  material   reviewing  many  cases   publicly  available   –  Facilitates  personalized,  customized  self-­‐directed  or   •  Lack  of  reward  systems  and  recogni3on   guided  learning   –  Koppi,  Bogle  &  Bogle,  2005   14  
  15. 15. 6/12/12   How  is  exper3se  and  diagnos3c  mastery  in  radiology  developed?   Take  Advantage  of  Regular  Clinical   •  1.  exper3se  in  radiology  is  directly  correlated  with  experience   Ac3vi3es  as  Source  of  RLOs   •  •  2.  case  experience  should  be  cumula3ve   3.  case  experience  should  include  full  spectrum  of  the  presenta3on  of  any  given  diagnosis   •  4.  case  experience  should  include  material  of  increasing  complexity   •  Day  to  day  work  à  Systema3c  Case  collec3on   •  5.  increasing  complexity  involves  cases  showing  typical,  then  less  typical,  less  easily   detectable  features,  with  less  associated  features,  or  more  confounding  associated   (annotated  in  PACS,  or  use  of  mobile  phone   features,  or  more  than  one  pathology,  and  progressively  less  or  contradictory  addi3onal   informa3on   camera  and  online  repositories)   •  6.  radiology  task  is  simple  three  step  process  -­‐  see,  recognize,  diagnose   •  Weekly  clinical  case  audits  (5%)  à  Good   •  7.  this  requires  perceptual  learning  or  pagern  recogni3on,  categorical  learning  or  the   ability  to  list  key  features,  and  diagnos3c  reasoning  or  the  ability  to  correlate  with  the   examples  of  normal  and  discrepancies  used   clinical  sehng   for  teaching  and  self  directed  learning   •  8.  experts  use  rapid  wholis3c  percep3on  linked  to  experience,  which  are  mental  schemas   or  representa3ons  of  the  key  features  of  a  wide  range  of  pathology   9.  experts  use  rapid  200msec  big  picture  percep3on  to  generate  the  most  likely   •  Weekly  clinical-­‐radiology-­‐pathology  rounds  à   •  hypothesis,  which  then  drives  deliberate  search  for  key  features   case  material  reused  for  residency  self   •  10.  experts  rapidly  decide  on  whether  radiograph  or  scan  is  normal,  or  has  a  likely   abnormality,  and  then  use  visual  search  and  reasoning  to  confirm  diagnosis   directed  learning  or  assessment   •  11.  online  case  repositories  can  accelerate  the  development  of  exper3se,  by  widening  the     •  case  experience    of  radiologists,  beyond  that  of  individual  day  to  day  direct  experience   12.  the  key  is  to  use  case  material  in  a  deliberate  way   Take  Home  Points   Future  role  of  educator  (s)  •  Mastery  is  the  ability  to  recognize  all  possible  presenta3ons  of  a   par3cular  condi3on,  and  differen3ate  this  from  look-­‐alike’s  and   differen3al  diagnoses  –  with  speed,  accuracy,  confidence   •  Knowledge  transmission  to  knowledge  filterer,  •  Start  with  common  clinical  condi3ons/presenta3ons   editor,  facilitator  •  Gain  exposure  to  all  possible  presenta3ons,  and  go  deeper  to   understand  the  underlying  anatomical/pathological/clinical   •  Coach,  resource  person,  systema3cally  collect   background  •  This  way  will  recognize  atypical  condi3ons,  or  a  new  condi3on   (full  range)  and  link  resources  /  form  interest  •  Systema3c  deliberate  prac3ce  (with  feedback)  is  the  key  à   groups   training  effect  /  lots  of  prac3ce  •  Aim  for  at  least  40  to  50  examples  of  each  diagnosis   •  Ini3ate  discussion,  ask  ques3ons,  give  advice,  •  Plan  for  spaced  (distributed)  rather  than  massed  prac3ce  –   provide  informa3on,  suggest  resources   allocate  3me  for  systema3c  prac3ce  •  Use  image  repositories  intelligently  –  digital,  hardcopy,   textbooks/journals   15  
  16. 16. 6/12/12   Typical  textbook  example(s)  à  Aim  to  gain  exposure  to  many,  many  more  (the  whole  spectrum)  Thank  you   Q  and  A   16