Use	  of	  Subimages	  in	  Fish	  Species	  Iden5fica5on:	  A	  Qualita5ve	  Study	  15	  June	  2011	  JCDL	  OGawa	     ...
Talk	  overview	  •  A	  user	  study	  about	  use	  of	  subimages	  and	  of	  a	     prototype	  digital	  library	  i...
Mo5va5on:	  many	  scholarly	  tasks	  involve	  working	  with	  subdocuments	                                           ...
Problems	  •  Informa5on	  is	  heterogeneous,	  voluminous,	     distributed	  across	  loca5ons,	  and	  it	  is	  chall...
A	  digital	  library	  (DL)	  includes	  a	  repository	  of	  collec5ons	  and	  metadata	  +	  services	  	            ...
Scenario	  While	  learning	  how	  to	  iden5fy	  fish,	  a	  student	  might	  make	  use	  of	  an	  advanced	  query	  ...
Searching	  for	  such	  fine-­‐grain	  contextual	  informa5on	  is	  not	  supported	  by	  current	  DLs	  	    Darter	 ...
Superimposed	  informa5on	  (SI)	  enables	  working	  with	  contextualized	  subdocuments	         superimposed	  (new)	...
Hypothesis	                              enhanced	                              support	  to	  SI-­‐DL	   =	     +	     sc...
SuperIDR	  –	  a	  prototype	  personal	  SI-­‐DL	  •  Enables	  marking	  and	  annota5ng	  images	  •  Combines	  text-­...
Task:	  fish	  species	  iden5fica5on	       Involves	  working	  with	  parts	  of	  fishes	  and	  fish	  images	  1a	  	  P...
SuperIDR:	  species	  details	  interface	                                                 12	  
Results	  of	  a	  classroom-­‐based	  user	  study	  of	  SuperIDR	                                                      ...
A	  qualita5ve	  study	  of	  subimage	  and	  SuperIDR	  use	  •  How	  do	  people	  use	  subimages	  in	  fish	     ide...
Study	  design	  criteria	   Maximize	  use	  of	  SuperIDR	    Par5cipants	  with	             Use	  for	  a	  long	  dur...
-­‐	  Study	  overview	  -­‐	  Informed	  consent	  -­‐	  SuperIDR	  setup	                   Qualita5ve	  study	  procedu...
-­‐	  Study	  overview	  -­‐	  Informed	  consent	  -­‐	  SuperIDR	  setup	                   Qualita5ve	  study	  procedu...
-­‐	  Study	  overview	  -­‐	  Informed	  consent	  -­‐	  SuperIDR	  setup	                   Qualita5ve	  study	  procedu...
-­‐	  Study	  overview	  -­‐	  Informed	  consent	  -­‐	  SuperIDR	  setup	                   Qualita5ve	  study	  procedu...
Data	                             Interview	  responses	  analysis	  was	                                                 ...
Analyzed	  par5cipants	  based	  on	  6	  par5cipants:	                        fisheries	  and	  fish	  iden5fica5on	        ...
A	  qualita5ve	  study	  of	  subimage	  and	  SuperIDR	  use	  •  How	  do	  people	  use	  subimages	  in	  fish	     ide...
Subimage/annota5on	  characteris5cs	  •  940	  subimages/	  annota5ons	  •  13	  recurring	  types	                       ...
Color:	          Red	  spots	  or	           bars,	  some	             blue	  bars	  LocaFon:	  Eyes	  almost	  on	  top	 ...
Morphological	  comparison:	                                   Tail	  fins	  with	  pointed	  lobes	  about	  equal	  in	  ...
Comparison	  with	  other	  informaFon	  objects	  (species):	                                                            ...
A	  qualita5ve	  study	  of	  subimage	  and	  SuperIDR	  use	  •  How	  do	  people	  use	  subimages	  in	  fish	     ide...
Strategies	  and	  contexts	  of	  subimage	  use	  in	  fish	  iden5fica5on	  •  Subimages	  are	  necessary	  in	  fish	  i...
Subimages	  in	  species	  learning	  methods	     Fish	  drawing	  in	  a	  book	     Notecard	                          ...
Subimage	  use	  in	  SuperIDR	  •  Marking	  and	  annota5on	      (940	  subimages	  and	  annota5ons)	  •  Browse	  	  ...
Comparing	  similar	  species	                                        31	  
SuperIDR:	  combined	  search	  Query	  interface	                              Search	  results	                         ...
Complex	  object	  as	  a	  query	  
Context	  	  “...	  It	  would	  not	  work	  if	  you	  said	  that	  this	  fish	  has	  dark	  spots.	  You	  know	  you...
SI-­‐DL	      “It	  [SuperIDR]	  is	  pulling	  together	  different	  ways	  of	  gekng	  to	      informa5on	  ...	  So,	...
Recommenda5ons	  for	  design	  of	  an	  SI-­‐DL	  •  Subdocuments	  as	  first-­‐class	  digital	  objects	  in	  a	     ...
Recommenda5ons	  for	  design	  of	  an	  SI-­‐DL	  •  Mul5ple	  ways	  to	  describe,	  access,	  browse,	     search,	  ...
Conclusions	  •  Working	  with	  subimages	  is	  important	  and	  necessary	     in	  fish	  species	  iden5fica5on	  •  ...
Resources	  and	  pointers	  hGp://umurthy.com/research/superidr	  	  •  SuperIDR	  informa5on	  and	  papers	  •  Uma	  M...
Acknowledgments	  •  Par5cipants	  •  Dr.	  Naren	  Ramakrishnan,	  Yinlin	  Chen,	  Nadia	     Kozievitch,	  Evandro	  Ra...
Thank	  you	                                                     ?	  Contact:	  Uma	  Murthy	  (umurthy@vt.edu)	  Lin	  Tz...
Appendix	                 42	  
Photo	  aGribu5ons	  (Flickr)	  •  A	  digital	  library	  by	  HacksHaven	  •  Art	  History	  With	  Chris	  And	  Mac	 ...
Photo	  aGribu5ons	  (Flickr)	  •  Everyone	  gekng	  in	  on	  macroinvertebrate	     iden5fica5on	  by	  USFWS/Southeast	...
SuperIDR	  architecture	  
Study	  setup	                       46	  
Summary	  of	  findings	  of	  qualita5ve	  study	  •  13	  types	  of	  subimages/annota5ons	  from	  940	     subimages/a...
Upcoming SlideShare
Loading in...5
×

Use of Subimages in Fish Species Identification - JCDL 2011 Talk

524

Published on

Slides from talk at JCDL 2011

1 Comment
1 Like
Statistics
Notes
  • Uma Murphy's dissertation work. Nice qualitative study of a digital library/annotation system in use. See also http://umurthy.com/research/superidr
    JCDL 2011 paper, 'Use of Subimages in Fish Species Identification: A Qualitative Study'
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
No Downloads
Views
Total Views
524
On Slideshare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
9
Comments
1
Likes
1
Embeds 0
No embeds

No notes for slide

Use of Subimages in Fish Species Identification - JCDL 2011 Talk

  1. 1. Use  of  Subimages  in  Fish  Species  Iden5fica5on:  A  Qualita5ve  Study  15  June  2011  JCDL  OGawa   hGp://umurthy.com/research/superidr    Uma  Murthy  Eric  Hallerman   Lin  Tzy  Li  Edward  Fox   Ricardo  Torres  Manuel  Perez-­‐Quinones   Lois  Delcambre  
  2. 2. Talk  overview  •  A  user  study  about  use  of  subimages  and  of  a   prototype  digital  library  in  fish  species   iden5fica5on  •  Findings  and  implica5ons   2  
  3. 3. Mo5va5on:  many  scholarly  tasks  involve  working  with  subdocuments   3  
  4. 4. Problems  •  Informa5on  is  heterogeneous,  voluminous,   distributed  across  loca5ons,  and  it  is  challenging  to   manage,  organize,  access,  retrieve,  and  use.  •  Tools/methods  (including  paper-­‐based  and  digital)   are  not  well-­‐integrated.   Ineffec5ve  and  inefficient  task  execu5on   4  
  5. 5. A  digital  library  (DL)  includes  a  repository  of  collec5ons  and  metadata  +  services     5  
  6. 6. Scenario  While  learning  how  to  iden5fy  fish,  a  student  might  make  use  of  an  advanced  query  that  involves  searching  for  subimages,  in  context  of  other  informa5on,  incl.  other  subimages:   Find  me  species  that   are  darters  that  have   a  dorsal  fin  that   looks  like  this,  which   is  connected  to   another  dorsal  fin   that  looks  like  this,   which  might  has  an   orange-­‐colored  edge   6  
  7. 7. Searching  for  such  fine-­‐grain  contextual  informa5on  is  not  supported  by  current  DLs     Darter   Find  me  species  that   Two  dorsal   are  darters  that  have   Orange  edge   fins  connected   a  dorsal  fin  that   looks  like  this,  which   is  connected  to   another  dorsal  fin   that  looks  like  this,   which  might  have  an   orange  hue  on  its   edge   7  
  8. 8. Superimposed  informa5on  (SI)  enables  working  with  contextualized  subdocuments   superimposed  (new)   informa)on   base  (exis)ng)   informa)on   8  
  9. 9. Hypothesis   enhanced   support  to  SI-­‐DL   =   +   scholarly     tasks  with   subdocuments   9  
  10. 10. SuperIDR  –  a  prototype  personal  SI-­‐DL  •  Enables  marking  and  annota5ng  images  •  Combines  text-­‐  and  content-­‐based  image   descrip5on,  retrieval,  and  browsing,  including   for  subimages   10  
  11. 11. Task:  fish  species  iden5fica5on   Involves  working  with  parts  of  fishes  and  fish  images  1a    Paired  fins  absent;  jaws  absent,  mouth  in  an  oral          disk  (the  disk  mostly  surrounded  by  a  fleshy  hood          in  larvae);  7  external  gill  openings  present  in  row          behind  eye  ..........  Lampreys  -­‐  Petromyzon5dae    p.  ooo  1b    Paired  fins  present  (at  least  1  set);  jaws  present;          1  external  gill  opening  per  side  ...................................  2  2a    Caudal  fin  heterocercal  or  abbreviate  heterocercal          (Figure  5)  ............................................................  3  2b    Caudal  fin  protocercal  (Figure  13,  Part  2,  upper  leh)          or  homocercal  (Figure  5)  ............................................  6  Dichotomous  keys   Personal  Notes   11  
  12. 12. SuperIDR:  species  details  interface   12  
  13. 13. Results  of  a  classroom-­‐based  user  study  of  SuperIDR   13  
  14. 14. A  qualita5ve  study  of  subimage  and  SuperIDR  use  •  How  do  people  use  subimages  in  fish   iden5fica5on?   –  Characteris5cs  of  subimages  and  related   informa5on   –  Contexts  and  strategies  of  working  with   subimages  in  fish  iden5fica5on  •  How  does  SuperIDR  support  that  use?   14  
  15. 15. Study  design  criteria   Maximize  use  of  SuperIDR   Par5cipants  with   Use  for  a  long  dura5on   In  natural  sekng  and   interest  in  fish  ID   targeted  tasks  Collect  data,  in  mul5ple  ways  and  from  mul5ple  sources,  on  subimage  and  SuperIDR  use  in  fish  ID   15  
  16. 16. -­‐  Study  overview  -­‐  Informed  consent  -­‐  SuperIDR  setup   Qualita5ve  study  procedures  -­‐  SuperIDR  training  -­‐  Pre-­‐study  interview   SuperIDR  use  in   work/study  (diary)   Unknown  specimen   iden5fica5on  tasks   SuperIDR  use  in   work/study  (diary)   Unknown  specimen   iden5fica5on  tasks   SuperIDR  use  in   work/study  (diary)   Post-­‐study  interview   16  
  17. 17. -­‐  Study  overview  -­‐  Informed  consent  -­‐  SuperIDR  setup   Qualita5ve  study  procedures  -­‐  SuperIDR  training  -­‐  Pre-­‐study  interview   SuperIDR  use  in   work/study  (diary)   Unknown  specimen   iden5fica5on  tasks   SuperIDR  use  in   work/study  (diary)   Unknown  specimen   iden5fica5on  tasks   SuperIDR  use  in   work/study  (diary)   Post-­‐study  interview   17  
  18. 18. -­‐  Study  overview  -­‐  Informed  consent  -­‐  SuperIDR  setup   Qualita5ve  study  procedures  -­‐  SuperIDR  training  -­‐  Pre-­‐study  interview   SuperIDR  use  in   work/study  (diary)   Unknown  specimen   idenFficaFon  tasks   SuperIDR  use  in   work/study  (diary)   Unknown  specimen   idenFficaFon  tasks   SuperIDR  use  in   work/study  (diary)   Post-­‐study  interview   18  
  19. 19. -­‐  Study  overview  -­‐  Informed  consent  -­‐  SuperIDR  setup   Qualita5ve  study  procedures  -­‐  SuperIDR  training  -­‐  Pre-­‐study  interview   SuperIDR  use  in   work/study  (diary)   Unknown  specimen   iden5fica5on  tasks   SuperIDR  use  in   work/study  (diary)   Unknown  specimen   iden5fica5on  tasks   SuperIDR  use  in   work/study  (diary)   Post-­‐study  interview   19  
  20. 20. Data   Interview  responses  analysis  was   Screen  captures  &  informed  by   spoken  thoughts  Grounded   Diary  Theory     entries   SuperIDR  logs  Species  ID  responses   Database   Species  ID  materials   image   20  
  21. 21. Analyzed  par5cipants  based  on  6  par5cipants:   fisheries  and  fish  iden5fica5on   experience,  current  projects   3  groups   and  fish  iden5fica5on  prac5ces  P2  (male),  P6  (male):  1-­‐2  years  of  experience,  undergraduates  (UG)  or  recent    UG  P1  (male),  P5  (female):  2-­‐5  years  of  experience,  Master’s  students,  working  on  theses  and/or  teaching/research  P3  (male),  P4  (female):  8-­‐10  years  of  experience,  PhD  students,  working  on  research  projects   21  
  22. 22. A  qualita5ve  study  of  subimage  and  SuperIDR  use  •  How  do  people  use  subimages  in  fish   iden5fica5on?   –  Characteris5cs  of  subimages  and  related   informa5on   –  Contexts  and  strategies  of  working  with   subimages  in  fish  iden5fica5on  •  How  does  SuperIDR  support  that  use?   22  
  23. 23. Subimage/annota5on  characteris5cs  •  940  subimages/  annota5ons  •  13  recurring  types   23  
  24. 24. Color:   Red  spots  or   bars,  some   blue  bars  LocaFon:  Eyes  almost  on  top  of  head   24  
  25. 25. Morphological  comparison:   Tail  fins  with  pointed  lobes  about  equal  in  length  ConnecFon/relaFonship:  Dorsal  fins  connected   25  
  26. 26. Comparison  with  other  informaFon  objects  (species):   Body  similar  to  cyprinella  CombinaFon  (color  and  count):  One  orange  and  one  blue  band  on  first  dorsal  fin   26  
  27. 27. A  qualita5ve  study  of  subimage  and  SuperIDR  use  •  How  do  people  use  subimages  in  fish   iden5fica5on?   –  Characteris5cs  of  subimages  and  related   informa5on   –  Contexts  and  strategies  of  working  with   subimages  in  fish  iden5fica5on  •  How  does  SuperIDR  support  that  use?   27  
  28. 28. Strategies  and  contexts  of  subimage  use  in  fish  iden5fica5on  •  Subimages  are  necessary  in  fish  iden5fica5on  •  In  learning  about  a  species  •  In  iden5fying  species     (top-­‐down  approach,  compare  similar  species)   Percina   Etheostoma   Perca   Ammocrypta   …..   Family   Genera   Species   28  
  29. 29. Subimages  in  species  learning  methods   Fish  drawing  in  a  book   Notecard   Printed  list   with  notes   Table  with   species   informaFon   29  
  30. 30. Subimage  use  in  SuperIDR  •  Marking  and  annota5on   (940  subimages  and  annota5ons)  •  Browse     (subimages  in  species  details,  comparison,  and   search  results)  •  Search   (subimages  in  text,  image,  and  combined  search,   and  in  complex  objects  as  queries)   30  
  31. 31. Comparing  similar  species   31  
  32. 32. SuperIDR:  combined  search  Query  interface   Search  results   32  
  33. 33. Complex  object  as  a  query  
  34. 34. Context    “...  It  would  not  work  if  you  said  that  this  fish  has  dark  spots.  You  know  you  get  hundreds  of  species  with  dark  spots.  But,  if  you  got  down  to  a  few  species  and  you  need  to  know  how  many  they  have  ..."  [P1  interview]  Manually  working  with  informaFon  It  depends  on  how  dis5nct  that  species  [is]  and  how  many  other  species  are  similar  to  that  species,  I  guess  …  I  would  never  trust  the  result,  I  guess,  100%  …you  know,  based  on  just  one  picture  and  a  liGle  bit  of  wriGen  text.  I  would  always  want  to  pull  up  other  species  that  are  somewhat  similar  and  just  do  a  visual  inspec5on  myself  to  be  sure  that  it  just  was  not  some  bad  [query]  image  that  I  used  or  a  bad  search  term."  [P3  interview]   34  
  35. 35. SI-­‐DL   “It  [SuperIDR]  is  pulling  together  different  ways  of  gekng  to   informa5on  ...  So,  not  only  do  I  have  a  taxonomy  [and]   dichotomous  key,  but  it  is  also  supported  by  images,  many   images  that  I  have  loaded  in  myself,  that  I  can  compare  and   contrast  right  there  in  the  program  [SuperIDR].  I  can  annotate   the  images,  so  I  know  that  I  kind  of  looking  somewhat  into   their  future  [use].  And  it  kind  of  just  pulls  all  those  tools   together,  more  so  than  [pulling  together]  informa5on.  It  gives   me  many  ways  of  accessing  the  same  informa5on.  The  more   ways  you  can  come  to  that  informa5on,  the  beGer  [it  is].   Because  it  is  always  going  to  make  you  more  confident  about   the  decision  that  you  are  making."  [P1  interview]   35  
  36. 36. Recommenda5ons  for  design  of  an  SI-­‐DL  •  Subdocuments  as  first-­‐class  digital  objects  in  a   DL   –  Store,  index,  access,  retrieve,  and  reuse  •  Preserving  and  using  context   –  Stored  in  metadata/links  to  digital  objects   –  Co-­‐presence  of  mul5ple  subimages   –  Along  with  other  criteria  (e.g.  family  and  loca5on)   36  
  37. 37. Recommenda5ons  for  design  of  an  SI-­‐DL  •  Mul5ple  ways  to  describe,  access,  browse,   search,  and  visualize  subdocuments   –  Search:  text  retrieval,  CBIR,  combined  search,  complex   objects  as  queries  •  Support  for  manually  working  with  as  well  as   automa5cally  processing  subdocuments   –  Subimage  annotation –  Auto-­‐complete,  and  search  refinement  (e.g.  using  an   ontology)   37  
  38. 38. Conclusions  •  Working  with  subimages  is  important  and  necessary   in  fish  species  iden5fica5on  •  An  SI-­‐DL  provides  enhanced  support  to  such   scholarly  tasks   –  Integrated  func5onality   –  Subdocuments  as  first-­‐class  objects  in  a  DL  •  Future  work:  Develop  web  interface  to  leverage   online  collec5ons  (Flickr)  to  study  applicability  in   other  domains   38  
  39. 39. Resources  and  pointers  hGp://umurthy.com/research/superidr    •  SuperIDR  informa5on  and  papers  •  Uma  Murthy’s  Ph.D.  disserta5on     “Digital  libraries  with  superimposed  informa5on  –  suppor5ng   scholarly  tasks  that  involve  fine-­‐grained  informa5on”     –  Includes  SuperIDR  source  code  •  Upcoming  panel  in  GHC  2011  on  conduc5ng   user  studies  remotely   39  
  40. 40. Acknowledgments  •  Par5cipants  •  Dr.  Naren  Ramakrishnan,  Yinlin  Chen,  Nadia   Kozievitch,  Evandro  Ramos,  Tiago  Falcao,   Seungwon  Yang,  Ricardo  Quintana-­‐Cas5llo,   Monika  Akbar,  Laurian  Vega,  Stacy  Branham,   Tejinder  Judge  •  Microsoh  tablet  PC  grant,  NSF  (DUE-­‐0435059   and  0840668),  CAPES  (BEX  1385/10-­‐0),  and   CPqD  Founda5on   40  
  41. 41. Thank  you   ?  Contact:  Uma  Murthy  (umurthy@vt.edu)  Lin  Tzy  Li  (lintzyli@ic.unicamp.br)  Edward  A.  Fox  (fox@vt.edu)    Web:   ?  hGp://umurthy.com/research/superidr     41  
  42. 42. Appendix   42  
  43. 43. Photo  aGribu5ons  (Flickr)  •  A  digital  library  by  HacksHaven  •  Art  History  With  Chris  And  Mac  6/9:  Manet:   Lecture  (Mme  Manet  and  Leon)  by   moonflowerdragon  •  Korean  music  by  Homies  In  Heaven  •  Old  annota5ons  by  Lorianne  DiSabato  •  Reading  Annota5on  by  Rosa  Say  
  44. 44. Photo  aGribu5ons  (Flickr)  •  Everyone  gekng  in  on  macroinvertebrate   iden5fica5on  by  USFWS/Southeast  •  studying  by  English106  •  My  1959  diary  entry  for  “the  day  the  music   died”  by  Snap  Man  •  database  2  by  Tim  Morgan  
  45. 45. SuperIDR  architecture  
  46. 46. Study  setup   46  
  47. 47. Summary  of  findings  of  qualita5ve  study  •  13  types  of  subimages/annota5ons  from  940   subimages/annota5ons  •  Subimages  are  important  and  necessary  in  fish   iden5fica5on  •  Iden5fica5on  top  down  way  •  Learning  using  mul5ple  methods  •  Context  is  important  •  Combined  search  and  using  a  complex  object  as  a   query  •  SI-­‐DL  –  bringing  together  capabili5es   47  
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×