Photographer Paths: Sequence Alignment of Geotagged Photos for Exploration-based Route Planning

1,620 views

Published on

Slides for the talk I gave at CSCW 2013, held in San Antonio, TX, USA.

The full paper reference is:

El Ali, A., van Sas, S. & Nack, F. (2013). Photographer Paths: Sequence Alignment of Geotagged Photos for Exploration-based Route Planning. In proceedings of the 16th ACM Conference on Computer Supported Cooperative Work and Social Computing (CSCW '13), 2013, San Antonio, Texas.

Paper link: http://staff.science.uva.nl/~elali/pdfs/p985-el-ali.pdf

Published in: Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
1,620
On SlideShare
0
From Embeds
0
Number of Embeds
10
Actions
Shares
0
Downloads
12
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Photographer Paths: Sequence Alignment of Geotagged Photos for Exploration-based Route Planning

  1. 1. Photographer Paths: Sequence Alignment of GeotaggedPhotos for Exploration-based Route Planning!Feb.  26,  2013 Abdallah  ‘Abdo’  El  Ali   Sicco  van  Sas   Frank  Nack   h6p://staff.science.uva.nl/~elali/  
  2. 2. Outline! I.  Introduc3on   II.  Photographer  Paths   III.  User  Evalua3on   IV.  Results   V.  Discussion  &  Future  Work  2
  3. 3. Introduction3
  4. 4. 4
  5. 5. 5
  6. 6. Pffttt…6
  7. 7.    We  don’t  always  want  to   supply  user  preferences   7
  8. 8.  Social  and  local  interpreta3on  of  city  places  and  routes  8
  9. 9. Off-­‐the-­‐beaten  track,  social  trails,  9  ≠  Lonely  Planet!    
  10. 10.  Assump3on:    Loca3ons  of  photographs  are  poten3ally  interes3ng  10
  11. 11.  But  sequen3al  property  needs  to  be  captured!  11
  12. 12.  Sequence  Alignment  methods  12
  13. 13.  Analysis  of  mobility  behavior  of  city   photographers:     where  photographers  have  been     in  what  order  they  have  been   there     how  closely  their  movements   parallel  those  of  other   photographers  13 By Keiichi Matsuda via supercolossal
  14. 14. Research Questions!  How  can  walkable  route  plans  be  automaCcally  generated  for  residents  (and  tourists)   that  would  like  to  explore  a  city?    And  are  these  route  plans  desirable?    Three  factors:    1)  Which  data  sources?    2)  Which  methods  to  generate  routes?                      3)  User  percep3ons  compared  to  fastest   and  popular  routes?    14
  15. 15. Photographer Paths!15
  16. 16. Approach!  1)  Crawl  Flickr    geotags,  3mestamps    2)  Map  each  geotag/loca3on  in  a  sequence  to  a  cell  in  a  par33oned  grid  map    3)  Mul3ple  Sequence  Alignment  on  photographer  routes  to  find  aligned  loca3on  sequences      These  alignments  are  Photographer  Route  Segments  (PRSs)  16
  17. 17. Dataset!   Flickr  geotagged  photos  within  Amsterdam,  The  Netherlands       Area:  17.3  km  N-­‐S  and  24.7  km  E-­‐W  (center)     5-­‐year  period  (Jan.  2006  -­‐  Dec.  2010)     Aeributes:       owner  ID     photo  ID     date  and  3me-­‐stamp     la3tude  and  longitude  (street  level  accuracy)     Database:  426,372  photos  17
  18. 18. Preprocessing!   Sequence  inference  with  following  constraints:     photo  taken  within  4  hours  from  previous  photo  and  in  same  order     minimum  2  or  more  different  loca3ons  (or  nodes)     early  experiments  determined  125  x  125m  cells  in  center  of  Amsterdam  grid  suitable     1691  routes  (average  length  of  9.92  loca3ons)     1130  unique  photographers  18
  19. 19. Sequence Alignment!   To  find  photographer  paths  from  Photographer  Route  Segments  (PRSs),  constraints  set:     PRS  has  minimum  4  photographers  with  minimum  2  aligned  nodes/loca3ons   !#**% Photographer Route Segments (PRSs)! !"#$% !"**% !+**%%&()*+$,*-.*)/*0$ !***% $**% ##&% +%,-./.012,-314%        231  PRSs         #**% !&% )%,-./.012,-314%  (average  length  of  2.61  nodes)   "%,-./.012,-314% "**% !"#$ +**% !(% !*!% +% !*% !% )% *% +% )% "% % 1)-.*$&2/342)0$ 19
  20. 20. PRSs in Amsterdam!20
  21. 21. PRS Aggregation!   Modified  Dijkstra’s  shortest  path  algorithm   & P1N1 P2N1 & # ! P1N2 ! " # !" #$% Start P1N3 P2N2 End21
  22. 22. PRS Aggregation to Crude Routes! PRSs CM Photographer Route WW Photographer Route22
  23. 23. User Evaluation23
  24. 24. Laboratory Study Design!   ~45  min.  Quan3ta3ve/Qualita3ve  lab-­‐based  study     15  par3cipants  (10  m,  5  f)  aged  between  21-­‐35  (M  =  29.2;  SD  =  3.3)     Interac3ve  web-­‐based  prototype  route  planner     Expert  route  evalua3on  by  ‘city  residents’  (lived  in  Amsterdam  >  1  year)     Plain  routes  to  avoid  informa3on  type  bias  24
  25. 25. Laboratory Study Design!   Two  scenarios:     Route  1:  Central  Sta3on  to  Museumplein    anernoon  scenario  favoring  explora3on     Route  2:  Waterlooplein  to  Westerkerk    evening  scenario  favoring  efficiency     Baseline  comparisons:       Photo  Density  (PD)  route:  highest  density  of  photos  (over  5  year  period)  in  grid  cells   along  route       Google  Maps  (GM)  route:  shortest  route  between  two  loca3ons     Counterbalanced  within-­‐subject  design       Route  Varia3on  (IV):  Photographer  Paths  vs.  Photo  Density  vs.  Google  Maps  25
  26. 26. Central Station to Museumplein (CM)! Photographer Paths Photo Density Google Maps route (5.36 km) route (3.83 km) route (3.35 km)26
  27. 27. Waterlooplein to Westerkerk (CM)! Photographer Paths route (2.28 km) Photo Density route (2.60 km) Google Maps route (1.59 km)27
  28. 28. Laboratory Study Design!  Data  collected:   1.  AerakDiff2  (Hassenzahl,  2003)  UX  ques3onnaire  responses  [7-­‐point  seman3c   differen3al  scale]:  Usability,  Hedonic  Quali3es  (Iden3ty,  S3mula3on),  Aerack3veness   2.  Two-­‐part  semi-­‐structured  interviews    Part  1:  Route  preferences,  feedback  on  Photographer  Paths    Part  2:  Inves3ga3on  of  visualized  informa3on  types  (visualized  info  type  handouts):      a)  Google  maps      b)  Color  coded  PRSs  (PP  route)      c)  Density  geopoints  (PD  route)    d)  Thumbnail  photo  geopoints      e)  Foursquare  POIs  28
  29. 29. a) d) b) e) c)29
  30. 30. Web Survey Study!   Short  web-­‐based  survey  for  CM  and  WW  routes  and  varia3ons     Basic  demographics  collected:  age,  gender,  years  in  Amsterdam     Sta3c  route  images,  no  counterbalancing     82  par3cipants  (55  m,  27  f)  aged  between  17-­‐62  (M=  27.6;  SD=  6.1)       Most  lived  in  Amsterdam  for  more  than  3  years  (44/82)       Some  between  1-­‐3  years  (15/82)     Less  than  a  year  (11/82)       Only  visited  before  (12/82)  30
  31. 31. Results31
  32. 32. AttrakDiff2 ! Central Station to Museumplein (CM) Route "# ** ** ** ** * ** ** * $# %# !"#$% &# !%# !$# !"# ()*+),-#./)0123#4.5# 6789:1-#./)0123#!#;87:,23# 6789:1-#./)0123#!# =>()-,?7:7@@#4=AA5# 46.!;5# <,+/0),9:#46.!<5# &("$)*$% ** !"#"$%$& * !"#"$%$ B929*()CB7(#)2B@# B929#D7:@123# E99*07#F)C@#32
  33. 33. AttrakDiff2 ! Waterlooplein to Westerkerk (WW) Route ** ** "# ** ** $# %# !"#$% &# !%# !$# !"# ()*+),-#./)0123#4.5# 6789:1-#./)0123#!#;87:,23# 6789:1-#./)0123#!# =>()-,?7:7@@#4=AA5# 46.!;5# <,+/0),9:#46.!<5# &("$)*$% !"#"$%$& ** B929*()CB7(#)2B@# B929#D7:@123# E99*07#F)C@#33
  34. 34. Route Preference! Lab  Study     CM  route:  most  chose  to  follow  the  PP  route  (9/15),  PD  route  (4/15),  GM  route  (2/15)     “One  of  the  routes  [PP]  was  long  and  took  many  detours,  and  I  thought  that  was  a   very  aFracHve  route!”     WW  route:  most  chose  to  follow  GM  route  (10/15),  PD  route  (4/15),  no  route  (1/15)       “You  are  going  for  coffee  so  you  just  want  to  get  there,  unlike  in  the  first  [CM]   scenario  where  it  is  a  nice  day  and  you  have  Hme.”   Web  Survey   •           CM  route:  GM  (40/82),  PD  (23/82),  PP  route  (10/82),  neither  (9/82)   •   No  experimenter  steering;  many  Amsterdam  residents  know  the  city  already  quite   well!   •   “I  would  not  easily  walk  these  routes...  who  in  Amsterdam  walks?  ;)”   •           WW  route:  GM  (67/82),  PD  (6/82),  PP  route  (3/82),  neither  (6/82)  34
  35. 35. Digital Information Aids! Lab  Study                Interview:  Part  1     “How  many  persons  (focus  on  city  photographers)  took  a  given  route  segment  over  a  certain  3me   period  (e.g.,  1  year)?”     Useful  (8/15)  for  exploring  a  city  one  already  knows     Not  sure  (4/15)     Depends  on  which  photographers  (2/15)     Not  for  me  (1/15)    Interview:  Part  2     Found  PP  info  type  aerac3ve  (10/15),  but  combine  with  Photo  thumbnails  (3/10)  and  POIs  (3/10)  35
  36. 36. Digital Information Aids! Web  Survey     POIs  along  a  route  (51x)     Route  distance  (51x)     Comments  along  a  route  (ranked  by  highest  ra3ngs  or  recency)  (24x)     Expert  travel  guides  (22x)     Photos  of  route  segments  (17x)     No  digital  aids  (13x)     Number  of  photographers  that  took  a  given  path  over  a  Hme  period  (9x)     Number  of  photos  along  a  route  over  a  3me  period  (9x)  36
  37. 37. Discussion37
  38. 38. Discussion!   Discrepancy  between  lab-­‐study  and  web  survey     Quick  web  survey  insufficient?       Visualiza3on/explana3on  of  digital  aids  important?     Proof-­‐of-­‐concept  approach  requires  real-­‐world  ‘outdoor’  evalua3on     Different  street  grid  network     Scalability  to  larger  ci3es     More  context-­‐awareness  38
  39. 39. Take Home Message! Going  towards  data-­‐driven  explora3on-­‐based  route  planners…     Some3mes  it’s  the  journey,  not  the  des3na3on     A  quan3ta3ve  approach  may  oversimplify  human  needs  for  explora3on     But  some3mes  we  want  an  automa3c  solu3on,  so  as  not  to  be  bothered   with  supplying  user  preferences  and  encounter  serendipity  39
  40. 40. Questions40 h6p://staff.science.uva.nl/~elali/  
  41. 41. References!  1.  Cheng,  A.-­‐J.,  Chen,  Y.-­‐Y.,  Huang,  Y.-­‐T.,  Hsu,  W.  H.,  and  Liao,  H.-­‐Y.  M.  Personalized  travel   recommenda3on  by  mining  people  aeributes  from  community-­‐contributed  photos.  In  Proc.   MM  ’11,  ACM  (2011),  83–92.    2.  M.  Clements,  P.  Serdyukov,  A.  P.  de  Vries,  and  M.  J.  Reinders.  Using  flickr  geotags  to  predict   user  travel  behaviour.  In  Proc.  SIGIR  ’10,  pages  851–852.  ACM  Press,  2010.   3.  M.  De  Choudhury,  M.  Feldman,  S.  Amer-­‐Yahia,  N.  Golbandi,  R.  Lempel,  and  C.  Yu.   Automa3c  construc3on  of  travel  i3neraries  using  social  breadcrumbs.  In  Proc.  HT  ’10,  pages   35–44.  ACM  Press,  2010.   4.  F.  Girardin,  F.  Calabrese,  F.  D.  Fiore,  C.  Ra|,  and  J.  Blat.  Digital  footprin3ng:  Uncovering   tourists  with  user-­‐generated  content.  IEEE  Pervasive  Compu3ng,  7:36–43,  October  2008.   5.  N.  Shoval  and  M.  Isaacson.  Sequence  alignment  as  a  method  for  human  ac3vity  analysis  in   space  and  3me.  Annals  of  the  Associa3on  of  American  Geographers,  97(2):282–297,  2007.   6.  A.  Vaccari,  F.  Calabrese,  B.  Liu,  and  C.  Ra|.  Towards  the  socioscope:  an  informa3on  system   for  the  study  of  social  dynamics  through  digital  traces.  In  Proc.  GIS  ’09,  pages  52–61.  ACM   Press,  2009.   7.  Hassenzahl,  M.,  Burmester,  M.,  and  Koller,  F.  AerakDiff:  Ein  Fragebogen  zur  Messung   wahrgenommener  hedonischer  und  pragma3scher  Qualit¨at.  Mensch  &  Computer  2003.   Interak3on  in  Bewegung  (2003),  187–196.   8.  Lu,  X.,  Wang,  C.,  Yang,  J.-­‐M.,  Pang,  Y.,  and  Zhang,  L.  Photo2trip:  genera3ng  travel  routes   from  geo-­‐tagged  photos  for  trip  planning.  In  MM  ’10,  ACM  (2010),  143–152.   9.  Wilson,  C.  Ac3vity  paeerns  in  space  and  3me:  calcula3ng  representa3ve  hagerstrand   trajectories.  TransportaHon  35  (2008),  485–499.  41
  42. 42. Related Work!   Sequence  Alignment  (SA)  methods:     Borrowed  from  bioinforma3cs  and  later  3me  geography     Time  geography  systema3cally  analyzes  and  explores  the  sequen3al  dimension  of  human  spa3al   and  temporal  ac3vity  (Shoval  &  Isaacson,  2007).       Visualize  human  movement  on  2-­‐D  plane:  x-­‐  &  y-­‐  axis    longitude  and  la3tude;  z-­‐axis    3me      useful  for  analyzing  sequences  of  human  ac3vity  (in  this  case,  photo-­‐taking  behavior  of   photographers)     Photo-­‐based  City  Modeling:     Understand  tourist  site  aerac3veness  based  on  geotagged  photos  (Girardin  et  al.,  2008)     Construct  inter-­‐city  travel  i3neraries  (De  Choudhury  et  al.,  2010)     Generate  personalized  Point-­‐of-­‐Interest  (POI)  recommenda3ons  of  where  to  go  in  a  city  based  on   the  users  travel  history  in  other  ci3es  (Clements  et  al.,  2010)      Approaches  focus  on  describing  loca3ons,  not  on  fine-­‐grained  within-­‐city  routes  that  connect   them     Non-­‐efficiency  Driven  Route  Planners     Automa3c  genera3on  of  travel  plans  based  on  millions  of  photos  (Lu  et  al.,  2010)     Personalized  data-­‐driven  travel  route  recommenda3ons  (Cheng  et  al.  2011)      Systems  geared  towards  recommending  hotspots  and  popular  routes,  not  off-­‐beat  explora3on  42 routes  
  43. 43. Sequence Alignment Overview! Input:  two  sequences  over  the  same  alphabet   Output:  an  alignment  of  the  two  sequences   Example:      Source:  GCGCATGGATTGAGCGA      Target:    TGCGCCATTGATGACCA     A  possible  alignment:      -­‐GCGC-­‐ATGGATTGAGCGA      TGCGCCATTGAT-­‐GACC-­‐A   Three  opera3ons  (each  with  cost):     Perfect  matches  (MATCH)     Mismatches  (DEL)     Inser3ons  &  dele3ons  (INDEL)     The  less  distance  cost,  the  higher  the  similarity  between  two  sequences   (Shoval & Isaacson, 2007)43
  44. 44. Multiple Sequence Alignment Overview!   Used  ClustalTXY  sonware  (Wilson  et  al.,  2008)  for  photo  alignment:     makes  full  use  of  mul3ple  pairwise  sequence  alignments,  where  alignments  are  computed  for   similarity  in  parallel     uses  a  progressive  heuris3c  to  apply  mul3ple  sequence  alignment  (MSA)     allows  elements  to  be  represented  with  up  to  12-­‐character  words,  which  allows  unique   representa3on  of  small  map  regions,  used  for  represen3ng  the  geotagged  photos     to  deal  with  differences  in  sequence  length,  ClustalTXY  adds  gap  openings  and  extensions  to   sequences.       MSA  in  3  stages:    1)  Pairwise  alignments  are  computed  for  all  sequences    2)  Aligned  sequences  are  grouped  together  in  a  dendogram  based  on  similarity    3)  Dendogram  used  as  a  guide  for  mul3ple  alignment  44

×