Design	
  and	
  implementa.on	
  of	
  a	
  system	
  
    for	
  the	
  improved	
  searching	
  and	
  
 accessing	
  of	
  real-­‐world	
  SOS	
  services	
  
                             	
  
       Ben	
  Knoechel	
  (bcknoech@ucalgary.ca)	
  
      Chih-­‐Yuan	
  Huang	
  (huangcy@ucalgary.ca)	
  
       Steve	
  Liang	
  (steve.liang@ucalgary.ca)	
  
                   University	
  of	
  Calgary	
  
                                	
  
Outline	
  
•    Introduc.on	
  
•    Methodology	
  
•    Experimental	
  Results	
  
•    Conclusions	
  




                                        2	
  
Introduc.on	
  




                  3	
  
Introduc.on	
  
•  The	
  Open	
  Geospa.al	
  Consor.um	
  (OGC)	
  has	
  
   developed	
  the	
  Sensor	
  Web	
  Enablement	
  (SWE)	
  
•  SWE	
  is	
  a	
  set	
  of	
  standards	
  to	
  allow	
  people	
  to	
  
   share	
  and	
  interact	
  with	
  sensor	
  data	
  over	
  the	
  
   Internet	
  
•  The	
  Sensor	
  Observa.on	
  Service	
  (SOS)	
  is	
  a	
  
   standard	
  for	
  sharing	
  data	
  collected	
  by	
  sensors	
  
    –  SOS	
  will	
  refer	
  to	
  version	
  1.0.0	
  

                                                                             4	
  
Sensor	
  Observa.on	
  Service	
  
•  SOS	
  defines	
  three	
  core	
  opera.ons	
  
   –  GetCapabili.es	
  
   –  DescribeSensor	
  
   –  GetObserva.on	
  
•  A	
  GetObserva.on	
  request	
  must	
  define	
  an	
  
   observa.on	
  offering	
  and	
  at	
  least	
  one	
  observed	
  
   property	
  
•  Addi.onal	
  filters	
  include	
  spa.al-­‐temporal	
  
   bounding	
  box,	
  feature	
  of	
  interest	
  
                                                                   5	
  
Phenomenon	
  Layer	
  
•  A	
  SOS	
  service	
  may	
  provide	
  varied	
  data	
  
•  We	
  define	
  a	
  phenomenon	
  layer	
  to	
  refer	
  to	
  
   data	
  specific	
  to	
  a	
  single	
  observa.on	
  offering	
  
   and	
  observed	
  property	
  
•  A	
  phenomenon	
  layer	
  can	
  be	
  used	
  to	
  generate	
  
   more	
  specific	
  data	
  requests	
  to	
  a	
  SOS	
  service	
  
•  This	
  defini.on	
  is	
  specific	
  to	
  our	
  group	
  


                                                                          6	
  
Enhancing	
  SOS	
  
•  SOS	
  accomplishes	
  the	
  goal	
  of	
  transferring	
  
   sensor	
  data	
  
•  We	
  can	
  extend	
  the	
  u.lity	
  of	
  SOS	
  by	
  designing	
  
   a	
  system	
  to	
  collect	
  data	
  from	
  mul.ple	
  SOS	
  
   services	
  
•  Our	
  team’s	
  use	
  case:	
  
    –  To	
  display	
  all	
  the	
  sensors	
  measuring	
  the	
  same	
  
       phenomenon	
  from	
  all	
  SOS	
  services	
  

                                                                                7	
  
Problems	
  
•  A	
  number	
  of	
  problems	
  were	
  encountered	
  
    when	
  developing	
  a	
  system	
  for	
  our	
  use	
  case	
  
1.  Large	
  variety	
  of	
  the	
  observed	
  property	
  URI	
  
    represen.ng	
  the	
  same	
  phenomenon	
  
2.  Sensor-­‐centric	
  SOS	
  services	
  
3.  Non-­‐compliant	
  SOS	
  services	
  



                                                                         8	
  
Different	
  Observed	
  Property	
  URIs	
  For	
  
               Wind	
  Speed	
  
urn:x-­‐ogc:def:property:OGC::WindSpeed
urn:ogc:def:property:universityofsaskatchewan:ip3:windspeed
urn:ogc:def:phenomenon:OGC:1.0.30:windspeed
urn:ogc:def:phenomenon:OGC:1.0.30:WindSpeeds
urn:ogc:def:phenomenon:OGC:windspeed
urn:ogc:def:property:geocens:geocensv01:windspeed
urn:ogc:def:property:noaa:ndbc:Wind	
  Speed
urn:ogc:def:property:OGC::WindSpeed
urn:ogc:def:property:ucberkeley:odm:Wind	
  Speed	
  Avg	
  MS
urn:ogc:def:property:ucberkeley:odm:Wind	
  Speed	
  Max	
  MS
hdp://ws.sensordatabus.org/Ows/Swe.svc/?
service=SOS&request=GetResourceByID&ResourceID=urn:renci:sdb:property:1.0.0:Wind
%20Speed
hdp://marinemetadata.org/cf#wind_speed
                                                                                   9	
  
An	
  Example	
  of	
  a	
  Sensor-­‐Centric	
  SOS	
  
                           Service	
  
                                                   	
  
Observation Offering                                      Observed Property
                                                  http://ws.sensordatabus.org/Ows/Swe.svc/?
    offering_44040        service=SOS&request=GetResourceByID&ResourceID=urn:renci:sdb:property:1.0.0:Wind%20Speed

                                                  http://ws.sensordatabus.org/Ows/Swe.svc/?
offering_nos.8411250.WL   service=SOS&request=GetResourceByID&ResourceID=urn:renci:sdb:property:1.0.0:Wind%20Speed

                                                  http://ws.sensordatabus.org/Ows/Swe.svc/?
    offering_pwaw3        service=SOS&request=GetResourceByID&ResourceID=urn:renci:sdb:property:1.0.0:Wind%20Speed

                                                  http://ws.sensordatabus.org/Ows/Swe.svc/?
    offering_mqtt2        service=SOS&request=GetResourceByID&ResourceID=urn:renci:sdb:property:1.0.0:Wind%20Speed

                                                  http://ws.sensordatabus.org/Ows/Swe.svc/?
offering_nws.SBCP.metar   service=SOS&request=GetResourceByID&ResourceID=urn:renci:sdb:property:1.0.0:Wind%20Speed




                                                                                                                10	
  
Proposed	
  Solu.on	
  
•  We	
  propose	
  a	
  logical	
  layer	
  service	
  to	
  manage	
  
   heterogeneous	
  observed	
  property	
  URIs	
  
•  We	
  propose	
  a	
  transla8on	
  engine	
  to	
  provide	
  
   efficient	
  communica.on	
  with	
  SOS	
  services	
  




                                                                           11	
  
Contribu.on	
  
•  Our	
  system	
  has	
  three	
  contribu.ons	
  
   1.  Summarizes	
  and	
  inves.gates	
  the	
  current	
  status	
  
       of	
  SOS	
  services	
  online	
  today	
  
   2.  Highlights	
  issues	
  with	
  developing	
  a	
  system	
  to	
  
       address	
  a	
  common	
  use	
  case	
  
   3.  Presents	
  a	
  system	
  architecture	
  for	
  handling	
  
       these	
  solu.ons	
  



                                                                             12	
  
Related	
  Work	
  
•  The	
  OGC	
  Catalogue	
  Service	
  provides	
  
   informa.on	
  about	
  available	
  OGC	
  services	
  
•  Not	
  all	
  SOS	
  services	
  are	
  registered	
  in	
  a	
  
   Catalogue	
  Service	
  
•  Also,	
  the	
  system	
  must	
  also	
  account	
  for	
  
   communica.ng	
  with	
  mul.ple	
  SOS	
  services	
  



                                                                       13	
  
Related	
  Work	
  
•  A	
  search	
  engine	
  is	
  commonly	
  used	
  for	
  
   answering	
  advanced	
  queries	
  
•  This	
  is	
  a	
  difficult	
  process	
  due	
  to	
  varying	
  URNs	
  
   of	
  observed	
  proper.es	
  
•  This	
  also	
  doesn’t	
  account	
  for	
  managing	
  
   communica.on	
  with	
  mul.ple	
  SOS	
  services	
  



                                                                               14	
  
Methodology	
  




                  15	
  
Overview	
  




               16	
  
Logical	
  Layer	
  Service	
  
•  Manages	
  and	
  maps	
  the	
  logical	
  concept	
  of	
  an	
  
   observed	
  property	
  to	
  corresponding	
  
   phenomenon	
  layers	
  
•  We	
  use	
  a	
  dic.onary	
  to	
  manage	
  our	
  list	
  of	
  
   observed	
  proper.es	
  
•  The	
  dic.onary	
  is	
  manually	
  created	
  by	
  a	
  data	
  
   processing	
  team	
  that	
  interprets	
  sensor	
  data	
  
   by	
  phenomenon	
  types	
  

                                                                          17	
  
Logical	
  Layer	
  Service	
  
•  Dic.onary	
  terms	
  are	
  managed	
  by	
  hierarchies	
  
   to	
  facilitate	
  user	
  searches	
  
•  The	
  mapping	
  from	
  dic.onary	
  terms	
  to	
  
   phenomenon	
  layers	
  is	
  also	
  done	
  manually	
  
•  The	
  textual	
  informa.on	
  of	
  the	
  observa.on	
  
   offering	
  and	
  observed	
  property	
  URI	
  are	
  
   interpreted	
  manually	
  and	
  mapped	
  to	
  
   corresponding	
  dic.onary	
  terms	
  

                                                                   18	
  
Logical	
  Layer	
  Service	
  
            Hierarchy	
                        Dic.onary	
                       Phenomenon	
  Layer	
  


                Earth	
  
                                                  .de	
  level	
  
                                                 water	
  temp	
  
Atmosphere	
                Hydro	
             soil	
  moisture	
  
                                                  soil	
  temp	
  
                                                  air	
  temp	
  

  Air	
         Soil	
  
                                                                                        1




(1)	
  hdp://app.geocens.ca:8171/sos,	
  Temperature,	
  urn:ogc:def:property:noaa:ndbc:Air	
  Temperature	
  
                                                                                                           19	
  
Transla.on	
  Engine	
  
•  The	
  transla.on	
  engine	
  has	
  four	
  components	
  
   1.    Controller	
  
   2.    SOS	
  Connector	
  
   3.    Feeder	
  
   4.    Cache	
  Database	
  




                                                                  20	
  
Transla.on	
  Engine	
  




                           21	
  
Client	
  
•  Web	
  based	
  front	
  end	
  
•  Users	
  select	
  a	
  hierarchy,	
  and	
  use	
  the	
  hierarchy	
  
   to	
  select	
  an	
  observed	
  property	
  
•  The	
  client	
  automa.cally	
  loads	
  all	
  data	
  for	
  that	
  
   observed	
  property	
  from	
  all	
  available	
  SOS	
  
   services	
  



                                                                        22	
  
Client	
  




             23	
  
Experimental	
  Results	
  




                              24	
  
Data	
  Mapping	
  
•  Data	
  mapping	
  is	
  performed	
  manually,	
  we	
  will	
  
   not	
  focus	
  on	
  valida.ng	
  correctness	
  of	
  
   mapping	
  
•  Instead,	
  we	
  will	
  summarize	
  the	
  overall	
  
   mappings	
  




                                                                   25	
  
Data	
  Mapping	
  
•  The	
  dic.onary	
  defines	
  76	
  observed	
  proper.es	
  
•  The	
  mapping	
  defines	
  phenomenon	
  layers,	
  so	
  
   the	
  user	
  does	
  not	
  need	
  to	
  create	
  mul.ple	
  
   requests	
  to	
  mul.ple	
  SOS	
  services	
  
•  For	
  example,	
  there	
  are	
  369	
  wind	
  speed	
  
   phenomenon	
  layers,	
  across	
  7	
  different	
  SOS	
  
   services	
  


                                                                  26	
  
Top	
  Ten	
  Observed	
  Proper.es	
  with	
  Highest	
  Number	
  of	
  
                             Mappings	
  
                                	
  
 Observed Property URN      Number of Mappings        Number of Services

horizontal_wind_speed              402                        2
    wind_direction                 369                        7
     wind_speed                    55                        12
dew_point_temperature              31                         4
 min_air_temperature               25                        14
   air_temperature                 25                        14
 max_air_temperature               20                        10
dry_bulb_temperature               19                         9
mean_air_temperature               18                         9
 atmospheric_pressure              18                        14
                                                                           27	
  
Number	
  of	
  instances	
  in	
  the	
  real-­‐world	
  SOS	
  
  services	
  and	
  the	
  logical	
  layer	
  service	
  




                                                               28	
  
Data	
  Access	
  
•  To	
  evaluate	
  the	
  performance	
  of	
  data	
  access,	
  
   we	
  used	
  our	
  system	
  approach	
  versus	
  the	
  
   naïve	
  approach	
  
•  We	
  downloaded	
  recent	
  observa.on	
  data	
  from	
  
   sensors	
  
•  Tes.ng	
  was	
  performed	
  on	
  Acer	
  Aspire	
  M3910	
  
   with	
  Intel®	
  Core	
  ™	
  i7-­‐870	
  @	
  2.93GHz	
  and	
  4	
  
   gigabyte	
  DIMM	
  Synchronous	
  1333	
  MHz	
  

                                                                        29	
  
Data	
  loading	
  latencies	
  by	
  using	
  naïve	
  
  solu.on	
  and	
  proposed	
  scheme	
  




                                                       30	
  
Conclusions	
  




                  31	
  
Conclusions	
  
•  A	
  logical	
  layer	
  service	
  and	
  transla.on	
  engine	
  
   are	
  proposed	
  to	
  solve	
  data	
  access	
  to	
  mul.ple,	
  
   heterogeneous	
  SOS	
  services	
  
•  The	
  data	
  mapping	
  aggregates	
  similar	
  
   phenomenon	
  layers	
  
•  The	
  proposed	
  data	
  access	
  scheme	
  is	
  efficient	
  
   when	
  compared	
  to	
  a	
  naïve	
  approach	
  


                                                                       32	
  
Future	
  Work	
  
•  The	
  work	
  here	
  can	
  be	
  extended	
  by	
  
   inves.ga.ng	
  automa.c	
  and	
  semi-­‐automa.c	
  
   methods	
  of	
  mapping,	
  dic.onary	
  genera.on,	
  
   and	
  informa.on	
  retrieval	
  
•  We	
  can	
  leverage	
  recent	
  research	
  in	
  the	
  
   seman.c	
  web	
  and	
  data	
  mining	
  for	
  opening	
  the	
  
   sensor	
  web	
  to	
  end	
  users	
  for	
  data	
  browsing	
  


                                                                    33	
  
Acknowledgements	
  




                       34	
  

Design and implementation of a system for the improved searching and accessing of real-world SOS services

  • 1.
    Design  and  implementa.on  of  a  system   for  the  improved  searching  and   accessing  of  real-­‐world  SOS  services     Ben  Knoechel  (bcknoech@ucalgary.ca)   Chih-­‐Yuan  Huang  (huangcy@ucalgary.ca)   Steve  Liang  (steve.liang@ucalgary.ca)   University  of  Calgary    
  • 2.
    Outline   •  Introduc.on   •  Methodology   •  Experimental  Results   •  Conclusions   2  
  • 3.
  • 4.
    Introduc.on   •  The  Open  Geospa.al  Consor.um  (OGC)  has   developed  the  Sensor  Web  Enablement  (SWE)   •  SWE  is  a  set  of  standards  to  allow  people  to   share  and  interact  with  sensor  data  over  the   Internet   •  The  Sensor  Observa.on  Service  (SOS)  is  a   standard  for  sharing  data  collected  by  sensors   –  SOS  will  refer  to  version  1.0.0   4  
  • 5.
    Sensor  Observa.on  Service   •  SOS  defines  three  core  opera.ons   –  GetCapabili.es   –  DescribeSensor   –  GetObserva.on   •  A  GetObserva.on  request  must  define  an   observa.on  offering  and  at  least  one  observed   property   •  Addi.onal  filters  include  spa.al-­‐temporal   bounding  box,  feature  of  interest   5  
  • 6.
    Phenomenon  Layer   • A  SOS  service  may  provide  varied  data   •  We  define  a  phenomenon  layer  to  refer  to   data  specific  to  a  single  observa.on  offering   and  observed  property   •  A  phenomenon  layer  can  be  used  to  generate   more  specific  data  requests  to  a  SOS  service   •  This  defini.on  is  specific  to  our  group   6  
  • 7.
    Enhancing  SOS   • SOS  accomplishes  the  goal  of  transferring   sensor  data   •  We  can  extend  the  u.lity  of  SOS  by  designing   a  system  to  collect  data  from  mul.ple  SOS   services   •  Our  team’s  use  case:   –  To  display  all  the  sensors  measuring  the  same   phenomenon  from  all  SOS  services   7  
  • 8.
    Problems   •  A  number  of  problems  were  encountered   when  developing  a  system  for  our  use  case   1.  Large  variety  of  the  observed  property  URI   represen.ng  the  same  phenomenon   2.  Sensor-­‐centric  SOS  services   3.  Non-­‐compliant  SOS  services   8  
  • 9.
    Different  Observed  Property  URIs  For   Wind  Speed   urn:x-­‐ogc:def:property:OGC::WindSpeed urn:ogc:def:property:universityofsaskatchewan:ip3:windspeed urn:ogc:def:phenomenon:OGC:1.0.30:windspeed urn:ogc:def:phenomenon:OGC:1.0.30:WindSpeeds urn:ogc:def:phenomenon:OGC:windspeed urn:ogc:def:property:geocens:geocensv01:windspeed urn:ogc:def:property:noaa:ndbc:Wind  Speed urn:ogc:def:property:OGC::WindSpeed urn:ogc:def:property:ucberkeley:odm:Wind  Speed  Avg  MS urn:ogc:def:property:ucberkeley:odm:Wind  Speed  Max  MS hdp://ws.sensordatabus.org/Ows/Swe.svc/? service=SOS&request=GetResourceByID&ResourceID=urn:renci:sdb:property:1.0.0:Wind %20Speed hdp://marinemetadata.org/cf#wind_speed 9  
  • 10.
    An  Example  of  a  Sensor-­‐Centric  SOS   Service     Observation Offering Observed Property http://ws.sensordatabus.org/Ows/Swe.svc/? offering_44040 service=SOS&request=GetResourceByID&ResourceID=urn:renci:sdb:property:1.0.0:Wind%20Speed http://ws.sensordatabus.org/Ows/Swe.svc/? offering_nos.8411250.WL service=SOS&request=GetResourceByID&ResourceID=urn:renci:sdb:property:1.0.0:Wind%20Speed http://ws.sensordatabus.org/Ows/Swe.svc/? offering_pwaw3 service=SOS&request=GetResourceByID&ResourceID=urn:renci:sdb:property:1.0.0:Wind%20Speed http://ws.sensordatabus.org/Ows/Swe.svc/? offering_mqtt2 service=SOS&request=GetResourceByID&ResourceID=urn:renci:sdb:property:1.0.0:Wind%20Speed http://ws.sensordatabus.org/Ows/Swe.svc/? offering_nws.SBCP.metar service=SOS&request=GetResourceByID&ResourceID=urn:renci:sdb:property:1.0.0:Wind%20Speed 10  
  • 11.
    Proposed  Solu.on   • We  propose  a  logical  layer  service  to  manage   heterogeneous  observed  property  URIs   •  We  propose  a  transla8on  engine  to  provide   efficient  communica.on  with  SOS  services   11  
  • 12.
    Contribu.on   •  Our  system  has  three  contribu.ons   1.  Summarizes  and  inves.gates  the  current  status   of  SOS  services  online  today   2.  Highlights  issues  with  developing  a  system  to   address  a  common  use  case   3.  Presents  a  system  architecture  for  handling   these  solu.ons   12  
  • 13.
    Related  Work   • The  OGC  Catalogue  Service  provides   informa.on  about  available  OGC  services   •  Not  all  SOS  services  are  registered  in  a   Catalogue  Service   •  Also,  the  system  must  also  account  for   communica.ng  with  mul.ple  SOS  services   13  
  • 14.
    Related  Work   • A  search  engine  is  commonly  used  for   answering  advanced  queries   •  This  is  a  difficult  process  due  to  varying  URNs   of  observed  proper.es   •  This  also  doesn’t  account  for  managing   communica.on  with  mul.ple  SOS  services   14  
  • 15.
  • 16.
  • 17.
    Logical  Layer  Service   •  Manages  and  maps  the  logical  concept  of  an   observed  property  to  corresponding   phenomenon  layers   •  We  use  a  dic.onary  to  manage  our  list  of   observed  proper.es   •  The  dic.onary  is  manually  created  by  a  data   processing  team  that  interprets  sensor  data   by  phenomenon  types   17  
  • 18.
    Logical  Layer  Service   •  Dic.onary  terms  are  managed  by  hierarchies   to  facilitate  user  searches   •  The  mapping  from  dic.onary  terms  to   phenomenon  layers  is  also  done  manually   •  The  textual  informa.on  of  the  observa.on   offering  and  observed  property  URI  are   interpreted  manually  and  mapped  to   corresponding  dic.onary  terms   18  
  • 19.
    Logical  Layer  Service   Hierarchy   Dic.onary   Phenomenon  Layer   Earth   .de  level   water  temp   Atmosphere   Hydro   soil  moisture   soil  temp   air  temp   Air   Soil   1 (1)  hdp://app.geocens.ca:8171/sos,  Temperature,  urn:ogc:def:property:noaa:ndbc:Air  Temperature   19  
  • 20.
    Transla.on  Engine   • The  transla.on  engine  has  four  components   1.  Controller   2.  SOS  Connector   3.  Feeder   4.  Cache  Database   20  
  • 21.
  • 22.
    Client   •  Web  based  front  end   •  Users  select  a  hierarchy,  and  use  the  hierarchy   to  select  an  observed  property   •  The  client  automa.cally  loads  all  data  for  that   observed  property  from  all  available  SOS   services   22  
  • 23.
  • 24.
  • 25.
    Data  Mapping   • Data  mapping  is  performed  manually,  we  will   not  focus  on  valida.ng  correctness  of   mapping   •  Instead,  we  will  summarize  the  overall   mappings   25  
  • 26.
    Data  Mapping   • The  dic.onary  defines  76  observed  proper.es   •  The  mapping  defines  phenomenon  layers,  so   the  user  does  not  need  to  create  mul.ple   requests  to  mul.ple  SOS  services   •  For  example,  there  are  369  wind  speed   phenomenon  layers,  across  7  different  SOS   services   26  
  • 27.
    Top  Ten  Observed  Proper.es  with  Highest  Number  of   Mappings     Observed Property URN Number of Mappings Number of Services horizontal_wind_speed 402 2 wind_direction 369 7 wind_speed 55 12 dew_point_temperature 31 4 min_air_temperature 25 14 air_temperature 25 14 max_air_temperature 20 10 dry_bulb_temperature 19 9 mean_air_temperature 18 9 atmospheric_pressure 18 14 27  
  • 28.
    Number  of  instances  in  the  real-­‐world  SOS   services  and  the  logical  layer  service   28  
  • 29.
    Data  Access   • To  evaluate  the  performance  of  data  access,   we  used  our  system  approach  versus  the   naïve  approach   •  We  downloaded  recent  observa.on  data  from   sensors   •  Tes.ng  was  performed  on  Acer  Aspire  M3910   with  Intel®  Core  ™  i7-­‐870  @  2.93GHz  and  4   gigabyte  DIMM  Synchronous  1333  MHz   29  
  • 30.
    Data  loading  latencies  by  using  naïve   solu.on  and  proposed  scheme   30  
  • 31.
  • 32.
    Conclusions   •  A  logical  layer  service  and  transla.on  engine   are  proposed  to  solve  data  access  to  mul.ple,   heterogeneous  SOS  services   •  The  data  mapping  aggregates  similar   phenomenon  layers   •  The  proposed  data  access  scheme  is  efficient   when  compared  to  a  naïve  approach   32  
  • 33.
    Future  Work   • The  work  here  can  be  extended  by   inves.ga.ng  automa.c  and  semi-­‐automa.c   methods  of  mapping,  dic.onary  genera.on,   and  informa.on  retrieval   •  We  can  leverage  recent  research  in  the   seman.c  web  and  data  mining  for  opening  the   sensor  web  to  end  users  for  data  browsing   33  
  • 34.