Introduction to OMG DDS (1 hour, 45 slides)

4,190 views

Published on

0 Comments
4 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
4,190
On SlideShare
0
From Embeds
0
Number of Embeds
6
Actions
Shares
0
Downloads
245
Comments
0
Likes
4
Embeds 0
No embeds

No notes for slide

Introduction to OMG DDS (1 hour, 45 slides)

  1. 1. Your  systems.  Working  as  one.   Introduc)on  to  OMG  DDS  OMG  Technical  Mee9ng,  Washington  D.C.,  March  2013  Gerardo  Pardo-­‐Castellote,  Ph.D.      [gerardo@r9.com]    CTO,  Real-­‐Time  Innova9ons,  Inc.    [www.r9.com]  Co-­‐chair  of  the  OMG  Data-­‐Distribu9on  SIG  
  2. 2. Systems  that  interact  with  the  Real  World  •  Must  adapt  to  changing  environment  •  Cannot  stop  processing  the  informa9on  •  Live  within  world-­‐imposed  9ming  Beyond  tradi9onal  interpreta9on  of  real-­‐9me   ©  2012  RTI  •  ALL  RIGHTS  RESERVED   2  
  3. 3. Challenge:      More  Data,  More  Speed,  More  Sources  TRENDS:  •  Growing  Informa9on  Volume  •  Lowering  Decision  Latency  •  Increasing  System  Availability  •  Accelera9ng  technology  inser9on   and  deployment  Next-­‐genera)on  systems  needs:  •  Scalability  •  Integra9on  &  Evolu9on  •  Robustness  &  Availability  •  Performance  •  Security   ©  2012  RTI  •  ALL  RIGHTS  RESERVED   3   3  
  4. 4. “Real  World”  Systems  are  integrated   using  a  Data  Model  •  Grounded  on  the  “physics”  of  the  problem  domain   –  Tied  to  the  nature  of  the  sensors  and  real  objects  in  the  system  (vehicles,   device  types,  …)  •  Provides  governance  across  disparate  teams  &  organiza9ons   –  The  “N^2”  integra9on  problem  is  reduced  to  a  “N”  problem  •  Increased  decoupling  from  use-­‐cases  and  components   –  Avoids  over  constraining  applica9ons  •  Open,  Evolvable,  Plaborm-­‐Independent   –  The  use-­‐cases,  algorithms  might  change  between  missions  or  versions  of   the  system  App   App   App   Realizing  this  data-­‐model  requires  a  middleware  infrastructure   ©  2012  RTI  •  ALL  RIGHTS  RESERVED   4  
  5. 5. DDS:    Standards-­‐based  Integra9on  Infrastructure  for  Cri9cal  Applica9ons   Streaming   Sensors   Events   Data   Real-­‐Time   Enterprise   Actuators   Applica)ons   Applica)ons   ©  2012  RTI  •  ALL  RIGHTS  RESERVED   5  
  6. 6. Family  of  Specifica9ons  2008   2009   2010   2010   2013   2013  UML  Profile   DDS  for   DDS     DDS-­‐STD-­‐C++   DDS   RPC  over  for  DDS   Lw  CCM   X-­‐Types   DDS-­‐JAVA5   Security   DDS   App   2004   App   App   DDS  Spec   DDS   2006   DDS   DDS   Implementa9on   DDS   Implementa9on   Implementa9on   Interoperablity   Network  /  TCP  /  UDP  /  IP   ©  2012  RTI  •  ALL  RIGHTS  RESERVED   6   6  
  7. 7. Broad  Adop9on  •  Vendor  independent   Cross-­‐vendor  portability! –  API  for  portability   –  Wire  protocol  for  interoperability  •  Mul9ple  implementa9ons   –  10  of  API   DDS  API   –  8  support  RTPS  •  Heterogeneous   Middleware   –  C,  C++,  Java,  .NET  (C#,  C++/CLI),  ADA,   Python,  Scala,  …   DDS  Real-­‐Time     Publish-­‐Subscribe   –  Linux,  Windows,  VxWorks,  Integrity,   Wire  Protocol  (RTPS)   other  embedded  &  real-­‐9me  •  Loosely  coupled   Cross-­‐vendor  interoperability! ©  2012  RTI  •  ALL  RIGHTS  RESERVED   7  
  8. 8. Interoperability  between  the  applica)ons  implemented  by  six  different  vendors  (March  2012)   OCI   ETRI   PrismTech   IBM   RTI   TwinOaks   8  
  9. 9. DDS  mandated  by  key  DoD  Programs  •  UK  Generic  Vehicle  Architecture   –  Mandates  DDS  for  vehicle  comm.   –  Mandates  DDS-­‐RTPS  for  interop.  •  DISR   –  Mandates  DDS  for  Pub-­‐Sub  API   –  Mandates  DDS-­‐RTPS  for  Interop  •  Army,  OSD   –  UCS,  Unmanned  Vehicle  Control   –  FACE  (Future  Airborne  Capability   Environment)  •  US  Navy  Open  Architecture   –  Mandates  DDS  for  Pub-­‐Sub  •  SPAWAR  NESI   –  Mandates  DDS  for  Pub-­‐Sub  SOA   ©  2012  RTI  •  ALL  RIGHTS  RESERVED   9  
  10. 10. DDS  Deployed  Applica9on  Examples   ©  2012  RTI  •  ALL  RIGHTS  RESERVED   10  
  11. 11. Everyday  Example:  Calendaring   Alterna)ve  Process  #1    (message-­‐centric):   1.  Email:  “Mee9ng  Monday  at  10:00.”   2.  Email:  “Here’s  dial-­‐in  info  for  mee9ng…”   3.  Email:  “Mee9ng  moved  to  Tuesday”   4.  You:  “Where  do  I  have  to  be?  When?”   5.  You:  (siFing  through  email  messages…)   ©  2012  RTI  •  ALL  RIGHTS  RESERVED   11  
  12. 12. Example:  Calendaring   Alterna)ve  Process  #2:   1.  Calendar:  (add  meeIng  Monday  at  10:00)   2.  Calendar:  (add  dial-­‐in  info)   3.  Calendar:  (move  meeIng  to  Tuesday)   4.  You:  “Where  do  I  have  to  be?  When?”   5.  You:  (check  calendar.  Contains   consolidated-­‐state)  The  difference  is  state!    The  infrastructure  consolidates  changes  and    maintains  it   ©  2012  RTI  •  ALL  RIGHTS  RESERVED   12  
  13. 13. Data-­‐Centric  Middleware  allows  applica9ons  to  be  integrated  to  the  Informa9on  Model   APP   APP   APP   APP   Standard  API   Data   Model   Standard  Mapping(*)   DDS  Global  Data  Space  No  custom  mappings  /  code  necessary  Direct  support  for  data-­‐centric  ac9ons:  create,  dispose,  read/take   Copyright  ©  2010  RTI  -­‐  All  rights  Reserved.  .   13  
  14. 14. Integra9ng  components  to  generic    middleware  technology   Comp   Comp   Comp   Comp   Custom     Integra9on   Data   Model   Custom  Mapping   Middleware  Ar)facts  Akin  to  implemen9ng  an  OO  design  on  a  Procedural  Language:  Requires  mapping  inheritance,  encapsula9on,  excep9ons,  …   Copyright  ©  2010  RTI  -­‐  All  rights  Reserved.  .   14  
  15. 15. Tradi9onal  data-­‐centric  technologies  not  suited  to  scalable  near  real-­‐9me  systems  Other  data-­‐centric  technologies:   –  Databases:  SQL   –  Web:  HTTP  (mostly)  •  …assume  the  world  changes  slowly  •  …use  network  resources  inefficiently   Not  scalable  •  …are  highly  centralized   100  apps  =>  100x  load   Slow   A  few  updates/sec   App   App   Server   App   App   State   App   App   Unreliable   Failure  here  kills  many  apps   ©  2012  RTI  •  ALL  RIGHTS  RESERVED   15  
  16. 16. DDS  is  decentralized.  Can  be  deployed  without  servers/brokers  DDS:  •  …allows  you  to  observe  frequent  changes  •  …uses  network  resources  efficiently  •  …is  peer-­‐to-­‐peer  and  decentralized   Fast   Scalable   Managed   Reliable   100,000’s  updates/sec   Load  indep.  #  apps   with  QoS   No  single  pt.  failure   App   App   App   App   App   App                                                    DDS    Data-­‐Centric  Bus   ©  2012  RTI  •  ALL  RIGHTS  RESERVED   16  
  17. 17. The  data-­‐centric   communicaIons  model   ©  2012  RTI  •  ALL  RIGHTS  RESERVED   17  
  18. 18. DDS  Adressing:  Data-­‐Objects  in  the  Global  Data  Space  •  Domain:  world  you’re  talking  about   Domain  •  Topic:  group  of  similar  objects   (e.g.  Yellowstone  Park)   –  Similar  structure  (“type”)            what   –  Similar  way  they  change            when   Topic   over  9me  (“QoS”)                              how   (e.g.  bears  in  the  park)  •  Instance:  individual  object   Instance   Instance   Booboo   (e.g.  Yogi     the  bear)  •  DataWriter:  source  of  observa9ons  about  a  set   of  data-­‐objects  (Topic)  •  DataReader:  observer  of  a  set  of  data-­‐objects   Topic   (Topic)   Snow  Depth  Sensors   ID  ID  ID  ID   ID  ID  ID   ID  ID  ID  ID  ID  ID  ID  ID   ID  ID  ID  ID  ID  ID  ID  ID   GPS   GPS   GPS   GPS   GPS   GPS   GPS   GPS   GPS   GPS   GPS   GPS   GPS   GPS   GPS   GPS   GPS   GPS   GPS   GPS   GPS   GPS   GPS   value   value   value   value   value   value   value   value   value   value   value   value   value   value   value   value   value   value   value   value   value   value   value  
  19. 19. Data-­‐Centric  Qos-­‐Aware  Pub-­‐Sub  Model   Virtual,  decentralized  global  data  space   Source Speed Power Phase (Key) WPT1 37.4 122.0 -12.20 WPT2 10.7 74.0 -12.23 WPTN 50.2 150.07 -11.98 Persistence   Recording  CRUD  opera9ons   Service   Service   ©  2012  RTI  •  ALL  RIGHTS  RESERVED   19  
  20. 20. ShapesDemo  Demo:  Publish-­‐Subscribe   ©  2012  RTI  •  ALL  RIGHTS  RESERVED   20  
  21. 21. Data-­‐Centric   example  Communica9ons  Model   Data   Domain   Data   Domain  New Writer   Par9cipant   Reader   Par9cipant   “Alarm”   Got new “Alarm”  subscriber data! Offered Requested Listener   QoS Listener   QoS •  Par)cipants  scope  the  global  data  space  (domain)   •  Topics  define  the  data-­‐objects  (collec9ons  of  subjects)   •  DataWriters  publish  data  on  Topics   •  DataReaders  subscribe  to  data  on  Topics   •  QoS  Policies  are  used  configure  the  system   •  Listeners  are  used  to  no9fy  the  applica9on  of  events   ©  2012  RTI  •  ALL  RIGHTS  RESERVED   21  
  22. 22. Quality  of  Service  (QoS)  •  Aside  from  the  actual  data  (WHAT)  to  be  delivered,   users  ozen  need  to  specify  HOW  to  send  it  …   …  reliably  (or  “send  and  forget”)   …  how  much  data  (all  data  ,  last  5  samples,  every  2  secs)   …  how  long  before  data  is  regarded  as  ‘stale’  and  is  discarded   …  how  many  publishers  of  the  same  data  is  allowed   …  how  to  ‘failover’  if  an  exis9ng  publisher  stops  sending  data   …  how  to  detect  “dead”  applica9ons   …  …  •  These  op9ons  are  controlled  by  formally-­‐defined       Quality  of  Service  (QoS)  policies  
  23. 23. Real-­‐Time  Quality  of  Service  Control   Collision ; 1 Hz; Subscriber   avoidance Reliable application ry get histo Best effort; 0.2 HZ; Publisher   Radar  Track   get history Subscriber   Best e ffo •  Publish reliably no hist rt; 0.01 Hz; •  10 Hz ory Airline •  Keep 10 minute history Subscriber   operations (6000 samples) center Backup   Publisher   Database, real-time log•  Quality  of  Service  (QoS)  determined  per-­‐en6ty  •  QoS  Contract:  Request  -­‐  Offered  •  Publishing  and  subscribing  applica9ons  can  be  no9fied  when  QoS  contract   is  violated     –  e.g.  Messages  lost  or  deadlines  missed  •  High  availability  via  automa9c  failover  
  24. 24. Real-­‐Time  Quality  of  Service  (QoS)   QoS Policy QoS Policy DURABILITY USER DATA User  QoS  Cache   HISTORY TOPIC DATA LIFESPAN GROUP DATA WRITER DATA LIFECYCLE PARTITION Presenta)on  Resources   READER DATA LIFECYCLE PRESENTATION ENTITY FACTORY DESTINATION ORDER RESOURCE LIMITS OWNERSHIP Availability   RELIABILITY OWNERSHIP STRENGTH Delivery   TIME BASED FILTER LIVELINESS DEADLINE LATENCY BUDGET Transport   CONTENT FILTERS TRANSPORT PRIORITY
  25. 25. ShapesDemo   Qos  in  Ac9on   •  Detec9ng   presence   •  History  cache   •  Dele9ng   objects   •  Ownership   •  Liveliness   •  Filtering   •  Durability  ©  2009  Real-­‐Time  Innova9ons,  Inc.     25  
  26. 26. OperaIonal  robustness  and   performance   ©  2012  RTI  •  ALL  RIGHTS  RESERVED   26  
  27. 27. Architecture  for  the  next-­‐genera9on  systems  •  Exis9ng  technologies  are  reaching  robustness/performance/scalability  limits  •  DDS  provides  a  fundamentally  new  DataBus  architecture  and  approach   –  Powerful  data-­‐centric  model   –  Ultra-­‐scalable  and  robust   –  Fully  decentralized,  peer-­‐to-­‐peer,    “no  bo}lenecks”  architecture   –  Superior  Wire  Protocol   –  Standards-­‐based,  mul9-­‐plaborm   Brokers  as   choke-­‐points   Connext  DDS  Approach   Single-­‐lane  traffic   No  priori9za9on   ©  2012  RTI  •  ALL  RIGHTS  RESERVED   27  
  28. 28. Performance   Performance  results  are  vendor-­‐specific   These  results  are  for  RTI  Connext  DDS.  Average  Latency  (Microseconds)   400   Number  of  Subscribers   350   1  (1  per  CPU  and  NIC)   300   20  (1  per  CPU  and  NIC)   250   40  (1  per  CPU,  2  per  NIC)   200   150   •  Reliable  mul9cast   100   •  Fully  meshed,  reliable   50   0   Orders  of   magnitude  faster   than  IT  solu9ons   Throughput  (Messages  per  Seconds)   Fastest  DDS  solu)on   ©  2012  RTI  •  ALL  RIGHTS  RESERVED   28  
  29. 29. Scalability   Scalability  results  are  vendor-­‐specific   These  results  are  for  RTI  Connext  DDS.   •  Scalable 600,000   Performance!! •  Millions of dataPer  Subscriber  (200  Bytes)   500,000   elements! Messages  per  Second   •  .5m updates/sec 400,000   (batched)! •  10s µs latency! 300,000   •  1000s consumers per update! 200,000   •  Orders  of  magnitude   more  scalable   100,000   than  IT  solu9ons   0   1    ~1000  subscribers,  <  15%  throughput  decrease   0                                      200                                      400                                      600                                      800                                       1,000   Number  of  Subscribers   ©  2012  RTI  •  ALL  RIGHTS  RESERVED   29  
  30. 30. Comparison  with  other  technologies   Comparison  results  are  vendor-­‐specific   These  results  compare  with  RTI  Connext  DDS.   Message  Length  (samples)   Adapted  from  Vanderbilt  presenta9on  at  July  2006  OMG  Workshop  on  RT  Systems   ©  2012  RTI  •  ALL  RIGHTS  RESERVED   30  
  31. 31. Common  use-­‐cases  /  paRerns   ©  2012  RTI  •  ALL  RIGHTS  RESERVED   31  
  32. 32. Common  Use  Cases   •  Isola9ng  subsystems   •  Detec9ng  presence  of  applica9ons   •  Discovering  publishers  and  subscribers   •  Keeping  a  “last-­‐value”  cache   •  Monitoring  the  health  of  applica9ons   •  Monitoring  the  health  of  data   •  Reliable  data  delivery   •  Building  a  highly-­‐available  system   •  Managing  network  load  and  behavior  3/24/13   ©  2012  RTI  •  COMPANY  CONFIDENTIAL   32  
  33. 33. Isola9ng  Subsystems  •  Concept  of  Domains  and  Domain  Par9cipants   N4  App  4   •   Container  for   N1  App  1   Pub/Sub   Pub/Sub   applica9ons  that      want   (A,B/C,D)   (D/C,E,F)   to  communicate   •   Applica9ons  can  join   N2  App  2   N4  App  5   or  leave  a  domain  in   any  order   Subscribe   Domain Publish   (C)   (C)   •   New  Applica9ons  are   “Auto-­‐Discovered”   N3  App  3   N5  App  6   •   An  applica9on  that   Pub/Sub   Subscribe   (E,F/A,C)   (B,C)   has  joined  a  domain  is   also  called  a  “Domain   Par9cipant”   Single  ‘Domain’  System  
  34. 34. Isola9ng  Subsystems  •  Use  mul9ple  domains  tor  scalability,   modularity  and  isola9on   Node  1  -­‐  App  1   Node  4  -­‐  App  1   Pub/Sub   Domain  A Pub/Sub   Node  2  -­‐  App  1   Node  4  -­‐  App  2   Subscribe   Publish   Domain  B   Node  3  -­‐  App  1   Node  5  -­‐  App  1   Pub/Sub   Subscribe   Domain  C   Node  5  -­‐  App  2   Node  6  -­‐  App  1   Added  Func.   Pub/Sub   Pub/Sub   Mul)ple  Domain  System  
  35. 35. Detec9ng  presence  of  applica9on  •  DDS  has  a  buil9n  discovery  service  •  It  provides  the  means  for  an  applica9on  to   discover  the  presence  of  other  par9cipants  on   the  Domain   –  The  Topic  “DCPSPar9cipants”  can  be  read  as  a   regular  Topic  to  see  when  DomainPar9cipants  join   and  leave  the  network  •  Applica9ons  can  also  include  meta-­‐data  that  is   sent  along  by  DDS  discovery  
  36. 36. Discover  Publishers  and  Subscribers   •  DDS  provides  a  mechanism  to  detect  en99es,   their  capabili9es  and  requirements   •  An  applica9on  can  discover  all  the  other   en99es  in  the  system  that  match  the   requirements   –  The  Topics  “DCPSPublica9ons”,   “DCPSSubscrip9ons”,  “DCPSTopics”,  and   “DCPSPar9cipants”    can  be  read  to  observe  the   other  en99es  in  the  domain  3/24/13   ©  2012  RTI  •  COMPANY  CONFIDENTIAL   36  
  37. 37. Publishing  data  that  outlives  it  source   •  The  concept  of  durability  in  the  global  dataspace   –  Vola9le   •  No  durability   –  Transient  local   •  Durability  maintained  by  the  publishing  applica9on   –  Transient   •  Durability  maintained  by  a  3rd  party  applica9on  (persistence   service)   –  Persistent   •  Durability  of  data  is  maintained  by  a  3rd  party  and  saved  to   disk   •  Durability  maintained  azer  restart  3/24/13   ©  2012  RTI  •  COMPANY  CONFIDENTIAL   37  
  38. 38. Keeping  a  “last  value”  cache   •  A  last-­‐value  cache  is  already  built-­‐in  into  every   Writer  in  the  system   •  A  late  joiner  will  automa9cally  ini9alize  to  the   last  value   •  Last  value  cache  can  be  configure  with  history   depth  greater  than  1   •  The  Persistence  Service  can  be  used  to  provide   a  last  value  cache  for  durable  data  3/24/13   ©  2012  RTI  •  COMPANY  CONFIDENTIAL   38  
  39. 39. Monitoring  the  health  of  applica9ons   •  DDS  has  mechanisms  to  monitor  presence,  health  and   ac9vity  of  all  en99es   •  Supports  a  concept  of  liveliness   –  Automa9c   •  Managed  by  the  infrastructure   –  Manually  asserted   •  Managed  by  the  applica9on   •  What  does  it  mean  if  the  applica9on  no  longer   publishes  data?   –  No  news,  good  news?   –  Applica9on  has  crashed?   –  Applica9on  is  disconnected?  3/24/13   ©  2012  RTI  •  COMPANY  CONFIDENTIAL   39  
  40. 40. Monitor  the  health  of  data-­‐objects   •  Concept  of  deadline   –  DDS  can  monitor  the  ac9vity  of  each  individual   data-­‐instance  in  the  system   –  If  an  instance  is  not  updated  according  to  the   requirements  of  the  receiving  applica9on,  the   applica9on  is  no9fied   –  Trigger  for  built-­‐in  failover  mechanism   •  Concept  of  lifespan   –  DDS  understands  if  a  data-­‐object  has  outlived  its   purpose  and  is  considered  ‘stale’  data  3/24/13   ©  2012  RTI  •  COMPANY  CONFIDENTIAL   40  
  41. 41. Ensuring  a  reliable  data  delivery   •  DDS  supports  a  transport  independent   efficient  reliability  protocol   –  In-­‐order  delivery   –  Reliable  delivery   –  Detec9on  and  no9fica9on  of  data  loss   –  Very  configurable   •  Warning  thresholds   •  Heartbeats,  gaps,  acks,  nacks,  blocking  or  non-­‐blocking   behaviour  3/24/13   ©  2012  RTI  •  COMPANY  CONFIDENTIAL   41  
  42. 42. Building  a  high-­‐available  system   •  High  Availability  has  mul9ple  facets,  many  of   which  can  be  handled  by  DDS   –  Detec9on  of  presence   •  DISCOVERY   –  Detec9on  of  health  and  ac9vity   •  LIVELINESS,  DEADLINE,  LIFESPAN   –  Survive  applica9on  and  system  failures   •  DURABILITY   –  Ability  to  handle  redundant  data  source  and  failover   •  OWNERSHIP   –  Reliable  data  transfer   •  RELIABILITY  3/24/13   ©  2012  RTI  •  COMPANY  CONFIDENTIAL   42  
  43. 43. Manage  network  load  &  behaviour   •  DDS  provides  mechanism  to  limit  the  data-­‐ rates   –  Filter  by  9me   –  Filter  by  content   –  Par99ons   –  Use  of  mul9cast   –  Configured  reliability  level   –  TransportPriority  QoS  policy   –  LatencyBudget  QoS  policy  3/24/13   ©  2012  RTI  •  COMPANY  CONFIDENTIAL   43  
  44. 44. Conclusions  •  DDS  is  a  mature  interna9onal  Standard  from  OMG   –  Plaborm  Neutral:  Opera9ng  systems  and  Programming   Languages   –  Deployed  worldwide  in  Military  systems  and  other   Demanding  real-­‐9me  applica9ons  •  DDS  Is  mandated  by  DoD  for  Publish-­‐Subscribe  and   data-­‐distribu9on  applica9ons  •  DDS  is  an  ideal  integra9on  plaborm  for  Intelligent   Systems   –  Highly  Tunable  via  Quality  of  Service  (QoS)   –  Rich  services  (persistence,  filtering,  high-­‐availability)   ©  2012  RTI  •  ALL  RIGHTS  RESERVED   44  
  45. 45. Find  out  more…   www.r9.com   dds.omg.org   community.r9.com   www.omg.org   demo.r9.com   www.youtube.com/real9meinnova9ons   blogs.r9.com   www.twi}er.com/RealTimeInnov   www.facebook.com/RTIsozware   www.slideshare.net/GerardoPardo   ©  2012  RTI  •  ALL  RIGHTS  RESERVED   45  
  46. 46. Thank You! dds.omg.org                                                                         www.omg.org                 ©  2012  RTI  •  ALL  RIGHTS  RESERVED   46  

×