High Performance Distributed Computing with DDS and Scala

1,569 views
1,457 views

Published on

This presentation provides a DDS tutorial and an introduction to programming DDS with the Scala Programming Language.

Published in: Technology
0 Comments
3 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
1,569
On SlideShare
0
From Embeds
0
Number of Embeds
36
Actions
Shares
0
Downloads
81
Comments
0
Likes
3
Embeds 0
No embeds

No notes for slide

High Performance Distributed Computing with DDS and Scala

  1. 1. High Performance Distributed Computing with DDS and ScalaOpenSplice DDS Angelo CORSARO, Ph.D. Chief Technology Officer OMG DDS Sig Co-Chair PrismTech angelo.corsaro@prismtech.com
  2. 2. OpenSplice DDS Background
  3. 3. Data Distribution Service For Real-Time Systems Copyright  2011,  PrismTech  –    All  Rights  Reserved. ☐ Introduced in 2004 to address the Data Distribution Real-Timeliness Scale challenges faced by a wide class of Defense and Aerospace ApplicationsOpenSplice DDS Key requirement for the Parallelism Determinism Throughput, Availability Scalability, Persistence, Security ☐ standard were to deliver very Systemic Signal Data Real-Time Information Near Real-Time Fault- Tolerant Information Complex Information high and predictable Processing Processing Management Processing Processing performance while scaling from embedded to ultra- Parallel Systems Distributed Systems large-scale deployments
  4. 4. Data Distribution Service For Real-Time Systems Copyright  2011,  PrismTech  –    All  Rights  Reserved. ☐ Recommended by key Real-Timeliness Scale administration worldwide, e.g. DoD, MoD, EUROCAE, etc.OpenSplice DDS Widely adopted across Parallelism Determinism Throughput, Availability Scalability, Persistence, Security ☐ several different domains, Systemic Signal Data Real-Time Information Near Real-Time Fault- Tolerant Information Complex Information e.g., Automated Trading, Processing Processing Management Processing Processing Simulations, SCADA, Telemetry, etc. Parallel Systems Distributed Systems
  5. 5. Core Standard Copyright  2011,  PrismTech  –    All  Rights  Reserved. Application ☐ DDS is nowadays a family Data Centric Publish Subscriber (DCPS) of standards Content Ownership Durability SubscriptionOpenSplice DDS ☐ The two standards at the Minimum Profile core are: ☐ DDS v1.2 -- API DDS Interoperability Wire Protocol - DDSI-RTPS ☐ DDSI v2.1 -- Wire Protocol UDP/IP Application
  6. 6. Standards: What For? Copyright  2011,  PrismTech  –    All  Rights  Reserved. Common “Language” Data Model +OpenSplice DDS QoS Requirements Interoperability DDSI Wire Interoperability
  7. 7. OpenSplice DDS DDS Application API Standard Standards: What For? Portability Copyright  2011,  PrismTech  –    All  Rights  Reserved.
  8. 8. Defense and Aerospace Copyright  2011,  PrismTech  –    All  Rights  Reserved.OpenSplice DDS Integrated Modular Vetronics Training & Simulation Systems Naval Combat Systems Air Traffic Control & Management Unmanned Air Vehicles Aerospace Applications
  9. 9. Commercial Applications Copyright  2011,  PrismTech  –    All  Rights  Reserved.OpenSplice DDS Agricultural Vehicle Systems Large Scale SCADA Systems Smart Cities Train Control Systems Complex Medical Devices High Frequency Auto-Trading
  10. 10. OpenSplice DDS D D BaS ics
  11. 11. Data Distribution Service For Real-Time Systems DDS provides a Topic-Based Publish/ Copyright  2011,  PrismTech  –    All  Rights  Reserved. Subscribe abstraction based on: Data Reader Data Writer ☐ Topics: data distribution subject’sOpenSplice DDS Data Reader Data TopicD Writer DataWriters: data producers TopicA ☐ Data TopicB Reader Data Writer ☐ DataReaders: data consumers TopicC ... Data Data Writer Reader DDS Global Data Space
  12. 12. Data Distribution Service For Real-Time Systems Copyright  2011,  PrismTech  –    All  Rights  Reserved. ☐ DataWriters and DataReaders are automatically and Data Reader dynamically matched by the Data Writer DDS Dynamic DiscoveryOpenSplice DDS Data Reader Data TopicD Writer TopicA ☐ A rich set of QoS allows to Data Reader TopicB control existential, temporal, Data Writer TopicC ... and spatial properties of data Data Data Writer Reader DDS Global Data Space
  13. 13. OpenSplice DDS DDS Entities
  14. 14. Domain Participant Domain (e.g. Domain 123) Domain Participant Copyright  2011,  PrismTech  –    All  Rights  Reserved. Topic Partition (e.g. “Telemetry”, “Shapes”, ) T1 Publisher T1 T3 Subscriber TxOpenSplice DDS Topic Instances/Samples Ty Tb Ta DataWrter DataReader Tc ☐ A DomainParticipant provides access to a DDS Domain ☐ DDS Domains are identified by natural numbers, such as 0, 1, 2, ... ☐ Each DDS Domain has associated some unique communication and discovery endpoints, automatically deduced by the Domain ID
  15. 15. OpenSplice DDS Topics
  16. 16. DDS Topics “Circle”, “Square”, “Triangle”, ... ☐ A Topic defines a class of streams, Copyright  2011,  PrismTech  –    All  Rights  Reserved. also called topic instances Name ☐ A Topic has associated a unique Topic name, a user defined extensible Typ SOpenSplice DDS type and a set of QoS policies DURABILITY, Qo e DEADLINE, ShapeType ☐ QoS Policies capture the Topic PRIORITY, … non-functional invariants struct ShapeType { ☐ Topics can be discovered or @Key string color; locally defined long x; long y; long shapesize; };
  17. 17. Topic Instances ☐ Each unique key value Copyright  2011,  PrismTech  –    All  Rights  Reserved. Instances Instances identifies a unique stream of data color =”Green” DDS not only struct ShapeType { ☐ color =”red” @Key string color; TopicOpenSplice DDS long x; long y; demultiplexes “streams” color = “Blue” long shapesize;}; but provides also lifecycle information ☐ A DDS DataWriter can write multiple instances
  18. 18. OpenSplice DDS Topic Types
  19. 19. Topic Types Type System ☐ DDS features a structural type system and the sub-type Copyright  2011,  PrismTech  –    All  Rights  Reserved. relationship ( <: ) defined by this type system is used to guide subscription matching ☐ In general, a DataReader[X] will match a DataWriter[Y], whereOpenSplice DDS X and Y are DDS Topic types, iff Y <: X ☐ Annotations available to: ☐ Constraint the subtype relationship ☐ Control the set of attributes considered when performing the subtype test
  20. 20. Type-Level Annotations ☐ The nature of a DDS Topic Type can be controlled through Copyright  2011,  PrismTech  –    All  Rights  Reserved. annotations applicable at a type or attribute level ☐ Specifically the @Extensibility annotation controls the allowed “extensibility”, in the sense of sub-typing for the given typeOpenSplice DDS ☐ Possible values are: ☐ Final: the type is sealed and can’t be changed ☐ Extensible: Only monotonic extensions are allowed, e.g. “adding attributes to the end” ☐ Mutable: Arbitrary extensions are allowed such as attribute re-ordering, removal, etc.
  21. 21. Attribute-Level Annotations ☐ Attribute semantics can also be controlled via an extensible Copyright  2011,  PrismTech  –    All  Rights  Reserved. annotation system ☐ Some of the most commonly used annotations are: @Optional: This attribute is allowed to be “nil”OpenSplice DDS ☐ ☐ @Shared: This attribute behaves like a pointer ☐ @MustUnderstand: This attribute must be present in order to match subscriptions ☐ @ID: Ordinal value associated to the type ☐ @Key: This attribute is part of the topic key
  22. 22. Publishers and SubscribersOpenSplice DDS
  23. 23. Publishers / Subscribers Domain (e.g. Domain 123) Domain Participant Copyright  2011,  PrismTech  –    All  Rights  Reserved. Topic Partition (e.g. “Telemetry”, “Shapes”, ) T1 Publisher T1 T3 Subscriber TxOpenSplice DDS Topic Instances/Samples Ty Tb Ta DataWrter DataReader Tc Publishers and Subscribers in DDS play two roles: ☐ Coordinate actions across DataReaders/DataWriters ☐ e.g. Atomic writes across data-writers ☐ Control access to DDS partitions, a mechanism provided by DDS to further
  24. 24. DDS Partitions Domain Partitions DDS Publishers and Copyright  2011,  PrismTech  –    All  Rights  Reserved. ☐ Subscribers can specify a M:0:0 M:0:1 M:0:2 list of regular expressions representing the partitionsOpenSplice DDS to join M:1:0 M:1:1 M:1:2 ☐ Actual partition can be any string, but some M:2:0 M:2:1 M:2:2 structure can be given in order mimic hierarchy
  25. 25. DDS Partitions Matching Domain Partitions Copyright  2011,  PrismTech  –    All  Rights  Reserved. M:0:0 M:0:1 M:0:2 M:0:*OpenSplice DDS M:1:0 M:1:1 M:1:2 M:2:0 M:2:1 M:2:2
  26. 26. DDS Partitions Matching Domain Partitions Copyright  2011,  PrismTech  –    All  Rights  Reserved. M:0:0 M:0:1 M:0:2OpenSplice DDS M:1:0 M:1:1 M:1:2 M:2:0 M:2:1 M:2:2 M:*:2
  27. 27. DDS Partitions Matching Domain Partitions Copyright  2011,  PrismTech  –    All  Rights  Reserved. M:0:0 M:0:1 M:0:2OpenSplice DDS M:1:0 M:1:1 M:1:2 M:2:0 M:2:1 M:2:2 M:0:0, M:1:1, M:2:2
  28. 28. DataReaders & DataWritersOpenSplice DDS
  29. 29. DataReader / DataWriter Domain (e.g. Domain 123) Domain Participant Copyright  2011,  PrismTech  –    All  Rights  Reserved. Topic Partition (e.g. “Telemetry”, “Shapes”, ) T1 Publisher T1 T3 Subscriber TxOpenSplice DDS Topic Instances/Samples Ty Tb Ta DataWrter DataReader Tc ☐ DataWriters allow to write samples for a specific Topic with a given QoS ☐ DataReaders allow to read samples for a specific Topic with a given QoS ☐ The Samples written/read are those belonging to the partitions associated with the Publisher/Subscriber
  30. 30. Subscription MatchingOpenSplice DDS
  31. 31. Subscription Matching For a DataWriter and a DataReader to match the following Copyright  2011,  PrismTech  –    All  Rights  Reserved. conditions have to hold: ☐ The Topic Name has to be the identicalOpenSplice DDS ☐ Said X the Topic Type associated with the DataWriter and Y the Topic Type associated with the Data reader, then X <: Y (recall that annotations control the result subtype test...) ☐ The QoS with which the DataWriter produces data matches or exceeds the DataReader QoS
  32. 32. Type Matching struct ShapeType3D { struct ShapeType3D { @Key @Key string color; string color; long x; long x; long y; long y; Copyright  2011,  PrismTech  –    All  Rights  Reserved. long shapesize; DW long shapesize; DR1 long z; long z; }; }; CircleOpenSplice DDS Circle ☐ As ShapeType3D <: ShapeType DW and DR2 match Circle struct ShapeType { DDS will take care of projecting @Key ☐ DR2 string color; ShapeType3D into ShapeType long x; long y; long shapesize; };
  33. 33. Type Matching @Extensibility(MUTABLE_EXTENSIBILITY) @Extensibility(MUTABLE_EXTENSIBILITY) struct ShapeTypeC { struct ShapeTypeA { @Key @ID(0) @Key @ID(0) string color; string color; @ID(10) @MustUnderstand @ID(10) @MustUnderstand DW1 long x; DR1 Copyright  2011,  PrismTech  –    All  Rights  Reserved. long x; @ID(11) @MustUnderstand @ID(11) @MustUnderstand long y; }; long y; @ID(12) @Optional long z; @ID(20) @Optional ☐ In this example, as a long shapesize; }; Circle Circle consequence ofOpenSplice DDS annotations, DR1 Circle Circle and DR2 match @Extensibility(MUTABLE_EXTENSIBILITY) struct ShapeTypeB { DW1 and DW2 @Key @ID(0) @Extensibility(MUTABLE_EXTENSIBILITY) string color; struct ShapeTypeA { @ID(10) @MustUnderstand @Key @ID(0) long x; DR2 string color; @ID(11) @MustUnderstand DW2 @ID(10) @MustUnderstand long y; }; long x; @ID(11) @MustUnderstand long y; @ID(12) @Optional long z; @ID(20) @Optional long shapesize; };
  34. 34. Anatomy of a DDS ApplicationOpenSplice DDS
  35. 35. Stepping into ScalaOpenSplice DDS Angelo CORSARO, Ph.D. Chief Technology Officer OMG DDS Sig Co-Chair PrismTech angelo.corsaro@prismtech.com
  36. 36. What is Scala Copyright  2011,  PrismTech  –    All  Rights  Reserved. ☐ Scala (pronounced Skah-lah) stands for “Scalable language” ☐ It is a language that carefully and creatively blends Object Oriented and Functional constructs into a statically typedOpenSplice DDS language with sophisticated type inference ☐ Scala targets the JVM and .NET runtime and is 100% compatible with Java
  37. 37. Why Should You Care? Scala is simple to write, extremely compact and easy to Copyright  2011,  PrismTech  –    All  Rights  Reserved. ☐ understand ☐ Scala is strongly typed with a structural type systemOpenSplice DDS ☐ Scala is an extensible language (many construct are build in the standard library) ☐ Scala makes it easy to design Domain Specific Language
  38. 38. Case Study: Complex NumbersOpenSplice DDS
  39. 39. Complex Numbers Copyright  2011,  PrismTech  –    All  Rights  Reserved. ☐ To explore some of the nice features of Scala, let’s see how we might design a Complex number class ☐ What we expect to be able to do is all mathematical operationsOpenSplice DDS between complex numbers as well as scalar multiplications and division ☐ [(1+i2)+2*(3-i5)*(i4)]/(1+i3) ☐ ~(1+i2) [conjugate] ☐ !(3+i4) [Modulo]
  40. 40. Constructor ☐ Scala allows to implicitly create constructors and attributes starting from Copyright  2011,  PrismTech  –    All  Rights  Reserved. a simple argument list associated with the class declarationOpenSplice DDS class Complex(val re: Float, val im: Float)
  41. 41. In Java public class Complex { Copyright  2011,  PrismTech  –    All  Rights  Reserved. ! private final float re; ! private final float im; ! public Complex(float re, float im) { ! ! this.re = re;OpenSplice DDS ! ! this.im = im; ! } ! public Complex(float f) { ! ! this.re = f; ! ! this.im = 0; ! } ! public float re() { return re;} ! public float im() { return im;}
  42. 42. Methods ☐ Everything in Scala is a method even operators Copyright  2011,  PrismTech  –    All  Rights  Reserved. ☐ Methods name can be symbols such as *, !, +, etc.OpenSplice DDS def + (c: Complex) : Complex = Complex(re + c.re, im + c.im) or, taking advantage of type inference.... def + (c: Complex) = Complex(re + c.re, im + c.im)
  43. 43. In Java... Copyright  2011,  PrismTech  –    All  Rights  Reserved. public Complex add(Complex that) { return new Complex(this.re() + that.re(),OpenSplice DDS this.im() + that.im()); }
  44. 44. As a result... val result = Complex(1,2) + Complex(2,3) Copyright  2011,  PrismTech  –    All  Rights  Reserved.OpenSplice DDS Complex c1 = new Complex(1, 2); Complex c2 = new Complex (3,4); Complex c3 = c1.add(c2); or... Complex c3 = (new Complex(1, 2).add(new Complex (3,4));
  45. 45. Companion Object ☐ Scala does not have the concept of static methods/attributes ☐ On the other hand it provides built-in support for Singletons, Copyright  2011,  PrismTech  –    All  Rights  Reserved. which are specified with the “object” keyword as opposed to the “class” keywordOpenSplice DDS object Complex { The companion object, is def apply(real: Float, img: Float) = new Complex(real, img) the object associated with def apply(real: Float) = new Complex(real, 0) a class, which shares the same name and provides implicit def floatToReComplex (f: Float) = new ReComplex(f) typically helper methods implicit def intToReComplex(i : Int) = new ReComplex(i) }
  46. 46. “apply” Magic ☐ When an instance of a class is followed by parentheses with a list of zero Copyright  2011,  PrismTech  –    All  Rights  Reserved. or more parameters, the compiler invokes the apply method for that instance ☐ This is true for an object with a defined apply method (such as a companion object), as well as an instance of a class that defines anOpenSplice DDS apply method val result = Complex(1,2) is the same as.... val result = Complex.apply(1,2)
  47. 47. Negation and Scalar Multiplication Copyright  2011,  PrismTech  –    All  Rights  Reserved. ☐ In order to design a Complex class that is well integrated in our type system we should be able to support the following cases: ☐ -(a+ib) ☐ c*(a+ib)OpenSplice DDS ☐ (a+ib)*c ☐ How can we supporting something like -(a+ib) and c*(a+ib)?
  48. 48. Scala Unary Operators ☐ Scala allows to define unary operators for the following method Copyright  2011,  PrismTech  –    All  Rights  Reserved. identifiers +, -, !, ~ def unary_-() = Complex(-re, -im)OpenSplice DDS ! def unary_!() = Math.sqrt(re*re + im*im) ! def unary_~() = Complex(re, -im) as a result we can write: val result = -Complex(1,2) + ~Complex(2,3)
  49. 49. Scala Implicit Conversions Copyright  2011,  PrismTech  –    All  Rights  Reserved. ☐ The expression: val c3 = 3*Complex(5, 7) ☐ Is equivalent to:OpenSplice DDS val c3 = 3.*(Complex(5, 7)) ☐ Yet, the method to multiply a Integer to a Complex is not present in the Scala Int class ☐ What can we do to make the trick?
  50. 50. Scala Implicit Conversions Copyright  2011,  PrismTech  –    All  Rights  Reserved. ☐ Scala does not support Open Classes, thus allowing to add new methods to existing classes ☐ Yet Scala supports implicit conversions that can be used to achieve the same resultOpenSplice DDS ☐ Lets see how...
  51. 51. Scala Implicit Conversion object Complex { Copyright  2011,  PrismTech  –    All  Rights  Reserved. implicit def floatToReComplex (f: Float) = new ReComplex(f) implicit def intToReComplex(i : Int) = new ReComplex(i) }OpenSplice DDS class ReComplex(re: Float) { def * (that: Complex) = Complex(re*that.re,re*that.im) }
  52. 52. The Result is... val c3 = 3*Complex(5, 7) Copyright  2011,  PrismTech  –    All  Rights  Reserved. is converted automatically into: val c3 = ReComplex(3).*(Complex(5, 7))OpenSplice DDS
  53. 53. Putting it all together case class Complex(val re: Float, val im: Float) { // Unary Operators Copyright  2011,  PrismTech  –    All  Rights  Reserved. def unary_-() = Complex(-re, -im) // Binary Operators def unary_!() = Math.sqrt(re*re + im*im) def + (c: Complex) = Complex(re + c.re, im + c.im) def unary_~() = Complex(re, -im) def - (c: Complex) = Complex(re - c.re, im - c.im) // Formatting def * (f: Float) = Complex(f*re, f*im) override def toString() : String = {OpenSplice DDS def * (c: Complex) = Complex((re*c.re) - (im*c.im), if (im > 0) re + "+i" + im ((re*c.im) + (im*c.re))) else if (im < 0) re + "-i" + (-im) else re.toString def / (c: Complex) = { val d = c.re*c.re + c.im*c.im } Complex(((re*c.re) + (im + c.im))/d, } ((im*c.re) - (re*c.im))/d ) }
  54. 54. Functions, Closures and CurryingOpenSplice DDS
  55. 55. Functions Copyright  2011,  PrismTech  –    All  Rights  Reserved. ☐ Scala has first-class functions ☐ Functions can be defined and called, but equally functions can be defined as unnamed literals and passed as valuesOpenSplice DDS def inc(x: Int) = x + 1 val vinc = (x: Int) => x+1 inc(5) vinc(5) Notice once again the uniform access principle
  56. 56. Playing with Functions val list = List(1,2,3,4,5,6,7,8,9) Copyright  2011,  PrismTech  –    All  Rights  Reserved. val g5 = list.filter((x: Int) => x > 5) g5: List[Int] = List(6, 7, 8, 9)OpenSplice DDS Or with placeholder syntax val list = List(1,2,3,4,5,6,7,8,9) val g5 = list.filter(_ > 5) g5: List[Int] = List(6, 7, 8, 9)
  57. 57. Closures Copyright  2011,  PrismTech  –    All  Rights  Reserved. ☐ Scala allows you to define functions which include free variables meaning variables whose value is not bound to the parameter list ☐ Free variable are resolved at runtime considering the closure of visible variableOpenSplice DDS ☐ Example: def mean(e : Array[Float]) : Float = { def mean(e : Array[Float]) : Float = { var sum = 0.0F var sum = 0.0F e.foreach((x: Int) => sum += x) e.foreach(sum += _) return sum/e.length return sum/e.length } }
  58. 58. Currying ☐ Scala provides support for curried functions which are applied to Copyright  2011,  PrismTech  –    All  Rights  Reserved. multiple argument lists, instead of just one ☐ Currying is the mechanism Scala provides for introducing new control abstractionOpenSplice DDS def curriedSum(x: Int)(y: Int) = x + y curriedSum(1) { 3 +5 }
  59. 59. OpenSplice DDS Traits
  60. 60. Traits ☐ Scala supports single inheritance from classes but can mix-in multiple Copyright  2011,  PrismTech  –    All  Rights  Reserved. traits ☐ A trait is the unit of code reuse for Scala. It encapsulate methods and field definitions Traits usually expect a class to implement an abstract method, whichOpenSplice DDS ☐ constitutes the “narrow” interface that allows to implement a rich behaviour ☐ Traits are also very useful for dependency injection
  61. 61. Ordered Complex Numbers Copyright  2011,  PrismTech  –    All  Rights  Reserved. ☐ Our complex numbers are not comparable ☐ Let’s assume that we wanted to make them comparable, and lets supposed that we define the total order as based on the module of the complex numberOpenSplice DDS ☐ How can we implement this behavior?
  62. 62. Ordered Trait Copyright  2011,  PrismTech  –    All  Rights  Reserved. ☐ The Ordered[T] traits encapsulates the set of methods that allow to define a total ordering over a type ☐ All the behaviour is defined in terms of an abstract method, namely “compare”OpenSplice DDS ☐ Classes that mix-in this trait have to implement the “compare” method class Complex(val re: Float, val im: Float) extends Ordering[Complex] { def compare(x: Complex, y: Complex) = { if (x == y) 0 else if (!x > !y) 1 else -1 }
  63. 63. Case Classes &OpenSplice DDS Pattern Matching
  64. 64. Case Classes and Pattern Matching ☐ Case Classes and Pattern Matching are twin constructs that are pretty Copyright  2011,  PrismTech  –    All  Rights  Reserved. useful when dealing with tree-like recursive data structures ☐ These constructs allow to match patterns in an expression and reconstruct the object graph that makes it upOpenSplice DDS ☐ Lets see an example...
  65. 65. Case Classes and Pattern abstract class Expr case class Var(name: String) extends Expr Copyright  2011,  PrismTech  –    All  Rights  Reserved. case class Number(num: Float) extends Expr case class UnOp(operator: String, arg: Expr) extends Expr case class BinOp(operator: String, left: Expr, right: Expr)OpenSplice DDS def simplifyExpr(expr: Expr) : Expr = expr match { case UnOp("-", UnOp("-", e)) => e case BinOp("+", e, Number("0")) => e case BinOp("*", e, Number("1")) => e case _ => expr }
  66. 66. Type ParametrizationOpenSplice DDS
  67. 67. Type Parametrization Copyright  2011,  PrismTech  –    All  Rights  Reserved. ☐ Scala provides support for type parametrization and makes it available for both classes as well as traits trait Queue[T] { def head: TOpenSplice DDS def tail: Queue[T] def append(x: T) : Queue[T] } ☐ Scala allows to annotate the parametrized type to control the resulting type variance
  68. 68. Type Variance ☐ If S <: T is Queue[S] <: Queue[T]? Copyright  2011,  PrismTech  –    All  Rights  Reserved. ☐ By default Scala makes generic types nonvariant. This behaviour can be changed using the following annotations: ☐ Queue[+T] indicates that the the sub-typing is covariant in the parameter TOpenSplice DDS ☐ Queue[-T] indicates that the the sub-typing is contravariant in the parameter T
  69. 69. Anatomy of a DDS ApplicationOpenSplice DDS
  70. 70. Anatomy of a DDS Application [Scala API] Domain Domain Copyright  2011,  PrismTech  –    All  Rights  Reserved. val dp = DomainParticipant(domainId) Participant Publisher Topic SubscriberOpenSplice DDS DataWriter DataReader
  71. 71. Anatomy of a DDS Application [Scala API] Domain Domain Copyright  2011,  PrismTech  –    All  Rights  Reserved. val dp = DomainParticipant(domainId) Participant Session // Create a Topic Publisher Topic SubscriberOpenSplice DDS val topic = Topic[ShapeType](dp, “Circle”) // Create a Publisher / Subscriber val pub = Publisher(dp) val sub = Subscriber(dp) DataWriter DataReader
  72. 72. Anatomy of a DDS Application [Scala API] Domain Domain Copyright  2011,  PrismTech  –    All  Rights  Reserved. val dp = DomainParticipant(domainId) Participant Session // Create a Topic Publisher Topic SubscriberOpenSplice DDS val topic = Topic[ShapeType](dp, “Circle”) // Create a Publisher / Subscriber val pub = Publisher(dp) val sub = Subscriber(dp) Reader/Writers for User Defined for Types DataWriter DataReader // Create a DataWriter/DataWriter val writer = DataWriter[ShapeType](pub, topic) Reader/Writer for val reader = DataReader[ShapeType](sub, topic) application defined Topic Types
  73. 73. Anatomy of a DDS Application [Scala API] Domain Domain Copyright  2011,  PrismTech  –    All  Rights  Reserved. val dp = DomainParticipant(domainId) Participant Session // Create a Topic Publisher Topic SubscriberOpenSplice DDS val topic = Topic[ShapeType](dp, “Circle”) // Create a Publisher / Subscriber val pub = Publisher(dp) val sub = Subscriber(dp) Reader/Writers for User Defined for Types DataWriter DataReader // Write data val data = new ShapeType(“RED”, 131, 107, 75) writer write data Reader/Writer for // But you can also write like this... application defined writer ! data Topic Types // Read new data and print it on the screen (reader read) foreach (prinln)
  74. 74. Data Reader/Writer CachesOpenSplice DDS
  75. 75. Reader/Writer Caches Application Application Application Copyright  2011,  PrismTech  –    All  Rights  Reserved. DataWriter DataReader DataReader ...OpenSplice DDS DataWriter Cache DataReader Cache DataReader Cache Network
  76. 76. Reader/Writer Caches write( ) Application Application Copyright  2011,  PrismTech  –    All  Rights  Reserved. DataWriter DataReader DataReader ...OpenSplice DDS DataWriter Cache DataReader Cache DataReader Cache Network
  77. 77. Reader/Writer Caches Application Application Application Copyright  2011,  PrismTech  –    All  Rights  Reserved. DataWriter DataReader DataReader ...OpenSplice DDS DataWriter Cache DataReader Cache DataReader Cache Topic Instance Network
  78. 78. Reader/Writer Caches Application Application Application Copyright  2011,  PrismTech  –    All  Rights  Reserved. DataWriter DataReader DataReader ...OpenSplice DDS DataWriter Cache DataReader Cache DataReader Cache Network
  79. 79. Reader/Writer Caches Application Application Application Copyright  2011,  PrismTech  –    All  Rights  Reserved. DataWriter DataReader DataReader ...OpenSplice DDS DataWriter Cache DataReader Cache DataReader Cache Topic Instance Topic Instance Network
  80. 80. Reader/Writer Caches write( ) Application Application Copyright  2011,  PrismTech  –    All  Rights  Reserved. DataWriter DataReader DataReader ...OpenSplice DDS DataWriter Cache DataReader Cache DataReader Cache Network
  81. 81. Reader/Writer Caches Application Application Application Copyright  2011,  PrismTech  –    All  Rights  Reserved. DataWriter DataReader DataReader ...OpenSplice DDS DataWriter Cache DataReader Cache DataReader Cache Writer History Network
  82. 82. Reader/Writer Caches Application Application Application Copyright  2011,  PrismTech  –    All  Rights  Reserved. DataWriter DataReader DataReader ...OpenSplice DDS DataWriter Cache DataReader Cache DataReader Cache Network
  83. 83. Reader/Writer Caches Application Application Application Copyright  2011,  PrismTech  –    All  Rights  Reserved. DataWriter DataReader DataReader ...OpenSplice DDS DataWriter Cache DataReader Cache DataReader Cache Reader History Reader History Network
  84. 84. Reader/Writer Caches write( ) Application Application Copyright  2011,  PrismTech  –    All  Rights  Reserved. DataWriter DataReader DataReader ...OpenSplice DDS DataWriter Cache DataReader Cache DataReader Cache Network
  85. 85. Reader/Writer Caches Application Application Application Copyright  2011,  PrismTech  –    All  Rights  Reserved. DataWriter DataReader DataReader ...OpenSplice DDS DataWriter Cache DataReader Cache DataReader Cache Network
  86. 86. Reader/Writer Caches Application Application Application Copyright  2011,  PrismTech  –    All  Rights  Reserved. DataWriter DataReader DataReader ...OpenSplice DDS DataWriter Cache DataReader Cache DataReader Cache Network
  87. 87. Reader/Writer Caches Application Application Application Copyright  2011,  PrismTech  –    All  Rights  Reserved. DataWriter DataReader DataReader ...OpenSplice DDS DataWriter Cache DataReader Cache DataReader Cache Network
  88. 88. Reader/Writer Caches Application Application Application Copyright  2011,  PrismTech  –    All  Rights  Reserved. DataWriter DataReader DataReader ...OpenSplice DDS DataWriter Cache DataReader Cache DataReader Cache QoS Policies ‣ History ‣ Destination Order Network ‣ Presentation ‣ Partition QoS Policies ‣ Time Based Filter ‣ Reliability ‣ Durability ‣ Ownership ‣ History ‣ Transport Priority ‣ Latency Budget ‣ Time Based Filter
  89. 89. Writer Cache Properties ☐ An entry for each topic instance Copyright  2011,  PrismTech  –    All  Rights  Reserved. ☐ Each cache entry might contain one or more samples, the number of DataWriter samples is controlled by the History QoS PolicyOpenSplice DDS ... ☐ Depending on the setting of the Durability QoS Policy, the content of the cache may be made available DataWriter Cache to late joiners. The depth of the cache controls how many samples per instance will be delivered to late joiners
  90. 90. Reader Cache Properties ☐ An entry for each topic instance Copyright  2011,  PrismTech  –    All  Rights  Reserved. ☐ Each cache entry might contain one or more samples, the number of samples is controlled by the History QoS Policy DataReader ☐ History.KeepLast(n) keeps the last n samplesOpenSplice DDS per instance ☐ History.KeepAll maintains all the samples, ... modulo resource constraints ☐ For a given instance: DataReader Cache ☐ Samples from the same writer are inserted in write-order ☐ Samples from multiple-writers are ordered using time-stamps (source or reception depending on configurable a QoS Policy)
  91. 91. Eventual Consistency DDS caches provide eventual consistency semantics Copyright  2011,  PrismTech  –    All  Rights  Reserved. ☐ ☐ This means that a read will see the effect of a preceding write eventuallyOpenSplice DDS ☐ Furthermore, given a data-writer that is currently matching N readers, we can think of DDS as providing eventual consistency with W=0 and R=1 ☐ W: the number of Acks expected in order to return from a write ☐ R: the number of sources from which a read access data
  92. 92. Understanding HistoryOpenSplice DDS
  93. 93. Reader/Writer Caches Application Application Application Copyright  2011,  PrismTech  –    All  Rights  Reserved. DataWriter DataReader DataReader ...OpenSplice DDS DataWriter Cache DataReader Cache DataReader Cache Network Assumptions: DataReader configured with history depth of 2 and DataWriter with default history depth of 1
  94. 94. Reader/Writer Caches write( 1 ) Application Application Copyright  2011,  PrismTech  –    All  Rights  Reserved. DataWriter DataReader DataReader ...OpenSplice DDS DataWriter Cache DataReader Cache DataReader Cache Network Assumptions: DataReader configured with history depth of 2 and DataWriter with default history depth of 1
  95. 95. Reader/Writer Caches Application Application Application Copyright  2011,  PrismTech  –    All  Rights  Reserved. DataWriter DataReader DataReader 1 ...OpenSplice DDS DataWriter Cache DataReader Cache DataReader Cache Network Assumptions: DataReader configured with history depth of 2 and DataWriter with default history depth of 1
  96. 96. Reader/Writer Caches Application Application Application Copyright  2011,  PrismTech  –    All  Rights  Reserved. DataWriter DataReader DataReader 1 ...OpenSplice DDS DataWriter Cache DataReader Cache DataReader Cache 1 Network Assumptions: DataReader configured with history depth of 2 and DataWriter with default history depth of 1
  97. 97. Reader/Writer Caches Application Application Application Copyright  2011,  PrismTech  –    All  Rights  Reserved. DataWriter DataReader DataReader 1 1 1 ...OpenSplice DDS DataWriter Cache DataReader Cache DataReader Cache Network Assumptions: DataReader configured with history depth of 2 and DataWriter with default history depth of 1
  98. 98. Reader/Writer Caches write( 2 ) Application Application Copyright  2011,  PrismTech  –    All  Rights  Reserved. DataWriter DataReader DataReader 1 1 1 ...OpenSplice DDS DataWriter Cache DataReader Cache DataReader Cache Network Assumptions: DataReader configured with history depth of 2 and DataWriter with default history depth of 1
  99. 99. Reader/Writer Caches Application Application Application Copyright  2011,  PrismTech  –    All  Rights  Reserved. DataWriter DataReader DataReader 2 1 1 ...OpenSplice DDS DataWriter Cache DataReader Cache DataReader Cache Network Assumptions: DataReader configured with history depth of 2 and DataWriter with default history depth of 1
  100. 100. Reader/Writer Caches Application Application Application Copyright  2011,  PrismTech  –    All  Rights  Reserved. DataWriter DataReader DataReader 2 1 1 ...OpenSplice DDS DataWriter Cache DataReader Cache DataReader Cache 2 Network Assumptions: DataReader configured with history depth of 2 and DataWriter with default history depth of 1
  101. 101. Reader/Writer Caches Application Application Application Copyright  2011,  PrismTech  –    All  Rights  Reserved. DataWriter DataReader DataReader 2 1 1 ...OpenSplice DDS 2 2 DataWriter Cache DataReader Cache DataReader Cache Network Assumptions: DataReader configured with history depth of 2 and DataWriter with default history depth of 1
  102. 102. Reader/Writer Caches write( 3 ) Application Application Copyright  2011,  PrismTech  –    All  Rights  Reserved. DataWriter DataReader DataReader 2 1 1 ...OpenSplice DDS 2 2 DataWriter Cache DataReader Cache DataReader Cache Network Assumptions: DataReader configured with history depth of 2 and DataWriter with default history depth of 1
  103. 103. Reader/Writer Caches Application Application Application Copyright  2011,  PrismTech  –    All  Rights  Reserved. DataWriter DataReader DataReader 3 1 1 ...OpenSplice DDS 2 2 DataWriter Cache DataReader Cache DataReader Cache Network Assumptions: DataReader configured with history depth of 2 and DataWriter with default history depth of 1
  104. 104. Reader/Writer Caches Application Application Application Copyright  2011,  PrismTech  –    All  Rights  Reserved. DataWriter DataReader DataReader 3 1 1 ...OpenSplice DDS 2 2 DataWriter Cache DataReader Cache DataReader Cache 3 Network Assumptions: DataReader configured with history depth of 2 and DataWriter with default history depth of 1
  105. 105. Reader/Writer Caches Application Application Application Copyright  2011,  PrismTech  –    All  Rights  Reserved. DataWriter DataReader DataReader 3 2 2 ...OpenSplice DDS 3 3 DataWriter Cache DataReader Cache DataReader Cache Network Assumptions: DataReader configured with history depth of 2 and DataWriter with default history depth of 1
  106. 106. Caches or Streams?OpenSplice DDS
  107. 107. Dynamic View of a Stream Copyright  2011,  PrismTech  –    All  Rights  Reserved.OpenSplice DDS ... Stream: Set of samples written over time for a given topic instance.
  108. 108. Dynamic View of a Stream Assumptions: Reader History = KeepLast (n) WriterHistory = KeepLast (m) Copyright  2011,  PrismTech  –    All  Rights  Reserved. Writer HistoryOpenSplice DDS ... Stream: Set of samples written over time for a given topic instance.
  109. 109. Dynamic View of a Stream Assumptions: Reader History = KeepLast (n) WriterHistory = KeepLast (m) Copyright  2011,  PrismTech  –    All  Rights  Reserved. Samples ‘on Writer History the wire’OpenSplice DDS ... Stream: Set of samples written over time for a given topic instance.
  110. 110. Dynamic View of a Stream Assumptions: Reader History = KeepLast (n) WriterHistory = KeepLast (m) Copyright  2011,  PrismTech  –    All  Rights  Reserved. Samples ‘on Writer History the wire’ Reader HistoryOpenSplice DDS ... Stream: Set of samples written over time for a given topic instance.
  111. 111. Dynamic View of a Stream Assumptions: Reader History = KeepLast (n) WriterHistory = KeepLast (m) Copyright  2011,  PrismTech  –    All  Rights  Reserved. Samples ‘on Writer History the wire’ Reader History ‘Past’ SamplesOpenSplice DDS ... Stream: Set of samples written over time for a given topic instance.
  112. 112. Eventual View of a Stream Assumptions (Default Settings): Reader History = KeepLast (1) WriterHistory = KeepLast (1) Copyright  2011,  PrismTech  –    All  Rights  Reserved. Writer History Reader History ‘Past’ SamplesOpenSplice DDS Stream: Set of samples written over time for a given topic instance.
  113. 113. Eventual View of a Stream Assumptions: Reader History = KeepLast (n) with n > 1 WriterHistory = KeepLast (1) Copyright  2011,  PrismTech  –    All  Rights  Reserved. Writer History Reader History ‘Past’ SamplesOpenSplice DDS Stream: Set of samples written over time for a given topic instance.
  114. 114. Eventual View of a Stream Assumptions: Reader History = KeepLast (n) with n > 1 WriterHistory = KeepLast (m) with n > m > 1 Copyright  2011,  PrismTech  –    All  Rights  Reserved. Writer History Reader History ‘Past’ SamplesOpenSplice DDS Stream: Set of samples written over time for a given topic instance.
  115. 115. Interacting with the DataReader CacheOpenSplice DDS
  116. 116. Reading Data Samples ☐ Samples can be read from the Data Reader History Cache Copyright  2011,  PrismTech  –    All  Rights  Reserved. ☐ The action of reading a sample is non-destructive. Samples are not removed from the cacheOpenSplice DDS DataReader DataReader ... read ... DataReader Cache DataReader Cache
  117. 117. Taking Data Samples ☐ Samples can be taken from the Data Reader History Cache Copyright  2011,  PrismTech  –    All  Rights  Reserved. ☐ The action of taking a sample is destructive. Samples are removed from the cacheOpenSplice DDS DataReader DataReader ... take ... DataReader Cache DataReader Cache
  118. 118. Read vs. Take Copyright  2011,  PrismTech  –    All  Rights  Reserved. ☐ The read operation should always be access the latest know value for topics that represent distributed stateOpenSplice DDS ☐ The take operation should be used to get the last notification from a topic that represent an event
  119. 119. Data SelectorsOpenSplice DDS
  120. 120. Cherry Picking in DDS Copyright  2011,  PrismTech  –    All  Rights  Reserved. ☐ DDS provides some very flexible mechanisms for selecting the data to be read:OpenSplice DDS ☐ Data Content ☐ Data Status ☐ These mechanisms are composable
  121. 121. Content-Based Data SelectionOpenSplice DDS
  122. 122. Filters and Queries Application ☐ DDS Filters allow to control what gets into a DataReader cache Copyright  2011,  PrismTech  –    All  Rights  Reserved. Query ☐ DDS Queries allow to control what gets out of a DataReader cache DataReaderOpenSplice DDS ☐ Filters are defined by means of ... ContentFilteredTopics ... ... ... ☐ Queries operate in conjunction with DataReader Cache read operations Filter ☐ Filters and Queries are expressed as SQL where clauses
  123. 123. struct ShapeType { Filters @Key string color; long x; long y; [Scala API] }; long shapesize; /** Copyright  2011,  PrismTech  –    All  Rights  Reserved. * NOTE: The Scala API if not provided with DP/Sub/Pub assumes * default domains and default partition. **/ // Create a Topic val topic = Topic[ShapeType](“Circle”) // Define filter expression and parametersOpenSplice DDS val query = Query(“x < %0 AND y < %1”, List(“200”, “300”)) // Define content filtered topic val cftopic = ContentFilteredTopic[ShapeType](“Circle”, topic, query) // Create a DataReader for the content-filtered Topic val reader = DataReader[ShapeType](cftopic)

×