Download as PDF, PPTX
//
Define
the
publisher
and
link-‐it
up
to
the
“chirp:@user”
partition
val
pubQos
=
PublisherQos()
.withPolicy(Partition(Config.ChirpRootPartition
+
user))
implicit
val
pub
=
Publisher(pubQos)
//
Define
the
data
writer
and
make
it
reliable
and
for
the
time
being
//
Transient
Local…
Will
refined
this
in
the
Part
II
val
dwQos
=
DataWriterQos().withPolicies(
Reliability.Reliable,
Durability.TransientLocal)
//
Create
the
DataWriter
val
dw
=
DataWriter[ChirpAction](pub,
topic,
dwQos)
//
Chirp!!!
val
chirp
=
…
dw.write(chirp)](https://image.slidesharecdn.com/vortex-tutorial-part-1-oct2014-141009043108-conversion-gate02/85/PrismTech-Vortex-Tutorial-Part-1-45-320.jpg)
//
Define
the
subscribe
and
link-‐it
up
to
the
partition
associated
to
//
the
user
followed
val
subQos
=
PublisherQos()
.withPolicy(Partition(followerList.map(Config.ChirpRootPartition
+
_))
implicit
val
sub
=
Subscriber(subQos)
//
Define
the
data
reader
and
make
it
reliable
and
for
the
time
being
//
Transient
Local…
Will
refined
this
in
the
Part
II
val
drQos
=
DataReaderQos().withPolicies(
Reliability.Reliable,
Durability.TransientLocal)
//
Create
the
DataWriter
val
dr
=
DataReader[ChirpAction](sub,
topic,
drQos)](https://image.slidesharecdn.com/vortex-tutorial-part-1-oct2014-141009043108-conversion-gate02/85/PrismTech-Vortex-Tutorial-Part-1-46-320.jpg)
Uploaded byADLINK Technology IoT
PrismTech Vortex Tutorial Part 1
The document outlines the Vortex platform, emphasizing its role in seamless data sharing across various applications and its adherence to the OMG DDS standard. It describes the architecture and security model of Vortex, including features for creating a micro-blogging application called Chirpit, leveraging partitions and content filters for data distribution. The summary includes implementation details for data writers and readers, as well as plans for future enhancements involving application configuration and analytics.
More Related Content
Similar to PrismTech Vortex Tutorial Part 1
PrismTech Vortex Tutorial Part 1
- 1. Angelo Corsaro, PhD Chief Technology Officer angelo.corsaro@prismtech.com Vortex Tutorial Part I
- 2. CopyrightPrismTech,2014 Vortex enables seamless, ubiquitous,efficient and timely data sharing across mobile, embedded, desktop, cloud and web applications Vortex is based on the OMG DDS standard The Vortex Platform Vortex Device Tools Integration Vortex Cloud MaaS
- 3.
- 4.
- 5. CopyrightPrismTech,2014 Support for transportlevel security via TLS and DTLS Support for fine grained access control Access control plug-in compliant with DDS Security Vortex Security Model Arthur Dent Arthur Dent Ford Prerfect Zaphod Beeblebrox Marvin Trillian A(r,w), B(r) A(r,w), B(r,w), X(r) *(r,w) *(r) A(r,w), B(r,w), C(r,w) Ford Prerfect Zaphod Beeblebrox Trillian Marvin A B A,B X * * A,B,C Identity Access Rights Sessions are authenticated and communication is encrypted Only the Topic included as part of the access rights are visible and accessible
- 6.
- 7.
- 8. CopyrightPrismTech,2014 E-Health Hospital A PrivateCloud Analytics Big Data Store Hospital B
- 9.
- 10. CopyrightPrismTech,2014 SESAR PENS PENS: Pan EuropeanNetwork Service FDP: Flight Data Processor CWP: Controller Working Position IPv6 network SSM support FDP FDPFDP CWP Tower Device FDP FDPFDP Tower Device CWP FDP FDPFDP Tower Device CWP FDP FDPFDP CWP Tower Device EU Nation 4 EU Nation 1 EU Nation 2 EU Nation 3
- 11.
- 12. CopyrightPrismTech,2014 To explore thevarious features provided by the Vortex platform we will be designing and implementing a micro-blogging platform called ChirpIt. Specifically, we want to support the following features: ChirpIt users should be able to “chirp”, “re-chirp”, “like” and “dislike” trills as well as get detailed statistics The ChirpIt platform should provide information on trending topics — identified by hashtags — as well as trending users Third party services should be able to flexibly access slices of produced trills to perform their own trend analysis ChirpIt should scale to millions of users ChirpIt should be based on a Lambda Architecture ChirpIt Requirements
- 13. CopyrightPrismTech,2014 ChirpIt Architecture analytics data centre chirp emotions stats batch layer chirpit apps Cloud Messaging serving layer chirp emotions stats master dataset view view view chirp emotions stats 3rd party svcs Speed layer
- 14.
- 15. CopyrightPrismTech,2014 DDS provides aGlobal Data Space abstraction that allows applications to autonomously, anonymously, securely and efficiently share data DDS’ Global Data Space is fully distributed (decentralised), highly efficient and scalable Data Distribution Service (DDS) DDS Global Data Space ... Data Writer Data Writer Data Writer Data Reader Data Reader Data Reader Data Reader Data Writer TopicA QoS TopicB QoS TopicC QoS TopicD QoS
- 16. CopyrightPrismTech,2014 DataWriters and DataReadersare automatically and dynamically matched by the DDS Discovery A rich set of QoS allows to control existential, temporal, and spatial properties of data Data Distribution Service (DDS) DDS Global Data Space ... Data Writer Data Writer Data Writer Data Reader Data Reader Data Reader Data Reader Data Writer TopicA QoS TopicB QoS TopicC QoS TopicD QoS
- 17. CopyrightPrismTech,2014 Elegant and HighLevel Data Sharing Abstraction Polyglot and platform independent • Java, Scala, C, C++, C#, JavaScript, CoffeeScript etc. • Android, Windows, Linux, VxWorks, etc. Peer-to-Peer by nature, Brokered when useful Key Highlights
- 18. CopyrightPrismTech,2014 Content and TemporalFiltering (both sender and receiver filtering supported) Queries 20+ QoS to control existential, temporal, and spatial properties of data Key Highlights
- 19. CopyrightPrismTech,2014 High Performance andScalable • ~30 usec latency* • 7M msgs/sec node-to-node throughput* Key Highlights (*) Performance measured on Linux Host with i7 processor on a 1Gbps Ethernet network
- 20.
- 21.
- 22. CopyrightPrismTech,2014 DDS data liveswithin a domain A domain is identified with a non negative integer, such as 1, 3, 31 The number 0 identifies the default domain A domain represent an impassable communication plane Domain DDS Domain
- 23. CopyrightPrismTech,2014 Partitions are themechanism provided by DDS to organise information within a domain Access to partitions is controlled through QoS Policies Partitions are defined as strings: - “system:telemetry” - “system:log” - “data:row-2:col-3” Partitions addressed by name or regular expressions: - ”system:telemetry” - “data:row-2:col-*” Partitions Partitions
- 24.
- 25. CopyrightPrismTech,2014 A Topic definesa domain-wide information’s class A Topic is defined by means of a (name, type, qos) tuple, where • name: identifies the topic within the domain • type: is the programming language type associated with the topic. Types are extensible and evolvable • qos: is a collection of policies that express the non-functional properties of this topic, e.g. reliability, persistence, etc. Topic Topic Type Name QoS ... TopicA QoS TopicB QoS TopicC QoS TopicD QoS
- 26. CopyrightPrismTech,2014 As explained inthe previous slide a topic defines a class/type of information Topics can be defined as Singleton or can have multiple Instances Topic Instances are identified by means of the topic key A Topic Key is identified by a tuple of attributes -- like in databases Remarks: - A Singleton topic has a single domain-wide instance - A “regular” Topic can have as many instances as the number of different key values, e.g., if the key is an 8-bit character then the topic can have 256 different instances Topic and Instances
- 27.
- 28. CopyrightPrismTech,2014 Chirp Actions union ChirpBody switch (ChirpActionKind) { case CHIRP_KIND: string<128> chirp; case RECHIRP_KIND: string<128> chirp; string user; case LIKE_KIND: string luser; case DISLIKE_KIND: string duser; }; ! struct ChirpAction { ChirpHeader header; ChirpBody body; }; #pragma keylist ChirpAction header.id.cid struct ChirpId { string uid; string cid; }; ! enum ChirpActionKind { CHIRP_KIND, RECHIRP_KIND, LIKE_KIND, DISLIKE_KIND }; struct ChirpHeader { ChirpId id; Location location; unsigned long long timestamp; ChirpActionKind kind; };
- 29. CopyrightPrismTech,2014 ChirpIt Statistics struct ChirpStats { ChirpId id; unsigned long rechirps; unsigned long likes; }; #pragma keylist ChirpStats id.cid struct UserStats { string userId; unsigned long followers; unsigned long chirps; unsigned long followed; }; #pragma keylist UserStats userId
- 30. CopyrightPrismTech,2014 User Actions struct enum UserActionKind { FOLLOW, UNFOLLOW }; struct UserAction { UserActionKind kind; string fromUser; string toUser; };
- 31. CopyrightPrismTech,2014 App Config typedef sequence<string> Partitions; ! struct AppConfig { string userId; Partitions ps; unsigned short history; }; #pragma keylist UserConfig userId
- 32.
- 33. CopyrightPrismTech,2014 To make thesystem scale properly we need to ensure that the chirps, rechirps, etc., are distributed only to the interested parties As we have a single topic representing chirp actions, how can we ensure that information is received only by followers? DDS Provides two mechanisms: - Partitions: Allow to create information scopes - Content Filters: Allow to receive the data that satisfy certains predicates Scoping Information
- 34.
- 35.
- 36. CopyrightPrismTech,2014 In summary partitionsare used to scope information Each user will “join” a partition for each followed party Example: - If @drx follows @magneto, @wolverine, @cyclops and @mistique then he will receive ChirpActions from the partitions: • chirp:@magneto, chirp:@wolverine, chirp:@cyclops and chirp:@mistique - On the other hand, regardless of the people that follows @wolverine he will always and only write ChirpActions in the partitions • chirp:@wolverine Using Partitions to Scope Chirps
- 37.
- 38. CopyrightPrismTech,2014 A DataWriter (DW)is a strongly typed entity used to produce samples for one or more instances of a Topic, with a given QoS Conceptually, the DataWriter QoS should be the same as the Topic QoS or more stringent However, DDS does enforce a specific relationship between the Topic and DataWriter QoS DataWriter DW Type Topic QoS
- 39. CopyrightPrismTech,2014 The DataWriter controlsthe life-cycle of Topic Instances and allows to: Define a new topic instance Write samples for a topic instance Dispose the topic instance DataWriter DW Type Topic QoS
- 40.
- 41. CopyrightPrismTech,2014 A DataReader (DR)is a strongly typed entity used to access and/or consume samples for a Topic, with a given QoS Conceptually, the DataReader QoS should be the same as the Topic QoS or less stringent However, DDS does enforce a specific relationship between the Topic and DataReader QoS DataReader DR Type Topic QoS
- 42.
- 43. CopyrightPrismTech,2014 DomainParticipant: Provides accessto a data cloud -- called a domain in DDS Topic: Domain-wide definition of a kind of Information Publisher/Subscriber: Provide scope to data sharing through the concept of partitions DataReader/DataWriter: Allow to read/write data for a given topic in the partitions their Subscriber/Publisher are associated with. DDS Entities Domain (e.g. Domain 123) Domain Participant Topic Publisher DataWrter Subscriber DataReader Partition (e.g. “Telemetry”, “Shapes”, ) Topic Instances/Samples Ta Tb Tc Tx Ty T1 T1 T3
- 44.
- 45. CopyrightPrismTech,2014 ChirpOut App // Domain Participant defined by dependency injection // Define the topic val topic = Topic[ChirpAction](Config.ChirpActionTopic) // Define the publisher and link-‐it up to the “chirp:@user” partition val pubQos = PublisherQos() .withPolicy(Partition(Config.ChirpRootPartition + user)) implicit val pub = Publisher(pubQos) // Define the data writer and make it reliable and for the time being // Transient Local… Will refined this in the Part II val dwQos = DataWriterQos().withPolicies( Reliability.Reliable, Durability.TransientLocal) // Create the DataWriter val dw = DataWriter[ChirpAction](pub, topic, dwQos) // Chirp!!! val chirp = … dw.write(chirp)
- 46. CopyrightPrismTech,2014 ChirpIn // Domain Participant defined by dependency injection // Define the topic val topic = Topic[ChirpAction](Config.ChirpActionTopic) // Define the subscribe and link-‐it up to the partition associated to // the user followed val subQos = PublisherQos() .withPolicy(Partition(followerList.map(Config.ChirpRootPartition + _)) implicit val sub = Subscriber(subQos) // Define the data reader and make it reliable and for the time being // Transient Local… Will refined this in the Part II val drQos = DataReaderQos().withPolicies( Reliability.Reliable, Durability.TransientLocal) // Create the DataWriter val dr = DataReader[ChirpAction](sub, topic, drQos)
- 47. CopyrightPrismTech,2014 ChirpIn // React to new Chirp dr listen { case DataAvailable(_) => { // take vs. read will become clear in Part II val samples = dr.take() // For the time being simply print-‐out the new chirps samples .filter(s => (s.getData != null)) .foreach(s => displayChirp) } }
- 48. CopyrightPrismTech,2014 ChirpOut App // Definethe topic — assuming the topic is registered with the server…! topic = new dds.Topic(0, 'Chirp', ‘com.ChirpIt’)! // Define the data writer Qos! dwQos = new dds.DataWriterQos(! dds.Reliability.Reliable, ! dds.Durability.TranslientLocal,! dds.Partition(userPartition))! ! // Use an Option monad to maintain the writer! chirpWriter = z_.None! ! // Create the writer when the runtime is connected! runtime.onconnect(() ->! dw = new dds.DataWriter(runtime, topic, dwQos)! chirpWriter = z_.Some(dw)! )! ! // …! // Write chirps! chirp = …! chirpWriter.map((dw) -> dw.write(chirp))!
- 49. CopyrightPrismTech,2014 ChirpIn App // Definethe topic — assuming the topic is registered with the server…! topic = new dds.Topic(0, 'Chirp', ‘com.ChirpIt’)! // Define the data reader Qos! drQos = new dds.DataWriterQos(! dds.Reliability.Reliable, ! dds.Durability.TranslientLocal,! dds.Partition(followedList))! ! // Use an Option monad to maintain the reader! chirpReader = z_.None! ! // Create the writer when the runtime is connected! runtime.onconnect(() ->! dr = new dds.DataReader(runtime, topic, dwQos)! // Attach a listener! dr.addListener((chirp) -> diplayChirp(chirp))! chirpReader = z_.Some(dr)! )
- 50. CopyrightPrismTech,2014 ChirpIt Thus Far… datacentre chirp emotions stats chirp emotions stats chirp emotions stats chirp emotions stats
- 51.
- 52. CopyrightPrismTech,2014 $ ./chirpOut @wolverin chirp>>@magneto will win!$ ./chirpOut @magneto @magneto>> @wolverine chirping while asleep… again! Download the package zip or tar.gz package available at: - https://dl.dropboxusercontent.com/u/19238968/webcast/2014/10/chirpit.zip - https://dl.dropboxusercontent.com/u/19238968/webcast/2014/10/chirpit.tar.gz Extract the package somewhere in your file-system and open a terminal at that location and run the app as shown below ChirpIt Demo $ ./chirpIn @magneto @wolverinse @mystique @wolverin>> @magneto will win! @magneto>> @wolverine chirping while asleep… again!
- 53. CopyrightPrismTech,2014 In Part IIwe extend our implementation to manage application configuration storage and analytics To do so, we will need to introduce additional features provided by the DDS and the Vortex Platform, such as Durability and Integration with Big-Data Stores What’s Next
- 54. CopyrightPrismTech,2014 In this presentationwe have performed the first step toward implementing a micro-blogging platform The combination of Vortex Device and Vortex Cloud made it very easy to create and deploy our Internet-Scale, multi-device micro-blogging platform Some of the DDS QoS, such as Durability have already shown how historical data, i.e. chirps made while we were offline — can be provided by the platform The Part II will illustrate how the complete application fits together and what is the role of Vortex in the Lambda Architecture Summary
- 55.