Communication over the kinds of Data-Links used for unmanned vehicles presents important challenges dues to the low bandwidth, intermittent, and lower reliability of these links. Classic network protocols such as TCP do not operate well in this environment forcing application developers to implement their own reliability and session management. This presentation describes he issues and alternatives.
AESA Airborne Radar Theory and Operations Technical Training Course SamplerJim Jenkins
The revolutionary active electronically scanned array (AESA) Radar provides huge gains in performance and all the front line fighters in the world from the Americans (F35, F22, F18, F15, F16) to the Europeans, Russians and Chinese already have one or soon will. This four day seminar, which took 10,000 man hours to produce, is a comprehensive treatment on the latest systems engineering technology required to design the modes for an AESA to capitalize on the systems inherent multi role, wide bandwidth, fast beam switching, and high power capabilities. Steve Jobs once said “You must provide the tools to let people become their best”, and this seminar will include two indispensable tools for the AESA engineer. 1) A newly written 400+ page electronic book with interactive calculations and simulations on the more complicated seminar subjects like STAP and Automatic Target Recognition. 2) A professionally designed spread sheet (with software) for designing, capturing and predicting the detection performance of the AESA modes including the challenging Alert-Confirm waveform.
Tutorial on Wireless Communications and Networking with Drones and Unmanned A...saadwalid
A comprehensive tutorial on how drones and unmanned aerial vehicles (UAVs) can be used for wireless communications and networking purpose. The tutorial studies all aspects of drones from performance analysis to deployment, resource management, and security.
Sonar based obstacle avoidance for UAVsgaurav dhir
Basic Obstacle Avoidance Capability using SONAR sensor was incorporated within an Unmanned Aerial Vehicle. The SONAR sensor was mounted on Raspberry Pi and the system was interfaced with the Pixhawk Flight Controller. MAVLINK protocol was used to ensure communication between Pixhawk and RPI allowing Pixhawk to take necessary control action.
AESA Airborne Radar Theory and Operations Technical Training Course SamplerJim Jenkins
The revolutionary active electronically scanned array (AESA) Radar provides huge gains in performance and all the front line fighters in the world from the Americans (F35, F22, F18, F15, F16) to the Europeans, Russians and Chinese already have one or soon will. This four day seminar, which took 10,000 man hours to produce, is a comprehensive treatment on the latest systems engineering technology required to design the modes for an AESA to capitalize on the systems inherent multi role, wide bandwidth, fast beam switching, and high power capabilities. Steve Jobs once said “You must provide the tools to let people become their best”, and this seminar will include two indispensable tools for the AESA engineer. 1) A newly written 400+ page electronic book with interactive calculations and simulations on the more complicated seminar subjects like STAP and Automatic Target Recognition. 2) A professionally designed spread sheet (with software) for designing, capturing and predicting the detection performance of the AESA modes including the challenging Alert-Confirm waveform.
Tutorial on Wireless Communications and Networking with Drones and Unmanned A...saadwalid
A comprehensive tutorial on how drones and unmanned aerial vehicles (UAVs) can be used for wireless communications and networking purpose. The tutorial studies all aspects of drones from performance analysis to deployment, resource management, and security.
Sonar based obstacle avoidance for UAVsgaurav dhir
Basic Obstacle Avoidance Capability using SONAR sensor was incorporated within an Unmanned Aerial Vehicle. The SONAR sensor was mounted on Raspberry Pi and the system was interfaced with the Pixhawk Flight Controller. MAVLINK protocol was used to ensure communication between Pixhawk and RPI allowing Pixhawk to take necessary control action.
An unmanned aerial vehicle, commonly known as a drone, is an aircraft without any human pilot, crew, or passengers on board. UAVs are a component of an unmanned aircraft system, which includes adding a ground-based controller and a system of communications with the UAV. UAV's can be used in wireless communications.
Radars are very complex electronic and electromagnetic systems. Often they are
complex mechanical systems as well. Radar systems are composed of many different
subsystems, which themselves are composed of many different components. There is a great
diversity in the design of radar systems based on purpose, but the fundamental operation and
main set of subsystems is the same.
Counter-UAS: Legal Challenges and Solutions for Research and Development (Jac...DroneSec
Jacob Tewes (Kutak Rock)
Talk Recording: https://www.youtube.com/watch?v=JUQh6DS51Uw
The Global Drone Security Network (GDSN) is the only event of its kind focusing on Cyber-UAV security, Drone Threat Intelligence, Counter-UAS, and UTM security. Watch the full recording here: https://www.youtube.com/watch?v=vZ6sRr65cSk
Speaker: https://www.linkedin.com/in/jacob-tewes-b20b017/
DroneSec is a cyber-uav security and threat intelligence company who hosted this second series of the GDSN community event.
https://dronesec.com/
In order for the Internet of Things to function properly, a reliable wireless technology needs to be available. Radio frequencies are some of the most popular and effective means for IoT communication.
High performance browser networking ch1,2,3Seung-Bum Lee
Presentation material including summary of "High Performance Browser Networking" by Ilya Grigorik. This book includes very good summary of computer network not only for internet browsing but also multimedia streaming.
An unmanned aerial vehicle, commonly known as a drone, is an aircraft without any human pilot, crew, or passengers on board. UAVs are a component of an unmanned aircraft system, which includes adding a ground-based controller and a system of communications with the UAV. UAV's can be used in wireless communications.
Radars are very complex electronic and electromagnetic systems. Often they are
complex mechanical systems as well. Radar systems are composed of many different
subsystems, which themselves are composed of many different components. There is a great
diversity in the design of radar systems based on purpose, but the fundamental operation and
main set of subsystems is the same.
Counter-UAS: Legal Challenges and Solutions for Research and Development (Jac...DroneSec
Jacob Tewes (Kutak Rock)
Talk Recording: https://www.youtube.com/watch?v=JUQh6DS51Uw
The Global Drone Security Network (GDSN) is the only event of its kind focusing on Cyber-UAV security, Drone Threat Intelligence, Counter-UAS, and UTM security. Watch the full recording here: https://www.youtube.com/watch?v=vZ6sRr65cSk
Speaker: https://www.linkedin.com/in/jacob-tewes-b20b017/
DroneSec is a cyber-uav security and threat intelligence company who hosted this second series of the GDSN community event.
https://dronesec.com/
In order for the Internet of Things to function properly, a reliable wireless technology needs to be available. Radio frequencies are some of the most popular and effective means for IoT communication.
High performance browser networking ch1,2,3Seung-Bum Lee
Presentation material including summary of "High Performance Browser Networking" by Ilya Grigorik. This book includes very good summary of computer network not only for internet browsing but also multimedia streaming.
Master Class : TCP/IP Mechanics from Scratch to ExpertAbhishek Sagar
This is Master Class course on TCP/IP protocol - Transmission Control Protocol. Since it is Master Class course, this course discusses the internal design and functioning of complex transport layer protocol - TCP.
Almost all traffic on internet today is transported by TCP protocol. TCP, as where it stands today, mature and solid, is the result of over 25 yrs of research by network gurus. TCP is complicated and difficult to understand, therefore i have paid utmost attention to present the concept in most simplest way as possible without any loss of information.
Insights into the performance and configuration of TCP in Automotive Ethernet...RealTime-at-Work (RTaW)
The idea of using TCP in cars has been around for some time, as the first specification of Autosar TCP/IP stack dates back from early 2013. However, TCP has not been popular yet in cars and there has not been much published works on using TCP for in-vehicle communications so far.
TCP – the Transmission Control Protocol – provides connection-oriented reliable transmission between network applications. TCP is the cornerstone of the Internet – a hugely successful protocol over the last 40 years – if it is certainly a fine piece of engineering but it is definitely a complex one.
The question we explore in this study is what can we expect from TCP for on-board in-vehicle communication in terms of latencies & throughput and how to best configure TCP in a context for which
it has not been conceived. In particular, we will show that TCP configuration on the ECU sides should consider the amount of memory available in the switches and that traffic shaping policy, as available in TSN, can provide a nice performance boost for TCP communication.
A throughput analysis of tcp in adhoc networkscsandit
Transmission Control Protocol (TCP) is a connection oriented end-end reliable byte stream
transport layer protocol. It is widely used in the Internet.TCP is fine tuned to perform well in
wired networks. However the performance degrades in mobile ad hoc networks. This is due to
the characteristics specific to wireless networks, such as signal fading, mobility, unavailability
of routes. This leads to loss of packets which may arise either from congestion or due to other
non-congestion events. However TCP assumes every loss as loss due to congestion and invokes
the congestion control procedures. TCP reduces congestion window in response, causing unnecessary
degradation in throughput. In mobile ad hoc networks multi-hop path forwarding further
worsens the packet loss and throughput. To understand the TCP behavior and improve the
TCP performance over mobile ad hoc networks considerable research has been carried out. As
the research is still active in this area a comprehensive and in-depth study on the TCP throughput
and the various parameters that degrade the performance of TCP have been analyzed. The
analysis is done using simulations in Qualnet 5.0
DDS Security Version 1.2 was adopted in 2024. This revision strengthens support for long runnings systems adding new cryptographic algorithms, certificate revocation, and hardness against DoS attacks.
From its first use case that enabled distributed communications for US Navy ships to the autonomous systems of today, the DDS family of standards has enabled new generations of applications to run reliably, rapidly and securely, regardless of distance or scale.
To commemorate the 20th year milestone, the DDS Foundation is creating presentations that highlight the 14 specifications in the DDS standard, along with selected real-world use cases.
This presentation introduces some of the original use-cases and experiments, along with a brief history of the Standards.
A recorded video of the presentation is available at this URL
https://www.brighttalk.com/webcast/12231/602966
Introduction to DDS: Context, Information Model, Security, and Applications.Gerardo Pardo-Castellote
Introduction to the Data-Distribution Service (DDS): Context and Applications.
This 50 minute presentation summarizes the main features of DDS including the information model, the type system, and security as well as how typical applications use DDS.
It was presented at the Canadian Government Information Day in Ottawa on September 2018.
There is also a video of this presentation at https://www.youtube.com/watch?v=6iICap5G7rw.
This Object Management Group (OMG) RFP solicits submissions identifying and defining mechanisms to achieve integration between DDS infrastructures and TSN networks. The goal is to provide all artifacts needed to support the design, deployment and execution of DDS systems over TSN networks.
The DDS-TSN integration specification sought shall realize the following functionality:
● Define mechanisms that provide the information required for TSN-enabled networks to calculate any network schedules needed to deploy a DDS system.
OMG RFP
● Identify those parts of the set of the IEEE TSN standards that are relevant for a DDS-TSN integration and indicate how the DDS aspects are mapped onto, or related to, the associated TSN aspects. Examples include TSN- standardized information models for calculating system-wide schedules and configuring network equipment.
● Identify and specify necessary extensions to the [DDSI-RTPS] and [DDS- SECURITY] specifications, if any, to allow DDS infrastructures to use TSN- enabled networks as their transport while maintaining interoperability between different DDS implementations.
● Identify and specify necessary extensions to the DDS and DDS- XML specification, if any, to allow declaration of TSN-specific properties or quality of service attributes.
A NEW ARCHITECTURE PROPOSAL TO INTEGRATE OPC UA, DDS & TSN.
Suppliers and end users need a complete solution to address the complexity of future industrial automation systems. These systems require:
• Interoperability to allow devices and independent software applications from multiple suppliers to work together seamlessly
• Extensibility to incorporate future large or intelligent systems
• Performance and flexibility to handle challenging deployments and use cases
• Robustness to guarantee continuity of operation despite partial failures
• Integrity and fine-grained security to protect against cyber attacks
• Widespread support for an industry standard
This document proposes a new technical architecture to build this future. The design combines the best of the OPC Unified Architecture (OPC UA), Data Distribution Service (DDS), and Time-Sensitive Networking (TSN) standards. It will connect the factory floor to the enterprise, sensors to cloud, and real-time devices to work cells. This proposal aims to define and standardize the architecture to unify the industry.
Technical overview of the DDS for Extremely Resource-Constrained Environments (DDS-XRCE) specification.
This specification was adopted by the OMG in March 2018.
Demonstrates interoperability of 5 independent products that implement the Data-Distribution Service (DDS) Security Standard
(https://www.omg.org/spec/DDS-SECURITY/).
Tests the following implementations: RTI Connext DDS, Twin Oaks Computing CoreDX DDS, Kongsberg InterComm DDS, ADLink Vortex DDS Cafe, and Object Computing Inc OpenDDS.
This demonstration was performed at the OMG Meeting held in Reston, VA, USA in March 2018
Applying MBSE to the Industrial IoT: Using SysML with Connext DDS and SimulinkGerardo Pardo-Castellote
The benefits of Model-Based Systems Engineering (MBSE) and SysML are well established. As a result, users want to apply MBSE to larger and more complex Industrial IoT applications.
Industrial IoT applications can be very challenging: They are distributed. They deploy components across nodes spanning from small Devices to Edge computers to the Cloud. They often need mathematically-complex software. Moreover, they have strict requirements in terms of performance, robustness, and security.
SysML can model requirements, system components, behavior, interactions, and more. However, SysML does not provide a robust way to connect components running across different computers, especially when the security and quality of service of individual data-flows matter. SysML also does not provide all the tools needed to model and generate the (mathematical) code for complex dynamic systems.
A new “DDS + Simulink” MagicDraw SysML plugin has been developed to addresses these needs. It brings to MagicDraw users the capabilities of Connext DDS from RTI and Simulink from Mathworks:
The OMG Data-Distribution Service (DDS) is a secure and Qos-aware connectivity “databus”. DDS is considered the core connectivity framework for Software Integration and Autonomy by the Industrial Internet Consortium. Connext DDS is the leading implementation of the DDS standard, proven in 1000s of critical deployments.
Simulink is a tool for modeling and implementing the code needed for complex dynamic systems. It is widely deployed in many application domains including Automotive, Robotics, and Control Systems.
The new MagicDraw plugin defines a “DDS profile” for SysML that can model a distributed application connected using the DDS databus. The plugin can also generate the artifacts that configure the DDS databus (Topics, Data Types, Qos, etc.) and the adapters to Simulink and native code (e.g. C++ or Java).
By integrating three best-of class technologies: SysML, DDS and Simulink it is now possible to do MBSE for a wide range of Industrial IoT applications.
One of the most important challenges that system designers and system integrators face when deploying complex Industrial Internet of Things (IoT) systems is the integration of different connectivity solutions and standards. At RTI, we are constantly working to accelerate the Industrial IoT revolution. Over the past few years, we have developed standard connectivity gateways to ensure that DDS systems can easily integrate with other core connectivity frameworks.
This year, we developed a standard OPC UA/DDS Gateway, a bridge between two of the most well-known Industrial IoT connectivity frameworks. We are excited to announce that the gateway was just adopted by the Object Management Group (OMG).
In this webinar, we will dive deeper into the importance of choosing a baseline core connectivity standard for the Industrial IoT and how to ensure all system components are fully integrated. Attendees will also learn:
How the OPC UA/DDS Gateway specification was developed and how it works
How to leverage the Gateway to enable DDS and OPC UA applications to interoperate transparently
About the first standard connectivity gateway released with RTI Web Integration Service in Connext DDS 5.3
Gateways are a critical component of system interoperability and we will keep working to help companies accelerate Industrial IoT adoption.
This is the Beta 1 version of the OPC UA / DDS Gateway specification released by the Object Management Group in March 2018.
This specification defines a standard, vendor-independent, configurable gateway that enables interoperability and information exchange between systems that use DDS and systems that use OPC UA.
Data Distribution Service (DDS) is a family of standards from the Object Management Group (OMG) that provide connectivity, interoperability, and portability for Industrial Internet, cyber-physical, and mission-critical applications.
The DDS connectivity standards cover Publish-Subscribe (DDS), Service Invocation (DDS-RPC), Interoperability (DDS-RTPS), Information Modeling (DDS-XTYPES), Security (DDS-SECURITY), as well as programing APIs for C, C++, Java and other languages.
The OPC Unified Architecture (OPC UA) is an information exchange standard for Industrial Automation and related systems created by the OPC Foundation. The OPC UA standard provides an Addressing and Information Model for Data Access, Alarms, and Service invocation layered over multiple transport-level protocols such as Binary TCP and Web-Services.
DDS and OPC UA exhibit significant deployment similarities:
• Both enable independently developed applications to interoperate even when those applications come from different vendors, use different programming languages, or run on different platforms and operating systems.
• Both have significant traction within Industrial Automation systems.
• Both define standard protocols built on top of the TCP/ UDP/IP Internet stacks.
The two technologies may coexist within the same application domains; however, while there are solutions that bridge between DDS and OPC UA, these are based on custom mappings and cannot be relied to work across vendors and products.
This is the DDS-XRCE 1.0 Beta specification adopted by the OMG March 2018.
The purpose of DDS-XRCE is to enable resource-constrained devices to participate in DDS communication, while at the same time allowing those devices to be disconnected for long periods of time but still be discoverable by other DDS applications.
DDS-XRCE defines a wire protocol, the DDS-XRCE protocol, to be used between an XRCE Client and XRCE Agent. The XRCE Agent is a DDS Participant in the DDS Global Data Space. The DDS-XRCE protocol allows the client to use the XRCE Agent as a proxy in order to produce and consume data in the DDS Global Data Space.
Demonstrates interoperability of 5 independent products that implement the Data-Distribution Service (DDS) Security Standard
(https://www.omg.org/spec/DDS-SECURITY/).
Tests the following implementations: RTI Connext DDS, Twin Oaks Computing CoreDX DDS, Kongsberg InterComm DDS, ADLink Vortex DDS Cafe, and Object Computing Inc OpenDDS.
Demonstrates interoperability of 3 independent products that implement the Data-Distribution Service (DDS) Security Standard
(https://www.omg.org/spec/DDS-SECURITY/).
Tests the following implementations: RTI Connext DDS, Twin Oaks Computing CoreDX DDS, and Kongsberg InterComm DDS.
This specification provides the following additional facilities to DDS [DDS] implementations and users:
* Type System. The specification defines a model of the data types that can be used for DDS Topics. The type system is formally defined using UML. The Type System is de- fined in section 7.2 and its subsections. The structural model of this system is defined in the Type System Model in section 7.2.2. The framework under which types can be modi- fied over time is summarized in section 7.2.3, “Type Extensibility and Mutability.” The concrete rules under which the concepts from 7.2.2 and 7.2.3 come together to define compatibility in the face of such modifications are defined in section 7.2.4, “Type Com- patibility.”
* Type Representations. The specification defines the ways in which types described by the Type System may be externalized such that they can be stored in a file or communi- cated over a network. The specification adds additional Type Representations beyond the
DDS-XTypes version 1.2 1
one (IDL [IDL41]) already implied by the DDS specification. Several Type Representa- tions are specified in the subsections of section 7.3. These include IDL (7.3.1), XML (7.3.2), XML Schema (XSD) (7.3.3), and TypeObject (7.3.4).
* Data Representation. The specification defines multiple ways in which objects of the types defined by the Type System may be externalized such that they can be stored in a file or communicated over a network. (This is also commonly referred as “data serializa- tion” or “data marshaling.”) The specification extends and generalizes the mechanisms already defined by the DDS Interoperability specification [RTPS]. The specification in- cludes Data Representations that support data type evolution, that is, allow a data type to change in certain well-defined ways without breaking communication. Two Data Repre- sentations are specified in the subsections of section 7.4. These are Extended CDR (7.4.1, 7.4.2, and 7.4.3) and XML (7.4.4).
* Language Binding. The specification defines multiple ways in which applications can access the state of objects defined by the Type System. The submission extends and gen- eralizes the mechanism currently implied by the DDS specification (“Plain Language Binding”) and adds a Dynamic Language Binding that allows application to access data without compile-time knowledge of its type. The specification also defines an API to de- fine and manipulate data types programmatically. Two Language Bindings are specified in the subsections of section 7.5. These are the Plain Language Binding and the Dynamic Language Binding.
This specification defines the Security Model and Service Plugin Interface (SPI) architecture for compliant DDS implementations. The DDS Security Model is enforced by the invocation of these SPIs by the DDS implementation. This specification also defines a set of builtin implementations of these SPIs.
* Authentication Service Plugin. Provides the means to verify the identity of the application and/or user that invokes operations on DDS. Includes facilities to perform mutual authentication between participants and establish a shared secret.
* AccessControl Service Plugin. Provides the means to enforce policy decisions on what DDS related operations an authenticated user can perform. For example, which domains it can join, which Topics it can publish or subscribe to, etc.
* Cryptographic Service Plugin. Implements (or interfaces with libraries that implement) all cryptographic operations including encryption, decryption, hashing, digital signatures, etc. This includes the means to derive keys from a shared secret.
* Logging Service Plugin. Supports auditing of all DDS security-relevant events Data Tagging Service Plugin. Provides a way to add tags to data samples.
This document specifies the OMG Interface Definition Language (IDL). IDL is a descriptive language used to define data types and interfaces in a way that is independent of the programming language or operating system/processor platform.
The IDL specifies only the syntax used to define the data types and interfaces. It is normally used in connection with other specifications that further define how these types/interfaces are utilized in specific contexts and platforms.
This the the formal version 1.0 of the DDS Security specification released September 2016. OMG document number formal/2016-08-01.
DDS-Security defines the Security Model and Service Plugin Interface (SPI) architecture for compliant DDS implementations.
The DDS Security Model is enforced by the invocation of these SPIs by the DDS implementation. This specification also defines a set of builtin implementations of these SPIs.
* The specified builtin SPI implementations enable out-of-the box security and interoperability between compliant DDS applications.
* The use of SPIs allows DDS users to customize the behavior and technologies that the DDS implementations use for Information Assurance, specifically customization of Authentication, Access Control, Encryption, Message Authentication, Digital Signing, Logging and Data Tagging.
This specification is a response to the OMG RFP "eXtremely Resource Constrained Environments DDS (DDS- XRCE)"
It defines a DDS-XRCE Service based on a client-server protocol between a resource constrained, low-powered device (client) and an Agent (the server) that enables the device to communicate with a DDS network and publish and subscribe to topics in a DDS domain. The specifications purpose and scope is to ensure that applications based on different vendor’ implementations of the DDS-XRCE Service are compatible and interoperable.
This is the Joint submission by RTI, TwinOaks, and eProsima. Updated September 2017, OMG document number mars/2017-09-18.
UiPath Test Automation using UiPath Test Suite series, part 4DianaGray10
Welcome to UiPath Test Automation using UiPath Test Suite series part 4. In this session, we will cover Test Manager overview along with SAP heatmap.
The UiPath Test Manager overview with SAP heatmap webinar offers a concise yet comprehensive exploration of the role of a Test Manager within SAP environments, coupled with the utilization of heatmaps for effective testing strategies.
Participants will gain insights into the responsibilities, challenges, and best practices associated with test management in SAP projects. Additionally, the webinar delves into the significance of heatmaps as a visual aid for identifying testing priorities, areas of risk, and resource allocation within SAP landscapes. Through this session, attendees can expect to enhance their understanding of test management principles while learning practical approaches to optimize testing processes in SAP environments using heatmap visualization techniques
What will you get from this session?
1. Insights into SAP testing best practices
2. Heatmap utilization for testing
3. Optimization of testing processes
4. Demo
Topics covered:
Execution from the test manager
Orchestrator execution result
Defect reporting
SAP heatmap example with demo
Speaker:
Deepak Rai, Automation Practice Lead, Boundaryless Group and UiPath MVP
Essentials of Automations: Optimizing FME Workflows with ParametersSafe Software
Are you looking to streamline your workflows and boost your projects’ efficiency? Do you find yourself searching for ways to add flexibility and control over your FME workflows? If so, you’re in the right place.
Join us for an insightful dive into the world of FME parameters, a critical element in optimizing workflow efficiency. This webinar marks the beginning of our three-part “Essentials of Automation” series. This first webinar is designed to equip you with the knowledge and skills to utilize parameters effectively: enhancing the flexibility, maintainability, and user control of your FME projects.
Here’s what you’ll gain:
- Essentials of FME Parameters: Understand the pivotal role of parameters, including Reader/Writer, Transformer, User, and FME Flow categories. Discover how they are the key to unlocking automation and optimization within your workflows.
- Practical Applications in FME Form: Delve into key user parameter types including choice, connections, and file URLs. Allow users to control how a workflow runs, making your workflows more reusable. Learn to import values and deliver the best user experience for your workflows while enhancing accuracy.
- Optimization Strategies in FME Flow: Explore the creation and strategic deployment of parameters in FME Flow, including the use of deployment and geometry parameters, to maximize workflow efficiency.
- Pro Tips for Success: Gain insights on parameterizing connections and leveraging new features like Conditional Visibility for clarity and simplicity.
We’ll wrap up with a glimpse into future webinars, followed by a Q&A session to address your specific questions surrounding this topic.
Don’t miss this opportunity to elevate your FME expertise and drive your projects to new heights of efficiency.
Key Trends Shaping the Future of Infrastructure.pdfCheryl Hung
Keynote at DIGIT West Expo, Glasgow on 29 May 2024.
Cheryl Hung, ochery.com
Sr Director, Infrastructure Ecosystem, Arm.
The key trends across hardware, cloud and open-source; exploring how these areas are likely to mature and develop over the short and long-term, and then considering how organisations can position themselves to adapt and thrive.
DevOps and Testing slides at DASA ConnectKari Kakkonen
My and Rik Marselis slides at 30.5.2024 DASA Connect conference. We discuss about what is testing, then what is agile testing and finally what is Testing in DevOps. Finally we had lovely workshop with the participants trying to find out different ways to think about quality and testing in different parts of the DevOps infinity loop.
Transcript: Selling digital books in 2024: Insights from industry leaders - T...BookNet Canada
The publishing industry has been selling digital audiobooks and ebooks for over a decade and has found its groove. What’s changed? What has stayed the same? Where do we go from here? Join a group of leading sales peers from across the industry for a conversation about the lessons learned since the popularization of digital books, best practices, digital book supply chain management, and more.
Link to video recording: https://bnctechforum.ca/sessions/selling-digital-books-in-2024-insights-from-industry-leaders/
Presented by BookNet Canada on May 28, 2024, with support from the Department of Canadian Heritage.
UiPath Test Automation using UiPath Test Suite series, part 3DianaGray10
Welcome to UiPath Test Automation using UiPath Test Suite series part 3. In this session, we will cover desktop automation along with UI automation.
Topics covered:
UI automation Introduction,
UI automation Sample
Desktop automation flow
Pradeep Chinnala, Senior Consultant Automation Developer @WonderBotz and UiPath MVP
Deepak Rai, Automation Practice Lead, Boundaryless Group and UiPath MVP
Neuro-symbolic is not enough, we need neuro-*semantic*Frank van Harmelen
Neuro-symbolic (NeSy) AI is on the rise. However, simply machine learning on just any symbolic structure is not sufficient to really harvest the gains of NeSy. These will only be gained when the symbolic structures have an actual semantics. I give an operational definition of semantics as “predictable inference”.
All of this illustrated with link prediction over knowledge graphs, but the argument is general.
GraphRAG is All You need? LLM & Knowledge GraphGuy Korland
Guy Korland, CEO and Co-founder of FalkorDB, will review two articles on the integration of language models with knowledge graphs.
1. Unifying Large Language Models and Knowledge Graphs: A Roadmap.
https://arxiv.org/abs/2306.08302
2. Microsoft Research's GraphRAG paper and a review paper on various uses of knowledge graphs:
https://www.microsoft.com/en-us/research/blog/graphrag-unlocking-llm-discovery-on-narrative-private-data/
Connector Corner: Automate dynamic content and events by pushing a buttonDianaGray10
Here is something new! In our next Connector Corner webinar, we will demonstrate how you can use a single workflow to:
Create a campaign using Mailchimp with merge tags/fields
Send an interactive Slack channel message (using buttons)
Have the message received by managers and peers along with a test email for review
But there’s more:
In a second workflow supporting the same use case, you’ll see:
Your campaign sent to target colleagues for approval
If the “Approve” button is clicked, a Jira/Zendesk ticket is created for the marketing design team
But—if the “Reject” button is pushed, colleagues will be alerted via Slack message
Join us to learn more about this new, human-in-the-loop capability, brought to you by Integration Service connectors.
And...
Speakers:
Akshay Agnihotri, Product Manager
Charlie Greenberg, Host
JMeter webinar - integration with InfluxDB and GrafanaRTTS
Watch this recorded webinar about real-time monitoring of application performance. See how to integrate Apache JMeter, the open-source leader in performance testing, with InfluxDB, the open-source time-series database, and Grafana, the open-source analytics and visualization application.
In this webinar, we will review the benefits of leveraging InfluxDB and Grafana when executing load tests and demonstrate how these tools are used to visualize performance metrics.
Length: 30 minutes
Session Overview
-------------------------------------------
During this webinar, we will cover the following topics while demonstrating the integrations of JMeter, InfluxDB and Grafana:
- What out-of-the-box solutions are available for real-time monitoring JMeter tests?
- What are the benefits of integrating InfluxDB and Grafana into the load testing stack?
- Which features are provided by Grafana?
- Demonstration of InfluxDB and Grafana using a practice web application
To view the webinar recording, go to:
https://www.rttsweb.com/jmeter-integration-webinar
State of ICS and IoT Cyber Threat Landscape Report 2024 previewPrayukth K V
The IoT and OT threat landscape report has been prepared by the Threat Research Team at Sectrio using data from Sectrio, cyber threat intelligence farming facilities spread across over 85 cities around the world. In addition, Sectrio also runs AI-based advanced threat and payload engagement facilities that serve as sinks to attract and engage sophisticated threat actors, and newer malware including new variants and latent threats that are at an earlier stage of development.
The latest edition of the OT/ICS and IoT security Threat Landscape Report 2024 also covers:
State of global ICS asset and network exposure
Sectoral targets and attacks as well as the cost of ransom
Global APT activity, AI usage, actor and tactic profiles, and implications
Rise in volumes of AI-powered cyberattacks
Major cyber events in 2024
Malware and malicious payload trends
Cyberattack types and targets
Vulnerability exploit attempts on CVEs
Attacks on counties – USA
Expansion of bot farms – how, where, and why
In-depth analysis of the cyber threat landscape across North America, South America, Europe, APAC, and the Middle East
Why are attacks on smart factories rising?
Cyber risk predictions
Axis of attacks – Europe
Systemic attacks in the Middle East
Download the full report from here:
https://sectrio.com/resources/ot-threat-landscape-reports/sectrio-releases-ot-ics-and-iot-security-threat-landscape-report-2024/
3. UAVs part of larger integrated network
Vehicle LAN
Data Link
Ground Station
LAN
Avionics
Net Centric GIG
Tactical
Backbone
Real-Time
Ground Station
Backend
WAN
4. Characteristics of UAV Communications
In-Vehicle comm.
Data Link comm.
Ground Station comm.
Net-Centric Backbone comm.
5. Inside Vehicle Communications
Deeply-embedded, low-power
– Limited CPU speed
– General Purpose Processors and FPGAs
Memory constrained devices
– Limited RAM
– Flash filesystem or none
Dedicated IPC transports
– Back plane
Certification requirements
– DO-178B Operating Systems
Challenging
Environment to
operate on!
6. Data-Link Communications
Multiple traffic types:
– Sensor data streams
– Command & Control data
– Status, Intelligence, Mission, Supervisory
Different traffic requirements for each type:
– Urgency, Priority, Reliability, Volume
– Stealth operations
Challenging communications channel:
– Large latency, low throughput channel
– Lossy links
– Disconnections
– Asymmetric bandwidth (downlink vs uplink)
7. Data Link Types & Requirements
Low
Throughput
High
reliability
High integrity
Aggregate
Performance
for HRDL +
HCDL
High
Throughput
(streaming
data)
Moderate
Throughput
High Avail.
High Integrity
Reqs
C2 and
Status data
transfer in
emergency
Relay xfer:
High Altitude
Platform or
UAV
Sensor
Data
C2 Data
Status Data
(position
attitude)
Use
Back UpBeyond Line
of Sight
High
Capacity
(HCDL)
High
Reliability
(HRDL)
9. Ground Station Characteristics
Heterogeneous system
– RT storage & processing of sensor data
– Integration with display
– Integration with C2 / supervision systems
– Integration with net-centric back end
Multiple programming languages:
C/C++/Java/.NET
Multi-Platform: Linux/Windows/Embedded
Modular, reconfigurable
Varying assignments of Ground-Station to UAV
10. Ground Station Requirements
Be able to handle and adapt a variety of:
– CPUs, / Computer platforms
– Traffic flows
– Programming Languages
– Operating Systems
Provide a modular framework
– Support reconfiguration
– Support evolution, extensibility
– Support SOA tenets
Support operational use-cases
– Link fallback
– Multi-station handoff
13. Outline
UAV Communication Requirements
Why TCP-based solutions do not work
Implementing your own data-link protocol
Using middleware (DDS) for the Data-Link
Conclusions
14. TCP-based solutions do not work
for the Data Link
TCP has fundamental problems in the Data-Link…
– Un-tunable timers & congestion control algorithm
– Bad behavior on lossy networks & networks with dropouts
– Bad behavior on large latency links
The consequences are:
– Protocol Problems
Head of line blocking
Brittle connection-oriented model
Byte-oriented. Lacks prioritization
Inflexible reliability model. Not stealth
– Performance problems
Slow connect
Low link utilization
TCP
15. TCP problem: Head-of-line blocking
TCP funnels all traffic over single reliable stream
A Byte cannot be delivered until all previous Bytes
have been received
– A lost packet will “block” all future traffic until that
packet is repaired
– A large message will “block” all future traffic until it is
completely delivered
– IMAGE: Broken Bicycle blocking race car
– IMAGE: Large Tractor blocking race car
TCP’s “stream-oriented” reliability model not
suitable for Data Link
TCP
16. TCP issues: Brittle connections
TCP relies in hard-coded timers to establish connections
– SYN messages bust be responded before timeout
– SYN timer is 3 secs with doubling exponential backoff: 3s, 6s, 12s,…
– Implementations give up after fixed number of attempts
– Large latencies (> 60 sec) cause every TCP connection attempt to fail
– Some TCP implementations fail sooner: e.g 9 sec for Windows
TCP is bad a detecting disconnections
– To detect connection liveliness must use KEEP_ALIVE option
System-wide timeout defaults to 2 hours
– Common solution is periodic application messaging
Detection time non-deterministic. Order of minutes
TCP connection failure is drastic
– All state is lost
– No knowledge of what messages were delivered or not
– Application must do their message framing, sequence numbering and
acknowledgment to enable to continue upon re-connect
TCP
17. TCP issue: Low bandwidth use
‘Perfect storm’ for TCP protocol:
– TCP slow start
Ramp-up time ~ RTT*log(BW)
– RTT – roundtrip time (2 x latency)
– BW -- bandwidth
– Insufficient TCP buffersize given large RTT
To utilize a given BW TCP needs buffersize ~ BW*RTT
For 10 Mbps and 500msec buffsize ~ 640KB!
Typical Available/Configured TCP buffsize far smaller!
– TCP congestion-control algorithm misinterprets packet loss as
a sign of congestion
End result: long ramp-up times, low and/or unstable
bandwidth use
TCP
18. Details: Insufficient buffersize
TCP flow control is based on a send "window size“
– Send Window determines how much data can be outstanding (i.e.,
unacknowledged) in the network.
In long-delay networks require large send-windows to hold
large amount of “in flight data” without blocking sender
– DataInTransit ~ bandwidth X delay
Operating Systems limit/hardcode window size.
– TCP standard limits window to 64 KB (in practice 32KB due to
signed arithmetic)
– Required windows are much larger:
RTT 0.8, BW 1.54 Mbps requires 154 KB
New "large-window“ TCP extension (TCP-LW) allows
windows up to 232KB
– But that makes the slow-start problem bigger…
TCP
19. Details: Congestion control
TCP congestion avoidance is bad for lossy or long latency links:
– Mistakenly interprets packet loss as congestion
– Excessively long ramp-up for new connections
RED (random early detection) gateways requires each gateway to monitor its own
queue length. When imminent congestion is detected the TCP sender is notified. By
dropping a packet earlier than it would normally, RED sends an implicit notification of
congestion. The sender is effectively notified by the timeout of this packet. The
principle behind the RED approach is that a few earlier-than-usual drops may help
avoid more packet drops later on. The TCP sender can then reduce its window
before serious congestion occurs.
In TCP Vegas the TCP sender predicts when congestion is about to occur and
reduces its transmission window before intermediate routers drop packets
– TCP can keep track of the minimum round trip time seen during a transfer and use the most
recently observed round trip time to compute the data queued in the network.
– TCP can also keep track of the throughput before and after the congestion window
changes to estimate the network congestion level.
– If estimates indicate that the number of packets queued in the network is rising, it reduces
the congestion window. As it observes the number decreasing it increases the congestion
window.
Although neither approach has been widely adopted, both hold promise for satellite
networks. As we mentioned earlier, TCP congestion control responds to congestion
slowly because of latency. If such congestion can be avoided before it happens, it is
a big win for high-speed and long-delay networks.
TCP
20. TCP issue: reliability & congestion control
TCP acknowledgment is non-selective & blunt
– If a segment is lost, TCP will retransmit all data starting
from the lost segment without regard to the successful
transmission of later segments.
TCP congestion control fooled by lossy networks
– TCP considers this lost segment as an indication of
congestion and reduce its window size in half
TCP
21. TCP issue: chatty reliability protocol
TCP reliability requires constant ACKs from
receiver
– Even if all messages are received…
ACK traffic consumers power and bandwidth
ACK traffic prevents stealth operations (can
reveal position of ACKer)
Other protocols (best efforts or NACK only) may
be better suited…
TCP
22. Summary TCP protocol problems
TCP is inflexible
TCP protocol not well suited for Data Link
– Low performance
– Incorrect behavior
NASA and others have tried to spearhead
efforts to modify TCP…
– Research on “delay tolerant” networks
– Research on TCP: HACK, SACK, Trunk protocols
These efforts remain in the research domain
TCP “one size fits all” Qos not suitable for Data Link
TCP
23. Outline
UAV Communication Requirements
Why TCP-based solutions do not work
Implementing your own data-link protocol
Using middleware (DDS) for the Data-Link
Conclusions
24. Implementing your own data-link protocol
Session management
Data stream management
Buffering
Traffic Prioritization/Shaping
Fragmentation / Reassembly
Reliability
Redundant links/failover
25. General Architecture
To solve the reliability, flow control, and
disconnection issues we need:
– Data buffers at both ends
– a reliable comm. protocol sends the data from the
send buffer to the receive buffer
Sender
Application
Receiver
Application
Reliability Protocol
Send Buffer Receive Buffer
26. General Architecture (2)
To avoid head-of-line blocking we need
– Separate buffers for each traffic type
– Separate reliable data streams for each traffic type,
each should have its own separate session
Sender
Application
Receiver
ApplicationEach traffic type has its own session
Send Buffer Receive Buffer
27. Reliable Protocol
At a minimum the reliability protocol must
– Identify each message with sessionId and a
sequence number
– Send periodic HearBeats announcing which
sequence numbers should have been received
– Accept ACKs to record the messages and clear
from send buffer
– Accept NACKs for sequence numbers and send the
requested repairs
29. Company Confidential
Confirmed Reliability (TCP Style)
Some packet loss
01
02
03
04
01
02
03
04, HB
01
02
X
ACK 1-2, NACK 3
05
06
07
08
05
06
07
08, HB
06
07
08
ACK 1-8
03
04
05
X
X
Packets 04 and 05 are received but the
protocol drops them because a prior
packet 03 is missing.
This wastes valuable bandwidth
30. Reliable Protocol (II)
For performance the protocol should
– Accept received messages out of order and cache
them on the receiver buffer while the missing
messages are repaired
– Send selective NACKs (SACKs) for just the
sequence numbers that are missed
To handle large sensor data (e.g images)
– Fragment & re-assemble large messages
– Handle reliability on message fragments as well
To handle small updates
– Bundle small updates into batches
– Flush batches based on max delay or packet size
32. Company Confidential
Confirmed Reliability (Reader Cache + SACK)
Some packet loss
01
02
03
04
01
02
03
04, HB
01
02
X
04ACK 1-2, SACK 3
05
06
07
08
05
06
07
08, HB
05
06
07
08ACK 1-8
03
Packets 04 and 05 are received and
cached waiting for the repair of 03.
No bandwidth is wasted.
33. Reliable Protocol (III)
For performance on a wide variety of links the
protocol must
– Allow configuration of timers and buffer sizes
– Maintain liveliness of the link via KeepAlive
messages
– Allow sessions and buffers to survive link
disconnection
– Perform output shaping with rate limits
– Support prioritization between sessions/traffic types
– Support differential shaping for each traffic type
– …
34. Redundancy and Failover
Data-Link may deliver duplicate packets
Data might arrive from redundant transports
Failover requires multiple sources of the same
information
How does protocol identify/filter these duplicates?
– Needs VirtualSessionId identifying session
independent of data-link or source
– Reader queue must be 2-level. Second level
organized by VirtualSessionId filters-out duplicates
35. Stealth
Reliability should be tunable:
– Best-efforts mode. No ACK traffic
Sacrifices reliability
While ensures order & no duplicates
– A NACK-only limits backwards traffic
But requires smarter buffer management
– Full reliability. Both ACKs and NACKs
Ensures delivery to the receiving application
36. Example (best effort with packet loss)
01
02
03
04
01
02
03
04, HB
01
02
X
04
05
06
07
08
05
06
07
08, HB
05
06
07
08
Company Confidential
Packets 03 is permanently lost
Repair request would compromise
stealth.
Application notified of packet loss.
37. Stealth Reliability (no packet loss)
01
02
03
04
01
02
03
04, HB
01
02
03
04
05
06
07
08
05
06
07
08, HB
05
06
07
08
Stealth not compromised under
Normal operating conditions.
38. Stealth Reliability (some packet loss)
01
02
03
04
01
02
03
04, HB
01
02
X
04NACK 3
05
06
07
08
05
06
07
08, HB
05
06
07
08
03
Stealth minimally compromised
Only when some message is lost
39. Message Batching
write()
sender receiver
write()
sender
Send queue Receive queue
Send queue Receive queue
Without batching each
message is separately
sent. For small messages
protocol headers might be
bigger than payload
With batching messages
are held a little and
combined into larger
batches maximizing
throughout and minimizing
CPU
receiver
Transparent:
Receiver still
sees individual
messages
40. Reliability with Batching
Reliability must work even when messages are
batched
ACK or NACK of individual samples would
negate some of the benefits of batching…
=> Protocol must be batch aware so that it can
ACK/NACK complete batches!
B3
B2
B1
B3
B2
B1
ACK(B3), NACK(B2)
Repair B2
B3
B2
B1
write()
sender
receiver
41. Batching is hard but it pays!
RTI DDS 4.3b perftest results
0
100
200
300
400
500
600
700
800
900
1000
0 1000 2000 3000 4000 5000
Sample size (bytes)
Throughput(Mbps)
Linux Baseline
Linux 10Kb Batch
Intel Core2Duo Single-CPU Dual-Core 2.4GHz, 4MB cache
32-bit CentOS 5 (RHEL 5), 2GB memory, Intel E1000 NIC
42. Other considerations
Resource management:
– During disconnected operation buffers might fill or
overflow…
– Solution is smart caching:
Purge by age
Filter by frequency
Keep “one of each”
– requires additional insight onto the data
– Some object identifier (e.g. track Id)
Filter by content
44. Outline
UAV Communication Requirements
Why TCP-based solutions do not work
Implementing your own data-link protocol
Using middleware (DDS) for the Data-Link
Conclusions
45. Ethernet Wireless Radio Shared Memory cPCI 1553
Using a Network Middleware
Network middleware: A
library between the
operating system and the
application
It insulates application
from the raw network
Implements reliability,
caching, …
Hardware (e.g. Radio)
Network stack (e.g. IP)
Middleware
Application
Middleware
Application Application Application Application Application
46. Which middleware to use?
Standards based
Configurable via QoS
Not based on TCP
Manages Sessions/Fragmentation/Reliability…
Failover/handover supoort
Efficient use of bandwidth
Multi-platform
Embeddable, Certifiable…
Integration with net-centric back end
47. DDS mandated for data-distribution
DISR (formerly JTA)
– DoD Information Technology
Standards Registry
US Navy Open Architecture
FCS SOSCOE
– Future Combat System –
System of System Common
Operating Environment
SPAWAR NESI
– Net-centric Enterprise Solutions
for Interoperability
– Mandates DDS for Pub-Sub SOA
48. 48
European Air
Traffic Control
RETF (USA)
Train Communications
Tokyo Japan
Traffic Control
Boeing Army Future
Combat System
Boeing AWACS
program
US Navy, DD(X)
LCS, LPD-17
SeaSlice
and 13 other Navies
DDS Adoption
49. Insitu Unmanned Air Vehicle
“…we have seen a 30% increase in productivity based on not
having to handle data communication issues.” Gary Viviani,
VP of Engineering
Insitu is a recognized leader in the
exploding UAV space
The next generation of UAV’s
including the Scan Eagle and newer
platforms
Challenge is to have a successful
UAV mission which requires
impressive autonomy and reliable
ground control
DDS enables an information flow
that is much more orchestrated
and flexible allowing seamless
switch control between multiple
ground stations while
connecting reliably over
unreliable links
51. CLIP Mediator Bridge
Transportation
• Common Link Integration Processing (CLIP): U.S.
Air Force and Navy joint project to build Tactical
Data Link (TDL) aggregator
• Enables information exchange between platforms
with incompatible tactical data links
• Challenge: existing system had poor integration
with platform mission systems
• With Northrop Grumman, RTI helped architect,
design, develop & test mediator bridge between
platform systems and CLIP
– RTI Services built a ‘mediator’ bridge
between Air Force, Navy, NGC, B1, B52
– First NESI DDS Compliant Product
Defense
“Working with RTI has been
both effective and productive.”
– Jim Miller, CLIP Program Manager
55. AWACS Radar System Upgrade
Airborne control system for
surveillance, command & control and
battle management
Upgrading system to be open,
supportable, less expensive to
maintain and extend
DDS is standards-based, open and
extensible, reducing integration risk
DDS is a proven COTS solution,
reducing total cost of ownership over
in-house development
56. CAE SimXXI Flight Simulation
State-of-the-art full-flight simulator
from CAE
Challenge is communication between
subsystems (over IEEE 1394) with
low-latency data transfer
DDS chosen because it excels in real-
time performance and is simple to use
and integrate
58. Next-generation of the U.S. Navy
Aegis Weapon System
Challenge to share time-critical data
across highly distributed system
including radar, weapons, displays
and controls
Need to maximize future scalability
and flexibility
DDS provides real-time
communication infrastructure.
Standards-based & extensible for
future system enhancements
Lockheed Martin US Navy Aegis Open
Architecture Weapon System
60. Sample EU project using DDS
ESO Extremely Large Telescope (E-ELT)
– 43m diameter (see vehicles on picture!)
– 30.000 sensors send data on the bus
– RTI DDS used as middleware for critical data
communication and integration
INDRA i-TEC e-FDP ATM program
– European leader in Air Traffic Mgmt
applications
– ATM integration for UK, Spain and Germany
– RTI Used as integration solution for Flight Data
Management and Distribution
EADS Euro Hawk UAV program
– EADS selected RTI for European UAV program
– RTI is used as embedded middleware in UAV
versatile payload
61. Sample EU project using DDS
PLATH (Hamburg, Germany)
– Radio signal analysis experts
– Has decided to use RTI on a large scale for key
middleware services
Volkswagen R&D
– After thorough evaluation VW has selected RTI
as a middleware for their next generation
vehicular R&D platform,
– AUTOSAR, ECS, ECU context.
MBDA France & UK
– They have been using RTI for 2 years
– Vertical launch missile program « MOUV »
62. Sample EU project using DDS
BASE 10 RoboScout Technology Reference
System (TRS)
– BTSE is a German project focused company
specialized in the defense market within NATO. They
are experts in robotics integrating systems
engineering, system qualification, manufacturing and
long term support.
– Base 10 has been working with RTI for 1 year
– We delivered Quick-Start training and an architecture
study on how to implement RTI on the vehicular
platform data flows
– There are 5 different subsystems on the data bus and
communication link between RCC and RoboScout RS
– RTI is now implemented in the RCC (bottom left
picture)
– Next release of RoboScout will implement RTI in
vehicular platform and outside services (radio and
satellite data-links).
64. Dissecting Messaging Technologies
The alternatives:
Standards based:
– Web-Service/SOAP Based (WS-Eventing, WS-Notification…)
– JMS
– CORBA
– Real-Time Data-Distribution Service
Vendor-proprietary:
– ESBs
– IBM WebSphere MQ,
– TIBCO,
– 29West,
– Gigaspaces
Custom build
Architecture
Quality of Service
Performance
& Scalability
65. Best-of breed RT-Messaging: DDS
Data Distribution Service (DDS)
– High performance real-time data distribution
– Object Management Group (OMG)
DDS Standard API (v1.2)
– Specifies user-visible API
– Ensures application portability
– Adopted in June 2003, revised June 2005,2006
DDS Standard Wire Protocol (v 2.1)
– Real-Time Publish-Subscribe (RTPS)
– Ensures application interoperability
– Adopted in June 2006, revised July 2007, 2008
Real-time
Publish-Subscribe
(RTPS) Wire Protocol
DDS
Middleware
Data Distribution
Service API
Standards-based services for
application developers
Standard protocol for
interoperability
67. Message Quality of Service
Avoid a single source from
overwhelming the network. Prevent
large low-urgency data (e.g., file
downloads) from compromising the
performance of critical data (e.g.,
alarms and critical news updates).
Provide dedicated bandwidth to the most
critical data.
Control how much load and
bandwidth a particular
sender can inject into the
network. Control the peak
load, average load, and size
of a burst.
Flow
Control
Prioritize real-time flows like live audio
over traffic that may be buffered
(e.g., video replay).
Prioritize critical control information (e.g.,
live radar tracks) over non-time
critical information such as aircraft
schedule changes.
Specify the relative importance of
different messages and the
maximum acceptable delay
between the time the
message is sent and the time
it’s delivered to the reader(s).
Latency
Budget
Send live voice or video data. Send
sensor data (e.g., radar tracks),
traffic readings, CPU/network
statistics and readings.
Let the application decide
whether messages should be
confirmed and retried when
missed, or else sent as best
efforts.
Reliability
Example Use CasesPurposeQoS
68. Message Quality of Service
Avoid a single source from
overwhelming the network. Prevent
large low-urgency data (e.g., file
downloads) from compromising the
performance of critical data (e.g.,
alarms and critical news updates).
Provide dedicated bandwidth to the most
critical data.
Control how much load and
bandwidth a particular
sender can inject into the
network. Control the peak
load, average load, and size
of a burst.
Flow
Control
Prioritize real-time flows like live audio
over traffic that may be buffered
(e.g., video replay).
Prioritize critical control information (e.g.,
live radar tracks) over non-time
critical information such as aircraft
schedule changes.
Specify the relative importance of
different messages and the
maximum acceptable delay
between the time the
message is sent and the time
it’s delivered to the reader(s).
Latency
Budget
Send live voice or video data. Send
sensor data (e.g., radar tracks),
traffic readings, CPU/network
statistics and readings.
Let the application decide
whether messages should be
confirmed and retried when
missed, or else sent as best
efforts.
Reliability
Example Use CasesPurposeQoS
DDS JMS* (partial)
DDS
DDS
WS-* (partial)Proprietary
Proprietary
69. Message Quality of Service (Cont.)
Allow exploiting the differential service
capabilities of the network
infrastructure
Configure the network infrastructure to
prioritize messages ahead of others.
Controls the traffic class used for
the underlying network
transport.
Takes advantage of network
multicast infrastructure
Transport
Priority
Multicast
Prevent a rapidly changing source from
using a lot of resources and starving
other less-active sources.
Some applications may only be
interested in the last 100 events for
each server regardless of the time
interval when they occurred.
Control how many related
messages (e.g., successive
updates to a stock value or
successive readings of a
sensor) must be maintained
by the middleware and
delivered to readers.
History
Prevent data that loses value with age
(e.g., old stock values, old news, old
sensor readings) from using
valuable system resources, while
ensuring that needed historic
information is kept (e.g., transaction
records).
Control how long the data must
be kept by the middleware to
be delivered to readers.
Old data may be of little value
delivering it wastes
bandwidth and gets in the
way of the more recent data.
Lifespan
Example Use CasesPurposeQoS
70. Message Quality of Service (Cont.)
Allow exploiting the differential service
capabilities of the network
infrastructure
Configure the network infrastructure to
prioritize messages ahead of others.
Controls the traffic class used for
the underlying network
transport.
Takes advantage of network
multicast infrastructure
Transport
Priority
Multicast
Prevent a rapidly changing source from
using a lot of resources and starving
other less-active sources.
Some applications may only be
interested in the last 100 events for
each server regardless of the time
interval when they occurred.
Control how many related
messages (e.g., successive
updates to a stock value or
successive readings of a
sensor) must be maintained
by the middleware and
delivered to readers.
History
Prevent data that loses value with age
(e.g., old stock values, old news, old
sensor readings) from using
valuable system resources, while
ensuring that needed historic
information is kept (e.g., transaction
records).
Control how long the data must
be kept by the middleware to
be delivered to readers.
Old data may be of little value
delivering it wastes
bandwidth and gets in the
way of the more recent data.
Lifespan
Example Use CasesPurposeQoS
DDS JMS
DDS
DDS
Proprietary
71. Message Quality of Service (Cont.)
Allow consumers with slow CPU or
network (e.g. wireless)
Filter data at the source or in the
infrastructure. Avoid wasting CPU
and bandwidth delivering data that
is not of interest
Monitor aircraft in your airspace, alarms
in the immediate vicinity, stocks that
cross a threshold or in the industries
of interest…
Provide an application only the
data it needs
Filter messages based on
content as requested by the
consuming application
Content
Filtering
Prevent data that loses when application
crash
Allow short-living applications (e.g. cgi
scripts) to generate messages that
are received reliable even by
applications that join the network
later
Externalize message history so
that they survive beyond the
life of the application that
generates them
Deliver messages reliably in the
presence of application
failure and re-starts.
Persisten
ce
Example Use CasesPurposeQoS
72. Message Quality of Service (Cont.)
Allow consumers with slow CPU or
network (e.g. wireless)
Filter data at the source or in the
infrastructure. Avoid wasting CPU
and bandwidth delivering data that
is not of interest
Monitor aircraft in your airspace, alarms
in the immediate vicinity, stocks that
cross a threshold or in the industries
of interest…
Provide an application only the
data it needs
Filter messages based on
content as requested by the
consuming application
Content
Filtering
Prevent data that loses when application
crash
Allow short-living applications (e.g. cgi
scripts) to generate messages that
are received reliable even by
applications that join the network
later
Externalize message history so
that they survive beyond the
life of the application that
generates them
Deliver messages reliably in the
presence of application
failure and re-starts.
Persisten
ce
Example Use CasesPurposeQoS
DDS JMS
DDS
Proprietary
Proprietary
73. Non-real-time Soft real-time Hard real-time Extreme real-time
Java/RMIJava/JMS
CORBA
MPI
Java RTSJ (soft RT) RTSJ (hard RT)
Web Services
MessagingTechnologiesandStandardsMessagingTechnologiesandStandards
Data Distribution Service / DDS
RT CORBA
Adapted from NSWC-DD OA Documentation
Data Distribution Service spans a
very wide spectrum of application needs
74. Top reasons to use DDS
Flexibility and Power of the data-centric model
Performance & Scalability
Rich set of built-in services
Interoperability across platforms and Languages
Provides/integrates Pub-Sub into SOA
75. #1 DDS Data-Centric Model
Data WriterData Writer
Data WriterData Writer
Data ReaderData Reader
Data Reader
Data Reader
Data Writer
“Global Data Space” generalizes Subject-Based Addressing
– Data objects addressed by DomainId, Topic and Key
– Domains provide a level of isolation
– Topic groups homogeneous subjects (same data-type & meaning)
– Key is a generalization of subject
Key can be any set of fields, not limited to a “x.y.z …” formatted string
76. #1 DDS Data-Centric Model
Data WriterData Writer
Data WriterData Writer
Data ReaderData Reader
Data Reader
Data Reader
Data Writer
Data Object
“Global Data Space” generalizes Subject-Based Addressing
– Data objects addressed by DomainId, Topic and Key
– Domains provide a level of isolation
– Topic groups homogeneous subjects (same data-type & meaning)
– Key is a generalization of subject
Key can be any set of fields, not limited to a “x.y.z …” formatted string
77. #1 DDS Data-Centric Model
Data WriterData Writer
Data WriterData Writer
Data ReaderData Reader
Data Reader
Data Reader
Data Writer
Topic
“Global Data Space” generalizes Subject-Based Addressing
– Data objects addressed by DomainId, Topic and Key
– Domains provide a level of isolation
– Topic groups homogeneous subjects (same data-type & meaning)
– Key is a generalization of subject
Key can be any set of fields, not limited to a “x.y.z …” formatted string
78. #1 DDS Data-Centric Model
Data WriterData Writer
Data WriterData Writer
Data ReaderData Reader
Data Reader
Data Reader
Data Writer
Key (subject)
“Global Data Space” generalizes Subject-Based Addressing
– Data objects addressed by DomainId, Topic and Key
– Domains provide a level of isolation
– Topic groups homogeneous subjects (same data-type & meaning)
– Key is a generalization of subject
Key can be any set of fields, not limited to a “x.y.z …” formatted string
79. Company Confidential
Topic: “Market Data”
Subject Filter (for a Reader)
Field
Value
Symbol Type Exchange
Payload
* * NYSE *
Subject Filter (for a Reader)
SourceField
Value
Symbol Type Exchange Payload
REUTERS * EQ NYSE Volume > x, Ask < y
Payload Filter (for a Reader)
Topic: “Order Entry”
Topic: “Market Data”
Subscriptions: By Topic, Subject, Content
Symbol OrderKind Stop Limit
SourceField
Value
Symbol Type Exchange
Payload
* * * * *
Volume Bid Ask …
OrderNumber …
80. DDS Demo: Concepts
Topics
– Square, Circle, Triangle
– Attributes
Data types (schemas)
– Shape (color, x, y, size)
Color is instance Key
– Attributes
Shape & color used for key
QoS
– Deadline, Liveliness
– Reliability, Durability
– History, Partition
– OwnershipControl Area:
Allows selection of objects and QoS
Display Area:
Shows state of objects
Start demo
81. QoS: Quality of Service
TRANSPORT PRIORITYCONTENT FILTERS
PRESENTATIONLIFESPAN
DESTINATION ORDERENTITY FACTORY
LATENCY BUDGETDEADLINE
LIVELINESSTIME BASED FILTER
OWNERSHIP STRENGTHRELIABILITY
OWNERSHIPRESOURCE LIMITS
PARTITIONWRITER DATA LIFECYCLE
GROUP DATAREADER DATA LIFECYCLE
TOPIC DATAHISTORY (per subject)
USER DATADURABILITY
QoS PolicyQoS Policy
82. Tunable Reliability Protocol
Configurable AckNack reply
times to eliminate storms
Fully configurable to bound
latency and overhead
– Heartbeats, delays, buffer
sizes
Consumer /
Reader
Producer /
Writer
Reliable
•Guaranteed
Ordered Delivery
•“Best effort” also
supportedS7
S5
S6
S4
S3
S2
S7 S6 S5 S4 S3 S2 S1
S1
S7
S5
S6
S4
S3
S2
S1
Performance can be tracked
by senders and recipients
– Configurable high/low
watermark, Buffer full
Flexible handling of slow
recipients
– Dynamically remove slow
receivers
83. High-Throughput via Aggregation
Increases throughput by aggregating smaller
messages into larger network packets
User tunable
– # packets to aggregate for delivery
– Aggregate packet size
– Max elapsed time before data is sent
– Manual flush at any time
write()
Full or timeout
84. Demo: Quality of Service (QoS)
Topics
– Square, Circle, Triangle
– Attributes
Data types (schemas)
– Shape (color, x, y, size)
Color is instance Key
– Attributes
Shape & color used for key
QoS
– Deadline, Liveliness
– Reliability, Durability
– History, Partition
– Ownership
RTI DDS delivers
Writers and readers state
Their needs
Start demo
85. #2 Performance & Scalability
DDS was designed to support high performance
RTI DDS was developed to maximize performance and minimize
jitter
Advanced techniques employed:
– Pre-allocation of memory
Never allocate/free memory in the critical path
– Use dedicated threads per receive port
Minimize thread switching
Avoid expensing operating system calls (e.g. select())
– Maximize concurrency
Carefully design critical sections
Patented concurrent mutex-free thread-safe data structures
– Employ high-performance data-access APIs
Read data by array (no additional copies)
Scatter/gather APIs to access transport.
Buffer loaning for zero copy access
86. Latency – (Linear Scale)
DDS/GSOAP/JMS/Notification Service Comparison - Latency
0
500
1000
1500
2000
2500
4 8 16 32 64 128 256 512 1024 2048 4096 8192 16384
Message Size (bytes)
DDS JMS Notification Service
Message Length (samples)
Adapted from Vanderbilt presentation at July 2006 OMG Workshop on RT Systems
88. Performance: Looking under the hood…
Increases performance for large messages, resulting in
higher throughput and lower latency. Reduces
CPU consumption on both sender and receiver.
Makes performance scale better with message size.
Operating system network-stack
technology that allows an
application to put and get data from
the network buffers “by reference,”
without performing extra copy
operations.
Zero Copy
Decouples sender and receiver, providing more
predictable performance for the writer and
reducing latency jitter.
Allows multiple write operations to be performed
concurrently over multiple channels, batched, or
optimized in other ways.
Middleware technology that allows a write
operation to be processed by a
separate thread and not block the
application thread that performed
the write.
Asynchronous Writes
Enables multicast use for larger (greater than 64KB)
messages.
Prevents “Head of Line” blocking where a high-priority
message is queued behind a large message.
Reduces jitter. Provides better performance in less
reliable networks (wireless / WANs).
Middleware technology that breaks a large
message into smaller units, delivers
them separately, and then
reassembles them prior to
deliverance to the application.
Message
Fragmentation
Greatly increases the throughput for small messages.
Reduces bandwidth and processor utilization for small
messages.
Middleware technology that combines
multiple messages into a single unit.
Message
Batching
Provides the most efficient way to send messages to
multiple receivers.
Reduces bandwidth, reduces overhead on the sender,
and minimizes latency and jitter.
Internet technology that allows a single
UDP message to be delivered to
many receivers.
Multicast
Why It MattersDescriptionPerformance &
Scalability
Technique
89. Company Confidential
Performance:
RTI DDS Low Latency and Jitter
0
50
100
150
200
250
300
350
400
32 64 128 256 512 1024 2048 4096 8192
Maximum
99.99%
99%
Median
Minimum
Reliable, ordered delivery over
Gigabit Ethernet between 2.0 GHz Opteron processors
running 32-bit Red Hat Enterprise Linux 4.0
Message/Data Size (bytes)
Latency(microseconds)
Latency and Jitter on Unloaded Network without Message Batching
91. 563,498 556,896
535,883
365,760
0
100,000
200,000
300,000
400,000
500,000
600,000
1 subscriber 20 subscribers
(1 per CPU and NIC)
40 subscribers
(1 per core, 2 per NIC)
72 subscribers
(1 per core, 2-8 per NIC)
MessagesperSecond
Scalability:
RTI DDS Reliable Multicast Performance
200 Byte messages
GBit Ethernet
Single publishing thread
All data subscribed
No message loss –
throttled to slowest
subscriber
CentOS 5, 32-bit
CPUs
– 2.4 GHz Intel Core 2 Duo
E6600
– 2.4 GHz Intel Core 2 Quad
Q6600
– 2.33 GHz Intel Xeon
E5345
– 2.4 GHz AMD Opteron
8216
NICs
– Intel PRO/1000
– Broadcom NetXtreme II
Throughput with batching
92. 0
100
200
300
400
500
600
700
800
900
1,000
32 64 128 256 512 1024 2048 4096 8192 16384
Message Size (bytes)
MegabitsperSecond
Native C++
.NET (C#)
Java
Performance:
RTI DDS High Performance across all Languages
Windows XP Pro SP2
32-bit
Reliable multicast
Gigabit Ethernet
2.4 GHz Intel Core 2
Quad Q6600
Single Intel PRO/1000
NIC
Four producer and
consumer threads
Throughput: Megabits per Second with batching
93. #3 Powerful Services & Tools
– High-Availability
– Persistent Data
– Recording service
– Relational Database bridge
– Development & Monitoring Tools
94. DDS High Availability via Redundancy
Owner determined per subject
Only extant writer with highest strength can publish a subject (or
topic for non-keyed topics)
Automatic failover when highest strength writer:
– Loses liveliness
– Misses a deadline
– Stops writing the subject
Shared Ownership allows any writer to update the subject
Producer / Writer
strength=10
Topic T1
I1 I2
Producer / Writer
strength=5
Producer / Writer
strength=1
I1 Primary
I1 Backup
I2 Primary
I2 Backup
95. DDS Data Persistence
A standalone service that persists data outside of
the context of a DataWriter
Data
Writer
Global
Data Space
Data
Reader
Persistence
Service
Persistence
Service
Data
Reader
Data
Writer
Permanent
Storage
Permanent
Storage
Can be configured for:
• Redundancy
• Load balancing
Demo:
1. PersistenceService
2. ShapesDemo
3. Application failure
4. Application (ShapesDemo) re-start
5. Persistence Svc failure
6. Application re-start
Cleanup database
96. DDS Real-Time Recording Service
Applications:
– Future analysis and
debugging
– Post-mortem
– Compliance checking
– Replay for testing and
simulation purposes
Record high-rate data
arriving in real-time
Non-intrusive – multicast
reception
Demo:
1. Start RecorderService
2. Start ShapesDemo
3. See output files
4. Convert to: HTML XML CSV
5. View Data: HTML XML CSV
98. DDS Enables Event Processing
CEP: programmable engines used to transform “data” into “information”
CEP engines are programmed using a derivative of SQL
CEP engines save time: They can implement a lot of the application logic:
– Classification, Correlation, Aggregation, Filter, Cleansing, Pattern Detection, etc.
DDS is the perfect ‘data’ and ‘information’ pipe for CEP engines
– Use high-speed data streams (1,000-1,000,000 msg/sec)
– Require latency measured in sub-milliseconds
– Demand access to events from a heterogeneous systems
CEP Engine
Dashboards
Applications
Alerts
RTI Global
Data Space
Market Data
Trades
Low Latency Messages
99. Tools provide insight into a distributed
system
RTI Analyzer
– Understand connections
and data flow
– Tune QoS properties
without changing code
RTI Scope
– Capture and monitor packet
payloads
– Collect time histories of
Topic values
RTI Protocol Analyzer
– Sniff the wire and analyze
traffic
100. #4 Interoperability between platforms & languages
Data accessible to all interested applications:
– Data distribution (publishers and subscribers): DDS
– Data management (storage, retrieval, queries): SQL
– ESB Integration, Business process integration: WSDL
– Legacy Java Integration: JMS
DBMS
DBMSDBMS
Global Data Space
Distributed
Node
Distributed
Node
Distributed
Node
Distributed
Node
Distributed
Node
SQL JMS
DDS SQL
DDSWSDL
D T
101. DDS: Multi- Architecture Support
• Same API for all platforms
• Language Independence: C, C++, Java™, C#, .NET, ADA
• Enterprise and Embedded Support
VxWorks®, INTEGRITY®, LynxOS®
Linux, Solaris, Windows
• Prototype on any platform
Linux
RTI DDS
Windows
RTI DDS
Integrity
RTI DDS
VxWorks
RTI DDS
103. #5 Provides Real-Time Pub-Sub in SOA
Real-Time
Devices Fault
Tolerance
Auditing &
Recording
Tools &
Visualization
Database
Event
Processing
Real-Time Pub-Sub/Caching/Messaging
SOA &
Real-Time
Web Services
WS-DDS
104. Real-Time SOA Architecture/Implementation
RT Architecture/Technology
High Performance
Event-Driven/Publish-Subscribe
Small footprint
Quality of Service
Support for embedded
environments
Support for unreliable & low-
bandwidth networks
Traditional Enterprise
Low Performance
Client-Server
Centralized (Server-based)
TCP based
DDS Data Bus
105. Conclusions
Implementing your own Data-Link Protocol is HARD
The simplest, most flexible solution is to use middleware to handle the reliability,
caching, failover…
Middleware must have special features to support specialized needs of Data Link:
Robust to packet loss, disconnects, good use of bandwidth, etc.
DDS the best choice today
– Is a mature international Standard from OMG
Platform Neutral: Operating systems and Programming Languages
Deployed worldwide in Military systems and other Demanding real-time applications
– It is mandated by US DoD for Publish-Subscribe and data-distribution applications
– It is ideally suited to UAVs
Highly Tunable via Quality of Service (QoS)
Flexible reliability model overcomes TCP problems
Can accommodate unreliable & high-latency transports
Uses bandwidth Efficiently
Rich services (persistence, filtering, high-availability)