The past few years have witnessed a tremendous increase in the amount of real-time data that applications in domains, such as, web analytics, social media, automated trading, smart cities, smart grids, etc., have to deal with. The challenges faced by these applications, commonly called Big Data Applications, are manifold as the staggering growth in volumes is complicating the collection, storage, analysis and distribution of data.
In this Webcast we focus our attention on the challenges tied to the collection and distribution of the large volumes of data characteristic of Big Data applications – the first and last stage on the pipeline shown in the Figure below – and partly on its storage.
Big data applications have to be capable of collecting as well as distributing massive amounts of data, much of which needs to be timely processed. As a result these applications need to optimally exploit networking as well as computing resources. Some of all of this data also need to be saved on Big Data stores for further analysis, thus an effective liaison has to be created between the data distribution technology and the storage technology.
This webcast explain the problems that OpenSplice DDS can solve for Big Data applications, it will introduce some specific extensions designed for dealing with throughputs of several millions of messages per second, and finally will show how OpenSplice DDS can be integrated with Big Data Stores, such as Cassandra and HBase.