This document summarizes a presentation about data stream networks and how they help the IoT ecosystem. It defines a data stream network as a secure, global network for two-way communication and real-time data push. It discusses how data stream networks provide services like publish/subscribe messaging, global replication and reliability, presence detection, storage and playback, and access control. Finally, it outlines how data stream networks help solve IoT pain points like presence detection, scalability, bandwidth usage, and security by enabling real-time communication, analytics, message routing and more between mobile/IoT/web applications.
14-10242017_IoT_Precipitating-the-Cloud-Local-IoT-Framework-with-Intel-and-AW...Amazon Web Services
Learning Objectives:
- How to gather sensor data on the gateway
- How to setup an Amazon Greengrass core
- How to create business logic using Lambda with Greengrass and How to connect to AWS IoT
Ending the Tyranny of Expensive Security Tools: A New HopeMichele Chubirka
A long time ago, in a galaxy far far away, AV was invented. Then firewalls and IDS and SIEM and NAC and DLP and on and on.
With all these products, it seems like a career in information security is really more about managing tools than defeating a galactic empire of hackers and miscreants. But like the Rebel Alliance, you can take back your enterprise, because many of our existing monitoring systems and network devices also have security functionality. Moreover, there are many excellent open source applications that work just as well as commercial ones.
14-10242017_IoT_Precipitating-the-Cloud-Local-IoT-Framework-with-Intel-and-AW...Amazon Web Services
Learning Objectives:
- How to gather sensor data on the gateway
- How to setup an Amazon Greengrass core
- How to create business logic using Lambda with Greengrass and How to connect to AWS IoT
Ending the Tyranny of Expensive Security Tools: A New HopeMichele Chubirka
A long time ago, in a galaxy far far away, AV was invented. Then firewalls and IDS and SIEM and NAC and DLP and on and on.
With all these products, it seems like a career in information security is really more about managing tools than defeating a galactic empire of hackers and miscreants. But like the Rebel Alliance, you can take back your enterprise, because many of our existing monitoring systems and network devices also have security functionality. Moreover, there are many excellent open source applications that work just as well as commercial ones.
Incident handlers manage security incidents by understanding common attack techniques, vectors and tools as well as defending against and/or responding to such attacks when they occur. In this talk we will discuss modern attacks, techniques, how to defend & respond to those threats.
Getting started on IoT with AWS and NodeMCU for less than 5€Stylight
In this workshop we talk about how NodeMCU can be used to create a cost-effective solution to deploy your Internet of Things project, together with AWS IoT.
Demystifying Wireless Security Using Open Source OptionsMichele Chubirka
Wireless LANs are often the soft underbelly of an organization's network. Users and guests demand easy access, but corporate resources still need to be protected. An enterprise could break the bank with expensive tools and consultants trying to maintain compliance and minimize risk.
The good news is that there are lots of excellent, well-documented open source (i.e., free) tools available to test and monitor your wireless network. And they don't even require a tin-foil hat.
Enhancing traffic analysis with elk and snortYaashan Raj
The slides present about how does Snort detecting the intruders meanwhile how to analyse the traffic using ELK Stack. It will be more helpful for you to detect the intruders and to analyse using colorful virtualization.
A brief introduction to LoRaWAN given at the Webnesday in St. Gallen on January 11th 2017. The focus is to give an idea on what LoRaWAN is, why it helps for IoT applications and how to use it (in Switzerland).
This is the presentation from Null/OWASP/g4h Bangalore October MeetUp by Narayanan Subramaniam.
technology.inmobi.com/events/null-october-meetup
Matriux is a GNU/Linux, Debian based security distribution designed for penetration testing and cyber forensic investigations. It is a distribution designed for security enthusiasts and professionals, can also be used normally as your default OS.
In the presentation , we will see how we can turn any system into a powerful penetration testing toolkit, without having to install any software into your hardisk. Matriux is designed to run from a Live environment like a CD / DVD or USB stick or it can easily be installed to your hard disk in a few steps.
Threat actors, criminal groups have traditionally been ahead of the detection curve; circumventing automate defenses and stealthily compromising an entire enterprise. As defenders, its crucial to adopt an offensive mindset to level the playing field.
In this presentation, we will first discuss on the various types of techniques used by threat actors which can aid them to stealthily compromise an entire organisation . Next we will look at detection. With the use of open source tooling, we will discuss how defenders can hunt for evidence of compromise by adopting an offensive mindset.
Introduction to DDS: Context, Information Model, Security, and Applications.Gerardo Pardo-Castellote
Introduction to the Data-Distribution Service (DDS): Context and Applications.
This 50 minute presentation summarizes the main features of DDS including the information model, the type system, and security as well as how typical applications use DDS.
It was presented at the Canadian Government Information Day in Ottawa on September 2018.
There is also a video of this presentation at https://www.youtube.com/watch?v=6iICap5G7rw.
Incident handlers manage security incidents by understanding common attack techniques, vectors and tools as well as defending against and/or responding to such attacks when they occur. In this talk we will discuss modern attacks, techniques, how to defend & respond to those threats.
Getting started on IoT with AWS and NodeMCU for less than 5€Stylight
In this workshop we talk about how NodeMCU can be used to create a cost-effective solution to deploy your Internet of Things project, together with AWS IoT.
Demystifying Wireless Security Using Open Source OptionsMichele Chubirka
Wireless LANs are often the soft underbelly of an organization's network. Users and guests demand easy access, but corporate resources still need to be protected. An enterprise could break the bank with expensive tools and consultants trying to maintain compliance and minimize risk.
The good news is that there are lots of excellent, well-documented open source (i.e., free) tools available to test and monitor your wireless network. And they don't even require a tin-foil hat.
Enhancing traffic analysis with elk and snortYaashan Raj
The slides present about how does Snort detecting the intruders meanwhile how to analyse the traffic using ELK Stack. It will be more helpful for you to detect the intruders and to analyse using colorful virtualization.
A brief introduction to LoRaWAN given at the Webnesday in St. Gallen on January 11th 2017. The focus is to give an idea on what LoRaWAN is, why it helps for IoT applications and how to use it (in Switzerland).
This is the presentation from Null/OWASP/g4h Bangalore October MeetUp by Narayanan Subramaniam.
technology.inmobi.com/events/null-october-meetup
Matriux is a GNU/Linux, Debian based security distribution designed for penetration testing and cyber forensic investigations. It is a distribution designed for security enthusiasts and professionals, can also be used normally as your default OS.
In the presentation , we will see how we can turn any system into a powerful penetration testing toolkit, without having to install any software into your hardisk. Matriux is designed to run from a Live environment like a CD / DVD or USB stick or it can easily be installed to your hard disk in a few steps.
Threat actors, criminal groups have traditionally been ahead of the detection curve; circumventing automate defenses and stealthily compromising an entire enterprise. As defenders, its crucial to adopt an offensive mindset to level the playing field.
In this presentation, we will first discuss on the various types of techniques used by threat actors which can aid them to stealthily compromise an entire organisation . Next we will look at detection. With the use of open source tooling, we will discuss how defenders can hunt for evidence of compromise by adopting an offensive mindset.
Introduction to DDS: Context, Information Model, Security, and Applications.Gerardo Pardo-Castellote
Introduction to the Data-Distribution Service (DDS): Context and Applications.
This 50 minute presentation summarizes the main features of DDS including the information model, the type system, and security as well as how typical applications use DDS.
It was presented at the Canadian Government Information Day in Ottawa on September 2018.
There is also a video of this presentation at https://www.youtube.com/watch?v=6iICap5G7rw.
This is the presentation at the OSIsoft EMEA User Conference in London, 16 October 2017.
Please note that "Open Edge Module" and "FogLAMP" are synonyms.
This presentation goes through several topics areas that are of specific interest in developing IoT Gateway solutions. IoT is a popular area of development that presents unique challenges like hardware and operating system selection, product life-cycle support and maintainability, software architectural solutions, connectivity, security, secure updates, and API availability. We discuss technologies and concepts like Hardware acceleration support, Linux kernel maintenance, Edge networking, LXC/Docker/KVM, Zigbee, 6loPAN, BLE, IoTivity, Allseen Alliance, SELinux and Trusted boot.
The aim of the presentation is to give an overview of the challenges in building an IoT Gateway and the Solutions available using Embedded Linux.
This presentation was delivered at LinuxCon Japan 2016 by Jim Gallagher
Big data conference europe real-time streaming in any and all clouds, hybri...Timothy Spann
Biography
Tim Spann is a Principal DataFlow Field Engineer at Cloudera where he works with Apache NiFi, MiniFi, Pulsar, Apache Flink, Apache MXNet, TensorFlow, Apache Spark, big data, the IoT, machine learning, and deep learning. Tim has over a decade of experience with the IoT, big data, distributed computing, streaming technologies, and Java programming. Previously, he was a senior solutions architect at AirisData and a senior field engineer at Pivotal. He blogs for DZone, where he is the Big Data Zone leader, and runs a popular meetup in Princeton on big data, the IoT, deep learning, streaming, NiFi, the blockchain, and Spark. Tim is a frequent speaker at conferences such as IoT Fusion, Strata, ApacheCon, Data Works Summit Berlin, DataWorks Summit Sydney, and Oracle Code NYC. He holds a BS and MS in computer science.
Talk
Real-Time Streaming in Any and All Clouds, Hybrid and Beyond
Today, data is being generated from devices and containers living at the edge of networks, clouds and data centers. We need to run business logic, analytics and deep learning at the scale and as events arrive.
Tools:
Apache Flink, Apache Pulsar, Apache NiFi, MiNiFi, DJL.ai Apache MXNet.
References:
https://www.datainmotion.dev/2019/11/introducing-mm-flank-apache-flink-stack.html
https://www.datainmotion.dev/2019/08/rapid-iot-development-with-cloudera.html
https://www.datainmotion.dev/2019/09/powering-edge-ai-for-sensor-reading.html
https://www.datainmotion.dev/2019/05/dataworks-summit-dc-2019-report.html
https://www.datainmotion.dev/2019/03/using-raspberry-pi-3b-with-apache-nifi.html
Source Code: https://github.com/tspannhw/MmFLaNK
FLiP Stack
StreamNative
Pachube: an open, easy to use, secure & scalable platform for building the 'I...pachube
Pachube (http://www.pachube.com/) is a data brokerage platform for the internet of things, managing millions of datapoints per day from thousands of individuals, organisations & companies around the world.
Pachube's powerful and scalable infrastructure enables you to build 'Internet of Things' products and store, share & discover realtime sensor, energy and environment data from objects, devices & buildings around the world.
Software Infrastructure for a National Research PlatformIan Foster
A presentation at the First National Research Platform workshop. "The purpose of this workshop is to bring together representatives from interested institutions to discuss implementation strategies for deployment of interoperable Science DMZs at a national scale." I present eight desirable properties for a software infrastructure for such a platforms, and describe our experience realizing these properties in the Globus system.
Maruthi Prithivirajan, Head of ASEAN & IN Solution Architecture, Neo4j
Get an inside look at the latest Neo4j innovations that enable relationship-driven intelligence at scale. Learn more about the newest cloud integrations and product enhancements that make Neo4j an essential choice for developers building apps with interconnected data and generative AI.
zkStudyClub - Reef: Fast Succinct Non-Interactive Zero-Knowledge Regex ProofsAlex Pruden
This paper presents Reef, a system for generating publicly verifiable succinct non-interactive zero-knowledge proofs that a committed document matches or does not match a regular expression. We describe applications such as proving the strength of passwords, the provenance of email despite redactions, the validity of oblivious DNS queries, and the existence of mutations in DNA. Reef supports the Perl Compatible Regular Expression syntax, including wildcards, alternation, ranges, capture groups, Kleene star, negations, and lookarounds. Reef introduces a new type of automata, Skipping Alternating Finite Automata (SAFA), that skips irrelevant parts of a document when producing proofs without undermining soundness, and instantiates SAFA with a lookup argument. Our experimental evaluation confirms that Reef can generate proofs for documents with 32M characters; the proofs are small and cheap to verify (under a second).
Paper: https://eprint.iacr.org/2023/1886
A tale of scale & speed: How the US Navy is enabling software delivery from l...sonjaschweigert1
Rapid and secure feature delivery is a goal across every application team and every branch of the DoD. The Navy’s DevSecOps platform, Party Barge, has achieved:
- Reduction in onboarding time from 5 weeks to 1 day
- Improved developer experience and productivity through actionable findings and reduction of false positives
- Maintenance of superior security standards and inherent policy enforcement with Authorization to Operate (ATO)
Development teams can ship efficiently and ensure applications are cyber ready for Navy Authorizing Officials (AOs). In this webinar, Sigma Defense and Anchore will give attendees a look behind the scenes and demo secure pipeline automation and security artifacts that speed up application ATO and time to production.
We will cover:
- How to remove silos in DevSecOps
- How to build efficient development pipeline roles and component templates
- How to deliver security artifacts that matter for ATO’s (SBOMs, vulnerability reports, and policy evidence)
- How to streamline operations with automated policy checks on container images
Essentials of Automations: The Art of Triggers and Actions in FMESafe Software
In this second installment of our Essentials of Automations webinar series, we’ll explore the landscape of triggers and actions, guiding you through the nuances of authoring and adapting workspaces for seamless automations. Gain an understanding of the full spectrum of triggers and actions available in FME, empowering you to enhance your workspaces for efficient automation.
We’ll kick things off by showcasing the most commonly used event-based triggers, introducing you to various automation workflows like manual triggers, schedules, directory watchers, and more. Plus, see how these elements play out in real scenarios.
Whether you’re tweaking your current setup or building from the ground up, this session will arm you with the tools and insights needed to transform your FME usage into a powerhouse of productivity. Join us to discover effective strategies that simplify complex processes, enhancing your productivity and transforming your data management practices with FME. Let’s turn complexity into clarity and make your workspaces work wonders!
Pushing the limits of ePRTC: 100ns holdover for 100 daysAdtran
At WSTS 2024, Alon Stern explored the topic of parametric holdover and explained how recent research findings can be implemented in real-world PNT networks to achieve 100 nanoseconds of accuracy for up to 100 days.
UiPath Test Automation using UiPath Test Suite series, part 6DianaGray10
Welcome to UiPath Test Automation using UiPath Test Suite series part 6. In this session, we will cover Test Automation with generative AI and Open AI.
UiPath Test Automation with generative AI and Open AI webinar offers an in-depth exploration of leveraging cutting-edge technologies for test automation within the UiPath platform. Attendees will delve into the integration of generative AI, a test automation solution, with Open AI advanced natural language processing capabilities.
Throughout the session, participants will discover how this synergy empowers testers to automate repetitive tasks, enhance testing accuracy, and expedite the software testing life cycle. Topics covered include the seamless integration process, practical use cases, and the benefits of harnessing AI-driven automation for UiPath testing initiatives. By attending this webinar, testers, and automation professionals can gain valuable insights into harnessing the power of AI to optimize their test automation workflows within the UiPath ecosystem, ultimately driving efficiency and quality in software development processes.
What will you get from this session?
1. Insights into integrating generative AI.
2. Understanding how this integration enhances test automation within the UiPath platform
3. Practical demonstrations
4. Exploration of real-world use cases illustrating the benefits of AI-driven test automation for UiPath
Topics covered:
What is generative AI
Test Automation with generative AI and Open AI.
UiPath integration with generative AI
Speaker:
Deepak Rai, Automation Practice Lead, Boundaryless Group and UiPath MVP
UiPath Test Automation using UiPath Test Suite series, part 5DianaGray10
Welcome to UiPath Test Automation using UiPath Test Suite series part 5. In this session, we will cover CI/CD with devops.
Topics covered:
CI/CD with in UiPath
End-to-end overview of CI/CD pipeline with Azure devops
Speaker:
Lyndsey Byblow, Test Suite Sales Engineer @ UiPath, Inc.
Sudheer Mechineni, Head of Application Frameworks, Standard Chartered Bank
Discover how Standard Chartered Bank harnessed the power of Neo4j to transform complex data access challenges into a dynamic, scalable graph database solution. This keynote will cover their journey from initial adoption to deploying a fully automated, enterprise-grade causal cluster, highlighting key strategies for modelling organisational changes and ensuring robust disaster recovery. Learn how these innovations have not only enhanced Standard Chartered Bank’s data infrastructure but also positioned them as pioneers in the banking sector’s adoption of graph technology.
GraphRAG is All You need? LLM & Knowledge GraphGuy Korland
Guy Korland, CEO and Co-founder of FalkorDB, will review two articles on the integration of language models with knowledge graphs.
1. Unifying Large Language Models and Knowledge Graphs: A Roadmap.
https://arxiv.org/abs/2306.08302
2. Microsoft Research's GraphRAG paper and a review paper on various uses of knowledge graphs:
https://www.microsoft.com/en-us/research/blog/graphrag-unlocking-llm-discovery-on-narrative-private-data/
Goodbye Windows 11: Make Way for Nitrux Linux 3.5.0!SOFTTECHHUB
As the digital landscape continually evolves, operating systems play a critical role in shaping user experiences and productivity. The launch of Nitrux Linux 3.5.0 marks a significant milestone, offering a robust alternative to traditional systems such as Windows 11. This article delves into the essence of Nitrux Linux 3.5.0, exploring its unique features, advantages, and how it stands as a compelling choice for both casual users and tech enthusiasts.
Observability Concepts EVERY Developer Should Know -- DeveloperWeek Europe.pdfPaige Cruz
Monitoring and observability aren’t traditionally found in software curriculums and many of us cobble this knowledge together from whatever vendor or ecosystem we were first introduced to and whatever is a part of your current company’s observability stack.
While the dev and ops silo continues to crumble….many organizations still relegate monitoring & observability as the purview of ops, infra and SRE teams. This is a mistake - achieving a highly observable system requires collaboration up and down the stack.
I, a former op, would like to extend an invitation to all application developers to join the observability party will share these foundational concepts to build on:
GraphSummit Singapore | The Art of the Possible with Graph - Q2 2024Neo4j
Neha Bajwa, Vice President of Product Marketing, Neo4j
Join us as we explore breakthrough innovations enabled by interconnected data and AI. Discover firsthand how organizations use relationships in data to uncover contextual insights and solve our most pressing challenges – from optimizing supply chains, detecting fraud, and improving customer experiences to accelerating drug discoveries.
Generative AI Deep Dive: Advancing from Proof of Concept to ProductionAggregage
Join Maher Hanafi, VP of Engineering at Betterworks, in this new session where he'll share a practical framework to transform Gen AI prototypes into impactful products! He'll delve into the complexities of data collection and management, model selection and optimization, and ensuring security, scalability, and responsible use.
Unlocking Productivity: Leveraging the Potential of Copilot in Microsoft 365, a presentation by Christoforos Vlachos, Senior Solutions Manager – Modern Workplace, Uni Systems
13. Publish/Subscribe Messaging
• Protocol-independent pub/sub core messaging service
• Messages up to 32kb w/ TLS encryption
• Lightweight channel handling: no limit on number of
channels & no cost impact
• Publish 1-to-1, 1-to-many, many-to-many
// publish
pubnub.publish({
channel : “hw_test”,
message : “Hello World!”
});
// subscribe
pubnub.subscribe({
channel : "hw_test",
message : receiver
})
14. Global Replication & Reliability
• 14 data centers located on 4 continents
• 99.999% uptime guaranteed
• All messages replicated to all 14 data centers by default
• Automatically manages dropped connections & reconnects
• Message caching catches devices up on missed data
15. PubNub Presence
• Built into API for all channels
• Automatically detect join/leave or device status change events
• Trigger events/behaviors based on presence status changes
• API method for retrieving list of device in a given
state & device location
• Set custom status changes, i.e. GPS location, or any custom
name/value pairs
// presence
pubnub.subscribe({
channel : “hw_test",
presence : pres_reciever,
message : receiver
});
16. Storage & Playback
• Default 24 hour storage of all messages;
unlimited retention available
• Catch users up on message history when they come online
• Load a configuration onto a device when it is booted up
• Audit past messages for compliance
• Play back messages at same rate they were received
// storage and playback
pubnub.history({
channel : “hw_test",
callback : sp_reciever
})
17. Access Manager
• Token-based access control scheme (similar to OAuth)
• Grant access to individual channels or groups of channels
• Read / write controlled at token level
• Tokens can auto-expire based on time (TTL)
• Grant/revoke tokens manually
// Key Authorization
var pubnub = PUBNUB.init({
publish_key : "pam",
subscribe_key : "pam",
auth_key : "3y8uiaj"
})
21. The different
components of IoT
• Data stream network does the
heavy lifting
• Smart network that manages
“things” and the data that flows
between them.
Things
Data Stream
Network
Mobile
applications
22. IoT pain points
Presence
detection
Decreased time to
market
Storing offline
messages
Minimum bandwidth
usage
Cross platform
support
security
Read/write
access
Bidirectional data
flow
Power drain
24. IoT pain points
• Detect machine connections
• Monitor user and machine
presence
• Global scale
Presence Detection
• Storage of realtime data
• History to retrieve messages
• Replay messages in order
received
Message History
25. • Sensors
• Always on devices
• Protocol independent approach
Bandwidth/power usage
• Millions of devices
• Plug and play configuration
• Decreased time to market
Scalability
27. Enterprise-Grade Security
Cross-platform AES & TLS
Message envelope meta-
data for smart routing &
analytics
Fine-grain token-based
access control
Auto-expiration of tokens
& manual revocation
No inbound open ports
removes attack vector
Can route to different data
centers to thwart regional
attacks.
Safe Harbor Compliant
Safe for HIPAA and PII
Intelligent Data Routing
Access Control Attack Prevention LegislativeEncryption
28. Why a Data Stream Network matters for
IoT?
• Realtime communication
• Analytics
• Message Routing
• Scalability
Mobile/IoT/web
29. IoT Use cases
• Control and monitor devices
• Stream vehicle data
• Trigger automatic actions
• Monitor device metadata
• http://www.pubnub.com/solutions/
Home Automation Connected/Smart cars
Wearables/ Connected devices
Coffe pot monitor : https://vimeo.com/134363392
PubNub security camera : https://vimeo.com/131005832
PubNub IoT house : https://vimeo.com/114900127
Motion controlled servos with leap motion and the Pi :
https://vimeo.com/136779399
Hi everybody, thank you for coming. My name is Bhavana srinivas and I work as a developer evangelist for PubNub.
PubNub is a global DSN that provides easy to use API for real time , reliable signaling between sensors, mobile devices, embedded chips, or software applications, anywhere around the globe in a secure manner. We have 70+ SDKs that we support, to let you build and scale real time applications for mobile, web and IoT.
As I go through my presentation, you will see what a data stream network means and how it will benefit IoT.
My twitter handle is @bhavana1110 and you can reach me at bhavana@pubnub.com. I have some cards and data sheets and frisbees with me as well in case you are interested.
I will walk you through what a data stream network is, what you can do with it and why it is criticial for IoT.
Lets start of by understanding what this even means and then move on to how we can use it for IoT.
BUILD SLIDE
More and more companies/software's needed the ability to control and work with real time data. Whether you needed stock updates, multi player game movements, screen sharing, voice over IP, telephony etc, you had this new data in motion, as compared to the traditional data in rest, which can be pulled out of a database every now and then.
So data is being constantly emitted be it social streams or financial streams, and there was no way to route it, or manage it or any kind of infrastructure to get the data to the devices that needed it in real time.
more and more companies were spending all their time and effort to build this kind of infrastructure than focussing on their application logic.
So just to make this really clear where PubNub sits, this is the evolution of the Internet. MAKE NOTES FROM TODDS PRESENTATION.
Lets go back 25 years or so. People were building the core infrastructure for the internet. So you had companies like Verizon, Comcast and level3 that were offering peering, routing or PoPs that speak TCP/UDP. Basically, connecting you to the internet.
Fast forward 10/15 years, you had all these webisites with tonnes of informtaion , all trying to scale. So you had CDNS like Akamai, cogent etc, that made copies of the information on these sites, and put them all over the world. If you were trying to access yahoo weather info from Japan, you would probably be connected to a data center that’s diff from someone accessing that information from California. These guys were speaking difffernet protocols like HTTP to do caching services etc.
A data stream network is similar to this, but meamt for two way communication to and from these data centers. U still have data centers all over the world, but you are publishing and subscribing data from them, and also speak other protocols like websockets,, MQTT, which are constantly evolving.
The current infrastructure for data at rest is pretty mature and well understood..They have been used for the past so many years, and they make it very easy to build mobile and web apps.
For data in motion on the other hand, its first of all not compatible with the infrastructure available for data at rest applications. The software or technologies themselves are relatively new. There are several open source technologies that can be used, but when you have to scale it becomes a problem. They are not secure, since they are new as well, and hasn’t been documented well.
So what we’ve just seen is that pretty much the whole internet is architected to handle data at rest. We’re interested in data in motion – streaming data. Similar to how you wouldn’t build your own ISP or CDN, what we’re suggesting is you shouldn’t try to architect your own DSN either.
You can roll your own, but…
Time to build is costly and increases time to market.
Hidden costs of scaling up are disproportionately expensive.
Ongoing maintenance and orchestration requires a dedicated team of experts.
Flexibility and expansion potential constrained by limited feature sets.
Security is an ongoing risk.
When you are devloping an application on a small scale , it is easy.
So does it make sense for everyone building apps, build hteir own dsn? No. why? But in the real world, you have a lot of problems like
unrealiable connectiviity’
different networks, - devices are always moving from one network to another. Within tunnels or losing powerr anf hence connectivity.
diff devices to be supported -
Latency
scaling
security
and many more.
Add notes from Todds presentation on what a global DSN is.
This is why using a Data stream network helps,, since it provides a way to connect, scale and secure real time data streams for rhe web, mobile and IoT.
Massive numbers of connections across any device
Extremely low latency
Unmatched reliability / uptime
End to end security over our network
Connect
Support several platforms
Powered by publish/subscribe messaging
Storage & Playback functionality
Secure
End to end security: TLS/SSL & AES included by default
PubNub Access Manager for token-based access control
No open ports to the web
EU SafeHarbor, HIPAA compliant
Scale
Global data replication to 14 data centers
Intelligent routing automatically connects users to closest PoP
Message & bandwidth efficiency with PubNub Stream Controller
Add scale easily, automatically, and economically
Audience simulation service for smooth rollouts & QA
Manage
Developer console & debug tools
Stream analytics shows data stream activity in real time
Steam syndication design pattern makes it easy to monetize data streams
Business model scales with your project’s success
No year-long contracts
PubNub excels because we deliver:
If you’ve had a chance to align on required capabilities before the call, list them here. If not, think about taking the time to fill this in with the prospect during the presentation.
Real-time data transfer speeds
Real-time location tracking
Extremely high reliability (works in the real world)
Easy, efficient scalability
Robust security built in
Developer tools to build in preferred environment
Business model that scales with your success
Add-on features to enable your use case
Explain the reasoning behind each of these features as per Todd’s video.
A data stream network has the ability to manage bi-directional communication between the devuces connected to them. This can be 1-1, 1-many or many-to-many, basucally the ability to publish and subscribe to messages.
For several applicattions, you want to be able to know who is part of the conversastion, or who has joined that channel of communicaationEssentially you wajnt to know the presence on that channel. ThisThis is similar to ,the green dot you see in a chat app like skype, when a person is online. Or a grey when they leave.
This is being used a lot in IoT, where you want to know when your remote controlled thermostat comes online, or also know when that connection goes off.
Now that being said, say you have several users on a chat channel, or several devices that are sending and receiving information using the data stream network.
. You are able to receive messages when you are connected to the network. But say your devuce went offline for a day, month or year. You want to be able to see the history of the streams that you missed on that channel and played back at the same rate at which it was received.
Also, in order to secutre the deivices that send/receive messages through the network, you want to be able to give
fine grained read/ write access on a channel and token level.
This way, only devices that have access can publish or subscribe.
What is the current state of IoT.
What are the different verticals
What are some of the common problems – no bi directional comm/ no uniformity ex all devices cant talk to each other/ memory/bandwidth/security/
How does a data stream network/pubnub solve it
So, as is often described, there are many machine to machine communications within the category of IoT. You have smart home appliances to wearables that may or may not be plugged into medical diagnostics to connected cars and transportation.
These, along with many other categories are the “Things” in IoT.
But when describing the “Internet” part of IoT, most presentations simply had some sort of amorphous cloud that connected everything magically.
The point is that everyone seems to talk about the big data and the devices that make up the IoT, but not the lines to the “cloud” that connects it all.
People are working on different verticals. And a very common application is when a sensor or a device is sending information back to the user. But you want to be able to do more than just receiving info from your thermostat. You want to be able to take some actions based on that value. You want to be able to control the devices you own, remotely, irrrespective of where you are in the world. You want to be able to do this in real time, cos noone wants their garage door to open five min after you press a button. You want it to happen instantly.
And that’s where a data stream network like pubnub fits in. It is the communication layer that connects the IoT.
What is the best way to build an IoT application that handles everything from controlling home appliances remotely to inform an aero-engine manufacturer that one of its engines need servicing in the opposite side of the world to gathering information from sensors destributed in the wild?
The starting point for Internet of Things applications are the things themselves. These edge devices typically have no screen (although that's not always the case), a low-power processor, some sort of embedded operating system and a way of communicating (usually wirelessly) using one or more communication protocols. The things may connect directly to the Internet, to neighboring things or to an Internet gateway device – typically a plastic box with blinking lights.
The next tier of the system, is a software and infrastructure that runs in a corporate data center or in the cloud and receives and organizes the streams of data coming from the things. Software running in the ingestion tier is usually also responsible for managing things and updating their firmware when necessary. After this comes the analytics tier; this takes the organized data and processes it. Finally, there's the end-user tier, the application that the end user actually sees and interacts with. This may be an enterprise application, a Web app or, perhaps, a mobile app.
So from the previous image, you can see how there are so many problems to solve startting from the “tjings in the internet of things, the network or infrastructure trat will support them, and the devices that run the application.
Lets start with the things – there are so many of them, from MCUs to MPUs, all runninf diff Oses, diff platforms etc. A samsung device doesn’t talk to an LG one which doesn’t talk to Apple devices., Whehn they are so varied, you need cross platform support.
Let me go through some of the obstacles in building an IoT application.
power drain : PubNub’s publish / subscribe paradigm reduces “chattiness” to keep power drain to a minimum
bandwidth : Protocol-independent approach uses the minimum bandwidth necessary to monitor and control IoT devices
Firmware: Use secure, private channels to remotely upgrade firmware with no user action needed
time to market : Don’t have to waste months of your team’s time architecting and maintaining a realtime infrastructure in-house
device provisioning: Make your IoT device plug-and-play, even through home and business firewalls
The way we do security is that the device opens a connection to Pn, and we keep that connection opne, so that now data canonly flow back through the pn nextwork to the device.
Remote Firmwarae upgrades – we don’t provide the FTP server that people usually need for this. We have a design practise to do this.
The server first lets its devices know that a new firmware update is available by broadcasting an alert message on a channel that all devices can read.
Then the devices that are online (and we know which ones they are through the use of Presence) download the update immediately.
For those devices that are not online, they can download the update from a cache on the channel when they are booted back up and available.
For some applications, its all about collecting data from the device like a sensor . For those, uni directional communication is more that enough
Data collection is:
* Easy
* Everyone is focusing on it today
[CLICK FOR ANIMATION]
Device Control is:
* Harder
* With smart home, connected cars, industrial valve controls, robots, etc. we don’t just want to read their status, we also want to control the devices
The promise of IoT is with device control, it’s not just about receiving the data from the devices
Built into API for all channels
Automatically detect join/leave or device status change events
Trigger events/behaviors based on presence status changesstate & device location
Set custom status changes, i.e. GPS location, or any custom name/value pairs
We have spotty connections everywhrre, when a handoff happens. In remote areas and when you go under tunnerls. This is when devices go offline. There is a meed for the device to automatically try to reconnect, and come back online as soon as possible. When it does, it needs to access all the information that might be lost, at the same rate that it was sent out,.
Default 24 hour storage of all messages; unlimited retention available
Catch users up on message history when they come online
Load a configuration onto a device when it is booted up
Audit past messages for compliance
Play back messages at same rate they were received
These sensors and hardare devices are becoming very small and cheap., but the constraint is power and memory. They have low program memory, and also cant last too long on, unless they conserve battery. The device must be able to send and receive messahes , by not spending too much energy .
When building an application, it is usually targeted to a large number of users. Once these devuces are deployed, it is necessary to ensure that every one of them can communicate with the DSN. When bought, these devices will prove ineffective if they need to be configured to work with the house/office firewalls. People expect it to be plug and play. You bring a new speaker at hime, that connects to the internet, you expect to be able to set it up really easily.
Already, too many manufacturers worry more about getting their IoT product to market quickly than about securing it. In some cases, the devices are so small that it’s hard to build in the right protection.
how do you go about managing the username/passwords for your ever increasing number of connected devices and appliances? What about the privacy of your information? Take as an example various Internet connected video cameras with easily defeated security controls or baby monitors.
So offloading security to the network is the right answer for many developers.
Jeep Hack (August 2015)
KQED.org (August 2015): http://ww2.kqed.org/futureofyou/2015/08/03/millions-of-americans-use-medical-devices-that-are-vulnerable-to-hacking/
Deep brain neurotransmitters
Gastric stimulators
Foot drop implants
Cochlear implants
Cardiac defibrillators / pacemakers
Insulin pumps
Security has to be a forethought more than anything else. The race to market for new devices means that security is not included in the design, or it is severely limited or poorly implemented. ’
You need to be able to encrypt the message using maybe TLS to secure the message from the device to the network, and trhen use something like AES that does the encryption even when it goes through the networlk.
OPEN PORTS: most devices on the internet, are also listening to any messages that come to them throufh the network. They always have open ports. This is asking for trouble since anyone can hack into that and sned them the wrong messages. You want to be able to open a connection to your device tham the other way rounf to be more secure.
TLS + AES:
ACCESS CONTROL – giving different users diff read/write permissions.
That's why it usually makes more sense to build an application on top of a ready-made "Internet of Things platform," Gillett adds. These platforms usually include an ingestion tier that carries out time-series archiving for incoming data, as well as an analytics tier, thin provisioning, activation and management capabilities, a real-time message bus, and an API to allow communication between the platform and applications built on top of it.
Build vs buy argument. How scale also helps.
A data stream network lets you manage, connect, scale your realitme applications for mobile web and IoT. At pubnub as well, we have seen several customers , over 2000, in a lot of verticals like chat, social apps, screen share, collaborative apps and IoT. With IoT, we have seen people using us for home automatrion, devcice contril, connected cars and connected devuces,
We have seen and built a lot of demos from hobbysists like raspberry pi and arduino community. They involve bidirectional messaging to and from the device.