Open Source
For
Rethinking IoT
Francis da Costa
www.chirp-networks.com 2
Francis daCosta: Past Experience and Expertise
• Real time Embedded systems and control
• Robotic...
www.chirp-networks.com 3
Its Different Out There:
The IoT won’t be much like the traditional Internet:
• Scope – it’s 100 ...
www.chirp-networks.com 4
The Challenge: Moore’s Vs Metcalfe’s at the Edge (IoT)
Time
M2M Social Traffic  Overhead at Edge
...
www.chirp-networks.com 5
Simpler Devices Must Rule
Next wave of the Internet is Machines to Machines Ecosystems
• Lots mor...
www.chirp-networks.com 6
IoT Data Characteristics
Machine to Machine (M2M)
• Terse – not oriented to humans
• Repetitive
•...
www.chirp-networks.com 7
Self-Classification Lesson from Nature: Pollen “Chirps”
Pollen propagates everywhere, but only sp...
www.chirp-networks.com 8
Self Classification Lesson from Nature: Birdsong “Chirps”
All birds derive some information, but ...
www.chirp-networks.com 9
Chirp Overhead = 4 bytes, 1 byte payload
IPV6 Overhead = 40 bytes, 1 byte payload
Note that Chirp...
www.chirp-networks.com 10
Public Section (mandatory) Private Section (optional)
Total chirp length with 2 Byte Public Fiel...
www.chirp-networks.com 11
Self-Classification is Key to Discover / Subscribe
• Massive amounts of data published from tril...
www.chirp-networks.com 12
Discover/Publish/Subscribe Lesson from Nature: Affinities
Underlying event not seen, but affinit...
www.chirp-networks.com 13
Home NetworkHome Network
Smoke Alarm
Ventilation
Lawn
Sprinkler

Local Weather Forecasts
Home “...
www.chirp-networks.com 14
Initial application:
air conditioning
control
Affinity by time-of-day correlation: elevator acti...
www.chirp-networks.com 15
• Discovery: Many useful data sources may be unknown.
- Self-classification (“Pressure sensor”, ...
www.chirp-networks.com 16
Scalability Lesson from Nature: Trees
End devices don’t communicate with one another, so “tree” ...
www.chirp-networks.com 17
Emerging IoT Architecture
End Devices
Propagator
Mesh Network
Filter
Gateway
Integrator
Function...
www.chirp-networks.com 18
Scalability of IoT Architecture
• End Devices can be cheap, simple, low power,
unmanaged
• Proto...
www.chirp-networks.com 19
Lesson from Nature: Local Autonomy
Devices operate independently, but may act in concert with ex...
www.chirp-networks.com 20
IP
IP
IP
IP
Round-Trip Control Loops
Dual Isochronous Control Loops
IP overhead for
every end de...
www.chirp-networks.com 21
Importance of Open Source
• Basic Chirp self-classification taxonomy will be (and must
be) Open ...
www.chirp-networks.com 22
Even “Closed” Flows Will Offer Open Source Information
• Specifics of data stream may be private...
www.chirp-networks.com 23
Internet of Things can only Happen through Open Source
• Too big, too much data, too unmanageabl...
www.chirp-networks.com 24
Support Slides
www.chirp-networks.com 25
Scalability: Managing Broadcasts, Distribution, Addressing
Chirps from end devices
Collection
IP...
www.chirp-networks.com 26
Scalability: Loading “Buses”
Chirps to-and-from end devices
Buses to/from
different
destinations...
www.chirp-networks.com 27
• Developed on Open Source platform: OpenWrt, et al
• Build structured trees among themselves
- ...
www.chirp-networks.com 28
Security Must be Incremental to Open Source Format
• Basic Chirp published and open to all
- As ...
www.chirp-networks.com 29
Public Section (mandatory) Private Section (optional)
DNA Pointer: 4 bytes, 8 bit Marker (1010)
...
www.chirp-networks.com 30
Upstream Remote Pressure Sensor
Local Low Level Control
Lawn Sprinkler
Actuator Valve
Real and V...
www.chirp-networks.com 31
Chirp Networks: Scalable Publish/Subscribe for the Edge
Chirp Networks are
• Scalable with Moore...
Upcoming SlideShare
Loading in...5
×

Open Source For Self Classification of Data Stream in the Internet Of Things.

438

Published on

It is well understood that the Internet of Things represents unprecedented challenges of scope, with trillions of appliances, sensors, and actuators expected to be connected over the coming years. What is not yet appreciated is that current peer-to-peer networking schemes will be unable to create the kind of publish / discover / subscribe architectures that will be needed. Instead, a new type of self-classified data stream is needed, which can only be enabled by open-source collaborations in defining and implementing the emerging architecture of the IoT. Mr. daCosta will explore the implications of open source for end devices and networking equipment, as well as describing how even proprietary IoT data flows can help build an open source implementation.

Presented at IoTWorldEvent, June 16, 2014

Published in: Internet, Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
438
On Slideshare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
Downloads
12
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Transcript of "Open Source For Self Classification of Data Stream in the Internet Of Things. "

  1. 1. Open Source For Rethinking IoT Francis da Costa
  2. 2. www.chirp-networks.com 2 Francis daCosta: Past Experience and Expertise • Real time Embedded systems and control • Robotics, Machine Learning • Distributed networking intelligence • Real time Publish / Discover / Subscribe • Real time Scheduling at network level. • Founder/CTO; MeshDynamics (Mesh networking) • Founder/CTO; Knowmadic (“Big” Data) • Founder/CTO: Advanced Cybernetics Group (Robotics) • IoT Stealth Startup: (Chirp Networks) http://www.linkedin.com/in/francisdacosta/
  3. 3. www.chirp-networks.com 3 Its Different Out There: The IoT won’t be much like the traditional Internet: • Scope – it’s 100 times bigger • Simple – majority of end devices “dumb” • Scalable – All control cannot be centralized • Subscription-based – too much data otherwise • Security – must be incremental • Standards – Must be Open-Source with privacy extensions Why Open Source is Essential For IoT
  4. 4. www.chirp-networks.com 4 The Challenge: Moore’s Vs Metcalfe’s at the Edge (IoT) Time M2M Social Traffic  Overhead at Edge Metcalfe’s Law is Exponential  Traffic within M2M Social Communities increase as O (n2)‐ Overtakes Moore’s Law at Edge of the Network, where IoT resides  Moore’s Law is Linear: “2X improvement Every 18 months”. CPU Improvement is O (n) Slow Down depicts slower adoption by resource constrained edge devices (e.g. IoT)  A new approach is needed for networking at the “Edge”
  5. 5. www.chirp-networks.com 5 Simpler Devices Must Rule Next wave of the Internet is Machines to Machines Ecosystems • Lots more Processor, Memory, Protocol stacks • Human Oriented Consumption (external) • Assumed often “Always On” • Centrally-managed naming (MACID, et al) • Often Limited to microcontrollers etc. • Consumption for local use (Internal) • Many remote with Intermittent power • Built by millions of manufacturers worldwide … many cannot afford traditional IP protocol overhead Humans Oriented Ecosystem Machines Oriented Ecosystem
  6. 6. www.chirp-networks.com 6 IoT Data Characteristics Machine to Machine (M2M) • Terse – not oriented to humans • Repetitive • Individual messages not critical • Meaning comes from combination with other data sourcoes – “Small Data” • Consumption and Generation is mostly Local – M2M communities • Often unidirectional, • Self-classified – new and necessary concept
  7. 7. www.chirp-networks.com 7 Self-Classification Lesson from Nature: Pollen “Chirps” Pollen propagates everywhere, but only specific receivers decode “message”
  8. 8. www.chirp-networks.com 8 Self Classification Lesson from Nature: Birdsong “Chirps” All birds derive some information, but only specific receivers fully participate
  9. 9. www.chirp-networks.com 9 Chirp Overhead = 4 bytes, 1 byte payload IPV6 Overhead = 40 bytes, 1 byte payload Note that Chirp lacks: • Universally unique ID • Error correction/retransmission Minimal overhead for end device, but it must be applied elsewhere: -“Propagator” Nodes/Network (discussed later) Public Chirp Classification Private payload IoT Data is Different, Protocol Must be Different: Chirps
  10. 10. www.chirp-networks.com 10 Public Section (mandatory) Private Section (optional) Total chirp length with 2 Byte Public Field, 4 bit Marker, 1 Byte Payload = 5.0 bytes 5.0 Bytes with 1 Byte Payload 6.0 Bytes with 2 Byte Payload 7.0 Bytes with 3 Byte Payload 8.0 Bytes with 4 Byte Payload 04 22 243 06 02 255 03 12 22 243 16 23 255 4 251 6 Marker Pointer (4 bits) 8 bit Marker 4 bit Marker (Subscribed) Agent ID Agent ID Self-Classification “Pollen/Birdsong” for IoT: Chirps
  11. 11. www.chirp-networks.com 11 Self-Classification is Key to Discover / Subscribe • Massive amounts of data published from trillions of devices • Servers (“Integrator Functions”) will discover and subscribe to interesting small data flows • This requires self-classification at end device - Basic: Type of device (moisture sensor, streetlight, etc.) from open-source taxonomy - Incremental: unlimited additional classification through private fields • Published data may be open to all or proprietary
  12. 12. www.chirp-networks.com 12 Discover/Publish/Subscribe Lesson from Nature: Affinities Underlying event not seen, but affinities are visible.
  13. 13. www.chirp-networks.com 13 Home NetworkHome Network Smoke Alarm Ventilation Lawn Sprinkler  Local Weather Forecasts Home “Health” Advisor Application subscribes to many data streams Time of Day Utility Pricing Known Publish/Subscribe Affinities (“Pollen”)
  14. 14. www.chirp-networks.com 14 Initial application: air conditioning control Affinity by time-of-day correlation: elevator activity Affinity by location: lighting control Integrator function seeks additional candidate data sources by affinities. Builds more refined causal models. Accelerate Learning through Affinities Affinity by type of data: peak energy cost variations Discovered Publish/Subscribe Affinities (“Pheromones”)
  15. 15. www.chirp-networks.com 15 • Discovery: Many useful data sources may be unknown. - Self-classification (“Pressure sensor”, “irrigation valve”, “Flow Sensor” etc.) permits discovery of data “affinities” - Open Source top-level taxonomy crucial to scale and discovery • Subscriber Based: Small data flows may be discovered, selected, and incorporated by Integrator Functions • Dynamic: New flows may be added and existing sources aged- out over time • Primarily for Local Consumption at Edge: “Small” data. Publish / Subscribe for the IoT
  16. 16. www.chirp-networks.com 16 Scalability Lesson from Nature: Trees End devices don’t communicate with one another, so “tree” better than “web”
  17. 17. www.chirp-networks.com 17 Emerging IoT Architecture End Devices Propagator Mesh Network Filter Gateway Integrator Function Chirp Data Streams “Small” Data Flows “Big Data” Analysis
  18. 18. www.chirp-networks.com 18 Scalability of IoT Architecture • End Devices can be cheap, simple, low power, unmanaged • Protocol sophistication only in Propagator Nodes • Prune and trim broadcasts – building “buses” • Optional distributed intelligent agents in Propagator Nodes - Extend subscription preferences of Integrator Functions - Add security and proprietary functions - Extends “Software Defined Network” publish/ subscribe functionality to edge of network
  19. 19. www.chirp-networks.com 19 Lesson from Nature: Local Autonomy Devices operate independently, but may act in concert with external “cues”
  20. 20. www.chirp-networks.com 20 IP IP IP IP Round-Trip Control Loops Dual Isochronous Control Loops IP overhead for every end device Lightweight chirp protocol only IP IP IP Managing Local Autonomy Traditional Emerging IoT “Propagator”
  21. 21. www.chirp-networks.com 21 Importance of Open Source • Basic Chirp self-classification taxonomy will be (and must be) Open Source for broad dissemination - Some number of top-level classifications - Sensors, appliances, actuators, etc. • Working groups and SIGs may refine sub-classifications - Most also open to public for non-local consumers • Enterprises and OEMs may develop custom and proprietary extensions for Private fields • Open Source networking implementations (OpenWrt, et al) to accelerate large scale deployment
  22. 22. www.chirp-networks.com 22 Even “Closed” Flows Will Offer Open Source Information • Specifics of data stream may be private, but “affinities” are still observable - Information about number, location, and activity of devices (and much more) - Adds information to Open Source Taxonomy • Analogous to CAPTCHA™ environment • Every additional end device potentially adds to Open Source knowledge base
  23. 23. www.chirp-networks.com 23 Internet of Things can only Happen through Open Source • Too big, too much data, too unmanageable • Lessons from nature - Only publish /discover / subscribe can scale - Self-classified data needed so receivers may select • Open Source taxonomy • Rapid proliferation of Propagator Nodes Functionality • Chirp Networks is developing prototype for military. • Leverages Open Source (OpenWRT, MAC802.11)
  24. 24. www.chirp-networks.com 24 Support Slides
  25. 25. www.chirp-networks.com 25 Scalability: Managing Broadcasts, Distribution, Addressing Chirps from end devices Collection IPv6 Packet Pruning Bundling Propagator Node Context GPS Location MotionTemperature Propagator Node MAC ID and IP Subnet Propagator Nodes create “Small Data” flows from Chirp data streams
  26. 26. www.chirp-networks.com 26 Scalability: Loading “Buses” Chirps to-and-from end devices Buses to/from different destinations Propagator Node Buses to/from different integrator functions Chirps unloaded/ reloaded Real Time Scheduling at Propagator Node
  27. 27. www.chirp-networks.com 27 • Developed on Open Source platform: OpenWrt, et al • Build structured trees among themselves - Path discovery, routing, redundancy, fail-over - Simplicity through “near-optimal” routing • Manage multicast: pruning, forwarding, spoofing, etc. • Optional integrated Publishing Agents participate in publish/subscribe bus, machine learning • Offer wide variety of end device interfaces: wired, wireless, optical, etc. Propagator Nodes –Networking Capabilities
  28. 28. www.chirp-networks.com 28 Security Must be Incremental to Open Source Format • Basic Chirp published and open to all - As in nature’s pheromones, pollen and birdsong • Private fields within Chirp may create “lock-and-key” relationship in OEM and proprietary applications - As in pollen – receiver determines • Further security achieved through distributed agents in Propagator Nodes • Secure data may still flow through Propagator Node network with open data, but is unintelligible - From nature: air transports both proprietary (e.g., pollen) and open “signals” (e.g., pheromones)
  29. 29. www.chirp-networks.com 29 Public Section (mandatory) Private Section (optional) DNA Pointer: 4 bytes, 8 bit Marker (1010) 12 22 243 16 23 255 4 251 6 Public Agent ID is 4.8.255 (4 byte Public, 8 bit Marker, DNA 255 (Subscribed) Agent states: Classification is 8.8.8.8 (1 byte each) Decrypted Chirp Class: 4.8.22.243.16.23. Its payload requires another Agent Private Agent 1.4.6 (specific for 4.8.22.243.16.23) decodes (251) Chirp with public (open) payloads have shorter classifications e.g. Chirp Class 4.8.22: Temp=243F Pressure=16psi Humidity=23%. Larger Packets intended for slower transport. Enterprises may define their (internal) classification schemes. Discovery of “unknown” chirp classes detected, addressed in SIGs. Distributed, organic growth of chirp classification taxonomy. Security Must be Incremental
  30. 30. www.chirp-networks.com 30 Upstream Remote Pressure Sensor Local Low Level Control Lawn Sprinkler Actuator Valve Real and Virtual Sensor                Publish/Subscribe Stream Remote Control Model refinement Down Stream Remote Flow Sensor Q: Did Sprinkler Turn On? A: No. Integrator Local Integrator Propagator Relay Node CAUSAL INFERENCE ENGINE 1. Upstream Water Pressure good 2. Downstream Flow Absent “ON” “OFF” Local and Remote Control Loops
  31. 31. www.chirp-networks.com 31 Chirp Networks: Scalable Publish/Subscribe for the Edge Chirp Networks are • Scalable with Moore’s Law (linear) • Secure: Chirp to IP bridging is through distributed agent network • Dynamic: Supports Temporal M2M communities through logical mesh • Reliable: Deterministic Latency and Jitter through tree based topology • Practical: Economies of Scale favor moving M2M overhead to Propagators Simple Secure Devices Scalable Propagator Network of Chirp Agents and Pub/Sub Routers 1 2 3 Small Data Pub/Sub Exchange Gateway/Filters BigDataIntegrators
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×