"Taming the Dragon": Zenoh's Genesis

Taming the Dragon — Ep. 1
Zenoh Genesis
CEO/CTO
angelo@zettascale.tech
Angelo Corsaro, PhD

Six Episodes
Nov 22nd — Zenoh Genesis
Nov 29th — Zenoh: Getting Started
Dec 6th — ROS/ROS2 Robot-to-Anything with Zenoh
Dec 13th — Zenoh on Microcontrollers and Low Power Networks
Dec 20th — Integrating DataBases and Messaging Protocols with Zenoh
Jan 6th — Data Flow Programming with Zenoh-Flow
All episodes run live on the given day at 10.00 CET and 18.00 CET to facilitate
participation from around the globe!

Mission
Bring to every connected human and machine the unconstrained
freedom to communicate, compute and store — anywhere, at any
scale, e
ffi
ciently and securely

Values
Zettlers embrace the Seven Knightly Virtues of Courage, Justice,
Mercy, Generosity, Faith, Nobility and Hope.

The Team
Management team with 10+
years of experience in
running SW companies / BUs
Strong thought and
technological leadership
Over 50% of the Engineers
hold a PhD
We are ~50 shiny Zettlers and
growing quickly

World Talents
Several of our key technical
sta
ff
s are well recognised
world experts
DDS was literally born in our
team — Cyclone DDS is our
brand new DDS implementation
We are the inventor of Zenoh —
the most powerful and
e
ffi
cient protocol yet!

IoT & IIoT
We were involved in building
some of the very
fi
rst IoT and
IIoT systems
In 2008 we were involved with
the Nice’s Connected
Boulevards, one of the world
fi
rst Smart Cities
In 2014 we part for the core
team that build the Fog Platform
for Barcelona

It was
Laborious…
Building these systems was
laborious
We had to stitch several
technologies together already
to make data
fl
ow end-to-end
We had to stitch a few more to
deal with data storage, etc.

MQTT
MQTT, was popularised as the
protocol for IoT, yet it supports only
client/broker architecture
Requires TCP/IP, thus making it
inconvenient for constrained
networks
The choice of topic-names has an
impact on bandwidth usage
Push Mode Only
MQTT
Broker
Pub
Pub
Pub
Sub
Sub
Sub
Sub
Sub
Pub
Pub
Sub
Sub

OMG DDS
Designed to work on a wired-
network with low message loss
Does not work o
ff
the shelf over
the Internet
Hard to scale on at Internet Scale
without breaking its semantics
Steep Learning Curve
Push Mode Only
DDS Global Data Space
...
Data
Writer
Data
Writer
Data
Writer
Data
Reader
Data
Reader
Data
Reader
Data
Reader
Data
Writer
TopicA
QoS
TopicB
QoS
TopicC
QoS
TopicD
QoS

Kafka
Designed for the Data-Centre
“Centralised” Architecture
Supports only Pull Mode

Chaos
The situation was extremely
messy, yet it seamed that
just a few of us where
bothered by it
Everyone was pushing for
the technology they had
adopted or were selling and
ignoring the challenges…
We couldn’t!

A New Beginning
We decided to take up the
challenge to design a new
protocol that could work in
the Cloud-to-Device
continuum
We set us-up for the
additional challenge to unify
data in motion and data at
rest and as a consequence
bring location transparency
to data at rest

Dragons teach us that if we want to climb high we have to do it against the wind.
Pub/Sub/Query protocol that Uni
fi
es data
in motion, data at rest and computations
from embedded microcontrollers up the
data centre
Provides location-transparent
abstractions for high performance pub/
sub and distributed queries across
heterogeneous systems
Provides universal abstractions for
cloud-to-device data-
fl
ow programming

• Pub / Sub (push)
Peer
put(room/1/temp, 19)
Peer
declare_subscriber
(room/1/temp)
Pub / Sub... but not only!

Peer
declare_subscriber
(**/temp).mode(Pull)
Peer
Peer
declare_subscriber
(room/1/temp)
pull()
• Pub / Sub (pull)

Peer
declare_subscriber
Peer
get(room/**/hum)
Peer
put(room/2/hum, 42)
(room/**)
Peer
Peer
declare_subscriber
(room/1/temp)
pull()
• Pub / Store / Get

Peer
declare_subscriber
Peer
declare_queryable
(elec_consumption)
Peer
get(elec_consumption)
Peer
get(room/**/hum)
Peer
put(room/2/hum, 42)
(room/**)
Peer
Peer
declare_subscriber
(room/1/temp)
pull()
• Get / Reply
• Pub / Store / Get

Any topology
Peer-to-peer
Clique and mesh topologies
Mesh
Peer
Peer
Peer
Peer
Peer
Clique
Peer
Peer
Peer
Peer

Any topology
Peer-to-peer
Brokered
Client
Client
Brokered
Clients communicate
through a router or a peer
Mesh
Peer
Peer
Peer
Peer
Peer
Router
Client Client Client
Clique
Peer
Peer
Peer
Peer

Any topology
Peer-to-peer
Brokered Routed
Client
Client
Routed
Routers forward data to and
from peers and clients
Brokered
Clients communicate
Mesh
Peer
Peer
Peer
Peer
Peer
Router
Client Client Client Client
Router
Router
Client
Clique
Peer
Peer
Peer
Peer

Router
Router
Any topology
Peer-to-peer
Brokered Routed
Router
Router
Router
Client
Client
Routed
Routers forward data to and
from peers and clients
Brokered
Clients communicate
Mesh
Peer
Peer
Peer
Peer
Peer
Router
Client Client Client Client
Router
Router
Client
Clique
Peer
Peer
Peer
Peer

Runs Everywhere
Written in Rust for security, safety and performance
Native libraries and API bindings for many programming
languages, e.g., Rust, C/C++, Python, REST, C# Kotlin
Supports network technologies from transport layer
down-to the data link
Available on embedded and extremely constrained
devices and networks — 5 bytes minimal overhead
Data Link
Network
Transport
Physical

Why Rust?
70% of security vulnerability are
due to memory safety issues
Rust is memory and
concurrency safe, high
performance, productive and
has a great ecosystem

24
Zenoh on Your Favourite MCU

Abstractions
Resource. A named data, in other terms a (key, value)
(e.g. home/kitchen/sensor/temp, 21.5
home/kitchen/sensor/hum, 0.67)
(e.g. home/*/sensor/air?co2>12[humidity])
Selector. An expression identifying a set of resources
(e.g. home/kitchen/sensor/*
home/**/temp
Key expression. An expression identifying a set of keys

Abstractions
Publisher. A spring of values for a key expression
Subscriber. A sink of values for a key expression
Queryable. A well of values for a key expression
(e.g. home/kitchen/sensor/temp
home/kitchen/sensor/* )
(e.g. home/kitchen/sensor/temp
home/kitchen/sensor/*)
(e.g. home/**)

Zenoh’s Abstractions Universality
Zenoh’s abstraction are universal since they allow
to express the key patterns in distributed
computing, namely:
Publish/Subscribe. Trivially supported by Zenoh’s
Publisher and Subscriber
Remote Computation. A Queryable represents a
generalised computation, since it can
transparently deal with replication and partitioning
Storage. Represented by the combination of a
Queryable and a Subscriber
Additionally all these primitives enjoy location
transparency by the virtue of being data-centric.

Pub/Sub
Subscriber
fleet/*/robot/*/position
Storage
fleet/1/**
Storage
fleet/2/**
Publisher
fleet/2/robot/1/position
Publisher
fleet/1/robot/2/position
ZZ
Subscriber
fleet/2/robot/*/position

Query
Queryable
fleet/*/camera/position
Storage
fleet/1/robot/**
Storage
fleet/2/robot/**

Query
Queryable
fleet/*/camera/position
Storage
fleet/1/robot/**
Storage
fleet/2/robot/** Get
fleet/*/*/position

Primitives
open/close — Open/Close a zenoh session.
declare_subscriber — Declares a subscriber with a user provided call-
back that will be triggered when data is available.
declare_publisher — Declares a publisher and optimise the
communication stack for repetitive publications. Notice that Zenoh
does not require a publisher in order to perform publications, this is just
an optimisation.
declare_queryable — Declares a queryable with a user provided call-
back that will be triggered whenever a query needs to be answered.

Primitives
put — puts a value for a key expression.
pull — Pulls data for a pull subscriber.
get — Issues a distributed query and returns a stream of
results. The query target, coverage and consolidation depends
on policies.

Lorem ipsum dolor sit amet
Easy to Use

Extensible
SSE
…
Zenoh Plugins
Ease integration of other technologies
…

Zenoh vs.
MQTT & Kafka
Zenoh can deliver ~60Gbps which
is 15x higher throughput than Kafka
and up to 32x than MQTT for
8Kbyte payloads
Zenoh delivers over 60x or higher
throughput than MQTT for payload
larger than 32Kbytes
Zenoh’s latency 10usec compared
to 25usec for MQTT and 75usec for
Kafka

Zenoh vs DDS
Discovery
Open Robotics performed a
comparative evaluation of
Zenoh and DDS (full report)
The results showed how
much more e
ffi
cient zenoh
discovery is compared to
DDS
1
8
www.openrobotics.org
www.openrobotics.org
Some interesting results
Discovery traffic versus application traffic
Some interesting results
Multicast traffic versus unicast and TCP traffic
(Multicast is a great way to bring down a poorly-configured network)

DDS Kafka MQTT Zenoh
Pub/Sub (Push) ✔
𐄂
✔ ✔
Pub/Sub (Pull)
𐄂
✔
𐄂
✔
Distributed Query
𐄂𐄂𐄂
✔
High Performance ✔
𐄂𐄂
✔
Constrained HW
𐄂𐄂
✔ ✔
Constrained Networks
𐄂𐄂𐄂
✔
Internet Scale
Deployment
𐄂
✔ ✔ ✔
Standard Based ✔
𐄂
✔
𐄂
QoS ✔
𐄂
✔ ✔
Peer2Peer ✔
𐄂𐄂
✔
Brokered
𐄂
✔ ✔ ✔
Routed
𐄂𐄂𐄂
✔
Feature Matrix

It's very important to inspire the next generation.
Zenoh is adopted on high visibility projects in robotics and
autonomous vehicles for on device and V2X/R2X
Next-Gen Devices built on Zenoh

Strong Endorsement #1
OpenRobotics openly taking a
position on Zenoh after
performing a comparative
evaluation against DDS
OpenRobotics calling for
contributions/fundings to have
Zenoh natively supported in ROS2
as an alternative of DDS by June
2023

Strong Endorsement #2
Zenoh selected by the ITU-T:
FGAI4AD-01 Automated driving safety data
protocol – Speci
fi
cation
At the result of a 2 years work-group evaluating
all available technologies for Intelligence
Transport Systems (ITS) communication and
having discarded DDS and MQTT among others
I n t e r n a t i o n a l T e l e c o m m u n i c a t i o n U n i o n
ITU-T Technical Report
TELECOMMUNICATION
STANDARDIZATION SECTOR
OF ITU
(29 September 2022)
Focus Group on AI for autonomous and assisted driving
(FG-AI4AD)
FGAI4AD-01
Automated driving safety data protocol
– Specification

Zenoh’s Key Highlights
Most wire/power/memory ef
fi
cient protocol in the market to
provide connectivity to extremely constrained targets
Supports push and pull pub/sub along with distributed queries
Resource keys are represented as integers on the wire, these
integer are local to a session => good for wire ef
fi
ciency
Supports for peer-to-peer and routed communication.
Support for zero-copy.
Ordered reliable data delivery and fragmentation.
Minimal wire overhead for user data is 5 bytes
Data Link
Network
Transport
Physical

Concluding Remarks
Zenoh has been designed to overcome the
limitations of existing protocols and provide
with a uni
fi
ed abstraction from the data-
centre to the micro-controller for dealing with
data in motion and data at rest
It is written in Rust for security and
performance
It has a vibrant and quickly growing community
Its has impressive performance

Patience, persistence and perspiration
make an unbeatable combination for
success.
Thank You

"Taming the Dragon": Zenoh's Genesis

Recommended

Recommended

More Related Content

Similar to "Taming the Dragon": Zenoh's Genesis

Similar to "Taming the Dragon": Zenoh's Genesis (20)

More from ZettaScaleTechnology

More from ZettaScaleTechnology (7)

Recently uploaded

Recently uploaded (20)

"Taming the Dragon": Zenoh's Genesis