Taming the Dragon — Ep. 2
Getting started
Luca Cominardi, PhD
Product conductor
luca@zettascale.tech
Six Episodes
Nov 22nd — Zenoh Genesis
Nov 29th — Zenoh: Getting Started
Dec 6th — ROS/ROS2 Robot-to-Anything with Zenoh
Dec 13th — Zenoh on Microcontrollers and Low Power Networks
Dec 20th — Integrating DataBases and Messaging Protocols with Zenoh
Jan 6th — Data Flow Programming with Zenoh-Flow
All episodes run live on the given day at 10.00 CET and 18.00 CET to facilitate
participation from around the globe!
Uni
fi
es data in motion, data at rest and
computations from embedded
microcontrollers up to data centres
Provides a location-transparent API for high
performance pub/sub and distributed queries
across heterogeneous systems
Facilitates geo-distributed storage and
integration with third party technology in a
plug-and-play fashion
Runs everywhere
Native libraries and API bindings for many
programming languages
Over various network technologies: from
transport layer to data link
On embedded and constrained devices
Data Link
Network
Transport
Physical
…
Any topology
Peer-to-peer
Clique and mesh topologies
Brokered
Clients communicate
through a router or a peer
Routed
Routers forward data to and
from peers and clients
Clique
Mesh
Router
Router
Router
Router
Peer
Peer
Peer
Peer
Peer
Peer
Peer Peer
Peer
Client
Client
Client
Client
Brokered
Routed
SSE
…
Extensible
Enhance zenoh with plugins
Interact with other technologies
…
Get your API
COTS and embedded
devices
x86 and ARM targets
Native libraries and
bindings
Zenoh API
Rust
• Native library
• Async support
Python
• Binding based on
Rust library
C
• Binding based on
Rust library
• Native library with
zenoh-pico
Coming soon…
API bindings:
• C++
• Java
• …
https://github.com/eclipse-zenoh/
zenoh
Rust
# Stable
[dependencies]
zenoh = "0.6.0-beta.1"
# Nightly
[dependencies]
zenoh = { git = "https://github.com/
eclipse-zenoh/zenoh.git" }
https://github.com/eclipse-zenoh/
zenoh-python
Python
$ pip install eclipse-zenoh
$ pip install eclipse-zenoh-nightly
Let’s write our first Zenoh app
Distribute values published on a given key expression to all the
subscribers matching that speci
fi
c key expression
Pub/sub, what for?
MATCHING
NOT MATCHING
(room/kitchen/temp, 21.5)
room/kitchen/*
home/alarm/status
room/kitchen/temp
room/**/temp
MATCHING
MATCHING
PUB
SUBS
The temp now is: 21.5
Python examples: https://github.com/eclipse-zenoh/
zenoh-python/tree/master/examples
Zenoh carries some
metadata information
related to the encoding of
the value to facilitate
decoding
Encoding
Python examples: https://github.com/eclipse-zenoh/
zenoh-python/tree/master/examples
Peer-to-peer is a
communication model
where zenoh peers talk
directly with each other
without the intervention of
any zenoh infrastructure
What is
peer-to-peer?
Zenoh performs scouting
to discover who is around,
e.g. other routers or peers
Scouting can be based on
multicast, DNS, etc.
What is
scouting?
Peer
multicast
network
Who is
there?
It’s XXX!
Reach me at IP A.
It’s YYY!
Reach me at IP B.
Router
Client
Request on-demand some values via a distributed query
matching a given key expression
Query/reply, what for?
MATCHING
NOT MATCHING
room/bedroom/temp
home/alarm/status
room/kitchen/temp
room/bathroom/temp
MATCHING
MATCHING
GET
QUERYABLES
What is the
temp now?
room/*/temp
The temp now is: 25
The temp now is: 21
The temp now is: 27
Rust examples: https://github.com/eclipse-zenoh/
zenoh/tree/master/examples
The temp now is: 21.5
What is the
latest temp?
Latest temp
is: 21.5
Store data as soon as it is published and allow to retrieve it on-
demand. It is a combination of a subscriber and a queryable.
Distributed storage, what for?
(room/kitchen/temp, 21.5)
PUB
room/*/temp
SUB QUERYABLE
room/*/temp
GET
room/*/temp
Rust examples: https://github.com/eclipse-zenoh/
zenoh/tree/master/examples
Subscriber
Queryable
All replies produced by the matching queryables are sent
back to the querier
routed?
routing
Stable
Nightly
Install zenohd from Deb
Stable
Install zenohd from Eclipse
Install zenohd from source
$ git clone https://github.com/eclipse-zenoh/
zenoh.git
$ cd zenoh
$ cargo build --release --all-targets
# The compiled zenohd can be found at:
# ./target/release/zenohd
Install zenohd from Docker
$ docker pull eclipse/zenoh:latest
$ docker pull eclipse/zenoh:master
$ echo "deb [trusted=yes] https://
download.eclipse.org/zenoh/zenoh/latest/ /" |
sudo tee -a /etc/apt/sources.list > /dev/null
$ sudo apt update
$ sudo apt install zenoh
Stable Nightly
https://download.eclipse.org/zenoh/zenoh/latest/
Peer
Local comm stays peer-to-peer
Remote comm via the zenoh router
All comm via the zenoh router
Client
Router
Peer
Peer
Peer
Peer
Router
Client
Client
Client Client
Changing the communication model in Zenoh does NOT require
changing the application logic nor the usage of a di
ff
erent API
It is enough to:
• Have at least one zenoh router up and running in the system
• Set the mode to “peer” / “client” in the application
From peer-to-peer to routed
let mut conf = Config::default();
config.set_mode(Some(WhatAmI::Client));
// Or WhatAmI::Peer
config.connect.endpoints.push(
"tcp/localhost:7447".parse().unwrap());
config.listen.endpoints.push(
"tcp/localhost:7448".parse().unwrap());
config.scouting.multicast.set_enabled(Some(false));
config = zenoh.Config.from_json5({
"mode": "client", // Or "peer"
"connect": { "endpoints":
[ "tcp/localhost:7447" ] },
"listen": { "endpoints":
[ "tcp/localhost:7449" ] },
"scouting": {
"multicast": { "enabled": false } }
})
Con
fi
g
fi
le
Rust
Python
{
"mode": “client", // Or "peer"
"connect": { "endpoints":
[ "tcp/localhost:7447" ] },
"listen": { "endpoints":
[ "tcp/localhost:7449" ] },
"scouting": {
"multicast": { "enabled": false } }
}
-l, --listen <ENDPOINT>...
Endpoints to listen on for incoming sessions.
Only peers and routers listen for incoming
sessions.
-e, --connect <ENDPOINT>...
Endpoints to connect to. Clients, peers, and
routers will try to open a session with the
provided list of endpoints.
--no-multicast-scouting
Disable multicast scouting for automatic
discovery and connection in a LAN.
No changes to
the code with a
config file!
Con
fi
guration from CLI Con
fi
guration from
fi
le
-l, --listen <ENDPOINT>...
An endpoint on which this router will listen
on for incoming sessions. Repeat this option
to open several listeners.
-e, --connect <ENDPOINT>...
An endpoint this router will try to
connect to. Repeat this option to connect to
several routers.
(router 1)$ zenohd -l tcp/localhost:7447
(router 2)$ zenohd -l tcp/localhost:7448
--rest-http-port 8001 -e tcp/localhost:7447
// conf1.json5
{
mode: "router",
listen: { endpoints: [ "tcp/localhost:7447" ] },
plugins: { rest: { http_port: 8000, }, },
}
// conf2.json5
{
mode: "router",
listen: { endpoints: [ "tcp/localhost:7448" ] },
connect: { endpoints: [ "tcp/localhost:7447" ] },
plugins: { rest: { http_port: 8001, }, },
}
(router 1)$ zenohd -c conf1.json5
(router 2)$ zenohd -c conf2.json5
Default config: https://github.com/eclipse-zenoh/zenoh/blob/master/DEFAULT_CONFIG.json5
$ zenohd --help
If the router goes down,
only the client subscriber
will stop receiving data
If the router comes back,
the client will reconnect
and resume receiving data
Example 1:
Pub/Sub
Router
Peer
Peer
Client
PUB
SUB
SUB
Connect two routers: one
on a local machine and
one on the cloud
Queries received from the
routed on the cloud are
forwarded downstream
Example 2:
Query/Reply
Router
Peer Peer
Router
http://demo.zenoh.io:8000/room/**
STORAGE QUERYABLE
GET
Plugins and backends
• Carries DDS tra
ffi
c on top of Zenoh
• Transparent to DDS participants
• Maps Zenoh to HTTP methods
• It support HTML5 Server Sent Events
• Built-in zenoh router
Webserver
DDS
MQTT
https://github.com/eclipse-zenoh/zenoh-plugin-mqtt
REST
• Serves web content (e.g., pages) that
is stored in Zenoh storages
• Carries MQTT tra
ffi
c on top of Zenoh
• Transparent to MQTT clients
https://github.com/eclipse-zenoh/zenoh-plugin-dds
Plugins
https://github.com/eclipse-zenoh/zenoh-plugin-
webserver
• A volatile key-value store
• Useful for last-value fast caching
• Built-in zenoh router
• A geo-distributed key-value store
• Alignment of decentralised storages
• Built-in zenoh router
• Multiple backends
In-memory
Filesystem
Storage plugin
• A persistent key-value store
• Useful for serving large
fi
les (e.g. CDN)
Storage and backends
https://github.com/eclipse-zenoh/zenoh-backend-filesystem
• A persistent database for time series
• Useful for performing data analytics
Amazon S3
And many more to come…
In
fl
uxDB
• Out-of-the box integration with cloud
storage
https://github.com/eclipse-zenoh/zenoh-backend-influxdb
Storage and backends
• A persistent key-value store
• Useful for last-value persistence
RocksDB
https://github.com/eclipse-zenoh/zenoh-backend-rocksdb
https://github.com/eclipse-zenoh/zenoh-backend-s3
Don’t forget to visit
Zenoh’s website…
https://zenoh.io/
… Zenoh’s GitHub …
https://github.com/
eclipse-zenoh/zenoh
… and to join Zenoh’s
Discord server!
https://discord.gg/
2GJ958VuHs
Luca Cominardi, PhD
Product conductor
Thank You

"Taming the Dragon": Get Started with Zenoh

  • 1.
    Taming the Dragon— Ep. 2 Getting started Luca Cominardi, PhD Product conductor luca@zettascale.tech
  • 2.
    Six Episodes Nov 22nd— Zenoh Genesis Nov 29th — Zenoh: Getting Started Dec 6th — ROS/ROS2 Robot-to-Anything with Zenoh Dec 13th — Zenoh on Microcontrollers and Low Power Networks Dec 20th — Integrating DataBases and Messaging Protocols with Zenoh Jan 6th — Data Flow Programming with Zenoh-Flow All episodes run live on the given day at 10.00 CET and 18.00 CET to facilitate participation from around the globe!
  • 3.
    Uni fi es data inmotion, data at rest and computations from embedded microcontrollers up to data centres Provides a location-transparent API for high performance pub/sub and distributed queries across heterogeneous systems Facilitates geo-distributed storage and integration with third party technology in a plug-and-play fashion
  • 4.
    Runs everywhere Native librariesand API bindings for many programming languages Over various network technologies: from transport layer to data link On embedded and constrained devices Data Link Network Transport Physical …
  • 5.
    Any topology Peer-to-peer Clique andmesh topologies Brokered Clients communicate through a router or a peer Routed Routers forward data to and from peers and clients Clique Mesh Router Router Router Router Peer Peer Peer Peer Peer Peer Peer Peer Peer Client Client Client Client Brokered Routed
  • 6.
    SSE … Extensible Enhance zenoh withplugins Interact with other technologies …
  • 7.
  • 8.
    COTS and embedded devices x86and ARM targets Native libraries and bindings Zenoh API Rust • Native library • Async support Python • Binding based on Rust library C • Binding based on Rust library • Native library with zenoh-pico Coming soon… API bindings: • C++ • Java • …
  • 9.
    https://github.com/eclipse-zenoh/ zenoh Rust # Stable [dependencies] zenoh ="0.6.0-beta.1" # Nightly [dependencies] zenoh = { git = "https://github.com/ eclipse-zenoh/zenoh.git" }
  • 10.
    https://github.com/eclipse-zenoh/ zenoh-python Python $ pip installeclipse-zenoh $ pip install eclipse-zenoh-nightly
  • 11.
    Let’s write ourfirst Zenoh app
  • 12.
    Distribute values publishedon a given key expression to all the subscribers matching that speci fi c key expression Pub/sub, what for? MATCHING NOT MATCHING (room/kitchen/temp, 21.5) room/kitchen/* home/alarm/status room/kitchen/temp room/**/temp MATCHING MATCHING PUB SUBS The temp now is: 21.5
  • 13.
  • 14.
    Zenoh carries some metadatainformation related to the encoding of the value to facilitate decoding Encoding
  • 15.
  • 16.
    Peer-to-peer is a communicationmodel where zenoh peers talk directly with each other without the intervention of any zenoh infrastructure What is peer-to-peer?
  • 17.
    Zenoh performs scouting todiscover who is around, e.g. other routers or peers Scouting can be based on multicast, DNS, etc. What is scouting? Peer multicast network Who is there? It’s XXX! Reach me at IP A. It’s YYY! Reach me at IP B. Router Client
  • 18.
    Request on-demand somevalues via a distributed query matching a given key expression Query/reply, what for? MATCHING NOT MATCHING room/bedroom/temp home/alarm/status room/kitchen/temp room/bathroom/temp MATCHING MATCHING GET QUERYABLES What is the temp now? room/*/temp The temp now is: 25 The temp now is: 21 The temp now is: 27
  • 19.
  • 20.
    The temp nowis: 21.5 What is the latest temp? Latest temp is: 21.5 Store data as soon as it is published and allow to retrieve it on- demand. It is a combination of a subscriber and a queryable. Distributed storage, what for? (room/kitchen/temp, 21.5) PUB room/*/temp SUB QUERYABLE room/*/temp GET room/*/temp
  • 21.
  • 22.
    All replies producedby the matching queryables are sent back to the querier
  • 26.
  • 27.
    Stable Nightly Install zenohd fromDeb Stable Install zenohd from Eclipse Install zenohd from source $ git clone https://github.com/eclipse-zenoh/ zenoh.git $ cd zenoh $ cargo build --release --all-targets # The compiled zenohd can be found at: # ./target/release/zenohd Install zenohd from Docker $ docker pull eclipse/zenoh:latest $ docker pull eclipse/zenoh:master $ echo "deb [trusted=yes] https:// download.eclipse.org/zenoh/zenoh/latest/ /" | sudo tee -a /etc/apt/sources.list > /dev/null $ sudo apt update $ sudo apt install zenoh Stable Nightly https://download.eclipse.org/zenoh/zenoh/latest/
  • 28.
    Peer Local comm stayspeer-to-peer Remote comm via the zenoh router All comm via the zenoh router Client Router Peer Peer Peer Peer Router Client Client Client Client
  • 29.
    Changing the communicationmodel in Zenoh does NOT require changing the application logic nor the usage of a di ff erent API It is enough to: • Have at least one zenoh router up and running in the system • Set the mode to “peer” / “client” in the application From peer-to-peer to routed
  • 30.
    let mut conf= Config::default(); config.set_mode(Some(WhatAmI::Client)); // Or WhatAmI::Peer config.connect.endpoints.push( "tcp/localhost:7447".parse().unwrap()); config.listen.endpoints.push( "tcp/localhost:7448".parse().unwrap()); config.scouting.multicast.set_enabled(Some(false)); config = zenoh.Config.from_json5({ "mode": "client", // Or "peer" "connect": { "endpoints": [ "tcp/localhost:7447" ] }, "listen": { "endpoints": [ "tcp/localhost:7449" ] }, "scouting": { "multicast": { "enabled": false } } }) Con fi g fi le Rust Python { "mode": “client", // Or "peer" "connect": { "endpoints": [ "tcp/localhost:7447" ] }, "listen": { "endpoints": [ "tcp/localhost:7449" ] }, "scouting": { "multicast": { "enabled": false } } } -l, --listen <ENDPOINT>... Endpoints to listen on for incoming sessions. Only peers and routers listen for incoming sessions. -e, --connect <ENDPOINT>... Endpoints to connect to. Clients, peers, and routers will try to open a session with the provided list of endpoints. --no-multicast-scouting Disable multicast scouting for automatic discovery and connection in a LAN. No changes to the code with a config file!
  • 31.
    Con fi guration from CLICon fi guration from fi le -l, --listen <ENDPOINT>... An endpoint on which this router will listen on for incoming sessions. Repeat this option to open several listeners. -e, --connect <ENDPOINT>... An endpoint this router will try to connect to. Repeat this option to connect to several routers. (router 1)$ zenohd -l tcp/localhost:7447 (router 2)$ zenohd -l tcp/localhost:7448 --rest-http-port 8001 -e tcp/localhost:7447 // conf1.json5 { mode: "router", listen: { endpoints: [ "tcp/localhost:7447" ] }, plugins: { rest: { http_port: 8000, }, }, } // conf2.json5 { mode: "router", listen: { endpoints: [ "tcp/localhost:7448" ] }, connect: { endpoints: [ "tcp/localhost:7447" ] }, plugins: { rest: { http_port: 8001, }, }, } (router 1)$ zenohd -c conf1.json5 (router 2)$ zenohd -c conf2.json5 Default config: https://github.com/eclipse-zenoh/zenoh/blob/master/DEFAULT_CONFIG.json5 $ zenohd --help
  • 32.
    If the routergoes down, only the client subscriber will stop receiving data If the router comes back, the client will reconnect and resume receiving data Example 1: Pub/Sub Router Peer Peer Client PUB SUB SUB
  • 33.
    Connect two routers:one on a local machine and one on the cloud Queries received from the routed on the cloud are forwarded downstream Example 2: Query/Reply Router Peer Peer Router http://demo.zenoh.io:8000/room/** STORAGE QUERYABLE GET
  • 34.
  • 35.
    • Carries DDStra ffi c on top of Zenoh • Transparent to DDS participants • Maps Zenoh to HTTP methods • It support HTML5 Server Sent Events • Built-in zenoh router Webserver DDS MQTT https://github.com/eclipse-zenoh/zenoh-plugin-mqtt REST • Serves web content (e.g., pages) that is stored in Zenoh storages • Carries MQTT tra ffi c on top of Zenoh • Transparent to MQTT clients https://github.com/eclipse-zenoh/zenoh-plugin-dds Plugins https://github.com/eclipse-zenoh/zenoh-plugin- webserver
  • 36.
    • A volatilekey-value store • Useful for last-value fast caching • Built-in zenoh router • A geo-distributed key-value store • Alignment of decentralised storages • Built-in zenoh router • Multiple backends In-memory Filesystem Storage plugin • A persistent key-value store • Useful for serving large fi les (e.g. CDN) Storage and backends https://github.com/eclipse-zenoh/zenoh-backend-filesystem
  • 37.
    • A persistentdatabase for time series • Useful for performing data analytics Amazon S3 And many more to come… In fl uxDB • Out-of-the box integration with cloud storage https://github.com/eclipse-zenoh/zenoh-backend-influxdb Storage and backends • A persistent key-value store • Useful for last-value persistence RocksDB https://github.com/eclipse-zenoh/zenoh-backend-rocksdb https://github.com/eclipse-zenoh/zenoh-backend-s3
  • 38.
    Don’t forget tovisit Zenoh’s website… https://zenoh.io/ … Zenoh’s GitHub … https://github.com/ eclipse-zenoh/zenoh … and to join Zenoh’s Discord server! https://discord.gg/ 2GJ958VuHs
  • 39.
    Luca Cominardi, PhD Productconductor Thank You