Orion Context Broker 20221220

Managing Context Information at
Large Scale: Introduction
Fermín Galán Márquez - fermin.galanmarquez@telefonica.com
(Reference Orion Context Broker version: 3.8.0)

• Context Management in FIWARE
• Orion Context Broker
• Creating and pulling data
• Pushing data and notifications
• Batch operations
2
Introduction

Being “Smart” requires first being “Aware”
Boiler
• Manufacturer
• Last revision
• Product id
• temperature
Users
• Name-Surname
• Birthday
• Preferences
• Location
Flowerpot
• Humidity
• Watering plan
Context Information
Application
3
• Implementing a Smart Application requires gathering and
managing context information, referring to values of
attributes characterizing relevant entities
• There are many application domains where management of
context info is relevant: smart home, smart agrifood, smart
industry, smart logistics, smart cities
standard API

4
Ship
• Company
• Speed
• Location
• …
Truck
• Driver
• Location
• …
Container
• Dimension
• Temperature
• …
Context Information
Application
standard API

Bus
• Location
• No. passengers
• Driver
• Licence plate
Citizen
• Name-Surname
• Birthday
• Preferences
• Location
Shop
• Location
• Business name
• Franchise
• offerings
Context Information
Application
5
standard API

Context Information Management in Smart Cities
• Systems dealing with management of city services or third-
party apps (subject to access control policies) can consume
and produce context info
• Overall city governance can rely on context information
available (real-time and historic) to monitor KPIs and run
BigData analysis
6
City Services
Bus
• Location
• No. passengers
• Driver
• Licence plate
Citizen
• Name-Surname
• Birthday
• Preferences
• Location
Shop
• Location
• Business name
• Franchise
• offerings
Context Information
City Governance System
City Services
Third-party Apps

Different sources of context need to be handle
• Context information may come from many sources:
– Existing systems
– Users, through mobile apps
– Sensor networks
• Source of info for a given entity may vary over time
Place = “X”, temperature = 30º
What’s the current temperature in place “X”?
A sensor in a
pedestrian street
The Public Bus Transport
Management system
A person from his smartphone
It’s too hot!
7
standard API

Different sources of context need to be handle
• Context information may come from many sources:
– Existing systems
– Users, through mobile apps
– Sensor networks
• Source of info for a given entity may vary over time
8
Street = “X”, traffic = high
What’s the current traffic in
street “X”?
Standard API
A sensor in a pedestrian street
The Public Bus Transport
Management system Citizen’s car app or
smartphone
Notify me the changes of
traffic in street “X”

A non-intrusive approach is required
• Capable to integrate with existing or future systems without impact
in their architectures, but bringing a common context information
hub
• Info about attributes of one entity may come from different
systems, which work either as Context Producers or Context
Providers
Application/Service
Standard API
System A System B
Context Producer Context Provider
attribute “location” attribute “driver”
9

Context Management in FIWARE
• The FIWARE Context Broker GE implements the NGSI API: a
simple yet powerful standard API for managing Context
information complying with the requirements of a smart
enabled system
• The FIWARE NGSI API is Restful: any web/backend
programmer gets quickly used to it
Application/Service
Context Broker
NGSI API
Boiler
• Manufacturer
• Last revision
• Product id
• temperature
Users
• Name-Surname
• Birthday
• Preferences
• Location
Flowerpot
• Humidity
• Watering plan
10

FIWARE NGSI: “The SNMP for IoT”
• Capturing data from, or Acting upon, IoT devices becomes
as easy as to read/change the value of attributes linked to
context entities using a Context Broker
Context Broker
NGSI API
NGSI API
GET /v2/entities/lamp1/attrs/presenceSensor
PUT /v2/entities/lamp1/attrs/status/value
“light on”
Setting up the value of attribute
“status” to “light on” triggers
execution of a function in the IoT
device that switches the lamp on
Issuing a get operation on the
“presenceSensor” attribute
enables the application to get
info about presence of people
near the lamp
11

Connecting to the Internet of Things
• Capturing data from, or Acting upon, IoT devices becomes
as easy as to read/change the value of attributes linked to
context entities using a Context Broker
Context Broker
GET /v2/entities/lamp1/attrs/humidity
PUT /v2/entities/lamp1/attrs/status/value
“watering”
Setting up the value of attribute
“status” to “watering” triggers
execution of a function in the IoT
device that waters the plant
Issuing a get operation on the
“humidity” attribute enables the
application to find out whether
the plant has to be watered
12
NGSI API
NGSI API

• Main functions:
– Context management
– Context availability management (advanced topic) (*)
• HTTP and REST-based
– JSON payload support
• Context in NGSI is based in an entity-attribute model:
Attributes
• Name
• Type
• Value
Entity
• EntityId
• EntityType
1 n
has
13
Orion Context Broker
(*) With some limitations in current Orion version, see https://github.com/telefonicaid/fiware-
orion/blob/master/doc/manuals/orion-api.md#registration-implementation-differences

Orion Context Broker
Context
Producers
Context
Consumers
subscriptions
update
query
notify
notify
update
update
DB
1026
1026
14
Orion Context Broker in a nutshell

GET <cb_host>:1026/version
{
"orion" : {
"version" : "3.8.0",
"uptime" : "0 d, 0 h, 0 m, 5 s",
"git_hash" : "...",
"compile_time" : "nodate",
"compiled_by" : "fermin",
"compiled_in" : "centollo",
"release_date" : "nodate",
"machine" : "x86_64",
"doc" : "https://fiware-orion.readthedocs.org/en/3.8.0/",
"libversions": …
}
}
15
Orion Context Broker – check health

16
Orion Context Broker Basic Operations
Entities
• GET /v2/entities
• Retrieve all entities
• POST /v2/entities
• Creates an entity
• GET /v2/entities/{entityID}
• Retrieves an entity
• [PUT|PATCH|POST] /v2/entities/{entityID}
• Updates an entity (different “flavors”)
• DELETE /v2/entities/{entityID}
• Deletes an entity

17
Orion Context Broker Basic Operations
Attributes
• GET /v2/entities/{entityID}/attrs/{attrName}
• Retrieves an attribute’s data
• PUT /v2/entities/{entityID}/attrs/{attrName}
• Updates an attribute’s data
• DELETE /v2/entities/{entityID}/attrs/{attrName}
• Deletes an attribute
• GET /v2/entities/{entityID}/attrs/{attrName}/value
• Retrieves an attribute’s value
• PUT /v2/entities/{entityID}/attrs/{attrName}/value
• Updates an attribute’s value

Context Broker operations: create & pull data
• Context Producers publish data/context elements by invoking the update
operations on a Context Broker.
• Context Consumers can retrieve data/context elements by invoking the query
operations on a Context Broker
Context Consumer
query
Context Producer
update
Context Broker
18

Quick Usage Example: Car Create
201 Created
19
POST <cb_host>:1026/v2/entities
Content-Type: application/json
...
{
"id": "Car1",
"type": "Car",
"speed": {
"type": "Float",
"value": 98
}
}
You can add "?options=upsert" to
the URL. In that case, the entity is
updated if already exists.

Quick Usage Example: Car Speed Update (1)
PUT <cb_host>:1026/v2/entities/Car1/attrs/speed
...
{
"type": "Float",
"value": 110
}
204 No Content
…
20
In the case of id ambiguity, you can use
"?type=Car" to specify entity type

Quick Usage Example: Car Speed Query (1)
200 OK
...
{
"type": "Float",
"value": 110,
"metadata": {}
}
21
You can get all the attributes of the entity using the
entity URL:
GET/v2/entities/Car1/attrs
GET <cb_host>:1026/v2/entities/Car1/attrs/speed

Quick Usage Example: Car Speed Update (2)
PUT <cb_host>:1026/v2/entities/Car1/attrs/speed/value
Content-Type: text/plain
...
115
204 No Content
…
22

Quick Usage Example: Car Speed Query (2)
23
200 OK
...
115.000000
GET <cb_host>:1026/v2/entities/Car1/attrs/speed/value
Accept: text/plain

201 Created
...
Quick Usage Example: Room Create (1)
...
{
"id": "Room1",
"type": "Room",
"temperature": {
"type": "Float",
"value": 24
},
"pressure": {
"type": "Integer",
"value": 718
}
}
24

204 No Content
…
Quick Usage Example: Room Update (1)
25
PATCH <cb_host>:1026/v2/entities/Room1/attrs
...
{
"temperature“: {
"type": "Float",
"value": 25
},
"pressure": {
"type": "Integer",
"value": 720
}
}

Quick Usage Example: Room Query (1)
26
200 OK
...
{
"pressure": {
"type": "Integer",
"value": 720,
"metadata": {}
},
"temperature": {
"type": "Float",
"value": 25,
"metadata": {}
}
}
GET <cb_host>:1026/v2/entities/Room1/attrs

Quick Usage Example: Room Query (2)
27
200 OK
...
{
"pressure": 720,
"temperature": 25
}
GET <cb_host>:1026/v2/entities/Room1/attrs?options=keyValues

201 Created
...
Quick Usage Example: Room Create (2)
...
{
"id": "Room2",
"type": "Room",
"temperature": {
"type": "Float",
"value": 29
},
"pressure": {
"type": "Integer",
"value": 730
}
}
28

Quick Usage Example: Filters (1)
29
200 OK
...
[
{
"id": "Room2",
"pressure": 730,
"temperature": 29,
"type": "Room"
}
]
GET <cb_host>:1026/v2/entities?options=keyValues&q=temperature>27

Quick Usage Example: Filters (2)
30
200 OK
...
[
{
"id": "Room1",
"pressure": 720,
"temperature": 25,
"type": "Room"
}
]
GET <cb_host>:1026/v2/entities?options=keyValues&q=pressure==715..725
The full description of the Simple
Query Language for filtering can be
found in the NGSIv2 Specification
document

Context Broker operations: push data
• Context Consumers can subscribe to receive context information that satisfy
certain conditions using the subscribe operation. Such subscriptions may
have an expiration time.
• The Context Broker notifies updates on context information to subscribed
Context Consumers by invoking the notify operation they export
subId = subscribeContext (consumer, expr, expiration)
Context Consumer
notify (subId, data/context)
Context Broker
Application
31

Quick Usage Example: Subscription
POST <cb_host>:1026/v2/subscriptions
…
{
"subject": {
"entities": [
{
"id": "Room1",
"type": "Room"
}
],
"condition": {
"attrs": [ "temperature" ]
}
},
"notification": {
"http": {
"url": "http://<host>:<port>/publish"
},
"attrs": [ "temperature" ]
},
"expires": "2026-04-05T14:00:00.000Z"
}
201 Created
Location: /v2/subscriptions/51c0ac9ed714fb3b37d7d5a8
...
32

25
19
Quick Usage Example: Notification
33

POST /publish HTTP/1.1
Content-type: application/json; charset=utf-8
Ngsiv2-AttrsFormat: normalized
…
{
"subscriptionId": "574d720dbef222abb860534a",
"data": [
{
"id": "Room1",
"type": "Room",
"temperature": {
"type": "Float",
"value": 19,
"metadata": {}
}
}
]
}
Quick Usage Example: Notification
34

List existing subscriptions
35
200 OK
…
[{
"id": " 51c0ac9ed714fb3b37d7d5a8 ",
"expires": "2026-04-05T14:00:00.000Z",
"status": "active",
"subject": {
"entities": [{
"id": "Room1",
"type": "Room"
}],
"condition": {
"attrs": ["temperature"]
}
},
"notification": {
"timesSent": 3,
"lastNotification": "2016-05-31T11:19:32.000Z",
"lastSuccess": "2016-05-31T11:19:32.000Z",
"attrs": ["temperature"],
"attrsFormat": "normalized",
"http": {
"url": "http://localhost:1028/publish"
}
}
}]
The full description of the
subscription object (including all
its fields) can be found in the
NGSIv2 Specification
GET <cb_host>:1026/v2/subscriptions

Orion Context Broker batch operations
36
• Batch query and batch update
• They are equivalent in functionality to previously described RESTful
operations
• All them use POST as verb and the /v2/op URL prefix, including
operation parameters in the JSON payload
• They implement extra functionality that cannot be achieved with
RESTful operations, e.g. to create several entities with the same
operation
• They are not a substitute but a complement to RESTful operations

201 Created
...
Batch Operation Example: Create Several Rooms
POST <cb_host>:1026/v2/op/update
Conten-Type: application/json
...
{
"actionType": "append",
"entities": [
{
"type": "Room",
"id": "Room3",
"temperature": {
"value": 21.2,
"type": "Float"
},
"pressure": {
"value": 722,
"type": "Integer"
}
},
…
37
…
{
"type": "Room",
"id": "Room4",
"temperature": {
"value": 31.8,
"type": "Float"
},
"pressure": {
"value": 712,
"type": "Integer"
}
}
]
}

How to get Orion? (Docker containers)
38
• Assuming docker is installed in your system
• Documentation in https://github.com/telefonicaid/fiware-
orion/tree/develop/docker
• Quick guide
git clone https://github.com/telefonicaid/fiware-orion.git
cd fiware-orion/docker
sudo docker-compose up
• That’s all!
– curl localhost:1026/version
• Installing Orion docker in less than 1 minute
– https://www.youtube.com/watch?v=6taR7e20H9U

Would you like to play with this?
39
• Have a look to the FIWARE Reference Tutorial
application
– git clone https://github.com/Fiware/tutorials.TourGuide-
App.git
– cd tutorials.TourGuide-App/
– docker-compose up orion
– curl localhost:1026/version
• Self-explanatory README.md at root directory
• Open a Postman session and rock and roll
– Postman collection:
https://github.com/Fiware/tutorials.TourGuide-
App/blob/develop/contrib/CampusParty2016.postman_coll
ection

NGSI Go
The NGSI Go is a command-line interface supporting FIWARE NGSIv2
API and Orion Admin API, which simplifies syntax. It's a powerful tool
and easy to use. It has different commands for Orion as shown:
– NGSI commands to manage NGSI Entity, subscription, registration
– Admin commands for FIWARE Orion
– Print version
– Copy or remove entities at once
– Create template of subscription or registration
– Notification receiver
– Integrated access token management for Keystone (Thinking Cities
platform), Keyrock and so on.
• Source code: https://github.com/lets-fiware/ngsi-go
• How to install: https://github.com/lets-fiware/ngsi-go#install
• Full documentation: https://ngsi-go.letsfiware.jp/
*NGSI Go is a third-party open-source software made by FIWARE community.
40

Would you like to know more?
41
• References
– This presentation, available at
https://github.com/telefonicaid/fiware-
orion#introductory-presentations
– NGSIv2 Specification
• https://github.com/telefonicaid/fiware-
orion/blob/master/doc/manuals/orion-api.md
– Documentation at ReadTheDocs
• https://fiware-orion.readthedocs.io
– Orion support through StackOverflow
• Ask your questions using the “fiware-orion” tag
• Look for existing questions at
http://stackoverflow.com/questions/tagged/fiware-orion

Managing Context Information at
Large Scale: Advanced Topics
Fermín Galán Márquez - fermin.galanmarquez@telefonica.com
(Reference Orion Context Broker version: 3.8.0)

43
Orion functionality
43
43
Pagination
Metadata
Compound attribute/metadata values
Type browsing
Geo-location
Query filters
DateTime support
Custom notifications
Notification status
Attribute/metadata filtering
Special attribute/metadata
Registrations & context providers
Multitenancy
Creating & pulling data
Pushing data
Subscriptions & Notifications
Batch operations
Transient entities
Oneshot subscriptions
Subscription
improvements
Unrestricted text attributes
Flow control
MQTT notifications
Update operators
Subscription auto-disabling
Subscription based in alteration type
Covered subscriptions

• Pagination
• Metadata
• Compound attribute/metadata values
• Type browsing
• Geo-location
• Update operators
• Query filters
• DateTime support
• Transient entities
• Unrestricted text attributes
• Flow control
• Custom notifications
• Notification status
• Oneshot subscriptions
• Subscription auto-disabling
• Subscriptions based in alteration type
• Covered subscriptions
• MQTT notifications
• Subscription improvements
• Attribute/metadata filtering and special
attribute/metadata
• Registrations & context providers
• Multitenancy
• Service paths
• CORS
• Notifying services in private networks
44
Advanced Features

• Pagination helps clients organize query and
discovery requests with a large number of
responses.
• Three URI parameters:
– limit
• Number of elements per page (default: 20, max: 1000)
– offset
• Number of elements to skip (default: 0)
– count (option)
• Returns total elements (default: not return)
45
Pagination

• Example, querying the first 100 entries:
– GET <orion_host>:1026/v2/entities?limit=100&options=count
• The first 100 elements are returned, along with the
following header in the response:
– Fiware-Total-Count: 322
• Now we now there are 322 entities, we can keep querying
the broker for them:
– GET <orion_host>:1026/v2/entities?offset=100&limit=100
46
Pagination

• By default, results are ordered by entity creation date
• This behavior can be overridden using orderBy URI parameter
– A comma-separated list of attributes (including system/builtin
attributes), id (for entity ID) and type (for entity type). Results are
ordered by the first field. On ties, the results are ordered by the
second field and so on. A "!" before the field name means that the
order is reversed.
• Example: get the first 20 entities ordered by temp in ascending
order, then humidity in descending order
GET <orion_host>:1026/v2/entities?limit=20&offset=0&orderBy=temp,!humidity
• dateCreated and dateModified can be used to ordering by
entity creation and modification date, respectively
47
Pagination

• Users may attach metadata to attributes
• Reserved metadata: dateCreated, dateModified, previousValue,
actionType, ignoreType
• By default, metadata are not removed on updates, except if
overrideMetadata option is used
• Examples:
48
…
"temperature": {
"type": "Float",
"value": 26.5,
"metadata": {
"accuracy": {
"type": "Float",
"value": 0.9
}
}
}
…
…
"temperature": {
"type": "Float",
"value": 26.5,
"metadata": {
"average": {
"type": "Float",
"value": 22.4
}
}
}
…
Metadata

Attributes
• Name
• Type
• Value
Entity
• EntityId
• EntityType
Metadata
• Name
• Type
• Value
49
Complete NGSI Model
1 n
has
1 n
has

• Attributes and metadata can have a structured
value. Vectors and key-value maps are
supported.
• It maps directly to JSON's objects and arrays.
50
Compound Attribute/Metadata Values

• Example: we have
a car whose four
wheels' pressure
we want to
represent as a
compound
attribute for a car
entity. We would
create the car
entity like this:
{
"type": "Car",
"id": "Car1",
"tirePressure": {
"type": "kPa",
"value": {
"frontRight": 120,
"frontLeft": 110,
"backRight": 115,
"backLeft": 130
}
}
}
51
Compound Attribute/Metadata Values

• GET /v2/types
• Retrieve a list of all entity types currently in Orion,
including their corresponding attributes and entities
count
• GET /v2/types/{typeID}
• Retrieve attributes and entities count associated to an
entity type
PRO TIP
GET /v2/contextTypes?options=values
Retrieves just a list of all entity types without any extra info
52
Type Browsing

• Entities can have an attribute
that specifies its location
• Several attribute types can be
used
– geo:point (for points)
– geo:line (for lines)
– geo:box (for boxes)
– geo:polygon (for polygons)
– geo:json (for arbitrary geometries, in
GeoJson standard)
• Example: create an entity called
Madrid
…and create a couple more towns:
• Leganés
• Alcobendas
{
"type": "City",
"id": "Madrid",
"position": {
"type": "geo:point",
"value": "40.418889, -3.691944"
}
}
53
Geo-location
null is also supported as value,
meaning "no location". This may be
useful in some cases.

Geo-location – Max distance
55
GET <cb_host>:1026/v2/entities?
idPattern=.*&
type=City&
georel=near;maxDistance:13500&
geometry=point&
coords=40.418889,-3.691944

Geo-location – Min distance
56
GET <cb_host>:1026/v2/entities?
idPattern=.*&
type=City&
georel=near;minDistance:13500&
geometry=point&
coords=40.418889,-3.691944

• Apart from near, the following georel can be
used
– georel=coveredBy
– georel=intersects
– georel=equals
– georel=disjoint
• See NGSIv2 Specification for a detailed
description
57
More geo-relationships

• Regular attribute update uses particular values (eg. 12, "on",
etc).
• This can be inefficient and cause race conditions in some cases
– Eg. several context producers updating a common counter attribute
58
Update operators
CP A
GET /v2/entity/E/attrs/count
CP B
count = 10
Sum 2 Sum 3
GET /v2/entity/E/attrs/count
10
10
10 +2 = 12
10 +3 = 13
count = 12
count = 13
PUT /v2/entity/E/attrs/count (12)
PUT /v2/entity/E/attrs/count (13)
Final value is 13 but it
should be 15!
Calculation
at CP:
Calculation
at CP:

• Update operators can be used in attribute updates
• Eg. "increase the value of attribute by 2"
• Full list of current operators:
– $inc: increase by a given value
– $mul: multiply by a given value
– $max, $min: max/min comparing provided and existing value
– $push, $addToSet: add elements to array
– $pull, $pullAll: remove elements from array
– $set, $unset: add/remove sub-keys in object
59
Update operators
…
"count": {
"type": "Number",
"value": { "$inc": 2 }
}
…

• Simpler work for context producers and avoiding race
conditions
– No need of calculation in context producers
60
Update operators
CP A
PUT /v2/entity/E/attrs/count
{$inc: 2}
CP B
count = 10
Sum 2 Sum 3
count = 12
count = 15
Final result is correct
PUT /v2/entity/E/attrs/count
{$inc: 3}

• For the GET /v2/entities operation
• By entity type
• By entity id list
• By entity id pattern (regex)
• By entity type pattern (regex)
• By geographical location
– Described in detail in previous slides
• Filters can be used simultaneously (i.e. like AND condition)
61
GET <cb_host>:1026/v2/entities?type=Room
GET <cb_host>:1026/v2/entities?id=Room1,Room2
GET <cb_host>:1026/v2/entities?idPattern=^Room[2-5]
GET <cb_host>:1026/v2/entities?typePattern=T[ABC]
Query filters

• By attribute value (q)
• By metadata value (mq)
• See full details about q and mq query language in NGSIv2 specification
62
GET <cb_host>:1026/v2/entities?q=temperature>25
GET <cb_host>:1026/v2/entities?q=tirePressure.frontRight >130
attribute name
attribute sub-key (for compound attribute values only)
GET <cb_host>:1026/v2/entities?mq=temperature.avg>25
GET <cb_host>:1026/v2/entities?mq=tirePressure.accuracy.frontRight >90
metadata sub-key (for compound
metadata values only)
attribute name
metadata name
Query filters

63
…
{
"subject": {
"entities": [
{
"id": “Car5",
"type": "Car"
},
{
"idPattern": "^Room[2-5]",
"type": "Room"
},
{
"id": "D37",
"typePattern": "Type[ABC]"
},
],
"condition": {
"attrs": [ "temperature" ],
"expression": {
"q": "temperature>40",
"mq": "humidity.avg==80..90",
"georel": "near;maxDistance:100000",
"geometry": "point",
"coords": "40.418889,-3.691944"
}
}
},
…
}
• Filters can be also used in
subscriptions
– id
– type
– id pattern
– type pattern
– attribute values
– metadata value
– geographical location
Query filters

• Orion implements date support
– Based on ISO ISO8601 format with milliseconds
precision, including partial representations and
timezones
• See https://github.com/telefonicaid/fiware-
orion/blob/master/doc/manuals/orion-api.md#datetime-
support for syntax details
• null is also supported meaning "no date"
– Use reserved attribute type DateTime to express a date
– Date-based filters are supported
64
Datetime support

• Attribute value arithmetic filters can be used with dates as if they
were numbers
• Entity dateModified and dateCreated special attributes, to get
entity creation and last modification timestamps
– They are shown in query responses using
attrs=dateModified,dateCreated
• Entity dateModified and dateCreated special metadata, to get
attribute creation and last modification timestamps
– They are shown in query responses using
metadata=dateModified,dateCreated
65
POST /v2/entities
…
{
"id": "John",
"birthDate": {
"type": "DateTime",
"value": "1979-10-14T07:21:24.238Z"
}
}
GET /v2/entities?q=birthDate<1985-01-01T00:00:00
Example: create entity John,
with birthDate attribute using
type DateTime
Datetime support

• Orion will automatically delete the entity, once dateExpires time
is reached
– In fact transient entities may remain in the database up to 60 seconds (or a bit
more, if the database load is high) after expiration date
• BE CAREFUL! Entities removed this way cannot be recovered
• As dateModified and dateCreated (previously described),
dateExpires is not retrieved by default
– It is shown in query responses using attrs=dateExpires
• dateExpires can be used as any othe DatetTime attribute in
filter, e.g.:
66
POST /v2/entities
…
{
"id": "Warning1",
"dateExpires": {
"type": "DateTime",
"value": “2018-07-30T12:00:00Z"
}
}
GET /v2/entities?q=dateExpires<2018-08-01T00:00:00
Example: create entity
Warning1, to expire at noon
on July 30th, 2018
Transient entities

• By default, Orion limits the set of allowed chars in order to
avoid injection attacks
– List of forbidden chars:
orion/blob/master/doc/manuals/orion-api.md#general-
syntax-restrictions
• You can avoid this check in attribute values using the
special attribute type TextUnrestricted
• Use it at your own risk!
67
Unrestricted text attributes
…
"description": {
"type": "TextUnrestricted",
"value": "Hey! I'm using <forbidden chars>"
}
…

• By default, Orion updates responses are decoupled from the
sending of the notifications triggered by these updates
• This may cause that if the client sending updates is much faster
than the receiver processing notifications, saturation will happen
68
Flow control
Client Receiver
…
client sending
updates too fast
at some point in time, Orion
notification queue will saturate
and new notifications will be
discarded
receiver is not
so fast
…

• Orion can implement a flow control mechanism, based in the
-notifFlowControl CLI option and flowControl option in update
requests
• Using flow control, Orion doesn’t response updates immediately.
It introduces a delay based on occupation size of the notification
queue (the more the occupation, the more the delay)
69
Flow control
Client Receiver
…
…
?options=flowControl
delays
introduced
by Orion

• More information on flow control
– Functionality description: https://fiware-
orion.readthedocs.io/en/master/admin/perf_tuning/inde
x.html#updates-flow-control-mechanism
– Detailed example/tutorial:
orion/blob/master/test/flowControlTest/README.md
70
Flow control

• Apart from the standard formats defined in the
previous slides NGSIv2 allows to re-define all the
notification aspects
• httpCustom is used instead of http, with the
following subfields
– URL query parameters
– HTTP method
– HTTP headers
– Payload (not necessarily JSON!)
• A simple macro substitution language based on ${..}
syntax can be used to “fill the gaps” with entity data (id,
type or attribute values) and extra information (service,
servicePath or authToken)
– Exception: this cannot be used in HTTP method field
71

72
…
"httpCustom": {
"url": "http://foo.com/entity/${id}",
"headers": {
"Content-Type": "text/plain"
},
"method": "PUT",
"qs": {
"type": "${type}"
},
"payload": "The temperature is ${temp} degrees"
}
…
PUT http://foo.com/entity/DC_S1-D41?type=Room
Content-Length: 31
The temperature is 23.4 degrees
PUT /v2/entities/DC_S1-D41/attrs/temp/value?type=Room
…
23.4
Custom notification configuration
update
notification
Text based payload

73
…
"httpCustom": {
"method": "PUT",
"qs": {
"type": "${type}"
},
"json": {
"t": "${temp}",
"s": "${status}"
}
}
…
PUT http://foo.com/entity/DC_S1-D41?type=Room
Content-Length: 19
{ "t": 23.4, "s": "on"}
PUT /v2/entities/DC_S1-D41/attrs?type=Room
…
{ "temp": {"value": 23.4, "type": "Number"},
"status": {"value": "on", "type": "Text"} }
update
notification
JSON based payload

74
…
"httpCustom": {
"method": "PUT",
"ngsi": {
"id": "stamp:${id}"
"A:map": { "value": "${A}", "type": "Text"}
"oldId": { "value": "${id}", "type": "Text"}
}
}
…
PUT http://foo.com/entity/DC_S1-D41
Content-Length: 140
{
"id": "stamp:DC_S1-D41",
"A": { "value": "foo", "type": "Text" },
"A:map": { "value": "foo", "type": "Text" },
"oldId": { "value": "DC_S1-D41", "type": "Text" }
}
PUT /v2/entities/DC_S1-D41/attrs?type=Room
…
{ "A": {"value": "foo", "type": "Text"} }
update
notification
NGSI patching in payload
Notification uses NGSIv2
compliant format (as in
regular notifications)!

• Detailed information in the notifications
element
– timesSent: total number of notifications
attempts (both successful and failed)
– failsCounter: number of consecutive
notifications fails
• failsCounter > 0 means subscription is in
failing state
– lastSuccess: last time that notification was
successfully sent
– lastFailure: last time that notification was
tried and failed
– lastNotification: last time the notification was
sent (either success or failure)
• Corollary: lastNotification value is the same
than either lastFailure or lastSuccess
– lastFailureReason: cause of last failure (text)
– lastSuccessCode: the HTTP code (number)
returned by receiving endpoint last time a
successful notification was sent
• lastSuccessCode and
lastFailureReason fields only in HTTP
notifications (not in MQTT ones)
75
200 OK
…
[{
"id": " 51c0ac9ed714fb3b37d7d5a8 ",
"status": "active",
"subject": { … },
"notification": {
"timesSent": 3,
"failsCounter": 1,
"lastNotification": "2016-05-31T11:19:32.000Z",
"lastSuccess": "2016-05-31T10:07:32.000Z",
"lastFailure": "2016-05-31T11:19:32.000Z",
"lastSuccessCode": 200,
"lastFailureReason": "Timeout was reached",
…
}
}]
Notification status

76
Notification diagnosis workflow
"status" field
value?
"lastSuccessCode“
field value?
Subscription is inactive
so notifications are not
being sent. Update
subscription to
activate it.
Notifications are being
sent and the receiving
endpoint confirms
correct reception.
Notifications are
being sent but the
receiving endpoint
is reporting HTTP
error. Check
receiving endpoint
(e.g. logs, etc.)
Some problem is
precluding
notifications to be
sent. In HTTP
subscriptions, the
"lastFailureReason"
field value should
provide additional
information on it.
inactive
Subscription is expired
so notifications are not
being sent. Update
subscription to make it
permanent or extend
expiration time
expired
active/oneshot
YES
2xx
4xx,
5xx
Only for HTTP notifications
failsCounter > 0?
NO

• A variant of the active
status, so when the
subscription is triggered
one time (i.e. a
notification is sent) it
automatically steps to
inactive status
• An inactive subscription
can step to oneshot,
starting again the process
• Initial notification upon
subscription creation or
update is not sent in this
case
77
Oneshot subscription
200 OK
…
[{
"id": "51c0ac..",
"status": "oneshot",
…
}
}]
200 OK
…
[{
"id": "51c0ac..",
"status": "inactive",
…
}
}]
Subscription is
triggered
PATCH /v2/subscriptions/51c0ac..
{
"status": "oneshot"
}

• A maxFailsLimit can be specified for subscriptions
so when the number of consecutive notifications
overpasses that value, the subscription
automatically passes to inactive state
• failsCounter is reset to 0 whenever a successful
notification is sent
• This allow to protect Orion against failing
notification endpoints what would consume
notification resources (pool workers, etc.)
• When a subscription is auto-disabled, a trace about
it is printed in logs:
78
Subscription auto-disabling
POST /v2/subscriptions
{
"subject": { … },
"notification": {
"maxFailsLimit": 3,
…
}
}
failsCounter > maxFailsLimit => status := inactive
time=... | lvl=WARN | ... | msg= Subscription 51c0ac9ed714fb3b37d7d5a8
automatically disabled due to failsCounter (4) overpasses maxFailsLimit (3)

• By default, subscriptions are triggered
when entities are created or changed (i.e.
update with a value different from the
previous value)
• This can be changed with alterationTypes
field (within condition) to specify a list of
the following (OR condition):
– entityUpdate: entity update (no matter if is
actually changed or not)
– entityChange: entity actual change
– entityCreate: entity is created
– entityDelete: entity is deleted
• The alteration type which triggered the
subscription can be included in
notifications by adding the special
alterationType attribute (which possible
values are the same tokens above)
79
Subscriptions based in alteration type
POST /v2/subscriptions
{
"subject": {
…
"condition": {
"alterationTypes":
["entityCreate","entityDelete"]
}
},
"notification": {
…
"attrs": [ "alterationType", "*" ]
}
}

• By default, attributes that doesn't exist in the entity are not
notified
• This can be changed with covered field (within
notification) set to true to specify that non existing
attributes has to be notified
– As they don't exist, they use null as value
• We use the term "covered" in the sense the notification
"covers" completely all the attributes in the attrs field within
notification
• This feature can be useful for those notification endpoints
that are not flexible enough for a variable set of attributes
and needs always the same set of incoming attributes in
every received notification
80

81
SUB 1:
"notification": {
...
"attrs": [
"temp",
"hum"
]
}
SUB 2 (covered)
"notification": {
...
"attrs": [
"temp",
"hum
],
"covered": true
}
POST /v2/entities/E/attrs/temp
{ ...
"value": 19,
"type": "Number"
}
NOTIF1:
…
{
…
"temp": {
"type": "Float",
"value": 19,
"metadata": {}
}
}
NOTIF2 (covered)
…
{
…
"temp": {
"type": "Number",
"value": 19,
"metadata": {}
},
"hum": {
"type": "None",
"value": null,
"metadata": {}
}
}

• Using mqtt instead of http in the subscription payload
• Taking advantage of MQTT features (shared subscriptions, QoS, etc.)
• MQTT authentication based in user/password is supported
• Efficient connection management (removing idle connections to MQTT brokers if
they are not in use, base din -mqttMaxAge)
• Also supporting custom notifications (mqttCustom)
82
MQTT notifications
…
{
"subject": {
"entities": [
{ "idPattern": ".*" }
],
},
"notification": {
"mqtt": {
"url": "mqtt://<mqtt_broker>:1883",
"topic": "/notif",
"qos": 1
}
}
}
Context Broker
MQTT Broker
…
MQTT broker subscribers

• By default updates not actually changing attribute
value don’t trigger notification but this behavior can be
overridden by the forcedUpdate URI param option,
eg.:
– PATCH /v2/entities/E/attrs?options=forcedUpdate
– PUT /v2/entities/E/attrs/A?options=forcedUpdate
– POST /v2/op/update?options=forcedUpdate
• onlyChangedAttributes (within notification) can be
set to true in order to notify only attributes that
change (in combination with the ones specified in attrs
or exceptAttrs)
• Per-subscription notification timeout can be specified
for HTTP subscriptions in timeout field (within
http/httpCustom within notification)
83
Additional subscription improvements

• By default all attribute are included in query
responses or notifications
• The attrs field (as parameter in GET operations
and as notification sub-field in subscriptions)
can be used to specify a filtering list
• The attrs field can be also used to explicitly
include some special attributes (not included by
default)
– dateCreated, dateModified, dateExpires: described
in previous slide
• The “*” can be used as an alias of “all the
(regular) attributes”
84
Attributes filtering and special attributes

• Examples
– Include only attributes temp and lum
• In queries: GET /v2/entities?attrs=temp,lum
• In subscriptions: "attrs": [ "temp", "lum" ]
– Include dateCreated and not any other attribute
• In queries: GET /v2/entities?attrs=dateCreated
• In subscriptions: "attrs": [ "dateCreated" ]
– Include dateModified and all the other (regular)
attributes
• In queries: GET /v2/entities?attrs=dateModified,*
• In subscriptions: "attrs": [ "dateModified", "*" ]
– Include all attributes (same effect that not using attrs,
not very interesting)
• In queries: GET /v2/entities?attrs=*
• In subscriptions: "attrs": [ "*" ]
85
Attributes filtering and special attributes

• By default all attribute metadata are included in query responses
and notifications
• The metadata field (as parameter in GET operations and as
notification sub-field in subscriptions) can be used to specify a
filtering list
• The metadata field can be also used to explicitly include some
special metadata (not included by default)
– dateCreated, dateModified: described in previous slide
– actionType: which value is the action type corresponding to the
update triggering the notification: “update”, “append” or “delete” (*)
– previousValue: which provides the value of the attribute previous to
processing the update that triggers the notification
• The “*” can be used as an alias of “all the (regular) metadata”
86
(*) actionType “delete” not yet supported by Orion in 3.8.0.
Metadata filtering and special metadata

• Examples
– Include only metadata MD1 and MD2
• In queries: GET /v2/entities?metadata=MD1,MD2
• In subscriptions: "metadata": [ "MD1", "MD2" ]
– Include previousValue and not any other metadata
• In queries: GET /v2/entities?metadata=previousValue
• In subscriptions: "attrs": [ "previousValue" ]
– Include actionType and all the other (regular) metadata
• In queries: GET /v2/entities?metadata=actionType,*
• In subscriptions: "attrs": [ "actionType", "*" ]
– Include all metadata (same effect that not using
metadata, not very interesting)
• In queries: GET /v2/entities?metadata=*
• In subscriptions: "metadata": [ "*" ]
87
Metadata filtering and special metadata

• Uncached queries and updates
88
Application
ContextBroker ContextProvider
1. register(provider= )
db
2. query 3. query
4. data
5. data
Context
Consumer
Registration & Context Providers

POST <cb_host>:1026/v2/registrations
…
{
"description": "Registration for Car1",
"dataProvided": {
"entities": [
{
"id": "Car1",
"type": "Car"
}
],
"attrs": [
"speed"
]
},
"provider": {
"http": {
"url": "http://contextprovider.com/Cars"
},
"legacyForwarding": true
}
}
201 Created
Location: /v2/registrations/5a82be3d093af1b94ac0f730
…
89
(*) With some limitations in current Orion version, see https://github.com/telefonicaid/fiware-
orion/blob/master/doc/manuals/orion-api.md#registration-implementation-differences

90
GET <cb_host>:1026/v2/entities/Car1/attrs
ContextBroker ContextProvider
db
query
data
200 OK
...
{
"type": "Float",
"value": 110,
"metadata": {}
}
?options=skipForwarding can be used to avoid
query forwarding, in which case query is evaluated using
exclusively Context Broker local context information

• Simple multitenant model based on
logical database separation.
• It eases tenant-based authorization
provided by other components.
• Just use an additional HTTP header
called "Fiware-Service", whose value
is the tenant name. Example:
Fiware-Service: Tenant1
Context
Broker
Tenant1
Tenant2
…
91
Multitenancy

• A service path is a hierarchical scope assigned to an entity
at creation time (with POST /v2/entities).
92
Service Paths

• In order to use a service path we put in a new HTTP header
called "Fiware-ServicePath". For example:
Fiware-ServicePath: /Madrid/Gardens/ParqueNorte/Parterre1
• Properties:
– A query on a service path will look only into the specified node
– Use "ParentNode/#" to include all child nodes
– Queries without Fiware-ServicePath resolve to "/#"
– Entities will fall in the "/" node by default
ParqueNorte
Parterre2
Parterre1
93
Service Paths

• Properties (continued):
– You can OR a query using a comma (,)
operator in the header
• For example, to query all street lights that are either in
ParqueSur or in ParqueOeste you would use:
ServicePath: Madrid/Gardens/ParqueSur,
Madrid/Gardens/ParqueOeste
• You can OR up to 10 different scopes.
– Maximum scope levels: 10
• Scope1/Scope2/.../Scope10
– You can have the same element IDs in
different scopes (be careful with this!)
– You can't change scope once the element is
created
– One entity can belong to only one scope
– It works not only with queries, but also with
subscriptions/notifications
– It works not only in NGSI10, but also with
registrations/discoveries (NGSI9)
ParqueNorte
Parterre1
light1
light1
A B
A or B
94
Service Paths

• Useful for programming clients that run entirely in
browser without backend
– Full support (including pre-flight requests) in NGSIv2
– Partial support (only for GET requests) in NGSIv1
• CLI parameters related with CORS
– -corsOrigin
– -corsMaxAge
• See details
– https://fiware-
orion.readthedocs.io/en/master/user/cors/index.html
Cross-Origin Resource Sharing (CORS)

96
Context Consumer
notify
Context Broker
Private Network
Context Consumer
Context Broker
Private Network
./ngrok http 8080
http://9762d35a.ngrok.io
ngrok.io
Notifying services in private networks

• References
– This presentation, available at
orion#introductory-presentations
– NGSIv2 Specification
• https://github.com/telefonicaid/fiware-
orion/blob/master/doc/manuals/orion-api.md
– Documentation at ReadTheDocs
• https://fiware-orion.readthedocs.io
– Orion support through StackOverflow
• Ask your questions using the “fiware-orion” tag
• Look for existing questions at
http://stackoverflow.com/questions/tagged/fiware-orion
97
Would you like to know more?

Thank you!
http://fiware.org
Follow @FIWARE on Twitter

Integration with existing systems
• Context adapters will be developed to interface with existing systems (e.g.,
municipal services management systems in a smart city) acting as Context
Providers, Context Producers, or both
• Some attributes from a given entity may be linked to a Context Provider
while other attributes may be linked to Context Producers
queryContext (e1,
attr1, attr2)
Context Provider
queryContext (e1,
attr1)
Context Consumer
updateContext (e1,
attr2)
Application
Context Broker
System B
System A
99

Integration with sensor networks
• The backend IoT Device Management GE enables creation and
configuration of NGSI IoT Agents that connect to sensor networks
• Each NGSI IoT Agent can behave as Context Consumers or Context
Providers, or both
FIWARE Context Broker
IoT
Agent-1
IoT
Agent-2
IoT
Agent-n
IoT Agent
Manager
create/monitor
FIWARE Backend IoT
Device Management
OMA NGSI API (northbound interface)
(southbound interfaces)
MQTT
ETSI M2M IETF CoAP
100

• Federation of infrastructures (private/public regions)
• Automated GE deployment
Cloud
• Complete Context Management Platform
• Integration of Data and Media Content
Data
•Easy plug&play of devices using multiple protocols
•Automated Measurements/Action Context updates
IoT
•Visualization of data (operation dashboards)
•Publication of data sets/services
Apps
•Easy support of UIs with advanced web-based 3D and AR
capabilities
•Visual representation of context information.
Web UI
•Advanced networking capabilities (SDN) and Middleware
•Interface to robots
I2ND
•Security Monitoring
•Built-in Identity/Access/Privacy Management
Security
Context Management in FIWARE
101

Context/Data Management Platform
Applications
OMA NGSI-9/10
Processing/Analysis
Algorithms
Gathered data is
injected for
processing/analysis
Distributed
Context
Sources Complex Event
Processing
(PROTON)
BigData
(COSMOS)
Processed data is
injected for
processing/analysi
s
Data generated either by CEP
or BigData is published
Gathered data injected
for CEP-like processing
Direct
bigdata
injection
Programming of
rules
102
FI-WARE Context/Data Management Platform

• Used by /v2/op/update (batch operation)
• Conventional actionTypes
– append: to append (or update if the attribute already
exists)
– update: to update
– delete: to delete
• Special actionTypes
– appendStrict: strict append (returns error if some of
the attributes to add already exists)
– replace: delete all the entity attributes, next append
the ones in the update request
103
Special update action types

Orion Context Broker 20221220

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Similar to Orion Context Broker 20221220

Similar to Orion Context Broker 20221220 (20)

More from Fermin Galan

More from Fermin Galan (17)

Recently uploaded

Recently uploaded (20)

Orion Context Broker 20221220

Editor's Notes