NGSI-LD and Smart Data Models: Standard Access to Digital Twin Data - 15 July 2020 Corresponding webinar recording: https://youtu.be/MBx23ypORLk Understanding the basis of context information management, NGSI-LD and smart Data Models Chapter: Core Difficulty: 2 Audience: Any Technical Speaker: Juanjo Hierro (CTO, FIWARE Foundation), Alberto Abella (Data Modeling Expert and Technical Evangelist, FIWARE Foundation)

1. NGSI-LD and Smart Data Models:
Standard Access to Digital Twin Data
Juanjo Hierro
CTO
Alberto Abella
Data modelling expert
FIWARE Foundation

2. Introduction
Juan José Hierro
CTO
FIWARE foundation
1
Alberto Abella
Data modelling expert
FIWARE foundation

3. Learning goals
▪ Understanding how Context / Digital Twin concepts are supported in FIWARE and
they provide the basis for architecting solutions powered by FIWARE
▪ Understanding the Smart Data Models initiative and how it will help you on your
projects
▪ Understanding how to use smart data models and how to contribute to them
2

4. Index
Context / Digital Twin Data Management in FIWARE
The Smart data Models Initiative
- Principles
- Structure
- Repositories
- Frontend
- Examples
- Example 1
- Example 2
- License of Data models
- Contribution
- Governance
- Documentation
- Roadmap
3

5. Context / Digital Twin Data Management in FIWARE
4

6. 5
FIWARE was created to ease solutions supporting the Smart Digital Life
building around open standards for managing Context / Digital Twin Data
that blurs the frontiers among application domains and enable the Data Economy

7. What are we referring to as Digital Twin?
▪ Digital Twin = Digital representation of an asset
• Characterized by attributes
□ Properties
□ Relationships 🡪🡪 Linked Data
• Values of attributes may change over time (or not)
• Typically have a location (but it is not a must requirement)
▪ (digital representation of) Context = Digital Twins Collection
▪ Digital Twin = Asset Administrative Shell in RAMI 4.0
▪ Basis for the development of any Smart Solution:
• Standard API for getting access to Digital Twin data (context)
• Common Data Models associated to Digital Twin classes
6

8. Modeling Context using Digital Twins
for Cities …
7
Entities
(Digital
Twins)
Bus
• Location
• No. passengers
• Driver
• License plate
Citizen
• Birthday
• Preferences
• Location
• ToDo list
Incident /
claim
• Date
• Location
• Type
• Issuer
• Description
Shop
• Location
• Business name
• Franchise
• offerings
Attribut
e
Process / Analyze
/ Monitor
Digital Twin
representation
Context
(Real World)
capture actuate
update
notify /
query

9. Modeling Context using Digital Twins for
Agrifood …
8
Tractor
• Location
• Speed
• Planed route
Crop
• Humidity
• Leaf area
• Age
Drone
• Location
• Battery level
• Speed
• Planed route
Attribut
e
Entities
(Digital
Twins)
Process / Analyze
/ Monitor
Digital Twin
representation
Context
(Real World)
capture actuate
update
notify /
query

10. Modeling Context using Digital Twins for
Manufacturing …
9
Pallet
• id
• product
• Items quantity
• Layers
• Size
• Weight
Transport robot
• Id
• location
• speed
• transported items
• destination
Operator
• Id
• location
• assigned task
• profile
Shopfloor Door
• Id
• location
• status (open/close)
Entities
(Digital
Twins)
Process / Analyze
/ Monitor
Digital Twin
representation
Context
(Real World)
capture actuate
update
notify /
query

11. Modeling Context using Digital Twins for
Energy …
10
Wind
Turbine
• location
• power
• wind speed
• pitch angle
Energy
Storage
• active power
• reactive power
• SoC
• SoH
Substation
• Hi voltage
• Lo voltage
• nominal power
• power flow
Attribut
e
Entities
(Digital
Twins)
Process / Analyze
/ Monitor
Digital Twin
representation
Context
(Real World)
capture actuate
update
notify /
query

12. Modeling Context using Digital Twins for
Energy …
11
Attribut
e
Tanker
• Driver
• Location
• Max Volume
• Current Level
• Speed
• Direction
Gas Tank
• Station
• Max Volume
• Current Level
• Min Threshold
• Temperature
Station
• Location
• Owner
• SLA
Entities
(Digital
Twins)
Process / Analyze
/ Monitor
Digital Twin
representation
Context
(Real World)
capture actuate
update
notify /
query

13. Integration at multiple levels
12
Digital Twin
representation
Digital Twin
representation
Digital Twin
representation
Architecting
Smart Solutions
Integrating systems and
data within organizations
(system of systems)
Sharing Data across
organizations
3rd
systems
sensors
Smart Solution
System 3
System 4
System 1
System 2
Smart City
Smart Building
Smart
Logistics
Smart Grid

14. NGSI: a standard API for accessing Context / Digital Twin data
13
Process / Analyze
/ Monitor
Digital Twin
representation
Context
(Real World)
capture actuate
update
notify /
query
Application/Service
FIWARE NGSI API
(NGSIv2 → NGSI-LD)
Bus
• Location
• No. passengers
• Driver
• Licence plate
Citizen
• Name-Surname
• Birthday
• Preferences
• Location
• ToDo list
Shop
• Location
• Business name
• Franchise
• offerings
Context Broker

15. Endorsement at global level
14
TM Forum supports FIWARE
NGSI for real-time access to
context information in cities
TM Forum and FIWARE
collaborate in development
of data marketplace platform
components
TM Forum and FIWARE also
collaborate in definition of
common smart data models
in collaboration with cities
ETSI created Jan 2017 an
Industry Specification Group
(ISG CIM) for defining a
Context Information
Management API
FIWARE NGSIv2 provided the
basis for the NGSI-LD specs
published by ETSI
FIWARE provides several
open source implementations
of ETSI NGSI-LD
The FIWARE Context Broker
Technology has been
selected as a new CEF
(Connecting Europe Facility)
Building Block
recommended to public and
private sector for
publication of right-time
context data
The European Data portal
will support the publication
of right-time Open Data
The GSMA has published a
Reference Architecture for
IoT Big Data Ecosystem
which recommends to
mobile operators
NGSI-LD plays the core role
in the defined Reference
Architecture

16. Smart Data Models
▪ FIWARE Foundation is
collaborating with relevant
organizations towards definition
of common data models for
multiple application domains
• Smart Cities
• Smart Agrifood
• Smart Energy
• Smart Environment
• Smart Manufacturing
• …
▪ Defined data models rely on
well-established ”de-facto”
standards (e.g., schema.org,
SHAREF or IEC CIM in Energy)
15
https://github.com/smart-data-m
odels

17. Vertical Smart Solution for Energy: Reference Architecture
▪ Four major layers:
• Data acquisition
• Data management
• Data processing, analysis and
visualization
• Application layer
▪ Ability to integrate third IoT
platforms or use FIWARE IoT
Agents part of the IDAS Framework
▪ Integration with most popular
Apache processing engines (Spark,
Flink, Hadoop)
▪ Advanced web mashup and
Business Intelligence components
16

18. Smart Grid Reference Architecture
17

19. The Smart data models initiative
18

20. And what will happen to the data models you use when ….
▪ … your project ends.
▪ … you have to update it. Could you afford it?
▪ … you were asked to make it interoperable with other/new initiatives?
▪ … you need to update it to new regulations / standards?
Wouldn’t be more efficient and useful to do it together?
19

21. 20
Principles

22. ▪ A community site with detailed data models available for open use for multiple sectors
▪ Together with other relevant organizations in the curation of the different domains and subjects
▪ Providing coherence and consistency between data models across different domains
▪ To create a method for AGILE standardization and evolution these data models
▪ To provide extended usefulness to FIWARE platform users in terms of:
− Extended interoperability
− Reduced time dedicated to data model codying
− Accumulated experience tested in real case scenarios
− Mapped to be integrated with other platforms
▪ Using open licensing to allow extensive use and adoption
▪ Used in real case scenarios (and based on real use cases)
▪ Based on git platform and github as development frontend
▪ Consensus as main decision method
▪ Based on widely adopted standards (including ontologies and international schemas (i.e. schema.org)
Principles of Smart data models initiative
21

23. 22
Structure

24. data-models
Umbrella repo
Cross
Sector
Smart
Manufacturing
Subject NSubject 1
Smart
Water
Smart
Robotics
Smart
Agrifood
Smart
Cities
Smart
Environment
Smart
Destinations
Smart
Sensoring
Subject 2 Subject 3 Subject 4
DOMAINS
REPOSITORIES
Readme
pointing to the
list of subjects
General info or
shared
resources
DATA-MODELS
- Guides for coding new data models
- Template for new data models and examples
- Directory for scripting tools to check data models
- Inventory of domains and data models
- Inventory of attributes and terms
- @Context for json-ld
SUBJECTS’ REPOSITORIES
Readme pointing to the list of data models for the objects
Contributors.md
subject-schema.json
DATA MODELS
README.md
/doc/spec.md
/examples
schema.json
Current Adopters
Structure: domains and subjects compile data models
23

25. GITHUB
http://github.com/smart-data-models
- Oriented to developers
- All resources available
- Contribution by PR
- Issues on data models
SITE (wp)
http://data-models.fiware.org
- Oriented to end users
- News on updates (subscription)
- Check attributes and enumerations
Structure: Webs
24

26. - Global data about the initiative
○ List of data models
○ List of attributes
○ Required attributes
- Domains’ repositories
- Updated daily to the last commit of Subjects
- Free contribution repo
- Under request
Domain repositories and others. http://github.com/smart-data-models/specs
Structure: Github. Domains
25

27. - Subjects as submodules of the Domain
- Updated daily to last commit
- Data models in the subject
Domain repositories and others.
https://github.com/smart-data-models/dataModel.WaterNetworkManagement
- Shared elements for all the
Data models in the subject
Structure: Github. Domains
26

28. - Agreement for release data models with open license - Contact options
- Presentation on the governance of the initiative
- Tools & links for learning about data modelling
- Manual submission of data model (with help) - Newslists on different Domains
- News updates on the data models
- Document for future contributors to data models
- List of data models
- Attributes database
- Coding instructions
Frontpage. https://data-models.fiware.org
Structure: WP
27

29. Frontpage. http://data-models.fiware.org/index.php/subscriptions-page/
Frontpage
1 list per domain
1 mail a week (unless very important changes)
Structure: WP
28

30. 29
Example of use

31. 30
Example 1. Look for an attribute for your data model
1. Look for a parameter into the attributes database (i.e. temperature)
2. Explore the different data models related
3. Review the specification

32. 31
Example 2. Creating an entity on NGSI and other systems
1. Browse github till retrieve GTFS trip payload (json)
2. Open editor to run the queries into a NGSI engine
3. Get example csv (raw)
4. Convert into SQL
5. Create a SQL database into a DB editor
By -stk - Own work, CC0,
https://commons.wikimedia.org/w/index.php?curid=47287176

33. 32
License of data models

34. 33
Licensing Data Models
1. Preferred: Creative Commons 4.0
2. Apache 2.0 or other open licenses could be approved as long as they:
a. Recognise contributions
b. Allow free reuse of the data models
c. Do not impose other restrictions to use and adoption
3. Contributors has to fill a form to provide rights to the initiative for releasing with
thouse licenses (not losing their IPR)

35. 34
Contribution

36. 35
Management of contributions
CONTRIBUTORS
● Anybody can contribute as long as he/she meets guidelines (for checking automation)
● Participation can be as individual or representing an organization
● Contributors are explicitly recognised and vote relevant changes and contributions
● Contribution has to be based on real use of the data model
● Contribution manual explains:
a. Options for contribution (PR, issue, form)
b. Link to the guidelines for contribution
c. Documents needed for a complete data model (reduction coming)

37. 36
Governance

38. CO-ORDINATION
▪ Technical steering board is the governing body
▪ Domains’ and Subjects’ repositories are coordinated independently
▪ Domains are coordinated by Organizations relevant in the sector
▪ Subjects are coordinated either individuals or organizations
▪ Technical steering board look after consistency and participation
▪ Technical steering board members are elected each year (or when necessary)
▪ Consensus procedure (conflict resolution)
*Final approval pending
Governance*
37

39. 38
Documentation

40. doc directory
schema.json
Descriptions
Automatic
An script generates the file
Manual
Part is done automatically
but manual review required
Data model Root directory
Payload
ExamplesPayload
Examples
Payload
Examples
Json
Jsonld
1
T
Scripts
model.yaml
Examples directory
2
Payload
Examples
csv
4
T
T
Available
Tested
To be created
swagger.yaml
3
README.md
spec.md
5
6
specES.md
specJP.md
specFR.md
specDE.md
specFI.md
specCN.md
Source of truth
7
context.jsonld
terms.jsonld
subject-external-
context.jsonld
Subject
directory
8
Data model
Root directory
Document structure
39

41. 1. The contributor provides the schema.json either new or updated
2. A script generates the templates for the swagger.yaml + model.yaml, and contributor
includes/updates the descriptions
3. A script generates the json and json-ld examples
4. A script generates csv examples out of the json and json-ld examples
5. From the schema.yaml and with other things added though a template that occasionally
has to be updated by the management it is created the swagger.yaml
6. The model.yaml allows the creation of a spec.md
7. The swagger.yaml + model.yaml allows the creation of a README.md (markdown)
8. The swagger.yaml + model.yaml are translated
9. Based on swagger_XX.yaml + model_XX.yaml the translations README_XX.md are
created
10. swagger.yaml + model.yaml also enables creation of JSON-LD and NGSI-LD
@context files
Explanation
40

42. - Root
- schema.json
- LICENSE.md
- CURRENT-ADOPTERS.md
- model.yaml
- swagger.yaml
- README.md
- README-XX.md (XX = ES,JP,DE,FI,FR, etc)
- docs
- spec.md
- examples
- example.* (json, jsonld, csv)
- example-normalized.* (json, jsonld, csv)
- translations
- model-XX.yaml (XX = ES,JP,DE,FI,FR, etc)
- swagger-XX.yaml (XX = ES,JP,DE,FI,FR, etc)
Items in BLUE are generated
and should not be edited.
Items in RED involve manual
input and form the “source of
truth”.
Items in PURPLE are
initially generated, but may
also be updated manually
● Items in BOLD are
supplied
● Items in Italics are
generated
Files in data model directory
41

43. 42
Roadmap

44. 1. Dramatic increase in the # of data models
− Adaptation of existing standards (Cities, Agrifood, Manufacturing, Energy, Water,
Robotics, Smart Destinations, etc)
2. Engagement of additional relevant organizations and bodies
3. Automation tools for reducing the workload for contributors
4. Survey to users to get actual needs
5. Growth the community relying on the decentralized governance
Roadmap
43

45. Thank you!
http://fiware.org
Follow @FIWARE on Twitter
contact:
alberto.abella@fiware.org

46. Q & A
45

47. 46
Complementary materials

48. LEVELS OF ACCEPTANCE OF DATA MODELS*
1. Harmonized
a. Designed
b. Follow guidelines
c. Complete documentation
d. Tested on NGSI
e. Contributors appear on CONTRIBUTORS.md
2. Local Standardization
a. There are projects that will be using the data model.
b. Endorsement by organization
3. Global standardization
a. There is group of interested on the data model (evolution)
b. There is several real case scenario documented on CURRENT-ADOPTERS
* Approval pending
47

49. HOW A DATA MODEL IS ACCEPTED
HARMONIZED
● Checked:
○ All documents are complete
○ Properties meet the guidelines
○ Payloads validate against the schema
○ Payloads validate against NGSI
○ Spec meet the template
○ Agreement between contributors
○ Contributor agreement filled by the authors
48

50. HOW A DATA MODEL IS ACCEPTED
LOCAL STANDARDIZATION
● Data model is used in projects:
○ There are one or more projects implementing the data model
● Organization supporting the data model
○ An organisation of the domain adopts / endorses the data model
■ Endorsement (letter of)
■ Dissemination of data model
● Maintenance
○ There is a group maintaining/curating the data model
49

51. HOW A DATA MODEL IS ACCEPTED
GLOBAL STANDARDIZATION
● Data model is used in real scenarios:
○ There are one or more real use cases using it
○ The CURRENT-ADOPTERS.md includes unless one real case scenario
● Global endorsements
○ There are organizations of the sector from several countries endorsing the
data model (or an international organization of the sector)
50