FIWARE Summit, Porto, 8 May 2018
Dr Thanasis Poulakidas, INTRASOFT Intl
BDVA/Boost 4.0 Big Data reference architecture
Driving the Boost 4.0 RA
Boost 4.0
Reference
Architecture
Pilot & System
Requirements
Existing Software
Frameworks
BDV Reference Model
Standards
and Policies
Industrial Data Spaces (IDS)
FIWARE
Big Data Europe platform
Hyperledger Fabric
5 lighthouse pilot areas
x 2 lighthouse factories
+ 3 replication factories = 13
Roadmap
Q1
M 1-6
Q2
Task 2.1 Scoping, Use-case Analysis & Pilots‘
Requirements
WP2 – BOOST4.0 Reference Architecture
Q3
M 7-12
Q4 Q5
M 13- 18
Q6 Q7 Q80
WP3 – BOOST4.0 Big Data Interoperable
Pipeline & Analytics Platform
Task 2.2 Horizontal Analysis and Industry
Platform Requirements
Task 2.3 Pilot-scale planning and coordination
Task 2.4 BOOST 4.0 Big Data Architecture
Task 2.5 Data Governance & Protection
Task 2.6 Standardisation & Certification
Task 3.1 BOOST 4.0 Industrial Data Space &
Infrastructure
Task 3.2 Semantic Models, Vocabularies &
Registry
Task 3.3 Industry Data Space Connectors &
Context Information Management
Task 3.4 Industrial Data Space Security &
Trusted Transactions
Task 3.5 Big Data Models for Cognitive
Manufacturing
Task 3.6 Big Data Analytics Platform
M2
D3.1
D2.1
D2.3
D2.5
D2.7
D2.2
D2.6
D3.2
D3.4
M6
M6
M2
First specs ready
Scoping, use case
analysis requirements
v1
WP 4, 5, 6, 7, 8 – BOOST4.0 Pilots
First version of
Architecture ready
(support T[45678].1)
D3.3
Pilot Requirements
and Use Cases
Specification v1
Big Data Models and
Analytics Platform v1
BOOST 4.0 Reference
Architecture
Specification v1
M 19-24
Deliverables
v2
Deliverables
v2
M3
BDV Reference Model
CybersecurityandTrust
Data Management
External Data Sources
PLM systems, Production data acquisition systems
(RFID, etc.), ERP/MES, Open Web-APIs
Things/Assets, Sensors and Actuators
Collaborative
Analytics Service
Marketplace
Facilitates access to
Open Data Model
(ODM), Data Space
Areas (DSA), and Data
Analytics Services
(DAS)
DataSharingPlatforms
Standards
Infrastructure
Cloud, Hybrid Cloud, Edge,
HPC, …
Development–EngineeringandDevOps
Data Protection
Data Processing
Data Analytics
Data Visualisation and User Interaction
Communication&Connectivity
Slightly adapted for
manufacturing
Principles and
techniques for data
management:
Collection, Preparation,
Curation, Linking,
Access, Sharing – Data
Market / Data Spaces,
SQL/NoSQL DBs
Privacy and anonymisation
mechanisms to facilitate data
protection –also related to
Cybersecurity
Optimized and scalable
architectures: Batch,
Interactive,
Streaming/Real-time
Descriptive, Diagnostic,
Predictive, Prescriptive,
Machine Learning and AI,
Statistics, …
Advanced visualisation
approaches for improved
user experience: 1D, 2D, 3D,
4D, VR/AR
Marketplaces, Industrial
Data Platforms and
Personal Data Platforms
Tool chains for BDV systems
addressing: system/software
engineering, devops, quality
assurance
Effective communication
and connectivity
mechanisms to support
for BD (includes 5G)
Security and trust beyond
privacy and
anonymization. E.g.
blockchain technologies,
smart contracts and
encryption
6 Big Data types:
1. Structured data
2. Time series data
3. Geospatial data
4. Media, Image, Video and
Audio data
5. Text data, including Natural
Language Processing data and
Genomics representations
6. Graph data, Network/Web
data and Metadata
• “Place” existing
frameworks
• Gather pilots’
requirements
(incl. BD)
• Describe
existing legacy
systems
• Compile
components
/platforms
/software that
will be provided
by Boost 4.0
partners
• Specify system
requirements
• Boost 4.0 RA
specs
Approach to Boost 4.0 RA
Smart Digital
Engineering
Smart Planning
& Management
Smart
Operations &
Digital
Workspace
Smart
Connected
Production
Smart
Maintenance &
Service
Solutions
Components
• “Place” existing
frameworks
• Gather pilots’
requirements
(incl. BD)
• Describe
existing legacy
systems
• Compile
components
/platforms
/software that
will be provided
by Boost 4.0
partners
• Specify system
requirements
• Boost 4.0 RA
specs
Components
Solutions
Use case title
Vertical (area)
Author/Company/Email
Actors/ Stakeholders and their
roles and responsibilities
Goals
Current Solutions
Big data Science (collection,
curation, analysis, action)
Compute (System)
Storage
Networking
Software
Big data Specific Challenges
(Gaps)
Data Source
(distributed/centralized)
Volume (size)
Velocity (e.g. real time)
Variety (multiple
datasets, mashup)
Variability
(rate of change)
Big data Specific Challenges in
Manufacturing
Veracity (Robustness
Issues, semantics)
Visualization
Data Quality (syntax)
Data Types
Data Analytics
Security and Privacy
Requirements
Highlight issues for generalizing
this Use
case (e.g. for ref. architecture)
More Information (URLs)
Note: <additional comments>
Approach to Boost 4.0 RA
Smart Digital
Engineering
Smart Planning
& Management
Smart
Operations &
Digital
Workspace
Smart
Connected
Production
Smart
Maintenance &
Service
Boost 4.0 + BDE
Semantic BD
Fabric
IDS
IDS
Apps
BDE
Applications
over Apache
Value
Added
Services
BDE
Applic
ations
FIWARE
• “Place” existing
frameworks
• Gather pilots’
requirements
(incl. BD)
• Describe
existing legacy
systems
• Compile
components
/platforms
/software that
will be provided
by Boost 4.0
partners
• Specify system
requirements
• Boost 4.0 RA
specs
Approach to Boost 4.0 RA
Smart Digital
Engineering
Smart Planning
& Management
Smart
Operations &
Digital
Workspace
Smart
Connected
Production
Smart
Maintenance &
Service
Solutions
Components
Legacy system name …
Type Database, OPC, etc.
Details
APIs
SQL data access, OPC-UA SDK,
etc.
Data
Description
• Bill of materials
• Products
• Schedule
• Etc.
Format
XML, etc.
Big Data
Characteristics
(if applicable)
Data Source
(distributed/centralized)
Volume (size)
Velocity (e.g. real time)
Variety (multiple
datasets, mashup)
Variability (rate of
change)
Other Big Data
Science
(collection,
curation,
analysis, action -
if applicable)
Veracity (Robustness
Issues, semantics)
Visualization
Data Analytics
Boost 4.0 + BDE
Semantic BD
Fabric
IDS
IDS
Apps
BDE
Applications
over Apache
Value
Added
Services
BDE
Applic
ations
FIWARE
• “Place” existing
frameworks
• Gather pilots’
requirements
(incl. BD)
• Describe
existing legacy
systems
• Compile
components
/platforms
/software that
will be provided
by Boost 4.0
partners
• Specify system
requirements
• Boost 4.0 RA
specs
Approach to Boost 4.0 RA
Smart Digital
Engineering
Smart Planning
& Management
Smart
Operations &
Digital
Workspace
Smart
Connected
Production
Smart
Maintenance &
Service
Solutions
Components
Boost 4.0 + BDE
Semantic BD
Fabric
IDS
IDS
Apps
BDE
Applications
over Apache
Value
Added
Services
BDE
Applic
ations
FIWARE
ID
BC-<Partner short name>-<Ascending id>
e.g. BC-INTRA-1
Responsible partner …
Tool name …
Overall Description …
Details
Functionalities offered
1. Data analysis
2. User interface
3. Etc.
Include here Big Data functionalities, if applicable: data
management, data processing, data analytics, visualization
Data input Description
• Sensor data from robots
• Production data
• Etc.
Format XML, etc.
Data Output Description • …
Format XML, etc.
Integration requirements MES/ERP, PLM, Production data acquisition systems, Other
Boost4.0 component, APIs, etc.
• “Place” existing
frameworks
• Gather pilots’
requirements
(incl. BD)
• Describe
existing legacy
systems
• Compile
components
/platforms
/software that
will be provided
by Boost 4.0
partners
• Specify system
requirements
• Boost 4.0 RA
specs
Approach to Boost 4.0 RA
Smart Digital
Engineering
Smart Planning
& Management
Smart
Operations &
Digital
Workspace
Smart
Connected
Production
Smart
Maintenance &
Service
Solutions
Components
IDS FIWARE
BDE
Semantic
BD Fabric
IDS
IDS
Apps
BDE
Applications
BDE
Applications
Value Added
Services
Value Added
Services
BDE
Applications
From Requirements to Technology
specification
Requirements Analysis Design Implementation
Functional & Non-
functional
requirements
Design
requirements
e.g. graph db
Implementation
requirements
e.g. standard use
Conceptual
solution
Concrete
technology
Specification for
the software
influence
influence
influence
refined into
refined into
Represents an
implementation-
independent solution e.g.
Persistence
a refinement of the
conceptual solution. It
assumes some details of the
implementation e.g.
Document-oriented database
a refinement of the
concrete technology which
specifies the exact
implementation, e.g.
User
requirement
Reference
Reference ID Overall Description
• explains what the software does
• describes its application, including relevant benefits,
objectives, and goals
• describes general factors affecting it and its
requirements
Specific
Requirement
s
Feature
Introduction &
Purpose of feature
Introduction & Purpose of feature
Stimulus Response
Sequence
Input & output description
Functional
Requirements
Define the fundamental actions that must take. Set of “system shall …”
statements. e.g.:
• Validity checks on the inputs
• Exact sequence of operations
• Responses to exceptions
External
Interface
Requirement
s
User Interfaces
Specify the characteristics of each interface between the software and its
users
Hardware
Interfaces
Specify the characteristics of each interface between the software and
hardware components of the system.
Software Interfaces Specify the use of other required software components.
Communications
Interfaces
Specify the various interfaces needed for communication with other
software/hardware components
Performance Requirements
Specify the numerical requirements placed on the software. e.g.:
• Amount and type of information to be handled
• Response duration in time
@boost4_0 https://www.linkedin.com
/groups/12075988

FIWARE Global Summit - BDVA / Boost 4.0 Big Data Reference Architecture

  • 1.
    FIWARE Summit, Porto,8 May 2018 Dr Thanasis Poulakidas, INTRASOFT Intl BDVA/Boost 4.0 Big Data reference architecture
  • 2.
    Driving the Boost4.0 RA Boost 4.0 Reference Architecture Pilot & System Requirements Existing Software Frameworks BDV Reference Model Standards and Policies Industrial Data Spaces (IDS) FIWARE Big Data Europe platform Hyperledger Fabric 5 lighthouse pilot areas x 2 lighthouse factories + 3 replication factories = 13
  • 3.
    Roadmap Q1 M 1-6 Q2 Task 2.1Scoping, Use-case Analysis & Pilots‘ Requirements WP2 – BOOST4.0 Reference Architecture Q3 M 7-12 Q4 Q5 M 13- 18 Q6 Q7 Q80 WP3 – BOOST4.0 Big Data Interoperable Pipeline & Analytics Platform Task 2.2 Horizontal Analysis and Industry Platform Requirements Task 2.3 Pilot-scale planning and coordination Task 2.4 BOOST 4.0 Big Data Architecture Task 2.5 Data Governance & Protection Task 2.6 Standardisation & Certification Task 3.1 BOOST 4.0 Industrial Data Space & Infrastructure Task 3.2 Semantic Models, Vocabularies & Registry Task 3.3 Industry Data Space Connectors & Context Information Management Task 3.4 Industrial Data Space Security & Trusted Transactions Task 3.5 Big Data Models for Cognitive Manufacturing Task 3.6 Big Data Analytics Platform M2 D3.1 D2.1 D2.3 D2.5 D2.7 D2.2 D2.6 D3.2 D3.4 M6 M6 M2 First specs ready Scoping, use case analysis requirements v1 WP 4, 5, 6, 7, 8 – BOOST4.0 Pilots First version of Architecture ready (support T[45678].1) D3.3 Pilot Requirements and Use Cases Specification v1 Big Data Models and Analytics Platform v1 BOOST 4.0 Reference Architecture Specification v1 M 19-24 Deliverables v2 Deliverables v2 M3
  • 4.
    BDV Reference Model CybersecurityandTrust DataManagement External Data Sources PLM systems, Production data acquisition systems (RFID, etc.), ERP/MES, Open Web-APIs Things/Assets, Sensors and Actuators Collaborative Analytics Service Marketplace Facilitates access to Open Data Model (ODM), Data Space Areas (DSA), and Data Analytics Services (DAS) DataSharingPlatforms Standards Infrastructure Cloud, Hybrid Cloud, Edge, HPC, … Development–EngineeringandDevOps Data Protection Data Processing Data Analytics Data Visualisation and User Interaction Communication&Connectivity Slightly adapted for manufacturing Principles and techniques for data management: Collection, Preparation, Curation, Linking, Access, Sharing – Data Market / Data Spaces, SQL/NoSQL DBs Privacy and anonymisation mechanisms to facilitate data protection –also related to Cybersecurity Optimized and scalable architectures: Batch, Interactive, Streaming/Real-time Descriptive, Diagnostic, Predictive, Prescriptive, Machine Learning and AI, Statistics, … Advanced visualisation approaches for improved user experience: 1D, 2D, 3D, 4D, VR/AR Marketplaces, Industrial Data Platforms and Personal Data Platforms Tool chains for BDV systems addressing: system/software engineering, devops, quality assurance Effective communication and connectivity mechanisms to support for BD (includes 5G) Security and trust beyond privacy and anonymization. E.g. blockchain technologies, smart contracts and encryption 6 Big Data types: 1. Structured data 2. Time series data 3. Geospatial data 4. Media, Image, Video and Audio data 5. Text data, including Natural Language Processing data and Genomics representations 6. Graph data, Network/Web data and Metadata
  • 5.
    • “Place” existing frameworks •Gather pilots’ requirements (incl. BD) • Describe existing legacy systems • Compile components /platforms /software that will be provided by Boost 4.0 partners • Specify system requirements • Boost 4.0 RA specs Approach to Boost 4.0 RA Smart Digital Engineering Smart Planning & Management Smart Operations & Digital Workspace Smart Connected Production Smart Maintenance & Service Solutions Components
  • 6.
    • “Place” existing frameworks •Gather pilots’ requirements (incl. BD) • Describe existing legacy systems • Compile components /platforms /software that will be provided by Boost 4.0 partners • Specify system requirements • Boost 4.0 RA specs Components Solutions Use case title Vertical (area) Author/Company/Email Actors/ Stakeholders and their roles and responsibilities Goals Current Solutions Big data Science (collection, curation, analysis, action) Compute (System) Storage Networking Software Big data Specific Challenges (Gaps) Data Source (distributed/centralized) Volume (size) Velocity (e.g. real time) Variety (multiple datasets, mashup) Variability (rate of change) Big data Specific Challenges in Manufacturing Veracity (Robustness Issues, semantics) Visualization Data Quality (syntax) Data Types Data Analytics Security and Privacy Requirements Highlight issues for generalizing this Use case (e.g. for ref. architecture) More Information (URLs) Note: <additional comments> Approach to Boost 4.0 RA Smart Digital Engineering Smart Planning & Management Smart Operations & Digital Workspace Smart Connected Production Smart Maintenance & Service Boost 4.0 + BDE Semantic BD Fabric IDS IDS Apps BDE Applications over Apache Value Added Services BDE Applic ations FIWARE
  • 7.
    • “Place” existing frameworks •Gather pilots’ requirements (incl. BD) • Describe existing legacy systems • Compile components /platforms /software that will be provided by Boost 4.0 partners • Specify system requirements • Boost 4.0 RA specs Approach to Boost 4.0 RA Smart Digital Engineering Smart Planning & Management Smart Operations & Digital Workspace Smart Connected Production Smart Maintenance & Service Solutions Components Legacy system name … Type Database, OPC, etc. Details APIs SQL data access, OPC-UA SDK, etc. Data Description • Bill of materials • Products • Schedule • Etc. Format XML, etc. Big Data Characteristics (if applicable) Data Source (distributed/centralized) Volume (size) Velocity (e.g. real time) Variety (multiple datasets, mashup) Variability (rate of change) Other Big Data Science (collection, curation, analysis, action - if applicable) Veracity (Robustness Issues, semantics) Visualization Data Analytics Boost 4.0 + BDE Semantic BD Fabric IDS IDS Apps BDE Applications over Apache Value Added Services BDE Applic ations FIWARE
  • 8.
    • “Place” existing frameworks •Gather pilots’ requirements (incl. BD) • Describe existing legacy systems • Compile components /platforms /software that will be provided by Boost 4.0 partners • Specify system requirements • Boost 4.0 RA specs Approach to Boost 4.0 RA Smart Digital Engineering Smart Planning & Management Smart Operations & Digital Workspace Smart Connected Production Smart Maintenance & Service Solutions Components Boost 4.0 + BDE Semantic BD Fabric IDS IDS Apps BDE Applications over Apache Value Added Services BDE Applic ations FIWARE ID BC-<Partner short name>-<Ascending id> e.g. BC-INTRA-1 Responsible partner … Tool name … Overall Description … Details Functionalities offered 1. Data analysis 2. User interface 3. Etc. Include here Big Data functionalities, if applicable: data management, data processing, data analytics, visualization Data input Description • Sensor data from robots • Production data • Etc. Format XML, etc. Data Output Description • … Format XML, etc. Integration requirements MES/ERP, PLM, Production data acquisition systems, Other Boost4.0 component, APIs, etc.
  • 9.
    • “Place” existing frameworks •Gather pilots’ requirements (incl. BD) • Describe existing legacy systems • Compile components /platforms /software that will be provided by Boost 4.0 partners • Specify system requirements • Boost 4.0 RA specs Approach to Boost 4.0 RA Smart Digital Engineering Smart Planning & Management Smart Operations & Digital Workspace Smart Connected Production Smart Maintenance & Service Solutions Components IDS FIWARE BDE Semantic BD Fabric IDS IDS Apps BDE Applications BDE Applications Value Added Services Value Added Services BDE Applications
  • 10.
    From Requirements toTechnology specification Requirements Analysis Design Implementation Functional & Non- functional requirements Design requirements e.g. graph db Implementation requirements e.g. standard use Conceptual solution Concrete technology Specification for the software influence influence influence refined into refined into Represents an implementation- independent solution e.g. Persistence a refinement of the conceptual solution. It assumes some details of the implementation e.g. Document-oriented database a refinement of the concrete technology which specifies the exact implementation, e.g. User requirement Reference Reference ID Overall Description • explains what the software does • describes its application, including relevant benefits, objectives, and goals • describes general factors affecting it and its requirements Specific Requirement s Feature Introduction & Purpose of feature Introduction & Purpose of feature Stimulus Response Sequence Input & output description Functional Requirements Define the fundamental actions that must take. Set of “system shall …” statements. e.g.: • Validity checks on the inputs • Exact sequence of operations • Responses to exceptions External Interface Requirement s User Interfaces Specify the characteristics of each interface between the software and its users Hardware Interfaces Specify the characteristics of each interface between the software and hardware components of the system. Software Interfaces Specify the use of other required software components. Communications Interfaces Specify the various interfaces needed for communication with other software/hardware components Performance Requirements Specify the numerical requirements placed on the software. e.g.: • Amount and type of information to be handled • Response duration in time
  • 11.