SlideShare a Scribd company logo
Rocking the boat with big
data
www.bigdataocean.eu
Dr. Spiros Mouzakitis (NTUA)
Deputy Project Manager
GiannisTsapelas(NTUA)
Senior Researcher
Around 80% of global trade by volume is carried
by sEA
….and around 70% of global
trade by value
CHALLEN
GESo Undeveloped sharing of Blue Data between
enterprises and entities of the maritime domain
and other domains
o Lack of agreed standards and formats
o Huge potential from cross-sectorial blue data
applications – still unexploited
o Out-of-the-box Big Data solution that enables
advanced queries and analytics on cross-sector
data - missing
Develop a Maritime Big Data platform
that delivers out-of-the-box, value-added data and
analytic services for maritime applications
by exploiting cross-sector data streams
4
BigDataOcean platform (services)
Find maritime data
Query / interlink datasets Compose analytic services
Visualize datasets
Create (real-time) dashboards
Request on-demand services
Port
Authorities
Maritime
Research
Centers
Ship-owning
companies
Maritime
software
suppliers
Service
providers at
sea level
Maritime
advisory firms
BigDataOcean end-user applications
Vessel Fault
Detection,
Predictive
Maintenance
and Fuel
Consumption
Reduction
Maritime
security and
anomaly
detection
Oil spill
modeling
Wave power
generation
8BDV PPP Technical Committee Meeting
Case 1 - Fault Prediction & Proactive Maintenance
and Fuel Consumption
9
Case 1 - Fault Prediction & Proactive Maintenance
and Fuel Consumption
10
• Damage and mechanical failures detection and predictive
maintenance of vessel equipment
• Investigation of the impact of the environmental conditions
and the operational decisions taken on the vessel's fuel
consumption
Goal
• Ship owners, Maritime EquipmentConstructors
Stakeholders
BigDataOcean Solution
Analytics and knowledge
base about fuel
consumption and repairs
Prediction models for
maintenance and fuel
consumption
Maritime Regulations, Flag
& Ship Classification
Provisions
Plant Maintenance
System for Main Engine
& spare parts
Vessel
routes
In-situ Observations
Cross-domain
Forecast Data
Port information
BigDataOcean
Solution
Business Benefits
● Shipping companies and Maritime Equipment Constructors
 Minimum repairs, maintenance cost and fuel consumption
 Maximum vessels’ use and financial benefit
 Minimize environment impact
 Reliability and innovation
 Advanced Data analytics related to maintenance and fuel
consumption
Case 2 – Mare protection – Oil Spill dispersion
forecast
16
Case 2 – Mare protection – Oil Spill dispersion
forecast
17
• Provide to the end users oil spill drift forecasting and
simulation services for the marine environment
• Enhance the efficiency in managing oil spill pollution risks.
Goal
• Emergency Response Companies, National Entities
(PublicAuthorities), NGOs, Marine Research Institutes,
Shipping companies and Oil drilling companies
Stakeholders
Natura /
Protecte
d areas
In-situ Observations
BigDataOcean Solution
Graphical output
Cross-domain
Forecast Data
POSEIDON
OSM
Oil spill scenario
submission
Location, Rate,
Nature & characteristics
Ocean Circulation
Forecast
Weather
Forecast
BigDataOcean
Solution
AIS
Data
BigDataOcean
Wave
Forecast
Oil Spill Dispersion Forecast
Acquisition
High Risk PollutionAreas
UnderwaterAccident
Business Benefits
● Extended knowledge, models and enriched datasets
● New products addressed to environment protection
organizations and maritime authorities for rapid intervention
against oil spills in the sea
● Control and limit impact and damage on the coast and on
essential resources and structures.
● Efficiency in the protection of the marine environment and of
the marine life.
Case 3 – Wave Power as Clean Energy Source
23
Case 3 – Wave Power as Clean Energy Source
24
• Evaluation of wave energy potential and contribution to
development of wave energy solutions.
Goal
• Offshore Renewables Service Providers, Offshore Pilot
Zone Concessionaires,WEC developers, Energy
Producers, Hydrographic Centres
Stakeholders
BigDataOcean Solution
Waves In-situ
Observations
Cross-domain
Vessel
routes
Port
s
Protected
areas
Wave models
output
Instruments
Data
BigDataOcean
Solution
Wave ResourceAssessment
Wave Energy study and
Forecast
DataVisualisations
Business Benefits
● Wave resource characterization in your selected location or
area, based in historical data.
● Wave forecast.
● Assessment of Wave Energy Converters energy generation,
allowing to compare your device with others.
● Forecast of energy generation for your WEC device.
Case 4 – Security and anomaly path detection
Case 4 – Security and anomaly path detection
• Identify vessel routes based on their motion patterns to act
proactively and minimise threats at sea.
Goal
• Port authorities, Ocean Observatories, Port/Cargo
Community systems,Transport and Logistics companies,
Harbour Pilots and Maritime Consultants
Stakeholders
BigDataOcean solution
AIS
Anomaly Detection
BigDataOcean
Solution
Visualization
Anomaly
detection
services
Analytics
AIS
data
Weather Data
Location of Sea Ports
List of “High Risk” Vessels
List of security
incidents & vessels
implicated
Nautical Information Maps
Feedback from vessel’s
crew & domain experts
Business Benefits
● Effectively handle the information volume from tracking
technologies and AIS data
● identify patterns of behavior and vessel risk profile
● proactively minimize the impact of possible threats
BigDataOcean Consortium
35
35
Development
Semantics
Pilot
Pilot
PilotPilot
Requirements
Exploitation
Coordination &
Development
Pilot
Timeplan
Analysis and Development
2017 6/2019
Start
MS4:
Prototype
MS2:
Requirements
and needs
Q1 Q2 Q3 Q4 Q1 Q2 Q3 Q4 Q1 Q2
MS1:
Maritime data
value chain
definition
MS3:
Methodology
& Architecture
MS5: Final
Platform
MS6:
Business
Plan,
Lessons
Learnt
Evaluation
Market preparation
We are here
Lessons learned: exploitation
37
BDO Commercial Partnership for the platform
1. BDO Platform Services
2. BDO End-UsersApplication Services
3. BDO On-Demand Data Services and custom solutions
4. BDO Non-Technical Package
• Training
• Consulting
• Data science coaching
• Contract deals with maritime companies
• Focus on B2B applications
Semantic Challenges
● Plethora of file formats and metadata standards coming from diverse data sources
● Ability to perform advanced queries that combine those big datasets
● Enable ad hoc queries
Our Approach
● Automatic ingestion system to a harmonized, semantically aware schema (BDO
canonical Model)
● Based on DCAT, NETCDF convention standards (cfconventions.org)
Performance Challenges
● Performance issues
 POSTGRESQL and relational databases typically used in legacy maritime applications –
Serious scalability issues for the size of the datasets
 NoSQL databases -> Good performance for simple queries, but bad performance for
queries that combine data
 Distributed, wide column stores (e.g. Cassandra) -> better optimised for known queries
Our Approach
● Presto with Apache Hive
Lessons learned: current solution
Solution for Query Designer
queries
Solution for further query
optimization
● Caching JOIN queries performed by
users
● Pre-join datasets for pilot
applications
● Adoption of Apache Parquet storage
and ORF file formats
HDFS
Query performance over million of rows: from 5 minutes to 5 seconds!
BDO Architecture
Demo
www.bigdataocean.eu
Questions
www.bigdataocean.eu

More Related Content

Similar to BDVe Webinar Series - Big Data Ocean - Rocking the boat with Big Data

Vessel Efficiency competition company elevator pitches - London
Vessel Efficiency competition company elevator pitches - LondonVessel Efficiency competition company elevator pitches - London
Vessel Efficiency competition company elevator pitches - London
KTN
 
SeaDataCloud - Introduction to SeaDataNet infrastructure
SeaDataCloud - Introduction to SeaDataNet infrastructureSeaDataCloud - Introduction to SeaDataNet infrastructure
SeaDataCloud - Introduction to SeaDataNet infrastructure
EUDAT
 
DSD-INT 2019 Global Data Services and Analysis Frameworks-Luijendijk
DSD-INT 2019 Global Data Services and Analysis Frameworks-LuijendijkDSD-INT 2019 Global Data Services and Analysis Frameworks-Luijendijk
DSD-INT 2019 Global Data Services and Analysis Frameworks-Luijendijk
Deltares
 
The Roadmap to a Lifesaving Digital Ecosystem
The Roadmap to a Lifesaving Digital EcosystemThe Roadmap to a Lifesaving Digital Ecosystem
The Roadmap to a Lifesaving Digital Ecosystem
William Roberts
 
Progetto INNO ed esempi di applicazioni nel campo della GEOMATICA - P.Cau
Progetto INNO ed esempi di applicazioni nel campo della GEOMATICA - P.CauProgetto INNO ed esempi di applicazioni nel campo della GEOMATICA - P.Cau
Progetto INNO ed esempi di applicazioni nel campo della GEOMATICA - P.Cau
Sardegna Ricerche
 
Handling Big Data in Ship Performance & Navigation Monitoring.
Handling Big Data in Ship Performance & Navigation Monitoring.Handling Big Data in Ship Performance & Navigation Monitoring.
Handling Big Data in Ship Performance & Navigation Monitoring.
Lokukaluge Prasad Perera
 
The Environmental Futures & Big Data Impact Lab: Plymouth Launch Event Slides
The Environmental Futures & Big Data Impact Lab: Plymouth Launch Event SlidesThe Environmental Futures & Big Data Impact Lab: Plymouth Launch Event Slides
The Environmental Futures & Big Data Impact Lab: Plymouth Launch Event Slides
The Environmental Futures & Big Data Impact Lab
 
Nuno Catarino: NextGEOSS project
Nuno Catarino: NextGEOSS projectNuno Catarino: NextGEOSS project
Nuno Catarino: NextGEOSS project
EOSC-hub project
 
Ocean Data Factory Sweden
Ocean Data Factory SwedenOcean Data Factory Sweden
Ocean Data Factory Sweden
Robin Teigland
 
Ocean Data Factory - Application for Funding
Ocean Data Factory - Application for FundingOcean Data Factory - Application for Funding
Ocean Data Factory - Application for Funding
Robin Teigland
 
BDS 2015
BDS 2015BDS 2015
BDS 2015
nidhi sachdeva
 
CV Sigve Hamilton Aspelund 062013
CV Sigve Hamilton Aspelund 062013CV Sigve Hamilton Aspelund 062013
CV Sigve Hamilton Aspelund 062013
Sigve Hamilton Aspelund
 
Big Data in Oil and Gas: How to Tap Its Full Potential
Big Data in Oil and Gas: How to Tap Its Full PotentialBig Data in Oil and Gas: How to Tap Its Full Potential
Big Data in Oil and Gas: How to Tap Its Full Potential
Hitachi Vantara
 
Introduction to the GSDI Marine SDI Best Practice Webinar
Introduction to the GSDI Marine SDI Best Practice WebinarIntroduction to the GSDI Marine SDI Best Practice Webinar
Introduction to the GSDI Marine SDI Best Practice Webinar
GSDI Association
 
TGS Corporate Capabilities
TGS Corporate CapabilitiesTGS Corporate Capabilities
TGS Corporate Capabilities
TGS
 
Offshore lytics rolloos_midih_presentation_oc2
Offshore lytics rolloos_midih_presentation_oc2Offshore lytics rolloos_midih_presentation_oc2
Offshore lytics rolloos_midih_presentation_oc2
MIDIH_EU
 
Marinet en 2017
Marinet en 2017Marinet en 2017
Marinet en 2017
Dmitry Tseitlin
 
Aerial Data Management and The Digital Enterprise
Aerial Data Management and The Digital EnterpriseAerial Data Management and The Digital Enterprise
Aerial Data Management and The Digital Enterprise
Advisian
 
Presentation of YARIS platform
Presentation of YARIS platformPresentation of YARIS platform
Presentation of YARIS platform
EU GoGIN Project
 
Using Data Integration to Deliver Intelligence to Anyone, Anywhere
Using Data Integration to Deliver Intelligence to Anyone, AnywhereUsing Data Integration to Deliver Intelligence to Anyone, Anywhere
Using Data Integration to Deliver Intelligence to Anyone, Anywhere
Safe Software
 

Similar to BDVe Webinar Series - Big Data Ocean - Rocking the boat with Big Data (20)

Vessel Efficiency competition company elevator pitches - London
Vessel Efficiency competition company elevator pitches - LondonVessel Efficiency competition company elevator pitches - London
Vessel Efficiency competition company elevator pitches - London
 
SeaDataCloud - Introduction to SeaDataNet infrastructure
SeaDataCloud - Introduction to SeaDataNet infrastructureSeaDataCloud - Introduction to SeaDataNet infrastructure
SeaDataCloud - Introduction to SeaDataNet infrastructure
 
DSD-INT 2019 Global Data Services and Analysis Frameworks-Luijendijk
DSD-INT 2019 Global Data Services and Analysis Frameworks-LuijendijkDSD-INT 2019 Global Data Services and Analysis Frameworks-Luijendijk
DSD-INT 2019 Global Data Services and Analysis Frameworks-Luijendijk
 
The Roadmap to a Lifesaving Digital Ecosystem
The Roadmap to a Lifesaving Digital EcosystemThe Roadmap to a Lifesaving Digital Ecosystem
The Roadmap to a Lifesaving Digital Ecosystem
 
Progetto INNO ed esempi di applicazioni nel campo della GEOMATICA - P.Cau
Progetto INNO ed esempi di applicazioni nel campo della GEOMATICA - P.CauProgetto INNO ed esempi di applicazioni nel campo della GEOMATICA - P.Cau
Progetto INNO ed esempi di applicazioni nel campo della GEOMATICA - P.Cau
 
Handling Big Data in Ship Performance & Navigation Monitoring.
Handling Big Data in Ship Performance & Navigation Monitoring.Handling Big Data in Ship Performance & Navigation Monitoring.
Handling Big Data in Ship Performance & Navigation Monitoring.
 
The Environmental Futures & Big Data Impact Lab: Plymouth Launch Event Slides
The Environmental Futures & Big Data Impact Lab: Plymouth Launch Event SlidesThe Environmental Futures & Big Data Impact Lab: Plymouth Launch Event Slides
The Environmental Futures & Big Data Impact Lab: Plymouth Launch Event Slides
 
Nuno Catarino: NextGEOSS project
Nuno Catarino: NextGEOSS projectNuno Catarino: NextGEOSS project
Nuno Catarino: NextGEOSS project
 
Ocean Data Factory Sweden
Ocean Data Factory SwedenOcean Data Factory Sweden
Ocean Data Factory Sweden
 
Ocean Data Factory - Application for Funding
Ocean Data Factory - Application for FundingOcean Data Factory - Application for Funding
Ocean Data Factory - Application for Funding
 
BDS 2015
BDS 2015BDS 2015
BDS 2015
 
CV Sigve Hamilton Aspelund 062013
CV Sigve Hamilton Aspelund 062013CV Sigve Hamilton Aspelund 062013
CV Sigve Hamilton Aspelund 062013
 
Big Data in Oil and Gas: How to Tap Its Full Potential
Big Data in Oil and Gas: How to Tap Its Full PotentialBig Data in Oil and Gas: How to Tap Its Full Potential
Big Data in Oil and Gas: How to Tap Its Full Potential
 
Introduction to the GSDI Marine SDI Best Practice Webinar
Introduction to the GSDI Marine SDI Best Practice WebinarIntroduction to the GSDI Marine SDI Best Practice Webinar
Introduction to the GSDI Marine SDI Best Practice Webinar
 
TGS Corporate Capabilities
TGS Corporate CapabilitiesTGS Corporate Capabilities
TGS Corporate Capabilities
 
Offshore lytics rolloos_midih_presentation_oc2
Offshore lytics rolloos_midih_presentation_oc2Offshore lytics rolloos_midih_presentation_oc2
Offshore lytics rolloos_midih_presentation_oc2
 
Marinet en 2017
Marinet en 2017Marinet en 2017
Marinet en 2017
 
Aerial Data Management and The Digital Enterprise
Aerial Data Management and The Digital EnterpriseAerial Data Management and The Digital Enterprise
Aerial Data Management and The Digital Enterprise
 
Presentation of YARIS platform
Presentation of YARIS platformPresentation of YARIS platform
Presentation of YARIS platform
 
Using Data Integration to Deliver Intelligence to Anyone, Anywhere
Using Data Integration to Deliver Intelligence to Anyone, AnywhereUsing Data Integration to Deliver Intelligence to Anyone, Anywhere
Using Data Integration to Deliver Intelligence to Anyone, Anywhere
 

More from Big Data Value Association

Data Privacy, Security in personal data sharing
Data Privacy, Security in personal data sharingData Privacy, Security in personal data sharing
Data Privacy, Security in personal data sharing
Big Data Value Association
 
Key Modules for a trsuted and privacy preserving personal data marketplace
Key Modules for a trsuted and privacy preserving personal data marketplaceKey Modules for a trsuted and privacy preserving personal data marketplace
Key Modules for a trsuted and privacy preserving personal data marketplace
Big Data Value Association
 
GDPR and Data Ethics considerations in personal data sharing
GDPR and Data Ethics considerations in personal data sharingGDPR and Data Ethics considerations in personal data sharing
GDPR and Data Ethics considerations in personal data sharing
Big Data Value Association
 
Intro - Three pillars for building a Smart Data Ecosystem: Trust, Security an...
Intro - Three pillars for building a Smart Data Ecosystem: Trust, Security an...Intro - Three pillars for building a Smart Data Ecosystem: Trust, Security an...
Intro - Three pillars for building a Smart Data Ecosystem: Trust, Security an...
Big Data Value Association
 
Three pillars for building a Smart Data Ecosystem: Trust, Security and Privacy
Three pillars for building a Smart Data Ecosystem: Trust, Security and PrivacyThree pillars for building a Smart Data Ecosystem: Trust, Security and Privacy
Three pillars for building a Smart Data Ecosystem: Trust, Security and Privacy
Big Data Value Association
 
Market into context - Three pillars for building a Smart Data Ecosystem: Trus...
Market into context - Three pillars for building a Smart Data Ecosystem: Trus...Market into context - Three pillars for building a Smart Data Ecosystem: Trus...
Market into context - Three pillars for building a Smart Data Ecosystem: Trus...
Big Data Value Association
 
BDV Skills Accreditation - Future of digital skills in Europe reskilling and ...
BDV Skills Accreditation - Future of digital skills in Europe reskilling and ...BDV Skills Accreditation - Future of digital skills in Europe reskilling and ...
BDV Skills Accreditation - Future of digital skills in Europe reskilling and ...
Big Data Value Association
 
BDV Skills Accreditation - Big Data skilling in Emilia-Romagna
BDV Skills Accreditation - Big Data skilling in Emilia-Romagna BDV Skills Accreditation - Big Data skilling in Emilia-Romagna
BDV Skills Accreditation - Big Data skilling in Emilia-Romagna
Big Data Value Association
 
BDV Skills Accreditation - EIT labels for professionals
BDV Skills Accreditation - EIT labels for professionalsBDV Skills Accreditation - EIT labels for professionals
BDV Skills Accreditation - EIT labels for professionals
Big Data Value Association
 
BDV Skills Accreditation - Recognizing Data Science Skills with BDV Data Scie...
BDV Skills Accreditation - Recognizing Data Science Skills with BDV Data Scie...BDV Skills Accreditation - Recognizing Data Science Skills with BDV Data Scie...
BDV Skills Accreditation - Recognizing Data Science Skills with BDV Data Scie...
Big Data Value Association
 
BDV Skills Accreditation - Objectives of the workshop
BDV Skills Accreditation - Objectives of the workshopBDV Skills Accreditation - Objectives of the workshop
BDV Skills Accreditation - Objectives of the workshop
Big Data Value Association
 
BDV Skills Accreditation - Welcome introduction to the workshop
BDV Skills Accreditation - Welcome introduction to the workshopBDV Skills Accreditation - Welcome introduction to the workshop
BDV Skills Accreditation - Welcome introduction to the workshop
Big Data Value Association
 
BDV Skills Accreditation - Definition and ensuring of digital roles and compe...
BDV Skills Accreditation - Definition and ensuring of digital roles and compe...BDV Skills Accreditation - Definition and ensuring of digital roles and compe...
BDV Skills Accreditation - Definition and ensuring of digital roles and compe...
Big Data Value Association
 
BigDataPilotDemoDays - I BiDaaS Application to the Manufacturing Sector Webinar
BigDataPilotDemoDays - I BiDaaS Application to the Manufacturing Sector WebinarBigDataPilotDemoDays - I BiDaaS Application to the Manufacturing Sector Webinar
BigDataPilotDemoDays - I BiDaaS Application to the Manufacturing Sector Webinar
Big Data Value Association
 
BigDataPilotDemoDays - I-BiDaaS Application to the Financial Sector Webinar
BigDataPilotDemoDays - I-BiDaaS Application to the Financial Sector WebinarBigDataPilotDemoDays - I-BiDaaS Application to the Financial Sector Webinar
BigDataPilotDemoDays - I-BiDaaS Application to the Financial Sector Webinar
Big Data Value Association
 
Virtual BenchLearning - Data Bench Framework
Virtual BenchLearning - Data Bench FrameworkVirtual BenchLearning - Data Bench Framework
Virtual BenchLearning - Data Bench Framework
Big Data Value Association
 
Virtual BenchLearning - DeepHealth - Needs & Requirements for Benchmarking
Virtual BenchLearning - DeepHealth - Needs & Requirements for BenchmarkingVirtual BenchLearning - DeepHealth - Needs & Requirements for Benchmarking
Virtual BenchLearning - DeepHealth - Needs & Requirements for Benchmarking
Big Data Value Association
 
Virtual BenchLearning - I-BiDaaS - Industrial-Driven Big Data as a Self-Servi...
Virtual BenchLearning - I-BiDaaS - Industrial-Driven Big Data as a Self-Servi...Virtual BenchLearning - I-BiDaaS - Industrial-Driven Big Data as a Self-Servi...
Virtual BenchLearning - I-BiDaaS - Industrial-Driven Big Data as a Self-Servi...
Big Data Value Association
 
Policy Cloud Data Driven Policies against Radicalisation - Technical Overview
Policy Cloud Data Driven Policies against Radicalisation - Technical OverviewPolicy Cloud Data Driven Policies against Radicalisation - Technical Overview
Policy Cloud Data Driven Policies against Radicalisation - Technical Overview
Big Data Value Association
 
Policy Cloud Data Driven Policies against Radicalisation - Participatory poli...
Policy Cloud Data Driven Policies against Radicalisation - Participatory poli...Policy Cloud Data Driven Policies against Radicalisation - Participatory poli...
Policy Cloud Data Driven Policies against Radicalisation - Participatory poli...
Big Data Value Association
 

More from Big Data Value Association (20)

Data Privacy, Security in personal data sharing
Data Privacy, Security in personal data sharingData Privacy, Security in personal data sharing
Data Privacy, Security in personal data sharing
 
Key Modules for a trsuted and privacy preserving personal data marketplace
Key Modules for a trsuted and privacy preserving personal data marketplaceKey Modules for a trsuted and privacy preserving personal data marketplace
Key Modules for a trsuted and privacy preserving personal data marketplace
 
GDPR and Data Ethics considerations in personal data sharing
GDPR and Data Ethics considerations in personal data sharingGDPR and Data Ethics considerations in personal data sharing
GDPR and Data Ethics considerations in personal data sharing
 
Intro - Three pillars for building a Smart Data Ecosystem: Trust, Security an...
Intro - Three pillars for building a Smart Data Ecosystem: Trust, Security an...Intro - Three pillars for building a Smart Data Ecosystem: Trust, Security an...
Intro - Three pillars for building a Smart Data Ecosystem: Trust, Security an...
 
Three pillars for building a Smart Data Ecosystem: Trust, Security and Privacy
Three pillars for building a Smart Data Ecosystem: Trust, Security and PrivacyThree pillars for building a Smart Data Ecosystem: Trust, Security and Privacy
Three pillars for building a Smart Data Ecosystem: Trust, Security and Privacy
 
Market into context - Three pillars for building a Smart Data Ecosystem: Trus...
Market into context - Three pillars for building a Smart Data Ecosystem: Trus...Market into context - Three pillars for building a Smart Data Ecosystem: Trus...
Market into context - Three pillars for building a Smart Data Ecosystem: Trus...
 
BDV Skills Accreditation - Future of digital skills in Europe reskilling and ...
BDV Skills Accreditation - Future of digital skills in Europe reskilling and ...BDV Skills Accreditation - Future of digital skills in Europe reskilling and ...
BDV Skills Accreditation - Future of digital skills in Europe reskilling and ...
 
BDV Skills Accreditation - Big Data skilling in Emilia-Romagna
BDV Skills Accreditation - Big Data skilling in Emilia-Romagna BDV Skills Accreditation - Big Data skilling in Emilia-Romagna
BDV Skills Accreditation - Big Data skilling in Emilia-Romagna
 
BDV Skills Accreditation - EIT labels for professionals
BDV Skills Accreditation - EIT labels for professionalsBDV Skills Accreditation - EIT labels for professionals
BDV Skills Accreditation - EIT labels for professionals
 
BDV Skills Accreditation - Recognizing Data Science Skills with BDV Data Scie...
BDV Skills Accreditation - Recognizing Data Science Skills with BDV Data Scie...BDV Skills Accreditation - Recognizing Data Science Skills with BDV Data Scie...
BDV Skills Accreditation - Recognizing Data Science Skills with BDV Data Scie...
 
BDV Skills Accreditation - Objectives of the workshop
BDV Skills Accreditation - Objectives of the workshopBDV Skills Accreditation - Objectives of the workshop
BDV Skills Accreditation - Objectives of the workshop
 
BDV Skills Accreditation - Welcome introduction to the workshop
BDV Skills Accreditation - Welcome introduction to the workshopBDV Skills Accreditation - Welcome introduction to the workshop
BDV Skills Accreditation - Welcome introduction to the workshop
 
BDV Skills Accreditation - Definition and ensuring of digital roles and compe...
BDV Skills Accreditation - Definition and ensuring of digital roles and compe...BDV Skills Accreditation - Definition and ensuring of digital roles and compe...
BDV Skills Accreditation - Definition and ensuring of digital roles and compe...
 
BigDataPilotDemoDays - I BiDaaS Application to the Manufacturing Sector Webinar
BigDataPilotDemoDays - I BiDaaS Application to the Manufacturing Sector WebinarBigDataPilotDemoDays - I BiDaaS Application to the Manufacturing Sector Webinar
BigDataPilotDemoDays - I BiDaaS Application to the Manufacturing Sector Webinar
 
BigDataPilotDemoDays - I-BiDaaS Application to the Financial Sector Webinar
BigDataPilotDemoDays - I-BiDaaS Application to the Financial Sector WebinarBigDataPilotDemoDays - I-BiDaaS Application to the Financial Sector Webinar
BigDataPilotDemoDays - I-BiDaaS Application to the Financial Sector Webinar
 
Virtual BenchLearning - Data Bench Framework
Virtual BenchLearning - Data Bench FrameworkVirtual BenchLearning - Data Bench Framework
Virtual BenchLearning - Data Bench Framework
 
Virtual BenchLearning - DeepHealth - Needs & Requirements for Benchmarking
Virtual BenchLearning - DeepHealth - Needs & Requirements for BenchmarkingVirtual BenchLearning - DeepHealth - Needs & Requirements for Benchmarking
Virtual BenchLearning - DeepHealth - Needs & Requirements for Benchmarking
 
Virtual BenchLearning - I-BiDaaS - Industrial-Driven Big Data as a Self-Servi...
Virtual BenchLearning - I-BiDaaS - Industrial-Driven Big Data as a Self-Servi...Virtual BenchLearning - I-BiDaaS - Industrial-Driven Big Data as a Self-Servi...
Virtual BenchLearning - I-BiDaaS - Industrial-Driven Big Data as a Self-Servi...
 
Policy Cloud Data Driven Policies against Radicalisation - Technical Overview
Policy Cloud Data Driven Policies against Radicalisation - Technical OverviewPolicy Cloud Data Driven Policies against Radicalisation - Technical Overview
Policy Cloud Data Driven Policies against Radicalisation - Technical Overview
 
Policy Cloud Data Driven Policies against Radicalisation - Participatory poli...
Policy Cloud Data Driven Policies against Radicalisation - Participatory poli...Policy Cloud Data Driven Policies against Radicalisation - Participatory poli...
Policy Cloud Data Driven Policies against Radicalisation - Participatory poli...
 

Recently uploaded

Econ3060_Screen Time and Success_ final_GroupProject.pdf
Econ3060_Screen Time and Success_ final_GroupProject.pdfEcon3060_Screen Time and Success_ final_GroupProject.pdf
Econ3060_Screen Time and Success_ final_GroupProject.pdf
blueshagoo1
 
一比一原版爱尔兰都柏林大学毕业证(本硕)ucd学位证书如何办理
一比一原版爱尔兰都柏林大学毕业证(本硕)ucd学位证书如何办理一比一原版爱尔兰都柏林大学毕业证(本硕)ucd学位证书如何办理
一比一原版爱尔兰都柏林大学毕业证(本硕)ucd学位证书如何办理
hqfek
 
一比一原版格里菲斯大学毕业证(Griffith毕业证书)学历如何办理
一比一原版格里菲斯大学毕业证(Griffith毕业证书)学历如何办理一比一原版格里菲斯大学毕业证(Griffith毕业证书)学历如何办理
一比一原版格里菲斯大学毕业证(Griffith毕业证书)学历如何办理
lzdvtmy8
 
Template xxxxxxxx ssssssssssss Sertifikat.pptx
Template xxxxxxxx ssssssssssss Sertifikat.pptxTemplate xxxxxxxx ssssssssssss Sertifikat.pptx
Template xxxxxxxx ssssssssssss Sertifikat.pptx
TeukuEriSyahputra
 
Discovering Digital Process Twins for What-if Analysis: a Process Mining Appr...
Discovering Digital Process Twins for What-if Analysis: a Process Mining Appr...Discovering Digital Process Twins for What-if Analysis: a Process Mining Appr...
Discovering Digital Process Twins for What-if Analysis: a Process Mining Appr...
Marlon Dumas
 
06-20-2024-AI Camp Meetup-Unstructured Data and Vector Databases
06-20-2024-AI Camp Meetup-Unstructured Data and Vector Databases06-20-2024-AI Camp Meetup-Unstructured Data and Vector Databases
06-20-2024-AI Camp Meetup-Unstructured Data and Vector Databases
Timothy Spann
 
Namma-Kalvi-11th-Physics-Study-Material-Unit-1-EM-221086.pdf
Namma-Kalvi-11th-Physics-Study-Material-Unit-1-EM-221086.pdfNamma-Kalvi-11th-Physics-Study-Material-Unit-1-EM-221086.pdf
Namma-Kalvi-11th-Physics-Study-Material-Unit-1-EM-221086.pdf
22ad0301
 
社内勉強会資料_Hallucination of LLMs               .
社内勉強会資料_Hallucination of LLMs               .社内勉強会資料_Hallucination of LLMs               .
社内勉強会資料_Hallucination of LLMs               .
NABLAS株式会社
 
A gentle exploration of Retrieval Augmented Generation
A gentle exploration of Retrieval Augmented GenerationA gentle exploration of Retrieval Augmented Generation
A gentle exploration of Retrieval Augmented Generation
dataschool1
 
一比一原版多伦多大学毕业证(UofT毕业证书)学历如何办理
一比一原版多伦多大学毕业证(UofT毕业证书)学历如何办理一比一原版多伦多大学毕业证(UofT毕业证书)学历如何办理
一比一原版多伦多大学毕业证(UofT毕业证书)学历如何办理
eoxhsaa
 
一比一原版莱斯大学毕业证(rice毕业证)如何办理
一比一原版莱斯大学毕业证(rice毕业证)如何办理一比一原版莱斯大学毕业证(rice毕业证)如何办理
一比一原版莱斯大学毕业证(rice毕业证)如何办理
zsafxbf
 
一比一原版斯威本理工大学毕业证(swinburne毕业证)如何办理
一比一原版斯威本理工大学毕业证(swinburne毕业证)如何办理一比一原版斯威本理工大学毕业证(swinburne毕业证)如何办理
一比一原版斯威本理工大学毕业证(swinburne毕业证)如何办理
actyx
 
Cell The Unit of Life for NEET Multiple Choice Questions.docx
Cell The Unit of Life for NEET Multiple Choice Questions.docxCell The Unit of Life for NEET Multiple Choice Questions.docx
Cell The Unit of Life for NEET Multiple Choice Questions.docx
vasanthatpuram
 
Data Scientist Machine Learning Profiles .pdf
Data Scientist Machine Learning  Profiles .pdfData Scientist Machine Learning  Profiles .pdf
Data Scientist Machine Learning Profiles .pdf
Vineet
 
原版一比一多伦多大学毕业证(UofT毕业证书)如何办理
原版一比一多伦多大学毕业证(UofT毕业证书)如何办理原版一比一多伦多大学毕业证(UofT毕业证书)如何办理
原版一比一多伦多大学毕业证(UofT毕业证书)如何办理
mkkikqvo
 
Sample Devops SRE Product Companies .pdf
Sample Devops SRE  Product Companies .pdfSample Devops SRE  Product Companies .pdf
Sample Devops SRE Product Companies .pdf
Vineet
 
Digital Marketing Performance Marketing Sample .pdf
Digital Marketing Performance Marketing  Sample .pdfDigital Marketing Performance Marketing  Sample .pdf
Digital Marketing Performance Marketing Sample .pdf
Vineet
 
一比一原版(UO毕业证)渥太华大学毕业证如何办理
一比一原版(UO毕业证)渥太华大学毕业证如何办理一比一原版(UO毕业证)渥太华大学毕业证如何办理
一比一原版(UO毕业证)渥太华大学毕业证如何办理
bmucuha
 
原版一比一爱尔兰都柏林大学毕业证(UCD毕业证书)如何办理
原版一比一爱尔兰都柏林大学毕业证(UCD毕业证书)如何办理 原版一比一爱尔兰都柏林大学毕业证(UCD毕业证书)如何办理
原版一比一爱尔兰都柏林大学毕业证(UCD毕业证书)如何办理
tzu5xla
 
Drownings spike from May to August in children
Drownings spike from May to August in childrenDrownings spike from May to August in children
Drownings spike from May to August in children
Bisnar Chase Personal Injury Attorneys
 

Recently uploaded (20)

Econ3060_Screen Time and Success_ final_GroupProject.pdf
Econ3060_Screen Time and Success_ final_GroupProject.pdfEcon3060_Screen Time and Success_ final_GroupProject.pdf
Econ3060_Screen Time and Success_ final_GroupProject.pdf
 
一比一原版爱尔兰都柏林大学毕业证(本硕)ucd学位证书如何办理
一比一原版爱尔兰都柏林大学毕业证(本硕)ucd学位证书如何办理一比一原版爱尔兰都柏林大学毕业证(本硕)ucd学位证书如何办理
一比一原版爱尔兰都柏林大学毕业证(本硕)ucd学位证书如何办理
 
一比一原版格里菲斯大学毕业证(Griffith毕业证书)学历如何办理
一比一原版格里菲斯大学毕业证(Griffith毕业证书)学历如何办理一比一原版格里菲斯大学毕业证(Griffith毕业证书)学历如何办理
一比一原版格里菲斯大学毕业证(Griffith毕业证书)学历如何办理
 
Template xxxxxxxx ssssssssssss Sertifikat.pptx
Template xxxxxxxx ssssssssssss Sertifikat.pptxTemplate xxxxxxxx ssssssssssss Sertifikat.pptx
Template xxxxxxxx ssssssssssss Sertifikat.pptx
 
Discovering Digital Process Twins for What-if Analysis: a Process Mining Appr...
Discovering Digital Process Twins for What-if Analysis: a Process Mining Appr...Discovering Digital Process Twins for What-if Analysis: a Process Mining Appr...
Discovering Digital Process Twins for What-if Analysis: a Process Mining Appr...
 
06-20-2024-AI Camp Meetup-Unstructured Data and Vector Databases
06-20-2024-AI Camp Meetup-Unstructured Data and Vector Databases06-20-2024-AI Camp Meetup-Unstructured Data and Vector Databases
06-20-2024-AI Camp Meetup-Unstructured Data and Vector Databases
 
Namma-Kalvi-11th-Physics-Study-Material-Unit-1-EM-221086.pdf
Namma-Kalvi-11th-Physics-Study-Material-Unit-1-EM-221086.pdfNamma-Kalvi-11th-Physics-Study-Material-Unit-1-EM-221086.pdf
Namma-Kalvi-11th-Physics-Study-Material-Unit-1-EM-221086.pdf
 
社内勉強会資料_Hallucination of LLMs               .
社内勉強会資料_Hallucination of LLMs               .社内勉強会資料_Hallucination of LLMs               .
社内勉強会資料_Hallucination of LLMs               .
 
A gentle exploration of Retrieval Augmented Generation
A gentle exploration of Retrieval Augmented GenerationA gentle exploration of Retrieval Augmented Generation
A gentle exploration of Retrieval Augmented Generation
 
一比一原版多伦多大学毕业证(UofT毕业证书)学历如何办理
一比一原版多伦多大学毕业证(UofT毕业证书)学历如何办理一比一原版多伦多大学毕业证(UofT毕业证书)学历如何办理
一比一原版多伦多大学毕业证(UofT毕业证书)学历如何办理
 
一比一原版莱斯大学毕业证(rice毕业证)如何办理
一比一原版莱斯大学毕业证(rice毕业证)如何办理一比一原版莱斯大学毕业证(rice毕业证)如何办理
一比一原版莱斯大学毕业证(rice毕业证)如何办理
 
一比一原版斯威本理工大学毕业证(swinburne毕业证)如何办理
一比一原版斯威本理工大学毕业证(swinburne毕业证)如何办理一比一原版斯威本理工大学毕业证(swinburne毕业证)如何办理
一比一原版斯威本理工大学毕业证(swinburne毕业证)如何办理
 
Cell The Unit of Life for NEET Multiple Choice Questions.docx
Cell The Unit of Life for NEET Multiple Choice Questions.docxCell The Unit of Life for NEET Multiple Choice Questions.docx
Cell The Unit of Life for NEET Multiple Choice Questions.docx
 
Data Scientist Machine Learning Profiles .pdf
Data Scientist Machine Learning  Profiles .pdfData Scientist Machine Learning  Profiles .pdf
Data Scientist Machine Learning Profiles .pdf
 
原版一比一多伦多大学毕业证(UofT毕业证书)如何办理
原版一比一多伦多大学毕业证(UofT毕业证书)如何办理原版一比一多伦多大学毕业证(UofT毕业证书)如何办理
原版一比一多伦多大学毕业证(UofT毕业证书)如何办理
 
Sample Devops SRE Product Companies .pdf
Sample Devops SRE  Product Companies .pdfSample Devops SRE  Product Companies .pdf
Sample Devops SRE Product Companies .pdf
 
Digital Marketing Performance Marketing Sample .pdf
Digital Marketing Performance Marketing  Sample .pdfDigital Marketing Performance Marketing  Sample .pdf
Digital Marketing Performance Marketing Sample .pdf
 
一比一原版(UO毕业证)渥太华大学毕业证如何办理
一比一原版(UO毕业证)渥太华大学毕业证如何办理一比一原版(UO毕业证)渥太华大学毕业证如何办理
一比一原版(UO毕业证)渥太华大学毕业证如何办理
 
原版一比一爱尔兰都柏林大学毕业证(UCD毕业证书)如何办理
原版一比一爱尔兰都柏林大学毕业证(UCD毕业证书)如何办理 原版一比一爱尔兰都柏林大学毕业证(UCD毕业证书)如何办理
原版一比一爱尔兰都柏林大学毕业证(UCD毕业证书)如何办理
 
Drownings spike from May to August in children
Drownings spike from May to August in childrenDrownings spike from May to August in children
Drownings spike from May to August in children
 

BDVe Webinar Series - Big Data Ocean - Rocking the boat with Big Data

  • 1. Rocking the boat with big data www.bigdataocean.eu Dr. Spiros Mouzakitis (NTUA) Deputy Project Manager GiannisTsapelas(NTUA) Senior Researcher
  • 2. Around 80% of global trade by volume is carried by sEA ….and around 70% of global trade by value
  • 3. CHALLEN GESo Undeveloped sharing of Blue Data between enterprises and entities of the maritime domain and other domains o Lack of agreed standards and formats o Huge potential from cross-sectorial blue data applications – still unexploited o Out-of-the-box Big Data solution that enables advanced queries and analytics on cross-sector data - missing
  • 4. Develop a Maritime Big Data platform that delivers out-of-the-box, value-added data and analytic services for maritime applications by exploiting cross-sector data streams 4
  • 5.
  • 6. BigDataOcean platform (services) Find maritime data Query / interlink datasets Compose analytic services Visualize datasets Create (real-time) dashboards Request on-demand services
  • 8. BigDataOcean end-user applications Vessel Fault Detection, Predictive Maintenance and Fuel Consumption Reduction Maritime security and anomaly detection Oil spill modeling Wave power generation 8BDV PPP Technical Committee Meeting
  • 9. Case 1 - Fault Prediction & Proactive Maintenance and Fuel Consumption 9
  • 10. Case 1 - Fault Prediction & Proactive Maintenance and Fuel Consumption 10 • Damage and mechanical failures detection and predictive maintenance of vessel equipment • Investigation of the impact of the environmental conditions and the operational decisions taken on the vessel's fuel consumption Goal • Ship owners, Maritime EquipmentConstructors Stakeholders
  • 11. BigDataOcean Solution Analytics and knowledge base about fuel consumption and repairs Prediction models for maintenance and fuel consumption Maritime Regulations, Flag & Ship Classification Provisions Plant Maintenance System for Main Engine & spare parts Vessel routes In-situ Observations Cross-domain Forecast Data Port information BigDataOcean Solution
  • 12.
  • 13.
  • 14.
  • 15. Business Benefits ● Shipping companies and Maritime Equipment Constructors  Minimum repairs, maintenance cost and fuel consumption  Maximum vessels’ use and financial benefit  Minimize environment impact  Reliability and innovation  Advanced Data analytics related to maintenance and fuel consumption
  • 16. Case 2 – Mare protection – Oil Spill dispersion forecast 16
  • 17. Case 2 – Mare protection – Oil Spill dispersion forecast 17 • Provide to the end users oil spill drift forecasting and simulation services for the marine environment • Enhance the efficiency in managing oil spill pollution risks. Goal • Emergency Response Companies, National Entities (PublicAuthorities), NGOs, Marine Research Institutes, Shipping companies and Oil drilling companies Stakeholders
  • 18. Natura / Protecte d areas In-situ Observations BigDataOcean Solution Graphical output Cross-domain Forecast Data POSEIDON OSM Oil spill scenario submission Location, Rate, Nature & characteristics Ocean Circulation Forecast Weather Forecast BigDataOcean Solution AIS Data BigDataOcean Wave Forecast Oil Spill Dispersion Forecast Acquisition High Risk PollutionAreas UnderwaterAccident
  • 19.
  • 20.
  • 21.
  • 22. Business Benefits ● Extended knowledge, models and enriched datasets ● New products addressed to environment protection organizations and maritime authorities for rapid intervention against oil spills in the sea ● Control and limit impact and damage on the coast and on essential resources and structures. ● Efficiency in the protection of the marine environment and of the marine life.
  • 23. Case 3 – Wave Power as Clean Energy Source 23
  • 24. Case 3 – Wave Power as Clean Energy Source 24 • Evaluation of wave energy potential and contribution to development of wave energy solutions. Goal • Offshore Renewables Service Providers, Offshore Pilot Zone Concessionaires,WEC developers, Energy Producers, Hydrographic Centres Stakeholders
  • 25. BigDataOcean Solution Waves In-situ Observations Cross-domain Vessel routes Port s Protected areas Wave models output Instruments Data BigDataOcean Solution Wave ResourceAssessment Wave Energy study and Forecast DataVisualisations
  • 26.
  • 27.
  • 28. Business Benefits ● Wave resource characterization in your selected location or area, based in historical data. ● Wave forecast. ● Assessment of Wave Energy Converters energy generation, allowing to compare your device with others. ● Forecast of energy generation for your WEC device.
  • 29. Case 4 – Security and anomaly path detection
  • 30. Case 4 – Security and anomaly path detection • Identify vessel routes based on their motion patterns to act proactively and minimise threats at sea. Goal • Port authorities, Ocean Observatories, Port/Cargo Community systems,Transport and Logistics companies, Harbour Pilots and Maritime Consultants Stakeholders
  • 31. BigDataOcean solution AIS Anomaly Detection BigDataOcean Solution Visualization Anomaly detection services Analytics AIS data Weather Data Location of Sea Ports List of “High Risk” Vessels List of security incidents & vessels implicated Nautical Information Maps Feedback from vessel’s crew & domain experts
  • 32.
  • 33.
  • 34. Business Benefits ● Effectively handle the information volume from tracking technologies and AIS data ● identify patterns of behavior and vessel risk profile ● proactively minimize the impact of possible threats
  • 36. Timeplan Analysis and Development 2017 6/2019 Start MS4: Prototype MS2: Requirements and needs Q1 Q2 Q3 Q4 Q1 Q2 Q3 Q4 Q1 Q2 MS1: Maritime data value chain definition MS3: Methodology & Architecture MS5: Final Platform MS6: Business Plan, Lessons Learnt Evaluation Market preparation We are here
  • 37. Lessons learned: exploitation 37 BDO Commercial Partnership for the platform 1. BDO Platform Services 2. BDO End-UsersApplication Services 3. BDO On-Demand Data Services and custom solutions 4. BDO Non-Technical Package • Training • Consulting • Data science coaching • Contract deals with maritime companies • Focus on B2B applications
  • 38. Semantic Challenges ● Plethora of file formats and metadata standards coming from diverse data sources ● Ability to perform advanced queries that combine those big datasets ● Enable ad hoc queries Our Approach ● Automatic ingestion system to a harmonized, semantically aware schema (BDO canonical Model) ● Based on DCAT, NETCDF convention standards (cfconventions.org)
  • 39. Performance Challenges ● Performance issues  POSTGRESQL and relational databases typically used in legacy maritime applications – Serious scalability issues for the size of the datasets  NoSQL databases -> Good performance for simple queries, but bad performance for queries that combine data  Distributed, wide column stores (e.g. Cassandra) -> better optimised for known queries Our Approach ● Presto with Apache Hive
  • 40. Lessons learned: current solution Solution for Query Designer queries Solution for further query optimization ● Caching JOIN queries performed by users ● Pre-join datasets for pilot applications ● Adoption of Apache Parquet storage and ORF file formats HDFS Query performance over million of rows: from 5 minutes to 5 seconds!

Editor's Notes

  1. Using the capabilities of the big data ocean platform we have created 4 applications that deal with crucial challenges for the maritime industry – these applications are our flagship products to kickstart promote the platform
  2. Its main goal is to support the first and most important step of the wave energy convertors. Where is the best place to place the WEC and and it’s the potential of the installation depending on the location at sea. Its stakeholders are….