Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Rocking the boat with big
data
www.bigdataocean.eu
Dr. Spiros Mouzakitis (NTUA)
Deputy Project Manager
GiannisTsapelas(NTU...
Around 80% of global trade by volume is carried
by sEA
….and around 70% of global
trade by value
CHALLEN
GESo Undeveloped sharing of Blue Data between
enterprises and entities of the maritime domain
and other domains
o ...
Develop a Maritime Big Data platform
that delivers out-of-the-box, value-added data and
analytic services for maritime app...
BigDataOcean platform (services)
Find maritime data
Query / interlink datasets Compose analytic services
Visualize dataset...
Port
Authorities
Maritime
Research
Centers
Ship-owning
companies
Maritime
software
suppliers
Service
providers at
sea leve...
BigDataOcean end-user applications
Vessel Fault
Detection,
Predictive
Maintenance
and Fuel
Consumption
Reduction
Maritime
...
Case 1 - Fault Prediction & Proactive Maintenance
and Fuel Consumption
9
Case 1 - Fault Prediction & Proactive Maintenance
and Fuel Consumption
10
• Damage and mechanical failures detection and p...
BigDataOcean Solution
Analytics and knowledge
base about fuel
consumption and repairs
Prediction models for
maintenance an...
Business Benefits
● Shipping companies and Maritime Equipment Constructors
 Minimum repairs, maintenance cost and fuel co...
Case 2 – Mare protection – Oil Spill dispersion
forecast
16
Case 2 – Mare protection – Oil Spill dispersion
forecast
17
• Provide to the end users oil spill drift forecasting and
sim...
Natura /
Protecte
d areas
In-situ Observations
BigDataOcean Solution
Graphical output
Cross-domain
Forecast Data
POSEIDON
...
Business Benefits
● Extended knowledge, models and enriched datasets
● New products addressed to environment protection
or...
Case 3 – Wave Power as Clean Energy Source
23
Case 3 – Wave Power as Clean Energy Source
24
• Evaluation of wave energy potential and contribution to
development of wav...
BigDataOcean Solution
Waves In-situ
Observations
Cross-domain
Vessel
routes
Port
s
Protected
areas
Wave models
output
Inst...
Business Benefits
● Wave resource characterization in your selected location or
area, based in historical data.
● Wave for...
Case 4 – Security and anomaly path detection
Case 4 – Security and anomaly path detection
• Identify vessel routes based on their motion patterns to act
proactively an...
BigDataOcean solution
AIS
Anomaly Detection
BigDataOcean
Solution
Visualization
Anomaly
detection
services
Analytics
AIS
d...
Business Benefits
● Effectively handle the information volume from tracking
technologies and AIS data
● identify patterns ...
BigDataOcean Consortium
35
35
Development
Semantics
Pilot
Pilot
PilotPilot
Requirements
Exploitation
Coordination &
Develo...
Timeplan
Analysis and Development
2017 6/2019
Start
MS4:
Prototype
MS2:
Requirements
and needs
Q1 Q2 Q3 Q4 Q1 Q2 Q3 Q4 Q1 ...
Lessons learned: exploitation
37
BDO Commercial Partnership for the platform
1. BDO Platform Services
2. BDO End-UsersAppl...
Semantic Challenges
● Plethora of file formats and metadata standards coming from diverse data sources
● Ability to perfor...
Performance Challenges
● Performance issues
 POSTGRESQL and relational databases typically used in legacy maritime applic...
Lessons learned: current solution
Solution for Query Designer
queries
Solution for further query
optimization
● Caching JO...
BDO Architecture
Demo
www.bigdataocean.eu
Questions
www.bigdataocean.eu
BDVe Webinar Series - Big Data Ocean - Rocking the boat with Big Data
BDVe Webinar Series - Big Data Ocean - Rocking the boat with Big Data
BDVe Webinar Series - Big Data Ocean - Rocking the boat with Big Data
BDVe Webinar Series - Big Data Ocean - Rocking the boat with Big Data
BDVe Webinar Series - Big Data Ocean - Rocking the boat with Big Data
BDVe Webinar Series - Big Data Ocean - Rocking the boat with Big Data
BDVe Webinar Series - Big Data Ocean - Rocking the boat with Big Data
BDVe Webinar Series - Big Data Ocean - Rocking the boat with Big Data
BDVe Webinar Series - Big Data Ocean - Rocking the boat with Big Data
BDVe Webinar Series - Big Data Ocean - Rocking the boat with Big Data
BDVe Webinar Series - Big Data Ocean - Rocking the boat with Big Data
You’ve finished this document.
Download and read it offline.
Upcoming SlideShare
What to Upload to SlideShare
Next
Upcoming SlideShare
What to Upload to SlideShare
Next
Download to read offline and view in fullscreen.

0

Share

BDVe Webinar Series - Big Data Ocean - Rocking the boat with Big Data

Download to read offline

BigDataOcean brings a digital revolution to the maritime industry by creating a large maritime big data infrastructure that enables collaborative, data-driven intelligence. BigDataOcean will allow analytics based on diverse data resources, coming from public and private providers. In this webinar, Spyros Mouzakitis and Giannis Tsapelas will present a demo of the BigDataOcean platform and discuss the challenges and lessons learned so far.

  • Be the first to like this

BDVe Webinar Series - Big Data Ocean - Rocking the boat with Big Data

  1. 1. Rocking the boat with big data www.bigdataocean.eu Dr. Spiros Mouzakitis (NTUA) Deputy Project Manager GiannisTsapelas(NTUA) Senior Researcher
  2. 2. Around 80% of global trade by volume is carried by sEA ….and around 70% of global trade by value
  3. 3. CHALLEN GESo Undeveloped sharing of Blue Data between enterprises and entities of the maritime domain and other domains o Lack of agreed standards and formats o Huge potential from cross-sectorial blue data applications – still unexploited o Out-of-the-box Big Data solution that enables advanced queries and analytics on cross-sector data - missing
  4. 4. Develop a Maritime Big Data platform that delivers out-of-the-box, value-added data and analytic services for maritime applications by exploiting cross-sector data streams 4
  5. 5. BigDataOcean platform (services) Find maritime data Query / interlink datasets Compose analytic services Visualize datasets Create (real-time) dashboards Request on-demand services
  6. 6. Port Authorities Maritime Research Centers Ship-owning companies Maritime software suppliers Service providers at sea level Maritime advisory firms
  7. 7. BigDataOcean end-user applications Vessel Fault Detection, Predictive Maintenance and Fuel Consumption Reduction Maritime security and anomaly detection Oil spill modeling Wave power generation 8BDV PPP Technical Committee Meeting
  8. 8. Case 1 - Fault Prediction & Proactive Maintenance and Fuel Consumption 9
  9. 9. Case 1 - Fault Prediction & Proactive Maintenance and Fuel Consumption 10 • Damage and mechanical failures detection and predictive maintenance of vessel equipment • Investigation of the impact of the environmental conditions and the operational decisions taken on the vessel's fuel consumption Goal • Ship owners, Maritime EquipmentConstructors Stakeholders
  10. 10. BigDataOcean Solution Analytics and knowledge base about fuel consumption and repairs Prediction models for maintenance and fuel consumption Maritime Regulations, Flag & Ship Classification Provisions Plant Maintenance System for Main Engine & spare parts Vessel routes In-situ Observations Cross-domain Forecast Data Port information BigDataOcean Solution
  11. 11. Business Benefits ● Shipping companies and Maritime Equipment Constructors  Minimum repairs, maintenance cost and fuel consumption  Maximum vessels’ use and financial benefit  Minimize environment impact  Reliability and innovation  Advanced Data analytics related to maintenance and fuel consumption
  12. 12. Case 2 – Mare protection – Oil Spill dispersion forecast 16
  13. 13. Case 2 – Mare protection – Oil Spill dispersion forecast 17 • Provide to the end users oil spill drift forecasting and simulation services for the marine environment • Enhance the efficiency in managing oil spill pollution risks. Goal • Emergency Response Companies, National Entities (PublicAuthorities), NGOs, Marine Research Institutes, Shipping companies and Oil drilling companies Stakeholders
  14. 14. Natura / Protecte d areas In-situ Observations BigDataOcean Solution Graphical output Cross-domain Forecast Data POSEIDON OSM Oil spill scenario submission Location, Rate, Nature & characteristics Ocean Circulation Forecast Weather Forecast BigDataOcean Solution AIS Data BigDataOcean Wave Forecast Oil Spill Dispersion Forecast Acquisition High Risk PollutionAreas UnderwaterAccident
  15. 15. Business Benefits ● Extended knowledge, models and enriched datasets ● New products addressed to environment protection organizations and maritime authorities for rapid intervention against oil spills in the sea ● Control and limit impact and damage on the coast and on essential resources and structures. ● Efficiency in the protection of the marine environment and of the marine life.
  16. 16. Case 3 – Wave Power as Clean Energy Source 23
  17. 17. Case 3 – Wave Power as Clean Energy Source 24 • Evaluation of wave energy potential and contribution to development of wave energy solutions. Goal • Offshore Renewables Service Providers, Offshore Pilot Zone Concessionaires,WEC developers, Energy Producers, Hydrographic Centres Stakeholders
  18. 18. BigDataOcean Solution Waves In-situ Observations Cross-domain Vessel routes Port s Protected areas Wave models output Instruments Data BigDataOcean Solution Wave ResourceAssessment Wave Energy study and Forecast DataVisualisations
  19. 19. Business Benefits ● Wave resource characterization in your selected location or area, based in historical data. ● Wave forecast. ● Assessment of Wave Energy Converters energy generation, allowing to compare your device with others. ● Forecast of energy generation for your WEC device.
  20. 20. Case 4 – Security and anomaly path detection
  21. 21. Case 4 – Security and anomaly path detection • Identify vessel routes based on their motion patterns to act proactively and minimise threats at sea. Goal • Port authorities, Ocean Observatories, Port/Cargo Community systems,Transport and Logistics companies, Harbour Pilots and Maritime Consultants Stakeholders
  22. 22. BigDataOcean solution AIS Anomaly Detection BigDataOcean Solution Visualization Anomaly detection services Analytics AIS data Weather Data Location of Sea Ports List of “High Risk” Vessels List of security incidents & vessels implicated Nautical Information Maps Feedback from vessel’s crew & domain experts
  23. 23. Business Benefits ● Effectively handle the information volume from tracking technologies and AIS data ● identify patterns of behavior and vessel risk profile ● proactively minimize the impact of possible threats
  24. 24. BigDataOcean Consortium 35 35 Development Semantics Pilot Pilot PilotPilot Requirements Exploitation Coordination & Development Pilot
  25. 25. Timeplan Analysis and Development 2017 6/2019 Start MS4: Prototype MS2: Requirements and needs Q1 Q2 Q3 Q4 Q1 Q2 Q3 Q4 Q1 Q2 MS1: Maritime data value chain definition MS3: Methodology & Architecture MS5: Final Platform MS6: Business Plan, Lessons Learnt Evaluation Market preparation We are here
  26. 26. Lessons learned: exploitation 37 BDO Commercial Partnership for the platform 1. BDO Platform Services 2. BDO End-UsersApplication Services 3. BDO On-Demand Data Services and custom solutions 4. BDO Non-Technical Package • Training • Consulting • Data science coaching • Contract deals with maritime companies • Focus on B2B applications
  27. 27. Semantic Challenges ● Plethora of file formats and metadata standards coming from diverse data sources ● Ability to perform advanced queries that combine those big datasets ● Enable ad hoc queries Our Approach ● Automatic ingestion system to a harmonized, semantically aware schema (BDO canonical Model) ● Based on DCAT, NETCDF convention standards (cfconventions.org)
  28. 28. Performance Challenges ● Performance issues  POSTGRESQL and relational databases typically used in legacy maritime applications – Serious scalability issues for the size of the datasets  NoSQL databases -> Good performance for simple queries, but bad performance for queries that combine data  Distributed, wide column stores (e.g. Cassandra) -> better optimised for known queries Our Approach ● Presto with Apache Hive
  29. 29. Lessons learned: current solution Solution for Query Designer queries Solution for further query optimization ● Caching JOIN queries performed by users ● Pre-join datasets for pilot applications ● Adoption of Apache Parquet storage and ORF file formats HDFS Query performance over million of rows: from 5 minutes to 5 seconds!
  30. 30. BDO Architecture
  31. 31. Demo www.bigdataocean.eu
  32. 32. Questions www.bigdataocean.eu

BigDataOcean brings a digital revolution to the maritime industry by creating a large maritime big data infrastructure that enables collaborative, data-driven intelligence. BigDataOcean will allow analytics based on diverse data resources, coming from public and private providers. In this webinar, Spyros Mouzakitis and Giannis Tsapelas will present a demo of the BigDataOcean platform and discuss the challenges and lessons learned so far.

Views

Total views

278

On Slideshare

0

From embeds

0

Number of embeds

0

Actions

Downloads

2

Shares

0

Comments

0

Likes

0

×