Wind power big_data_and_iot_forum_bigdataeurope

Big Data Europe
for System Monitoring
BigDataEurope in Wind Power Big Data and IoT Forum31-mars-17
F. Mouzakis (CRES)
and BDE consortium

BigDataEurope in 2nd Wind Power Big Data and IoT Forum Workshop, Amsterdam 31/3/17
System Monitoring p.2
Overview
Project outline
Big Data Europe platform
Data acquisition challenge
Use case in WT monitoring
BDE opportunities
www.big-data-europe.eu

Project rationale
Show societal value of Big Data
Lower barrier for using big data technologies
o Required effort and resources
o Limited data science skills
Help establishing cross-lingual/
organizational/domain Data Value Chains

BDE Consortium

Overview

Big Data
Landscape
(Matt Turck)

Tool groups
Big Data
Technologies
Data Storage
Technologies
Data
Processing
Workflow
Coordination
Querying/
Processing
Search
Data
Export/
Import
Data
Analysis
Text Mining

Open source technologies for Big Data Apps
www.big-data-europe.eu 8

Big Data Technologies vs 3Vs
Volume
VelocityVariety
Storm

BDE Platform
BDE platform consists of 3 layers:
• the hardware layer
• a resource manager – Docker Swarm – and
• Big Data applications running on top
An application can be seen as a pipeline consisting of multiple components,
like HDFS, Spark and Kafka, which are wired together in order to solve a specific Big Data
problem. The components will be packaged in Docker containers and glued together
with Docker Compose.

Platform architecture

Platform goals
Low total cost of ownership
Simple to get started with Big Data
Cater for widely varying use cases
Embrace emerging Big Data technologies
Simple integration with custom components

https://www.big-data-europe.eu/howto-install-the-bde-platform/
https://github.com/big-data-europe
Youtube: Getting Started With BDE Platform
Install BDE platform
Technical support: TENFORCE
Aad Versteden aad.versteden@tenforce.com

Platform architecture (components)

General requirements in System monitoring
Modular distributed system using standard Ethernet network.
Specs compliant with International Standards.
Robust and reliable for 24/7 standalone non-stop operation.
Scalable and reconfigurable.
Time Synchronization across all modules (GPS or Master module).
Embedded processing capabilities.
Send notifications and alarms.
Data storage

Typical DAQ system core component (1/2)
NI-CompactRIO platform http://www.ni.com/compactrio/
Designed for harsh environments
-20º to -55ºC temperature, 5grms vibration, 30g shock
Low power requirements
~10W at 9-30V for battery powered standalone operation
Data acquisition based on FPGA hardware offering fast I/O response times
and increased reliability
Source: NI.com

Typical DAQ system core component (2/2)
Various modules for Analogue I/O and Digital I/O
High precision (24-bit delta-sigma) A/D converters (s.g. modules)
16-bit 100kHz Simultaneous Sampling AI modules.
XML - Configuration file
Raw (high speed) data sent over the Ethernet and stored on a NAS.
All data packets are time-stamped with common GPS or local time.
Backup storage on each module USB port, in case of network loss.
Source: NI.com

Architecture
Real-Time microprocessor:
• Runs a real-time OS (NI Linux Real-Time)
• Communicates and uploads data
• Performs statistical analysis
• Stores and compresses raw data
FPGA microprocessor:
• Acquires signals (analogue & digital I/O)
• Runs without OS at high clock rates and no
jitter
• Executes different tasks with true parallelism
• Performs low-level signal processing
(i.e.: filtering, control)
• Assures non-stop operation
Source: NI.com

Field-Programmable Gate Array (FPGA)
- Reconfigurable Hardware chip.
- Analogous to a printed circuit board with unconnected components on it.
- Connections in an FPGA circuit are dynamically defined in software.

Example of FPGA implementation
Boolean operations (green lines) based on
digital I/O (A,B,C,D,F) execute in parallel with
math operations (orange lines) based on
analogue I/O (W, X, Y, Z).
The compiler produces an optimized bitstream
file, with all the routing paths, between millions
of logic gates, memory blocks and I/O channels.
Source: NI.com

Typical configuration
. . .
Gigabit Local Area Network (LAN)
Network Attached Storage (NAS)System Monitoring PC

Raw Data storage – Existing File Formats
ASCII files
Characteristics
• Human-readable
• Easily portable to other applications such as
Microsoft Excel
Considerations
• Significantly larger disk footprint
• Slow read and write

Binary files
Characteristics
• Compact file size
• Fast streaming (read & write)
Considerations
• Not human-readable
• Not easily exchangeable

XML files
Characteristics
• Stores complex data structures
• Shows display in a Web browser or in a text
editor
Considerations
• Even larger disk footprint
• Front-end schema design
• Does not stream

Database files
Characteristics
• Store data centrally
• Organize and query test results with SQL
Considerations
• Time intensive startup effort
• Requires maintenance
• Potentially high cost

Binary Structured (TDMS – NI Technical Data Management Technology)
Characteristics
• Single streaming binary file
• Three levels of hierarchy for better organization
File, groups, and channels
• Customizable, descriptive properties at each level
• User-defined Meta data for campaign properties
Considerations
• Third party format

File Format Comparison
Source: NI.com

Title: System monitoring in energy production units – Research on CMS
Objective: Provide a tool for handling the analysis of large data sets, tailored for
distributed and highly sampled sensors; offer to the community open data and tools
Field/Stakeholders: The pilot addresses RES developers, researchers in CMS systems,
manufacturers, CMS and IT service providers in wind energy field
Other Use: Sensor data cases (transport, manufacturing and other related industrial fields)
Use Case (Scenario, Pilot)

The management of large number of distributed energy production systems present challenges in
operation monitoring, maintenance and production forecasting. In renewable energy production, the
stochastic nature of the resource input, poses an additional challenge.
In order to meet the above challenges, an extensive network of sensors operates and produces data
continually supporting decision making. In research field experimental systems are generating data that
feed procedures under development.
Constraints in data volume lead to degradation of their value.
For the reduction of energy cost, the increase of production reliability and the increase of renewables
penetration (all major aspects of the societal challenge) the new developments in Information
Technology have to be combined with the Operation Technology exploiting in a holistic
approach the data collection, management and analytics.
Problem Definition

Type: SCADA and CMS data on a Wind Turbine specifically instrumented for BDE
Status: Open
Sensors: Operational parameters, Vibration, Mechanical Loading, Power Quality etc
Format: Third party (TDMS technical data management streaming) – preprocessing foreseen
Acquisition technology: Field Programmable Gate Arrays (FPGA)
Sampling rate: from 10s/s for operational parameters up to 64ks/s for vibration & power quality
Streaming volume: 3 distributed units yield ~14Gb/hour continuously
Scope: Long term monitoring for drive train condition monitoring research
Analytics: engineering signal analysis, research on parametrics, fault identification and loop with
updated methodologies on raw data
Operation of pilot: 1 year (on-going)
Primary Content/Data Involved

CRES Wind Farm
Monitored WT
Neg-Micon 750kW

The monitoring system provides indicatively the following data
for the condition monitoring system:
- system operational statuses/parameters
- wind turbine output electrical power
- nacelle yaw position, yaw motor electrical power
- wind speed from nacelle anemometer (1~10s/s)
and additionally:
- mechanical loads on tower top and base cross section
(~100s/s)
- rotor thrust & tower torsion
- HSS torque (on shaft coupling the gearbox with the
generator) (64ks/s)
- power quality current & voltage (64ks/s)
- vibration at gearbox various stages (64ks/s)
- acoustic emission signals at gearbox (Ms/s)
CRES CM system on NegMicon 750kW WT

SC3 Pilot measuring system
Gearbox and Drive train DAQ systems
Operation DAQ
system
Supervising PC
Local storage
unit

Bearing vibration
sensors at
gearbox HSS
(accelerometers)
Power Quality voltage and current probes
SC3 Pilot sample sensors

BDE pilot CM module topology
7.2Gb/hour (i2)
0.1Gb/hour (r4)
7.2Gb/hour (i2)
?
Source: NI.com

Sample BDE pilot timeseries

Basic analytics
Raw time series
Statistics and
correlations
Dynamic analysis
Fatigue analysis

Pilot concept

Pilot hardware setup

Remarks – BDE opportunities
IT developments offer plethora of tools for data management/analytics
Big Data Europe (www.big-data-europe.eu) offers a easy-to-use generic
platform to encourage community’s entrance to big data technologies
Pilot cases show applicability of tools and deliver open data
You are invited for including your case in BDI pilots; support is provided by
BDE tech team

Thank you for your
attention
BDE consortium

Wind power big_data_and_iot_forum_bigdataeurope

Recommended

Recommended

More Related Content

More from BigData_Europe

More from BigData_Europe (20)

Recently uploaded

Recently uploaded (20)

Wind power big_data_and_iot_forum_bigdataeurope