Complex machines, e.g. trains or wind turbines, require very solid maintenance procedures. Anticipating the wear of a piece or the failure of a system allows a sensitive maintenance scheduling and prevention of catastrophic failures. The race towards efficiency has enabled the spreading of sensors that collect huge data about the current state of the different components of said machines. Collecting and storing this data can be considered a solvable problem. However, all that data is of no use by itself. An optimal maintenance can derive from decisions that can derive from information that can derive from that big lake of data. Session presented at Big Data Spain 2015 Conference 16th Oct 2015 Kinépolis Madrid http://www.bigdataspain.org Event promoted by: http://www.paradigmatecnologico.com Abstract:http://www.bigdataspain.org/program/fri/slot-36.html#spch36.2

2. Predicting failures on complex
machines
Ion Marqués

3. OUTLINE
NEM Solutions provides complete management solutions to businesses
responsible for the operation and maintenance of multi-system assets.

4. OUTLINE
NEM Solutions provides complete management solutions to businesses
responsible for the operation and maintenance of multi-system assets.
Nowadays, we have clients with thousands of assets, generating massive
volume of data.

5. OUTLINE
NEM Solutions provides complete management solutions to businesses
responsible for the operation and maintenance of multi-system assets.
Nowadays, we have clients with thousands of assets, generating massive
volume of data.
What we’ll see in the following 15 minutes:
1. The client’s needs
2. Our approach
3. The solution’s overview
4. The engine - the core of the solution.
5. How we did it, what did we learn.

6. DEMAND FOR EFFICIENT AND SUSTAINABLE TRANSPORTATION SYSTEMS.
HIGH SPEED & URBAN TRANSPORTATION NEEDS ON THE RISE.
INCREASING ENERGY NEEDS. ON & OFF SHORE RENEWABLES GROWING.
NEED FOR PRODUCTIVITY, RELIABILITY AND CONTINUOUS IMPROVEMENT.
THE CLIENTS’ NEEDS
REACTIVE
APPROACH
The business
under control
Avoid
surprises
The
unexpected
happens
Business
plan fails
BUSINESS &
KNOWLEDGE

7. Normality
model
definition
Normality
model
Vs = Failure
Symptoms
Real time
data
FUTURE
PROJECTION
FROM DATA
KNOWLEDGE
GENERATION
A.U.R.A: ARTIFICIAL INMUNE SYSTEM

8. OUR BIG DATA SOLUTION

9. THE WORKFLOW: 1st APPROACH
• We translate the calculations to a topology.
• Each topology node is a computational unit, i.e arithmetical operations,
symptom calculations, machine learning algorithm testings, …
• Each node is a Storm bolt. We had around 160 bolts each doing one task.

10. THE WORKFLOW: 1st APPROACH
• We translate the calculations to a topology.
• Each topology node is a computational unit, i.e arithmetical operations,
symptom calculations, machine learning algorithm testings, …
• Each node is a Storm bolt. We had around 160 bolts each doing one task.
• One “master” spout.
• If a bolt fails, all the
data must be re-
emmited!

11. THE WORKFLOW: 2nd APPROACH
• We translate the calculations to a topology.
• Each topology node is a computational unit, i.e arithmetical operations,
symptom calculations, machine learning algorithm testings, …
• Each node is a Storm bolt. We had around 160 bolts each doing one task.

12. THE WORKFLOW: 2nd APPROACH
• We translate the calculations to a topology.
• Each topology node is a computational unit, i.e arithmetical operations,
symptom calculations, machine learning algorithm testings, …
• Each node is a Storm bolt. We had around 160 bolts each doing one task.
• One spout per variable
• Too much
communication for our
case.
• Not efficient enough.

13. THE WORKFLOW: CURRENT APPROACH
• We translate the calculations to a simple topology.
• Non-codependant tasks are grouped into computational units.
• We have a few nodes, assigning one executor per task.

14. THE WORKFLOW: CURRENT APPROACH
• We translate the calculations to a simple topology.
• Non-codependant tasks are grouped into computational units.
• We have a few nodes, assigning one executor per task.
• Same parallelization.
• Less communication.
• Adapted to small
clusters.
• Better performance.

15. WE HAD:
 The knowledge about the industries’ needs.
 The machine learning methodologies to extract useful information.
 A successful non-scalable product.
CONCLUSION

16. WE HAD:
 The knowledge about the industries’ needs.
 The machine learning methodologies to extract useful information.
 A successful non-scalable product.
CONCLUSION
WE NEEDED:
o The means to make that product capable of processing massive amount
of data.
o To solve a key point: Embedding algorithms into a scalable streaming
framework.

17. • ROI: Industry demands tools that assist in making decisions affecting lots
of complex machines.
• In order to meet that particular demand, we need more than amazing
visualizations and simple data mining methods.
LEASONS LEARNED

18. • ROI: Industry demands tools that assist in making decisions affecting lots
of complex machines.
• In order to meet that particular demand, we need more than amazing
visualizations and simple data mining methods.
LEASONS LEARNED
Technically, it is a challenge:
• Kafka+Storm+Redis+Hbase can be a winning choice.
• There’s no free lunch, and every case is different.
• Translate your algorithms into a path the data will cross: A directed
graph, a topology. Then simplify. Fail. Try again.
• Your team must know your problem: From how heat in a wind rotor
behaves to how failures in Storm propagate.

19. LISTENING TO YOUR ASSETS
NEM Solutions
+34 943 30 93 28
info@nemsolutions.com
@NEMSolutions
Thank you!