This document discusses using recurrent neural networks for predictive maintenance. It begins by providing context on industry 4.0 and the growth of industrial automation. It then discusses predictive maintenance and how sensor data from industrial equipment can be used for failure prediction. The document outlines how a recurrent neural network model could be developed using streaming sensor data from manufacturing devices to identify abnormal behavior and predict needed maintenance. It describes the workflow of importing and preparing the data, developing and testing the model, and deploying it to generate alerts from new streaming data.

1. © 2017 MapR TechnologiesMapR Confidential 1
Predictive Maintenance Using
Recurrent Neural Networks
Justin Brandenburg
Data Scientist, MapR Data Technologies

2. © 2017 MapR TechnologiesMapR Confidential 2

3. © 2017 MapR TechnologiesMapR Confidential 3
• 907B$/y investment until 20201
• 1,6M operational industrial robots in the
world in 20152
• 2.6M by 20201
1: What Everyone Must Know About Industry 4.0, Forbes June 2016
2: International Federation of Robotics (IFR) study World Robotics 2016source: PwC 2016 Global Industry 4.0 Survey
Industry 4.0 is Now – Internet of Things (IoT)
Industry 4.0 systems1:
1. Interoperable
2. Information transparency
3. Technical assistance
4. Decentralized decision making

4. © 2017 MapR TechnologiesMapR Confidential 4
Predictive Maintenance
• Thinking about predictive maintenance by tying it into a revenue stream.
When your machines are up and running, you’re making money.
• The idea behind predictive maintenance is that the failure patterns of various
types of equipment are predictable. If we can predict when a piece of
hardware will fail accurately, and replace that component before it fails, we
can achieve much higher levels of operational efficiency.
• With many devices now including sensor data and other components that
send diagnosis reports, predictive maintenance using big data becomes
increasingly more accurate and effective.

5. © 2017 MapR TechnologiesMapR Confidential 5
Data is Generated One Event at a Time
“time” : “6:01.103”,
“event” : “RETWEET”,
“location” :
“lat” : 40.712784,
“lon” : -74.005941
“time: “5:04.120”,
“severity” : “CRITICAL”,
“msg” : “Service down”
“card_num” : 1234,
“merchant” : ”Apple”,
“amount” : 50

6. © 2017 MapR TechnologiesMapR Confidential 6
Stream
Welding Sensors
Stream
Press Sensors
Stream
Prod Line
Stream
Paint
Topic
Topic
Topic
Topic
IoT Real Time Monitoring
Data Platform

7. © 2017 MapR TechnologiesMapR Confidential 7
From Monitoring to Classifying to Predicting
• Monitoring still requires active involvement and quick response
– Oil well that is indicating increased temperature or volume
– Network traffic for bot net activity or insider threat
• What are we monitoring?
– Is this behavior normal?
• Based on what we are monitoring, can we predict what will
happen?

8. © 2017 MapR TechnologiesMapR Confidential 8
Classifying vs Predicting in IoT
• Classification
• Prediction – Can we predict when this non-normal behavior will
occur?
Non-Normal

9. © 2017 MapR TechnologiesMapR Confidential 9
9
Predictive Maintenance for Industrial Devices
Primary goal: Reduce unplanned downtime

10. © 2017 MapR TechnologiesMapR Confidential 10
Need for Scale: Deploy to a Real Factory
Tesla Factory photo by Paul Sakuma/AP

11. © 2017 MapR TechnologiesMapR Confidential 11
Solution Building Blocs

12. © 2017 MapR TechnologiesMapR Confidential 12
MapR Converged Data Platform
Files, Tables, Streams
together on same platform
Shared Services
On-Premise, In the Cloud, Hybrid
High Availability Real Time Security & Governance Multi-tenancy Disaster Recovery Global Namespace
Converge-X™ Data Fabric
Event Data
Streams
Analytics &
Machine Learning
Engines
Operational
Database
Cloud-scale
Data Store

13. © 2017 MapR TechnologiesMapR Confidential 13
Batch and Stream Processing on a Single Platform
Batch Feeds
Logistics Databases
MapR-ES
Batch Processing
Extract Features
Build Models
AI & Machine Learning
Topic
MapR-XD
MapR-DB
Device Health
Stream Processing
Derive Features
Apply Models
Operationalize Models
Topic
Topic
Device
Communication

14. © 2017 MapR TechnologiesMapR Confidential 14
TensorFlow
TensorFlow is an open source software library for numerical
computation using data flow graphs
• Developed by Google, released to open source community in Nov 2015 and
quickly became one of the most popular deep learning frameworks
• Two months after its release it had already become the most popular forked
ML GitHub repository
• Built on C++ with a Python interface

15. © 2017 MapR TechnologiesMapR Confidential 15
What is a Tensor?
A Tensor is a n-dimensional matrix
• 1D is a vector
• 2D (M x M) matrix/tensor is a square array of numbers (m numbers tall and
m numbers wide)
• M x M x M tensor is a cube array (m tall, m wide, m deep)

16. © 2017 MapR TechnologiesMapR Confidential 16
Why TensorFlow for this problem?
• TensorFlow has rich documentation
• Works on CPUs and GPUs (most DL frameworks can)
• Versions 1.x and above have increased utility with function
abstraction
• Once model is trained, tested and optimized, it can be deployed
to edge computing structures or containers

17. © 2017 MapR TechnologiesMapR Confidential 17
Deep Learning Implementations
Convolutional
Neural
Networks
Deep
Neural
Networks
Recurrent
Neural
Networks
providing lift for
classification and
forecasting models
feature extraction
and classification of
images
for sequence of
events (sentences
or time series)

18. © 2017 MapR TechnologiesMapR Confidential 18
Recurrent Neural Network at a Glance
• A neural network that can be used when your data is treated as a
sequence, where the particular order of the data-points matter
• Sometimes, the input is a sequence and the output is a single
vector, or the other way around.

19. © 2017 MapR TechnologiesMapR Confidential 19
Recurrent Neural Network Topology
Unrolling through time

20. © 2017 MapR TechnologiesMapR Confidential 20
What are we working with?
• Challenge: Sensor attached to an
automated manufacturing device
capture position and calibration at each
time stamp. Sensor is capturing real
time data on the device and its current
health. The data is stored for historical
analysis to identify trends and patterns
to determine if any devices need to be
taken out of production for health checks
and maintenance.
• Data: 2,014.dat files that, when
unpackaged, were xml format

21. © 2017 MapR TechnologiesMapR Confidential 21
Workflow
• Import Data into environment
• Perform data transformations
• Exploration of historical data
• Model Construction
• Model Testing
• Deploy Model into Streaming Consumer
• Integrate Visualization

22. © 2017 MapR TechnologiesMapR Confidential 22
Data Import
• MapR-FS allows for a user to ingest any file type
– Filesystem offers schema-on-read versus schema-on-write. Schema-on-
write necessitates knowing your schema before data is written and then
when data is read, it comes back in the schema defined up-front.
Schema-on-read allows data loading as-is, with no preprocessing thereby
removing obstacles for data capture.
• Data was uploaded in compressed file into MapR-FS
– Also can be done via NFS or scp
• Extract files from .zip file
• Extracted files were .dat filetypes, converted file extension to .xml
using standard bash commands

23. © 2017 MapR TechnologiesMapR Confidential 23
Data Transformation
• Using Spark to do bulk file ETL from many xml files to single csv
• XML format
• Dataframe

24. © 2017 MapR TechnologiesMapR Confidential 24
Data Exploration
• Use Jupyter Notebook for interactive data exploration and model
development

25. © 2017 MapR TechnologiesMapR Confidential 25
Data Prep and Model Building
• Use Jupyter Notebook for prepping data and developing your
model and setting hyper parameters

26. © 2017 MapR TechnologiesMapR Confidential 26
Model Training and Testing
• Use Jupyter
Notebook for
training and
testing the model

27. © 2017 MapR TechnologiesMapR Confidential 27
Model Deployment
• Can take tested RNN model and deploy the model on new data as
it streams from sensor attached to device
• Model will generate an alert if predicted metric exceeds historically
normal threshold

28. © 2017 MapR TechnologiesMapR Confidential 28
Visualization
• Dashboard can show in
real time trends and
behaviors of real time
sensor data and the next
period prediction

29. © 2017 MapR TechnologiesMapR Confidential 29
Microservices for Serving Deep Learning Models
Event Streams & DB
Advantages
• Deploy models to production as
microservices
• Use files, real-time streams and
databases in production
• Scales horizontally
• Support both real-time and batch
• May or may not be stateful

30. © 2017 MapR TechnologiesMapR Confidential 30
Points to Remember
• TF is just one ML tool among many (but a great one)
• Choosing the right one depends on your problem
– Ex: Supervised or Unsupervised learning
• How does this model or solution scale?
• Once a model is optimized and insight gained, how can I deploy
my model to help my organization?
• Tools are never used in isolation, the platform matters!
– Support the Workflow, Not Just Modeling

31. © 2017 MapR TechnologiesMapR Confidential 31
90+% of Machine Learning
Success is Data Logistics,
Not Learning

32. © 2017 MapR TechnologiesMapR Confidential 32
Q&A
ENGAGE WITH US
@mapr
Blog: https://mapr.com/blog/
MapR Academy
http://learn.mapr.com/