This document provides an overview of deep learning techniques including neural networks, convolutional neural networks (CNNs), and long short-term memory (LSTM) algorithms. It defines key concepts like Bayesian inference, heuristics, perceptrons, and backpropagation. It also describes how to configure neural networks by specifying hyperparameters, hidden layers, normalization methods, and training parameters. CNN architectures are explained including convolution, pooling, and applications in computer vision tasks. Finally, predictive maintenance using deep learning to predict equipment failures from sensor data is briefly discussed.

1. WELCOME TO:
Deep Learning AI Technologies
1. Neural Networks
2. CNN
By Adj Prof. Giuseppe Mascarella
giuseppe@valueamplify.com

2. https://docs.microsoft.com/en
-us/azure/machine-
learning/concept-deep-
learning-vs-machine-
learning?WT.mc_id=wiley-
book-lazzeri
(Incl Heuristic)

3. Stochastic Gradient Descent for Bayesian
Neural Networks
DEFINITIONS
A) BAYESIAN
Bayesian inference is a method of statistical inference in which Bayes'
theorem is used to update the probability for a hypothesis as more evidence
or information becomes available.
B) Heuristic
Experimenting, evaluating possible answers or solutions, or by trial and
error
Shortcuts to make decisions based on past events or traits that are
representative of or similar to the current situation.

4. Heuristics in
Neural Networks

5. What Is a Neural Network”
A is a set of interconnected layers.
The inputs are the first layer, and are connected to an output layer by
an acyclic graph comprised of weighted edges and nodes.
A NN is a DNN (Deep Neural Networks) when you have many hidden
layers has more accuracy.
All nodes in a layer are connected by the weighted edges to nodes in
the next layer.
To compute the output of the network for a particular input, the value
is set by calculating the weighted sum of the values of the nodes from
the previous layer.
An activation function is then applied to that weighted sum.

6. Neural Networks Basics
X1 - Is the weather good?
X2 - Does your boyfriend or girlfriend want
to accompany you?
X3 - Is the festival near public transit?
(You don't own a car).
The artificial
neuron are
called:
perceptron
• w1=6 for the weather
• w2=2
• w3=2 for the other
conditions.
• Threshold
(Bias Toward
Cheese)= 3
Biz Problem To Solve: Predict, Will John Go To The Cheese Festival This Weekend?
W=> Tool for a device that
makes decisions by weighing
up evidence and previous
results.
NAND Electrical Circuit Operator
0,1
Bias: how easy it is to get
the perceptron to fire
W
3

7. Sigmoid Neuron
• Learning algorithms which can
automatically tune neurons:
• Weights
• biases
8 vs 9
=
=

17. Backpropagation! The algo that computes the best Gradient

21. Cost Function: MSE (Mean Squared Error)
n is the total number of training inputs,
a is the vector of outputs from the network w/x as input
Sum is over all training inputs, x
Y=(m n +x)
C = w2-w1/(y2-y1)

23. Gradient Descent Function

27. How To Configure a NN (Part 1)
https://docs.microsoft.com/en-us/azure/machine-learning/studio-module-reference/two-class-neural-network
1. Category
2. Parameters (Hyperparameters)
3. Hidden Layers
4. Hidden Nodes, default in a hidden layer is 100 nodes.
5. Learning rate, define the size of the step taken at each iteration,
before correction. (i.e. a larger value for learning rate can cause the model to
converge faster, but it can overshoot local minima.)
6. Number of learning iterations, specify the maximum number of
times the algorithm should process the training cases.
7. Initial Learning Weights Diameter, specify the node weights at
the start of the learning process.
8. Momentum, specify a weight to apply during learning to nodes
from previous iterations.
https://distill.pub/2017/momentum/
Scripted, custom CNN
input Picture [28, 28];
hidden C1 [5, 12, 12]
from Picture convolve {
InputShape = [28, 28];
KernelShape = [ 5, 5];
Stride = [ 2, 2];
MapCount = 5;
}

29. How To Configure a NN (Part 2)
• Type of normalizer, select a method to use for feature normalization. The following normalization methods are
supported:
• Binning normalizer: The binning normalizer creates bins of equal size, and then normalizes every value in each bin,
dividing by the total number of bins.
• Gaussian normalizer: The Gaussian normalizer rescales the values of each feature to have mean 0 and variance 1. This is
done by computing the mean and the variance of each feature. For each instance, the mean value is subtracted, and the
result divided by the square root of the variance (the standard deviation).
• Min-max normalizer: The min-max normalizer linearly rescales every feature to the [0,1] interval.Rescaling to the [0,1]
interval is done by shifting the values of each feature so that the minimal value is 0, and then dividing by the new maximal
value (which is the difference between the original maximal and minimal values).
• Do not normalize: No normalization is performed.
• Shuffle examples option to shuffle cases between iterations. If you deselect this option, cases are processed in
exactly the same order each time you run the experiment.
• Random number seed, type a value to use as the seed. Specifying a seed value is useful when you want to
ensure repeatability across runs of the same experiment. Otherwise, a system clock value is used as the seed,
which can cause slightly different results each time you run the experiment.
• Allow unknown categorical levels option to create a grouping for unknown values in the training and
validation sets. The model might be less precise on known values but provide better predictions for new
(unknown) values. If you deselect this option, the model can accept only the values contained in the training
data.

30. NN
Configuration
Summary

31. Tune Model Hyperparameters
• Integrated train and tune: You configure a set of
parameters to use, and then let the module iterate over
multiple combinations, measuring accuracy until it finds a
"best" model. With most learner modules, you can choose
which parameters should be changed during the training
process, and which should remain fixed.
•Depending on how long you want the tuning process to
run, you might decide to exhaustively test all
combinations, or you could shorten the process by
establishing a grid of parameter combinations and testing
a randomized subset of the parameter grid.
• Cross validation with tuning: With this option, you
divide your data into some number of folds and then build
and test models on each fold. This method provides the
best accuracy and can help find problems with the dataset;
however, it takes longer to train.

32. Assignment: NN ML Experiment Reverse
Engineering
• https://gallery.azure.ai/Experiment/pranab-p181a35-nlp-final-exam

33. WELCOME TO:
Deep Learning AI Technologies
1. Neural Networks
2. CNN(Convoluted Networks)
By Adj Prof. Giuseppe Mascarella
giuseppe@valueamplify.com

34. CNN Definitions
A. Inspired by biological processes[7][8][9][10] in that
the connectivity pattern between neurons resembles the organization of the animal visual
cortex.
Individual cortical neurons respond to stimuli only in a restricted region of the visual field known
as the receptive field. The receptive fields of different neurons partially overlap such that they
cover the entire visual field.
B. The name “convolutional neural network” indicates that the network employs a mathematical
operation called convolution that is a mathematical operation on two functions (f and g) that
produces a third function (f*g) expressing how the shape of one is modified by the other.
C. The convolutional layer is the core building block of a CNN. The layer's parameters
consist of a set of learnable filters (or kernels), which have a small receptive field, but
extend through the full depth of the input volume.
D. They are also known as shift invariant or space invariant artificial neural networks
(SIANN), based on their shared-weights architecture and translation invariance
characteristics.

35. CNN
Applications:
image and video recognition, recommender systems, image
classification, medical image analysis, natural language processing, and
financial time series.
WHY?
CNNs use relatively little pre-processing compared to other image
classification algorithms.
This means that the network learns the filters that in traditional algorithms
were hand-engineered. This independence from prior knowledge and human effort
in feature design is a major advantage.

36. What Problem Is CNN Trying To Solve?
Adaptation for education purpose from: https://www.youtube.com/watch?v=2-Ol7ZB0MmU

37. CNN Deep Learning Is a Brilliant A Series Of
Classifiers Orchestrated in Layers

38. CNN Architecture

39. Derivative work from: A friendly introduction to Convolutional Neural Networks and Image Recognition by
Serrano

48. Apply Filter

49. Pooling Layer

50. Compare

53. Convolution Network

58. Create Random Numbers And Gradient Descend

59. CNN

60. LSTM (Long short-term memory) Algo
Long short-term memory (LSTM) is an artificial recurrent neural
network (RNN) architecture used in the field of deep learning. Unlike
standard feedforward neural networks, LSTM has feedback
connections.

61. • The aim of predictive maintenance (PdM) is first to predict when
equipment failure might occur, and secondly, to prevent the occurrence of
the failure by performing maintenance. ... When predictive maintenance is
working effectively as a maintenance strategy, maintenance is only
performed on machines when it is required.
• The asset of interest has a progressing degradation pattern, which is
reflected in the asset's sensor measurements." Some FMEA shows that
assets fail without a detectable degradation pattern. Also, sensor
(performance) data is only one piece of the picture. Asset configuration will
impact RUL also. If I swap an old fan blade (or a non-OEM fan blade, etc.)
into my engine, it will alter RUL. If the model doesn't know how many
cycles the fan blade has on it, it will break.