This document provides an overview of artificial neural networks (ANN) and their applications. It begins with introducing general concepts in artificial intelligence, machine learning, and ANN. It then discusses different ANN architectures like multilayer perceptrons and backpropagation. It also covers ANN learning paradigms such as supervised, unsupervised, and reinforcement learning. Finally, it provides examples of using ANN for applications in fields like solar energy prediction, structural analysis, and control systems.

1. Department of Engineering
American University of Iraq, Sulaimani (AUIS)
1

2. Contents
2
ARTIFICIAL
NEURAL
NETWORKS
REGRESSION
PERFORMANCE
PREDICTION
ANN FOR
RESEARCH
SAMPLE
EXERCISE

3. 3
ARTIFICIAL
NEURAL
NETWORKS

4.  GENERAL FRAMEWORK OF AI
 ARTIFICIAL INTELLIGENCE
 MACHINE LEARNING
 ARTIFICIAL NEURAL NETWORKS
 DEEP LEARNING
4
ARTIFICIAL
NEURAL
NETWORKS

5. 5
Image Source: aiso-lab.com

6. WHAT – WHY - HOW
6

7. Artificial Intelligence
7

8. Artificial Intelligence
 What is AI ?
Branch of computer science that focuses on creating systems that
are intelligent and independent.
 Why use AI ?
To enhances the speed, precision and effectiveness of human
efforts.
 How is AI implemented ?
AI can be developed through learning that is symbolic-based or
data-based.
8

9. Machine learning
 What is Machine Learning ?
Using learning paradigms in machines to achieve pattern
recognition and other human functions.
 Why use Machine Learning ?
Machines can utilize more data, and dimensions of data, to perform
functions such as pattern recognition.
 How is Machine Learning developed ?
Machines learn through either statistical learning or deep
learning.
9

10. The State-of-AI
10
Text AI
Visual AI
Interactive AI
Functional AI
Analytic AI
text recognition, speech-to-text conversion
machine translation, and content generation capabilities.
Chatbots and smart personal assistants.
Computer vision or augmented reality
Sentiment analysis and supplier risk assessment
Maintenance of breakdown in machines after spotting errors, etc.
Examples:
Alex Bekker. 5 Types of AI to Propel Your Business. Science soft.
Link: https://www.scnsoft.com/blog/artificial-intelligence-types

11. 11
Source: Alex Bekker,ScienceSoft. link: https://www.scnsoft.com/blog/artificial-intelligence-types
The State-of-AI

12. The State-of-AI
12
Symbolic learning
Machine learning Pattern recognition
Computer vision
Neural Networks Deep Learning
learning capabilities (or) reasoning capabilities
What about combining the two ?
Able to give explanations and can manipulate
complex data structures
Richa Bhatia. Understanding the difference between symbolic AI & non symbolic AI.
https://analyticsindiamag.com/understanding-difference-symbolic-ai-non-symbolic-ai/

13. 13
Symbolic learning
Symbolic learning
See
Process
Computer vision
Robotics
Interact
Symbolic
learning

14. 14
Human ability to
interact with the world
System that
can learn
Symbolic learning
Machine learning Pattern recognition
Computer vision
Neural Networks Deep Learning
Machine learning

15. 15
Robotic system
Sensor
Actuators
Non-AI
programming
AI
algorithm
AI and Robotics

16. ARTIFICIAL NEURAL NETWORKS
 What are Artificial Neural Networks ?
 Why use Artificial Neural Networks ?
 How are Artificial Neural Networks developed ?
Answers in the presentation > slides 15-41
16

17. Deep Learning
 What is Deep Learning ?
 Why use Deep Learning ?
 How is Deep Learning achieved ?
Answers in the presentation > slide 42
17

18. Artificial Neural Networks (ANN)
C
18

19. Artificial Neural Networks (ANN)
Motivation
19

20. Artificial Neural Networks (ANN)
Advantages
20

21. Artificial Neural Networks (ANN)
Components
21

22. Artificial Neural Networks (ANN)
C
22

23. Artificial Neural Networks (ANN)
C
23

24. Artificial Neural Networks (ANN)
C
24

25. Artificial Neural Networks (ANN)
Numerical
Data
Labeling
Images
Video
Text
Time-series
Clustering
Machine
perception
Classification Clustering
C
25

26. Artificial Neural Networks (ANN)
Classification
SUPERVISED LEARNING
Face detection
Object
Identification
Text
classification
Voice detection
Gesture
Identification
Transcript
speech-text
Create
Numerical
Data
Assign
to
data
TRAIN
THE
ANN
Create
correlation
26

27. Artificial Neural Networks (ANN)
Ni, D. X. (2007). Application of neural networks to character recognition. Proceedings
of students/faculty research day, CSIS, Pace University, May 4th.
Classification
C
27

28. Artificial Neural Networks (ANN)
Classification
https://www.analyticsindiamag.com/how-to-create-your-first-artificial-neural-network-in-python/ C
28

29. Artificial Neural Networks (ANN)
Regression
SUPERVISED LEARNING
Hardware
breakdown
Employee
turnover
Customer churn
Health
breakdown
Machine output
C
29

30. Artificial Neural Networks (ANN)
Regression
https://dataaspirant.com/2017/03/02/how-logistic-regression-model-works/ C
30

31. Artificial Neural Networks (ANN)
Regression
https://dataaspirant.com/2017/03/02/how-logistic-regression-model-works/ C
31

32. Artificial Neural Networks (ANN)
Regression
https://dataaspirant.com/2017/03/02/how-logistic-regression-model-works/ C
32

33. Artificial Neural Networks (ANN)
Clustering
UNSUPERVISED
LEARNING
Comparing documents, images, etc.
to surface similar items
Detecting anomalies, or unusual
behavior
C
33

34. Artificial Neural Networks (ANN)
Clustering
C
34

35. Artificial Neural Networks (ANN)
35
https://quantumcomputingtech.blogspot.com/2019/02/machine-learning-animated-gif.html
Clustering

36. Artificial Neural Networks (ANN)
REINFORCEMENT
Self-driving car
Customer management
Adaptive traffic signal
control
Games
Reinforcement
LEARNING
C
36

37. 37
Artificial Neural Networks (ANN)
REINFORCEMENT
This paradigm does not require a labelled input/output pairs to be presented and
does not need sub-optimal actions to be explicitly corrected.

38. Artificial Neural Networks (ANN)
Learning paradigms
SUPERVISED LEARNING
UNSUPERVISED LEARNING
REINFORCEMENT
LEARNING
38

39. Artificial Neural Networks (ANN)
Learning paradigms
Kaplan, S. (2017). Deep generative models for synthetic retinal image generation. C
39

40. Artificial Neural Networks (ANN)
Learning paradigms
Unsupervised learning Supervised learning Clustering: An Introduction to Supervised Machine Learning and Pattern
Classification: The Big Picture by Sebastian Raschka C
40

41. Artificial Neural Networks (ANN)
SINGLE LAYER
C
41

42. Artificial Neural Networks (ANN)
MULTIPLE LAYER PERCEPTRON (MLP)
C
42

43. Artificial Neural Networks (ANN)
MULTIPLE LAYER PERCEPTRON (MLP)
C
43

44. Artificial Neural Networks (ANN)
MULTIPLE LAYER PERCEPTRON (MLP)
C
44

45. Artificial Neural Networks (ANN)
MULTIPLE LAYER PERCEPTRON (MLP)
Jain & Mao, 1996 C
Deep
45

46. Artificial Neural Networks (ANN)
MULTIPLE LAYER PERCEPTRON (MLP)
A MLP consists of at least three layers of nodes: an input layer, a hidden layer and an
output layer. Except for the input nodes, each node is a neuron that uses a
nonlinear activation function.
MLP utilizes a supervised learning technique called backpropagation for training. Its
multiple layers and non-linear activation distinguish MLP from a linear perceptron. It can
distinguish data that is not linearly separable.
46

47. Artificial Neural Networks (ANN)
MULTIPLE LAYER PERCEPTRON (MLP)
Changing connection weights after each piece of data is processed, based on the
amount of error in the output compared to the expected result.
47

48. Artificial Neural Networks (ANN)
MULTIPLE LAYER PERCEPTRON (MLP)
MLPs are useful in research for their ability to solve problems stochastically, which
often allows approximate solutions for extremely complex problems like fitness
approximation.
MLPs are universal function approximators as shown by Cybenko's theorem, so they can
be used to create mathematical models by regression analysis. As classification is a
particular case of regression when the response variable is categorical, MLPs make good
classifier algorithms.
MLPs were a popular machine learning solution in the 1980s, finding applications in
diverse fields such as speech recognition, image recognition, and machine
translation software, but thereafter faced strong competition from much simpler (and
related) support vector machines. Interest in backpropagation networks returned due to
the successes of deep learning.
48

49. Artificial Neural Networks (ANN)
MULTIPLE LAYER PERCEPTRON (MLP)
49

50. Artificial Neural Networks (ANN)
FEED-FORWARD
Connections between the nodes do not form a cycle.
The first and simplest type of artificial neural network
devised.
50

51. Artificial Neural Networks (ANN)
BACKPROPAGATION
51

52. Artificial Neural Networks (ANN)
BACKPROPAGATION
52

53. Artificial Neural Networks (ANN)
BACKPROPAGATION
 Supervised learning
 Used at each layer to minimize the
error between the layer’s response
and the actual data
 The error at each hidden layer is an
average of the evaluated error
53

54. Artificial Neural Networks (ANN)
BACKPROPAGATION
 N is a neuron.
 Nw is one of N’s inputs weights
 Nout is N’s output.
 Nw = Nw +Δ Nw
 Δ Nw = Nout * (1‐ Nout)* NErrorFactor
 NErrorFactor = NExpectedOutput – NActualOutput
 This works only for the last layer, as we can know the actual output, and
the expected output.
54

55. Artificial Neural Networks (ANN)
Training stage
Validating stage
Testing stage
55

56. Contents
56
ARTIFICIAL
NEURAL
NETWORKS
REGRESSION
PERFORMANCE
PREDICTION
ANN FOR
RESEARCH
SAMPLE
EXERCISE

57. 57
REGRESSION
A measure of the relation between the mean value of one variable (e.g. output)
and corresponding values of other variables (e.g. time and cost).
Dependent
variable
Independent
variables

58. 58
REGRESSION
Dependent variable
Independent variables
Control variables

59. Regression and Correlation
59

60. Contents
60
ARTIFICIAL
NEURAL
NETWORKS
REGRESSION
PERFORMANCE
PREDICTION
ANN FOR
RESEARCH
SAMPLE
EXERCISE

61. PERFORMANCE
PREDICTION
Methodology
Performance
Prediction
Statistical
methods
Analysis
61

62. PV/T modeling and performance
prediction
• Why use artificial neural networks (ANN) in solar energy
technologies?
• How to use artificial neural networks (ANN) in solar energy
technologies?
• How to utilize regression and classification to invest in solar energy?
62

63. PV performance time-series prediction
Short-term power
forecasting
Estimating power loss
due to environment
Estimating the energetic
performance of PV/T
PV fault detection
Optimize PV array
inclination
Solar Energy Forecasting
Optimize maximum
power point tracking
63

64. PV performance time-series prediction
C
Senkal, O. & Kuleli, T. (2009). Estimation of solar radiation over Turkey using
artificial neural network and satellite data. Applied Energy, Vol. 86, pp. 1222–
64

65. PV performance time-series prediction
Almonacid, F., Rus, C., Hontoria, L., Fuentes, M. &
Nofuentes G. (2009). Characterisation of Si-crystalline PV
modules by artificial neural Networks. Renewable Energy, C
65

66. PV performance time-series prediction
Firefly algorithm (FA) and Particle Swarm Optimization (PSO) applied
to train NN
EXAMPLE 1: SHORT TERM POWER
FORECASTING
Demirdelen, T., Aksu, I. O., Esenboga, B., Aygul, K.,
Ekinci, F., & Bilgili, M. (2019). A new method for
generating short-term power forecasting based on artificial
neural networks and optimization methods for solar
photovoltaic power plants. In Solar Photovoltaic Power
Plants (pp. 165-189). Springer, Singapore.
Multi-Layer Feed Forward (MLFF) neural network structure
The data of 1MWPV power plant in Turkey is used to estimate output power
by
real-time data mining for short time prediction. 66

67. PV performance time-series prediction
Example 1
C
67

68. PV performance time-series prediction
Example 1
Ambient temperature [◦C]
Solar radiation [W/m2]
PV Panel temperature [◦C]
Input
Layer
Hidden
Layer
Output
Layer
24 8
Interconnections Interconnections
PV power (W)
68

69. PV/T performance time-series prediction
Particle Swarm Optimization (PSO) applied to train NN
EXAMPLE 2: ESTIMATING ENERGETIC
PERFORMANCE
Alnaqi, A. A., Moayedi, H., Shahsavar, A., & Nguyen, T.
K. (2019). Prediction of energetic performance of a
building integrated photovoltaic/thermal system thorough
artificial neural network and hybrid particle swarm
optimization models. Energy conversion and
management, 183, 137-148.
Multi-Layer Feed-Forward Back-Propagation (FFBP) neural network
structure
building integrated photovoltaic/thermal system
69

70. PV/T performance time-series prediction
Example 2
C
70

71. PV/T performance time-series prediction
Example 2
C
71

72. Statistical methods
• Coefficient of determination
• Mean Absolute Percentage Error
• Root Mean Square Error
• Mean Absolute Error
• Mean Square Error
difference between two continuous variables
how well a regression model is capable of describing a
data set
a measure of prediction accuracy of a forecasting
method in statistics,
a measure of the quality of an estimator
measure of the differences between
an estimator and the values observed
72

73. Statistical methods
COEFFICIENT OF DETERMINATION
a measure for how well a regression model is capable of
describing a data set.
used to evaluate the validity of the predictive
results compared to the actual (experimental)
real model results
𝒚𝒊= experimental value of (y)
𝒇𝒊= predicted value of (y)
𝒚𝒊= the mean of the experimental values
N = the number of values
73

74. Statistical methods
MEAN ABSOLUTE ERROR
a measure of difference between two continuous variables.
the average vertical distance between each point and the
identity line. MAE is also the average horizontal distance
between each point and the identity line.
74

75. Statistical methods
MEAN ABSOLUTE PERCENTAGE ERROR
a measure of prediction accuracy of a forecasting
method in statistics,
75

76. Statistical methods
MEAN SQUARE ERROR
measures the average of the squares of the errors—that is, the average
squared difference between the estimated values and what is estimated
The MSE is a measure of the quality of an estimator—it is always non-
negative, and values closer to zero are better.
76

77. Statistical methods
ROOT MEAN SQUARE ERROR
a measure for the difference between a data set and a corresponding fit. The RMS
asymptotically converges to the standard deviation from the model's predicted value
for sufficiently large sizes of data sets.
measure of the differences between values (sample or population values)
predicted by a model or an estimator and the values observed.
77

78. Applications in engineering disciplines
Civil engineering
78
Suryanita, R. (2016). The application of artificial neural
networks in predicting structural response of multistory
building in the region of Sumatra island. KnE Engineering.
Reinforced Concrete Building Models

79. Applications in engineering disciplines
Electrical engineering
79
Hiyama, T., Kouzuma, S., Imakubo,
T.:‘Identification of optimal operating pointof PV
modules using neural network for real time
maximum power trackingcontrol’,IEEE Trans.
Energy Convers., 1995,10, (2), pp. 360–367

80. Applications in engineering disciplines
Mechanical engineering
80
Ranasinghe, R. A. T. M., Jaksa, M. B., Kuo, Y. L., & Nejad, F. P. (2017). Application of artificial neural networks for
predicting the impact of rolling dynamic compaction using dynamic cone penetrometer test results. Journal of Rock
Mechanics and Geotechnical Engineering, 9(2), 340-349.
Predicting the impact
of rolling dynamic
compaction using
dynamic cone
penetrometer test
results

81. Applications in engineering disciplines
Control engineering
81
KACHULKOVA, S. Applications of Artificial Neural Networks in Control Systems. transfer, 1(1), 5.

82. 82
ARTIFICIAL
NEURAL
NETWORKS
REGRESSION
PERFORMANCE
PREDICTION
ANN FOR
RESEARCH
SAMPLE
EXERCISE

83. Tips to start utilizing machine learning and
ANN techniques for your previous,
current and future studies in your field…
PAST – PRESENT - FUTURE
83
ANN FOR
RESEARCH

84. 84
TIP #1 CLASSIFY THE VARIABLES
DEPENDENT
VARIABLES (DV)
INDEPENDENT
VARIABLES (IV)
CONTROL
VARIABLES (CV)

85. 85
TIP #2 ENSURE HIGH-QUALITY OF DATA
GIGO
GARBAGE IN =
GARBAGE OUT

86. GIGO, VIGO, VIVO
86

87. 87
TIP #2 ENSURE HIGH-QUALITY OF DATA
WHAT IS YOUR UNCERTAINTY
THRESHOLD?
WHAT MODEL ARE YOU USING FOR
UNCERTAINTY ANALYSIS?
WHAT IS YOUR UNCERTAINTY
THRESHOLD?

88. 88
Example of uncertainty model and
analysis
The uncertainty associated with the result is WR and those associated
with the independent variables are W1, W2, …, Wn.
Source: Kline, S. J. (1953). Describing uncertainty in single sample experiments. Mech.
Engineering, 75, 3-8.

89. 89
Example of uncertainty model and
analysis
Source: Sarafraz, M. M., Safaei, M. R., Leon, A. S., Tlili, I., Alkanhal, T. A., Tian, Z., ... & Arjomandi, M. (2019).
Experimental investigation on thermal performance of a PV/T-PCM (photovoltaic/thermal) system cooling with a PCM
and nanofluid. Energies, 12(13), 2572.

90. 90
Data filtration
Height of an adult male
by fixing the typo or possibly remeasuring the item or
person. If that’s not possible, you must delete the data
point because you know it’s an incorrect value.
Source: Statistics by Jim. Link: https://statisticsbyjim.com/basics/remove-outliers/ (access
date: 25th of July 2022)

91. 91
COLLECT AS MUCH DATA AS
POSSIBLE.
TIP #3 COLLECT BULK DATA
Once testing station, or setup, is established.
Record as much data as possible. With a
variety of design configurations.

92. 92
Number of input, outputs and hidden
nodes.
TIP #4 Select the components
Utilize the identified variables (IV, DV, CV)

93. 93
How much (%) of the data are you allocating for
training? Validating? Testing?
TIP #5 Choose the data splitting
percentage

94. 94
Use MATLAB/Python to create a simple
MLP Neural Network to process the data
TIP #6 Start with a simple MLP ANN

95. 95
ARTIFICIAL
NEURAL
NETWORKS
REGRESSION
PERFORMANCE
PREDICTION
ANN FOR
RESEARCH
SAMPLE
EXERCISE

96. 96
Simple exercise
Perform tasks 1-5

97. Food for Thought
• What is the difference between Artificial Intelligence and
computational intelligence?
• How to interpret skewness and kurtosis?
• What is the significance of the activation function?
• What is the significance of the bias?
97

98.  Identify the dependent and independent variables in your work.
 Assign the input and output nodes to your network.
 Conduct data pre-processing.
 Decide the type of problem you have.
 Choose an activation function.
 Choose the number of hidden nodes.
 Assign weights and biases.
 Download MATLAB and implement a simple MLP code.
 Carryout the training, validation and testing.
Homework
98

99. Textbook in PV/T
https://www.amazon.com/Photovoltaic-Thermal-Systems-Principles-
Applications/dp/3030278239/ref=sr_1_1?keywords=PV%2FT+al-waeli&qid=1563014343&s=gateway&sr=8-1
99

100. Review article in PV/T
100

101. Original article on ANN use for PV/T
Performance Prediction
101

102. It’s play time…
• https://playground.tensorflow.org/
102

103. Thank you
103