Convolutional neural networks (CNNs) are widely used for tasks like image classification, object detection, and face recognition. CNNs extract features from data using convolutional structures and are inspired by biological visual perception. Early CNNs include LeNet for handwritten text recognition and AlexNet which introduced ReLU and dropout to improve performance. Newer CNNs like VGGNet, GoogLeNet, ResNet and MobileNets aim to improve accuracy while reducing parameters. CNNs require activation functions, loss functions, and optimizers to learn from data during training. They have various applications in domains like computer vision, natural language processing and time series forecasting.
This is a presentation I gave as a short overview of LSTMs. The slides are accompanied by two examples which apply LSTMs to Time Series data. Examples were implemented using Keras. See links in slide pack.
Introduction For seq2seq(sequence to sequence) and RNNHye-min Ahn
This is my slides for introducing sequence to sequence model and Recurrent Neural Network(RNN) to my laboratory colleagues.
Hyemin Ahn, @CPSLAB, Seoul National University (SNU)
Deep learning (also known as deep structured learning or hierarchical learning) is the application of artificial neural networks (ANNs) to learning tasks that contain more than one hidden layer. Deep learning is part of a broader family of machine learning methods based on learning data representations, as opposed to task-specific algorithms. Learning can be supervised, partially supervised or unsupervised.
Convolutional Neural Networks : Popular Architecturesananth
In this presentation we look at some of the popular architectures, such as ResNet, that have been successfully used for a variety of applications. Starting from the AlexNet and VGG that showed that the deep learning architectures can deliver unprecedented accuracies for Image classification and localization tasks, we review other recent architectures such as ResNet, GoogleNet (Inception) and the more recent SENet that have won ImageNet competitions.
This is a presentation I gave as a short overview of LSTMs. The slides are accompanied by two examples which apply LSTMs to Time Series data. Examples were implemented using Keras. See links in slide pack.
Introduction For seq2seq(sequence to sequence) and RNNHye-min Ahn
This is my slides for introducing sequence to sequence model and Recurrent Neural Network(RNN) to my laboratory colleagues.
Hyemin Ahn, @CPSLAB, Seoul National University (SNU)
Deep learning (also known as deep structured learning or hierarchical learning) is the application of artificial neural networks (ANNs) to learning tasks that contain more than one hidden layer. Deep learning is part of a broader family of machine learning methods based on learning data representations, as opposed to task-specific algorithms. Learning can be supervised, partially supervised or unsupervised.
Convolutional Neural Networks : Popular Architecturesananth
In this presentation we look at some of the popular architectures, such as ResNet, that have been successfully used for a variety of applications. Starting from the AlexNet and VGG that showed that the deep learning architectures can deliver unprecedented accuracies for Image classification and localization tasks, we review other recent architectures such as ResNet, GoogleNet (Inception) and the more recent SENet that have won ImageNet competitions.
The slides includes an introduction to Long Short-term Memory (LSTM ) >> A novel approach in dealing with vanishing gradients in deep neural networks. Made for students, and anyone out there who'd love to learn about recurrent artificial neural networks, specifically of the LSTMs architecture.
Reference material has been attached to further your reading.
Deep Learning: Recurrent Neural Network (Chapter 10) Larry Guo
This Material is an in_depth study report of Recurrent Neural Network (RNN)
Material mainly from Deep Learning Book Bible, http://www.deeplearningbook.org/
Topics: Briefing, Theory Proof, Variation, Gated RNNN Intuition. Real World Application
Application (CNN+RNN on SVHN)
Also a video (In Chinese)
https://www.youtube.com/watch?v=p6xzPqRd46w
Explores the type of structure learned by Convolutional Neural Networks, the applications where they're most valuable and a number of appropriate mental models for understanding deep learning.
One-shot learning is an object categorization problem in computer vision. Whereas most machine learning based object categorization algorithms require training on hundreds or thousands of images and very large datasets, one-shot learning aims to learn information about object categories from one, or only a few, training images
This presentation on Recurrent Neural Network will help you understand what is a neural network, what are the popular neural networks, why we need recurrent neural network, what is a recurrent neural network, how does a RNN work, what is vanishing and exploding gradient problem, what is LSTM and you will also see a use case implementation of LSTM (Long short term memory). Neural networks used in Deep Learning consists of different layers connected to each other and work on the structure and functions of the human brain. It learns from huge volumes of data and used complex algorithms to train a neural net. The recurrent neural network works on the principle of saving the output of a layer and feeding this back to the input in order to predict the output of the layer. Now lets deep dive into this presentation and understand what is RNN and how does it actually work.
Below topics are explained in this recurrent neural networks tutorial:
1. What is a neural network?
2. Popular neural networks?
3. Why recurrent neural network?
4. What is a recurrent neural network?
5. How does an RNN work?
6. Vanishing and exploding gradient problem
7. Long short term memory (LSTM)
8. Use case implementation of LSTM
Simplilearn’s Deep Learning course will transform you into an expert in deep learning techniques using TensorFlow, the open-source software library designed to conduct machine learning & deep neural network research. With our deep learning course, you'll master deep learning and TensorFlow concepts, learn to implement algorithms, build artificial neural networks and traverse layers of data abstraction to understand the power of data and prepare you for your new role as deep learning scientist.
Why Deep Learning?
It is one of the most popular software platforms used for deep learning and contains powerful tools to help you build and implement artificial neural networks.
Advancements in deep learning are being seen in smartphone applications, creating efficiencies in the power grid, driving advancements in healthcare, improving agricultural yields, and helping us find solutions to climate change. With this Tensorflow course, you’ll build expertise in deep learning models, learn to operate TensorFlow to manage neural networks and interpret the results.
And according to payscale.com, the median salary for engineers with deep learning skills tops $120,000 per year.
You can gain in-depth knowledge of Deep Learning by taking our Deep Learning certification training course. With Simplilearn’s Deep Learning course, you will prepare for a career as a Deep Learning engineer as you master concepts and techniques including supervised and unsupervised learning, mathematical and heuristic aspects, and hands-on modeling to develop algorithms. Those who complete the course will be able to:
Learn more at: https://www.simplilearn.com/
From Conventional Machine Learning to Deep Learning and Beyond.pptxChun-Hao Chang
In this slide, Deep Learning are compared with Conventional Learning and the strength of DNN models will be explained.
The target audience are people who have the knowledge of Machine Learning or Data Mining but not familiar with Deep Learning.
In machine learning, a convolutional neural network is a class of deep, feed-forward artificial neural networks that have successfully been applied fpr analyzing visual imagery.
This is a presentation on Handwritten Digit Recognition using Convolutional Neural Networks. Convolutional Neural Networks give better results as compared to conventional Artificial Neural Networks.
The slides includes an introduction to Long Short-term Memory (LSTM ) >> A novel approach in dealing with vanishing gradients in deep neural networks. Made for students, and anyone out there who'd love to learn about recurrent artificial neural networks, specifically of the LSTMs architecture.
Reference material has been attached to further your reading.
Deep Learning: Recurrent Neural Network (Chapter 10) Larry Guo
This Material is an in_depth study report of Recurrent Neural Network (RNN)
Material mainly from Deep Learning Book Bible, http://www.deeplearningbook.org/
Topics: Briefing, Theory Proof, Variation, Gated RNNN Intuition. Real World Application
Application (CNN+RNN on SVHN)
Also a video (In Chinese)
https://www.youtube.com/watch?v=p6xzPqRd46w
Explores the type of structure learned by Convolutional Neural Networks, the applications where they're most valuable and a number of appropriate mental models for understanding deep learning.
One-shot learning is an object categorization problem in computer vision. Whereas most machine learning based object categorization algorithms require training on hundreds or thousands of images and very large datasets, one-shot learning aims to learn information about object categories from one, or only a few, training images
This presentation on Recurrent Neural Network will help you understand what is a neural network, what are the popular neural networks, why we need recurrent neural network, what is a recurrent neural network, how does a RNN work, what is vanishing and exploding gradient problem, what is LSTM and you will also see a use case implementation of LSTM (Long short term memory). Neural networks used in Deep Learning consists of different layers connected to each other and work on the structure and functions of the human brain. It learns from huge volumes of data and used complex algorithms to train a neural net. The recurrent neural network works on the principle of saving the output of a layer and feeding this back to the input in order to predict the output of the layer. Now lets deep dive into this presentation and understand what is RNN and how does it actually work.
Below topics are explained in this recurrent neural networks tutorial:
1. What is a neural network?
2. Popular neural networks?
3. Why recurrent neural network?
4. What is a recurrent neural network?
5. How does an RNN work?
6. Vanishing and exploding gradient problem
7. Long short term memory (LSTM)
8. Use case implementation of LSTM
Simplilearn’s Deep Learning course will transform you into an expert in deep learning techniques using TensorFlow, the open-source software library designed to conduct machine learning & deep neural network research. With our deep learning course, you'll master deep learning and TensorFlow concepts, learn to implement algorithms, build artificial neural networks and traverse layers of data abstraction to understand the power of data and prepare you for your new role as deep learning scientist.
Why Deep Learning?
It is one of the most popular software platforms used for deep learning and contains powerful tools to help you build and implement artificial neural networks.
Advancements in deep learning are being seen in smartphone applications, creating efficiencies in the power grid, driving advancements in healthcare, improving agricultural yields, and helping us find solutions to climate change. With this Tensorflow course, you’ll build expertise in deep learning models, learn to operate TensorFlow to manage neural networks and interpret the results.
And according to payscale.com, the median salary for engineers with deep learning skills tops $120,000 per year.
You can gain in-depth knowledge of Deep Learning by taking our Deep Learning certification training course. With Simplilearn’s Deep Learning course, you will prepare for a career as a Deep Learning engineer as you master concepts and techniques including supervised and unsupervised learning, mathematical and heuristic aspects, and hands-on modeling to develop algorithms. Those who complete the course will be able to:
Learn more at: https://www.simplilearn.com/
From Conventional Machine Learning to Deep Learning and Beyond.pptxChun-Hao Chang
In this slide, Deep Learning are compared with Conventional Learning and the strength of DNN models will be explained.
The target audience are people who have the knowledge of Machine Learning or Data Mining but not familiar with Deep Learning.
In machine learning, a convolutional neural network is a class of deep, feed-forward artificial neural networks that have successfully been applied fpr analyzing visual imagery.
This is a presentation on Handwritten Digit Recognition using Convolutional Neural Networks. Convolutional Neural Networks give better results as compared to conventional Artificial Neural Networks.
This presentation is Part 2 of my September Lisp NYC presentation on Reinforcement Learning and Artificial Neural Nets. We will continue from where we left off by covering Convolutional Neural Nets (CNN) and Recurrent Neural Nets (RNN) in depth.
Time permitting I also plan on having a few slides on each of the following topics:
1. Generative Adversarial Networks (GANs)
2. Differentiable Neural Computers (DNCs)
3. Deep Reinforcement Learning (DRL)
Some code examples will be provided in Clojure.
After a very brief recap of Part 1 (ANN & RL), we will jump right into CNN and their appropriateness for image recognition. We will start by covering the convolution operator. We will then explain feature maps and pooling operations and then explain the LeNet 5 architecture. The MNIST data will be used to illustrate a fully functioning CNN.
Next we cover Recurrent Neural Nets in depth and describe how they have been used in Natural Language Processing. We will explain why gated networks and LSTM are used in practice.
Please note that some exposure or familiarity with Gradient Descent and Backpropagation will be assumed. These are covered in the first part of the talk for which both video and slides are available online.
A lot of material will be drawn from the new Deep Learning book by Goodfellow & Bengio as well as Michael Nielsen's online book on Neural Networks and Deep Learning as well several other online resources.
Bio
Pierre de Lacaze has over 20 years industry experience with AI and Lisp based technologies. He holds a Bachelor of Science in Applied Mathematics and a Master’s Degree in Computer Science.
https://www.linkedin.com/in/pierre-de-lacaze-b11026b/
A comprehensive tutorial on Convolutional Neural Networks (CNN) which talks about the motivation behind CNNs and Deep Learning in general, followed by a description of the various components involved in a typical CNN layer. It explains the theory involved with the different variants used in practice and also, gives a big picture of the whole network by putting everything together.
Next, there's a discussion of the various state-of-the-art frameworks being used to implement CNNs to tackle real-world classification and regression problems.
Finally, the implementation of the CNNs is demonstrated by implementing the paper 'Age ang Gender Classification Using Convolutional Neural Networks' by Hassner (2015).
Lecture conducted by me on Deep Learning concepts and applications. Discussed FNNs, CNNs, Simple RNNs and LSTM Networks in detail. Finally conducted a hands-on session on deep-learning using Keras and scikit-learn.
Sachpazis:Terzaghi Bearing Capacity Estimation in simple terms with Calculati...Dr.Costas Sachpazis
Terzaghi's soil bearing capacity theory, developed by Karl Terzaghi, is a fundamental principle in geotechnical engineering used to determine the bearing capacity of shallow foundations. This theory provides a method to calculate the ultimate bearing capacity of soil, which is the maximum load per unit area that the soil can support without undergoing shear failure. The Calculation HTML Code included.
Explore the innovative world of trenchless pipe repair with our comprehensive guide, "The Benefits and Techniques of Trenchless Pipe Repair." This document delves into the modern methods of repairing underground pipes without the need for extensive excavation, highlighting the numerous advantages and the latest techniques used in the industry.
Learn about the cost savings, reduced environmental impact, and minimal disruption associated with trenchless technology. Discover detailed explanations of popular techniques such as pipe bursting, cured-in-place pipe (CIPP) lining, and directional drilling. Understand how these methods can be applied to various types of infrastructure, from residential plumbing to large-scale municipal systems.
Ideal for homeowners, contractors, engineers, and anyone interested in modern plumbing solutions, this guide provides valuable insights into why trenchless pipe repair is becoming the preferred choice for pipe rehabilitation. Stay informed about the latest advancements and best practices in the field.
Courier management system project report.pdfKamal Acharya
It is now-a-days very important for the people to send or receive articles like imported furniture, electronic items, gifts, business goods and the like. People depend vastly on different transport systems which mostly use the manual way of receiving and delivering the articles. There is no way to track the articles till they are received and there is no way to let the customer know what happened in transit, once he booked some articles. In such a situation, we need a system which completely computerizes the cargo activities including time to time tracking of the articles sent. This need is fulfilled by Courier Management System software which is online software for the cargo management people that enables them to receive the goods from a source and send them to a required destination and track their status from time to time.
Immunizing Image Classifiers Against Localized Adversary Attacksgerogepatton
This paper addresses the vulnerability of deep learning models, particularly convolutional neural networks
(CNN)s, to adversarial attacks and presents a proactive training technique designed to counter them. We
introduce a novel volumization algorithm, which transforms 2D images into 3D volumetric representations.
When combined with 3D convolution and deep curriculum learning optimization (CLO), itsignificantly improves
the immunity of models against localized universal attacks by up to 40%. We evaluate our proposed approach
using contemporary CNN architectures and the modified Canadian Institute for Advanced Research (CIFAR-10
and CIFAR-100) and ImageNet Large Scale Visual Recognition Challenge (ILSVRC12) datasets, showcasing
accuracy improvements over previous techniques. The results indicate that the combination of the volumetric
input and curriculum learning holds significant promise for mitigating adversarial attacks without necessitating
adversary training.
Event Management System Vb Net Project Report.pdfKamal Acharya
In present era, the scopes of information technology growing with a very fast .We do not see any are untouched from this industry. The scope of information technology has become wider includes: Business and industry. Household Business, Communication, Education, Entertainment, Science, Medicine, Engineering, Distance Learning, Weather Forecasting. Carrier Searching and so on.
My project named “Event Management System” is software that store and maintained all events coordinated in college. It also helpful to print related reports. My project will help to record the events coordinated by faculties with their Name, Event subject, date & details in an efficient & effective ways.
In my system we have to make a system by which a user can record all events coordinated by a particular faculty. In our proposed system some more featured are added which differs it from the existing system such as security.
Water scarcity is the lack of fresh water resources to meet the standard water demand. There are two type of water scarcity. One is physical. The other is economic water scarcity.
Democratizing Fuzzing at Scale by Abhishek Aryaabh.arya
Presented at NUS: Fuzzing and Software Security Summer School 2024
This keynote talks about the democratization of fuzzing at scale, highlighting the collaboration between open source communities, academia, and industry to advance the field of fuzzing. It delves into the history of fuzzing, the development of scalable fuzzing platforms, and the empowerment of community-driven research. The talk will further discuss recent advancements leveraging AI/ML and offer insights into the future evolution of the fuzzing landscape.
Cosmetic shop management system project report.pdfKamal Acharya
Buying new cosmetic products is difficult. It can even be scary for those who have sensitive skin and are prone to skin trouble. The information needed to alleviate this problem is on the back of each product, but it's thought to interpret those ingredient lists unless you have a background in chemistry.
Instead of buying and hoping for the best, we can use data science to help us predict which products may be good fits for us. It includes various function programs to do the above mentioned tasks.
Data file handling has been effectively used in the program.
The automated cosmetic shop management system should deal with the automation of general workflow and administration process of the shop. The main processes of the system focus on customer's request where the system is able to search the most appropriate products and deliver it to the customers. It should help the employees to quickly identify the list of cosmetic product that have reached the minimum quantity and also keep a track of expired date for each cosmetic product. It should help the employees to find the rack number in which the product is placed.It is also Faster and more efficient way.
Industrial Training at Shahjalal Fertilizer Company Limited (SFCL)MdTanvirMahtab2
This presentation is about the working procedure of Shahjalal Fertilizer Company Limited (SFCL). A Govt. owned Company of Bangladesh Chemical Industries Corporation under Ministry of Industries.
Forklift Classes Overview by Intella PartsIntella Parts
Discover the different forklift classes and their specific applications. Learn how to choose the right forklift for your needs to ensure safety, efficiency, and compliance in your operations.
For more technical information, visit our website https://intellaparts.com
2. A Survey of Convolutional Neural
Networks:
Analysis, Applications, and Prospects
Zewen Li, Wenjie Yang, Shouheng Peng, Fan Liu, Member, IEEE
3. Introduction to Convolution Neural Network (CNN)
•Applications using CNN –
•Face recognition
•Autonomous vehicles
•Self-service supermarket
•Intelligent medical treatment
4. Emergence of CNN
• McCulloch and Pitts – First mathematical MP model of
neurons
• Rosenblatt - Added learning capability to MP model
• Hinton – Proposed multi-layer feedforward network trained
by the error Back Propagation – BP network
• Waibel - Time Delay Neural Network (TDNN) for speech
recognition
• LeCun – First convolution network (LeNet) to recognize
handwritten text
5. Overview of CNN
• Feedforward neural network
• Extracts features from data from convolution structures
• Architecture inspired by visual perception
• Biological neuron corresponds to an artificial neuron
• CNN kernels represent different receptors that can respond to various
features
• Activation function transmit signal to next neuron if it exceeds certain
threshold
• Loss functions and optimizers teach the whole CNN system to learn
6. Advantages of CNN
• Local connections – Each neuron connected to not all but
small no. of neurons. Reduces parameters and speed up
convergence.
• Weight sharing - Connections share same weights
• Down-sampling dimensionality reduction.
• These characteristics make CNN most representative
algorithms
7. Components of CNN
• Convolution - pivotal step for feature extraction. Output is feature
map
• Padding - introduced to enlarge the input with zero value
• Stride – Control the density of convolution
• Pooling - Obviate redundancy or down sampling
8. LeNet - 5
• Composed of 7 trainable layers containing 2 convolutional layers, 2
pooling layers, and 3 fully-connected layers
• NN characteristics of local receptive fields, shared weights, and spatial
or temporal subsampling, ensures shift, scale, and distortion
• Used for handwriting recognition
9. AlexNet
• Has 8 layers, containing 5 convolutional layers and 3 fully-connected
layers
• uses ReLU as the activation function of CNN to solve gradient
vanishing
• Dropout was used in last few layers to avoid overfitting
• Local Response Normalization (LRN) to enhance generalization of
model
10. AlexNet
• Employ 2 powerful GPUs, two feature maps generated by two GPUs
can be combined as the final output
• Enlarges dataset and calculates average of their predictions as final
result
• Principal Component Analysis (PCA) to change the RGB values of
training set
11. VGGNet
• LRN layer was removed
• VGGNets use 3 × 3 convolution kernels rather than 5 × 5 or 5 × 5
ones, since several small kernels have the same receptive field and
more nonlinear variations compared with larger ones.
12. GoogLeNet - Inception v1
• CNN formed by stacking with Inception modules
• Inception v1 deploys 1 × 1, 3 × 3, 5 × 5 convolution kernels to
construct a “wide” network
• Convolution kernels with different sizes can extract the feature maps
of different scales of the image
• 1 × 1 convolution kernel is used to reduce the number of channels,
i.e., reduce computational cost
13. GoogLeNet - Inception v2
• Output of every layer is normalized to increase the robustness of
model and train it with high learning rate
• Single 5 × 5 convolutional layers can be replaced by two 3 × 3 ones
• One n x n convolutional layer can be replaced byone 1 x n and one n x
1 convolutional layer
• Filter banks expanded wider to improve high dimensional
representations
14. ResNet
• Two layer residual block constructed by the shortcut connection
• 50-layer ResNet, 101-layer ResNet, and 152-layer ResNet utilize three-
layer residual blocks
• Three-layer residual block is also called the bottleneck module
because the two ends of the block are narrower than the middle
• Can mitigate the gradient vanishing problem since the gradient can
directly flow through shortcut connections
•
15. DCGAN
• GAN has generative model G and a discriminative model D
• The model G with random noise z generates a sample G(z) that
subjects to the data distribution data learned by G.
• The model D can determine whether the input sample is real data x
or generated data G(z).
• Both G and D can be nonlinear functions. The aim of G is to generate
real data, the aim of D is to distinguish fake data generated by G from
the real data
16. MobileNets
• lightweight models proposed by Google for embedded devices such
as mobile phones
• depth-wise separable convolutions and several advanced techniques
to build thin deep neural networks.
17. ShuffleNets
• Series of CNN-based models to solve the problem of insufficient
computing power of mobile devices
• Combine pointwise group convolution, channel shuffle, which
significantly reduce the computational cost with little loss of accuracy
18. GhostNet
• As large amounts of redundant features are extracted by existing
CNNs for image cognition, GhostNet is used to reduce computational
cost effectively
• Similar feature maps in traditional convolution layers are called ghost
• Traditional convolution layers divided into two parts
• Less convolution kernels are directly used in feature extraction
• These features are processed in linear transformation to acquire
multiple feature maps. They proved that Ghost module applies to
other CNN models
19. Activation function
• In a multilayer neural network, there is a function between two layers
which is called activation function
• Determines which information should be transmitted to the next
neuron
• If no activation function, input layer will be linear function of the
output
• Nonlinear functions are introduced as activation functions to enhance
ability of neural network
20. Types of activation function
• Sigmoid function can map a real number to (0, 1), so it can be used
for binary classification problems.
• Tanh function maps a real number to (-1, 1), achieves normalization.
This makes the next layer easier to learn.
• Rectified Linear Unit (ReLU), when x is less than 0, its value is 0; when
x is greater than or equal to 0, its value is x itself. Speeds up learning.
• ELU function has a negative value, so the average value of its output is
close to 0, making the rate of convergence faster than ReLU.
21. Loss/Cost function
• Calculates the distance between the predicted value and the actual
value
• Used as a learning criterion of the optimization problem
• Common loss functions Mean Absolute Error (MAE), Mean Square
Error (MSE), Cross Entropy
22. Rules of Thumb for Loss Function Selection
• CNN models for regression problems, choose L1 loss or L2 loss as the
loss function.
• For classification problems, select the rest of the loss functions
• Cross entropy loss is the most popular choice, with a softmax layer in
the end.
• The selection of loss function in CNNs also depends on the
application scenario. For example, when it comes to face recognition,
contrastive loss and triplet loss are turned out to be the commonly-
used ones nowadays.
23. Optimizer
• In convolutional neural networks, need to optimize non-convex
functions.
• Mathematical methods require huge computing power, so optimizers
are used in the training process to minimize the loss function for
getting optimal network parameters within acceptable time.
• Common optimization algorithms are Momentum, RMSprop, Adam,
etc.
24. Applications of one-dimensional CNN
• Time Series Prediction
• Electrocardiogram (ECG) time series, weather forecast, and traffic flow
prediction, highway traffic flow prediction
• Signal Identification
• ECG signal identification, structural damage identification, and system fault
identification
25. Applications of two-dimensional CNN
• Image Classification
• medical image classification, traffic scenes related classification, classify
breast cancer tissues
• Object Detection
• Image Segmentation
• Face Recognition
27. Conclusion
• Due to the advantages of convolutional neural networks, such as local
connection, weight sharing, and down-sampling dimensionality reduction,
they have been widely deployed in both research and industry projects
• First, we discussed basic building blocks of CNN and how to construct a
CNN-based model from scratch
• Secondly, some excellent CNN networks
• Third, we introduce activation functions, loss functions, and optimizers for
CNN
• Fourth, we discuss some typical applications of CNN
• CNN can be refined further in terms of model size, security, and easy
hyperparameters selection. Moreover, there are lots of problems that
convolution is hard to handle, such as low generalization ability, lack of
equivariance, and poor crowded-scene results, so that several promising
directions are pointed.