Ashray Bhandare, profile picture
Uploaded byAshray Bhandare
PPTX, PDF19,205 views

Convolutional Neural Networks

The document provides an overview of convolutional neural networks (CNNs) and their layers. It begins with an introduction to CNNs, noting they are a type of neural network designed to process 2D inputs like images. It then discusses the typical CNN architecture of convolutional layers followed by pooling and fully connected layers. The document explains how CNNs work using a simple example of classifying handwritten X and O characters. It provides details on the different layer types, including convolutional layers which identify patterns using small filters, and pooling layers which downsample the inputs.

Embed presentation

Downloaded 811 times
Convolutional Neural Networks Ashray Bhandare Anil Sehgal
Page  2 Agenda  Introduction  Archetecture Overview – How ConvNet Works  ConvNet Layers – Convolutional Layer – Pooling Layer – Normalization Layer (ReLU) – Fully-Connected Layer  Demos & Parameters – Hyper Parameters – Mnist dataset on AWS – CIFAR-10 ConvNetJS  Case Studies EECS6980:006 Social Network Analysis
Page  3 Agenda  Introduction  Archetecture Overview – How ConvNet Works  ConvNet Layers – Convolutional Layer – Pooling Layer – Normalization Layer (ReLU) – Fully-Connected Layer  Demos & Parameters – Hyper Parameters – Mnist dataset on AWS – CIFAR-10 ConvNetJS  Case Studies EECS6980:006 Social Network Analysis
Page  4 Introduction  A convolutional neural network (or ConvNet) is a type of feed-forward artificial neural network  The architecture of a ConvNet is designed to take advantage of the 2D structure of an input image.    A ConvNet is comprised of one or more convolutional layers (often with a pooling step) and then followed by one or more fully connected layers as in a standard multilayer neural network. EECS6980:006 Social Network Analysis VS
Page  5 Agenda  Introduction  Archetecture Overview – How ConvNet Works  ConvNet Layers – Convolutional Layer – Pooling Layer – Normalization Layer (ReLU) – Fully-Connected Layer  Demos & Parameters – Hyper Parameters – Mnist dataset on AWS – CIFAR-10 ConvNetJS  Case Studies EECS6980:006 Social Network Analysis
Page  6 Motivation behind ConvNets  Consider an image of size 200x200x3 (200 wide, 200 high, 3 color channels) – a single fully-connected neuron in a first hidden layer of a regular Neural Network would have 200*200*3 = 120,000 weights. – Due to the presence of several such neurons, this full connectivity is wasteful and the huge number of parameters would quickly lead to overfitting  However, in a ConvNet, the neurons in a layer will only be connected to a small region of the layer before it, instead of all of the neurons in a fully-connected manner. – the final output layer would have dimensions 1x1xN, because by the end of the ConvNet architecture we will reduce the full image into a single vector of class scores (for N classes), arranged along the depth dimension EECS6980:006 Social Network Analysis
Page  7 MLP VS ConvNet EECS6980:006 Social Network Analysis Input Hidden Output Input Hidden Output Multilayered Perceptron: All Fully Connected Layers Convolutional Neural Network With Partially Connected Convolution Layer
Page  8 MLP vs ConvNet A regular 3-layer Neural Network. A ConvNet arranges its neurons in three dimensions (width, height, depth), as visualized in one of the layers. EECS6980:006 Social Network Analysis
Page  9 Agenda  Introduction  Archetecture Overview – How ConvNet Works  ConvNet Layers – Convolutional Layer – Pooling Layer – Normalization Layer (ReLU) – Fully-Connected Layer  Demos & Parameters – Hyper Parameters – Mnist dataset on AWS – CIFAR-10 ConvNetJS  Case Studies EECS6980:006 Social Network Analysis
Page  10 How ConvNet Works  For example, a ConvNet takes the input as an image which can be classified as ‘X’ or ‘O’  In a simple case, ‘X’ would look like: X or OCNN A two-dimensional array of pixels
Page  11 How ConvNet Works  What about trickier cases? CNN X CNN O
Page  12 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 1 -1 -1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 -1 -1 1 -1 1 -1 -1 -1 -1 -1 -1 -1 1 -1 -1 -1 -1 -1 -1 -1 1 -1 1 -1 -1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 -1 -1 1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 -1 -1 1 1 -1 1 -1 -1 -1 -1 -1 -1 -1 1 -1 -1 -1 -1 -1 -1 -1 1 -1 1 1 -1 -1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 = ? How ConvNet Works – What Computer Sees
Page  13 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 1 -1 -1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 -1 -1 1 -1 1 -1 -1 -1 -1 -1 -1 -1 1 -1 -1 -1 -1 -1 -1 -1 1 -1 1 -1 -1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 -1 -1 1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 -1 -1 1 1 -1 1 -1 -1 -1 -1 -1 -1 -1 1 -1 -1 -1 -1 -1 -1 -1 1 -1 1 1 -1 -1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 =x How ConvNet Works
Page  14 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 X -1 -1 -1 -1 X X -1 -1 X X -1 -1 X X -1 -1 -1 -1 X 1 -1 1 -1 -1 -1 -1 -1 -1 -1 1 -1 -1 -1 -1 -1 -1 -1 1 -1 1 X -1 -1 -1 -1 X X -1 -1 X X -1 -1 X X -1 -1 -1 -1 X -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 How ConvNet Works – What Computer Sees  Since the pattern doesnot match exactly, the computer will not be able to classify this as ‘X’
Page  15 Agenda  Introduction  Archetecture Overview – How ConvNet Works  ConvNet Layers – Convolutional Layer – Pooling Layer – Normalization Layer (ReLU) – Fully-Connected Layer  Demos & Parameters – Hyper Parameters – Mnist dataset on AWS – CIFAR-10 ConvNetJS  Case Studies EECS6980:006 Social Network Analysis
Page  16 ConvNet Layers (At a Glance)  CONV layer will compute the output of neurons that are connected to local regions in the input, each computing a dot product between their weights and a small region they are connected to in the input volume.  RELU layer will apply an elementwise activation function, such as the max(0,x) thresholding at zero. This leaves the size of the volume unchanged.  POOL layer will perform a downsampling operation along the spatial dimensions (width, height).  FC (i.e. fully-connected) layer will compute the class scores, resulting in volume of size [1x1xN], where each of the N numbers correspond to a class score, such as among the N categories. EECS6980:006 Social Network Analysis
Page  17 Agenda  Introduction  Archetecture Overview – How ConvNet Works  ConvNet Layers – Convolutional Layer – Pooling Layer – Normalization Layer (ReLU) – Fully-Connected Layer  Demos & Parameters – Hyper Parameters – Mnist dataset on AWS – CIFAR-10 ConvNetJS  Case Studies EECS6980:006 Social Network Analysis
Page  18 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 X -1 -1 -1 -1 X X -1 -1 X X -1 -1 X X -1 -1 -1 -1 X 1 -1 1 -1 -1 -1 -1 -1 -1 -1 1 -1 -1 -1 -1 -1 -1 -1 1 -1 1 X -1 -1 -1 -1 X X -1 -1 X X -1 -1 X X -1 -1 -1 -1 X -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 Recall – What Computer Sees  Since the pattern doesnot match exactly, the computer will not be able to classify this as ‘X’  What got changed?
Page  19 = = =  Convolution layer will work to identify patterns (features) instead of individual pixels Convolutional Layer
Page  20 1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 1 -1 1 -1 1 -1 -1 1 -1 1 -1 1 -1 1 -1 1 Convolutional Layer - Filters  The CONV layer’s parameters consist of a set of learnable filters.  Every filter is small spatially (along width and height), but extends through the full depth of the input volume.  During the forward pass, we slide (more precisely, convolve) each filter across the width and height of the input volume and compute dot products between the entries of the filter and the input at any position.
Page  21 1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 1 -1 1 -1 1 -1 -1 1 -1 1 -1 1 -1 1 -1 1 Convolutional Layer - Filters  Sliding the filter over the width and height of the input gives 2-dimensional activation map that responds to that filter at every spatial position.
Page  22 Convolutional Layer - Strides Input Hidden Output Input Hidden Output 1-D convolution Filters: 1 Filter Size: 2 Stride: 2 1-D convolution Filters: 1 Filter Size: 2 Stride: 1 • The distance that filter is moved across the input from the previous layer each activation is referred to as the stride.
Page  23 Convolutional Layer - Padding  Sometimes it is convenient to pad the input volume with zeros around the border.  Zero padding is allows us to preserve the spatial size of the output volumes EECS6980:006 Social Network Analysis Input Hidden Output 1 D convolution with: Filters: 1 Filter Size: 2 Stride: 2 Padding: 1 0 0 Input Hidden Output 1 D convolution with: Filters: 2 Filter Size: 2 Stride: 2 Padding: 1 0 0
Page  24 1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 1 -1 -1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 -1 -1 1 -1 1 -1 -1 -1 -1 -1 -1 -1 1 -1 -1 -1 -1 -1 -1 -1 1 -1 1 -1 -1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 -1 -1 1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 Convolutional Layer – Filters – Navigation Example  Strides = 1, Filter Size = 3 X 3 X 1, Padding = 0
Page  25 1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 1 -1 -1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 -1 -1 1 -1 1 -1 -1 -1 -1 -1 -1 -1 1 -1 -1 -1 -1 -1 -1 -1 1 -1 1 -1 -1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 -1 -1 1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 Convolutional Layer – Filters – Navigation Example
Page  26 1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 1 -1 -1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 -1 -1 1 -1 1 -1 -1 -1 -1 -1 -1 -1 1 -1 -1 -1 -1 -1 -1 -1 1 -1 1 -1 -1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 -1 -1 1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 Convolutional Layer – Filters – Navigation Example
Page  27 1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 1 -1 -1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 -1 -1 1 -1 1 -1 -1 -1 -1 -1 -1 -1 1 -1 -1 -1 -1 -1 -1 -1 1 -1 1 -1 -1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 -1 -1 1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 Convolutional Layer – Filters – Navigation Example
Page  28 1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 1 -1 -1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 -1 -1 1 -1 1 -1 -1 -1 -1 -1 -1 -1 1 -1 -1 -1 -1 -1 -1 -1 1 -1 1 -1 -1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 -1 -1 1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 Convolutional Layer – Filters – Navigation Example
Page  29 1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 1 -1 -1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 -1 -1 1 -1 1 -1 -1 -1 -1 -1 -1 -1 1 -1 -1 -1 -1 -1 -1 -1 1 -1 1 -1 -1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 -1 -1 1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 Convolutional Layer – Filters – Navigation Example
Page  30 1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 1 -1 -1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 -1 -1 1 -1 1 -1 -1 -1 -1 -1 -1 -1 1 -1 -1 -1 -1 -1 -1 -1 1 -1 1 -1 -1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 -1 -1 1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 Convolutional Layer – Filters – Navigation Example
Page  31 1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 1 -1 -1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 -1 -1 1 -1 1 -1 -1 -1 -1 -1 -1 -1 1 -1 -1 -1 -1 -1 -1 -1 1 -1 1 -1 -1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 -1 -1 1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 Convolutional Layer – Filters – Navigation Example
Page  32 1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 1 -1 -1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 -1 -1 1 -1 1 -1 -1 -1 -1 -1 -1 -1 1 -1 -1 -1 -1 -1 -1 -1 1 -1 1 -1 -1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 -1 -1 1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 Convolutional Layer – Filters – Navigation Example
Page  33 1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 1 -1 -1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 -1 -1 1 -1 1 -1 -1 -1 -1 -1 -1 -1 1 -1 -1 -1 -1 -1 -1 -1 1 -1 1 -1 -1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 -1 -1 1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 Convolutional Layer – Filters – Computation Example
Page  34 1 1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 1 -1 -1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 -1 -1 1 -1 1 -1 -1 -1 -1 -1 -1 -1 1 -1 -1 -1 -1 -1 -1 -1 1 -1 1 -1 -1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 -1 -1 1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 Convolutional Layer – Filters – Computation Example
Page  35 1 1 1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 1 -1 -1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 -1 -1 1 -1 1 -1 -1 -1 -1 -1 -1 -1 1 -1 -1 -1 -1 -1 -1 -1 1 -1 1 -1 -1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 -1 -1 1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 Convolutional Layer – Filters – Computation Example
Page  36 1 1 1 1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 1 -1 -1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 -1 -1 1 -1 1 -1 -1 -1 -1 -1 -1 -1 1 -1 -1 -1 -1 -1 -1 -1 1 -1 1 -1 -1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 -1 -1 1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 Convolutional Layer – Filters – Computation Example
Page  37 1 1 1 1 1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 1 -1 -1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 -1 -1 1 -1 1 -1 -1 -1 -1 -1 -1 -1 1 -1 -1 -1 -1 -1 -1 -1 1 -1 1 -1 -1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 -1 -1 1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 Convolutional Layer – Filters – Computation Example
Page  38 1 1 1 1 1 1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 1 -1 -1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 -1 -1 1 -1 1 -1 -1 -1 -1 -1 -1 -1 1 -1 -1 -1 -1 -1 -1 -1 1 -1 1 -1 -1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 -1 -1 1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 Convolutional Layer – Filters – Computation Example
Page  39 1 1 1 1 1 1 1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 1 -1 -1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 -1 -1 1 -1 1 -1 -1 -1 -1 -1 -1 -1 1 -1 -1 -1 -1 -1 -1 -1 1 -1 1 -1 -1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 -1 -1 1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 Convolutional Layer – Filters – Computation Example
Page  40 1 1 1 1 1 1 1 1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 1 -1 -1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 -1 -1 1 -1 1 -1 -1 -1 -1 -1 -1 -1 1 -1 -1 -1 -1 -1 -1 -1 1 -1 1 -1 -1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 -1 -1 1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 Convolutional Layer – Filters – Computation Example
Page  41 1 1 1 1 1 1 1 1 1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 1 -1 -1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 -1 -1 1 -1 1 -1 -1 -1 -1 -1 -1 -1 1 -1 -1 -1 -1 -1 -1 -1 1 -1 1 -1 -1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 -1 -1 1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 Convolutional Layer – Filters – Computation Example
Page  42 1 1 1 1 1 1 1 1 1 1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 1 -1 -1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 -1 -1 1 -1 1 -1 -1 -1 -1 -1 -1 -1 1 -1 -1 -1 -1 -1 -1 -1 1 -1 1 -1 -1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 -1 -1 1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 Convolutional Layer – Filters – Computation Example
Page  43 1 1 1 1 1 1 1 1 1 1 1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 1 -1 -1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 -1 -1 1 -1 1 -1 -1 -1 -1 -1 -1 -1 1 -1 -1 -1 -1 -1 -1 -1 1 -1 1 -1 -1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 -1 -1 1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 Convolutional Layer – Filters – Computation Example
Page  44 1 1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 1 -1 -1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 -1 -1 1 -1 1 -1 -1 -1 -1 -1 -1 -1 1 -1 -1 -1 -1 -1 -1 -1 1 -1 1 -1 -1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 -1 -1 1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 Convolutional Layer – Filters – Computation Example
Page  45 1 1 -1 1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 1 -1 -1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 -1 -1 1 -1 1 -1 -1 -1 -1 -1 -1 -1 1 -1 -1 -1 -1 -1 -1 -1 1 -1 1 -1 -1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 -1 -1 1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 Convolutional Layer – Filters – Computation Example
Page  46 1 1 -1 1 1 1 -1 1 1 1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 1 -1 -1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 -1 -1 1 -1 1 -1 -1 -1 -1 -1 -1 -1 1 -1 -1 -1 -1 -1 -1 -1 1 -1 1 -1 -1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 -1 -1 1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 Convolutional Layer – Filters – Computation Example
Page  47 1 1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 1 -1 -1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 -1 -1 1 -1 1 -1 -1 -1 -1 -1 -1 -1 1 -1 -1 -1 -1 -1 -1 -1 1 -1 1 -1 -1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 -1 -1 1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 1 1 -1 1 1 1 -1 1 1 55 1 1 -1 1 1 1 -1 1 1 Convolutional Layer – Filters – Computation Example
Page  48 1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 1 -1 -1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 -1 -1 1 -1 1 -1 -1 -1 -1 -1 -1 -1 1 -1 -1 -1 -1 -1 -1 -1 1 -1 1 -1 -1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 -1 -1 1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 = 0.77 -0.11 0.11 0.33 0.55 -0.11 0.33 -0.11 1.00 -0.11 0.33 -0.11 0.11 -0.11 0.11 -0.11 1.00 -0.33 0.11 -0.11 0.55 0.33 0.33 -0.33 0.55 -0.33 0.33 0.33 0.55 -0.11 0.11 -0.33 1.00 -0.11 0.11 -0.11 0.11 -0.11 0.33 -0.11 1.00 -0.11 0.33 -0.11 0.55 0.33 0.11 -0.11 0.77 Convolutional Layer – Filters – Computation Example Input Size (W): 9 Filter Size (F): 3 X 3 Stride (S): 1 Filters: 1 Padding (P): 0 9 X 9 7 X 7 Feature Map Size = 1+ (W – F + 2P)/S = 1+ (9 – 3 + 2 X 0)/1 = 7
Page  49 1 -1 -1 -1 1 -1 -1 -1 1 0.33 -0.11 0.55 0.33 0.11 -0.11 0.77 -0.11 0.11 -0.11 0.33 -0.11 1.00 -0.11 0.55 -0.11 0.11 -0.33 1.00 -0.11 0.11 0.33 0.33 -0.33 0.55 -0.33 0.33 0.33 0.11 -0.11 1.00 -0.33 0.11 -0.11 0.55 -0.11 1.00 -0.11 0.33 -0.11 0.11 -0.11 0.77 -0.11 0.11 0.33 0.55 -0.11 0.33 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 1 -1 -1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 -1 -1 1 -1 1 -1 -1 -1 -1 -1 -1 -1 1 -1 -1 -1 -1 -1 -1 -1 1 -1 1 -1 -1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 -1 -1 1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 = 0.77 -0.11 0.11 0.33 0.55 -0.11 0.33 -0.11 1.00 -0.11 0.33 -0.11 0.11 -0.11 0.11 -0.11 1.00 -0.33 0.11 -0.11 0.55 0.33 0.33 -0.33 0.55 -0.33 0.33 0.33 0.55 -0.11 0.11 -0.33 1.00 -0.11 0.11 -0.11 0.11 -0.11 0.33 -0.11 1.00 -0.11 0.33 -0.11 0.55 0.33 0.11 -0.11 0.77 -1 -1 1 -1 1 -1 1 -1 -1 1 -1 1 -1 1 -1 1 -1 1 0.33 -0.55 0.11 -0.11 0.11 -0.55 0.33 -0.55 0.55 -0.55 0.33 -0.55 0.55 -0.55 0.11 -0.55 0.55 -0.77 0.55 -0.55 0.11 -0.11 0.33 -0.77 1.00 -0.77 0.33 -0.11 0.11 -0.55 0.55 -0.77 0.55 -0.55 0.11 -0.55 0.55 -0.55 0.33 -0.55 0.55 -0.55 0.33 -0.55 0.11 -0.11 0.11 -0.55 0.33 = = -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 1 -1 -1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 -1 -1 1 -1 1 -1 -1 -1 -1 -1 -1 -1 1 -1 -1 -1 -1 -1 -1 -1 1 -1 1 -1 -1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 -1 -1 1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 1 -1 -1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 -1 -1 1 -1 1 -1 -1 -1 -1 -1 -1 -1 1 -1 -1 -1 -1 -1 -1 -1 1 -1 1 -1 -1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 -1 -1 1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 Convolutional Layer – Filters – Output Feature Map  Output Feature Map of One complete convolution: – Filters: 3 – Filter Size: 3 X 3 – Stride: 1  Conclusion: – Input Image: 9 X 9 – Output of Convolution: 7 X 7 X 3
Page  50 0.33 -0.11 0.55 0.33 0.11 -0.11 0.77 -0.11 0.11 -0.11 0.33 -0.11 1.00 -0.11 0.55 -0.11 0.11 -0.33 1.00 -0.11 0.11 0.33 0.33 -0.33 0.55 -0.33 0.33 0.33 0.11 -0.11 1.00 -0.33 0.11 -0.11 0.55 -0.11 1.00 -0.11 0.33 -0.11 0.11 -0.11 0.77 -0.11 0.11 0.33 0.55 -0.11 0.33 0.77 -0.11 0.11 0.33 0.55 -0.11 0.33 -0.11 1.00 -0.11 0.33 -0.11 0.11 -0.11 0.11 -0.11 1.00 -0.33 0.11 -0.11 0.55 0.33 0.33 -0.33 0.55 -0.33 0.33 0.33 0.55 -0.11 0.11 -0.33 1.00 -0.11 0.11 -0.11 0.11 -0.11 0.33 -0.11 1.00 -0.11 0.33 -0.11 0.55 0.33 0.11 -0.11 0.77 0.33 -0.55 0.11 -0.11 0.11 -0.55 0.33 -0.55 0.55 -0.55 0.33 -0.55 0.55 -0.55 0.11 -0.55 0.55 -0.77 0.55 -0.55 0.11 -0.11 0.33 -0.77 1.00 -0.77 0.33 -0.11 0.11 -0.55 0.55 -0.77 0.55 -0.55 0.11 -0.55 0.55 -0.55 0.33 -0.55 0.55 -0.55 0.33 -0.55 0.11 -0.11 0.11 -0.55 0.33 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 1 -1 -1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 -1 -1 1 -1 1 -1 -1 -1 -1 -1 -1 -1 1 -1 -1 -1 -1 -1 -1 -1 1 -1 1 -1 -1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 -1 -1 1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 Convolutional Layer – Output
Page  51 Agenda  Introduction  Archetecture Overview – How ConvNet Works  ConvNet Layers – Convolutional Layer – Pooling Layer – Normalization Layer (ReLU) – Fully-Connected Layer  Demos & Parameters – Hyper Parameters – Mnist dataset on AWS – CIFAR-10 ConvNetJS  Case Studies EECS6980:006 Social Network Analysis
Page  52 Rectified Linear Units (ReLUs) 0.77 -0.11 0.11 0.33 0.55 -0.11 0.33 -0.11 1.00 -0.11 0.33 -0.11 0.11 -0.11 0.11 -0.11 1.00 -0.33 0.11 -0.11 0.55 0.33 0.33 -0.33 0.55 -0.33 0.33 0.33 0.55 -0.11 0.11 -0.33 1.00 -0.11 0.11 -0.11 0.11 -0.11 0.33 -0.11 1.00 -0.11 0.33 -0.11 0.55 0.33 0.11 -0.11 0.77 0.77
Page  53 0.77 0 Rectified Linear Units (ReLUs) 0.77 -0.11 0.11 0.33 0.55 -0.11 0.33 -0.11 1.00 -0.11 0.33 -0.11 0.11 -0.11 0.11 -0.11 1.00 -0.33 0.11 -0.11 0.55 0.33 0.33 -0.33 0.55 -0.33 0.33 0.33 0.55 -0.11 0.11 -0.33 1.00 -0.11 0.11 -0.11 0.11 -0.11 0.33 -0.11 1.00 -0.11 0.33 -0.11 0.55 0.33 0.11 -0.11 0.77
Page  54 0.77 0 0.11 0.33 0.55 0 0.33 Rectified Linear Units (ReLUs) 0.77 -0.11 0.11 0.33 0.55 -0.11 0.33 -0.11 1.00 -0.11 0.33 -0.11 0.11 -0.11 0.11 -0.11 1.00 -0.33 0.11 -0.11 0.55 0.33 0.33 -0.33 0.55 -0.33 0.33 0.33 0.55 -0.11 0.11 -0.33 1.00 -0.11 0.11 -0.11 0.11 -0.11 0.33 -0.11 1.00 -0.11 0.33 -0.11 0.55 0.33 0.11 -0.11 0.77
Page  55 0.77 0 0.11 0.33 0.55 0 0.33 0 1.00 0 0.33 0 0.11 0 0.11 0 1.00 0 0.11 0 0.55 0.33 0.33 0 0.55 0 0.33 0.33 0.55 0 0.11 0 1.00 0 0.11 0 0.11 0 0.33 0 1.00 0 0.33 0 0.55 0.33 0.11 0 0.77 Rectified Linear Units (ReLUs) 0.77 -0.11 0.11 0.33 0.55 -0.11 0.33 -0.11 1.00 -0.11 0.33 -0.11 0.11 -0.11 0.11 -0.11 1.00 -0.33 0.11 -0.11 0.55 0.33 0.33 -0.33 0.55 -0.33 0.33 0.33 0.55 -0.11 0.11 -0.33 1.00 -0.11 0.11 -0.11 0.11 -0.11 0.33 -0.11 1.00 -0.11 0.33 -0.11 0.55 0.33 0.11 -0.11 0.77
Page  56 ReLU layer 0.77 0 0.11 0.33 0.55 0 0.33 0 1.00 0 0.33 0 0.11 0 0.11 0 1.00 0 0.11 0 0.55 0.33 0.33 0 0.55 0 0.33 0.33 0.55 0 0.11 0 1.00 0 0.11 0 0.11 0 0.33 0 1.00 0 0.33 0 0.55 0.33 0.11 0 0.77 0.33 0 0.11 0 0.11 0 0.33 0 0.55 0 0.33 0 0.55 0 0.11 0 0.55 0 0.55 0 0.11 0 0.33 0 1.00 0 0.33 0 0.11 0 0.55 0 0.55 0 0.11 0 0.55 0 0.33 0 0.55 0 0.33 0 0.11 0 0.11 0 0.33 0.33 0 0.55 0.33 0.11 0 0.77 0 0.11 0 0.33 0 1.00 0 0.55 0 0.11 0 1.00 0 0.11 0.33 0.33 0 0.55 0 0.33 0.33 0.11 0 1.00 0 0.11 0 0.55 0 1.00 0 0.33 0 0.11 0 0.77 0 0.11 0.33 0.55 0 0.33 0.33 -0.11 0.55 0.33 0.11 -0.11 0.77 -0.11 0.11 -0.11 0.33 -0.11 1.00 -0.11 0.55 -0.11 0.11 -0.33 1.00 -0.11 0.11 0.33 0.33 -0.33 0.55 -0.33 0.33 0.33 0.11 -0.11 1.00 -0.33 0.11 -0.11 0.55 -0.11 1.00 -0.11 0.33 -0.11 0.11 -0.11 0.77 -0.11 0.11 0.33 0.55 -0.11 0.33 0.77 -0.11 0.11 0.33 0.55 -0.11 0.33 -0.11 1.00 -0.11 0.33 -0.11 0.11 -0.11 0.11 -0.11 1.00 -0.33 0.11 -0.11 0.55 0.33 0.33 -0.33 0.55 -0.33 0.33 0.33 0.55 -0.11 0.11 -0.33 1.00 -0.11 0.11 -0.11 0.11 -0.11 0.33 -0.11 1.00 -0.11 0.33 -0.11 0.55 0.33 0.11 -0.11 0.77 0.33 -0.55 0.11 -0.11 0.11 -0.55 0.33 -0.55 0.55 -0.55 0.33 -0.55 0.55 -0.55 0.11 -0.55 0.55 -0.77 0.55 -0.55 0.11 -0.11 0.33 -0.77 1.00 -0.77 0.33 -0.11 0.11 -0.55 0.55 -0.77 0.55 -0.55 0.11 -0.55 0.55 -0.55 0.33 -0.55 0.55 -0.55 0.33 -0.55 0.11 -0.11 0.11 -0.55 0.33
Page  57 Agenda  Introduction  Archetecture Overview – How ConvNet Works  ConvNet Layers – Convolutional Layer – Pooling Layer – Normalization Layer (ReLU) – Fully-Connected Layer  Demos & Parameters – Hyper Parameters – Mnist dataset on AWS – CIFAR-10 ConvNetJS  Case Studies EECS6980:006 Social Network Analysis
Page  58 Pooling Layer  The pooling layers down-sample the previous layers feature map.  Its function is to progressively reduce the spatial size of the representation to reduce the amount of parameters and computation in the network  The pooling layer often uses the Max operation to perform the downsampling process EECS6980:006 Social Network Analysis
Page  59 1.00 Pooling  Pooling Filter Size = 2 X 2, Stride = 2
Page  60 1.00 0.33 Pooling  Pooling Filter Size = 2 X 2, Stride = 2
Page  61 1.00 0.33 0.55 Pooling  Pooling Filter Size = 2 X 2, Stride = 2
Page  62 1.00 0.33 0.55 0.33 Pooling  Pooling Filter Size = 2 X 2, Stride = 2
Page  63 1.00 0.33 0.55 0.33 0.33 Pooling  Pooling Filter Size = 2 X 2, Stride = 2
Page  64 1.00 0.33 0.55 0.33 0.33 1.00 0.33 0.55 0.55 0.33 1.00 0.11 0.33 0.55 0.11 0.77 Pooling  Pooling Filter Size = 2 X 2, Stride = 2
Page  65 1.00 0.33 0.55 0.33 0.33 1.00 0.33 0.55 0.55 0.33 1.00 0.11 0.33 0.55 0.11 0.77 0.33 0.55 1.00 0.77 0.55 0.55 1.00 0.33 1.00 1.00 0.11 0.55 0.77 0.33 0.55 0.33 0.55 0.33 0.55 0.33 0.33 1.00 0.55 0.11 0.55 0.55 0.55 0.11 0.33 0.11 0.11 0.33 Pooling
Page  66 Layers get stacked -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 1 -1 -1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 -1 -1 1 -1 1 -1 -1 -1 -1 -1 -1 -1 1 -1 -1 -1 -1 -1 -1 -1 1 -1 1 -1 -1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 -1 -1 1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 1.00 0.33 0.55 0.33 0.33 1.00 0.33 0.55 0.55 0.33 1.00 0.11 0.33 0.55 0.11 0.77 0.33 0.55 1.00 0.77 0.55 0.55 1.00 0.33 1.00 1.00 0.11 0.55 0.77 0.33 0.55 0.33 0.55 0.33 0.55 0.33 0.33 1.00 0.55 0.11 0.55 0.55 0.55 0.11 0.33 0.11 0.11 0.33
Page  67 Layers Get Stacked - Example 224 X 224 X 3 224 X 224 X 64 CONVOLUTION WITH 64 FILTERS 112 X 112 X 64 POOLING (DOWNSAMPLING)
Page  68 Deep stacking -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 1 -1 -1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 -1 -1 1 -1 1 -1 -1 -1 -1 -1 -1 -1 1 -1 -1 -1 -1 -1 -1 -1 1 -1 1 -1 -1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 -1 -1 1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 1.00 0.55 0.55 1.00 0.55 1.00 1.00 0.55 1.00 0.55 0.55 0.55
Page  69 Agenda  Introduction  Archetecture Overview – How ConvNet Works  ConvNet Layers – Convolutional Layer – Pooling Layer – Normalization Layer (ReLU) – Fully-Connected Layer  Demos & Parameters – Hyper Parameters – Mnist dataset on AWS – CIFAR-10 ConvNetJS  Case Studies EECS6980:006 Social Network Analysis
Page  70 Fully connected layer  Fully connected layers are the normal flat feed-forward neural network layers.  These layers may have a non-linear activation function or a softmax activation in order to predict classes.  To compute our output, we simply re- arrange the output matrices as a 1-D array. 1.00 0.55 0.55 1.00 0.55 1.00 1.00 0.55 1.00 0.55 0.55 0.55 1.00 0.55 0.55 1.00 1.00 0.55 0.55 0.55 0.55 1.00 1.00 0.55
Page  71 Fully connected layer  A summation of product of inputs and weights at each output node determines the final prediction X O 0.55 1.00 1.00 0.55 0.55 0.55 0.55 0.55 1.00 0.55 0.55 1.00
Page  72 Putting it all together -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 1 -1 -1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 -1 -1 1 -1 1 -1 -1 -1 -1 -1 -1 -1 1 -1 -1 -1 -1 -1 -1 -1 1 -1 1 -1 -1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 -1 -1 1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 X O
Page  73 Agenda  Introduction  Archetecture Overview – How ConvNet Works  ConvNet Layers – Convolutional Layer – Pooling Layer – Normalization Layer (ReLU) – Fully-Connected Layer  Parameters & Demos – Hyper Parameters – Mnist dataset on AWS – CIFAR-10 ConvNetJS  Case Studies EECS6980:006 Social Network Analysis
Page  74 Hyperparameters (knobs)  Convolution – Filter Size – Number of Filters – Padding – Stride  Pooling – Window Size – Stride  Fully Connected – Number of neurons
Page  75 Agenda  Introduction  Archetecture Overview – How ConvNet Works  ConvNet Layers – Convolutional Layer – Pooling Layer – Normalization Layer (ReLU) – Fully-Connected Layer  Parameters & Demos – Hyper Parameters – Mnist dataset on AWS – CIFAR-10 ConvNetJS  Case Studies EECS6980:006 Social Network Analysis
Page  76 Agenda  Introduction  Archetecture Overview – How ConvNet Works  ConvNet Layers – Convolutional Layer – Pooling Layer – Normalization Layer (ReLU) – Fully-Connected Layer  Parameters & Demos – Hyper Parameters – Mnist dataset on AWS – CIFAR-10 ConvNetJS  Case Studies EECS6980:006 Social Network Analysis
Page  77 Case Studies  LeNet – 1998  AlexNet – 2012  ZFNet – 2013  VGG – 2014  GoogLeNet – 2014  ResNet – 2015 EECS6980:006 Social Network Analysis
Page  78 References  Karpathy, A. (n.d.). CS231n Convolutional Neural Networks for Visual Recognition. Retrieved from http://cs231n.github.io/convolutional-networks/#overview  Rohrer, B. (n.d.). How do Convolutional Neural Networks work?. Retrieved from http://brohrer.github.io/how_convolutional_neural_networks_work.html  Brownlee, J. (n.d.). Crash Course in Convolutional Neural Networks for Machine Learning. Retrieved from http://machinelearningmastery.com/crash-course-convolutional-neural-networks/  Lidinwise (n.d.). The revolution of depth. Retrieved from https://medium.com/@Lidinwise/the-revolution-of- depth-facf174924f5#.8or5c77ss  Nervana. (n.d.). Tutorial: Convolutional neural networks. Retrieved from https://www.nervanasys.com/convolutional-neural-networks/ EECS6980:006 Social Network Analysis
Page  79 Questions EECS6980:006 Social Network Analysis
Page  80 Thank you!! EECS6980:006 Social Network Analysis

More Related Content

PPTX
Convolutional Neural Network and Its Applications
byKasun Chinthaka Piyarathna
 
PPTX
Convolutional Neural Network - CNN | How CNN Works | Deep Learning Course | S...
bySimplilearn
 
PDF
Convolutional Neural Networks (CNN)
byGaurav Mittal
 
PPTX
Introduction to CNN
byShuai Zhang
 
PPTX
Convolution Neural Network (CNN)
byBasit Rafiq
 
PPTX
Convolutional neural network
byMojammilHusain
 
PPTX
CNN and its applications by ketaki
byKetaki Patwari
 
PPTX
Machine Learning - Convolutional Neural Network
byRichard Kuo
 
Convolutional Neural Network and Its Applications
byKasun Chinthaka Piyarathna
 
Convolutional Neural Network - CNN | How CNN Works | Deep Learning Course | S...
bySimplilearn
 
Convolutional Neural Networks (CNN)
byGaurav Mittal
 
Introduction to CNN
byShuai Zhang
 
Convolution Neural Network (CNN)
byBasit Rafiq
 
Convolutional neural network
byMojammilHusain
 
CNN and its applications by ketaki
byKetaki Patwari
 
Machine Learning - Convolutional Neural Network
byRichard Kuo
 

What's hot

PPTX
Cnn
byNirthika Rajendran
 
PDF
Convolutional Neural Network Models - Deep Learning
byMohamed Loey
 
PDF
Convolutional neural network
byYan Xu
 
PPTX
Convolution Neural Network (CNN)
bySuraj Aavula
 
PDF
Resnet
byashwinjoseph95
 
PDF
Deep Learning - Convolutional Neural Networks
byChristian Perone
 
PPTX
Activation functions
byPRATEEK SAHU
 
PPTX
What Is Deep Learning? | Introduction to Deep Learning | Deep Learning Tutori...
bySimplilearn
 
PDF
Deep Feed Forward Neural Networks and Regularization
byYan Xu
 
PPTX
Deep Learning - CNN and RNN
byAshray Bhandare
 
PPTX
Artifical Neural Network and its applications
bySangeeta Tiwari
 
PPTX
Activation function
byAstha Jain
 
PPTX
Image classification using cnn
bySumeraHangi
 
PPTX
Introduction to Deep Learning
byOswald Campesato
 
PPTX
Convolutional Neural Network (CNN)
byMuhammad Haroon
 
PPTX
AlexNet
byBertil Hatt
 
PPTX
Convolutional neural network
byFerdous ahmed
 
PDF
Transfer Learning: An overview
byjins0618
 
PPTX
cnn ppt.pptx
byrohithprabhas1
 
PPTX
Batch normalization presentation
byOwin Will
 
Cnn
byNirthika Rajendran
 
Convolutional Neural Network Models - Deep Learning
byMohamed Loey
 
Convolutional neural network
byYan Xu
 
Convolution Neural Network (CNN)
bySuraj Aavula
 
Resnet
byashwinjoseph95
 
Deep Learning - Convolutional Neural Networks
byChristian Perone
 
Activation functions
byPRATEEK SAHU
 
What Is Deep Learning? | Introduction to Deep Learning | Deep Learning Tutori...
bySimplilearn
 
Deep Feed Forward Neural Networks and Regularization
byYan Xu
 
Deep Learning - CNN and RNN
byAshray Bhandare
 
Artifical Neural Network and its applications
bySangeeta Tiwari
 
Activation function
byAstha Jain
 
Image classification using cnn
bySumeraHangi
 
Introduction to Deep Learning
byOswald Campesato
 
Convolutional Neural Network (CNN)
byMuhammad Haroon
 
AlexNet
byBertil Hatt
 
Convolutional neural network
byFerdous ahmed
 
Transfer Learning: An overview
byjins0618
 
cnn ppt.pptx
byrohithprabhas1
 
Batch normalization presentation
byOwin Will
 

Similar to Convolutional Neural Networks

PDF
CNN Basics.pdf
byNiharikaThakur32
 
PDF
Faire de la reconnaissance d'images avec le Deep Learning - Cristina & Pierre...
byJedha Bootcamp
 
PPTX
Your First Deep Learning project With CNN (workshop).pptx
byMuhammad Fahad Bashir
 
PDF
lecture16.pdf000000000000000000000000000
bythed3942
 
PPTX
Mnist report ppt
byRaghunandanJairam
 
PDF
_AI_Stanford_Super_#DeepLearning_Cheat_Sheet!_😊🙃😀🙃😊.pdf
bySongsDrizzle
 
PDF
super-cheatsheet-deep-learning.pdf
byDeanSchoolofElectron
 
PDF
Mnist report
byRaghunandanJairam
 
PPTX
DEEP LEARNING UNIT 2 all about deep learning
bypavanforsocials
 
PPTX
Convolutional Neural Network - Computer Vision.pptx
bykmpslab
 
PPTX
Convolutional Neural Networks
bymilad abbasi
 
PPTX
Cnn
byMehrnaz Faraz
 
PPTX
build a Convolutional Neural Network (CNN) using TensorFlow in Python
byKv Sagar
 
PDF
Overview of Convolutional Neural Networks
byananth
 
PDF
Convolutional_neural_network mechanism.pptx.pdf
bySwathiSoman5
 
PPTX
Introduction to Neural Networks and Deep Learning
byVahid Mirjalili
 
PPTX
Convolutional Neural Network (CNN)of Deep Learning
byalihassaah1994
 
PPTX
CNN 2 for 2025-2026 Advanced AI lecture...
byMohammedshaker50
 
PPTX
Deep Learning
byPierre de Lacaze
 
PDF
dfdshofdifhdifhdfhgfoighfgofgfgfgfgdfdfdfdf
bynguyenhoangy207
 
CNN Basics.pdf
byNiharikaThakur32
 
Faire de la reconnaissance d'images avec le Deep Learning - Cristina & Pierre...
byJedha Bootcamp
 
Your First Deep Learning project With CNN (workshop).pptx
byMuhammad Fahad Bashir
 
lecture16.pdf000000000000000000000000000
bythed3942
 
Mnist report ppt
byRaghunandanJairam
 
_AI_Stanford_Super_#DeepLearning_Cheat_Sheet!_😊🙃😀🙃😊.pdf
bySongsDrizzle
 
super-cheatsheet-deep-learning.pdf
byDeanSchoolofElectron
 
Mnist report
byRaghunandanJairam
 
DEEP LEARNING UNIT 2 all about deep learning
bypavanforsocials
 
Convolutional Neural Network - Computer Vision.pptx
bykmpslab
 
Convolutional Neural Networks
bymilad abbasi
 
Cnn
byMehrnaz Faraz
 
build a Convolutional Neural Network (CNN) using TensorFlow in Python
byKv Sagar
 
Overview of Convolutional Neural Networks
byananth
 
Convolutional_neural_network mechanism.pptx.pdf
bySwathiSoman5
 
Introduction to Neural Networks and Deep Learning
byVahid Mirjalili
 
Convolutional Neural Network (CNN)of Deep Learning
byalihassaah1994
 
CNN 2 for 2025-2026 Advanced AI lecture...
byMohammedshaker50
 
Deep Learning
byPierre de Lacaze
 
dfdshofdifhdifhdfhgfoighfgofgfgfgfgdfdfdfdf
bynguyenhoangy207
 

Recently uploaded

PDF
Columbus Data and Analytics Wednesday - People Analytics 101: Making Sense of...
byTim Wilson
 
DOCX
Data Analyst Training Course | Advanced Data Analytics Certification | GVT Ac...
byGVT Academy
 
PDF
chapter one introduction to Probability theory.pdf
bySeid42
 
PPTX
Organizational Structure and Design Topic 4
bydavenieralayaay
 
PPTX
Social Media Audit of Starbucks Coffee Company.pptx
bydiviap21
 
PDF
Unlocking the Future of Logistics & Supply Chain
byKunal Suri
 
PPTX
End to End Supply Chain Management- Analytics Case Study.
byNandhuNandha1
 
PPT
Microsoft_SQL_ServerConsolidationTWP.ppt
byssuser8c3398
 
PDF
US Digital Fan Engagement Index 2026: Benchmarking Online Demand for Sports T...
byHyperset Group Ltd
 
PDF
Trustworthy AI : Governance of AI through Ethics
byKarim Baïna
 
PPTX
Top Institute for Data Analyst Course in Navi Mumbai
byPatil sagar
 
PDF
LECTURE - Overcoming the AI Failure Rate - AIG, DQM.pdf
bySami Laine
 
PDF
Understanding Data Analytics: Concepts, Types, and Use Cases
byThe IoT Academy
 
PDF
Analysis of Union Budget 2026-27: Major Announcements and their Impact
byDeeshiPavecha
 
PPTX
The Singularity Sprint (2026): Neuralink + the AI Arms Race — Ridwan Sassman
byridwansassman
 
PDF
Khan Traders Fish Meal – MSDS Material Safety Data Sheet
byKhan Traders Karachi
 
PDF
PathRAG rag pipeline PathRAG rag pipeline .pdf
byhotslix023
 
PPTX
Lecture 14- Introduction to Hypothesis Testing.pptx
bygbadamosiayomide
 
PPT
data collection tool ppt.ppttttdataadfttg
byMuhammadodil
 
PDF
Step by Step Guide to Buying a Old or Aged Verified Paxful Accounts in US.pdf
byTopSelleriTDotCom
 
Columbus Data and Analytics Wednesday - People Analytics 101: Making Sense of...
byTim Wilson
 
Data Analyst Training Course | Advanced Data Analytics Certification | GVT Ac...
byGVT Academy
 
chapter one introduction to Probability theory.pdf
bySeid42
 
Organizational Structure and Design Topic 4
bydavenieralayaay
 
Social Media Audit of Starbucks Coffee Company.pptx
bydiviap21
 
Unlocking the Future of Logistics & Supply Chain
byKunal Suri
 
End to End Supply Chain Management- Analytics Case Study.
byNandhuNandha1
 
Microsoft_SQL_ServerConsolidationTWP.ppt
byssuser8c3398
 
US Digital Fan Engagement Index 2026: Benchmarking Online Demand for Sports T...
byHyperset Group Ltd
 
Trustworthy AI : Governance of AI through Ethics
byKarim Baïna
 
Top Institute for Data Analyst Course in Navi Mumbai
byPatil sagar
 
LECTURE - Overcoming the AI Failure Rate - AIG, DQM.pdf
bySami Laine
 
Understanding Data Analytics: Concepts, Types, and Use Cases
byThe IoT Academy
 
Analysis of Union Budget 2026-27: Major Announcements and their Impact
byDeeshiPavecha
 
The Singularity Sprint (2026): Neuralink + the AI Arms Race — Ridwan Sassman
byridwansassman
 
Khan Traders Fish Meal – MSDS Material Safety Data Sheet
byKhan Traders Karachi
 
PathRAG rag pipeline PathRAG rag pipeline .pdf
byhotslix023
 
Lecture 14- Introduction to Hypothesis Testing.pptx
bygbadamosiayomide
 
data collection tool ppt.ppttttdataadfttg
byMuhammadodil
 
Step by Step Guide to Buying a Old or Aged Verified Paxful Accounts in US.pdf
byTopSelleriTDotCom
 

Convolutional Neural Networks

  • 1.
  • 2.
    Page  2 Agenda Introduction  Archetecture Overview – How ConvNet Works  ConvNet Layers – Convolutional Layer – Pooling Layer – Normalization Layer (ReLU) – Fully-Connected Layer  Demos & Parameters – Hyper Parameters – Mnist dataset on AWS – CIFAR-10 ConvNetJS  Case Studies EECS6980:006 Social Network Analysis
  • 3.
    Page  3 Agenda Introduction  Archetecture Overview – How ConvNet Works  ConvNet Layers – Convolutional Layer – Pooling Layer – Normalization Layer (ReLU) – Fully-Connected Layer  Demos & Parameters – Hyper Parameters – Mnist dataset on AWS – CIFAR-10 ConvNetJS  Case Studies EECS6980:006 Social Network Analysis
  • 4.
    Page  4 Introduction A convolutional neural network (or ConvNet) is a type of feed-forward artificial neural network  The architecture of a ConvNet is designed to take advantage of the 2D structure of an input image.    A ConvNet is comprised of one or more convolutional layers (often with a pooling step) and then followed by one or more fully connected layers as in a standard multilayer neural network. EECS6980:006 Social Network Analysis VS
  • 5.
    Page  5 Agenda Introduction  Archetecture Overview – How ConvNet Works  ConvNet Layers – Convolutional Layer – Pooling Layer – Normalization Layer (ReLU) – Fully-Connected Layer  Demos & Parameters – Hyper Parameters – Mnist dataset on AWS – CIFAR-10 ConvNetJS  Case Studies EECS6980:006 Social Network Analysis
  • 6.
    Page  6 Motivationbehind ConvNets  Consider an image of size 200x200x3 (200 wide, 200 high, 3 color channels) – a single fully-connected neuron in a first hidden layer of a regular Neural Network would have 200*200*3 = 120,000 weights. – Due to the presence of several such neurons, this full connectivity is wasteful and the huge number of parameters would quickly lead to overfitting  However, in a ConvNet, the neurons in a layer will only be connected to a small region of the layer before it, instead of all of the neurons in a fully-connected manner. – the final output layer would have dimensions 1x1xN, because by the end of the ConvNet architecture we will reduce the full image into a single vector of class scores (for N classes), arranged along the depth dimension EECS6980:006 Social Network Analysis
  • 7.
    Page  7 MLPVS ConvNet EECS6980:006 Social Network Analysis Input Hidden Output Input Hidden Output Multilayered Perceptron: All Fully Connected Layers Convolutional Neural Network With Partially Connected Convolution Layer
  • 8.
    Page  8 MLPvs ConvNet A regular 3-layer Neural Network. A ConvNet arranges its neurons in three dimensions (width, height, depth), as visualized in one of the layers. EECS6980:006 Social Network Analysis
  • 9.
    Page  9 Agenda Introduction  Archetecture Overview – How ConvNet Works  ConvNet Layers – Convolutional Layer – Pooling Layer – Normalization Layer (ReLU) – Fully-Connected Layer  Demos & Parameters – Hyper Parameters – Mnist dataset on AWS – CIFAR-10 ConvNetJS  Case Studies EECS6980:006 Social Network Analysis
  • 10.
    Page  10 HowConvNet Works  For example, a ConvNet takes the input as an image which can be classified as ‘X’ or ‘O’  In a simple case, ‘X’ would look like: X or OCNN A two-dimensional array of pixels
  • 11.
    Page  11 HowConvNet Works  What about trickier cases? CNN X CNN O
  • 12.
    Page  12 -1-1 -1 -1 -1 -1 -1 -1 -1 -1 1 -1 -1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 -1 -1 1 -1 1 -1 -1 -1 -1 -1 -1 -1 1 -1 -1 -1 -1 -1 -1 -1 1 -1 1 -1 -1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 -1 -1 1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 -1 -1 1 1 -1 1 -1 -1 -1 -1 -1 -1 -1 1 -1 -1 -1 -1 -1 -1 -1 1 -1 1 1 -1 -1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 = ? How ConvNet Works – What Computer Sees
  • 13.
    Page  13 -1-1 -1 -1 -1 -1 -1 -1 -1 -1 1 -1 -1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 -1 -1 1 -1 1 -1 -1 -1 -1 -1 -1 -1 1 -1 -1 -1 -1 -1 -1 -1 1 -1 1 -1 -1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 -1 -1 1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 -1 -1 1 1 -1 1 -1 -1 -1 -1 -1 -1 -1 1 -1 -1 -1 -1 -1 -1 -1 1 -1 1 1 -1 -1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 =x How ConvNet Works
  • 14.
    Page  14 -1-1 -1 -1 -1 -1 -1 -1 -1 -1 X -1 -1 -1 -1 X X -1 -1 X X -1 -1 X X -1 -1 -1 -1 X 1 -1 1 -1 -1 -1 -1 -1 -1 -1 1 -1 -1 -1 -1 -1 -1 -1 1 -1 1 X -1 -1 -1 -1 X X -1 -1 X X -1 -1 X X -1 -1 -1 -1 X -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 How ConvNet Works – What Computer Sees  Since the pattern doesnot match exactly, the computer will not be able to classify this as ‘X’
  • 15.
    Page  15 Agenda Introduction  Archetecture Overview – How ConvNet Works  ConvNet Layers – Convolutional Layer – Pooling Layer – Normalization Layer (ReLU) – Fully-Connected Layer  Demos & Parameters – Hyper Parameters – Mnist dataset on AWS – CIFAR-10 ConvNetJS  Case Studies EECS6980:006 Social Network Analysis
  • 16.
    Page  16 ConvNetLayers (At a Glance)  CONV layer will compute the output of neurons that are connected to local regions in the input, each computing a dot product between their weights and a small region they are connected to in the input volume.  RELU layer will apply an elementwise activation function, such as the max(0,x) thresholding at zero. This leaves the size of the volume unchanged.  POOL layer will perform a downsampling operation along the spatial dimensions (width, height).  FC (i.e. fully-connected) layer will compute the class scores, resulting in volume of size [1x1xN], where each of the N numbers correspond to a class score, such as among the N categories. EECS6980:006 Social Network Analysis
  • 17.
    Page  17 Agenda Introduction  Archetecture Overview – How ConvNet Works  ConvNet Layers – Convolutional Layer – Pooling Layer – Normalization Layer (ReLU) – Fully-Connected Layer  Demos & Parameters – Hyper Parameters – Mnist dataset on AWS – CIFAR-10 ConvNetJS  Case Studies EECS6980:006 Social Network Analysis
  • 18.
    Page  18 -1-1 -1 -1 -1 -1 -1 -1 -1 -1 X -1 -1 -1 -1 X X -1 -1 X X -1 -1 X X -1 -1 -1 -1 X 1 -1 1 -1 -1 -1 -1 -1 -1 -1 1 -1 -1 -1 -1 -1 -1 -1 1 -1 1 X -1 -1 -1 -1 X X -1 -1 X X -1 -1 X X -1 -1 -1 -1 X -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 Recall – What Computer Sees  Since the pattern doesnot match exactly, the computer will not be able to classify this as ‘X’  What got changed?
  • 19.
    Page  19 = = = Convolution layer will work to identify patterns (features) instead of individual pixels Convolutional Layer
  • 20.
    Page  20 1-1 -1 -1 1 -1 -1 -1 1 -1 -1 1 -1 1 -1 1 -1 -1 1 -1 1 -1 1 -1 1 -1 1 Convolutional Layer - Filters  The CONV layer’s parameters consist of a set of learnable filters.  Every filter is small spatially (along width and height), but extends through the full depth of the input volume.  During the forward pass, we slide (more precisely, convolve) each filter across the width and height of the input volume and compute dot products between the entries of the filter and the input at any position.
  • 21.
    Page  21 1-1 -1 -1 1 -1 -1 -1 1 -1 -1 1 -1 1 -1 1 -1 -1 1 -1 1 -1 1 -1 1 -1 1 Convolutional Layer - Filters  Sliding the filter over the width and height of the input gives 2-dimensional activation map that responds to that filter at every spatial position.
  • 22.
    Page  22 ConvolutionalLayer - Strides Input Hidden Output Input Hidden Output 1-D convolution Filters: 1 Filter Size: 2 Stride: 2 1-D convolution Filters: 1 Filter Size: 2 Stride: 1 • The distance that filter is moved across the input from the previous layer each activation is referred to as the stride.
  • 23.
    Page  23 ConvolutionalLayer - Padding  Sometimes it is convenient to pad the input volume with zeros around the border.  Zero padding is allows us to preserve the spatial size of the output volumes EECS6980:006 Social Network Analysis Input Hidden Output 1 D convolution with: Filters: 1 Filter Size: 2 Stride: 2 Padding: 1 0 0 Input Hidden Output 1 D convolution with: Filters: 2 Filter Size: 2 Stride: 2 Padding: 1 0 0
  • 24.
    Page  24 1-1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 1 -1 -1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 -1 -1 1 -1 1 -1 -1 -1 -1 -1 -1 -1 1 -1 -1 -1 -1 -1 -1 -1 1 -1 1 -1 -1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 -1 -1 1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 Convolutional Layer – Filters – Navigation Example  Strides = 1, Filter Size = 3 X 3 X 1, Padding = 0
  • 25.
    Page  25 1-1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 1 -1 -1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 -1 -1 1 -1 1 -1 -1 -1 -1 -1 -1 -1 1 -1 -1 -1 -1 -1 -1 -1 1 -1 1 -1 -1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 -1 -1 1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 Convolutional Layer – Filters – Navigation Example
  • 26.
    Page  26 1-1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 1 -1 -1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 -1 -1 1 -1 1 -1 -1 -1 -1 -1 -1 -1 1 -1 -1 -1 -1 -1 -1 -1 1 -1 1 -1 -1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 -1 -1 1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 Convolutional Layer – Filters – Navigation Example
  • 27.
    Page  27 1-1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 1 -1 -1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 -1 -1 1 -1 1 -1 -1 -1 -1 -1 -1 -1 1 -1 -1 -1 -1 -1 -1 -1 1 -1 1 -1 -1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 -1 -1 1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 Convolutional Layer – Filters – Navigation Example
  • 28.
    Page  28 1-1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 1 -1 -1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 -1 -1 1 -1 1 -1 -1 -1 -1 -1 -1 -1 1 -1 -1 -1 -1 -1 -1 -1 1 -1 1 -1 -1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 -1 -1 1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 Convolutional Layer – Filters – Navigation Example
  • 29.
    Page  29 1-1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 1 -1 -1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 -1 -1 1 -1 1 -1 -1 -1 -1 -1 -1 -1 1 -1 -1 -1 -1 -1 -1 -1 1 -1 1 -1 -1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 -1 -1 1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 Convolutional Layer – Filters – Navigation Example
  • 30.
    Page  30 1-1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 1 -1 -1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 -1 -1 1 -1 1 -1 -1 -1 -1 -1 -1 -1 1 -1 -1 -1 -1 -1 -1 -1 1 -1 1 -1 -1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 -1 -1 1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 Convolutional Layer – Filters – Navigation Example
  • 31.
    Page  31 1-1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 1 -1 -1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 -1 -1 1 -1 1 -1 -1 -1 -1 -1 -1 -1 1 -1 -1 -1 -1 -1 -1 -1 1 -1 1 -1 -1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 -1 -1 1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 Convolutional Layer – Filters – Navigation Example
  • 32.
    Page  32 1-1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 1 -1 -1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 -1 -1 1 -1 1 -1 -1 -1 -1 -1 -1 -1 1 -1 -1 -1 -1 -1 -1 -1 1 -1 1 -1 -1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 -1 -1 1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 Convolutional Layer – Filters – Navigation Example
  • 33.
    Page  33 1-1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 1 -1 -1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 -1 -1 1 -1 1 -1 -1 -1 -1 -1 -1 -1 1 -1 -1 -1 -1 -1 -1 -1 1 -1 1 -1 -1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 -1 -1 1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 Convolutional Layer – Filters – Computation Example
  • 34.
    Page  34 1 1-1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 1 -1 -1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 -1 -1 1 -1 1 -1 -1 -1 -1 -1 -1 -1 1 -1 -1 -1 -1 -1 -1 -1 1 -1 1 -1 -1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 -1 -1 1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 Convolutional Layer – Filters – Computation Example
  • 35.
    Page  35 11 1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 1 -1 -1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 -1 -1 1 -1 1 -1 -1 -1 -1 -1 -1 -1 1 -1 -1 -1 -1 -1 -1 -1 1 -1 1 -1 -1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 -1 -1 1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 Convolutional Layer – Filters – Computation Example
  • 36.
    Page  36 11 1 1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 1 -1 -1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 -1 -1 1 -1 1 -1 -1 -1 -1 -1 -1 -1 1 -1 -1 -1 -1 -1 -1 -1 1 -1 1 -1 -1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 -1 -1 1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 Convolutional Layer – Filters – Computation Example
  • 37.
    Page  37 11 1 1 1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 1 -1 -1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 -1 -1 1 -1 1 -1 -1 -1 -1 -1 -1 -1 1 -1 -1 -1 -1 -1 -1 -1 1 -1 1 -1 -1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 -1 -1 1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 Convolutional Layer – Filters – Computation Example
  • 38.
    Page  38 11 1 1 1 1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 1 -1 -1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 -1 -1 1 -1 1 -1 -1 -1 -1 -1 -1 -1 1 -1 -1 -1 -1 -1 -1 -1 1 -1 1 -1 -1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 -1 -1 1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 Convolutional Layer – Filters – Computation Example
  • 39.
    Page  39 11 1 1 1 1 1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 1 -1 -1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 -1 -1 1 -1 1 -1 -1 -1 -1 -1 -1 -1 1 -1 -1 -1 -1 -1 -1 -1 1 -1 1 -1 -1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 -1 -1 1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 Convolutional Layer – Filters – Computation Example
  • 40.
    Page  40 11 1 1 1 1 1 1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 1 -1 -1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 -1 -1 1 -1 1 -1 -1 -1 -1 -1 -1 -1 1 -1 -1 -1 -1 -1 -1 -1 1 -1 1 -1 -1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 -1 -1 1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 Convolutional Layer – Filters – Computation Example
  • 41.
    Page  41 11 1 1 1 1 1 1 1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 1 -1 -1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 -1 -1 1 -1 1 -1 -1 -1 -1 -1 -1 -1 1 -1 -1 -1 -1 -1 -1 -1 1 -1 1 -1 -1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 -1 -1 1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 Convolutional Layer – Filters – Computation Example
  • 42.
    Page  42 11 1 1 1 1 1 1 1 1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 1 -1 -1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 -1 -1 1 -1 1 -1 -1 -1 -1 -1 -1 -1 1 -1 -1 -1 -1 -1 -1 -1 1 -1 1 -1 -1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 -1 -1 1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 Convolutional Layer – Filters – Computation Example
  • 43.
    Page  43 1 11 1 1 1 1 1 1 1 1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 1 -1 -1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 -1 -1 1 -1 1 -1 -1 -1 -1 -1 -1 -1 1 -1 -1 -1 -1 -1 -1 -1 1 -1 1 -1 -1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 -1 -1 1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 Convolutional Layer – Filters – Computation Example
  • 44.
    Page  44 1 1-1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 1 -1 -1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 -1 -1 1 -1 1 -1 -1 -1 -1 -1 -1 -1 1 -1 -1 -1 -1 -1 -1 -1 1 -1 1 -1 -1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 -1 -1 1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 Convolutional Layer – Filters – Computation Example
  • 45.
    Page  45 11 -1 1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 1 -1 -1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 -1 -1 1 -1 1 -1 -1 -1 -1 -1 -1 -1 1 -1 -1 -1 -1 -1 -1 -1 1 -1 1 -1 -1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 -1 -1 1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 Convolutional Layer – Filters – Computation Example
  • 46.
    Page  46 11 -1 1 1 1 -1 1 1 1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 1 -1 -1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 -1 -1 1 -1 1 -1 -1 -1 -1 -1 -1 -1 1 -1 -1 -1 -1 -1 -1 -1 1 -1 1 -1 -1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 -1 -1 1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 Convolutional Layer – Filters – Computation Example
  • 47.
    Page  47 1 1-1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 1 -1 -1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 -1 -1 1 -1 1 -1 -1 -1 -1 -1 -1 -1 1 -1 -1 -1 -1 -1 -1 -1 1 -1 1 -1 -1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 -1 -1 1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 1 1 -1 1 1 1 -1 1 1 55 1 1 -1 1 1 1 -1 1 1 Convolutional Layer – Filters – Computation Example
  • 48.
    Page  48 1-1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 1 -1 -1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 -1 -1 1 -1 1 -1 -1 -1 -1 -1 -1 -1 1 -1 -1 -1 -1 -1 -1 -1 1 -1 1 -1 -1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 -1 -1 1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 = 0.77 -0.11 0.11 0.33 0.55 -0.11 0.33 -0.11 1.00 -0.11 0.33 -0.11 0.11 -0.11 0.11 -0.11 1.00 -0.33 0.11 -0.11 0.55 0.33 0.33 -0.33 0.55 -0.33 0.33 0.33 0.55 -0.11 0.11 -0.33 1.00 -0.11 0.11 -0.11 0.11 -0.11 0.33 -0.11 1.00 -0.11 0.33 -0.11 0.55 0.33 0.11 -0.11 0.77 Convolutional Layer – Filters – Computation Example Input Size (W): 9 Filter Size (F): 3 X 3 Stride (S): 1 Filters: 1 Padding (P): 0 9 X 9 7 X 7 Feature Map Size = 1+ (W – F + 2P)/S = 1+ (9 – 3 + 2 X 0)/1 = 7
  • 49.
    Page  49 1-1 -1 -1 1 -1 -1 -1 1 0.33 -0.11 0.55 0.33 0.11 -0.11 0.77 -0.11 0.11 -0.11 0.33 -0.11 1.00 -0.11 0.55 -0.11 0.11 -0.33 1.00 -0.11 0.11 0.33 0.33 -0.33 0.55 -0.33 0.33 0.33 0.11 -0.11 1.00 -0.33 0.11 -0.11 0.55 -0.11 1.00 -0.11 0.33 -0.11 0.11 -0.11 0.77 -0.11 0.11 0.33 0.55 -0.11 0.33 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 1 -1 -1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 -1 -1 1 -1 1 -1 -1 -1 -1 -1 -1 -1 1 -1 -1 -1 -1 -1 -1 -1 1 -1 1 -1 -1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 -1 -1 1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 = 0.77 -0.11 0.11 0.33 0.55 -0.11 0.33 -0.11 1.00 -0.11 0.33 -0.11 0.11 -0.11 0.11 -0.11 1.00 -0.33 0.11 -0.11 0.55 0.33 0.33 -0.33 0.55 -0.33 0.33 0.33 0.55 -0.11 0.11 -0.33 1.00 -0.11 0.11 -0.11 0.11 -0.11 0.33 -0.11 1.00 -0.11 0.33 -0.11 0.55 0.33 0.11 -0.11 0.77 -1 -1 1 -1 1 -1 1 -1 -1 1 -1 1 -1 1 -1 1 -1 1 0.33 -0.55 0.11 -0.11 0.11 -0.55 0.33 -0.55 0.55 -0.55 0.33 -0.55 0.55 -0.55 0.11 -0.55 0.55 -0.77 0.55 -0.55 0.11 -0.11 0.33 -0.77 1.00 -0.77 0.33 -0.11 0.11 -0.55 0.55 -0.77 0.55 -0.55 0.11 -0.55 0.55 -0.55 0.33 -0.55 0.55 -0.55 0.33 -0.55 0.11 -0.11 0.11 -0.55 0.33 = = -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 1 -1 -1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 -1 -1 1 -1 1 -1 -1 -1 -1 -1 -1 -1 1 -1 -1 -1 -1 -1 -1 -1 1 -1 1 -1 -1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 -1 -1 1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 1 -1 -1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 -1 -1 1 -1 1 -1 -1 -1 -1 -1 -1 -1 1 -1 -1 -1 -1 -1 -1 -1 1 -1 1 -1 -1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 -1 -1 1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 Convolutional Layer – Filters – Output Feature Map  Output Feature Map of One complete convolution: – Filters: 3 – Filter Size: 3 X 3 – Stride: 1  Conclusion: – Input Image: 9 X 9 – Output of Convolution: 7 X 7 X 3
  • 50.
    Page  50 0.33-0.11 0.55 0.33 0.11 -0.11 0.77 -0.11 0.11 -0.11 0.33 -0.11 1.00 -0.11 0.55 -0.11 0.11 -0.33 1.00 -0.11 0.11 0.33 0.33 -0.33 0.55 -0.33 0.33 0.33 0.11 -0.11 1.00 -0.33 0.11 -0.11 0.55 -0.11 1.00 -0.11 0.33 -0.11 0.11 -0.11 0.77 -0.11 0.11 0.33 0.55 -0.11 0.33 0.77 -0.11 0.11 0.33 0.55 -0.11 0.33 -0.11 1.00 -0.11 0.33 -0.11 0.11 -0.11 0.11 -0.11 1.00 -0.33 0.11 -0.11 0.55 0.33 0.33 -0.33 0.55 -0.33 0.33 0.33 0.55 -0.11 0.11 -0.33 1.00 -0.11 0.11 -0.11 0.11 -0.11 0.33 -0.11 1.00 -0.11 0.33 -0.11 0.55 0.33 0.11 -0.11 0.77 0.33 -0.55 0.11 -0.11 0.11 -0.55 0.33 -0.55 0.55 -0.55 0.33 -0.55 0.55 -0.55 0.11 -0.55 0.55 -0.77 0.55 -0.55 0.11 -0.11 0.33 -0.77 1.00 -0.77 0.33 -0.11 0.11 -0.55 0.55 -0.77 0.55 -0.55 0.11 -0.55 0.55 -0.55 0.33 -0.55 0.55 -0.55 0.33 -0.55 0.11 -0.11 0.11 -0.55 0.33 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 1 -1 -1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 -1 -1 1 -1 1 -1 -1 -1 -1 -1 -1 -1 1 -1 -1 -1 -1 -1 -1 -1 1 -1 1 -1 -1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 -1 -1 1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 Convolutional Layer – Output
  • 51.
    Page  51 Agenda Introduction  Archetecture Overview – How ConvNet Works  ConvNet Layers – Convolutional Layer – Pooling Layer – Normalization Layer (ReLU) – Fully-Connected Layer  Demos & Parameters – Hyper Parameters – Mnist dataset on AWS – CIFAR-10 ConvNetJS  Case Studies EECS6980:006 Social Network Analysis
  • 52.
    Page  52 RectifiedLinear Units (ReLUs) 0.77 -0.11 0.11 0.33 0.55 -0.11 0.33 -0.11 1.00 -0.11 0.33 -0.11 0.11 -0.11 0.11 -0.11 1.00 -0.33 0.11 -0.11 0.55 0.33 0.33 -0.33 0.55 -0.33 0.33 0.33 0.55 -0.11 0.11 -0.33 1.00 -0.11 0.11 -0.11 0.11 -0.11 0.33 -0.11 1.00 -0.11 0.33 -0.11 0.55 0.33 0.11 -0.11 0.77 0.77
  • 53.
    Page  53 0.770 Rectified Linear Units (ReLUs) 0.77 -0.11 0.11 0.33 0.55 -0.11 0.33 -0.11 1.00 -0.11 0.33 -0.11 0.11 -0.11 0.11 -0.11 1.00 -0.33 0.11 -0.11 0.55 0.33 0.33 -0.33 0.55 -0.33 0.33 0.33 0.55 -0.11 0.11 -0.33 1.00 -0.11 0.11 -0.11 0.11 -0.11 0.33 -0.11 1.00 -0.11 0.33 -0.11 0.55 0.33 0.11 -0.11 0.77
  • 54.
    Page  54 0.770 0.11 0.33 0.55 0 0.33 Rectified Linear Units (ReLUs) 0.77 -0.11 0.11 0.33 0.55 -0.11 0.33 -0.11 1.00 -0.11 0.33 -0.11 0.11 -0.11 0.11 -0.11 1.00 -0.33 0.11 -0.11 0.55 0.33 0.33 -0.33 0.55 -0.33 0.33 0.33 0.55 -0.11 0.11 -0.33 1.00 -0.11 0.11 -0.11 0.11 -0.11 0.33 -0.11 1.00 -0.11 0.33 -0.11 0.55 0.33 0.11 -0.11 0.77
  • 55.
    Page  55 0.770 0.11 0.33 0.55 0 0.33 0 1.00 0 0.33 0 0.11 0 0.11 0 1.00 0 0.11 0 0.55 0.33 0.33 0 0.55 0 0.33 0.33 0.55 0 0.11 0 1.00 0 0.11 0 0.11 0 0.33 0 1.00 0 0.33 0 0.55 0.33 0.11 0 0.77 Rectified Linear Units (ReLUs) 0.77 -0.11 0.11 0.33 0.55 -0.11 0.33 -0.11 1.00 -0.11 0.33 -0.11 0.11 -0.11 0.11 -0.11 1.00 -0.33 0.11 -0.11 0.55 0.33 0.33 -0.33 0.55 -0.33 0.33 0.33 0.55 -0.11 0.11 -0.33 1.00 -0.11 0.11 -0.11 0.11 -0.11 0.33 -0.11 1.00 -0.11 0.33 -0.11 0.55 0.33 0.11 -0.11 0.77
  • 56.
    Page  56 ReLUlayer 0.77 0 0.11 0.33 0.55 0 0.33 0 1.00 0 0.33 0 0.11 0 0.11 0 1.00 0 0.11 0 0.55 0.33 0.33 0 0.55 0 0.33 0.33 0.55 0 0.11 0 1.00 0 0.11 0 0.11 0 0.33 0 1.00 0 0.33 0 0.55 0.33 0.11 0 0.77 0.33 0 0.11 0 0.11 0 0.33 0 0.55 0 0.33 0 0.55 0 0.11 0 0.55 0 0.55 0 0.11 0 0.33 0 1.00 0 0.33 0 0.11 0 0.55 0 0.55 0 0.11 0 0.55 0 0.33 0 0.55 0 0.33 0 0.11 0 0.11 0 0.33 0.33 0 0.55 0.33 0.11 0 0.77 0 0.11 0 0.33 0 1.00 0 0.55 0 0.11 0 1.00 0 0.11 0.33 0.33 0 0.55 0 0.33 0.33 0.11 0 1.00 0 0.11 0 0.55 0 1.00 0 0.33 0 0.11 0 0.77 0 0.11 0.33 0.55 0 0.33 0.33 -0.11 0.55 0.33 0.11 -0.11 0.77 -0.11 0.11 -0.11 0.33 -0.11 1.00 -0.11 0.55 -0.11 0.11 -0.33 1.00 -0.11 0.11 0.33 0.33 -0.33 0.55 -0.33 0.33 0.33 0.11 -0.11 1.00 -0.33 0.11 -0.11 0.55 -0.11 1.00 -0.11 0.33 -0.11 0.11 -0.11 0.77 -0.11 0.11 0.33 0.55 -0.11 0.33 0.77 -0.11 0.11 0.33 0.55 -0.11 0.33 -0.11 1.00 -0.11 0.33 -0.11 0.11 -0.11 0.11 -0.11 1.00 -0.33 0.11 -0.11 0.55 0.33 0.33 -0.33 0.55 -0.33 0.33 0.33 0.55 -0.11 0.11 -0.33 1.00 -0.11 0.11 -0.11 0.11 -0.11 0.33 -0.11 1.00 -0.11 0.33 -0.11 0.55 0.33 0.11 -0.11 0.77 0.33 -0.55 0.11 -0.11 0.11 -0.55 0.33 -0.55 0.55 -0.55 0.33 -0.55 0.55 -0.55 0.11 -0.55 0.55 -0.77 0.55 -0.55 0.11 -0.11 0.33 -0.77 1.00 -0.77 0.33 -0.11 0.11 -0.55 0.55 -0.77 0.55 -0.55 0.11 -0.55 0.55 -0.55 0.33 -0.55 0.55 -0.55 0.33 -0.55 0.11 -0.11 0.11 -0.55 0.33
  • 57.
    Page  57 Agenda Introduction  Archetecture Overview – How ConvNet Works  ConvNet Layers – Convolutional Layer – Pooling Layer – Normalization Layer (ReLU) – Fully-Connected Layer  Demos & Parameters – Hyper Parameters – Mnist dataset on AWS – CIFAR-10 ConvNetJS  Case Studies EECS6980:006 Social Network Analysis
  • 58.
    Page  58 PoolingLayer  The pooling layers down-sample the previous layers feature map.  Its function is to progressively reduce the spatial size of the representation to reduce the amount of parameters and computation in the network  The pooling layer often uses the Max operation to perform the downsampling process EECS6980:006 Social Network Analysis
  • 59.
    Page  59 1.00 Pooling Pooling Filter Size = 2 X 2, Stride = 2
  • 60.
    Page  60 1.000.33 Pooling  Pooling Filter Size = 2 X 2, Stride = 2
  • 61.
    Page  61 1.000.33 0.55 Pooling  Pooling Filter Size = 2 X 2, Stride = 2
  • 62.
    Page  62 1.000.33 0.55 0.33 Pooling  Pooling Filter Size = 2 X 2, Stride = 2
  • 63.
    Page  63 1.000.33 0.55 0.33 0.33 Pooling  Pooling Filter Size = 2 X 2, Stride = 2
  • 64.
    Page  64 1.000.33 0.55 0.33 0.33 1.00 0.33 0.55 0.55 0.33 1.00 0.11 0.33 0.55 0.11 0.77 Pooling  Pooling Filter Size = 2 X 2, Stride = 2
  • 65.
    Page  65 1.000.33 0.55 0.33 0.33 1.00 0.33 0.55 0.55 0.33 1.00 0.11 0.33 0.55 0.11 0.77 0.33 0.55 1.00 0.77 0.55 0.55 1.00 0.33 1.00 1.00 0.11 0.55 0.77 0.33 0.55 0.33 0.55 0.33 0.55 0.33 0.33 1.00 0.55 0.11 0.55 0.55 0.55 0.11 0.33 0.11 0.11 0.33 Pooling
  • 66.
    Page  66 Layersget stacked -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 1 -1 -1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 -1 -1 1 -1 1 -1 -1 -1 -1 -1 -1 -1 1 -1 -1 -1 -1 -1 -1 -1 1 -1 1 -1 -1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 -1 -1 1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 1.00 0.33 0.55 0.33 0.33 1.00 0.33 0.55 0.55 0.33 1.00 0.11 0.33 0.55 0.11 0.77 0.33 0.55 1.00 0.77 0.55 0.55 1.00 0.33 1.00 1.00 0.11 0.55 0.77 0.33 0.55 0.33 0.55 0.33 0.55 0.33 0.33 1.00 0.55 0.11 0.55 0.55 0.55 0.11 0.33 0.11 0.11 0.33
  • 67.
    Page  67 LayersGet Stacked - Example 224 X 224 X 3 224 X 224 X 64 CONVOLUTION WITH 64 FILTERS 112 X 112 X 64 POOLING (DOWNSAMPLING)
  • 68.
    Page  68 Deepstacking -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 1 -1 -1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 -1 -1 1 -1 1 -1 -1 -1 -1 -1 -1 -1 1 -1 -1 -1 -1 -1 -1 -1 1 -1 1 -1 -1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 -1 -1 1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 1.00 0.55 0.55 1.00 0.55 1.00 1.00 0.55 1.00 0.55 0.55 0.55
  • 69.
    Page  69 Agenda Introduction  Archetecture Overview – How ConvNet Works  ConvNet Layers – Convolutional Layer – Pooling Layer – Normalization Layer (ReLU) – Fully-Connected Layer  Demos & Parameters – Hyper Parameters – Mnist dataset on AWS – CIFAR-10 ConvNetJS  Case Studies EECS6980:006 Social Network Analysis
  • 70.
    Page  70 Fullyconnected layer  Fully connected layers are the normal flat feed-forward neural network layers.  These layers may have a non-linear activation function or a softmax activation in order to predict classes.  To compute our output, we simply re- arrange the output matrices as a 1-D array. 1.00 0.55 0.55 1.00 0.55 1.00 1.00 0.55 1.00 0.55 0.55 0.55 1.00 0.55 0.55 1.00 1.00 0.55 0.55 0.55 0.55 1.00 1.00 0.55
  • 71.
    Page  71 Fullyconnected layer  A summation of product of inputs and weights at each output node determines the final prediction X O 0.55 1.00 1.00 0.55 0.55 0.55 0.55 0.55 1.00 0.55 0.55 1.00
  • 72.
    Page  72 Puttingit all together -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 1 -1 -1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 -1 -1 1 -1 1 -1 -1 -1 -1 -1 -1 -1 1 -1 -1 -1 -1 -1 -1 -1 1 -1 1 -1 -1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 -1 -1 1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 X O
  • 73.
    Page  73 Agenda Introduction  Archetecture Overview – How ConvNet Works  ConvNet Layers – Convolutional Layer – Pooling Layer – Normalization Layer (ReLU) – Fully-Connected Layer  Parameters & Demos – Hyper Parameters – Mnist dataset on AWS – CIFAR-10 ConvNetJS  Case Studies EECS6980:006 Social Network Analysis
  • 74.
    Page  74 Hyperparameters(knobs)  Convolution – Filter Size – Number of Filters – Padding – Stride  Pooling – Window Size – Stride  Fully Connected – Number of neurons
  • 75.
    Page  75 Agenda Introduction  Archetecture Overview – How ConvNet Works  ConvNet Layers – Convolutional Layer – Pooling Layer – Normalization Layer (ReLU) – Fully-Connected Layer  Parameters & Demos – Hyper Parameters – Mnist dataset on AWS – CIFAR-10 ConvNetJS  Case Studies EECS6980:006 Social Network Analysis
  • 76.
    Page  76 Agenda Introduction  Archetecture Overview – How ConvNet Works  ConvNet Layers – Convolutional Layer – Pooling Layer – Normalization Layer (ReLU) – Fully-Connected Layer  Parameters & Demos – Hyper Parameters – Mnist dataset on AWS – CIFAR-10 ConvNetJS  Case Studies EECS6980:006 Social Network Analysis
  • 77.
    Page  77 CaseStudies  LeNet – 1998  AlexNet – 2012  ZFNet – 2013  VGG – 2014  GoogLeNet – 2014  ResNet – 2015 EECS6980:006 Social Network Analysis
  • 78.
    Page  78 References Karpathy, A. (n.d.). CS231n Convolutional Neural Networks for Visual Recognition. Retrieved from http://cs231n.github.io/convolutional-networks/#overview  Rohrer, B. (n.d.). How do Convolutional Neural Networks work?. Retrieved from http://brohrer.github.io/how_convolutional_neural_networks_work.html  Brownlee, J. (n.d.). Crash Course in Convolutional Neural Networks for Machine Learning. Retrieved from http://machinelearningmastery.com/crash-course-convolutional-neural-networks/  Lidinwise (n.d.). The revolution of depth. Retrieved from https://medium.com/@Lidinwise/the-revolution-of- depth-facf174924f5#.8or5c77ss  Nervana. (n.d.). Tutorial: Convolutional neural networks. Retrieved from https://www.nervanasys.com/convolutional-neural-networks/ EECS6980:006 Social Network Analysis
  • 79.
    Page  79 Questions EECS6980:006Social Network Analysis
  • 80.
    Page  80 Thankyou!! EECS6980:006 Social Network Analysis