This document discusses convolutional neural networks (CNNs) for image recognition. It explains key CNN components like convolutional layers, pooling layers, hyperparameters like kernel size and stride. It provides code in TensorFlow to recognize handwritten digits from the MNIST dataset using a CNN model with convolutional and pooling layers. The code trains the model on MNIST data and evaluates test accuracy.

1. 합성곱 신경망
딥러닝을 위한 신경망 기초
nonezerok@gmail.com
영상학습에 절대강자인 CNN을 다룹니다.
향후 자신의 신경망 설계를 위해 꼭 필요한 기반 요소 및 관련 하이퍼 파라미터를 설명합니다.
MNIST 필기체 숫자영상 인식을 위한 Tensorflow 코드를 통해 하이퍼 파라미터 설정을 연습합니다.

2. 2
CNN in Practice for MNIST
fully-connected
10-neuron
output layer
(softmax)
http://neuralnetworksanddeeplearning.com/index.html
5X5 커널 20개 사용
각각 1개 층

3. 3
AlexNet (2012)
https://ml4a.github.io/ml4a/convnets/
더 복잡하게 구성할 수 있다.
stride
padding커널 크기, 커널 개수, 합성곱 층, 풀링 층 구성
커널 크기: 3X3

4. 4
ImageNet (http://www.image-net.org/) Data
https://www.youtube.com/watch?v=40riCqvRoMs (Fei Fei Li)

5. 5
1D 2D+1D
통합구조

6. 6
depth, stride, zero-padding
convolution layer

7. 7
https://tensorflow.blog/a-guide-to-convolution-arithmetic-for-deep-learning/

8. 8
convolution with filter 3x3, stride 2, pad 1

9. 99
a 7x7 input and a 3x3 filter with stride 1 and pad 0
we would get a 5x5 output
( −  + 2)

+ 1
 =
 − 1
2
,  = 1 → input volume = output volume

10. 10
https://github.com/vdumoulin/conv_arithmetic

11. 11
 = 2,  = 3,  = 2,  = 1 → 3 × 3
http://cs231n.github.io/convolutional-networks/ (animation)

12. 12
Robust to Spatial Location
pooling layer

13. 13
max pooling with filter 2x2, stride 2, pad 0

14. 14
max pooling with filter 3x3, stride 1, pad 0

15. 15
import tensorflow as tf
import matplotlib.pyplot as plt
from tensorflow.examples.tutorials.mnist import input_data
mnist = input_data.read_data_sets("MNIST_data/", one_hot=True)
x = tf.placeholder("float", shape=[None, 784])
y = tf.placeholder("float", shape=[None, 10])
x_image = tf.reshape(x, [-1, 28, 28, 1])
W_conv1 = tf.Variable(tf.truncated_normal([5,5,1,32], stddev=0.1))
b_conv1 = tf.Variable(tf.constant(0.1, shape=[32]))
[code in Tensorflow]
https://tensorflow.blog

16. 16
conv1 = tf.nn.relu(tf.nn.conv2d(x_image, W_conv1, strides=[1, 1, 1, 1],
padding='SAME') + b_conv1)
pool1 = tf.nn.max_pool(conv1, ksize=[1, 2, 2, 1], strides=[1, 2, 2, 1],
padding='SAME')
'''
conv1 = tf.layers.conv2d(x_image, 32, (5,5), strides=(1,1),
padding="same", activation=tf.nn.relu)
pool1 = tf.layers.max_pooling2d(conv1, pool_size=(2,2), strides=(2,2))
'''
conv2 = tf.layers.conv2d(pool1, 64, (5,5), strides=(1,1), padding="same",
activation=tf.nn.relu)
pool2 = tf.layers.max_pooling2d(conv2, pool_size=(2,2), strides=(2,2))
pool2_flat = tf.reshape(pool2, [-1, 7*7*64])
fc = tf.layers.dense(pool2_flat, 1024, activation=tf.nn.relu)
drop_prob = tf.placeholder("float")
fc_drop = tf.layers.dropout(fc, rate=drop_prob)
z = tf.layers.dense(fc_drop, 10)
y_hat = tf.nn.softmax(z)

17. 17
loss = tf.losses.softmax_cross_entropy(y, y_hat)
optimizer = tf.train.GradientDescentOptimizer(0.1)
train = optimizer.minimize(loss)
correct_prediction = tf.equal(tf.argmax(y, 1), tf.argmax(y_hat, 1))
accuracy = tf.reduce_mean(tf.cast(correct_prediction, "float"))
sess = tf.Session()
sess.run(tf.global_variables_initializer())

18. 18
costs = []
for i in range(200):
x_data, y_data = mnist.train.next_batch(100)
_, cost = sess.run([train, loss],
feed_dict={x:x_data, y:y_data, drop_prob:0.5})
if i%100 == 0:
'''training_accuracy = sess.run(accuracy,
feed_dict={x:x_data, y:y_data, drop_prob: 0.1})
print("%d, training accuracy: %g"%(i, training_accuracy))'''
print("%05d, cost: %g"%(i, cost))
costs.append(cost)
plt.plot(costs)
plt.show()
test_accuracy = sess.run(accuracy,
feed_dict={x:mnist.test.images[:100], y:mnist.test.labels[:100],
drop_prob: 1.0})
print("test accuracy: %g"%(test_accuracy))

19. 19
''' filter for the first convolution layer'''
filters = sess.run(W_conv1)
for i in range(32):
plt.subplot(4, 8, i+1)
filter = filters[:, :, 0, i]
plt.imshow(filter, interpolation="nearest", cmap="gray")
plt.show()