TensorFlow

1. Jiří Materna
Head of Research, Seznam.cz
TensorFlow

2. What is not TensorFlow

3. What is TensorFlow?

4. The MNIST database

5. Perceptron classifier for
MNIST 1/2

6. Perceptron classifier for
MNIST 2/2
import tensorflow as tf
x = tf.placeholder(tf.float32, [None, 784])
W = tf.Variable(tf.zeros([784, 10]))
b = tf.Variable(tf.zeros([10]))
y = tf.nn.softmax(tf.matmul(x, W) + b)
y_ = tf.placeholder(tf.float32, [None, 10])
cross_entropy = -tf.reduce_sum(y_*tf.log(y))
train_step =
tf.train.GradientDescentOptimizer(0.01).minimize(cross_entropy)
for i in range(1000):
batch_xs, batch_ys = mnist.train.next_batch(100)
sess.run(train_step, feed_dict={x: batch_xs, y_: batch_ys})

7. Convolutional Neural Nets

8. Convolution in image
recognition

9. Convolution for MNIST
#Convolutional Layer
W_conv1 = weight_variable([5, 5, 1, 32])
b_conv1 = bias_variable([32])
h_conv1 = tf.nn.relu(conv2d(x_image, W_conv1) + b_conv1)
h_pool1 = max_pool_2x2(h_conv1)
#Densely Connected Layer
W_fc1 = weight_variable([12 * 12 * 64, 1024])
b_fc1 = bias_variable([1024])
h_pool1_flat = tf.reshape(h_pool1, [-1, 12*12*64])
h_fc1 = tf.nn.relu(tf.matmul(h_pool1_flat, W_fc1) + b_fc1)
#Dropout
keep_prob = tf.placeholder("float")
h_fc1_drop = tf.nn.dropout(h_fc1, keep_prob)

10. Adult content recognition
https://www.youtube.com/watch?v=XGxXIoZxODg

11. Recurrent Neural Networks

12. Language Modeling with
LSTM
lstm = rnn_cell.BasicLSTMCell(lstm_size)
stacked_lstm = rnn_cell.MultiRNNCell([lstm] * number_of_layers)
# Initial state of the LSTM memory.
state = tf.zeros([batch_size, lstm_size])
loss = 0.0
for current_batch_of_chars in dataset:
# State is updated after processing each batch of chars.
output, state = stacked_lstm(current_batch_of_chars, state)
# The LSTM output can be used to make next word predictions
logits = tf.matmul(output, softmax_w) + softmax_b
probabilities = tf.nn.softmax(logits)
loss += loss_function(probabilities, target_words)

13. Generating poems

14. Too complex?

15. Try Keras
from keras.models import Sequential
from keras.layers.core import Dense, Dropout, Activation, Flatten
from keras.layers.convolutional import Convolution2D, MaxPooling2D
from keras.optimizers import SGD
model = Sequential()
model.add(Convolution2D(32, 5, 5, border_mode=‘same',
input_shape=(1, 28, 28)))
model.add(Activation('relu'))
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Dense(1024))
model.add(Flatten())
model.add(Activation('relu'))
model.add(Dropout(0.5))
model.add(Activation('softmax'))
sgd = SGD(lr=0.1)
model.compile(loss='categorical_crossentropy', optimizer=sgd)

16. Still too complex?

17. Time to change your job

18. Invitation
www.mlprague.com
twitter: @mlprague.com
Facebook: https://www.facebook.com/mlprague

19. Jiří Materna
Twitter: @JiriMaterna
www.mlguru.cz