SlideShare a Scribd company logo
1 of 98
Introduction to TensorFlow 2
Scala Meetup (PA) 06/27/2019
Oswald Campesato
ocampesato@yahoo.com
Highlights/Overview
 What is TensorFlow 2?
 Major Changes in TF 2
 Working with strings/arrays/tensors
 Working with TF 2 @tf.function decorator
 Working with TF 2 generators
 Working with TF 2 tf.data.Dataset
 Datasets in TF 1.x versus TF 2
 Working with TF 2 tf.keras
 CNNs, RNNs, LSTMs, Bidirectional LSTMs
 Reinforcement Learning/TF Agents
Highlights/Overview
This session does not cover:
 “the new vision” of TF 2
 “the common practices” of TF 2
 “lessons learned” from TF 1.x
 a detailed “roadmap” from TF 1.x to TF 2
What is TensorFlow 2?
An open source framework for ML and DL
Created by Google (TF 1.x released 11/2015)
Evolved from Google Brain
Linux and Mac OS X support (VM for Windows)
TF home page: https://www.tensorflow.org/
Status of TensorFlow 2
Alpha release: around March/2019
Other “Nightly” builds:
https://pypi.org/project/tf-nightly-2.0-preview/2.0.0.dev20190417/#files
Production release: TBD (later this year)
TF 2 Platform Support
Tested and supported on 64-bit systems
Ubuntu 16.04 or later
Windows 7 or later
macOS 10.12.6 (Sierra) or later (no GPU support)
Raspbian 9.0 or later
TF 2 Docker Images
 https://hub.docker.com/r/tensorflow/tensorflow/
$ docker run -it --rm tensorflow/tensorflow bash
 Start a CPU-only container with Python 2:
$ docker run -it --rm --runtime=nvidia tensorflow/tensorflow:latest-
gpu-py3 python
 Start a GPU container with Python 3 & Python interpreter:
$ docker run -it --rm -v $(realpath ~/notebooks):/tf/notebooks -p
8888:8888 tensorflow/tensorflow:late
Status of TensorFlow 1.x
1.13.1: current stable release
1.14: latest release
After TF 2 is in production:
Only security-related updates to TF 1.x
TF 1.x support for one year after TF 2 production
What is TensorFlow 2?
 Support for Python, Java, C++
 Desktop, Server, Mobile, Web
 CPU/GPU/TPU support
 Visualization via TensorBoard
 Can be embedded in Python scripts
 Installation: pip install tensorflow
 Ex: pip install tensorflow==2.0.0-alpha0
TensorFlow Use Cases (Generic)
Image recognition
Computer vision
Voice/sound recognition
Time series analysis
Language detection
Language translation
Text-based processing
Handwriting Recognition
Major Changes in TF 2
 TF 2 Eager Execution: default mode
 @tf.function decorator: instead of tf.Session()
 AutoGraph: graph generated in @tf.function()
 Generators: used with tf.data.Dataset (TF 2)
 Differentiation: calculates gradients
 tf.GradientTape: automatic differentiation
Removed from TensorFlow 2
 tf.Session()
 tf.placeholder()
 tf.global_initializer()
 feed_dict
 Variable scopes
 tf.contrib code
 tf.flags and tf.contrib
 Global variables
 => TF 1.x functions moved to compat.v1
Upgrading Files/Directories to TF 2
 The TF 2 upgrade script:
https://www.tensorflow.org/alpha/guide/upgrade
 Script name: tf_upgrade_v2
 install TF to get the upgrade script:
pip install tensorflow (NOT pip3)
Upgrading Files/Directories to TF 2
1) upgrade one file:
tf_upgrade_v2 --infile oldtf.py --outfile newtf.py
 The preceding command creates report.txt
Upgrade an entire directory:
tf_upgrade_v2 –intree mydir --outtree newtf.py –
copyotherfiles False
NB: do NOT make manual upgrade changes
Migrating to TensorFlow 2
 1) do not manually update parts of your code
 2) script won't work with functions with reordered arguments
 3) script assumes this TF statement: "import tensorflow as tf"
 4) the script does not reorder arguments
 5) the script adds keyword arguments to functions that have their
arguments reordered.
 Upgrade Jupyter notebooks and Python files in a GitHub repo:
http://tf2up.ml/
=> Replace prefix https://github.com/ with http://tf2up.m
What is a Tensor?
TF tensors are n-dimensional arrays
TF tensors are very similar to numpy ndarrays
scalar number: a zeroth-order tensor
vector: a first-order tensor
matrix: a second-order tensor
3-dimensional array: a 3rd order tensor
https://dzone.com/articles/tensorflow-simplified-
examples
TensorFlow Eager Execution
An imperative interface to TF
Fast debugging & immediate run-time errors
=> TF 2 requires Python 3.x (not Python 2.x)
No static graphs or sessions
TensorFlow Eager Execution
integration with Python tools
Supports dynamic models + Python control flow
support for custom and higher-order gradients
=> Default mode in TensorFlow 2.0
TensorFlow Eager Execution
 import tensorflow as tf
 x = [[2.]]
 # m = tf.matmul(x, x) <= deprecated
 m = tf.linalg.matmul(x, x)
 print("m:",m)
 print("m:",m.numpy())
#Output:
#m: tf.Tensor([[4.]], shape=(1,1),dtype=float32)
#m: [[4.]]
Basic TF 2 Operations
Working with Constants
Working with Strings
Working with Tensors
Working Operators
Arrays in TF 2
Multiplying Two Arrays
Convert Python Arrays to TF Arrays
Conflicting Types
TensorFlow “primitive types”
 tf.constant:
> initialized immediately and immutable
 import tensorflow as tf
 aconst = tf.constant(3.0)
 print(aconst)
# output for TF 2:
#tf.Tensor(3.0, shape=(), dtype=float32)
# output for TF 1.x:
#Tensor("Const:0", shape=(), dtype=float32)
TensorFlow “primitive types”
tf.Variable (a class):
+ initial value is required
+ updated during training
+ in-memory buffer (saved/restored from disk)
+ can be shared in a distributed environment
+ they hold learned parameters of a model
TensorFlow Arithmetic
import tensorflow as tf # arith1.py
a = tf.add(4, 2)
b = tf.subtract(8, 6)
c = tf.multiply(a, 3)
d = tf.div(a, 6)
 print(a) # 6
 print(b) # 2
 print(c) # 18
 print(d) # 1
 print("a:",a.numpy())
 print("b:",b.numpy())
 print("c:",c.numpy())
 print("d:",d.numpy())
TensorFlow Arithmetic
 tf.Tensor(6, shape=(), dtype=int32)
 tf.Tensor(2, shape=(), dtype=int32)
 tf.Tensor(18, shape=(), dtype=int32)
 tf.Tensor(1.0, shape=(), dtype=float64)
 6
 2
 18
 1.0
TF 2 Loops and Arrays
Using “for” loops
Using “while” loops
tf.reduce_prod()
tf.reduce_sum()
A “for” Loop Example
 import tensorflow as tf
 x = tf.Variable(0, name='x')
 for i in range(5):
 x = x + 1
 print("x:",x)
A “for” Loop Example
 x: tf.Tensor(1, shape=(), dtype=int32)
 x: tf.Tensor(2, shape=(), dtype=int32)
 x: tf.Tensor(3, shape=(), dtype=int32)
 x: tf.Tensor(4, shape=(), dtype=int32)
 x: tf.Tensor(5, shape=(), dtype=int32)
A “while” Loop Example
 import tensorflow as tf
 a = tf.constant(12)
 while not tf.equal(a, 1):
 if tf.equal(a % 2, 0):
 a = a / 2
 else:
 a = 3 * a + 1
 print(a)
A “while” Loop Example
 tf.Tensor(6.0, shape=(), dtype=float64)
 tf.Tensor(3.0, shape=(), dtype=float64)
 tf.Tensor(10.0, shape=(), dtype=float64)
 tf.Tensor(5.0, shape=(), dtype=float64)
 tf.Tensor(16.0, shape=(), dtype=float64)
 tf.Tensor(8.0, shape=(), dtype=float64)
 tf.Tensor(4.0, shape=(), dtype=float64)
 tf.Tensor(2.0, shape=(), dtype=float64)
 tf.Tensor(1.0, shape=(), dtype=float64)
Useful TF 2 APIs
tf.shape()
tf.rank()
tf.range()
tf.reshape()
tf.ones()
tf.zeros()
tf.fill()
 tf.random.normal()
 tf.truncated_normal()
Some TF 2 “lazy operators”
map()
filter()
flatmap()
batch()
take()
zip()
flatten()
Combined via “method chaining”
TF 2 “lazy operators”
 filter():
 uses Boolean logic to "filter" the elements in an array to
determine which elements satisfy the Boolean condition
 map(): a projection
 this operator "applies" a lambda expression to each input
element
 flat_map():
 maps a single element of the input dataset to a Dataset of
elements
TF 2 “lazy operators”
 batch(n):
 processes a "batch" of n elements during each
iteration
 repeat(n):
 repeats its input values n times
 take(n):
 operator "takes" n input values
TF 2 tf.data.Dataset
TF “wrapper” class around data sources
Located in the tf.data.Dataset namespace
Supports lambda expressions
Supports lazy operators
They use generators instead of iterators (TF 1.x)
tf.data.Dataset.from_tensors()
 Import tensorflow as tf
 #combine the input into one element
 t1 = tf.constant([[1, 2], [3, 4]])
 ds1 = tf.data.Dataset.from_tensors(t1)
# output: [[1, 2], [3, 4]]
tf.data.Dataset.from_tensor_slices()
 Import tensorflow as tf
 #separate element for each item
 t2 = tf.constant([[1, 2], [3, 4]])
 ds1 = tf.data.Dataset.from_tensor_slices(t2)
# output: [1, 2], [3, 4]
TF 2 Datasets: code sample
 import tensorflow as tf # tf2-dataset.py
 import numpy as np
x = np.arange(0, 10)
 # create a dataset from a Numpy array
ds = tf.data.Dataset.from_tensor_slices(x)
TF 1.x Datasets: iterator
 import tensorflow as tf # tf1x-plusone.py
 import numpy as np
 x = np.arange(0, 10)
 # create dataset object from Numpy array
 ds = tf.data.Dataset.from_tensor_slices(x)
 ds.map(lambda x: x + 1)
 # create a one-shot iterator <= TF 1.x
 iterator = ds.make_one_shot_iterator()
 for i in range(10):
 val = iterator.get_next()
 print("val:",val)
TF 2 Datasets: generator
import tensorflow as tf # tf2-plusone.py
import numpy as np
x = np.arange(0, 5) # 0, 1, 2, 3, 4
def gener():
for i in x:
yield (3*i)
ds = tf.data.Dataset.from_generator(gener, (tf.int64))
for value in ds.take(len(x)):
print("1value:",value)
for value in ds.take(2*len(x)):
print("2value:",value)
TF 2 Datasets: generator
 1value: tf.Tensor(0, shape=(), dtype=int64)
 1value: tf.Tensor(3, shape=(), dtype=int64)
 1value: tf.Tensor(6, shape=(), dtype=int64)
 1value: tf.Tensor(9, shape=(), dtype=int64)
 1value: tf.Tensor(12, shape=(), dtype=int64)
 2value: tf.Tensor(0, shape=(), dtype=int64)
 2value: tf.Tensor(3, shape=(), dtype=int64)
 2value: tf.Tensor(6, shape=(), dtype=int64)
 2value: tf.Tensor(9, shape=(), dtype=int64)
 2value: tf.Tensor(12, shape=(), dtype=int64)
TF 1.x map() and take()
 import tensorflow as tf # tf 1.12.0
 tf.enable_eager_execution()
 ds = tf.data.TextLineDataset("file.txt")
 ds = ds.map(lambda line:
tf.string_split([line]).values)
 try:
 for value in ds.take(2):
 print("value:",value)
 except tf.errors.OutOfRangeError:
 pass
TF 2 map() and take(): file.txt
 this is file line #1
 this is file line #2
 this is file line #3
 this is file line #4
 this is file line #5
TF 2 map() and take(): output
 ('value:', <tf.Tensor: id=16, shape=(5,),
dtype=string, numpy=array(['this', 'is',
'file', 'line', '#1'], dtype=object)>)
 ('value:', <tf.Tensor: id=18, shape=(5,),
dtype=string, numpy=array(['this', 'is',
'file', 'line', '#2'], dtype=object)>)
TF 2 map() and take()
 import tensorflow as tf
 import numpy as np
 x = np.array([[1],[2],[3],[4]])
 # make a ds from a numpy array
 ds = tf.data.Dataset.from_tensor_slices(x)
 ds = ds.map(lambda x: x*2).map(lambda x:
x+1).map(lambda x: x**3)
 for value in ds.take(4):
 print("value:",value)
TF 2 map() and take(): output
 value: tf.Tensor([27], shape=(1,), dtype=int64)
 value: tf.Tensor([125], shape=(1,), dtype=int64)
 value: tf.Tensor([343], shape=(1,), dtype=int64)
 value: tf.Tensor([729], shape=(1,), dtype=int64)
TF 2 batch(): EXTRA CREDIT
import tensorflow as tf
import numpy as np
x = np.arange(0, 12)
def gener():
i = 0
while(i < len(x/3)):
yield (i, i+1, i+2) # three integers at a time
i += 3
ds = tf.data.Dataset.from_generator(gener, (tf.int64,tf.int64,tf.int64))
third = int(len(x)/3)
for value in ds.take(third):
print("value:",value)
TF 2 batch() & zip(): EXTRA CREDIT
import tensorflow as tf
ds1 = tf.data.Dataset.range(100)
ds2 = tf.data.Dataset.range(0, -100, -1)
ds3 = tf.data.Dataset.zip((ds1, ds2))
ds4 = ds3.batch(4)
for value in ds.take(10):
print("value:",value)
TF 2: Working with tf.keras
tf.keras.layers
tf.keras.models
tf.keras.optimizers
tf.keras.utils
tf.keras.regularizers
TF 2: tf.keras.models
 tf.keras.models.Sequential(): most common
 clone_model(...): Clone any Model instance
 load_model(...): Loads a model saved via save_model
 model_from_config(...): Instantiates a Keras model from its
config
 model_from_json(...):
 Parses a JSON model config file and returns a model instance
 model_from_yaml(...):
 Parses a yaml model config file and returns a model instance
 save_model(...): Saves a model to a HDF5 file
TF 2: tf.keras.layers
 tf.keras.models.Sequential()
 tf.keras.layers.Conv2D()
 tf.keras.layers.MaxPooling2D()
 tf.keras.layers.Flatten()
 tf.keras.layers.Dense()
 tf.keras.layers.Dropout(0.1)
 tf.keras.layers.BatchNormalization()
 tf.keras.layers.embedding()
 tf.keras.layers.RNN()
 tf.keras.layers.LSTM()
 tf.keras.layers.Bidirectional (ex: BERT)
TF 2: Activation Functions
tf.keras.activations.relu
tf.keras.activations.selu
tf.keras.activations.elu
tf.keras.activations.linear
tf.keras.activations.sigmoid
tf.keras.activations.softmax
tf.keras.activations.softplus
tf.keras.activations.tanh
Others …
TF 2: Built-in Datasets
tf.keras.datasets.boston_housing
tf.keras.datasets.cifar10
tf.keras.datasets.cifar100
tf.keras.datasets.fashion_mnist
tf.keras.datasets.imdb
tf.keras.datasets.mnist
tf.keras.datasets.reuters
TF 2: “Experimental”
 tf.keras.experimental.CosineDecay
 tf.keras.experimental.CosineDecayRestarts
 tf.keras.experimental.LinearCosineDecay
 tf.keras.experimental.NoisyLinearCosineDecay
 tf.keras.experimental.PeepholeLSTMCell
TF 2: Optimizers
tf.optimizers.Adadelta
tf.optimizers.Adagrad
tf.optimizers.Adam
tf.optimizers.Adamax
tf.optimizers.SGD
TF 2; Callbacks
tf.keras.callbacks.Callback
tf.keras.callbacks.EarlyStopping
tf.keras.callbacks.LambaCallback
tf.keras.callbacks.ModelCheckpoint
tf.keras.callbacks.TensorBoard
tf.keras.callbacks.TerminateOnNan
TF 2: tf.losses
 tf.losses.BinaryCrossEntropy
 tf.losses.CategoricalCrossEntropy
 tf.losses.CategoricalHinge
 tf.losses.CosineSimilarity
 tf.losses.Hinge
 tf.losses.Huber
 tf.losses.KLD
 tf.losses.KLDivergence
 tf.losses.MAE
 tf.losses.MSE
 tf.losses.SparseCategoricalCrossentropy
Other TF 2 Namespaces
tf.keras.regularizers (L1 and L2)
tf.keras.utils (to_categorical)
tf.keras.preprocessing.text.Tokenizer
TF 1.x Model APIs
 tf.core (“reduced” in TF 2)
 tf.layers (deprecated in TF 2)
 tf.keras (the primary API in TF 2)
 tf.estimator.Estimator (implementation in tf.keras)
 tf.contrib.learn.Estimator (Deprecated in TF 2)
Other TF 2 Features
TF Probability (Linear Regression)
TF Agents (Reinforcement Learning)
TF Text (NLP)
TF Federated
TF Privacy
Tensor2Tensor
TF Ranking
TFX
TF 2 (Keras), DL, and RL
The remaining slides briefly discuss TF 2 and:
 CNNs (Convolutional Neural Networks)
 RNNs (Recurrent Neural Networks)
 LSTMs (Long Short Term Memory)
 Autoencoders
 Variational Autoencoders
 Reinforcement Learning
 Some Keras-based code blocks
 Some useful links
Neural Network: 3 Hidden Layers
Transition Between Layers
W = weight matrix between adjacent layers
x = an input vector of numbers
B = a bias vector of numbers
F = activation function (sigmoid, tanh, etc)
Y = output vector of numbers
Y = F(x*W + b)
TF 2/Keras and MLPs
 import tensorflow as tf
 . . .
 model = tf.keras.models.Sequential()
 model.add(tf.keras.layers.Dense(10, input_dim=num_pixels,
activation='relu'))
 model.add(tf.keras.layers.Dense(30, activation='relu'))
 model.add(tf.keras.layers.Dense(10, activation='relu'))
 model.add(tf.keras.layers.Dense(num_classes,
activation='softmax'))
 model.compile(tf.keras.optimizers.Adam(lr=0.01),
loss='categorical_crossentropy', metrics=['accuracy'])
TF 2/Keras and MLPs
 import tensorflow as tf
 . . .
 model = tf.keras.models.Sequential([
tf.keras.layers.Dense(10, input_dim=num_pixels, activation='relu’),
tf.keras.layers.Dense(30, activation='relu’),
tf.keras.layers.Dense(10, activation='relu’),
tf.keras.layers.Dense(num_classes, activation='softmax’),
tf.keras.optimizers.Adam(lr=0.01), loss='categorical_crossentropy',
metrics=['accuracy'])
])
CNNs: Convolution and Pooling
TF 2/Keras and CNNs
 import tensorflow as tf
 . . .
 model = tf.keras.models.Sequential([
 tf.keras. layers.Conv2D(30,(5,5),
input_shape=(28,28,1),activation='relu')),
 tf.keras.layers.MaxPooling2D(pool_size=(2, 2))),
 tf.keras.layers.Flatten(),
 tf.keras.layers.Dense(500, activation='relu’),
 tf.keras.layers.Dropout(0.5),
 tf.keras.layers.Dense(num_classes, activation='softmax’)
 ])
What are RNNs?
RNNs = Recurrent Neural Networks
RNNs capture/retain events in time
FF networks do not learn from the past
=> RNNs DO learn from the past
RNNs contain layers of recurrent neurons
Common Types of RNNs
1) Simple RNNs (minimal structure)
2) LSTMs (input, forget, output gates)
3) Gated Recurrent Unit (GRU)
NB: RNNs can be used with MNIST data
Some Use Cases for RNNs
1) Time Series Data
2) NLP (Natural Language Processing):
Processing documents
“analyzing” Trump’s tweets
3) RNNs and MNIST dataset
Recall: Feed Forward Structure
RNN Cell (left) and FF (right)
RNN “Big Picture” (cs231n)
RNN Neuron: same W for all steps
RNN and MNIST Dataset
Sample initialization values for RNN:
 n_steps = 28 # # of time steps
 n_inputs = 28 # number of rows
 n_neurons = 150 # 150 neurons in one layer
 n_outputs = 10 # 10 digit classes (0->9)
Remember that:
one layer = one memory cell
MNIST class count = size of output layer
TF 2/Keras and RNNs
import tensorflow as tf
...
model = tf.keras.models.Sequential()
model.add(tf.keras.layers.SimpleRNN(20,
input_shape=(input_length, input_dim),
return_sequences=False))
...
Problems with RNNs
lengthy training time
Unrolled RNN will be very deep
Propagating gradients through many layers
Prone to vanishing gradient
Prone to exploding gradient
What are LSTMs?
Long short-term memory (LSTM):
a model for the short-term memory
Contains more state than a “regular” RNN
which can last for a long period of time
can forget and remember things selectively
Hochreiter/Schmidhuber proposed in 1997
improved in 2000 by Felix Gers' team
set accuracy records in multiple apps
Features of LSTMs
Used in Google speech recognition + Alpha Go
they avoid the vanishing gradient problem
Can track 1000s of discrete time steps
Used by international competition winners
Use Cases for LSTMs
Connected handwriting recognition
Speech recognition
Forecasting
Anomaly detection
Pattern recognition
Advantages of LSTMs
well-suited to classify/process/predict time series
More powerful: increased memory (=state)
Can explicitly add long-term or short-term state
even with time lags of unknown size/duration between
events
The Primary LSTM Gates
Input, Forget, and Output gates (FIO)
The main operational block of LSTMs
an "information filter"
a multivariate input gate
some inputs are blocked
some inputs "go through"
"remember" necessary information
FIO gates use the sigmoid activation function
The cell state uses the tanh activation function
LSTM Gates and Cell State
 1) the input gate controls:
the extent to which a new value flows into the cell state
 2) the forget gate controls:
the extent to which a value remains in the cell state
 3) the output gate controls:
The portion of the cell state that’s part of the output
 4) the cell state maintains long-term memory
 => LSTMs store more memory than basic RNN cells
Common LSTM Variables
 # LSTM unrolled through 28 time steps (with MNIST):
 time_steps = 28
 # number of hidden LSTM units:
 num_units = 128
 # number of rows of 28 pixels:
 n_inputs = 28
 # number of pixels per row:
 input_size = 28
 # mnist has 10 classes (0-9):
 n_classes = 10
 # size of input batch:
 batch_size = 128
Basic LSTM Cells
LSTM Formulas
TF 2/Keras and LSTMs
 import tensorflow as tf
 . . .
model = tf.keras.models.Sequential()
model.add(tf.keras.layers.LSTMCell(6,batch_input_shape=(1,1,1),kernel_i
nitializer='ones',stateful=True))
model.add(tf.keras.layers.Dense(1))
. . .
=> LSTM versus LSTMCell:
https://stackoverflow.com/questions/48187283/whats-the-difference-
between-lstm-and-lstmcell
=> How to define a custom LSTM cell:
https://stackoverflow.com/questions/54231440/define-custom-lstm-cell-in-
keras
TF 2/Keras and LSTM/RNN/GRU
 import tensorflow as tf
 ...
 model = Sequential()
 model.add(tf.keras.layers.LSTM(64,
return_sequences=True, input_shape=(10, 64)))
 model.add(tf.keras.layers.SimpleRNN(32,
return_sequences=True))
 model.add(tf.keras.layers.GRU(10, ...)
 ...
TF 2/Keras & BiDirectional LSTMs
 import tensorflow as tf
 . . .
 model = Sequential()
 model.add(Bidirectional(LSTM(10,
return_sequences=True), input_shape=(5,10)))
 model.add(Bidirectional(LSTM(10)))
 model.add(Dense(5))
 model.add(Activation('softmax'))
 model.compile(loss='categorical_crossentropy',
optimizer='rmsprop')
Autoencoders
Variational Autoencoders (2013)
AEs, VAEs, and GANs
 https://jaan.io/what-is-variational-autoencoder-vae-tutorial/
 TensorFlow 2 and Autoencoders:
https://gist.github.com/AFAgarap/326af55e36be0529c507f1599f88c06e
 TensorFlow 2 and Variational Autoencoders:
 Convolutional Variational Autoencoder:
https://www.tensorflow.org/alpha/tutorials/generative/cvae
 Variational Autoencoders and GANS:
https://www.youtube.com/watch?v=KFX4apL7a4s
Reinforcement Learning
Reinforcement Learning
Reinforcement Learning
Deep Reinforcement Learning
Reinforcement Learning
 Dopamine on TensorFlow (Research Framework):
https://github.com/google/dopamine
 TF-Agents library for RL in TensorFlow:
https://github.com/tensorflow/agents
 Keras and Reinforcement Learning:
https://github.com/keras-rl/keras-rl
 OpenAI universe, DeepMind Lab, TensorForce
About Me: Recent Books
1)TensorFlow 2 Pocket Primer (2019)
2) TensorFlow 1.x Pocket Primer (2019)
3) Python for TensorFlow (2019)
4) C Programming Pocket Primer (2019)
5) RegEx Pocket Primer (2018)
6) Data Cleaning Pocket Primer (2018)
7) Angular Pocket Primer (2017)
8) Android Pocket Primer (2017)
9) CSS3 Pocket Primer (2016)
10) SVG Pocket Primer (2016)
11) Python Pocket Primer (2015)
12) D3 Pocket Primer (2015)
13) HTML5 Mobile Pocket Primer (2014)
What I do (Training)
Instructor at UCSC (Santa Clara):
Machine Learning (Python) (06/18/2019) 10 weeks
Deep Learning (Keras) (07/10/2019) 10 weeks
Deep Learning (Keras) (07/17/2019) 10 weeks
Machine Learning (Python) (09/09/2019) 10 weeks
DL, NLP, Deep RL (Adv Topics) (10/02/2019) 10 weeks
Deep Learning (Keras/TF 2) (10/04/2019) 10 weeks
UCSC link:
https://www.ucsc-extension.edu/certificate-program/offering/deep-
learning-and-artificial-intelligence-tensorflow

More Related Content

What's hot

Real Time Streaming Data with Kafka and TensorFlow (Yong Tang, MobileIron) Ka...
Real Time Streaming Data with Kafka and TensorFlow (Yong Tang, MobileIron) Ka...Real Time Streaming Data with Kafka and TensorFlow (Yong Tang, MobileIron) Ka...
Real Time Streaming Data with Kafka and TensorFlow (Yong Tang, MobileIron) Ka...confluent
 
Parquet Strata/Hadoop World, New York 2013
Parquet Strata/Hadoop World, New York 2013Parquet Strata/Hadoop World, New York 2013
Parquet Strata/Hadoop World, New York 2013Julien Le Dem
 
Text generation and_advanced_topics
Text generation and_advanced_topicsText generation and_advanced_topics
Text generation and_advanced_topicsankit_ppt
 
CPU Scheduling algorithms
CPU Scheduling algorithmsCPU Scheduling algorithms
CPU Scheduling algorithmsShanu Kumar
 
Introduction to Apache Flink
Introduction to Apache FlinkIntroduction to Apache Flink
Introduction to Apache Flinkdatamantra
 
pandas - Python Data Analysis
pandas - Python Data Analysispandas - Python Data Analysis
pandas - Python Data AnalysisAndrew Henshaw
 
Container Performance Analysis
Container Performance AnalysisContainer Performance Analysis
Container Performance AnalysisBrendan Gregg
 
Introduction to Python
Introduction to PythonIntroduction to Python
Introduction to PythonNowell Strite
 
Linux Profiling at Netflix
Linux Profiling at NetflixLinux Profiling at Netflix
Linux Profiling at NetflixBrendan Gregg
 
Natural language processing and transformer models
Natural language processing and transformer modelsNatural language processing and transformer models
Natural language processing and transformer modelsDing Li
 
Running High-Speed Serverless with nuclio
Running High-Speed Serverless with nuclioRunning High-Speed Serverless with nuclio
Running High-Speed Serverless with nuclioiguazio
 
Introduction to PyTorch
Introduction to PyTorchIntroduction to PyTorch
Introduction to PyTorchJun Young Park
 
Building Scalable, Highly Concurrent & Fault Tolerant Systems - Lessons Learned
Building Scalable, Highly Concurrent & Fault Tolerant Systems -  Lessons LearnedBuilding Scalable, Highly Concurrent & Fault Tolerant Systems -  Lessons Learned
Building Scalable, Highly Concurrent & Fault Tolerant Systems - Lessons LearnedJonas Bonér
 
Facebook architecture
Facebook architectureFacebook architecture
Facebook architecturemysqlops
 
Google's Pathways Language Model and Chain-of-Thought
Google's Pathways Language Model and Chain-of-ThoughtGoogle's Pathways Language Model and Chain-of-Thought
Google's Pathways Language Model and Chain-of-ThoughtVaclav1
 
tf.data: TensorFlow Input Pipeline
tf.data: TensorFlow Input Pipelinetf.data: TensorFlow Input Pipeline
tf.data: TensorFlow Input PipelineAlluxio, Inc.
 
A Tale of Three Deep Learning Frameworks: TensorFlow, Keras, & PyTorch with B...
A Tale of Three Deep Learning Frameworks: TensorFlow, Keras, & PyTorch with B...A Tale of Three Deep Learning Frameworks: TensorFlow, Keras, & PyTorch with B...
A Tale of Three Deep Learning Frameworks: TensorFlow, Keras, & PyTorch with B...Databricks
 

What's hot (20)

Real Time Streaming Data with Kafka and TensorFlow (Yong Tang, MobileIron) Ka...
Real Time Streaming Data with Kafka and TensorFlow (Yong Tang, MobileIron) Ka...Real Time Streaming Data with Kafka and TensorFlow (Yong Tang, MobileIron) Ka...
Real Time Streaming Data with Kafka and TensorFlow (Yong Tang, MobileIron) Ka...
 
Parquet Strata/Hadoop World, New York 2013
Parquet Strata/Hadoop World, New York 2013Parquet Strata/Hadoop World, New York 2013
Parquet Strata/Hadoop World, New York 2013
 
Text generation and_advanced_topics
Text generation and_advanced_topicsText generation and_advanced_topics
Text generation and_advanced_topics
 
CPU Scheduling algorithms
CPU Scheduling algorithmsCPU Scheduling algorithms
CPU Scheduling algorithms
 
Introduction to Apache Flink
Introduction to Apache FlinkIntroduction to Apache Flink
Introduction to Apache Flink
 
Apache Spark with Scala
Apache Spark with ScalaApache Spark with Scala
Apache Spark with Scala
 
pandas - Python Data Analysis
pandas - Python Data Analysispandas - Python Data Analysis
pandas - Python Data Analysis
 
Container Performance Analysis
Container Performance AnalysisContainer Performance Analysis
Container Performance Analysis
 
Introduction to Python
Introduction to PythonIntroduction to Python
Introduction to Python
 
Linux Profiling at Netflix
Linux Profiling at NetflixLinux Profiling at Netflix
Linux Profiling at Netflix
 
Data race
Data raceData race
Data race
 
Natural language processing and transformer models
Natural language processing and transformer modelsNatural language processing and transformer models
Natural language processing and transformer models
 
Running High-Speed Serverless with nuclio
Running High-Speed Serverless with nuclioRunning High-Speed Serverless with nuclio
Running High-Speed Serverless with nuclio
 
Introduction to PyTorch
Introduction to PyTorchIntroduction to PyTorch
Introduction to PyTorch
 
Python
PythonPython
Python
 
Building Scalable, Highly Concurrent & Fault Tolerant Systems - Lessons Learned
Building Scalable, Highly Concurrent & Fault Tolerant Systems -  Lessons LearnedBuilding Scalable, Highly Concurrent & Fault Tolerant Systems -  Lessons Learned
Building Scalable, Highly Concurrent & Fault Tolerant Systems - Lessons Learned
 
Facebook architecture
Facebook architectureFacebook architecture
Facebook architecture
 
Google's Pathways Language Model and Chain-of-Thought
Google's Pathways Language Model and Chain-of-ThoughtGoogle's Pathways Language Model and Chain-of-Thought
Google's Pathways Language Model and Chain-of-Thought
 
tf.data: TensorFlow Input Pipeline
tf.data: TensorFlow Input Pipelinetf.data: TensorFlow Input Pipeline
tf.data: TensorFlow Input Pipeline
 
A Tale of Three Deep Learning Frameworks: TensorFlow, Keras, & PyTorch with B...
A Tale of Three Deep Learning Frameworks: TensorFlow, Keras, & PyTorch with B...A Tale of Three Deep Learning Frameworks: TensorFlow, Keras, & PyTorch with B...
A Tale of Three Deep Learning Frameworks: TensorFlow, Keras, & PyTorch with B...
 

Similar to Introduction to TensorFlow 2

Introduction to TensorFlow 2 and Keras
Introduction to TensorFlow 2 and KerasIntroduction to TensorFlow 2 and Keras
Introduction to TensorFlow 2 and KerasOswald Campesato
 
Introduction to TensorFlow 2
Introduction to TensorFlow 2Introduction to TensorFlow 2
Introduction to TensorFlow 2Oswald Campesato
 
Working with tf.data (TF 2)
Working with tf.data (TF 2)Working with tf.data (TF 2)
Working with tf.data (TF 2)Oswald Campesato
 
Deep Learning, Scala, and Spark
Deep Learning, Scala, and SparkDeep Learning, Scala, and Spark
Deep Learning, Scala, and SparkOswald Campesato
 
Deep Learning and TensorFlow
Deep Learning and TensorFlowDeep Learning and TensorFlow
Deep Learning and TensorFlowOswald Campesato
 
TensorFlow for IITians
TensorFlow for IITiansTensorFlow for IITians
TensorFlow for IITiansAshish Bansal
 
Introduction to Deep Learning, Keras, and TensorFlow
Introduction to Deep Learning, Keras, and TensorFlowIntroduction to Deep Learning, Keras, and TensorFlow
Introduction to Deep Learning, Keras, and TensorFlowSri Ambati
 
Introduction to Deep Learning, Keras, and Tensorflow
Introduction to Deep Learning, Keras, and TensorflowIntroduction to Deep Learning, Keras, and Tensorflow
Introduction to Deep Learning, Keras, and TensorflowOswald Campesato
 
Introduction to Deep Learning and TensorFlow
Introduction to Deep Learning and TensorFlowIntroduction to Deep Learning and TensorFlow
Introduction to Deep Learning and TensorFlowOswald Campesato
 
Intro to Deep Learning, TensorFlow, and tensorflow.js
Intro to Deep Learning, TensorFlow, and tensorflow.jsIntro to Deep Learning, TensorFlow, and tensorflow.js
Intro to Deep Learning, TensorFlow, and tensorflow.jsOswald Campesato
 
TensorFlow in Your Browser
TensorFlow in Your BrowserTensorFlow in Your Browser
TensorFlow in Your BrowserOswald Campesato
 
Deep Learning in Your Browser
Deep Learning in Your BrowserDeep Learning in Your Browser
Deep Learning in Your BrowserOswald Campesato
 
Introduction to TensorFlow 2.0
Introduction to TensorFlow 2.0Introduction to TensorFlow 2.0
Introduction to TensorFlow 2.0Databricks
 
TensorFlow example for AI Ukraine2016
TensorFlow example  for AI Ukraine2016TensorFlow example  for AI Ukraine2016
TensorFlow example for AI Ukraine2016Andrii Babii
 
Tensorflow - Intro (2017)
Tensorflow - Intro (2017)Tensorflow - Intro (2017)
Tensorflow - Intro (2017)Alessio Tonioni
 
Boosting machine learning workflow with TensorFlow 2.0
Boosting machine learning workflow with TensorFlow 2.0Boosting machine learning workflow with TensorFlow 2.0
Boosting machine learning workflow with TensorFlow 2.0Jeongkyu Shin
 
Deep Learning in your Browser: powered by WebGL
Deep Learning in your Browser: powered by WebGLDeep Learning in your Browser: powered by WebGL
Deep Learning in your Browser: powered by WebGLOswald Campesato
 
Hands-on Learning with KubeFlow + Keras/TensorFlow 2.0 + TF Extended (TFX) + ...
Hands-on Learning with KubeFlow + Keras/TensorFlow 2.0 + TF Extended (TFX) + ...Hands-on Learning with KubeFlow + Keras/TensorFlow 2.0 + TF Extended (TFX) + ...
Hands-on Learning with KubeFlow + Keras/TensorFlow 2.0 + TF Extended (TFX) + ...Chris Fregly
 

Similar to Introduction to TensorFlow 2 (20)

Introduction to TensorFlow 2 and Keras
Introduction to TensorFlow 2 and KerasIntroduction to TensorFlow 2 and Keras
Introduction to TensorFlow 2 and Keras
 
Introduction to TensorFlow 2
Introduction to TensorFlow 2Introduction to TensorFlow 2
Introduction to TensorFlow 2
 
Working with tf.data (TF 2)
Working with tf.data (TF 2)Working with tf.data (TF 2)
Working with tf.data (TF 2)
 
Deep Learning, Scala, and Spark
Deep Learning, Scala, and SparkDeep Learning, Scala, and Spark
Deep Learning, Scala, and Spark
 
Deep Learning and TensorFlow
Deep Learning and TensorFlowDeep Learning and TensorFlow
Deep Learning and TensorFlow
 
TensorFlow for IITians
TensorFlow for IITiansTensorFlow for IITians
TensorFlow for IITians
 
TensorFlow Tutorial.pdf
TensorFlow Tutorial.pdfTensorFlow Tutorial.pdf
TensorFlow Tutorial.pdf
 
Introduction to Deep Learning, Keras, and TensorFlow
Introduction to Deep Learning, Keras, and TensorFlowIntroduction to Deep Learning, Keras, and TensorFlow
Introduction to Deep Learning, Keras, and TensorFlow
 
Introduction to Deep Learning, Keras, and Tensorflow
Introduction to Deep Learning, Keras, and TensorflowIntroduction to Deep Learning, Keras, and Tensorflow
Introduction to Deep Learning, Keras, and Tensorflow
 
Introduction to Deep Learning and TensorFlow
Introduction to Deep Learning and TensorFlowIntroduction to Deep Learning and TensorFlow
Introduction to Deep Learning and TensorFlow
 
H2 o berkeleydltf
H2 o berkeleydltfH2 o berkeleydltf
H2 o berkeleydltf
 
Intro to Deep Learning, TensorFlow, and tensorflow.js
Intro to Deep Learning, TensorFlow, and tensorflow.jsIntro to Deep Learning, TensorFlow, and tensorflow.js
Intro to Deep Learning, TensorFlow, and tensorflow.js
 
TensorFlow in Your Browser
TensorFlow in Your BrowserTensorFlow in Your Browser
TensorFlow in Your Browser
 
Deep Learning in Your Browser
Deep Learning in Your BrowserDeep Learning in Your Browser
Deep Learning in Your Browser
 
Introduction to TensorFlow 2.0
Introduction to TensorFlow 2.0Introduction to TensorFlow 2.0
Introduction to TensorFlow 2.0
 
TensorFlow example for AI Ukraine2016
TensorFlow example  for AI Ukraine2016TensorFlow example  for AI Ukraine2016
TensorFlow example for AI Ukraine2016
 
Tensorflow - Intro (2017)
Tensorflow - Intro (2017)Tensorflow - Intro (2017)
Tensorflow - Intro (2017)
 
Boosting machine learning workflow with TensorFlow 2.0
Boosting machine learning workflow with TensorFlow 2.0Boosting machine learning workflow with TensorFlow 2.0
Boosting machine learning workflow with TensorFlow 2.0
 
Deep Learning in your Browser: powered by WebGL
Deep Learning in your Browser: powered by WebGLDeep Learning in your Browser: powered by WebGL
Deep Learning in your Browser: powered by WebGL
 
Hands-on Learning with KubeFlow + Keras/TensorFlow 2.0 + TF Extended (TFX) + ...
Hands-on Learning with KubeFlow + Keras/TensorFlow 2.0 + TF Extended (TFX) + ...Hands-on Learning with KubeFlow + Keras/TensorFlow 2.0 + TF Extended (TFX) + ...
Hands-on Learning with KubeFlow + Keras/TensorFlow 2.0 + TF Extended (TFX) + ...
 

More from Oswald Campesato

Introduction to Deep Learning
Introduction to Deep LearningIntroduction to Deep Learning
Introduction to Deep LearningOswald Campesato
 
"An Introduction to AI and Deep Learning"
"An Introduction to AI and Deep Learning""An Introduction to AI and Deep Learning"
"An Introduction to AI and Deep Learning"Oswald Campesato
 
Introduction to Deep Learning for Non-Programmers
Introduction to Deep Learning for Non-ProgrammersIntroduction to Deep Learning for Non-Programmers
Introduction to Deep Learning for Non-ProgrammersOswald Campesato
 
Deep Learning and TensorFlow
Deep Learning and TensorFlowDeep Learning and TensorFlow
Deep Learning and TensorFlowOswald Campesato
 
Introduction to Deep Learning and Tensorflow
Introduction to Deep Learning and TensorflowIntroduction to Deep Learning and Tensorflow
Introduction to Deep Learning and TensorflowOswald Campesato
 
Deep Learning, Keras, and TensorFlow
Deep Learning, Keras, and TensorFlowDeep Learning, Keras, and TensorFlow
Deep Learning, Keras, and TensorFlowOswald Campesato
 
Java and Deep Learning (Introduction)
Java and Deep Learning (Introduction)Java and Deep Learning (Introduction)
Java and Deep Learning (Introduction)Oswald Campesato
 
Deep Learning: R with Keras and TensorFlow
Deep Learning: R with Keras and TensorFlowDeep Learning: R with Keras and TensorFlow
Deep Learning: R with Keras and TensorFlowOswald Campesato
 
Diving into Deep Learning (Silicon Valley Code Camp 2017)
Diving into Deep Learning (Silicon Valley Code Camp 2017)Diving into Deep Learning (Silicon Valley Code Camp 2017)
Diving into Deep Learning (Silicon Valley Code Camp 2017)Oswald Campesato
 
TypeScript and Deep Learning
TypeScript and Deep LearningTypeScript and Deep Learning
TypeScript and Deep LearningOswald Campesato
 
D3, TypeScript, and Deep Learning
D3, TypeScript, and Deep LearningD3, TypeScript, and Deep Learning
D3, TypeScript, and Deep LearningOswald Campesato
 
D3, TypeScript, and Deep Learning
D3, TypeScript, and Deep LearningD3, TypeScript, and Deep Learning
D3, TypeScript, and Deep LearningOswald Campesato
 

More from Oswald Campesato (18)

Introduction to Deep Learning
Introduction to Deep LearningIntroduction to Deep Learning
Introduction to Deep Learning
 
"An Introduction to AI and Deep Learning"
"An Introduction to AI and Deep Learning""An Introduction to AI and Deep Learning"
"An Introduction to AI and Deep Learning"
 
Introduction to Deep Learning for Non-Programmers
Introduction to Deep Learning for Non-ProgrammersIntroduction to Deep Learning for Non-Programmers
Introduction to Deep Learning for Non-Programmers
 
Deep Learning and TensorFlow
Deep Learning and TensorFlowDeep Learning and TensorFlow
Deep Learning and TensorFlow
 
Introduction to Deep Learning and Tensorflow
Introduction to Deep Learning and TensorflowIntroduction to Deep Learning and Tensorflow
Introduction to Deep Learning and Tensorflow
 
Deep Learning, Keras, and TensorFlow
Deep Learning, Keras, and TensorFlowDeep Learning, Keras, and TensorFlow
Deep Learning, Keras, and TensorFlow
 
C++ and Deep Learning
C++ and Deep LearningC++ and Deep Learning
C++ and Deep Learning
 
Scala and Deep Learning
Scala and Deep LearningScala and Deep Learning
Scala and Deep Learning
 
Java and Deep Learning (Introduction)
Java and Deep Learning (Introduction)Java and Deep Learning (Introduction)
Java and Deep Learning (Introduction)
 
Deep Learning: R with Keras and TensorFlow
Deep Learning: R with Keras and TensorFlowDeep Learning: R with Keras and TensorFlow
Deep Learning: R with Keras and TensorFlow
 
Java and Deep Learning
Java and Deep LearningJava and Deep Learning
Java and Deep Learning
 
Diving into Deep Learning (Silicon Valley Code Camp 2017)
Diving into Deep Learning (Silicon Valley Code Camp 2017)Diving into Deep Learning (Silicon Valley Code Camp 2017)
Diving into Deep Learning (Silicon Valley Code Camp 2017)
 
Android and Deep Learning
Android and Deep LearningAndroid and Deep Learning
Android and Deep Learning
 
Introduction to Kotlin
Introduction to KotlinIntroduction to Kotlin
Introduction to Kotlin
 
TypeScript and Deep Learning
TypeScript and Deep LearningTypeScript and Deep Learning
TypeScript and Deep Learning
 
D3, TypeScript, and Deep Learning
D3, TypeScript, and Deep LearningD3, TypeScript, and Deep Learning
D3, TypeScript, and Deep Learning
 
D3, TypeScript, and Deep Learning
D3, TypeScript, and Deep LearningD3, TypeScript, and Deep Learning
D3, TypeScript, and Deep Learning
 
Angular and Deep Learning
Angular and Deep LearningAngular and Deep Learning
Angular and Deep Learning
 

Recently uploaded

AI in Action: Real World Use Cases by Anitaraj
AI in Action: Real World Use Cases by AnitarajAI in Action: Real World Use Cases by Anitaraj
AI in Action: Real World Use Cases by AnitarajAnitaRaj43
 
CORS (Kitworks Team Study 양다윗 발표자료 240510)
CORS (Kitworks Team Study 양다윗 발표자료 240510)CORS (Kitworks Team Study 양다윗 발표자료 240510)
CORS (Kitworks Team Study 양다윗 발표자료 240510)Wonjun Hwang
 
Top 10 CodeIgniter Development Companies
Top 10 CodeIgniter Development CompaniesTop 10 CodeIgniter Development Companies
Top 10 CodeIgniter Development CompaniesTopCSSGallery
 
Frisco Automating Purchase Orders with MuleSoft IDP- May 10th, 2024.pptx.pdf
Frisco Automating Purchase Orders with MuleSoft IDP- May 10th, 2024.pptx.pdfFrisco Automating Purchase Orders with MuleSoft IDP- May 10th, 2024.pptx.pdf
Frisco Automating Purchase Orders with MuleSoft IDP- May 10th, 2024.pptx.pdfAnubhavMangla3
 
AI mind or machine power point presentation
AI mind or machine power point presentationAI mind or machine power point presentation
AI mind or machine power point presentationyogeshlabana357357
 
Harnessing Passkeys in the Battle Against AI-Powered Cyber Threats.pptx
Harnessing Passkeys in the Battle Against AI-Powered Cyber Threats.pptxHarnessing Passkeys in the Battle Against AI-Powered Cyber Threats.pptx
Harnessing Passkeys in the Battle Against AI-Powered Cyber Threats.pptxFIDO Alliance
 
Simplifying Mobile A11y Presentation.pptx
Simplifying Mobile A11y Presentation.pptxSimplifying Mobile A11y Presentation.pptx
Simplifying Mobile A11y Presentation.pptxMarkSteadman7
 
Human Expert Website Manual WCAG 2.0 2.1 2.2 Audit - Digital Accessibility Au...
Human Expert Website Manual WCAG 2.0 2.1 2.2 Audit - Digital Accessibility Au...Human Expert Website Manual WCAG 2.0 2.1 2.2 Audit - Digital Accessibility Au...
Human Expert Website Manual WCAG 2.0 2.1 2.2 Audit - Digital Accessibility Au...Skynet Technologies
 
Event-Driven Architecture Masterclass: Engineering a Robust, High-performance...
Event-Driven Architecture Masterclass: Engineering a Robust, High-performance...Event-Driven Architecture Masterclass: Engineering a Robust, High-performance...
Event-Driven Architecture Masterclass: Engineering a Robust, High-performance...ScyllaDB
 
Intro to Passkeys and the State of Passwordless.pptx
Intro to Passkeys and the State of Passwordless.pptxIntro to Passkeys and the State of Passwordless.pptx
Intro to Passkeys and the State of Passwordless.pptxFIDO Alliance
 
Tales from a Passkey Provider Progress from Awareness to Implementation.pptx
Tales from a Passkey Provider  Progress from Awareness to Implementation.pptxTales from a Passkey Provider  Progress from Awareness to Implementation.pptx
Tales from a Passkey Provider Progress from Awareness to Implementation.pptxFIDO Alliance
 
Observability Concepts EVERY Developer Should Know (DevOpsDays Seattle)
Observability Concepts EVERY Developer Should Know (DevOpsDays Seattle)Observability Concepts EVERY Developer Should Know (DevOpsDays Seattle)
Observability Concepts EVERY Developer Should Know (DevOpsDays Seattle)Paige Cruz
 
Vector Search @ sw2con for slideshare.pptx
Vector Search @ sw2con for slideshare.pptxVector Search @ sw2con for slideshare.pptx
Vector Search @ sw2con for slideshare.pptxjbellis
 
ADP Passwordless Journey Case Study.pptx
ADP Passwordless Journey Case Study.pptxADP Passwordless Journey Case Study.pptx
ADP Passwordless Journey Case Study.pptxFIDO Alliance
 
Design Guidelines for Passkeys 2024.pptx
Design Guidelines for Passkeys 2024.pptxDesign Guidelines for Passkeys 2024.pptx
Design Guidelines for Passkeys 2024.pptxFIDO Alliance
 
State of the Smart Building Startup Landscape 2024!
State of the Smart Building Startup Landscape 2024!State of the Smart Building Startup Landscape 2024!
State of the Smart Building Startup Landscape 2024!Memoori
 
Finding Java's Hidden Performance Traps @ DevoxxUK 2024
Finding Java's Hidden Performance Traps @ DevoxxUK 2024Finding Java's Hidden Performance Traps @ DevoxxUK 2024
Finding Java's Hidden Performance Traps @ DevoxxUK 2024Victor Rentea
 
Introduction to use of FHIR Documents in ABDM
Introduction to use of FHIR Documents in ABDMIntroduction to use of FHIR Documents in ABDM
Introduction to use of FHIR Documents in ABDMKumar Satyam
 
TEST BANK For Principles of Anatomy and Physiology, 16th Edition by Gerard J....
TEST BANK For Principles of Anatomy and Physiology, 16th Edition by Gerard J....TEST BANK For Principles of Anatomy and Physiology, 16th Edition by Gerard J....
TEST BANK For Principles of Anatomy and Physiology, 16th Edition by Gerard J....rightmanforbloodline
 
JavaScript Usage Statistics 2024 - The Ultimate Guide
JavaScript Usage Statistics 2024 - The Ultimate GuideJavaScript Usage Statistics 2024 - The Ultimate Guide
JavaScript Usage Statistics 2024 - The Ultimate GuidePixlogix Infotech
 

Recently uploaded (20)

AI in Action: Real World Use Cases by Anitaraj
AI in Action: Real World Use Cases by AnitarajAI in Action: Real World Use Cases by Anitaraj
AI in Action: Real World Use Cases by Anitaraj
 
CORS (Kitworks Team Study 양다윗 발표자료 240510)
CORS (Kitworks Team Study 양다윗 발표자료 240510)CORS (Kitworks Team Study 양다윗 발표자료 240510)
CORS (Kitworks Team Study 양다윗 발표자료 240510)
 
Top 10 CodeIgniter Development Companies
Top 10 CodeIgniter Development CompaniesTop 10 CodeIgniter Development Companies
Top 10 CodeIgniter Development Companies
 
Frisco Automating Purchase Orders with MuleSoft IDP- May 10th, 2024.pptx.pdf
Frisco Automating Purchase Orders with MuleSoft IDP- May 10th, 2024.pptx.pdfFrisco Automating Purchase Orders with MuleSoft IDP- May 10th, 2024.pptx.pdf
Frisco Automating Purchase Orders with MuleSoft IDP- May 10th, 2024.pptx.pdf
 
AI mind or machine power point presentation
AI mind or machine power point presentationAI mind or machine power point presentation
AI mind or machine power point presentation
 
Harnessing Passkeys in the Battle Against AI-Powered Cyber Threats.pptx
Harnessing Passkeys in the Battle Against AI-Powered Cyber Threats.pptxHarnessing Passkeys in the Battle Against AI-Powered Cyber Threats.pptx
Harnessing Passkeys in the Battle Against AI-Powered Cyber Threats.pptx
 
Simplifying Mobile A11y Presentation.pptx
Simplifying Mobile A11y Presentation.pptxSimplifying Mobile A11y Presentation.pptx
Simplifying Mobile A11y Presentation.pptx
 
Human Expert Website Manual WCAG 2.0 2.1 2.2 Audit - Digital Accessibility Au...
Human Expert Website Manual WCAG 2.0 2.1 2.2 Audit - Digital Accessibility Au...Human Expert Website Manual WCAG 2.0 2.1 2.2 Audit - Digital Accessibility Au...
Human Expert Website Manual WCAG 2.0 2.1 2.2 Audit - Digital Accessibility Au...
 
Event-Driven Architecture Masterclass: Engineering a Robust, High-performance...
Event-Driven Architecture Masterclass: Engineering a Robust, High-performance...Event-Driven Architecture Masterclass: Engineering a Robust, High-performance...
Event-Driven Architecture Masterclass: Engineering a Robust, High-performance...
 
Intro to Passkeys and the State of Passwordless.pptx
Intro to Passkeys and the State of Passwordless.pptxIntro to Passkeys and the State of Passwordless.pptx
Intro to Passkeys and the State of Passwordless.pptx
 
Tales from a Passkey Provider Progress from Awareness to Implementation.pptx
Tales from a Passkey Provider  Progress from Awareness to Implementation.pptxTales from a Passkey Provider  Progress from Awareness to Implementation.pptx
Tales from a Passkey Provider Progress from Awareness to Implementation.pptx
 
Observability Concepts EVERY Developer Should Know (DevOpsDays Seattle)
Observability Concepts EVERY Developer Should Know (DevOpsDays Seattle)Observability Concepts EVERY Developer Should Know (DevOpsDays Seattle)
Observability Concepts EVERY Developer Should Know (DevOpsDays Seattle)
 
Vector Search @ sw2con for slideshare.pptx
Vector Search @ sw2con for slideshare.pptxVector Search @ sw2con for slideshare.pptx
Vector Search @ sw2con for slideshare.pptx
 
ADP Passwordless Journey Case Study.pptx
ADP Passwordless Journey Case Study.pptxADP Passwordless Journey Case Study.pptx
ADP Passwordless Journey Case Study.pptx
 
Design Guidelines for Passkeys 2024.pptx
Design Guidelines for Passkeys 2024.pptxDesign Guidelines for Passkeys 2024.pptx
Design Guidelines for Passkeys 2024.pptx
 
State of the Smart Building Startup Landscape 2024!
State of the Smart Building Startup Landscape 2024!State of the Smart Building Startup Landscape 2024!
State of the Smart Building Startup Landscape 2024!
 
Finding Java's Hidden Performance Traps @ DevoxxUK 2024
Finding Java's Hidden Performance Traps @ DevoxxUK 2024Finding Java's Hidden Performance Traps @ DevoxxUK 2024
Finding Java's Hidden Performance Traps @ DevoxxUK 2024
 
Introduction to use of FHIR Documents in ABDM
Introduction to use of FHIR Documents in ABDMIntroduction to use of FHIR Documents in ABDM
Introduction to use of FHIR Documents in ABDM
 
TEST BANK For Principles of Anatomy and Physiology, 16th Edition by Gerard J....
TEST BANK For Principles of Anatomy and Physiology, 16th Edition by Gerard J....TEST BANK For Principles of Anatomy and Physiology, 16th Edition by Gerard J....
TEST BANK For Principles of Anatomy and Physiology, 16th Edition by Gerard J....
 
JavaScript Usage Statistics 2024 - The Ultimate Guide
JavaScript Usage Statistics 2024 - The Ultimate GuideJavaScript Usage Statistics 2024 - The Ultimate Guide
JavaScript Usage Statistics 2024 - The Ultimate Guide
 

Introduction to TensorFlow 2

  • 1. Introduction to TensorFlow 2 Scala Meetup (PA) 06/27/2019 Oswald Campesato ocampesato@yahoo.com
  • 2. Highlights/Overview  What is TensorFlow 2?  Major Changes in TF 2  Working with strings/arrays/tensors  Working with TF 2 @tf.function decorator  Working with TF 2 generators  Working with TF 2 tf.data.Dataset  Datasets in TF 1.x versus TF 2  Working with TF 2 tf.keras  CNNs, RNNs, LSTMs, Bidirectional LSTMs  Reinforcement Learning/TF Agents
  • 3. Highlights/Overview This session does not cover:  “the new vision” of TF 2  “the common practices” of TF 2  “lessons learned” from TF 1.x  a detailed “roadmap” from TF 1.x to TF 2
  • 4. What is TensorFlow 2? An open source framework for ML and DL Created by Google (TF 1.x released 11/2015) Evolved from Google Brain Linux and Mac OS X support (VM for Windows) TF home page: https://www.tensorflow.org/
  • 5. Status of TensorFlow 2 Alpha release: around March/2019 Other “Nightly” builds: https://pypi.org/project/tf-nightly-2.0-preview/2.0.0.dev20190417/#files Production release: TBD (later this year)
  • 6. TF 2 Platform Support Tested and supported on 64-bit systems Ubuntu 16.04 or later Windows 7 or later macOS 10.12.6 (Sierra) or later (no GPU support) Raspbian 9.0 or later
  • 7. TF 2 Docker Images  https://hub.docker.com/r/tensorflow/tensorflow/ $ docker run -it --rm tensorflow/tensorflow bash  Start a CPU-only container with Python 2: $ docker run -it --rm --runtime=nvidia tensorflow/tensorflow:latest- gpu-py3 python  Start a GPU container with Python 3 & Python interpreter: $ docker run -it --rm -v $(realpath ~/notebooks):/tf/notebooks -p 8888:8888 tensorflow/tensorflow:late
  • 8. Status of TensorFlow 1.x 1.13.1: current stable release 1.14: latest release After TF 2 is in production: Only security-related updates to TF 1.x TF 1.x support for one year after TF 2 production
  • 9. What is TensorFlow 2?  Support for Python, Java, C++  Desktop, Server, Mobile, Web  CPU/GPU/TPU support  Visualization via TensorBoard  Can be embedded in Python scripts  Installation: pip install tensorflow  Ex: pip install tensorflow==2.0.0-alpha0
  • 10. TensorFlow Use Cases (Generic) Image recognition Computer vision Voice/sound recognition Time series analysis Language detection Language translation Text-based processing Handwriting Recognition
  • 11. Major Changes in TF 2  TF 2 Eager Execution: default mode  @tf.function decorator: instead of tf.Session()  AutoGraph: graph generated in @tf.function()  Generators: used with tf.data.Dataset (TF 2)  Differentiation: calculates gradients  tf.GradientTape: automatic differentiation
  • 12. Removed from TensorFlow 2  tf.Session()  tf.placeholder()  tf.global_initializer()  feed_dict  Variable scopes  tf.contrib code  tf.flags and tf.contrib  Global variables  => TF 1.x functions moved to compat.v1
  • 13. Upgrading Files/Directories to TF 2  The TF 2 upgrade script: https://www.tensorflow.org/alpha/guide/upgrade  Script name: tf_upgrade_v2  install TF to get the upgrade script: pip install tensorflow (NOT pip3)
  • 14. Upgrading Files/Directories to TF 2 1) upgrade one file: tf_upgrade_v2 --infile oldtf.py --outfile newtf.py  The preceding command creates report.txt Upgrade an entire directory: tf_upgrade_v2 –intree mydir --outtree newtf.py – copyotherfiles False NB: do NOT make manual upgrade changes
  • 15. Migrating to TensorFlow 2  1) do not manually update parts of your code  2) script won't work with functions with reordered arguments  3) script assumes this TF statement: "import tensorflow as tf"  4) the script does not reorder arguments  5) the script adds keyword arguments to functions that have their arguments reordered.  Upgrade Jupyter notebooks and Python files in a GitHub repo: http://tf2up.ml/ => Replace prefix https://github.com/ with http://tf2up.m
  • 16. What is a Tensor? TF tensors are n-dimensional arrays TF tensors are very similar to numpy ndarrays scalar number: a zeroth-order tensor vector: a first-order tensor matrix: a second-order tensor 3-dimensional array: a 3rd order tensor https://dzone.com/articles/tensorflow-simplified- examples
  • 17. TensorFlow Eager Execution An imperative interface to TF Fast debugging & immediate run-time errors => TF 2 requires Python 3.x (not Python 2.x) No static graphs or sessions
  • 18. TensorFlow Eager Execution integration with Python tools Supports dynamic models + Python control flow support for custom and higher-order gradients => Default mode in TensorFlow 2.0
  • 19. TensorFlow Eager Execution  import tensorflow as tf  x = [[2.]]  # m = tf.matmul(x, x) <= deprecated  m = tf.linalg.matmul(x, x)  print("m:",m)  print("m:",m.numpy()) #Output: #m: tf.Tensor([[4.]], shape=(1,1),dtype=float32) #m: [[4.]]
  • 20. Basic TF 2 Operations Working with Constants Working with Strings Working with Tensors Working Operators Arrays in TF 2 Multiplying Two Arrays Convert Python Arrays to TF Arrays Conflicting Types
  • 21. TensorFlow “primitive types”  tf.constant: > initialized immediately and immutable  import tensorflow as tf  aconst = tf.constant(3.0)  print(aconst) # output for TF 2: #tf.Tensor(3.0, shape=(), dtype=float32) # output for TF 1.x: #Tensor("Const:0", shape=(), dtype=float32)
  • 22. TensorFlow “primitive types” tf.Variable (a class): + initial value is required + updated during training + in-memory buffer (saved/restored from disk) + can be shared in a distributed environment + they hold learned parameters of a model
  • 23. TensorFlow Arithmetic import tensorflow as tf # arith1.py a = tf.add(4, 2) b = tf.subtract(8, 6) c = tf.multiply(a, 3) d = tf.div(a, 6)  print(a) # 6  print(b) # 2  print(c) # 18  print(d) # 1  print("a:",a.numpy())  print("b:",b.numpy())  print("c:",c.numpy())  print("d:",d.numpy())
  • 24. TensorFlow Arithmetic  tf.Tensor(6, shape=(), dtype=int32)  tf.Tensor(2, shape=(), dtype=int32)  tf.Tensor(18, shape=(), dtype=int32)  tf.Tensor(1.0, shape=(), dtype=float64)  6  2  18  1.0
  • 25. TF 2 Loops and Arrays Using “for” loops Using “while” loops tf.reduce_prod() tf.reduce_sum()
  • 26. A “for” Loop Example  import tensorflow as tf  x = tf.Variable(0, name='x')  for i in range(5):  x = x + 1  print("x:",x)
  • 27. A “for” Loop Example  x: tf.Tensor(1, shape=(), dtype=int32)  x: tf.Tensor(2, shape=(), dtype=int32)  x: tf.Tensor(3, shape=(), dtype=int32)  x: tf.Tensor(4, shape=(), dtype=int32)  x: tf.Tensor(5, shape=(), dtype=int32)
  • 28. A “while” Loop Example  import tensorflow as tf  a = tf.constant(12)  while not tf.equal(a, 1):  if tf.equal(a % 2, 0):  a = a / 2  else:  a = 3 * a + 1  print(a)
  • 29. A “while” Loop Example  tf.Tensor(6.0, shape=(), dtype=float64)  tf.Tensor(3.0, shape=(), dtype=float64)  tf.Tensor(10.0, shape=(), dtype=float64)  tf.Tensor(5.0, shape=(), dtype=float64)  tf.Tensor(16.0, shape=(), dtype=float64)  tf.Tensor(8.0, shape=(), dtype=float64)  tf.Tensor(4.0, shape=(), dtype=float64)  tf.Tensor(2.0, shape=(), dtype=float64)  tf.Tensor(1.0, shape=(), dtype=float64)
  • 30. Useful TF 2 APIs tf.shape() tf.rank() tf.range() tf.reshape() tf.ones() tf.zeros() tf.fill()  tf.random.normal()  tf.truncated_normal()
  • 31. Some TF 2 “lazy operators” map() filter() flatmap() batch() take() zip() flatten() Combined via “method chaining”
  • 32. TF 2 “lazy operators”  filter():  uses Boolean logic to "filter" the elements in an array to determine which elements satisfy the Boolean condition  map(): a projection  this operator "applies" a lambda expression to each input element  flat_map():  maps a single element of the input dataset to a Dataset of elements
  • 33. TF 2 “lazy operators”  batch(n):  processes a "batch" of n elements during each iteration  repeat(n):  repeats its input values n times  take(n):  operator "takes" n input values
  • 34. TF 2 tf.data.Dataset TF “wrapper” class around data sources Located in the tf.data.Dataset namespace Supports lambda expressions Supports lazy operators They use generators instead of iterators (TF 1.x)
  • 35. tf.data.Dataset.from_tensors()  Import tensorflow as tf  #combine the input into one element  t1 = tf.constant([[1, 2], [3, 4]])  ds1 = tf.data.Dataset.from_tensors(t1) # output: [[1, 2], [3, 4]]
  • 36. tf.data.Dataset.from_tensor_slices()  Import tensorflow as tf  #separate element for each item  t2 = tf.constant([[1, 2], [3, 4]])  ds1 = tf.data.Dataset.from_tensor_slices(t2) # output: [1, 2], [3, 4]
  • 37. TF 2 Datasets: code sample  import tensorflow as tf # tf2-dataset.py  import numpy as np x = np.arange(0, 10)  # create a dataset from a Numpy array ds = tf.data.Dataset.from_tensor_slices(x)
  • 38. TF 1.x Datasets: iterator  import tensorflow as tf # tf1x-plusone.py  import numpy as np  x = np.arange(0, 10)  # create dataset object from Numpy array  ds = tf.data.Dataset.from_tensor_slices(x)  ds.map(lambda x: x + 1)  # create a one-shot iterator <= TF 1.x  iterator = ds.make_one_shot_iterator()  for i in range(10):  val = iterator.get_next()  print("val:",val)
  • 39. TF 2 Datasets: generator import tensorflow as tf # tf2-plusone.py import numpy as np x = np.arange(0, 5) # 0, 1, 2, 3, 4 def gener(): for i in x: yield (3*i) ds = tf.data.Dataset.from_generator(gener, (tf.int64)) for value in ds.take(len(x)): print("1value:",value) for value in ds.take(2*len(x)): print("2value:",value)
  • 40. TF 2 Datasets: generator  1value: tf.Tensor(0, shape=(), dtype=int64)  1value: tf.Tensor(3, shape=(), dtype=int64)  1value: tf.Tensor(6, shape=(), dtype=int64)  1value: tf.Tensor(9, shape=(), dtype=int64)  1value: tf.Tensor(12, shape=(), dtype=int64)  2value: tf.Tensor(0, shape=(), dtype=int64)  2value: tf.Tensor(3, shape=(), dtype=int64)  2value: tf.Tensor(6, shape=(), dtype=int64)  2value: tf.Tensor(9, shape=(), dtype=int64)  2value: tf.Tensor(12, shape=(), dtype=int64)
  • 41. TF 1.x map() and take()  import tensorflow as tf # tf 1.12.0  tf.enable_eager_execution()  ds = tf.data.TextLineDataset("file.txt")  ds = ds.map(lambda line: tf.string_split([line]).values)  try:  for value in ds.take(2):  print("value:",value)  except tf.errors.OutOfRangeError:  pass
  • 42. TF 2 map() and take(): file.txt  this is file line #1  this is file line #2  this is file line #3  this is file line #4  this is file line #5
  • 43. TF 2 map() and take(): output  ('value:', <tf.Tensor: id=16, shape=(5,), dtype=string, numpy=array(['this', 'is', 'file', 'line', '#1'], dtype=object)>)  ('value:', <tf.Tensor: id=18, shape=(5,), dtype=string, numpy=array(['this', 'is', 'file', 'line', '#2'], dtype=object)>)
  • 44. TF 2 map() and take()  import tensorflow as tf  import numpy as np  x = np.array([[1],[2],[3],[4]])  # make a ds from a numpy array  ds = tf.data.Dataset.from_tensor_slices(x)  ds = ds.map(lambda x: x*2).map(lambda x: x+1).map(lambda x: x**3)  for value in ds.take(4):  print("value:",value)
  • 45. TF 2 map() and take(): output  value: tf.Tensor([27], shape=(1,), dtype=int64)  value: tf.Tensor([125], shape=(1,), dtype=int64)  value: tf.Tensor([343], shape=(1,), dtype=int64)  value: tf.Tensor([729], shape=(1,), dtype=int64)
  • 46. TF 2 batch(): EXTRA CREDIT import tensorflow as tf import numpy as np x = np.arange(0, 12) def gener(): i = 0 while(i < len(x/3)): yield (i, i+1, i+2) # three integers at a time i += 3 ds = tf.data.Dataset.from_generator(gener, (tf.int64,tf.int64,tf.int64)) third = int(len(x)/3) for value in ds.take(third): print("value:",value)
  • 47. TF 2 batch() & zip(): EXTRA CREDIT import tensorflow as tf ds1 = tf.data.Dataset.range(100) ds2 = tf.data.Dataset.range(0, -100, -1) ds3 = tf.data.Dataset.zip((ds1, ds2)) ds4 = ds3.batch(4) for value in ds.take(10): print("value:",value)
  • 48. TF 2: Working with tf.keras tf.keras.layers tf.keras.models tf.keras.optimizers tf.keras.utils tf.keras.regularizers
  • 49. TF 2: tf.keras.models  tf.keras.models.Sequential(): most common  clone_model(...): Clone any Model instance  load_model(...): Loads a model saved via save_model  model_from_config(...): Instantiates a Keras model from its config  model_from_json(...):  Parses a JSON model config file and returns a model instance  model_from_yaml(...):  Parses a yaml model config file and returns a model instance  save_model(...): Saves a model to a HDF5 file
  • 50. TF 2: tf.keras.layers  tf.keras.models.Sequential()  tf.keras.layers.Conv2D()  tf.keras.layers.MaxPooling2D()  tf.keras.layers.Flatten()  tf.keras.layers.Dense()  tf.keras.layers.Dropout(0.1)  tf.keras.layers.BatchNormalization()  tf.keras.layers.embedding()  tf.keras.layers.RNN()  tf.keras.layers.LSTM()  tf.keras.layers.Bidirectional (ex: BERT)
  • 51. TF 2: Activation Functions tf.keras.activations.relu tf.keras.activations.selu tf.keras.activations.elu tf.keras.activations.linear tf.keras.activations.sigmoid tf.keras.activations.softmax tf.keras.activations.softplus tf.keras.activations.tanh Others …
  • 52. TF 2: Built-in Datasets tf.keras.datasets.boston_housing tf.keras.datasets.cifar10 tf.keras.datasets.cifar100 tf.keras.datasets.fashion_mnist tf.keras.datasets.imdb tf.keras.datasets.mnist tf.keras.datasets.reuters
  • 53. TF 2: “Experimental”  tf.keras.experimental.CosineDecay  tf.keras.experimental.CosineDecayRestarts  tf.keras.experimental.LinearCosineDecay  tf.keras.experimental.NoisyLinearCosineDecay  tf.keras.experimental.PeepholeLSTMCell
  • 56. TF 2: tf.losses  tf.losses.BinaryCrossEntropy  tf.losses.CategoricalCrossEntropy  tf.losses.CategoricalHinge  tf.losses.CosineSimilarity  tf.losses.Hinge  tf.losses.Huber  tf.losses.KLD  tf.losses.KLDivergence  tf.losses.MAE  tf.losses.MSE  tf.losses.SparseCategoricalCrossentropy
  • 57. Other TF 2 Namespaces tf.keras.regularizers (L1 and L2) tf.keras.utils (to_categorical) tf.keras.preprocessing.text.Tokenizer
  • 58. TF 1.x Model APIs  tf.core (“reduced” in TF 2)  tf.layers (deprecated in TF 2)  tf.keras (the primary API in TF 2)  tf.estimator.Estimator (implementation in tf.keras)  tf.contrib.learn.Estimator (Deprecated in TF 2)
  • 59. Other TF 2 Features TF Probability (Linear Regression) TF Agents (Reinforcement Learning) TF Text (NLP) TF Federated TF Privacy Tensor2Tensor TF Ranking TFX
  • 60. TF 2 (Keras), DL, and RL The remaining slides briefly discuss TF 2 and:  CNNs (Convolutional Neural Networks)  RNNs (Recurrent Neural Networks)  LSTMs (Long Short Term Memory)  Autoencoders  Variational Autoencoders  Reinforcement Learning  Some Keras-based code blocks  Some useful links
  • 61. Neural Network: 3 Hidden Layers
  • 62. Transition Between Layers W = weight matrix between adjacent layers x = an input vector of numbers B = a bias vector of numbers F = activation function (sigmoid, tanh, etc) Y = output vector of numbers Y = F(x*W + b)
  • 63. TF 2/Keras and MLPs  import tensorflow as tf  . . .  model = tf.keras.models.Sequential()  model.add(tf.keras.layers.Dense(10, input_dim=num_pixels, activation='relu'))  model.add(tf.keras.layers.Dense(30, activation='relu'))  model.add(tf.keras.layers.Dense(10, activation='relu'))  model.add(tf.keras.layers.Dense(num_classes, activation='softmax'))  model.compile(tf.keras.optimizers.Adam(lr=0.01), loss='categorical_crossentropy', metrics=['accuracy'])
  • 64. TF 2/Keras and MLPs  import tensorflow as tf  . . .  model = tf.keras.models.Sequential([ tf.keras.layers.Dense(10, input_dim=num_pixels, activation='relu’), tf.keras.layers.Dense(30, activation='relu’), tf.keras.layers.Dense(10, activation='relu’), tf.keras.layers.Dense(num_classes, activation='softmax’), tf.keras.optimizers.Adam(lr=0.01), loss='categorical_crossentropy', metrics=['accuracy']) ])
  • 66. TF 2/Keras and CNNs  import tensorflow as tf  . . .  model = tf.keras.models.Sequential([  tf.keras. layers.Conv2D(30,(5,5), input_shape=(28,28,1),activation='relu')),  tf.keras.layers.MaxPooling2D(pool_size=(2, 2))),  tf.keras.layers.Flatten(),  tf.keras.layers.Dense(500, activation='relu’),  tf.keras.layers.Dropout(0.5),  tf.keras.layers.Dense(num_classes, activation='softmax’)  ])
  • 67. What are RNNs? RNNs = Recurrent Neural Networks RNNs capture/retain events in time FF networks do not learn from the past => RNNs DO learn from the past RNNs contain layers of recurrent neurons
  • 68. Common Types of RNNs 1) Simple RNNs (minimal structure) 2) LSTMs (input, forget, output gates) 3) Gated Recurrent Unit (GRU) NB: RNNs can be used with MNIST data
  • 69. Some Use Cases for RNNs 1) Time Series Data 2) NLP (Natural Language Processing): Processing documents “analyzing” Trump’s tweets 3) RNNs and MNIST dataset
  • 70. Recall: Feed Forward Structure
  • 71. RNN Cell (left) and FF (right)
  • 73. RNN Neuron: same W for all steps
  • 74. RNN and MNIST Dataset Sample initialization values for RNN:  n_steps = 28 # # of time steps  n_inputs = 28 # number of rows  n_neurons = 150 # 150 neurons in one layer  n_outputs = 10 # 10 digit classes (0->9) Remember that: one layer = one memory cell MNIST class count = size of output layer
  • 75. TF 2/Keras and RNNs import tensorflow as tf ... model = tf.keras.models.Sequential() model.add(tf.keras.layers.SimpleRNN(20, input_shape=(input_length, input_dim), return_sequences=False)) ...
  • 76. Problems with RNNs lengthy training time Unrolled RNN will be very deep Propagating gradients through many layers Prone to vanishing gradient Prone to exploding gradient
  • 77. What are LSTMs? Long short-term memory (LSTM): a model for the short-term memory Contains more state than a “regular” RNN which can last for a long period of time can forget and remember things selectively Hochreiter/Schmidhuber proposed in 1997 improved in 2000 by Felix Gers' team set accuracy records in multiple apps
  • 78. Features of LSTMs Used in Google speech recognition + Alpha Go they avoid the vanishing gradient problem Can track 1000s of discrete time steps Used by international competition winners
  • 79. Use Cases for LSTMs Connected handwriting recognition Speech recognition Forecasting Anomaly detection Pattern recognition
  • 80. Advantages of LSTMs well-suited to classify/process/predict time series More powerful: increased memory (=state) Can explicitly add long-term or short-term state even with time lags of unknown size/duration between events
  • 81. The Primary LSTM Gates Input, Forget, and Output gates (FIO) The main operational block of LSTMs an "information filter" a multivariate input gate some inputs are blocked some inputs "go through" "remember" necessary information FIO gates use the sigmoid activation function The cell state uses the tanh activation function
  • 82. LSTM Gates and Cell State  1) the input gate controls: the extent to which a new value flows into the cell state  2) the forget gate controls: the extent to which a value remains in the cell state  3) the output gate controls: The portion of the cell state that’s part of the output  4) the cell state maintains long-term memory  => LSTMs store more memory than basic RNN cells
  • 83. Common LSTM Variables  # LSTM unrolled through 28 time steps (with MNIST):  time_steps = 28  # number of hidden LSTM units:  num_units = 128  # number of rows of 28 pixels:  n_inputs = 28  # number of pixels per row:  input_size = 28  # mnist has 10 classes (0-9):  n_classes = 10  # size of input batch:  batch_size = 128
  • 86. TF 2/Keras and LSTMs  import tensorflow as tf  . . . model = tf.keras.models.Sequential() model.add(tf.keras.layers.LSTMCell(6,batch_input_shape=(1,1,1),kernel_i nitializer='ones',stateful=True)) model.add(tf.keras.layers.Dense(1)) . . . => LSTM versus LSTMCell: https://stackoverflow.com/questions/48187283/whats-the-difference- between-lstm-and-lstmcell => How to define a custom LSTM cell: https://stackoverflow.com/questions/54231440/define-custom-lstm-cell-in- keras
  • 87. TF 2/Keras and LSTM/RNN/GRU  import tensorflow as tf  ...  model = Sequential()  model.add(tf.keras.layers.LSTM(64, return_sequences=True, input_shape=(10, 64)))  model.add(tf.keras.layers.SimpleRNN(32, return_sequences=True))  model.add(tf.keras.layers.GRU(10, ...)  ...
  • 88. TF 2/Keras & BiDirectional LSTMs  import tensorflow as tf  . . .  model = Sequential()  model.add(Bidirectional(LSTM(10, return_sequences=True), input_shape=(5,10)))  model.add(Bidirectional(LSTM(10)))  model.add(Dense(5))  model.add(Activation('softmax'))  model.compile(loss='categorical_crossentropy', optimizer='rmsprop')
  • 91. AEs, VAEs, and GANs  https://jaan.io/what-is-variational-autoencoder-vae-tutorial/  TensorFlow 2 and Autoencoders: https://gist.github.com/AFAgarap/326af55e36be0529c507f1599f88c06e  TensorFlow 2 and Variational Autoencoders:  Convolutional Variational Autoencoder: https://www.tensorflow.org/alpha/tutorials/generative/cvae  Variational Autoencoders and GANS: https://www.youtube.com/watch?v=KFX4apL7a4s
  • 96. Reinforcement Learning  Dopamine on TensorFlow (Research Framework): https://github.com/google/dopamine  TF-Agents library for RL in TensorFlow: https://github.com/tensorflow/agents  Keras and Reinforcement Learning: https://github.com/keras-rl/keras-rl  OpenAI universe, DeepMind Lab, TensorForce
  • 97. About Me: Recent Books 1)TensorFlow 2 Pocket Primer (2019) 2) TensorFlow 1.x Pocket Primer (2019) 3) Python for TensorFlow (2019) 4) C Programming Pocket Primer (2019) 5) RegEx Pocket Primer (2018) 6) Data Cleaning Pocket Primer (2018) 7) Angular Pocket Primer (2017) 8) Android Pocket Primer (2017) 9) CSS3 Pocket Primer (2016) 10) SVG Pocket Primer (2016) 11) Python Pocket Primer (2015) 12) D3 Pocket Primer (2015) 13) HTML5 Mobile Pocket Primer (2014)
  • 98. What I do (Training) Instructor at UCSC (Santa Clara): Machine Learning (Python) (06/18/2019) 10 weeks Deep Learning (Keras) (07/10/2019) 10 weeks Deep Learning (Keras) (07/17/2019) 10 weeks Machine Learning (Python) (09/09/2019) 10 weeks DL, NLP, Deep RL (Adv Topics) (10/02/2019) 10 weeks Deep Learning (Keras/TF 2) (10/04/2019) 10 weeks UCSC link: https://www.ucsc-extension.edu/certificate-program/offering/deep- learning-and-artificial-intelligence-tensorflow