Oracle Code One 2018 San Francisco session

1. Machine Learning Applied -
Contextual Chatbots Coding,
Oracle JET andTensorFlow
Andrejus Baranovskis, CEO andTechnical Expert, Red Samurai Consulting
Oracle ACE Director and Oracle Groundbreaker Ambassador

2. Oracle ExpertsTeam
ADF, JET, ORACLE FUSION, ORACLE CLOUD, MACHINE LEARNING
Oracle PaaS Partner Community Award for Outstanding Java Cloud
Service Contribution 2017

3. Session Goal
HowTo BuildYour Own Machine Learning Chatbot

4. AGENDA
• Technical Architecture
• Solution WalkThrough
• Machine Learning Introduction
• Implementation Points

5. TECHNICAL ARCHITECTURE

6. Machine Learning Chatbot Context Communication Chatbot UI
Classification Chatbot messaging
Oracle JET

7. Chatbot Custom application logic
Generic listener
Oracle JET

8. CHATBOT CONTEXT
• Chatbot framework needs a structure in which conversational intents are
defined (this can be JSON file)
• Conversational intent contains:
• tag (unique name)
• patterns (sentence patterns for neural network text classifier)
• responses (one will be used as a response)

9. SOLUTION WALKTHROUGH

11. GENTLE INTRODUCTIONTO MACHINE
LEARNING

12. LEARNING AND INFERENCE
Training data Feature vector Learning algorithm Model
Test data Feature vector Model Prediction

13. KEY PARAMETERS
• Cost Function - score for each candidate parameter, shows sum of
errors in predicting.The higher the cost, the worse the model
parameters will be
• Epoch - each step of looping through all data to update the model
parameters
• Learning rate - the size of the learning step

14. REGRESSION
Regression
algorithm
Input Output
Continuous
Continuous
Discrete

15. REGRESSION EXAMPLE
w - parameter to be found usingTensorFlow

16. CLASSIFICATION
f{x}
Input Output
DiscreteContinuous
Discrete
Classifier

17. CLASSIFICATION EXAMPLE
Linear boundary line learned from the training data - equal probability for both groups

18. WHYTENSORFLOW?
• TensorFlow has become the tool of choice to implement machine
learning solutions
• Developed by Google and supported by its flourishing community
• Gives a way to easily implement industry-standard code

19. CLASSIFICATION IMPLEMENTATION

20. STEP 1: PREPARING DATA
• Tokenise patterns into array of words
• Lower case and stem all words. Example: Pharmacy => pharm.
Attempt to represent related words
• Create list of classes - intents
• Create list of documents - combination between list of patterns and
list of intents

21. STEP 1: PREPARING DATA

22. STEP 2: PREPARINGTENSORFLOW INPUT
• [X: [0, 0, 0, 1, 0, 1, 0, 1, 0, 0, 0, 0, 0, 1, ...N],Y: [0, 0, 1, 0, 0, 0, ...M]]
• [X: [0, 1, 0, 0, 0, 1, 0, 0, 0, 0, 1, 0, 1, 0, ...N],Y: [0, 0, 0, 1, 0, 0, ...M]]
• Array representing pattern with 0/1. N = vocabulary size. 1 when
word position in vocabulary is matching word from pattern
• Array representing intent with 0/1. M = number of intents. 1 when
intent position in list of intents/classes is matching current intent

23. STEP 2: PREPARINGTENSORFLOW INPUT

24. STEP 3:TRAINING NEURAL NETWORK
• Use tflearn - deep learning library featuring a higher-level API forTensorFlow
• Define X input shape - equal to word vocabulary size
• Define two layers with 8 hidden neurones - optimal for text classification task
(based on experiments)
• DefineY input shape - equal to number of intents
• Apply regression to find the best equation parameters

25. STEP 3:TRAINING NEURAL NETWORK
• Define Deep Neural Network model (DNN)
• Run model.fit to construct classification model. Provide X/Y inputs,
number of epochs and batch size
• Per each epoch, multiple operations are executed to find optimal
model parameters to classify future input converted to array of 0/1

26. STEP 3:TRAINING NEURAL NETWORK
• Batch size:
• Smaller batch size requires less memory. Especially important for datasets
with large vocabulary
• Typically networks train faster with smaller batches.Weights and network
parameters are updated after each propagation
• The smaller the batch the less accurate estimate of the gradient (function
which describes the data) could be

27. STEP 3:TRAINING NEURAL NETWORK

28. STEP 4: INITIAL MODELTESTING
• Tokenise input sentence - split it into array of words
• Create bag of words (array with 0/1) for the input sentence - array
equal to the size of vocabulary, with 1 for each word found in input
sentence
• Run model.predict with given bag of words array, this will return
probability for each intent

29. STEP 4: INITIAL MODELTESTING

30. STEP 5: REUSETRAINED MODEL
• For better reusability, it is recommended to create separate
TensorFlow notebook, to handle classification requests
• We can reuse previously created DNN model, by loading it with
TensorFlow pickle

31. STEP 5: REUSETRAINED MODEL

32. STEP 6:TEXT CLASSIFICATION
• Define REST interface, so that function will be accessible outside
TensorFlow
• Convert incoming sentence into bag of words array and run
model.predict
• Consider results with probability higher than 0.25 to filter noise
• Return multiple identified intents (if any), together with assigned probability

33. STEP 6:TEXT CLASSIFICATION

34. IMPLEMENTATION POINTS

44. QUESTIONS

45. CONTACTS
• Andrejus Baranovskis (https://andrejusb.blogspot.com)
• Email: abaranovskis@redsamuraiconsulting.com
• Twitter: @andrejusb
• LinkedIn: https://www.linkedin.com/in/andrejus-baranovskis-251b392
• Web: http://redsamuraiconsulting.com

46. REFERENCES
• Source Code - https://github.com/abaranovskis-redsamurai/shenzhen
• Contextual Chatbot inTensorFlow - https://bit.ly/2pFbTw4
• TensorFlow Book - http://tensorflowbook.com/