Describes the approach to automate the process of training and deploying a general chatbot in the cloud

1. TRAINING CHATBOT IN THE
CLOUD
ALEX YU

2. INTRODUCTION
• Code Pipeline
• Deep Learning
• Challenges
• Limitations
• Tips

3. CODEPIPELINE
Training Training Progress Chatbot

4. AWS SERVICES
• S3
• CodePipeline
• CodeBuild
• Cloudformation

5. NEURAL NETWORK
• Definition
• Input
• Hidden
• Output

6. NEURAL NETWORK

7. DEEP LEARNING
• Basic neural network
• Convolutional neural network: image recognition
• Recurrent neural network: translation

8. TRAINING PROCESS
• Cloudformation Stack
• Pull down source
• Setup Webserver
• Start Training
• Save Model
• Write progress to file, go back to Training
• Upload model file to S3

9. CHATBOT PROCESS
• Cloudformation Stack
• Pull down source
• Pull down model file from S3
• Setup webserver
• Start chatbot

10. CHALLENGES
• GPU
• Big model file
• Long time of training
• Chatbot requires less resources than training

11. BAKE AMI
• Instance Type: G2
• OS: Ubuntu 14.04
• Installing GPU:cuda7.5, cudnn5.0
• Deep learning frameworks: torch7, tensorflow, theano
• Dependencies: npm, pip, luarocks

12. MODEL FILE
• Massive training data
• Complex neural networks
• Memory

13. FRAMEWORK
• Torch (Deep Learning) + Python (API)
• Lutorpy
• Pros
• Performance: tensorflow, theano
• No compile time
• Facebook ,Twitter, Nvidia
• Cons
• Lua
• Less libraries

14. COST
• G2.2xlarge – 0.65$/hr
• G2.8xlarge X 4
• 50000 data entries, 1000 hidden cells, 50 epoch -> 3 days
• Training: G2
• Chatbot: M4

15. LIMITATIONS
• Can’t pass in variables to codebuild,
• Codebuild timeout 8 hrs max

16. TIPS
• Code pipeline
• Setup separate cloudformation template for training and chatbot
• Use pre-baked images
• Use nvdia-docker
• Torch7 faster performance
• Use Jenkins to build souce code, fetch training data
• Set the variable in pipeline template, pass it as environment variable for codebuild
• Use lambda function to get progress
• Use serverless.js to build code pipeline

17. QUESTIONS ?

18. THANK YOU
http://www.linkedin.com/in/alexz-yu