딥러닝으로 학습된 모델을 SAM CLI를 이용하여 로컬 테스트, AWS Lambda로 배포하여 Sentiment Analysis API를 제공하는 방법을 공유합니다 :) -- 개발자를 위한 쿨하고 섹시한 봇 플랫폼 https://bothub.studio

1. Serving Sentiment Analysis API
on AWS Lambda
feat. SAM CLI & BotHub.Studio
2018.07.26
anson@deepnatural.io

2. 딥러닝으로 학습시킨 모델을 ✍
SAM으로 로컬 테스트하고 Lambda에 올린 후 ❤
BotHub.Studio로 빌드된 ChatBot에 적용 👍

3. 왜 Lambda에?

4. 딥러닝으로 학습시킨 다양한 자연언어처리 관련 모델들을 API로 구성
NLP as (Micro) Service
의도 분류, 감정 분석, 정보 추출, 응답 생성 등 다양한 모델들
수많은 언어들
딥러닝으로 새로운 모델을 만들때마다 Lambda Function 하나씩 추가하면 어떨까

5. Disk & Memory Limit
경험해보니 Lambda가 가진 제약이 있어..
😢

6. SageMaker를 추천합니다

7. https://www.youtube.com/watch?v=tSFF0osd7ZM

8. 하지만,
많은 EC2 Instance를 생성 / 관리하고, 또 비용이 우려된다면
Lambda도 괜찮은 방법이 아닐까(?)

9. 1. 딥러닝으로 감정 분석기 만들기

10. I love you, baby.
😄
😡

11. I don’t like you.
😄
😡

12. 딥러닝으로 감정 분석기 만들기
• IMDB Movie Review Corpus (EN)
• LSTM RNN
• Tensorflow 1.8
• Python 3.6
• Jupyter Notebook
• 개발은 MacAir에서 하고
트레이닝은 AWS GPU Instance에서
• p2.xlarge ( $0.9 / hr, Oregon )

13. 딥러닝으로 감정 분석기 만들기
IMDB Movie
Review Corpus
Coding, Preprocessing
Upload code & data
Training Model
Download trained model

14. 2. SAM으로 로컬 테스트 및 배포

15. https://github.com/awslabs/aws-sam-cli

16. $ pip install aws-sam-cli
$ sam --version
SAM CLI, version 0.5.0
$ sam init --runtime python3.6 --name sentiment
[+] Initializing project structure...
[SUCCESS] - Read sentiment/README.md for further instructions on how to proceed
[*] Project initialization is now complete
$ ls -l
README.md
hello_world/
requirements.txt
template.yaml
tests/
$ sam local start-api
2018-07-26 08:32:33 Mounting SentimentFunction at http://127.0.0.1:3000/sentiment [GET]
2018-07-26 08:32:33 You can now browse to the above endpoints to invoke your functions. You do not need to
restart/reload SAM CLI while working on your functions changes will be reflected instantly/automatically. You only need
to restart SAM CLI if you update your AWS SAM template
2018-07-26 08:32:33 * Running on http://127.0.0.1:3000/ (Press CTRL+C to quit)

17. 람다는 Amazon Linux 위에서 동작
시스템 의존성이 있는 라이브러리를 이용한다면 Amazon Linux에 맞게 파이썬 패키지를 세
팅해야 함
Tensorflow가 바로 그 의존적인 녀석 😔
Docker Amazon Linux 이미지를 받고 그 안에서 파이썬 패키지 준비

18. 람다는 Amazon Linux 위에서 동작
시스템 의존성이 있는 라이브러리를 이용한다면 Amazon Linux에 맞게 파이썬 패키지를 세팅해야 함
Tensorflow가 바로 그 의존적인 녀석 😔
Docker Amazon Linux 이미지를 받고 그 안에서 파이썬 패키지 준비
pip install tensorflow==1.8.0
sam deploy

19. $ docker run -v $(pwd):/outputs --name lambdapack -d amazonlinux:latest tail -f /dev/null
$ docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
493d257a3748 amazonlinux:latest "tail -f /dev/null" 6 hours ago Up 5 hours lambdapack

20. # buildPack.sh
dev_install () {
yum -y update
yum -y upgrade
yum install -y
wget
gcc
gcc-c++
python36-devel
python36-virtualenv
python36-pip
findutils
zlib-devel
zip
}
pip_rasterio () {
cd /home/
rm -rf env
python3 -m virtualenv env --python=python3
source env/bin/activate
cat $VIRTUAL_ENV/pip.conf
pip install tensorflow==1.8.0
deactivate
}
rm -rf lambdapack
mkdir lambdapack
cd lambdapack
cp -R /home/env/lib/python3.6/site-packages/* .
cp -R /home/env/lib64/python3.6/site-packages/* .
echo "original size $(du -sh /home/lambdapack | cut -f1)"
# cleaning libs
rm -rf external
find . -type d -name "tests" -exec rm -rf {} +
# cleaning
find -name "*.so" | xargs strip
find -name "*.so.*" | xargs strip
rm -r pip
rm -r pip-*
rm -r wheel
rm -r wheel-*
rm easy_install.py
find . -name *.pyc -delete
echo "stripped size $(du -sh /home/lambdapack | cut -f1)"
# compressing
zip -FS -r1 /outputs/pack.zip * > /dev/null
echo "compressed size $(du -sh /outputs/pack.zip | cut -f1)"
}
main () {
dev_install
아래 블로그를 참조했습니다 :)
https://beomi.github.io/2017/12/07/Deploy-Tensorflow-Keras-on-AWS-Lambda/

23. SAM 프로젝트 build 디렉토리에 설치된 패키지 복사
없어도 되는 패키지 및 서브 모듈 삭제
$ ls -al sentiment/build
total 128
drwxr-xr-x 53 sangwon staff 1.7K Jul 26 13:22 ./
drwxr-xr-x 7 sangwon staff 224B Jul 26 13:21 ../
drwxr-xr-x 11 sangwon staff 352B Jul 26 12:45 Markdown-2.6.11.dist-info/
drwxr-xr-x 10 sangwon staff 320B Jul 26 12:45 Werkzeug-0.14.1.dist-info/
-rw-r--r-- 1 sangwon staff 0B Jul 26 13:23 __init__.py
drwxr-xr-x 5 sangwon staff 160B Jul 26 12:45 __pycache__/
drwxr-xr-x 10 sangwon staff 320B Jul 26 12:45 absl/
drwxr-xr-x 9 sangwon staff 288B Jul 26 12:45 absl_py-0.3.0.dist-info/
-rw-r--r-- 1 sangwon staff 725B Jul 26 13:23 app.py
drwxr-xr-x 13 sangwon staff 416B Jul 26 12:45 astor/
drwxr-xr-x 10 sangwon staff 320B Jul 26 12:45 astor-0.7.1.dist-info/
drwxr-xr-x 8 sangwon staff 256B Jul 26 12:45 bleach/
drwxr-xr-x 9 sangwon staff 288B Jul 26 12:45 bleach-1.5.0.dist-info/
drwxr-xr-x 7 sangwon staff 224B Jul 26 12:45 certifi/
drwxr-xr-x 10 sangwon staff 320B Jul 26 12:45 certifi-2018.4.16.dist-info/
drwxr-xr-x 43 sangwon staff 1.3K Jul 26 12:45 chardet/
drwxr-xr-x 10 sangwon staff 320B Jul 26 12:45 chardet-3.0.4.dist-info/
drwxr-xr-x 5 sangwon staff 160B Jul 26 12:45 data/
-rw-r--r-- 1 sangwon staff 126B Jul 26 12:45 easy_install.py
drwxr-xr-x 41 sangwon staff 1.3K Jul 26 12:45 external/
drwxr-xr-x 8 sangwon staff 256B Jul 26 12:45 gast/
drwxr-xr-x 9 sangwon staff 288B Jul 26 12:45 gast-0.2.0.dist-info/
drwxr-xr-x 3 sangwon staff 96B Jul 26 12:45 google/
drwxr-xr-x 16 sangwon staff 512B Jul 26 12:45 grpc/
drwxr-xr-x 9 sangwon staff 288B Jul 26 12:45 grpcio-1.13.0.dist-info/
drwxr-xr-x 17 sangwon staff 544B Jul 26 12:45 html5lib/
drwxr-xr-x 9 sangwon staff 288B Jul 26 12:45 html5lib-0.9999999.dist-info/
drwxr-xr-x 11 sangwon staff 352B Jul 26 12:45 idna/
drwxr-xr-x 9 sangwon staff 288B Jul 26 12:45 idna-2.6.dist-info/
drwxr-xr-x 16 sangwon staff 512B Jul 26 12:45 markdown/
drwxr-xr-x 32 sangwon staff 1.0K Jul 26 13:22 numpy/
drwxr-xr-x 8 sangwon staff 256B Jul 26 12:45 numpy-1.15.0.dist-info/
drwxr-xr-x 7 sangwon staff 224B Jul 26 12:45 pkg_resources/
-rw-r--r-- 1 sangwon staff 539B Jul 26 13:22 protobuf-3.6.0-py3.6-nspkg.pth
drwxr-xr-x 8 sangwon staff 256B Jul 26 12:45 protobuf-3.6.0.dist-info/
drwxr-xr-x 21 sangwon staff 672B Jul 26 12:45 requests/
drwxr-xr-x 9 sangwon staff 288B Jul 26 12:45 requests-2.18.4.dist-info/
-rw-r--r-- 1 sangwon staff 8.8K Jul 26 13:23 sentiment_rnn.py
drwxr-xr-x 42 sangwon staff 1.3K Jul 26 12:45 setuptools/
drwxr-xr-x 11 sangwon staff 352B Jul 26 12:45 setuptools-40.0.0.dist-info/
drwxr-xr-x 9 sangwon staff 288B Jul 26 12:45 six-1.11.0.dist-info/
-rw-r--r-- 1 sangwon staff 30K Jul 26 13:22 six.py
drwxr-xr-x 18 sangwon staff 576B Jul 26 12:45 tensorboard/
drwxr-xr-x 9 sangwon staff 288B Jul 26 12:45 tensorboard-1.8.0.dist-info/
drwxr-xr-x 11 sangwon staff 352B Jul 26 12:45 tensorflow/
drwxr-xr-x 10 sangwon staff 320B Jul 26 12:45 tensorflow-1.8.0.dist-info/
drwxr-xr-x 9 sangwon staff 288B Jul 26 12:45 termcolor-1.1.0.dist-info/
-rw-r--r-- 1 sangwon staff 4.9K Jul 26 13:23 termcolor.py
drwxr-xr-x 16 sangwon staff 512B Jul 26 12:45 urllib3/
drwxr-xr-x 9 sangwon staff 288B Jul 26 12:45 urllib3-1.22.dist-info/
drwxr-xr-x 28 sangwon staff 896B Jul 26 12:45 werkzeug/
drwxr-xr-x 17 sangwon staff 544B Jul 26 12:45 wheel/
drwxr-xr-x 9 sangwon staff 288B Jul 26 12:45 wheel-0.31.1.dist-info/
$ ls -al lib
__pycache__/
absl/
absl_py-0.3.0.dist-info/
google/
numpy/
numpy-1.15.0.dist-info/
pkg_resources/
tensorflow/
tensorflow-1.8.0.dist-info/

24. du -hs data/checkpoints data/vocab2id.pkl sentiment/build
261M data/checkpoints
1000K data/vocab2id.pkl
209M sentiment/build
500M가 넘는 모델은 람다로 올리는 것이 불가능
Disk & Memory Limit 😱
Code: Deployment Package
Data: S3 -> /tmp
Lambda 첫 호출, 초기화 로직에서 /tmp 디렉토리로 다운로드

25. template.yaml 😇

26. # 로컬 테스트
$ sam local start-api
# AWS 람다로 배포
$ sam package …
$ sam deploy …

27. POST
{
"message": "Hi there! How's your day? It's very wonderful today!"
}
Response
{
"sentiment": 0.9833118915557861
}
POST
{
"message": "I didn't like it. The movie was so boring. Terrible.."
}
Response
{
"sentiment": 0.2814910411834717
}
😃
😡

28. 3. BotHub.Studio에서 Bot과 연동

29. 개발자를 위한 쿨하고 섹시한 봇 개발 플랫폼 - BotHub.Studio

31. I love you ❤
0.82342
I love you ❤
😻
BotHub.Studio

32. 감사합니다.
anson@deepnatural.io
https://bothub.studio