by Sunil Mallya, Startup Solutions Architect, AWS

1. © 2015, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Sunil Mallya, Startup Solutions Architect
January 19, 2017
MXNet + Lambda

2. AI Pipeline
• Data storage
• Pre-processing
• Feature extraction
• Model generation
• Model Analysis
• Feature engineering
• Evaluation and feedback

3. Continuous
scaling
No servers to
manage
Never pay for idle
– no cold servers
(only happy
accountants)
Benefits of AWS Lambda

4. Scaling Predictions with AWS Lambda
import mxnet as mx
bucket = 'my-model-bucket’
s3_client = boto3.client('s3')
f_params = 'resnet-18-0000.params'
f_symbol = 'resnet-18-symbol.json'
#params
f_params_file = tempfile.NamedTemporaryFile()
s3_client.download_file(bucket, f_params, f_params_file.name)
#symbol
f_symbol_file = tempfile.NamedTemporaryFile()
s3_client.download_file(bucket, f_symbol, f_symbol_file.name)
def lambda_handler(event, context):
model = load_model(f_params_file, f_symbol_file)
outside the context

5. SAM deployment
A common language to
describe the content of
a serverless
application across the
ecosystem.
Apache 2.0 licensed
GitHub project

6. SAM deployment example
aws cloudformation package
--template-file template.yaml
--output-template-file template-out.yaml
--s3-bucket <your-s3-bucket-name>
aws cloudformation deploy
--template-file <path-to-file/template-
out.yaml
--stack-name <STACK_NAME>
--capabilities CAPABILITY_IAM

7. DEMO
• The code does the following
• Load the model – RESNET
• Download the image
• Predict the labels
• Lets predict labels for

8. Is it ready for production?
$ wrk -t40 -c120 -d60s http://my_prediction_endpoint
Thread Stats Avg Stdev Max +/- Stdev
Latency 1.18s 96.82ms 1.98s 81.23%
Req/Sec 2.39 2.87 10.00 87.91%
4520 requests in 1.00m, 3.16MB read
Socket errors: connect 0, read 0, write 0, timeout 0
Requests/sec: 75.23
Transfer/sec: 53.85KB

9. Global latency for prediction
application
hosted
in Oregon

10. Thank You!
https://github.com/awslabs/mxnet-lambda
smallya@amazon.com