Deep Learning for Developers: An Introduction, Featuring Samsung SDS (AIM301-R1) - AWS re:Invent 2018

1. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Deep Learning for Developers: An Introduction Hagay Lupesko Engineering Leader – Deep Learning Amazon Web Services A I M 3 0 1 - R Seungjai Min Samsung SDS

2. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Agenda • Deep learning is a big deal! • Brief introduction to deep learning • Starting to code neural networks • Cardiac arrhythmia detection with deep learning, by Samsung SDS By the end of this session, you will understand what deep learning is, and how you can start using it today!

4. “Deep Learning is a superpower. With it you can make a computer see, synthesize novel art, translate language ... If that isn’t a superpower, I don’t know what is.” – Andrew Ng

5. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Artificial intelligence Machine learning Deep learning Can machines think? Can machines do what we can? (Turing, 1950) Machine Learning Data Answers Rules Traditional programming Data Rules Answers

11. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Inspired by the brain’s neurons We have ~100B of them, and ~1Q synapses ANN is a simple computation construct: w1 w2 wn x1 x2 xn Σ φ 𝑦 … 𝑦 = 𝜑(෍ 𝑗=1 𝑛 𝑤𝑗 𝑥𝑗) Artificial neurons

12. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Fully connected layer Networks are comprised of stacked layers (mostly) Composed of artificial neurons, each fully connected to the previous layer, and to the next Connections have an associated “weight” learned during training Fully connected layer

13. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Convolution layer Operator is “sliding” a kernel across the input, computing value at each position in the output tensor The kernel values are the learned parameters of the layer Convolution layers are effective in perceptual problems involving visual data

14. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Convolution layer When combined across multiple layers, convolution is able to learn hierarchical features, with increasing levels of abstraction Layer 1 – Edges Layer 2 – Curves Layer N – Classes

15. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Recurrent neural network layer In some problems, predicting a given point in time depends on input context from an earlier time RNN layer “remembers the past” – using loops! RNNs are effective for time series and natural language problems A X y

16. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Recurrent neural network layer We can “unroll” an RNN layer to understand how it works A x0 h0 A x1 h1 A x2 h2 A xt ht …A X y

17. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Neural network Output layer Input layer Hidden layers ManyMore… • Non-linear • Hierarchical feature learning • Scalable architecture • Computationally intensive

18. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Forward pass Backwards pass Input Data Neural Network Output Loss Back Propagate Update Weights Forward-backward repeats across multiple epochs, each epoch goes through the entire training dataset. Training neural networks

22. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Building a neural network for image classification from mxnet.gluon import nn net = nn.Sequential() net.add(nn.Conv2D(channels=6, kernel_size=5, activation='relu'), nn.MaxPool2D(pool_size=2, strides=2), nn.Conv2D(channels=16, kernel_size=3, activation='relu'), nn.MaxPool2D(pool_size=2, strides=2), nn.Flatten(), nn.Dense(120, activation="relu"), nn.Dense(84, activation="relu"), nn.Dense(10))

23. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Training a neural network for image classification loss_function = gluon.loss.SoftmaxCrossEntropyLoss() trainer = gluon.Trainer(net.collect_params(), 'sgd', {'learning_rate': 0.1}) for epoch in range(num_epochs): for X, Y in train_data: with autograd.record(): Y_hat = net(X) loss = loss_function(Y_hat, Y) loss.backward() trainer.step(batch_size)

25. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Using a pre-trained model # Loading a pre-trained model from mxnet.gluon.model_zoo import vision model = vision.resnet18_v1(pretrained=True) # Invoking inference on an input tensor x output = model(x) # Computing probabilities, and extracting the top 5 classes output.softmax().topk(k=5)

26. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Deploying a trained model Deploying trained models Amazon SageMaker – Cloud AWS Greengrass – IoT Amazon EC2 Amazon ECS Mobile or Edge devices Your own solution

27. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Inference on non-Python stacks // Set the CPU context List<Context> context = Arrays.asList(Context.cpu()); // Initialize the input descriptor List<DataDesc> inputDesc = Arrays.asList( new DataDesc("data", new Shape(new int[]{1, 3, 224, 224}), DType.Float32(), "NCHW")); // Initialize the model predictor Predictor predictor = new Predictor(app.modelPathPrefix, inputDesc, context, 0); // Prepare input image BufferedImage img = resizeImage(loadImageFromFile(app.inputImagePath), 224, 224); // predict float[][] result = predictor.predict(new float[][]{imagePreprocess(img)});

33. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. https://www.mayoclinic.org/tests-procedures/ekg/care-at-mayo-clinic/pcc-20384985 Need for AI in cardiac arrhythmia detection • Challenges • Over 300 million ECGs are recorded annually • One-day record has 80K beats / person • Benefits of applying AI • Expert clinicians and cardiologists can inspect only 0%~2.5% beats that are marked by AI • Save clinicians and cardiologists time • Increase the accuracy of diagnoses

34. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Challenges for traditional ML (1) Ventricular tachycardia from MIT-BIH 205 Atrial fibrillation from MIT-BIH 219 There are 40 different types of arrhythmia

35. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Challenges for traditional ML (2) Each person has his/her own normal heart beat pattern Normal beat Inverted T (normal but could be hyperkalemia) Large T (normal but could be coronary artery disease)

37. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Sequence learning using RNN Supervised learning x1 x2 xN Features (CNN result) …. RNN input gru2_1 gru1_1 gru1_2 gru1_n gru2_2 gru2_n yN y1 merge … Bi-GRU Forward Backward … Classifier

40. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Our initial efforts Enterprise solution requirements for low cost  No GPU for inference AWS instance type inference time GPU p2.8xlarge (4 GPUs) 20 mins p2.xlarge (1 GPU) 80 mins CPU c5.4xlarge (16 vCPUs) 120 mins Reduce inference time & maintain the target accuracy!

41. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Inference optimization 0.5 0.6 0.7 0.8 0.9 1 1.1 0 20 40 60 80 100 120 1 2 3 4 5 6 7 8 RelativeAccuracy Latency(mins) Series2 Series1 c5.2xlarge No. vCPUs 16 16 8 8 8 8 8 8 Algorithm Bi-GRU Bi-GRU Bi-GRU Fused RNN (Bi-GRU) Fused RNN (Bi-GRU) Fused RNN (Bi-RNN) Fused RNN (Bi-RNN) Fused RNN (Bi-RNN) Sample Size 256 128 128 128 128 128 128 64 MKL - - V V V V V V Optimization Bucketing Bucketing Bucketing Auto Data Balancing Bucketing Auto Data Balancing

44. Thank you! © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Hagay Lupesko Engineering Leader – Deep Learning Amazon Web Services Seungjai Min Head of Data Analytics Samsung SDS Meet & Greet: Come and meet us in person Aria East, Level 1, Willow Lounge

45. Time: 15 minutes after this session Location: Speaker Lounge (ARIA East, Level 1, Willow Lounge) Duration: 30 min.