Presentation covered building a signature verification system and deploying it to production. This includes resources usage as well as how the model was picked. Meetup held in Tokyo with Deep learning Otemachi.

1. Deploying Signature Verification
Deep Learning Otemachi
June 2018
1

2. Overview
• Skymind Overview
• Task Problem Statement
• Model Overview
• Infra/Hardware requirements
• Model Rotation and Maintenance
• Questions

3. DL4J
• Inference focused
• Import keras/tf/onnx
• JVM Cupy
• Persistable ETL
• Enterprise focus rather
than research (never
intended to be
pytorch/tf/chainer!)

4. We build SKIL

5. Problem Overview: Sig Verification

6. Model Overview
• Transfer learning helps domain adaptation and
sparse datasets (e.g. small number of
signatures)
• Per-person models allow us to make
assumptions for individuals when needed
• Implementation: Adapting papers!
• Easy to troubleshoot
• Simpler Model: Predictable resource usage on
feed forward networks

7. Important Notes on Accuracy
• Err on assuming wrong
• False Positive and Negative matters a lot
• Generalization with 60-70% accuracy is better
than perfection overfitting
• Decision Thresholds for accuracy should be
tuned relative to this assumption

8. Findings
• Besides public data, collected data from real
people
• Trained on a mix of signatures intended to be
fraudulent and other people’s
• Near perfect on person with neat signature
• Manages 70-80% F1 on people with semi clean
signatures

9. How Model was evaluated
• Trained on 4-6 examples each with early
stopping using transfer learning + 2 class
softmax
• 2-3 test samples in test set
• 2-3 in validation set (usually fraudulent)
• F1 scores based on validation set

10. Infra/Hardware requirements
• Maintain the base model separately
• Track individuals’ signatures and their accuracy
• When onboarding new signatures, use
hyperparameter search to autotrain new models
• Use decision learner and retraining to handle
individual cases
• Occasionally retrain base model when enough
new data comes in

11. Memory Usage (Training)
Model Class: org.deeplearning4j.nn.conf.MultiLayerConfiguration
Model Name: MultiLayerNetwork
Network Input: [InputTypeFeedForward(0)]
# Layers: 20
Layer Types: 4 x ConvolutionLayer, 5 x ActivationLayer, 3 x BatchNormalization, 4 x
DenseLayer, 3 x SubsamplingLayer, 1 x OutputLayer,
Inference Memory (FP32) 185,073,220 bytes + nExamples * 9,964,544 bytes
Training Memory (FP32): 532,637,644 bytes + nExamples * 35,671,040 bytes
Inference Memory Breakdown (FP32):
- PARAMETERS 177,546,820 bytes
- ACTIVATIONS nExamples * 9,964,544 bytes
- WORKING_MEMORY_VARIABLE 7,526,400 bytes
Training Memory Breakdown (CacheMode = NONE, FP32):
- PARAMETERS 177,546,820 bytes
- PARAMATER_GRADIENTS 177,546,820 bytes
- ACTIVATIONS nExamples * 9,964,544 bytes
- ACTIVATION_GRADIENTS nExamples * 10,655,744 bytes
- UPDATER_STATE 177,541,956 bytes
- WORKING_MEMORY_FIXED 2,048 bytes
- WORKING_MEMORY_VARIABLE nExamples * 15,052,800 bytes

12. Training
Training Memory (FP32): 532,637,644 bytes + nExamples * 35,671,040 bytes
Training Memory Breakdown (CacheMode = NONE, FP32):
- PARAMETERS 177,546,820 bytes
- PARAMATER_GRADIENTS 177,546,820 bytes
- ACTIVATIONS nExamples * 9,964,544 bytes
- ACTIVATION_GRADIENTS nExamples * 10,655,744 bytes
- UPDATER_STATE 177,541,956 bytes
- WORKING_MEMORY_FIXED 2,048 bytes
- WORKING_MEMORY_VARIABLE nExamples * 15,052,800 bytes

13. Memory Usage (Inference)
Inference Memory (FP32) 185,073,220 bytes + nExamples *
9,964,544 bytes
Inference Memory Breakdown (FP32):
- PARAMETERS 177,546,820 bytes
- ACTIVATIONS nExamples * 9,964,544 bytes
- WORKING_MEMORY_VARIABLE 7,526,400 bytes

14. Transactional Throughput
• ETL: Load from Disk/ Vector Creation
• Iteration time: Forward/Backward (update neural net)
• Samples/Second: Number of images processed /sec
• Batches/Second: Number of minibatches per second

15. Some Visualizations
5G of Heap Allocated
3g of memory on gpu
for neural net

16. Some Takeaways
• 224.931396 MB for inference with batch size of
4
• This is per model
• Model Rotation is required
• Python hogs all memory by default. You need a
platform to even manage memory properly.

17. Some Takeaways
• Timings matter for training and inference
• Constant monitoring is needed to ensure SLA
• Able to forecast job completion time
• Disk hardware and data pipeline matters
(load/transform data)

18. Thank You
18