Data scientists want Python for experimentation, engineers want production-gradesystems. This can create friction between departments and often leads to suboptimal solutions. In this talk we show how to access Deeplearning4J (DL4J) directly from Python, and discuss how to import some of your favorite frameworks into DL4J. This approach narrows the gap between science and engineering and brings Deep Learning models to production more easily. We close by giving a demo of real-time object detection with YOLO, using Skymind's intelligence layer (SKIL).

1. Max Pumperla Version 1.0
Snakes on a plane
Ship your Python on
enterprise machines

2. Skymind overview
● Deep learning for enterprise
● Globally distributed, remote
● Creators of Deeplearning4j (DL4J)
● 300k+ downloads per month

3. ● Python lead @ Skymind
● Chronic NIH syndrome
● DL4J & Keras core dev team
● Author of Elephas
● Coursera instructor
● Author of “Deep Learning and the
Game of Go”
About me

4. Python on Enterprise machines
I’ve had it with these $^!@# snakes on this #*$%* plane.
● Not what you expect to see
● Fun to watch (even if ironically)
● Potentially dangerous to everyone involved
● Heroes can pull it off

5. Science
Model handover
Engineering
● Scripted: Python, R, Julia, ...
● Notebooks / Interactivity
● Fast experimentation cycle
● Reproducibility
● Collaboration
● Typed: Java, C/C++, ...
● Distributed storage & training
● Real-time / streaming data
● Low latency predictions
● Fault tolerance
● Keep model in prod and up-to-date
● Updates, rollbacks etc.
The Real Language Wars ™
unfolding right here

6. But my shop is Python only!
● Lucky you
● Can be slow (beware of the GIL)
● Watch memory consumption
● Limited data access*
● Managing large projects is tough

7. Rest of the world
● Reimplement your model? Wait, your whole pipeline!
● Own (micro-)service?
● Containerize?
What about all the other requirements?

8. Deep Learning at Skymind
Deeplearning4j, ScalNet
Build, train, and deploy neural networks on
JVM and in Spark.
ND4J
High performance linear algebra on
GPU/CPU. Numpy for JVM.
SameDiff
Symbolic differentiation and computation
graphs.
Arbiter
Hyperparameter search for optimizing
neural networks.
Model Import
Import neural nets from TensorFlow,
Keras (Theano, Caffe), ONNX.
Jumpy
Python API for ND4J.

9. Keras Model import
● Active Keras contributors*
● 95% import coverage
● 95% DL4J feature parity
● One-liner to import models
● Can import Keras preprocessing
ComputationGraph network =
KerasModelImport.importKerasModelAndWeights(file)

10. TensorFlow Model import
● 70% import coverage
● One-liner to import models
● All major applications*
TFGraphMapper.getInstance().importGraph(file);

11. Jumpy and Numpy
● No overhead conversion (jnius)
● Go back and forth
● Works for Spark as well
import jumpy as jp
import numpy as np
x_np = np.random.random((100, 50))
x_jp = jp.array(x_np)
x_np = x_jp.numpy()

12. Parallel inference
● Multi-threading and auto-batching
● Optimal resource allocation

13. Elephas: Python interface for
distributed training

14. Distributed training on Spark
(Parameter sharing)

15. Integrated AI platform
SKIL

16. SKIL Python interface
skil_server = Skil()
work_space = WorkSpace(skil_server)
experiment = Experiment(work_space)
model = Model('model.h5', experiment)
deployment = Deployment(skil_server, "deployment")
service = model.deploy(deployment, scale=2)

17. Object Detection
● Decade ago: feature engineering
(‘member cascading Haar filters?… yikes)
● Today: dominated by deep learning
○ Sliding window DNNs
○ Faster R-CNN
○ YOLO
○ SSD

18. YOLO v3
“But maybe a better question is: “What are we going to do
with these detectors now that we have them?” A lot of the
people doing this research are at Google and Facebook. I
guess at least we know the technology is in good hands and
definitely won’t be used to harvest your personal
information and sell it to.... wait, you’re saying that’s exactly
what it will be used for?? Oh”
- Joseph Redmon

20. YOLO deployment and live
feed
in 12 lines of code
import skil
import cv2
skil_server = skil.Skil()
model = skil.Model('yolo_v2.pb', name='yolo-tf', model_id='yolo-3493723')
deployment = skil.Deployment(skil_server, 'yolo')
service = model.deploy(deployment, input_names=['input'], output_names=['output'], scale=2)
cap = cv2.VideoCapture(0)
while(True):
_, image = cap.read()
detection = service.detect_objects(image)
image = skil.utils.yolo.annotate_image(image, detection)
cv2.imshow('yolo', image)
Download first
SKIL knows your app is YOLO!

21. Live demos always go wrong, but
YOLO

22. Thank you! github.com/maxpumperla
twitter.com/maxpumperla