Deep learning and machine vision could diagnose cancer more accurately than people. We're training Inception-Resnet-v2, recently the world's most accurate classifier on Imagenet, in tensorflow with 600,000 images for this.

1. Mammography
with
Inception
Stephen Morrell
Bob Kemp
Karl Kalleberg
Zbigniew Wojna

2. The Use Case
• I believe AI will have a larger impact
than oil..
• Coming soon: (better) segmentation,
transfer learning, smaller datasets, 3D
4D, unsupervised learning, etc
• Breast cancer is the second most
common in the West after skin cancer
and a leading causes of death along
with cardiac events &strokes. All have
wide diagnostic funnels so suitable for
automation.
• Prevalence: 12% women in the US will
develop invasive breast cancer during
their lifetime.
• Low Accuracy in Mammograms
Technology is a gift of God. After the gift of life it is perhaps the greatest of
God's gifts. It is the mother of civilizations, of arts and of sciences.
Freeman Dyson

3. Date of download: 7/12/2016Source: 2016 American Medical Association
Mammography Accuracy

4. Histology & Micro-calcifications √ Masses X
Krupinski & Levenson, 2015
Pigeons Diagnosing Cancer

5. Alex Net

6. Image Net

7. A Karpathy on Imagenet: 5.1%
http://karpathy.github.io/2014/09/02/what-i-learned-from-competing-against-a-convnet-on-imagenet/

8. Dream Challenge
• Dataset 641k mammograms, 87k patients, panel data. Other
datasets for pre-training
• Infrastructure GPUs 2 x NVIDIA K80s
Docker
• Timing Competitive phase toMarch 2017 then community phase
• Project risks
1. Infrastructure & Network Development
2. Do mammograms contain enough info per se. Will they be superceded
3. Can we develop learning algorithms which classify accurately given limited
data. Dream has 0.5% = 3k labelled cancerous images.
• Opportunity familiarity with ConvNets & benchmarking. Tech could
be applied to other fields
• $1,000,000 prizes

9. Dream Challenge - Infrastructure

10. TensorFlow & Inception
Powerful Learning Framework
1. Gradient Descent and automated
differentiation
2. Regularisation: Weight decay, batch
normalisation, dropout
3. Analytics and visualisation with
Tensorboard:
a. Histograms of losses, activations,
gradients
b. Network visualisation
Fast Infrastructure
1. Data preprocessing infrastructure
2. Multi-GPUs and machines
3. Multi-threading
4. Hyperparameter tuning
a. Layer & activation speciMication
b. Learning rate schedules
c. Batch normalisation
d. etc
5. etc

11. # 1 Define Variables
weights = tf.Variable(tf.truncated_normal([IMAGE_PIXELS,
hidden1_units], stddev=1.0 / math.sqrt(float(IMAGE_PIXELS))),
name='weights‘)
biases = tf.Variable(tf.zeros([hidden1_units]), name='biases')
# 2 Hidden Layer Definition
hidden1 = tf.nn.relu(tf.matmul(images, weights) + biases)
logits = tf.matmul(hidden1, weights) + biases
# 3 Final Layers
cross_entropy = tf.nn.sparse_softmax_cross_entropy_with_logits(
logits, labels, name='xentropy')
loss = tf.reduce_mean(cross_entropy, name='xentropy_mean')
# 4 Add Training
train_op = optimizer.minimize(loss, global_step=global_step)
Graph Construction

12. # Initialise
with tf.Session() as sess:
init = tf.initialize_all_variables()
sess.run(init)
# Training Loop
for step in xrange(FLAGS.max_steps):
_, loss = sess.run(train_op, loss)
# Evaluation
true_count += sess.run(eval_correct, feed_dict=feed_dict)
Init, Train, Eval

13. GPU, Queue, Combine

14. • Benefits of TFRecords
• Starting Queuerunners
• Memory ‘leaks’
• Variable name clashes
• Dummy & simple networks
• Transfer learning
• Memory use scaling
Infrastructure Lessons

15. Questions