Slides for the paper titled "Verifiable and Energy Efficient Medical Image Analysis with Quantised Self-attentive Deep Neural Networks" presented at MICCAI 2022 FAIR Workshop, SIngapore. Paper Abstract: Convolutional Neural Networks have played a significant role in various medical imaging tasks like classification and segmentation. They provide state-of-the-art performance compared to classical image processing algorithms. However, the major downside of these methods is the high computational complexity, reliance on high-performance hardware like GPUs and the inherent black-box nature of the model. In this paper, we propose quantised stand-alone self-attention based models as an alternative to traditional CNNs. In the proposed class of networks, convolutional layers are replaced with stand-alone self-attention layers, and the network parameters are quantised after training. We experimentally validate the performance of our method on classification and segmentation tasks. We observe $50-80\%$ reduction in model size, $60-80\%$ lesser number of parameters, $40-85\%$ fewer FLOPs and $65-80\%$ more energy efficiency during inference on CPUs. The code will be available at \href {https://github.com/Rakshith2597/Quantised-Self-Attentive-Deep-Neural-Network}{https://github.com/Rakshith2597/Quantised-Self-Attentive-Deep-Neural-Network}.

1. Indian Institute of Technology Kharagpur
Kharagpur Learning, Imaging and Visualization Research Group
Verifiable and Energy Efficient Medical Image
Analysis with Quantised Self-attentive Deep
Neural Networks
Rakshith Sathish, Swanand Khare, and Debdoot Sheet

2. Indian Institute of Technology Kharagpur
Kharagpur Learning, Imaging and Visualization Research Group
Motivation
30 September 2022 Verifiable and Energy Efficient Medical Image Analysis with Quantised Self Attentive Deep Neural Networks | Rakshith Sathish 2
Image Sources:
[1] NIH-Chest Xray
dataset
[2] Med. Seg.
Decathlon
[3] link
[4] link
Detection/Classification [1] Segmentation [2]
Versatility
Architecture of residual network [3]
Model Complexity
NVIDIA GPU [4]
High-performance Hardware

3. Indian Institute of Technology Kharagpur
Kharagpur Learning, Imaging and Visualization Research Group
Challenges
30 September 2022 Verifiable and Energy Efficient Medical Image Analysis with Quantised Self Attentive Deep Neural Networks | Rakshith Sathish 3
No. of parameters,
FLOPs, etc Drop in performance
Model Complexity
High Performance Hardware
Image Source:

4. Indian Institute of Technology Kharagpur
Kharagpur Learning, Imaging and Visualization Research Group
Our Approach
30 September 2022 Verifiable and Energy Efficient Medical Image Analysis with Quantised Self Attentive Deep Neural Networks | Rakshith Sathish 4

5. Indian Institute of Technology Kharagpur
Kharagpur Learning, Imaging and Visualization Research Group
Concat
𝐶𝑖𝑛 × 𝑀 × 𝑁
Stand Alone Self-Attention
30 September 2022 Verifiable and Energy Efficient Medical Image Analysis with Quantised Self Attentive Deep Neural Networks | Rakshith Sathish 5
𝐶𝑜𝑢𝑡 × 1 × 1
Output
𝐶𝑜𝑢𝑡/2 × ℎ ×w
𝐶𝑜𝑢𝑡/2 × ℎ ×w 𝐶𝑜𝑢𝑡/2 × ℎ ×w
𝐶𝑜𝑢𝑡/2 × ℎ ×w
𝐊1
𝐊2
𝐄𝒄𝒐𝒍
𝐄𝒓𝒐𝒘
Split
Query (𝐐)
𝐶𝑜𝑢𝑡 × 1 × 1
Keys (𝐊)
𝐶𝑜𝑢𝑡 × ℎ × 𝑤
Values (𝐕)
𝐶𝑜𝑢𝑡 × ℎ ×w
Softmax
1 × ℎ ×w
Conv2d
(𝐶
𝑜𝑢𝑡
×
1
×
1)
Conv2d
(𝐶
𝑜𝑢𝑡
×
1
×
1)
Conv2d
(𝐶
𝑜𝑢𝑡
×
1
×
1)
𝐶𝑜𝑢𝑡 × ℎ ×w
𝐊′
Ref: Ramachandran, P., Parmar, N., Vaswani, A., Bello, I., Levskaya, A., Shlens, J.:
Stand-alone self-attention in vision models

6. Indian Institute of Technology Kharagpur
Kharagpur Learning, Imaging and Visualization Research Group
Experiments
30 September 2022 Verifiable and Energy Efficient Medical Image Analysis with Quantised Self Attentive Deep Neural Networks | Rakshith Sathish 6
Classification Segmentation
• NIH Chest X-ray dataset of 14 Common
Thorax Disease
• Multi-Label classification
• 30,805 patients
Sample Chest X-Ray from NIH dataset.
• Liver subset of medical decathlon dataset.
• Multi-Label classification
• 30,805 patients
• 131 ground truth paired 3D CT volumes
A Sample pair from the dataset

7. Indian Institute of Technology Kharagpur
Kharagpur Learning, Imaging and Visualization Research Group
Network Architecture: Classification
30 September 2022 Verifiable and Energy Efficient Medical Image Analysis with Quantised Self Attentive Deep Neural Networks | Rakshith Sathish 7

8. Indian Institute of Technology Kharagpur
Kharagpur Learning, Imaging and Visualization Research Group
Network Architecture: Segmentation
30 September 2022 Verifiable and Energy Efficient Medical Image Analysis with Quantised Self Attentive Deep Neural Networks | Rakshith Sathish 8

9. Indian Institute of Technology Kharagpur
Kharagpur Learning, Imaging and Visualization Research Group
Quantitative Results
30 September 2022 Verifiable and Energy Efficient Medical Image Analysis with Quantised Self Attentive Deep Neural Networks | Rakshith Sathish 9
Model Accuracy
ResNet-18 0.89
q-ResNet-18 0.88
ResNet-50 0.84
q-ResNet-50 0.83
SaDNN-cls (ours) 0.90
q-SaDNN-cls (ours) 0.89
Model DSC
UNet-small 0.88
q-UNet-small 0.88
SUMNet 0.89
q-SUMNet 0.89
SaDNN-seg (ours) 0.88
q-SaDNN-seg (ours) 0.85
Classification Segmentation

10. Indian Institute of Technology Kharagpur
Kharagpur Learning, Imaging and Visualization Research Group
Qualitative Results: Segmentation
30 September 2022 Verifiable and Energy Efficient Medical Image Analysis with Quantised Self Attentive Deep Neural Networks | Rakshith Sathish 10

11. Indian Institute of Technology Kharagpur
Kharagpur Learning, Imaging and Visualization Research Group
Computational Analysis: Classification
30 September 2022 Verifiable and Energy Efficient Medical Image Analysis with Quantised Self Attentive Deep Neural Networks | Rakshith Sathish 11
58.59% smaller than q-ResNet 18
80.75% smaller than q-ResNet 50
Model Size
65.93% fewer than ResNet 18
85.32% fewer than ResNet 50
MACs
59.17% lesser than ResNet 18
80.62% lesser than ResNet 50
Parameters

12. Indian Institute of Technology Kharagpur
Kharagpur Learning, Imaging and Visualization Research Group
Computational Analysis: Segmentation
30 September 2022 Verifiable and Energy Efficient Medical Image Analysis with Quantised Self Attentive Deep Neural Networks | Rakshith Sathish 12
73.06% smaller than q-Unet-small
64.94% smaller than q-SUMNet
Model Size
73.06% fewer than q-Unet-small
64.94% fewer than q-SUMNet
MACs
74.37% lesser than q-Unet-small
66.21% lesser than q-SUMNet
Parameters

13. Indian Institute of Technology Kharagpur
Kharagpur Learning, Imaging and Visualization Research Group
Energy Profiling
30 September 2022 Verifiable and Energy Efficient Medical Image Analysis with Quantised Self Attentive Deep Neural Networks | Rakshith Sathish 13
Model Energy (J)
UNet-small 502.78
q-UNet-small 25.13
SUMNet 979.75
q-SUMNet 48.97
SaDNN-seg
(ours)
637.00
q-SaDNN-seg
(ours)
31.87
Model Energy (J)
ResNet-18 20.93
q-ResNet-18 1.04
ResNet-50 48.53
q-ResNet-50 2.41
SaDNN-cls
(ours)
7.13
q-SaDNN-cls
(ours)
0.35
Operation
Energy (pJ)
MUL ADD
8-bit INT 0.2 0.03
16-bit FP 1.1 0.40
32-bit FP 3.7 0.90
Ref: Horowitz, M.: 1.1 computing’s energy problem (and what we can do about it).
In: IEEE Int. Solid-State Circuits Conf. Digest of Technical Papers. pp. 10–14 (2014)
Limitations
• Theoretical
• Specific to 45nm chipset

14. Indian Institute of Technology Kharagpur
Kharagpur Learning, Imaging and Visualization Research Group
Conclusion
30 September 2022 Verifiable and Energy Efficient Medical Image Analysis with Quantised Self Attentive Deep Neural Networks | Rakshith Sathish 14
We proposed a class of quantised self-attentive neural networks which can be
used for medical image classification and segmentation.
Compared to traditional CNN architectures, the proposed network has
• Fewer parameters
• Less MACs
• Smaller in size
• Good performance
• No significant drop after quantisation