Attention Preservation in Network Compression for Reliable Network Interpretation

1. Attribution Preservation in Network Compression
for Reliable Network Interpretation
NeurIPS 2020
Geondo Park*, June Yong Yang*, Sung Ju Hwang, Eunho Yang
{geondopark, laoconeth, sjhwang82, eunhoy}@kaist.ac.kr
*Equal contribution.

2. Safety-Critical Deep Learning
Safety-critical applications require:
• Reliability: Must provide constant uptime.
• Cloud based models are unreliable due to communication failures.
• Need to embed models directly on edge devices.
• Network compression is needed to fit them inside smaller spaces.
• Trust: Provide explanations to its decisions.
• ‘Why’ did the network ran a red light?
• Explainable AI(XAI) algorithms are needed
2
Autonomous Driving X-ray Diagnosis

3. Network Compression
Network Compression
• Make the network consume less computational(time, space) cost while maintaining
its prediction power
• Why: Embedding DNNs directly in edge devices(reliability)
• Knowledge distillation, Network pruning, Weight Quantization, etc.
3

4. Explainable AI
Explainable AI(XAI)
• DNNs are practically black boxes
• Produce human-understandable interpretations of decisions and predictions of
neural networks
• Why: Trustworthiness, transparency
• Input attribution, interpretable models, etc.
41) Yulong Wang., Pytorch Visual Attribution 2018, github.com/yulongwang12/visual-attribution

5. Attribution Deformation Problem
Compression Breaks Attribution
• Compressed networks produce deformed attribution maps compared to their former
selves and the ground truth segmentations.
• Happens across various compression methods and attribution methods
5

6. Attribution Matching to Preserve Attribution
• While compressing, make the attribution map of the compressing network follow the
map of the pre-compression network
• Employ a matching loss to keep the maps of the compressing network close to the
pre-compression network
6

7. Weight Collapsed Attribution Matching
• Generate attribution maps by collapsing them in the channel dimension
7
Equally weighted Sensitivity weighted
⋅⋅⋅
1
1
1
1
𝐴1
𝐴2
𝐴 𝐶−1
𝐴 𝐶
⋅⋅⋅
𝐴1
𝐴2
𝐴 𝐶−1
𝐴 𝐶
𝑈1(0.04)
𝑈2(0.1)
𝑈 𝐶−1(0.6)
𝑈 𝐶(0.14)

8. Stochastic Matching
• Generate attribution maps by collapsing them in the channel dimension
8
Stochastic sensitivity weighted
⋅⋅⋅
𝐴1
𝐴2
𝐴 𝐶−1
𝐴 𝐶
𝑈1 ⋅ 𝑅1(0.04)
𝑈2 ⋅ 𝑅2 (0)
𝑈 𝐶−1 ⋅ 𝑅 𝑐−1
𝑈 𝐶 ⋅ 𝑅 𝐶 (0.14)

9. Knowledge Distillation
Basic knowledge distillation
• Naïve Knowledge Distillation causes attribution map distortion
• Our framework effectively preserves the attribution map similarly to the teacher,
which in turn preserves attribution against ground truth.
• Attribution preservation also helps in preserving the predictive performance.
9

10. Unstructured Pruning
Weight magnitude based pruning
• Since unstructured pruning is relatively tolerable, attribution distortion occurs less
than other compression methods.
• Similar phenomena were observed: attribution score and predictive performance is
preserved.
10

11. Structured Pruning
L1-magnitude based pruning
• Similar phenomena were seen in the structured pruning method. As the structured
pruning prune with the unit of channels, attribution distortion occurs more than
unstructured pruning.
• Our framework also help preserving the attribution maps with structured pruning.
11

12. Sample Images(Structured Pruning)
12

13. Conclusion
• Discovered that naïve network compression causes the Attribution Deformation
Problem, which has not been considered before.
• Proposed and constructed the Attribution Map Matching framework, which enforces
the attribution maps of the compressed network to follow the pre-compression
network.
• Experiments show that our method indeed preserves various kinds of attribution.
Also, our method yields gains in terms of predictive performance.
14