Visual transformers: Attention is all you need for computer vision
•Download as PPTX, PDF•
0 likes•440 views
The document discusses visual transformers and attention mechanisms in computer vision. It summarizes recent work on applying transformers, originally used for natural language processing, to vision tasks. This includes Vision Transformers which treat images as sequences and apply self-attention. The document reviews key papers on attention mechanisms, the Transformer architecture, and applying transformers to computer vision through Vision Transformers.
Recommended
Recommended
More Related Content
What's hot
What's hot (20)
Recently uploaded
Recently uploaded (20)
Visual transformers: Attention is all you need for computer vision
- 1. Visual transformers Leo Pauly PhD student | Visual AI Advisors: Prof. David Hogg, Prof. Raul Fuentes University of Leeds, UK
- 2. Visual transformers Leo Pauly PhD student | Visual AI Advisors: Prof. David Hogg, Prof. Raul Fuentes University of Leeds, UK
- 3. Visual transformers Leo Pauly PhD student | Visual AI Advisors: Prof. David Hogg, Prof. Raul Fuentes University of Leeds, UK
- 4. Visual transformers Leo Pauly PhD student | Visual AI Advisors: Prof. David Hogg, Prof. Raul Fuentes University of Leeds, UK
- 5. Dosovitskiy et.al, ICLR 2021
- 6. Vaswani et.al, NeurlPS 2017 Dosovitskiy et.al, ICLR 2021
- 7. Vaswani et.al, NeurlPS 2017 Dosovitskiy et.al, ICLR 2021 Bahdanau et.al, ICLR 2015
- 8. Dosovitskiy et.al, ICLR 2021 Vaswani et.al, NeurlPS 2017 Bahdanau et.al, ICLR 2015 Sutskever et.al, NeurlPS 2014
- 9. Vaswani et.al, NeurlPS 2017 Sutskever et.al, NeurlPS 2014 Dosovitskiy et.al, ICLR 2021 Bahdanau et.al, ICLR 2015
- 10. Attention Mechanism yi=RNN(yi-1,c,si-1) s1 s2 y3 yo y1 y2 c Bahdanau et.al, ICLR 2015
- 11. Attention Mechanism yi=RNN(yi-1,c,si-1) s1 s2 y3 yo y1 y2 c Bahdanau et.al, ICLR 2015 • Bottleneck at the context vector (c) • Information loss • Back propagation issues
- 13. Attention Mechanism yi=RNN(yi-1,c,si-1) s1 s2 y3 yo y1 y2 c yi=RNN(yi-1,ci,si-1) ci=f(hj) j=1…Tx
- 14. Attention Mechanism s1 s2 y3 yo y1 y2 c yi=RNN(yi-1,ci,si-1) Figure from: https://medium.datadriveninvestor.com/attention-in-rnns-321fbcd64f05
- 15. Attention Mechanism s1 s2 y3 yo y1 y2 c yi=RNN(yi-1,ci,si-1) Figure from: https://medium.datadriveninvestor.com/attention-in-rnns-321fbcd64f05
- 16. Attention Mechanism s1 s2 y3 yo y1 y2 c yi=RNN(yi-1,ci,si-1) More reading: https://medium.datadriveninvestor.com/attention-in-rnns-321fbcd64f05
- 17. Attention Mechanism Figure from: https://trungtran.io/2019/03/29/neural-machine-translation-with-attention-mechanism/ x= y=
- 18. Attention is all you Need Vaswani et.al, NeurlPS 2017
- 19. Attention is all you Need
- 20. Attention is all you Need • Scaled dot product attention • Multi-headed attention • Self attention
- 21. Attention is all you Need
- 22. Attention is all you Need Basics explained Y (Query) X (Value) x1 x2 x3 x1 x2 x3 X Attention Map X Output x1 x2 x3 y1 y2 y3 XT (KeyT) y1 y2 y3 Q KT V =(Q.KT). V
- 23. Attention is all you Need Basics explained Y (Query) X (Value) x1 x2 x3 x1 x2 x3 XT (KeyT) y1 y2 y3
- 24. Attention is all you Need Basics explained Y (Query) X (Value) x1 x2 x3 x1 x2 x3 X XT (KeyT) y1 y2 y3 Q KT
- 25. Attention is all you Need Basics explained Y (Query) X (Value) x1 x2 x3 x1 x2 x3 X XT (KeyT) y1 y2 y3 Q KT Attention Map x1 x2 x3 y1 y2 y3
- 26. Attention is all you Need Basics explained Y (Query) X (Value) X XT (KeyT) Q KT Attention Map ‘I’ ‘am’ ‘Leo’ ‘Je’ ‘suis’ ‘leo’ ‘I’ ‘am’ ‘Leo’ ‘Je’ ‘suis’ ‘leo’ ‘I’ ‘am’ ‘Leo’
- 27. Attention is all you Need Basics explained Y (Query) X (Value) x1 x2 x3 x1 x2 x3 X XT (KeyT) y1 y2 y3 Q KT Attention Map x1 x2 x3 y1 y2 y3 X
- 28. Attention is all you Need Basics explained Y (Query) X (Value) x1 x2 x3 x1 x2 x3 X Attention Map X Output x1 x2 x3 y1 y2 y3 XT (KeyT) y1 y2 y3 Q KT V =(Q.KT). V
- 29. Attention is all you Need Basics explained Y (Query) X (Value) X Attention Map X Output XT (KeyT) Q KT V =(Q.KT). V ‘I’ ‘am’ ‘Leo’ ‘Je’ ‘suis’ ‘leo’ ‘I’ ‘am’ ‘Leo’ ‘Je’ ‘suis’ ‘leo’ ‘I’ ‘am’ ‘Leo’
- 30. Attention is all you Need
- 31. Attention is all you Need
- 32. Attention is all you Need Self attention !!! X
- 33. Attention is all you Need Transformer Architecture
- 34. Attention is all you Need
- 35. Vision Transformers Dosovitskiy et.al, ICLR 2021
- 43. • Transformers vs CNNs : Is it worth the hype ? Vision Transformers Insights Ref: https://youtu.be/TvVc1e_4648 ? MaaS ?
- 44. • Transformers vs CNNs : Is it worth the hype ? Vision Transformers Insights ? ?
- 45. • Transformers vs CNNs : Is it worth the hype ? Vision Transformers Insights Higher resolutions ?
- 46. Vision Transformers • Can we do (un)self-supervised pre-training ? Insights Goyal et.al, Arxiv 2021
- 47. • Architecture-level unification across domains Multi-modal AI systems Vision Transformers Insights
- 48. Q !