"Attention Is All You Need" (Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Łukasz Kaiser, https://bit.ly/2y7yAD2 presented by Maroua Maachou (Veepee)
5. Problems in usual Seq2Seq model
→ Difficulty to summarize long-term dependencies
→ The need of a contextual embedding for the elements
of the sequence
→ Slow and difficult models to train
17. CREDITS: This presentation template was created by Slidesgo, including
icons by Flaticon, and infographics & images by Freepik.
Please keep this slide for attribution.
Does anyone have any questions?
marouamaachou@gmail.com
THANKS
18. ■ The Paper : https://arxiv.org/pdf/1706.03762.pdf
■ The Illustrated Transformer : http://jalammar.github.io/illustrated-transformer/
■ The Annotated Transformer : https://nlp.seas.harvard.edu/2018/04/03/attention.html
■ Stanford NLP course :
https://www.youtube.com/watch?v=XXtpJxZBa2c&list=PLoROMvodv4rOhcuXMZkNm7j3fVwBBY4
2z&index=8
■ Attention and Augmented Recurrent Neural Networks : https://distill.pub/2016/augmented-rnns/
■ Other Attention Mechanisms :
https://lilianweng.github.io/lil-log/2018/06/24/attention-attention.html#a-family-of-attention-mecha
nisms
■ My Implementation : https://github.com/marouamaachou
RESOURCES