Download to read offline
![15Source: https://www.researchgate.net/figure/Historical-top5-error-rate-of-the-annual-winner-of-the-ImageNet-image-classification_fig7_303992986 [accessed 6 Sep, 2019]
Solving Vision
top5 error rate of the annual winner of the ImageNet image classification challenge
Introduction of CNN models](https://image.slidesharecdn.com/natural-language-in-the-ai-evolution-191017094211/85/Why-natural-language-is-next-step-in-the-AI-evolution-15-320.jpg)
Uploaded byinovex GmbH
Why natural language is next step in the AI evolution
The document discusses the evolution of AI, focusing on the complexity and significance of natural language processing (NLP). It emphasizes the challenges of understanding language due to its nuances and context, and highlights modern NLP techniques such as word-vectors, transformer models, and big language models like BERT. Additionally, it explores various applications of NLP including code generation, text summarization, and enhancing accessibility to information.
More Related Content
Similar to Why natural language is next step in the AI evolution
![English_final_ppt[2] final.pptx](https://cdn.slidesharecdn.com/ss_thumbnails/englishfinalppt2final-230928102934-d115ac20-thumbnail.jpg?width=640&height=640&fit=bounds)
Why natural language is next step in the AI evolution
- 1. Why natural languageis next step in the AI evolution Nico Kreiling Karlsruhe, 1.10.2019
- 2. Nico Kreiling › DataScientist @ inovex › Twitter: nicokreiling › www.techtiefen.de 2
- 3. › Most informationis available in either written or spoken words › Language projects fundamental real world concepts › Speach is everywhere 3 Language is important... ...but really difficult
- 4.
- 5. 5 Walking or even climbingcan be learned fast
- 6.
- 7. › Language hasmany syntactic relationships Stoiber probably thought, he was making a clear point › Language understanding ofen requires external knowledge Pippi feeds Mr. Nilsson regulary. › Language heavily depends on the context „The president has a great vision“ › Irony and sarcasm are topics on their own 7 Why is language complicated ?
- 8. 8 But it works... ...howdoes it?
- 9. Language is more than words 9 How does modernNLP work? Word-Vectors 0,11349057 0,84879879 0,93261575 ... 0,83844097 0,79705175 0,60343267 0,26889299 0,51101431 ... 0,79602685 0,24950905 0,18796149 0,83380881 0,65787064 ... 0,29058833 0,06187716 0,3424444 0,50918356 0,03611215 ... 0,6951714 0,87274548 0,07952056 0,35248701 0,86835106 ... 0,06748431 0,80904259
- 10. 10 How does modernNLP work? Word-Vectors King Queen Princess Prince
- 11. 11 How does modernNLP work? Word-Vectors King + Princess - Queen = Prince King Queen Princess Prince
- 12. 12source: https://explosion.ai/blog/deep-learning-formula-nlp How doesmodern NLP work? Encoder / Decoder Model Embedding Represent the text as Embeddings Encode Enrich the embedding with contextual information Attend Reduces the feature matrix to a single vectory Predict Use the vector to decide about the output
- 13. Grammer is alsoimportant. 13source: http://ruder.io/nlp-imagenet/ How does modern NLP work? Transformer and Self-Supervised Learning › Self Attention › Mask some words › Randomly add a second sentence Language is more than words. 0 0,2 0,4 0,6 0,8 1 Language is m ore than words Attention
- 14. 14 How does modernNLP work? Big Language Models + Transfer learning › Bert Large › 24-layer, 340M parameters › Bert Base (104 languages) › 12-layer, 110M parameters › 64 Tesla V100 GPUs for 79.2 hours › Cost of training $2074–$12,571 But we can build upon this model!
- 15. 15Source: https://www.researchgate.net/figure/Historical-top5-error-rate-of-the-annual-winner-of-the-ImageNet-image-classification_fig7_303992986 [accessed6 Sep, 2019] Solving Vision top5 error rate of the annual winner of the ImageNet image classification challenge Introduction of CNN models
- 16. › GLUE: Language Understanding ›SQuAD: Question Answering › SWAG: Common Sense Reasoning › Coref: Co-Reference Resolution 16 Solving Speach 0 10 20 30 40 50 60 70 80 90 100 GLUE SQuAD 1.1 SQuAD 2 SWAG Coref Performance on different NLP Challenges Sota Bert
- 17.
- 18. Songs as words 18 Spotify WordVectors Auto-Generated Playlists
- 19.
- 20.
- 21.
- 22. Abstractive text summerization usingon arbitary long texts using bidirectional RNNs 22https://www.inovex.de/blog/text-summarization-seq2seq-neural-networks/ Other applications we investigate in Fine-tune general purpose language models for text classificiation in special language environments Attention based RNNs for Q&A on database information using reinforcement learning Extract key aspects on long, complicated texts (medical records, legislative texts) Limit offenstive speach in public communication (bulletin boards, e-mails) Access previously not accessible data sources for easy information access (hotlines, chatbots...)
- 23. Vielen Dank Questions? Let‘s talk: NicoKreiling nicokreiling