Recommended
More Related Content
Similar to Generative AI and the Rise of Large Language Models
Similar to Generative AI and the Rise of Large Language Models (20)
Recently uploaded
Recently uploaded (20)
Generative AI and the Rise of Large Language Models
- 1. Generative AI The Rise of Language Models
- 2. AI is the new Black
- 3. Google releasing their new model with 80B params OralB dropping their AI toothbrush
- 4. AI, ML, DL, GI... AI ML DL GI Simulated intelligence exhibited by machines. Pattern Recognition from data (Linear Regression, Random Forests, ...) Uses Neural Networks to learn complex data relationships. Creates new content (images, text, music) based on patterns learned from existing data.
- 5. Large Language Models Petabytes of input data, Billions of parameters Designed to understand and interact with human language, grasp grammar, context and even cultural references Model that makes predictions or generates outputs based on input
- 7. Softmax Decoder Output Embeding The Transformer, centerpiece of GenAI Encoder Input Embedding Attention Layers, where every word becomes aware of its surroundings Pretrained in an unsupervised manner with large unlabeled datasets Fine tuned on supervised training to perform better Unlike RNN, runs sequences in parallel which makes them very fast Attention Layers
- 8. Softmax Embeding Decoder Embeding Decoder Decoder The Underlying Magic: Encoders and Decoders Encoder Embedding Encoder Encoder GPT (Generative Pretrained Transformer) BERT
- 9. But how does it actually work?
- 10. Prediction Word Embeddings Embeddings Vectorizing Words Text Corpus [ ? ] went to the woods... [ ? ] sold all their stocks in... [ ][ ] 0.7 0.5 1.1 3.4 0.2 0.1 9.1 4.4 Extract
- 11. Attention Mechanism The quick brown fox jumps over the lazy dog Q: Any adjectives in front of me? [ ] 0.2 0.1 9.1 4.4
- 12. Attention Mechanism The quick brown fox jumps over the lazy dog Q: Any adjectives in front of me? [ ] 0.2 0.3 7.8 4.4 Let me adjust my definition... Right here! I am!
- 13. Transfer Learning Acquired knowledge in the pretraining phase is transferred to downstream tasks Model is fine tuned to specific task using an annotated dataset Enables generalization to new tasks and reduces the need for large annotated datasets
- 14. As always, this sounds too good to be true...
- 15. Hallucinations Where do they come from? Due to: Limited Knowledge: The user is asking about something the model was not trained on Model: Limitations of the current LLM (small model, needs fine-tuning) Data: Most common technique is Retrieval Augmented Generation If Data is not well organized -> will retrieve wrong info Hallucination is inevitable: https://arxiv.org/abs/2401.11817
- 16. Hallucinations How to deal with them? Model Fine-Tuning Retrieval Augmented Generation (RAG) Prompt Engineering
- 18. LangChain Simplifies LLM-powered app development, connecting to diverse data sources for personalized experiences Supports chatbots, retrieval-augmented generation, summarization, and synthetic data. Integrates with Amazon, Google, Microsoft, OpenAI, etc. Chains with programming languages, platforms, financial data, web scraping, scientific papers.
- 19. LangChain