Download as PDF, PPTX
Uploaded byAWS Chicago
Jeff Maruschek: How does RAG REALLY work?
The document provides an overview of Retrieval-Augmented Generation (RAG) and its components, including retrieval, augmentation, and generation processes in generating responses from a foundation model. It discusses the significance of embeddings for text representation and their application in comparing text similarity, emphasizing the challenges associated with implementing RAG such as managing data sources and scaling retrieval mechanisms. Interactive demos for Amazon Bedrock's chatbot and knowledge bases are showcased to illustrate the functionalities of RAG.
More Related Content
Similar to Jeff Maruschek: How does RAG REALLY work?
Jeff Maruschek: How does RAG REALLY work?
- 1. © 2024, AmazonWeb Services, Inc. or its affiliates. All rights reserved. Jeff Maruschek Amazon Web Services Solutions Architect – DoD – Air Force Retrieval-Augmented Generation (RAG) Overview and Interactive Demos
- 2. © 2024, AmazonWeb Services, Inc. or its affiliates. All rights reserved. 2 Introduction
- 3. © 2024, AmazonWeb Services, Inc. or its affiliates. All rights reserved. 3 Introduction
- 4. © 2024, AmazonWeb Services, Inc. or its affiliates. All rights reserved. 4 djmoose on Mixcloud:
- 5. © 2024, AmazonWeb Services, Inc. or its affiliates. All rights reserved. What is RAG? What are Embeddings? Amazon Bedrock ChatBot Demo Amazon Bedrock Knowledge Bases Demo Agenda 5
- 6. © 2024, AmazonWeb Services, Inc. or its affiliates. All rights reserved. 6 What book genre is The Three-Body Problem?
- 7. © 2024, AmazonWeb Services, Inc. or its affiliates. All rights reserved. 7 What instrument does Joshua Bell play?
- 8. © 2024, AmazonWeb Services, Inc. or its affiliates. All rights reserved. 8 What instrument does the musician Joshua Bell play?
- 9. © 2024, AmazonWeb Services, Inc. or its affiliates. All rights reserved. 9 Who is Joshua Bell?
- 10. © 2024, AmazonWeb Services, Inc. or its affiliates. All rights reserved. 10
- 11. © 2024, AmazonWeb Services, Inc. or its affiliates. All rights reserved. 11
- 12. © 2024, AmazonWeb Services, Inc. or its affiliates. All rights reserved. • Retrieval ▪ Fetches the relevant content from the external knowledge base or data sources based on a user query • Augmentation ▪ Adding the retrieved relevant context to the user prompt, which goes as an input to the foundation model • Generation ▪ Response from the foundation model based on the augmented prompt. 12 What is RAG?
- 13. © 2024, AmazonWeb Services, Inc. or its affiliates. All rights reserved. 13
- 14. © 2024, AmazonWeb Services, Inc. or its affiliates. All rights reserved. What are embeddings? E M B E D D I N G M O D E L 0.027 -0.011 -0.023 … 0.025 -0.009 -0.025 … New York Paris Vector Embeddings Human Text • Numerical representation of text (vectors) that captures semantics and relationships between words. • Embedding models capture features and nuances of the text. • Rich embeddings can be used to compare text similarity. • Multilingual Text Embeddings can identify meaning in different languages. -0.011 0.021 0.013 … Animal -0.009 0.019 0.015 … Horse
- 15. © 2024, AmazonWeb Services, Inc. or its affiliates. All rights reserved. 16
- 16. © 2024, AmazonWeb Services, Inc. or its affiliates. All rights reserved. 17
- 17. © 2024, AmazonWeb Services, Inc. or its affiliates. All rights reserved. 18
- 18. © 2024, AmazonWeb Services, Inc. or its affiliates. All rights reserved. 19
- 19. © 2024, AmazonWeb Services, Inc. or its affiliates. All rights reserved. 20 Amazon Bedrock ChatBot Demo https://github.com/aws-samples/bedrock-claude-chat
- 20. © 2024, AmazonWeb Services, Inc. or its affiliates. All rights reserved. 21 Documents Used
- 21. © 2024, AmazonWeb Services, Inc. or its affiliates. All rights reserved. 22 Architecture
- 22. © 2024, AmazonWeb Services, Inc. or its affiliates. All rights reserved. However, when it comes to implementing RAG, there are challenges… Creating vector embeddings for large volumes of data Orchestration Managing multiple data sources Scaling retrieval mechanism Coding effort Incremental updates to vector store
- 23. © 2024, AmazonWeb Services, Inc. or its affiliates. All rights reserved. 24 Amazon Bedrock Knowledge Bases Demo https://aws.amazon.com/bedrock/knowledge-bases/
- 24. © 2024, AmazonWeb Services, Inc. or its affiliates. All rights reserved. 25 Documents Used
- 25. © 2024, AmazonWeb Services, Inc. or its affiliates. All rights reserved. Thank you! Jeff Maruschek Solutions Architect – DoD – Air Force jefmarus@amazon.com 26