Download as PDF, PPTX
![Semantic Similarity
Image from Sutor et al
Woman = [0.3, 0.4]
Queen = [0.3, 0.9]
King = [0.5, 0.7]
Woman = [0.3, 0.4]
Queen = [0.3, 0.9]
King = [0.5, 0.7]
Man = [0.5, 0.2]
Queen - Woman + Man = King
Queen = [0.3, 0.9]
- Woman = [0.3, 0.4]
[0.0, 0.5]
+ Man = [0.5, 0.2]
King = [0.5, 0.7]
Man = [0.5, 0.2]](https://image.slidesharecdn.com/beyondragvectordatabases-240422205449-15eb19b2/85/Beyond-Retrieval-Augmented-Generation-RAG-Vector-Databases-13-320.jpg)
![Vector Similarity Metric: L2 (Euclidean)
Queen = [0.3, 0.9]
King = [0.5, 0.7]
d(Queen, King) = √(0.3-0.5)2
+ (0.9-0.7)2
= √(0.2)2
+ (0.2)2
= √0.04 + 0.04
= √0.08 ≅ 0.28](https://image.slidesharecdn.com/beyondragvectordatabases-240422205449-15eb19b2/85/Beyond-Retrieval-Augmented-Generation-RAG-Vector-Databases-15-320.jpg)
![Vector Similarity Metric: Inner Product (IP)
Queen = [0.3, 0.9]
King = [0.5, 0.7]
Queen · King = (0.3*0.5) + (0.9*0.7)
= 0.15 + 0.63 = 0.78](https://image.slidesharecdn.com/beyondragvectordatabases-240422205449-15eb19b2/85/Beyond-Retrieval-Augmented-Generation-RAG-Vector-Databases-16-320.jpg)
![Queen = [0.3, 0.9]
King = [0.5, 0.7]
Vector Similarity Metric: Cosine
𝚹
cos(Queen, King) = (0.3*0.5)+(0.9*0.7)
√0.32
+0.92
* √0.52
+0.72
= 0.15+0.63 _
√0.9 * √0.74
= 0.78 _
√0.666
≅ 0.03](https://image.slidesharecdn.com/beyondragvectordatabases-240422205449-15eb19b2/85/Beyond-Retrieval-Augmented-Generation-RAG-Vector-Databases-17-320.jpg)
Uploaded byZilliz
Beyond Retrieval Augmented Generation (RAG): Vector Databases
The document discusses the significance and workings of vector databases, highlighting their ability to analyze vast amounts of unstructured data, which constitutes 80% of global data by 2025. It explains various vector similarity metrics, provides examples of use cases such as semantic search and recommendation systems, and outlines the architecture and advantages of using Milvus/Zilliz for large-scale data management. Vector databases are promoted as a solution for efficient storage, indexing, and querying of vector embeddings from unstructured data.
More Related Content
Similar to Beyond Retrieval Augmented Generation (RAG): Vector Databases
![Getting Started with Apache Spark: Big Data Made Simple [Free Meetup]](https://cdn.slidesharecdn.com/ss_thumbnails/apachesparkgettingstarted-260203175547-8361bcc3-thumbnail.jpg?width=640&height=640&fit=bounds)
Beyond Retrieval Augmented Generation (RAG): Vector Databases
- 1. Yujian Tang |Zilliz Beyond RAG: Vector Databases
- 2. Yujian Tang Senior DeveloperAdvocate, Zilliz yujian@zilliz.com https://www.linkedin.com/in/yujiantang https://www.twitter.com/yujian_tang Speaker
- 3. 01 Why VectorDatabases? CONTENTS 03 04 Vector Database Architecture 02 How Do Vector Databases Work? Use Cases
- 4. 01 Why VectorDatabases?
- 5. Compare data thatyou couldn’t compare before
- 6. Unstructured Data isEverywhere Unstructured data is any data that does not conform to a predefined data model. By 2025, IDC estimates there will be 175 zettabytes of data globally (that's 175 with 21 zeros), with 80% of that data being unstructured. Currently, 90% of unstructured data is never analyzed. Text Images Video and more!
- 7. Vector Databases Unstructured Data +ML = Vector Magic
- 8. Find Semantically SimilarData Apple made profits of $97 Billion in 2023 I like to eat apple pie for profit in 2023 Apple’s bottom line increased by record numbers in 2023
- 9. But wait! There’smore!
- 10. Use math toquantify relationships between entities
- 11. 02 How DoVector Databases Work?
- 12. Vector similarity isa mathematical measure of how close two vectors are
- 13. Semantic Similarity Image fromSutor et al Woman = [0.3, 0.4] Queen = [0.3, 0.9] King = [0.5, 0.7] Woman = [0.3, 0.4] Queen = [0.3, 0.9] King = [0.5, 0.7] Man = [0.5, 0.2] Queen - Woman + Man = King Queen = [0.3, 0.9] - Woman = [0.3, 0.4] [0.0, 0.5] + Man = [0.5, 0.2] King = [0.5, 0.7] Man = [0.5, 0.2]
- 14. Similarity metrics areways to measure distance in vector space
- 15. Vector Similarity Metric:L2 (Euclidean) Queen = [0.3, 0.9] King = [0.5, 0.7] d(Queen, King) = √(0.3-0.5)2 + (0.9-0.7)2 = √(0.2)2 + (0.2)2 = √0.04 + 0.04 = √0.08 ≅ 0.28
- 16. Vector Similarity Metric:Inner Product (IP) Queen = [0.3, 0.9] King = [0.5, 0.7] Queen · King = (0.3*0.5) + (0.9*0.7) = 0.15 + 0.63 = 0.78
- 17. Queen = [0.3,0.9] King = [0.5, 0.7] Vector Similarity Metric: Cosine 𝚹 cos(Queen, King) = (0.3*0.5)+(0.9*0.7) √0.32 +0.92 * √0.52 +0.72 = 0.15+0.63 _ √0.9 * √0.74 = 0.78 _ √0.666 ≅ 0.03
- 18. Vector Similarity Metrics Euclidean- Spatial distance Cosine - Orientational distance Inner Product - Both With normalized vectors, IP = Cosine
- 19. Indexes organize theway we access our data
- 20.
- 21. Hierarchical Navigable SmallWorlds (HNSW) Source: https://arxiv.org/ftp/arxiv/papers/1603/1603.09320.pdf
- 22.
- 23.
- 24. Indexes Overview - IVF= Intuitive, medium memory, performant - HNSW = Graph based, high memory, highly performant - Flat = brute force - SQ = bucketize across one dimension, accuracy x memory tradeoff - PQ = bucketize across two dimensions, more accuracy x memory tradeoff
- 25. Vector databases efficientlystore, index, and relate entities by a quantitative value
- 26.
- 27. What Does VectorData Look Like?
- 28. RAG RAG Inject your datavia a vector database like Milvus/Zilliz Query LLM Milvus Your Data Primary Use Case ● Factual Recall ● Forced Data Injection ● Cost Optimization
- 29. Common AI UseCases LLM Augmented Retrieval Expand LLMs' knowledge by incorporating external data sources into LLMs and your AI applications. Match user behavior or content features with other similar behaviors or features to make effective recommendations. Recommender System Search for semantically similar texts across vast amounts of natural language documents. Text/ Semantic Search Image Similarity Search Identify and search for visually similar images or objects from a vast collection of image libraries. Video Similarity Search Search for similar videos, scenes, or objects from extensive collections of video libraries. Audio Similarity Search Find similar audios from massive amounts of audio data to perform tasks such as genre classification, or recognize speech. Molecular Similarity Search Search for similar substructures, superstructures, and other structures for a specific molecule. Question Answering System Interactive QA chatbot that automatically answers user questions Multimodal Similarity Search Search over multiple types of data simultaneously, e.g. text and images
- 30.
- 31.
- 32.
- 33. 04 Vector DatabaseArchitecture
- 34. Why Not Usea SQL/NoSQL Database? ● Inefficiency in High-dimensional spaces ● Suboptimal Indexing ● Inadequate query support ● Lack of scalability ● Limited analytics capabilities ● Data conversion issues TL;DR: Vector operations are too computationally intensive for traditional database infrastructures
- 35. Why Not Usea Vector Search Library? ● Have to manually implement filtering ● Not optimized to take advantage of the latest hardware ● Unable to handle large scale data ● Lack of lifecycle management ● Inefficient indexing capabilities ● No built in safety mechanisms TL;DR: Vector search libraries lack the infrastructure to help you scale, deploy, and manage your apps in production.
- 36. What is Milvus/Zillizideal for? ○ Advanced filtering ○ Hybrid search ○ Durability and backups ○ Replications/High Availability ○ Sharding ○ Aggregations ○ Lifecycle management ○ Multi-tenancy ○ High query load ○ High insertion/deletion ○ Full precision/recall ○ Accelerator support (GPU, FPGA) ○ Billion-scale storage Purpose-built to store, index and query vector embeddings from unstructured data at scale.
- 37. Meta Storage Root QueryData Index Coordinator Service Proxy Proxy etcd Log Broker SDK Load Balancer DDL/DCL DML NOTIFICATION CONTROL SIGNAL Object Storage Minio / S3 / AzureBlob Log Snapshot Delta File Index File Worker Node QUERY DATA DATA Message Storage Access Layer Query Node Data Node Index Node High-level overview of Milvus’ Architecture
- 38. Start building with ZillizCloud today! zilliz.com/cloud
- 39. | © Copyright9/25/23 Zilliz 39 Appendix
- 40. Important Notes - Cosine,IP, and L2 are all the SAME rank order. - They differ in use case - L2 for when you need magnitude - Cosine for orientation - IP for magnitude and orientation - OR - Cosine = IP for normalized vectors
- 41.
- 42. Basic Idea You wantto use your data with a large language model
- 43. RAG vs FineTuning LLM Fine Tuning Augment an LLM by training it on your data Your Data “New” LLM Query Primary Use Case ● Style transfer
- 44. Takeaway Use RAG toforce the LLM to work with your data by injecting it via a vector database like Milvus or Zilliz
- 45. Chunking Considerations Chunk Size ChunkOverlap Character Splitters
- 46. How Does YourData Look? Conversation Data Documentation Data Lecture or Q/A Data
- 47.
- 48.
- 49.
- 50.
- 51.
- 52.
- 53. Your chunking strategydepends on what your data looks like and what you need from it. Takeaway:
- 54. Examining Embeddings Picking amodel What to embed Metadata
- 55. Embeddings Strategies Level 1:Embedding Chunks Directly Level 2: Embedding Sub and Super Chunks Level 3: Incorporating Chunking and Non-Chunking Metadata
- 56. Metadata Examples Chunking ● Paragraphposition ● Section header ● Larger paragraph ● Sentence Number ● … Non-Chunking ● Author ● Publisher ● Organization ● Role Based Access Control ● …
- 57. Your embeddings strategydepends on your accuracy, cost, and use case needs Takeaway:
- 58. Basic Idea Vector Databasesprovide the ability to inject your data via semantic similarity Considerations include: scale, performance, and flexibility
- 59. Milvus Architecture: Differentiation 1.Cloud Native, Distributed System Architecture 2. True Separation of Concerns 3. Scalable Index Creation Strategy with 512 MB Segments
- 60. Vector Databases arepurpose-built to handle indexing, storing, and querying vector data. Milvus & Zilliz are specifically designed for high performance and billion+ scale use cases. Takeaway:
- 61. Vector Database Resources GiveMilvus a Star! Chat with me on Discord!
- 62. Get Started Free Gotquestions? Stop by our booth! Milvus Open Source Self-Managed github.com/milvus-io/milvus Zilliz Cloud SaaS Fully-Managed zilliz.com/cloud