Chunking, Embeddings, and Vector Databases

1 | © Copyright 2023 Zilliz
1
Yujian Tang | Zilliz
Chunking, Embeddings, and
Vector Databases

2
Yujian Tang
Senior Developer Advocate, Zilliz
yujian@zilliz.com
https://www.linkedin.com/in/yujiantang
https://www.twitter.com/yujian_tang
Speaker

3
GPT-Cache
Key maintainer of the following Open-Source projects
Founded
Headquarters
Focus
2017
Redwood Shores, CA
Vector database company for enterprise-grade AI built on Milvus, the
popular open-source vector database that helps organizations quickly
create AI applications.
Zilliz at a Glance

4
01 Why RAG?
CONTENTS
03
04 Vector Databases
02 Chunking Considerations
Embedding Models and Strategies

5
01 RAG Overview

6
Basic Idea
You want to use your data with a large
language model

7
RAG vs Fine Tuning
RAG
Inject your data via a vector
database like Milvus/Zilliz
Query LLM
Milvus
Your Data
Primary Use Case
- Factual Recall
- Forced Data Injection
- Cost Optimization

8
RAG vs Fine Tuning
LLM
Fine Tuning
Augment an LLM by training it
on your data
Your Data
“New” LLM
Query
Primary Use Case
- Style transfer
- Domain specific usage

9
Takeaway
Use RAG to force the LLM to work with your data by injecting it
via a vector database like Milvus or Zilliz

10
02 Chunking

11
Chunking Considerations
Chunk Size
Chunk Overlap
Character Splitters

12
How Does Your Data Look?
Conversation
Data
Documentation
Data
Lecture or Q/A
Data

13
Examples

14
Examples

15
Examples

16
Examples

17
Examples

18
Examples

19
Your chunking strategy depends on what your data looks
like and what you need from it.
Takeaway:

20
03 Embeddings

21
Examining Embeddings
Picking a model
What to embed
Metadata

22
Embeddings Models

23
Embeddings Strategies
Level 1: Embedding Chunks Directly
Level 2: Embedding Sub and Super Chunks
Level 3: Incorporating Chunking and Non-Chunking Metadata

24
Metadata Examples
Chunking
- Paragraph position
- Section header
- Larger paragraph
- Sentence Number
- …
Non-Chunking
- Author
- Publisher
- Organization
- Role Based Access Control
- …

25
Your embeddings strategy depends on your accuracy,
cost, and use case needs
Takeaway:

26
04 Vector Databases

27
Basic Idea
Vector Databases provide the ability to inject your data via
semantic similarity
Considerations include: scale, performance, and flexibility

28
Semantic Similarity
Image from Sutor et al
Woman = [0.3, 0.4]
Queen = [0.3, 0.9]
King = [0.5, 0.7]
Woman = [0.3, 0.4]
Queen = [0.3, 0.9]
King = [0.5, 0.7]
Man = [0.5, 0.2]
Queen - Woman + Man = King
Queen = [0.3, 0.9]
- Woman = [0.3, 0.4]
[0.0, 0.5]
+ Man = [0.5, 0.2]
King = [0.5, 0.7]
Man = [0.5, 0.2]

29
Vector
Databases
Where do Vectors Come From?

30
Basic Functionality

31
Why Not Use a SQL/NoSQL Database?
• Inefficiency in High-dimensional spaces
• Suboptimal Indexing
• Inadequate query support
• Lack of scalability
• Limited analytics capabilities
• Data conversion issues
TL;DR: Vector operations are too computationally intensive for
traditional database infrastructures

32
Why Not Use a Vector Search Library?
• Have to manually implement filtering
• Not optimized to take advantage of the latest hardware
• Unable to handle large scale data
• Lack of lifecycle management
• Inefficient indexing capabilities
• No built in safety mechanisms
TL;DR: Vector search libraries lack the infrastructure to help you scale,
deploy, and manage your apps in production.

33
What is Milvus/Zilliz ideal for?
• Advanced filtering
• Hybrid search
• Durability and backups
• Replications/High Availability
• Sharding
• Aggregations
• Lifecycle management
• Multi-tenancy
• High query load
• High insertion/deletion
• Full precision/recall
• Accelerator support (GPU,
FPGA)
• Billion-scale storage
Purpose-built to store, index and query vector embeddings from unstructured data at scale.

34
Meta Storage
Root Query Data Index
Coordinator Service
Proxy
Proxy
etcd
Log Broker
SDK
Load Balancer
DDL/DCL
DML
NOTIFICATION
CONTROL SIGNAL
Object Storage
Minio / S3 / AzureBlob
Log Snapshot Delta File Index File
Worker Node QUERY DATA DATA
Message Storage
Access Layer
Query Node Data Node Index Node
High-level overview of Milvus’ Architecture

35
Milvus Architecture: Differentiation
1. Cloud Native, Distributed System Architecture
2. True Separation of Concerns
3. Scalable Index Creation Strategy with 512 MB Segments

36
Vector Databases are purpose-built to handle
indexing, storing, and querying vector data.
Milvus & Zilliz are specifically designed for high
performance and billion+ scale use cases.
Takeaway:

37
Vector Database Resources
Give Milvus a Star! Chat with me on Discord!

38
Start building
with Zilliz Cloud today!
zilliz.com/cloud

Chunking, Embeddings, and Vector Databases

Recommended

Recommended

More Related Content

Similar to Chunking, Embeddings, and Vector Databases

Similar to Chunking, Embeddings, and Vector Databases (20)

More from Zilliz

More from Zilliz (18)

Recently uploaded

Recently uploaded (20)

Chunking, Embeddings, and Vector Databases