Advanced Retrieval Augmented Generation Techniques

•

0 likes•33 views

While achieving a basic Retrieval Augmented Generation (RAG) is relatively straightforward, attaining superior results requires tuning and optimizing various factors, such as a careful selection of embedding models. Additionally, applying advanced techniques, such as multi-stage retrieval with rerankers, is essential. A methodology for quality evaluation is also critical to success in crafting the best strategy for your specific use case. This talk will introduce the landscape of available optimization techniques and provide advice on best practices.

Technology

1 | © Copyright 2024 Zilliz
1
1 | © Copyright 9/25/23 Zilliz
1 | © Copyright 9/25/23 Zilliz
Speaker
Jiang Chen
Ecosystem & AI Platform
jiang.chen@zilliz.com
@jiangc1010

2 | © Copyright 2024 Zilliz
2
Fantastic RAG Techniques
And Where to Find Them
Jiang Chen @ Zilliz

3 | © Copyright 2024 Zilliz
3
LLMs are great, but …
You still need to battle hallucination
with retriever, just like the Niffler

4 | © Copyright 2024 Zilliz
4
The evolution of AI made the semantic search of
unstructured data possible
Search by Probability
Statistical analyses of common
datasets established the foundation for
processing unstructured data, e.g. NLP,
and image classification
AI Model Breakthrough
The advancements in BERT, ViT, CBT
etc. have revolutionized semantic
analysis across unstructured data
Vectorization
Word2Vec, CNNs, Deep Speech pioneered
unstructured data embeddings, mapping the
words, images, videos into high-dimensional
vectors

5 | © Copyright 2024 Zilliz
5
01 Review of RAG basics
CONTENTS
02 Advanced RAG techniques
RAG in action with Milvus Lite
03

6 | © Copyright 2024 Zilliz
6
01 Review of RAG basics

7 | © Copyright 2024 Zilliz
7
Why RAG?
RAG vs. LLM
- Knowledge of LLM is out-of-date
- LLM can not get your private knowledge
- Hallucinations
- Transparency and interpretability
RAG vs. Fine-tune
- Fine-tune is expensive
- Fine-tune spent much time
- RAG is pluggable

9 | © Copyright 2024 Zilliz
9
02 Advanced RAG techniques

10 | © Copyright 2024 Zilliz
10
First thing first
Measure it before you attempts to improve it!

11 | © Copyright 2024 Zilliz
11
Indexing
Query Retrieval Prompt&
Generation

12 | © Copyright 2024 Zilliz
12
Types of RAG Enhancement Techniques
● Divide & Conquer
○ Query Enhancement: better express or process the query intent.
○ Indexing Enhancement: data cleanup, better parser and chunking
○ Retriever Enhancement: more retrievers and hybrid search strategy
○ Generator Enhancement: prompt engineering and more powerful LLM
● Thinking outside the box
○ Agents? Other tools than retriever?

13 | © Copyright 2024 Zilliz
13
Query Enhancement

16 | © Copyright 2024 Zilliz
16
What are the differences in features
between Milvus and Zilliz Cloud?
Sub query1: What are the features of Milvus?
Sub query2: What are the features of Zilliz Cloud?

18 | © Copyright 2024 Zilliz
18
Indexing Enhancement

19 | © Copyright 2024 Zilliz
19
Good dishes come from good ingredients
• Data collection
• Data cleaning
• Parsing & Chunking
• DNN-native data?

21 | © Copyright 2024 Zilliz
21
Retriever Enhancement

25 | © Copyright 2024 Zilliz
25
Generator Enhancement

32 | © Copyright 2024 Zilliz
32
03 RAG in action with Milvus Lite

34 | © Copyright 2024 Zilliz
34
Seamless integration with all popular AI toolkits

35 | © Copyright 2024 Zilliz
35
35 | © Copyright 9/25/23 Zilliz
35 | © Copyright 9/25/23 Zilliz
Simplify and streamline
the conversion of
unstructured data into
state-of-the-art vector
embeddings, using
intuitive UI and Restful
APIs.
Pipelines
Easy. High-quality. Scalable.
Simplify the workflow
for developers, from
converting
unstructured data into
searchable vectors to
retrieving them from
vector databases
Deliver excellence in
every phase of vector
search pipeline
development and
deployment,
regardless of their
expertise
Ensure scalability for
managing large
datasets and
high-throughput
queries, maintaining
high performance with
min. customization or
infra changes
Zilliz Cloud Pipelines

36 | © Copyright 2024 Zilliz
36
T H A N K Y O U

Dr Jesús Barrasa, Head of Solutions Architecture for EMEA, Neo4j Join us as we explore breakthrough innovations enabled by interconnected data and AI. Discover firsthand how organizations use relationships in data to uncover contextual insights and solve our most pressing challenges – from optimizing supply chains, detecting fraud, and improving customer experiences to accelerating drug discoveries.

Chunking, Embeddings, and Vector Databases

Zilliz

Retrieval Augmented Generation (RAG), is a popular method to use a large language model, a vector database, and some sort of prompt interface to build better chat bots. On the surface, it seems pretty simple to build a RAG app, but when it comes down to implementation, there are many details to hash out. These details include how to: chunk data, work with embeddings, and even how to select and use a vector database.

BT & Neo4j _ How Knowledge Graphs help BT deliver Digital Transformation.pptx

Neo4j

Delivered by Sreenath Gopalakrishna, Director of Software Engineering at BT, and Dr Jim Webber, Chief Scientist at Neo4j, at Gartner Data & Analytics Summit London 2024 this presentation examines how knowledge graphs and GenAI combine in real-world solutions. BT Group has used the Neo4j Graph Database to enable impressive digital transformation programs over the last 6 years. By re-imagining their operational support systems to adopt self-serve and data lead principles they have substantially reduced the number of applications and complexity of their operations. The result has been a substantial reduction in risk and costs while improving time to value, innovation, and process automation. Future innovation plans include the exploration of uses of EKG + Generative AI.

Introducing Milvus Lite: Easy-to-Install, Easy-to-Use vector database for you...

Zilliz

Government GraphSummit: Keynote - Graphs in Government

Neo4j

Jim Webber Ph.D., Chief Scientist, Neo4j Learn about the importance of graph technology, its evolution over the last few years and the impact it has had on the database and data analytics industry. This session will provide an overview of graph technology and talk about the past, present, and future of graphs and data management. Multiple use cases and customer examples will be covered, including examples of where graph databases and graph data science can assist and accelerate machine learning and artificial intelligence projects.

Neo4j Jesus Barrasa The Art of the Possible with Graph

Neo4j

The document discusses how massive trends like connected data, cloud innovation, and the rise of generative AI are transforming industries. It argues that to thrive in this new environment, organizations must turn data into insights and knowledge. Graph databases are presented as better for this task by preserving relationships that get lost with relational databases. The document promotes Neo4j's graph database platform and its capabilities for enabling insights, powering cloud applications, and combining with generative AI through knowledge graphs.

GraphSummit Toronto: Keynote - Innovating with Graphs

Neo4j

Neo4j GraphSummit London March 2023 Emil Eifrem Keynote.pptx

Neo4j

The Data Platform for Today's Intelligent Applications.pdf

Neo4j

Jarrod Lopiccolo - Big Data

RenoTahoeAMA

The document discusses big data, including what it is, current usage and challenges, and the future direction. It notes that while 64% of enterprises are deploying or planning big data projects, 56% still do not know what to do with big data. It recommends starting small with big data efforts, having a clear goal, and iterating as more is learned from the data. Visualizing data and tracking sentiment are presented as ways to start extracting useful insights from big data.

Introduction to Multilingual Retrieval Augmented Generation (RAG)

Zilliz

Retrieval augmented generation (RAG) is the most popular style of large language model application to emerge from 2023. The most basic style of RAG works by vectorizing your data and injecting it into a vector database like Milvus for retrieval to augment the text output generated by an LLM. This is just the beginning. One of the ways that we can extend RAG, and extend AI, is through multilingual use cases. Typical RAG is done in English using embedding models that are trained in English. In this talk, we’ll explore how RAG could work in languages other than English. We’ll explore French, Chinese, and Polish.

Are You Underestimating the Value Within Your Data? A conversation about grap...

Neo4j

CollabDays NL 2024 - Protecting and governing your sensitive data with Micros...

Jasper Oosterveld

In today's AI-driven workplace, the seamless flow of information brings with it the inherent risk of sensitive data exposure and unauthorized access. The rapid adoption of Copilot for Microsoft 365 underscores the critical need for safeguarding and governing your organization's sensitive data. As a Data Security Consultant and Microsoft MVP, I invite you to join me in unveiling practical tips and tricks for achieving this using Microsoft Purview Information Protection, Data Loss Prevention, Compliance Manager, and Communication Compliance. Together, we'll navigate the landscape of data security challenges and risks, exploring actionable insights to implement robust solutions for effectively protecting and governing your organization's data within an AI-driven environment.

GPT and Graph Data Science to power your Knowledge Graph

Neo4j

In this workshop at Data Innovation Summit 2023, we demonstrated how you could learn from the network structure of a Knowledge Graph and use OpenAI’s GPT engine to populate and enhance your Knowledge Graph. Key takeaways: 1. How Knowledge Graphs grow organically 2. How to deploy Graph Algorithms to learn from the topology of a graph 3. Integrate a Knowledge Graph with OpenAI’s GPT 4. Use Graph Node embeddings to feed Machine Learning workflow

Nordics Edition - The Neo4j Graph Data Platform Today & Tomorrow

Neo4j

raph Databases with Neo4j – Emil Eifrem

buildacloud

This document provides an overview of the graph database Neo4j. It discusses that Neo4j is a graph database with nodes, relationships, and properties that is well-suited for complex, highly connected data. Examples are given demonstrating how Neo4j can be used for network management in telecommunications companies and content management, access control, and collaboration at Adobe. Cypher, the query language for Neo4j, is also introduced.

Workshop - Architecting Innovative Graph Applications- GraphSummit Milan

Neo4j

Workshop 1. Architecting Innovative Graph Applications Join this hands-on workshop for beginners led by Neo4j experts guiding you to systematically uncover contextual intelligence. Using a real-life dataset we will build step-by-step a graph solution; from building the graph data model to running queries and data visualization. The approach will be applicable across multiple use cases and industries.

Wikibon 2018 Predictions

plburris

The Art of the Possible with Graph by Dr Jim Webber Neo4j.pptx

Neo4j

Disrupting with Data: Lessons from Silicon Valley

Anand Rajaraman

This document summarizes Anand Rajaraman's presentation on data-driven disruption. It discusses 5 generations of data-driven applications that have followed available data sources: 1) Leveraging private, structured data for competitive advantage. 2) Harnessing public data. 3) Leveraging "semi-public" social and mobile data shared with user consent. 4) Combining public, semi-public, and private data. 5) Adding artificial intelligence to leverage massive amounts of proprietary training data. The presentation covers lessons around startup opportunities in infrastructure, analytics, and intelligent applications, and discusses the difference between data-driven disruption versus optimization. It also discusses challenges around human-machine

Christian Lanng – Co Founder CEO, Trade Shift

Global Business Intelligence

This document discusses the future of procurement and how companies need to adapt to remain competitive. It notes that 52% of Fortune 500 companies from 15 years ago no longer exist due to lack of adaptability. It argues that companies prioritize process, inflexible systems, and cost savings over innovation, smart people, and value creation. The future will require agility, flexibility, digitization, and collaboration across the supply chain. Tradeshift aims to enable this by providing a flexible collaborative platform instead of single purpose tools, connecting the entire supply chain, and leveraging real-time data and analytics.

Streamlining Nonprofit Organizations: It’s All About the Cloud

4Good.org

You may have heard people talk about the latest advance in technology: “the cloud.” In this webinar, participants will learn what cloud computing is, how it’s changing our work, and how nonprofit organizations can take advantage of cloud-based services (e.g. Salesforce, Google Docs, Dropbox) to enhance their operations in all parts of their organizations to become more efficient with their limited resources and time. Participants will see real-life case studies of nonprofit organizations that are using these tools every day. We will offer a survey of cloud-based tools that organizations can use within their own organizations in the areas of cross-organizational management, program management, human resources, marketing and fundraising. At the end of the webinar, participants will leave with a list of tools and resources for further exploration, and with specific strategies about how to incorporate these tools into their day-to-day work.

Fueling AI with Great Data with Airbyte Webinar

Zilliz

Programming Foundation Models with DSPy - Meetup Slides

Zilliz

Similar to Advanced Retrieval Augmented Generation Techniques

Vector Databases 101 - An introduction to the world of Vector Databases

Zilliz

Neo4j Keynote: The Art of the Possible with Graph Technology

Neo4j

Keynote Presentation at GraphTalk Oslo 2023

Neo4j

The Art of the Possible with Graph - Sudhir Hasbe - GraphSummit London 14 Nov...

Neo4j

GraphSummit Toronto: Keynote - Innovating with Graphs

Neo4j

Neo4j GraphSummit London March 2023 Emil Eifrem Keynote.pptx

Neo4j

The Data Platform for Today's Intelligent Applications.pdf

Neo4j

Jarrod Lopiccolo - Big Data

RenoTahoeAMA

Introduction to Multilingual Retrieval Augmented Generation (RAG)

Zilliz

Are You Underestimating the Value Within Your Data? A conversation about grap...

Neo4j

CollabDays NL 2024 - Protecting and governing your sensitive data with Micros...

Jasper Oosterveld

GPT and Graph Data Science to power your Knowledge Graph

Neo4j

Nordics Edition - The Neo4j Graph Data Platform Today & Tomorrow

Neo4j

raph Databases with Neo4j – Emil Eifrem

buildacloud

Workshop - Architecting Innovative Graph Applications- GraphSummit Milan

Neo4j

Wikibon 2018 Predictions

plburris

The Art of the Possible with Graph by Dr Jim Webber Neo4j.pptx

Neo4j

Disrupting with Data: Lessons from Silicon Valley

Anand Rajaraman

Christian Lanng – Co Founder CEO, Trade Shift

Global Business Intelligence

Streamlining Nonprofit Organizations: It’s All About the Cloud

4Good.org

Similar to Advanced Retrieval Augmented Generation Techniques (20)

Vector Databases 101 - An introduction to the world of Vector Databases

Neo4j Keynote: The Art of the Possible with Graph Technology

Keynote Presentation at GraphTalk Oslo 2023

The Art of the Possible with Graph - Sudhir Hasbe - GraphSummit London 14 Nov...

GraphSummit Toronto: Keynote - Innovating with Graphs

Neo4j GraphSummit London March 2023 Emil Eifrem Keynote.pptx

The Data Platform for Today's Intelligent Applications.pdf

Jarrod Lopiccolo - Big Data

Introduction to Multilingual Retrieval Augmented Generation (RAG)

Are You Underestimating the Value Within Your Data? A conversation about grap...

CollabDays NL 2024 - Protecting and governing your sensitive data with Micros...

GPT and Graph Data Science to power your Knowledge Graph

Nordics Edition - The Neo4j Graph Data Platform Today & Tomorrow

raph Databases with Neo4j – Emil Eifrem

Workshop - Architecting Innovative Graph Applications- GraphSummit Milan

Wikibon 2018 Predictions

The Art of the Possible with Graph by Dr Jim Webber Neo4j.pptx

Disrupting with Data: Lessons from Silicon Valley

Christian Lanng – Co Founder CEO, Trade Shift

Streamlining Nonprofit Organizations: It’s All About the Cloud

More from Zilliz

Fueling AI with Great Data with Airbyte Webinar

Zilliz

Programming Foundation Models with DSPy - Meetup Slides

Zilliz

Generating privacy-protected synthetic data using Secludy and Milvus

Zilliz

During this demo, the founders of Secludy will demonstrate how their system utilizes Milvus to store and manipulate embeddings for generating privacy-protected synthetic data. Their approach not only maintains the confidentiality of the original data but also enhances the utility and scalability of LLMs under privacy constraints. Attendees, including machine learning engineers, data scientists, and data managers, will witness first-hand how Secludy's integration with Milvus empowers organizations to harness the power of LLMs securely and efficiently.

Building Production Ready Search Pipelines with Spark and Milvus

Zilliz

MemGPT: Introduction to Memory Augmented Chat

Zilliz

Copilot Workspace: What it is, how it works, why it matters

Zilliz

Infrastructure Challenges in Scaling RAG with Custom AI models

Zilliz

Building Retrieval-Augmented Generation (RAG) systems with open-source and custom AI models is a complex task. This talk explores the challenges in productionizing RAG systems, including retrieval performance, response synthesis, and evaluation. We’ll discuss how to leverage open-source models like text embeddings, language models, and custom fine-tuned models to enhance RAG performance. Additionally, we’ll cover how BentoML can help orchestrate and scale these AI components efficiently, ensuring seamless deployment and management of RAG systems in the cloud.

Full-RAG: A modern architecture for hyper-personalization

Zilliz

Mike Del Balso, CEO & Co-Founder at Tecton, presents "Full RAG," a novel approach to AI recommendation systems, aiming to push beyond the limitations of traditional models through a deep integration of contextual insights and real-time data, leveraging the Retrieval-Augmented Generation architecture. This talk will outline Full RAG's potential to significantly enhance personalization, address engineering challenges such as data management and model training, and introduce data enrichment with reranking as a key solution. Attendees will gain crucial insights into the importance of hyperpersonalization in AI, the capabilities of Full RAG for advanced personalization, and strategies for managing complex data integrations for deploying cutting-edge AI solutions.

Knowledge Graphs in Retrieval Augmented Generation with WhyHow.AI

Zilliz

Answer 'What's for Dinner?' with Vector Search and Natural Language using Hay...

Zilliz

What will you learn? Have you ever wanted a personal chef? You've probably heard the joke "being in a relationship is just asking each other 'what do you want to eat for dinner' until you die." Sure, you can just browse recipes online but who knows if they are any good? LLMs to the rescue! In this session, I'll demonstrate taking a dataset on Kaggle of my favorite cookbook recipes, pulling data into a Milvus vector database instance, and building an agentic Haystack RAG pipeline so I can search for tasty recipes with natural language. I'll even take it one step further with a function call to make an Amazon shopping list with the ingredients. Join us for this session to see how you can solve real-world problems with RAG and answer the age old question "what's for dinner?" Topics Covered - How to build a real-world RAG app - Getting started with Haystack - Ingesting data into Milvus

Introduction to Open Source RAG and RAG Evaluation

Zilliz

You’ve heard good data matters in Machine Learning, but does it matter for Generative AI applications? Corporate data often differs significantly from the general Internet data used to train most foundation models. Join me for a demo on building an open source RAG (Retrieval Augmented Generation) stack using Milvus vector database for Retrieval, LangChain, Llama 3 with Ollama, Ragas RAG Eval, and optional Zilliz cloud, OpenAI.

Emergent Methods: Multilingual narrative tracking in the news - real-time exp...

Zilliz

We present an architecture of embedding models, vector databases, LLMs, and narrow ML for tracking global news narratives across a variety of countries/languages/news sources in https://asknews.app/. As an example, we explore the real-time application of this architecture for tracking the news narrative surrounding the death of Russian opposition leader Alexei Navalny coming from Russian, French, and English sources

"I see eyes in my soup": How Delivery Hero implemented the safety system for ...

Zilliz

Exploring Multimodal Embeddings with Milvus

Zilliz

A Beginners Guide to Building a RAG App Using Open Source Milvus

Zilliz

Zilliz - Overview of Generative models in ML

Zilliz

Integrating Multimodal AI in Your Apps with Floom

Zilliz

Build streaming LLM with Timeplus and Zilliz

Zilliz

Beyond Retrieval Augmented Generation (RAG): Vector Databases

Zilliz

Voyage AI: cutting-edge embeddings and rerankers for search and RAG

Zilliz

More from Zilliz (20)

Fueling AI with Great Data with Airbyte Webinar

Programming Foundation Models with DSPy - Meetup Slides

Generating privacy-protected synthetic data using Secludy and Milvus

Building Production Ready Search Pipelines with Spark and Milvus

MemGPT: Introduction to Memory Augmented Chat

Copilot Workspace: What it is, how it works, why it matters

Infrastructure Challenges in Scaling RAG with Custom AI models

Full-RAG: A modern architecture for hyper-personalization

Knowledge Graphs in Retrieval Augmented Generation with WhyHow.AI

Answer 'What's for Dinner?' with Vector Search and Natural Language using Hay...

Introduction to Open Source RAG and RAG Evaluation

Emergent Methods: Multilingual narrative tracking in the news - real-time exp...

"I see eyes in my soup": How Delivery Hero implemented the safety system for ...

Exploring Multimodal Embeddings with Milvus

A Beginners Guide to Building a RAG App Using Open Source Milvus

Zilliz - Overview of Generative models in ML

Integrating Multimodal AI in Your Apps with Floom

Build streaming LLM with Timeplus and Zilliz

Beyond Retrieval Augmented Generation (RAG): Vector Databases

Voyage AI: cutting-edge embeddings and rerankers for search and RAG

Recently uploaded

WeTestAthens: Postman's AI & Automation Techniques

Postman

Unlock the Future of Search with MongoDB Atlas_ Vector Search Unleashed.pdf

Malak Abu Hammad

Discover how MongoDB Atlas and vector search technology can revolutionize your application's search capabilities. This comprehensive presentation covers: * What is Vector Search? * Importance and benefits of vector search * Practical use cases across various industries * Step-by-step implementation guide * Live demos with code snippets * Enhancing LLM capabilities with vector search * Best practices and optimization strategies Perfect for developers, AI enthusiasts, and tech leaders. Learn how to leverage MongoDB Atlas to deliver highly relevant, context-aware search results, transforming your data retrieval process. Stay ahead in tech innovation and maximize the potential of your applications. #MongoDB #VectorSearch #AI #SemanticSearch #TechInnovation #DataScience #LLM #MachineLearning #SearchTechnology

Salesforce Integration for Bonterra Impact Management (fka Social Solutions A...

Jeffrey Haguewood

Sidekick Solutions uses Bonterra Impact Management (fka Social Solutions Apricot) and automation solutions to integrate data for business workflows. We believe integration and automation are essential to user experience and the promise of efficient work through technology. Automation is the critical ingredient to realizing that full vision. We develop integration products and services for Bonterra Case Management software to support the deployment of automations for a variety of use cases. This video focuses on integration of Salesforce with Bonterra Impact Management. Interested in deploying an integration with Salesforce for Bonterra Impact Management? Contact us at sales@sidekicksolutionsllc.com to discuss next steps.

Energy Efficient Video Encoding for Cloud and Edge Computing Instances

Alpen-Adria-Universität

Recommendation System using RAG Architecture

fredae14

Nunit vs XUnit vs MSTest Differences Between These Unit Testing Frameworks.pdf

flufftailshop

When it comes to unit testing in the .NET ecosystem, developers have a wide range of options available. Among the most popular choices are NUnit, XUnit, and MSTest. These unit testing frameworks provide essential tools and features to help ensure the quality and reliability of code. However, understanding the differences between these frameworks is crucial for selecting the most suitable one for your projects.

System Design Case Study: Building a Scalable E-Commerce Platform - Hiike

Hiike

Digital Marketing Trends in 2024 | Guide for Staying Ahead

Wask

https://www.wask.co/ebooks/digital-marketing-trends-in-2024 Feeling lost in the digital marketing whirlwind of 2024? Technology is changing, consumer habits are evolving, and staying ahead of the curve feels like a never-ending pursuit. This e-book is your compass. Dive into actionable insights to handle the complexities of modern marketing. From hyper-personalization to the power of user-generated content, learn how to build long-term relationships with your audience and unlock the secrets to success in the ever-shifting digital landscape.

AWS Cloud Cost Optimization Presentation.pptx

HarisZaheer8

This presentation provides valuable insights into effective cost-saving techniques on AWS. Learn how to optimize your AWS resources by rightsizing, increasing elasticity, picking the right storage class, and choosing the best pricing model. Additionally, discover essential governance mechanisms to ensure continuous cost efficiency. Whether you are new to AWS or an experienced user, this presentation provides clear and practical tips to help you reduce your cloud costs and get the most out of your budget.

Taking AI to the Next Level in Manufacturing.pdf

ssuserfac0301

Read Taking AI to the Next Level in Manufacturing to gain insights on AI adoption in the manufacturing industry, such as: 1. How quickly AI is being implemented in manufacturing. 2. Which barriers stand in the way of AI adoption. 3. How data quality and governance form the backbone of AI. 4. Organizational processes and structures that may inhibit effective AI adoption. 6. Ideas and approaches to help build your organization's AI strategy.

Trusted Execution Environment for Decentralized Process Mining

LucaBarbaro3

Let's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with Slack

shyamraj55

Azure API Management to expose backend services securely

Dinusha Kumarasiri

Letter and Document Automation for Bonterra Impact Management (fka Social Sol...

Jeffrey Haguewood

Sidekick Solutions uses Bonterra Impact Management (fka Social Solutions Apricot) and automation solutions to integrate data for business workflows. We believe integration and automation are essential to user experience and the promise of efficient work through technology. Automation is the critical ingredient to realizing that full vision. We develop integration products and services for Bonterra Case Management software to support the deployment of automations for a variety of use cases. This video focuses on automated letter generation for Bonterra Impact Management using Google Workspace or Microsoft 365. Interested in deploying letter generation automations for Bonterra Impact Management? Contact us at sales@sidekicksolutionsllc.com to discuss next steps.

Monitoring and Managing Anomaly Detection on OpenShift.pdf

Tosin Akinosho

Monitoring and Managing Anomaly Detection on OpenShift Overview Dive into the world of anomaly detection on edge devices with our comprehensive hands-on tutorial. This SlideShare presentation will guide you through the entire process, from data collection and model training to edge deployment and real-time monitoring. Perfect for those looking to implement robust anomaly detection systems on resource-constrained IoT/edge devices. Key Topics Covered 1. Introduction to Anomaly Detection - Understand the fundamentals of anomaly detection and its importance in identifying unusual behavior or failures in systems. 2. Understanding Edge (IoT) - Learn about edge computing and IoT, and how they enable real-time data processing and decision-making at the source. 3. What is ArgoCD? - Discover ArgoCD, a declarative, GitOps continuous delivery tool for Kubernetes, and its role in deploying applications on edge devices. 4. Deployment Using ArgoCD for Edge Devices - Step-by-step guide on deploying anomaly detection models on edge devices using ArgoCD. 5. Introduction to Apache Kafka and S3 - Explore Apache Kafka for real-time data streaming and Amazon S3 for scalable storage solutions. 6. Viewing Kafka Messages in the Data Lake - Learn how to view and analyze Kafka messages stored in a data lake for better insights. 7. What is Prometheus? - Get to know Prometheus, an open-source monitoring and alerting toolkit, and its application in monitoring edge devices. 8. Monitoring Application Metrics with Prometheus - Detailed instructions on setting up Prometheus to monitor the performance and health of your anomaly detection system. 9. What is Camel K? - Introduction to Camel K, a lightweight integration framework built on Apache Camel, designed for Kubernetes. 10. Configuring Camel K Integrations for Data Pipelines - Learn how to configure Camel K for seamless data pipeline integrations in your anomaly detection workflow. 11. What is a Jupyter Notebook? - Overview of Jupyter Notebooks, an open-source web application for creating and sharing documents with live code, equations, visualizations, and narrative text. 12. Jupyter Notebooks with Code Examples - Hands-on examples and code snippets in Jupyter Notebooks to help you implement and test anomaly detection models.

Ocean lotus Threat actors project by John Sitima 2024 (1).pptx

SitimaJohn

Ocean Lotus cyber threat actors represent a sophisticated, persistent, and politically motivated group that poses a significant risk to organizations and individuals in the Southeast Asian region. Their continuous evolution and adaptability underscore the need for robust cybersecurity measures and international cooperation to identify and mitigate the threats posed by such advanced persistent threat groups.

Deep Dive: Getting Funded with Jason Jason Lemkin Founder & CEO @ SaaStr

saastr

Presentation of the OECD Artificial Intelligence Review of Germany

innovationoecd

Introduction of Cybersecurity with OSS at Code Europe 2024

Hiroshi SHIBATA

I develop the Ruby programming language, RubyGems, and Bundler, which are package managers for Ruby. Today, I will introduce how to enhance the security of your application using open-source software (OSS) examples from Ruby and RubyGems. The first topic is CVE (Common Vulnerabilities and Exposures). I have published CVEs many times. But what exactly is a CVE? I'll provide a basic understanding of CVEs and explain how to detect and handle vulnerabilities in OSS. Next, let's discuss package managers. Package managers play a critical role in the OSS ecosystem. I'll explain how to manage library dependencies in your application. I'll share insights into how the Ruby and RubyGems core team works to keep our ecosystem safe. By the end of this talk, you'll have a better understanding of how to safeguard your code.

Artificial Intelligence for XMLDevelopment

Octavian Nadolu

In the rapidly evolving landscape of technologies, XML continues to play a vital role in structuring, storing, and transporting data across diverse systems. The recent advancements in artificial intelligence (AI) present new methodologies for enhancing XML development workflows, introducing efficiency, automation, and intelligent capabilities. This presentation will outline the scope and perspective of utilizing AI in XML development. The potential benefits and the possible pitfalls will be highlighted, providing a balanced view of the subject. We will explore the capabilities of AI in understanding XML markup languages and autonomously creating structured XML content. Additionally, we will examine the capacity of AI to enrich plain text with appropriate XML markup. Practical examples and methodological guidelines will be provided to elucidate how AI can be effectively prompted to interpret and generate accurate XML markup. Further emphasis will be placed on the role of AI in developing XSLT, or schemas such as XSD and Schematron. We will address the techniques and strategies adopted to create prompts for generating code, explaining code, or refactoring the code, and the results achieved. The discussion will extend to how AI can be used to transform XML content. In particular, the focus will be on the use of AI XPath extension functions in XSLT, Schematron, Schematron Quick Fixes, or for XML content refactoring. The presentation aims to deliver a comprehensive overview of AI usage in XML development, providing attendees with the necessary knowledge to make informed decisions. Whether you’re at the early stages of adopting AI or considering integrating it in advanced XML development, this presentation will cover all levels of expertise. By highlighting the potential advantages and challenges of integrating AI with XML development tools and languages, the presentation seeks to inspire thoughtful conversation around the future of XML development. We’ll not only delve into the technical aspects of AI-powered XML development but also discuss practical implications and possible future directions.

Recently uploaded (20)

WeTestAthens: Postman's AI & Automation Techniques

Unlock the Future of Search with MongoDB Atlas_ Vector Search Unleashed.pdf

Salesforce Integration for Bonterra Impact Management (fka Social Solutions A...

Energy Efficient Video Encoding for Cloud and Edge Computing Instances

Recommendation System using RAG Architecture

Nunit vs XUnit vs MSTest Differences Between These Unit Testing Frameworks.pdf

System Design Case Study: Building a Scalable E-Commerce Platform - Hiike

Digital Marketing Trends in 2024 | Guide for Staying Ahead

AWS Cloud Cost Optimization Presentation.pptx

Taking AI to the Next Level in Manufacturing.pdf

Trusted Execution Environment for Decentralized Process Mining

Let's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with Slack

Azure API Management to expose backend services securely

Letter and Document Automation for Bonterra Impact Management (fka Social Sol...

Monitoring and Managing Anomaly Detection on OpenShift.pdf

Ocean lotus Threat actors project by John Sitima 2024 (1).pptx

Deep Dive: Getting Funded with Jason Jason Lemkin Founder & CEO @ SaaStr

Presentation of the OECD Artificial Intelligence Review of Germany

Introduction of Cybersecurity with OSS at Code Europe 2024

Artificial Intelligence for XMLDevelopment

Advanced Retrieval Augmented Generation Techniques

4. 4 | © Copyright 2024 Zilliz 4 The evolution of AI made the semantic search of unstructured data possible Search by Probability Statistical analyses of common datasets established the foundation for processing unstructured data, e.g. NLP, and image classification AI Model Breakthrough The advancements in BERT, ViT, CBT etc. have revolutionized semantic analysis across unstructured data Vectorization Word2Vec, CNNs, Deep Speech pioneered unstructured data embeddings, mapping the words, images, videos into high-dimensional vectors

7. 7 | © Copyright 2024 Zilliz 7 Why RAG? RAG vs. LLM - Knowledge of LLM is out-of-date - LLM can not get your private knowledge - Hallucinations - Transparency and interpretability RAG vs. Fine-tune - Fine-tune is expensive - Fine-tune spent much time - RAG is pluggable

12. 12 | © Copyright 2024 Zilliz 12 Types of RAG Enhancement Techniques ● Divide & Conquer ○ Query Enhancement: better express or process the query intent. ○ Indexing Enhancement: data cleanup, better parser and chunking ○ Retriever Enhancement: more retrievers and hybrid search strategy ○ Generator Enhancement: prompt engineering and more powerful LLM ● Thinking outside the box ○ Agents? Other tools than retriever?

16. 16 | © Copyright 2024 Zilliz 16 What are the differences in features between Milvus and Zilliz Cloud? Sub query1: What are the features of Milvus? Sub query2: What are the features of Zilliz Cloud?

35. 35 | © Copyright 2024 Zilliz 35 35 | © Copyright 9/25/23 Zilliz 35 | © Copyright 9/25/23 Zilliz Simplify and streamline the conversion of unstructured data into state-of-the-art vector embeddings, using intuitive UI and Restful APIs. Pipelines Easy. High-quality. Scalable. Simplify the workflow for developers, from converting unstructured data into searchable vectors to retrieving them from vector databases Deliver excellence in every phase of vector search pipeline development and deployment, regardless of their expertise Ensure scalability for managing large datasets and high-throughput queries, maintaining high performance with min. customization or infra changes Zilliz Cloud Pipelines

Advanced Retrieval Augmented Generation Techniques

Recommended

Recommended

More Related Content

Similar to Advanced Retrieval Augmented Generation Techniques

Similar to Advanced Retrieval Augmented Generation Techniques (20)

More from Zilliz

More from Zilliz (20)

Recently uploaded

Recently uploaded (20)

Advanced Retrieval Augmented Generation Techniques