SlideShare a Scribd company logo
1 of 16
Download to read offline
Integrating Multimodal AI in Your Apps
MaxBrin,Co-Founder&CEO
CTO Cyber @ Amdocs
Co-Founder & CEO @ Cards
Research & Innovation Lead @ CyberArk
Co-Founder & CEO of Hoger
Chief Architect @ Enabley
About me
What is Floom trying to solve?
The gap between AI Potential
and AI Integration
Where’s the gap?
No robust dev/dep infra
Technical barrier
No Centralized Management
Inspiration Deficit
Existing AI Integration Flow
AI
Model
App
Code
Library
Gateway
Logging, Caching,
Routing, Security etc.
Logic Coating
Data Ingestion, RAG,
Embeddings
Centralization
Floom, AI Orchestrator
Open-Source
Floom
AIOrchestrator
AI Model
Multimodal
App
AI Functions
Direct Inference + Agents
Extract Physical Addresses
Generate Speech from Text
Detect Objects in Image
Detect Emotion in Text
Classify PG rating in Video Ask questions about data
Invoke Tools for action Generate Image
AI Functions
Deploy
RAG (Context)
Now... Run it!
Why did we choose Milvus DB?
Lowest latency
Friendly, well documented APIs
Predictable under load!
Passed our rigorous production testing
How does Milvus DB enable Floom?
Floom relies on Milvus for RAG and general embeddings search (CS/L2/IP)
Default Vector DB for Floom Docker (part of docker-compose)
Default Vector DB for Floom Cloud (Zilliz Cloud)
Full RAG + Multimodal (Video!) experimentation
Most companies keep GenAI as POC
Most product managers focus on the classic GenAI examples
(text+code generation)
Generalizing GenAI is hard, but feasible and well worth it.
GenAI integration is currently reserved to GenAI enthusiasts, Data
Science or Innovation team
Using current dev/dep methodologies, GenAI is not yet
production-grade
Key Lessons
Roadmap:
AI Definition Markup Language
● Generalized approach towards developing and maintaining AI integrations
● Distilled to “AI Functions” (including both agents and direct inference)
● Easy to understand, build, debug and maintain across versatile teams
● Designed for developers with no to little prior AI/GenAI knowledge
● Easily packed in a distributable format
○ Definition (AI Functions: Inference + Agents)
○ Configuration
○ Orchestration
Let’s Floom.
Questions?

More Related Content

Similar to Integrating Multimodal AI in Your Apps with Floom

Similar to Integrating Multimodal AI in Your Apps with Floom (20)

Deploying Enterprise Search in PLM Context with Aras
Deploying Enterprise Search in PLM Context with ArasDeploying Enterprise Search in PLM Context with Aras
Deploying Enterprise Search in PLM Context with Aras
 
Beyond MDM: 5 Things You Must do to Secure Mobile Devices in the Enterprise
Beyond MDM: 5 Things You Must do to Secure Mobile Devices in the EnterpriseBeyond MDM: 5 Things You Must do to Secure Mobile Devices in the Enterprise
Beyond MDM: 5 Things You Must do to Secure Mobile Devices in the Enterprise
 
Home robots meet IBM Watson for Voice UI, and AI
Home robots meet IBM Watson for Voice UI, and AIHome robots meet IBM Watson for Voice UI, and AI
Home robots meet IBM Watson for Voice UI, and AI
 
The Ball Launch on 2013 Microsoft TechDays Part 1/2
The Ball Launch on 2013 Microsoft TechDays Part 1/2The Ball Launch on 2013 Microsoft TechDays Part 1/2
The Ball Launch on 2013 Microsoft TechDays Part 1/2
 
Preventive Maintenance and IoE
Preventive Maintenance and IoEPreventive Maintenance and IoE
Preventive Maintenance and IoE
 
Microsoft Cognitive Service, Tap into the Power of Machine Learning with Easy...
Microsoft Cognitive Service, Tap into the Power of Machine Learning with Easy...Microsoft Cognitive Service, Tap into the Power of Machine Learning with Easy...
Microsoft Cognitive Service, Tap into the Power of Machine Learning with Easy...
 
GDG Cloud Southlake 31: Santosh Chennuri and Festus Yeboah: Empowering Develo...
GDG Cloud Southlake 31: Santosh Chennuri and Festus Yeboah: Empowering Develo...GDG Cloud Southlake 31: Santosh Chennuri and Festus Yeboah: Empowering Develo...
GDG Cloud Southlake 31: Santosh Chennuri and Festus Yeboah: Empowering Develo...
 
From Reversing to Exploitation: Android Application Security in Essence
From Reversing to Exploitation: Android Application Security in EssenceFrom Reversing to Exploitation: Android Application Security in Essence
From Reversing to Exploitation: Android Application Security in Essence
 
Internet of Things, Cloud & Artificial Intelligence
Internet of Things, Cloud & Artificial IntelligenceInternet of Things, Cloud & Artificial Intelligence
Internet of Things, Cloud & Artificial Intelligence
 
SharePoint Mobile App Development with Xmarin
SharePoint Mobile App Development with XmarinSharePoint Mobile App Development with Xmarin
SharePoint Mobile App Development with Xmarin
 
From Reversing to Exploitation
From Reversing to ExploitationFrom Reversing to Exploitation
From Reversing to Exploitation
 
IBM MobileFirst Reference Architecture 1512 v3 2015
IBM MobileFirst Reference Architecture 1512 v3 2015IBM MobileFirst Reference Architecture 1512 v3 2015
IBM MobileFirst Reference Architecture 1512 v3 2015
 
Innovation morning power platform
Innovation morning power platformInnovation morning power platform
Innovation morning power platform
 
Frog Trade's Presentation
Frog Trade's PresentationFrog Trade's Presentation
Frog Trade's Presentation
 
The Big Connection: Integrating Cloud with Enterprise Systems
The Big Connection: Integrating Cloud with Enterprise SystemsThe Big Connection: Integrating Cloud with Enterprise Systems
The Big Connection: Integrating Cloud with Enterprise Systems
 
TechDayPakistan-Slides RAG with Cosmos DB.pptx
TechDayPakistan-Slides RAG with Cosmos DB.pptxTechDayPakistan-Slides RAG with Cosmos DB.pptx
TechDayPakistan-Slides RAG with Cosmos DB.pptx
 
Marco Polo
Marco PoloMarco Polo
Marco Polo
 
SharePoint Saturday Warsaw - Conversational AI applications in Microsoft Teams
SharePoint Saturday Warsaw - Conversational AI applications in Microsoft TeamsSharePoint Saturday Warsaw - Conversational AI applications in Microsoft Teams
SharePoint Saturday Warsaw - Conversational AI applications in Microsoft Teams
 
Azure Logic Apps & AI - Building Integration & AI Solutions
Azure Logic Apps & AI - Building Integration & AI SolutionsAzure Logic Apps & AI - Building Integration & AI Solutions
Azure Logic Apps & AI - Building Integration & AI Solutions
 
OUGN 2018 - Chatbot and the need to integrate
OUGN 2018 - Chatbot and the need to integrateOUGN 2018 - Chatbot and the need to integrate
OUGN 2018 - Chatbot and the need to integrate
 

More from Zilliz

More from Zilliz (19)

Emergent Methods: Multilingual narrative tracking in the news - real-time exp...
Emergent Methods: Multilingual narrative tracking in the news - real-time exp...Emergent Methods: Multilingual narrative tracking in the news - real-time exp...
Emergent Methods: Multilingual narrative tracking in the news - real-time exp...
 
Introduction to Multilingual Retrieval Augmented Generation (RAG)
Introduction to Multilingual Retrieval Augmented Generation (RAG)Introduction to Multilingual Retrieval Augmented Generation (RAG)
Introduction to Multilingual Retrieval Augmented Generation (RAG)
 
"I see eyes in my soup": How Delivery Hero implemented the safety system for ...
"I see eyes in my soup": How Delivery Hero implemented the safety system for ..."I see eyes in my soup": How Delivery Hero implemented the safety system for ...
"I see eyes in my soup": How Delivery Hero implemented the safety system for ...
 
Exploring Multimodal Embeddings with Milvus
Exploring Multimodal Embeddings with MilvusExploring Multimodal Embeddings with Milvus
Exploring Multimodal Embeddings with Milvus
 
A Beginners Guide to Building a RAG App Using Open Source Milvus
A Beginners Guide to Building a RAG App Using Open Source MilvusA Beginners Guide to Building a RAG App Using Open Source Milvus
A Beginners Guide to Building a RAG App Using Open Source Milvus
 
Zilliz - Overview of Generative models in ML
Zilliz - Overview of Generative models in MLZilliz - Overview of Generative models in ML
Zilliz - Overview of Generative models in ML
 
Build streaming LLM with Timeplus and Zilliz
Build streaming LLM with Timeplus and ZillizBuild streaming LLM with Timeplus and Zilliz
Build streaming LLM with Timeplus and Zilliz
 
Beyond Retrieval Augmented Generation (RAG): Vector Databases
Beyond Retrieval Augmented Generation (RAG): Vector DatabasesBeyond Retrieval Augmented Generation (RAG): Vector Databases
Beyond Retrieval Augmented Generation (RAG): Vector Databases
 
Chunking, Embeddings, and Vector Databases
Chunking, Embeddings, and Vector DatabasesChunking, Embeddings, and Vector Databases
Chunking, Embeddings, and Vector Databases
 
Introduction to Large Language Model Customization.pdf
Introduction to Large Language Model Customization.pdfIntroduction to Large Language Model Customization.pdf
Introduction to Large Language Model Customization.pdf
 
Voyage AI: cutting-edge embeddings and rerankers for search and RAG
Voyage AI: cutting-edge embeddings and rerankers for search and RAGVoyage AI: cutting-edge embeddings and rerankers for search and RAG
Voyage AI: cutting-edge embeddings and rerankers for search and RAG
 
Leverage Zilliz Serverless - Up to 50X Saving for Your Vector Storage Cost
Leverage Zilliz Serverless - Up to 50X Saving for Your Vector Storage CostLeverage Zilliz Serverless - Up to 50X Saving for Your Vector Storage Cost
Leverage Zilliz Serverless - Up to 50X Saving for Your Vector Storage Cost
 
Training state-of-the-art general text embedding
Training state-of-the-art general text embeddingTraining state-of-the-art general text embedding
Training state-of-the-art general text embedding
 
Fact vs. Fiction: Autodetecting Hallucinations in LLMs
Fact vs. Fiction: Autodetecting Hallucinations in LLMsFact vs. Fiction: Autodetecting Hallucinations in LLMs
Fact vs. Fiction: Autodetecting Hallucinations in LLMs
 
Vector Databases 101 - An introduction to the world of Vector Databases
Vector Databases 101 - An introduction to the world of Vector DatabasesVector Databases 101 - An introduction to the world of Vector Databases
Vector Databases 101 - An introduction to the world of Vector Databases
 
VectorDB Schema Design 101 - Considerations for Building a Scalable and Perfo...
VectorDB Schema Design 101 - Considerations for Building a Scalable and Perfo...VectorDB Schema Design 101 - Considerations for Building a Scalable and Perfo...
VectorDB Schema Design 101 - Considerations for Building a Scalable and Perfo...
 
Voyage AI Embedding Models for Retrieval Augmented Generation
Voyage AI Embedding Models for Retrieval Augmented GenerationVoyage AI Embedding Models for Retrieval Augmented Generation
Voyage AI Embedding Models for Retrieval Augmented Generation
 
Chat with your data, privately and locally
Chat with your data, privately and locallyChat with your data, privately and locally
Chat with your data, privately and locally
 
Introducing Milvus and new features in 2.4 release
Introducing Milvus and new features in 2.4 releaseIntroducing Milvus and new features in 2.4 release
Introducing Milvus and new features in 2.4 release
 

Recently uploaded

Easier, Faster, and More Powerful – Alles Neu macht der Mai -Wir durchleuchte...
Easier, Faster, and More Powerful – Alles Neu macht der Mai -Wir durchleuchte...Easier, Faster, and More Powerful – Alles Neu macht der Mai -Wir durchleuchte...
Easier, Faster, and More Powerful – Alles Neu macht der Mai -Wir durchleuchte...
panagenda
 
Hyatt driving innovation and exceptional customer experiences with FIDO passw...
Hyatt driving innovation and exceptional customer experiences with FIDO passw...Hyatt driving innovation and exceptional customer experiences with FIDO passw...
Hyatt driving innovation and exceptional customer experiences with FIDO passw...
FIDO Alliance
 
Harnessing Passkeys in the Battle Against AI-Powered Cyber Threats.pptx
Harnessing Passkeys in the Battle Against AI-Powered Cyber Threats.pptxHarnessing Passkeys in the Battle Against AI-Powered Cyber Threats.pptx
Harnessing Passkeys in the Battle Against AI-Powered Cyber Threats.pptx
FIDO Alliance
 

Recently uploaded (20)

State of the Smart Building Startup Landscape 2024!
State of the Smart Building Startup Landscape 2024!State of the Smart Building Startup Landscape 2024!
State of the Smart Building Startup Landscape 2024!
 
ChatGPT and Beyond - Elevating DevOps Productivity
ChatGPT and Beyond - Elevating DevOps ProductivityChatGPT and Beyond - Elevating DevOps Productivity
ChatGPT and Beyond - Elevating DevOps Productivity
 
Frisco Automating Purchase Orders with MuleSoft IDP- May 10th, 2024.pptx.pdf
Frisco Automating Purchase Orders with MuleSoft IDP- May 10th, 2024.pptx.pdfFrisco Automating Purchase Orders with MuleSoft IDP- May 10th, 2024.pptx.pdf
Frisco Automating Purchase Orders with MuleSoft IDP- May 10th, 2024.pptx.pdf
 
Event-Driven Architecture Masterclass: Engineering a Robust, High-performance...
Event-Driven Architecture Masterclass: Engineering a Robust, High-performance...Event-Driven Architecture Masterclass: Engineering a Robust, High-performance...
Event-Driven Architecture Masterclass: Engineering a Robust, High-performance...
 
Easier, Faster, and More Powerful – Alles Neu macht der Mai -Wir durchleuchte...
Easier, Faster, and More Powerful – Alles Neu macht der Mai -Wir durchleuchte...Easier, Faster, and More Powerful – Alles Neu macht der Mai -Wir durchleuchte...
Easier, Faster, and More Powerful – Alles Neu macht der Mai -Wir durchleuchte...
 
Continuing Bonds Through AI: A Hermeneutic Reflection on Thanabots
Continuing Bonds Through AI: A Hermeneutic Reflection on ThanabotsContinuing Bonds Through AI: A Hermeneutic Reflection on Thanabots
Continuing Bonds Through AI: A Hermeneutic Reflection on Thanabots
 
Easier, Faster, and More Powerful – Notes Document Properties Reimagined
Easier, Faster, and More Powerful – Notes Document Properties ReimaginedEasier, Faster, and More Powerful – Notes Document Properties Reimagined
Easier, Faster, and More Powerful – Notes Document Properties Reimagined
 
الأمن السيبراني - ما لا يسع للمستخدم جهله
الأمن السيبراني - ما لا يسع للمستخدم جهلهالأمن السيبراني - ما لا يسع للمستخدم جهله
الأمن السيبراني - ما لا يسع للمستخدم جهله
 
Hyatt driving innovation and exceptional customer experiences with FIDO passw...
Hyatt driving innovation and exceptional customer experiences with FIDO passw...Hyatt driving innovation and exceptional customer experiences with FIDO passw...
Hyatt driving innovation and exceptional customer experiences with FIDO passw...
 
Event-Driven Architecture Masterclass: Integrating Distributed Data Stores Ac...
Event-Driven Architecture Masterclass: Integrating Distributed Data Stores Ac...Event-Driven Architecture Masterclass: Integrating Distributed Data Stores Ac...
Event-Driven Architecture Masterclass: Integrating Distributed Data Stores Ac...
 
Harnessing Passkeys in the Battle Against AI-Powered Cyber Threats.pptx
Harnessing Passkeys in the Battle Against AI-Powered Cyber Threats.pptxHarnessing Passkeys in the Battle Against AI-Powered Cyber Threats.pptx
Harnessing Passkeys in the Battle Against AI-Powered Cyber Threats.pptx
 
JohnPollard-hybrid-app-RailsConf2024.pptx
JohnPollard-hybrid-app-RailsConf2024.pptxJohnPollard-hybrid-app-RailsConf2024.pptx
JohnPollard-hybrid-app-RailsConf2024.pptx
 
Generative AI Use Cases and Applications.pdf
Generative AI Use Cases and Applications.pdfGenerative AI Use Cases and Applications.pdf
Generative AI Use Cases and Applications.pdf
 
WebAssembly is Key to Better LLM Performance
WebAssembly is Key to Better LLM PerformanceWebAssembly is Key to Better LLM Performance
WebAssembly is Key to Better LLM Performance
 
Working together SRE & Platform Engineering
Working together SRE & Platform EngineeringWorking together SRE & Platform Engineering
Working together SRE & Platform Engineering
 
Portal Kombat : extension du réseau de propagande russe
Portal Kombat : extension du réseau de propagande russePortal Kombat : extension du réseau de propagande russe
Portal Kombat : extension du réseau de propagande russe
 
Microsoft CSP Briefing Pre-Engagement - Questionnaire
Microsoft CSP Briefing Pre-Engagement - QuestionnaireMicrosoft CSP Briefing Pre-Engagement - Questionnaire
Microsoft CSP Briefing Pre-Engagement - Questionnaire
 
ADP Passwordless Journey Case Study.pptx
ADP Passwordless Journey Case Study.pptxADP Passwordless Journey Case Study.pptx
ADP Passwordless Journey Case Study.pptx
 
Six Myths about Ontologies: The Basics of Formal Ontology
Six Myths about Ontologies: The Basics of Formal OntologySix Myths about Ontologies: The Basics of Formal Ontology
Six Myths about Ontologies: The Basics of Formal Ontology
 
TEST BANK For, Information Technology Project Management 9th Edition Kathy Sc...
TEST BANK For, Information Technology Project Management 9th Edition Kathy Sc...TEST BANK For, Information Technology Project Management 9th Edition Kathy Sc...
TEST BANK For, Information Technology Project Management 9th Edition Kathy Sc...
 

Integrating Multimodal AI in Your Apps with Floom

  • 2. MaxBrin,Co-Founder&CEO CTO Cyber @ Amdocs Co-Founder & CEO @ Cards Research & Innovation Lead @ CyberArk Co-Founder & CEO of Hoger Chief Architect @ Enabley About me
  • 3. What is Floom trying to solve? The gap between AI Potential and AI Integration
  • 4. Where’s the gap? No robust dev/dep infra Technical barrier No Centralized Management Inspiration Deficit
  • 5. Existing AI Integration Flow AI Model App Code Library Gateway Logging, Caching, Routing, Security etc. Logic Coating Data Ingestion, RAG, Embeddings Centralization
  • 7. AI Functions Direct Inference + Agents Extract Physical Addresses Generate Speech from Text Detect Objects in Image Detect Emotion in Text Classify PG rating in Video Ask questions about data Invoke Tools for action Generate Image
  • 11.
  • 12. Why did we choose Milvus DB? Lowest latency Friendly, well documented APIs Predictable under load! Passed our rigorous production testing
  • 13. How does Milvus DB enable Floom? Floom relies on Milvus for RAG and general embeddings search (CS/L2/IP) Default Vector DB for Floom Docker (part of docker-compose) Default Vector DB for Floom Cloud (Zilliz Cloud) Full RAG + Multimodal (Video!) experimentation
  • 14. Most companies keep GenAI as POC Most product managers focus on the classic GenAI examples (text+code generation) Generalizing GenAI is hard, but feasible and well worth it. GenAI integration is currently reserved to GenAI enthusiasts, Data Science or Innovation team Using current dev/dep methodologies, GenAI is not yet production-grade Key Lessons
  • 15. Roadmap: AI Definition Markup Language ● Generalized approach towards developing and maintaining AI integrations ● Distilled to “AI Functions” (including both agents and direct inference) ● Easy to understand, build, debug and maintain across versatile teams ● Designed for developers with no to little prior AI/GenAI knowledge ● Easily packed in a distributable format ○ Definition (AI Functions: Inference + Agents) ○ Configuration ○ Orchestration