.NET User Group Meetup with Christian Weyer about Large Language Models, Data & APIs - Integrating Generative AI Power into your solutions - with Python and .NET
Beyond Boundaries: Leveraging No-Code Solutions for Industry Innovation
Large Language Models, Data & APIs - Integrating Generative AI Power into your solutions - with Python and .NET
1. Large Language Models, Data & APIs
Integrating Generative AI Power
into your solutions - with Python & .NET
Christian Weyer
@christianweyer
CTO, Technology Catalyst
2. § Technology catalyst
§ AI-powered solutions
§ Pragmatic end-to-end architectures
§ Microsoft Regional Director
§ Microsoft MVP for Developer Technologies & Azure
ASPInsider, AzureInsider
§ Google GDE for Web Technologies
christian.weyer@thinktecture.com @christianweyer https://www.thinktecture.com
Large Language Models, Data & APIs
Integrating Generative AI Power into your solutions - with Python & .NET
Christian Weyer
Co-Founder & CTO @ Thinktecture AG
2
3. Large Language Models, Data & APIs
Integrating Generative AI Power into your solutions - with Python & .NET
Our journey
3
AI all-the-
things?
Integrating
LLMs
Selected
Scenarios
End-to-End
Exciting
Times…
Democratizing
Generative AI
4. Large Language Models, Data & APIs
Integrating Generative AI Power into your solutions - with Python & .NET
AI all-the-things?
4
5. Large Language Models, Data & APIs
Integrating Generative AI Power into your solutions - with Python & .NET
AI all-the-things?
5
Data
Science
Artificial Intelligence
Machine Learning
Unsupervised, supervised, reinforcement learning
Deep Learning
ANN, CNN, RNN etc.
NLP
Generative AI
GAN, VAE, Transformers etc.
Image / Video Generation
GAN, VAE
Large Language Models
Transformers
6. § LLMs generate text based on input
§ LLMs can understand text – this changes a lot
§ Prompts are the universal interface (“UI”) →
unstructured text with semantics
§ Human language evolves as a first-class citizen in
software architecture 🤯
Large Language Models, Data & APIs
Integrating Generative AI Power into your solutions - with Python & .NET
Large Language Models (LLMs)
6
Text… – really, just text?
7. § LLMs are programs
§ LLMs are highly specialized neural networks
§ LLMs use(d) lots of data
§ LLMs need a lot of resources to be operated
§ LLMs have an API to be used through
Large Language Models, Data & APIs
Integrating Generative AI Power into your solutions - with Python & .NET
Large Language Models demystified
7
8. Large Language Models, Data & APIs
Integrating Generative AI Power into your solutions - with Python & .NET
Integrating LLMs
8
9. § LLMs are always part of end-to-end architectures
§ HTTP/Web/REST APIs
§ Databases
§ Client apps (Web, desktop, mobile)
§ etc.
§ An LLM is ‘just’ an additional asset in your architecture
§ It is not the Holy Grail for everything!
Large Language Models, Data & APIs
Integrating Generative AI Power into your solutions - with Python & .NET
End-to-end architectures with LLMs
9
10. Large Language Models, Data & APIs
Integrating Generative AI Power into your solutions - with Python & .NET
Using LLMs: It’s just APIs !
Inference, FTW.
10
11. GPT-4 API access via OpenAI Playground
Large Language Models, Data & APIs
Integrating Generative AI Power into your solutions - with Python & .NET
DEMO
11
12. Large Language Models, Data & APIs
Integrating Generative AI Power into your solutions - with Python & .NET
The best tool for .NET developers to talk to LLMs!
12
🙈
13. § OSS framework for developing applications powered by LLMs
§ > 1000 contributors
§ Python and Typescript versions
§ Chains for sequences of LLM-related actions in code
§ Abstractions for
§ Prompts & LLMs (local and remote)
§ Memory
§ Vector stores
§ Tools
§ Loading text from a wide range of sources
Large Language Models, Data & APIs
Integrating Generative AI Power into your solutions - with Python & .NET
LangChain - building LLM-based applications
13
14. § Microsoft’s OSS framework to integrate LLMs into applications
§ .NET, Python, and Java versions
§ .NET 1.0 RTM in December 2023
§ Plugins encapsulate AI capabilities
§ Semantic functions for prompting
§ Native functions to run local code
§ Planners are orchestrating LLMS interactions
§ Not as broad feature set as LangChain
§ E.g., no concept/abstraction for loading data
Large Language Models, Data & APIs
Integrating Generative AI Power into your solutions - with Python & .NET
Semantic Kernel - building LLM-based applications
14
15. Large Language Models, Data & APIs
Integrating Generative AI Power into your solutions - with Python & .NET
Selected
Scenarios
15
16. Text generation
§ LLMs are good in generating text
§ Regular text
§ Code
§ SQL (beware!)
§ JSON
§ etc.
Large Language Models, Data & APIs
Integrating Generative AI Power into your solutions - with Python & .NET
Typical LLM scenarios:
16
17. Extracting meaning in text
§ LLM can be instructed to, e.g.
§ Do sentiment analysis
§ Extract information from text
§ Extracting structured information
§ JSON, TypeScript types, etc.
§ Via tools like Kor, TypeChat, or Open AI Functions
Large Language Models, Data & APIs
Integrating Generative AI Power into your solutions - with Python & .NET
Typical LLM scenarios:
17
18. Extracting structured data
(LangChain + Kor)
Large Language Models, Data & APIs
Integrating Generative AI Power into your solutions - with Python & .NET
DEMO
18
19. Large Language Models, Data & APIs
Integrating Generative AI Power into your solutions - with Python & .NET
Answering Questions on Data - Retrieval-augmented generation (RAG)
Cleanup & Split Text Embedding
Question
Text Embedding
Save
Query
Relevant Text
Question
A
n
s
w
e
r
LLM
19
Vector DB
Embedding
model
Embedding
model
💡
Indexing / Embedding QA
20. Learning about my company’s policies via Slack
(LangChain)
Large Language Models, Data & APIs
Integrating Generative AI Power into your solutions - with Python & .NET
DEMO
20
21. Chat with web site documents
(Semantic Kernel)
Large Language Models, Data & APIs
Integrating Generative AI Power into your solutions - with Python & .NET
DEMO
21
22. Large Language Models, Data & APIs
Integrating Generative AI Power into your solutions - with Python & .NET
Democratizing
Generative AI
22
23. Large Language Models, Data & APIs
Integrating Generative AI Power into your solutions - with Python & .NET
LLMs everywhere
OpenAI-related (cloud)
OpenAI
Azure OpenAI
Service
Big cloud providers
Google Model Garden
on Vertex AI
Amazon Bedrock
Other providers
Antrophic
Cohere
HuggingFace
… Open-source
Edge
IoT Server
Desktop Mobile
Web
Open-source
23
24. § Open-source community drives
innovation in Generative AI
§ HuggingFace is central place for it
§ Literally, every week a new and
“better” LLM shows up 🤓
§ Important factors
§ Use case
§ Parameter size
§ Quantization
§ Processing power needed
§ Mistral-based family shows big
potential for local use cases (7B params)
Large Language Models, Data & APIs
Integrating Generative AI Power into your solutions - with Python & .NET
Open-source LLMs thrive
24
25. Local RAG with Mistral OSS LLM
(llama.cpp & LM Studio)
Large Language Models, Data & APIs
Integrating Generative AI Power into your solutions - with Python & .NET
DEMO
25
26. Large Language Models, Data & APIs
Integrating Generative AI Power into your solutions - with Python & .NET
End-to-End
26
27. Ask for expert availability in my company systems
(Speech-to-text, my own API, GPT-4, Text-to-speech)
Large Language Models, Data & APIs
Integrating Generative AI Power into your solutions - with Python & .NET
DEMO
27
28. Large Language Models, Data & APIs
Integrating Generative AI Power into your solutions - with Python & .NET
Exciting Times
28
29. § LLMs enable new scenarios & use cases to
incorporate human language into software solutions
§ Fast moving and changing field
§ Every week something “big” happens in LLM space
§ Frameworks & ecosystem are evolving together with LLMs
§ Closed vs open LLMs
§ Competition drives invention & advancement
§ SISO (sh*t in, sh*t out)
§ Quality of results heavily depends on your data & input
Large Language Models, Data & APIs
Integrating Generative AI Power into your solutions - with Python & .NET
Current state
29
33. § Semantic Kernel
§ https://learn.microsoft.com/en-us/semantic-kernel/overview/
§ LangChain
§ https://www.langchain.com/
§ LangChain Agents
§ https://python.langchain.com/docs/modules/agents/
§ ReAct: Synergizing Reasoning and Acting in Language Models
§ https://react-lm.github.io/
§ Prompt Engineering Guide
§ https://www.promptingguide.ai/
§ OpenAI API reference
§ https://platform.openai.com/docs/api-reference
§ Azure OpenAI Service REST API reference
§ https://learn.microsoft.com/en-us/azure/ai-services/openai/reference
§ Hugging Face Inference Endpoints (for various OSS LLMs)
§ https://huggingface.co/docs/inference-endpoints/api_reference
§ Llama2 on Hugging Face
§ https://huggingface.co/blog/llama2
§ OWASP Top 10 for LLM Applications
§ https://owasp.org/www-project-top-10-for-large-language-model-applications/assets/PDF/OWASP-Top-10-for-LLMs-2023-slides-v1_0_1.pdf
Large Language Models, Data & APIs
Integrating Generative AI Power into your solutions - with Python & .NET
Links
33