This is Nicholas Roze-Freitas's speaker slides from the October Milvus Community Meetup.

1. CONFIDENTIAL
Building Agentic Systems
By Nicholas Roze-Freitas

2. My Background
• 10 Months in SAP Academy for Product & Engineering
• Studied Data Science and Math at UCSB
• Data Scientist building agents at SAP Concur

3. Agentic Systems?
• Different categories of Agentic Systems
• independent, fully autonomous
• More prescriptive implementation that
follows a flow
• LangChain’s blog writes often times Agentic
Systems in production are a combination

4. Complex
Reasoning
Dynamic
Decision-making
Unstructured
Data
When are Agents good to use?

5. Main Components Of An Agent
• LLM
• Tools
• Memory
• Short term
• Long term
Langgraph

6. Building Agents
• Tons of frameworks
• Langchain / Langgraph
• CrewAI
• Google ADK
• OpenAI Agents SDK
• … and more!
• Agents themselves are pretty simple
• Most frameworks implement the same loop
Also Langgraph

7. The Loop
• Feed Messages to LLM
• If LLM called tools
• Call tools & add results to messages
• If LLM responds with text
• Get user feedback if applicable
• Or do something custom
Again Langgraph

8. Some Hidden Challenges
• Security
• Malformed & incorrect tool calling
• Message history and context management
Agents
Sometimes

9. Hidden Challenge: Security
• LLMs can generate malicious outputs
• User/company data isolation
• Console commands
• Solution?
• Design interactions with implicit guardrails
• User/company based instances of tools
Agents
Sometimes
(again)

10. Hidden Challenge: Malformed Function Calls
• LLMs can fail to generate function calls
• Malformed function calling can break up
flows
• Solution?
• Define tools with primitives
• Implement custom error handling

11. Hidden Challenge: Context
• Agents can forget things
• Context helps agents understand their task and
get information
• Solution?
• Keep most important info at top of context
• Design RAG systems to pull context
dynamically
Agents Thinking
Really Hard

12. My Library of choice: LangChain & LangGraph

13. Why I like Lang(Chain/Graph)
• Offers high and low level APIs
• Represents agentic systems as a graph
• Low level APIs give the ability to build highly
customizable solutions
• Building blocks
• Mitigations
• Compliance
Some Companies Using
LangGraph

14. Getting Started
• LangGraph/Langchain offers high level APIs that are
easy to start with
• create_agent
• middleware
• Or get into the weeds and just start with LangGraph!

15. What I’m Excited About
• Fine-tuned small language models
• Large model inference is expensive & unnecessary
• Agent’s often handle specialized tasks

16. Thank you + Questions
Connect with me on LinkedIn @ Nicholas
Roze-Freitas