Recommended
Recommended
More Related Content
What's hot
What's hot (20)
Similar to PyBay23: Understanding LangChain Agents and Tools with Twilio (or with SMS).pptx
Similar to PyBay23: Understanding LangChain Agents and Tools with Twilio (or with SMS).pptx (20)
More from Elizabeth (Lizzie) Siegle
More from Elizabeth (Lizzie) Siegle (20)
Recently uploaded
Recently uploaded (20)
PyBay23: Understanding LangChain Agents and Tools with Twilio (or with SMS).pptx
- 1. Understanding LangChain Agents && Tools🛠 w/ Twilio ☎️ ( || SMS) @lizziepika PyBay 2023
- 2. Hi 👋🏽🌊 ● ☎️👩🏻💻Twilio devangelism since '17 ● 😍Python🐍, 🚴♀️, 🏃♀️,📚,☕️
- 3. Order of Operations - Intro to LLM Apps - What is LangChain?⛓️ - Modules🧱 - Live code w/ modules 🧱 - SMS with Twilio📲☎️ - What's Next? 🔜
- 4. "The hottest new programming language is English" 🔥- Andrej Karpathy
- 5. Apps around LLMs - Chatbots, virtual assistants - Content generation and automation - Summarization - Extraction - Sentiment analysis, text classification - Question answering systems - Code generation && debugging
- 6. - Framework for making LLM apps - "Chain" ⛓️ together components - Abstractions - 🐍 + JS What is LangChain?
- 7. LangChain Modules - Prompt templates: templates for different types of prompts - “chatbot” style templates, ELI5 question- answering, etc - LLMs: wrapper for OpenAI GPT-3, Cohere, etc - ⛓️Chains and Agents: who decides what actions to take? - 🧠Memory🧠: Short-term memory, long-term memory.
- 8. Prompt Templates - pre-defined recipes to generate LLM prompts - responsible for constructing the final prompt to pass to an LLM. - The same object can be reused with different data ie prompts
- 9. LLMs - Different providers (OpenAI, Cohere, Hugging Face, etc)
- 10. Chains ⛓️ - predetermined set of steps - Ex: - LLMChain (simple prompt + resp) - Chat over docs w/ chat history - Q&A with sources - API Chains
- 11. Agents - LLM determines actions - Action agents vs Plan-and- execute agents - Tool, memory access
- 12. Tools - Class instances agents use to perform specific tasks, utilities - Google Search, Bash, Python REPL, Zapier, ReadFileTool
- 13. Executor - Type of agent + set of tools - Calls agent -> returns next action -> calls tool req'd
- 20. Chains⛓️ vs Agents🕵 ♀️ - Chain = infra/tooling - Agent = unknown chain with tools - LLM decides actions
- 22. Why LangChain
- 23. Connect
- 25. Community - MLOps Community - Medium, Dev.to
- 27. What's Next? - Play🧱!! - different models, parameters - ☎️ promo code: SEATTLEJS23 - TF.js, Synaptic, 🧱.js
- 30. Thank you! 👋🏽 - @lizziepika - lsiegle@twilio.com - promo code: PYBAY23 Slides:
Editor's Notes
- studied general compsci in college, just a regular developer, who gained an interest in AI🤖 It's fun. I've been having so much fun this year just like playing around with it. To paraphrase the LangChain CEO/cofounder, now is a great time to be a builder.
- Agenda SMS = input to model via English, in our case
- Founding Member/Research Scientist OpenAI, Sr. Director of AI at Tesla, where he led the computer vision team of Tesla Autopilot. What does he mean? At an AI conf a few weeks ago and an attendee asked the founder/builder of LangChain about getting better output. He said be creating better "prompts" or input text that guide language models, like GPT, to produce the output we want. The better the input, the better the output.
- What takes inputs? LLM apps. These can include chatbots, virtual assistants, and they can be used to automatically generate summaries and synopses of long documents, making it easier for people to digest complex information. They could generate articles, blog posts, social media captions, product descriptions, and more. By training on vast amounts of text data, these models can capture the style, tone, and structure of different genres and produce coherent and contextually relevant content. Large language models can also assist in content automation by generating content based on specific parameters or prompts.
- I'm here today to talk about LangChain, which makes it easier to build LLM applications. They do not pay me, I'm just a fan. At its core, LangChain is an open source framework built around LLMs that we can use for chatbots, Generative Question-Answering (GQA), summarization, and much more. It provides abstractions for working with LLMs, along with a collection of implementations for each abstraction. Components are modular and easy-to-use, whether or not you are using the rest of the LangChain framework The CEO/builder went to AI meetups in SF (click) and heard pain points developers felt when working with LLMs, when building LLM apps. Also JS support is expanding, but Python provides more!
- The core idea of the library is that we can “chain” together different components, different model or LLM calls, to create more advanced use cases around LLMs. Chains may consist of multiple components from several modules which we will go over: reusable prompt that can be used to generate prompts for language models Won't dig into memory much in this talk
- Again, prompts are input to the model and are often structured in different ways before presenting themselves to the model so that we can get different results. LangChain provides several classes and functions to make constructing and working with prompts easy. A template is a reproducible, pre-defined way to generate prompts: may include instructions, few shot examples (where the model looks at examples), and specific context and questions appropriate for a given task. For Q&A, we could take a user’s question and reformat it for different Q&A styles, like conventional Q&A, a bullet list of answers, or even a summary of problems relevant to the given question.
- LangChain makes it easier to replace different models in your code via abstractions for working with LLMs The LLM module is simply a wrapper around different LLMs. This wrapper makes it simple for LangChain developers to communicate with different LLMs using a single interface, without having to worry about the difference between the different LLMs. What makes them different? Different parameters (determining how input data is transformed into the desired output ), accuracy, speed, training, scale, price…different company missions: While OpenAI's mission is Artificial General Intelligence (AGI), Cohere AI is focussed on fulfilling their customer's technological needs.
- Chains in LangChain (where the name comes from!) are wrappers around a series of single components, completing predetermined sequence of events that are hardcoded in code. Chains break down complex tasks into smaller steps w/ a series of interconnected components that work together to process user input. The output of one step is used as input for the next. In a basic application, you probably wouldn’t even need to use a chain, but in more complex applications involving multiple LLMs (such as when we get to Agents), the Chain component provides a standard interface for interacting with LLMs. Examples of Chains include: … API Chains (interact with third-party APIs)
- Use Agent = a LLM setup to run iteratively with some goals / tasks defined. LLM to determine what actions to take and in what order. often used to interact with data sources to assist with problem-solving. Agents have complex workflows where the model essentially talks to itself without a human forcing every part of the interaction Action agents: at each timestep, decide on the next action using the outputs of all previous actions Plan-and-execute agents: decide on the full sequence of actions up front, then execute them all without updating the plan Agent = memory + tool use (but some agents don’t fit all of those)
- Agents don't know everything! Tools are functions that agents can use to perform…interact with the outside world. These tools can be generic utilities (e.g. search–useful to search for things that happened after 2021 because OpenAI's models were trained on data up to 2021 ), other chains, or even other agents. …WriteFileTool
- Executor = an Agent + a set of Tools. More specifically, it's a running instance of an agent that is used to execute tasks based on the agent’s decisions and configured tools. The agents decide which tools to use, and the executors executes them
- Been talking a lot, now I'm going to show you. let's dig into some code
- "chain' is infrastructure/tooling, a sequence of actions that are hardcoded in code. "agent" is worker with tools, utilities, an unknown chain that depends on the user's input and the model has more say. Whereas a chain defines an immediate input/output process, the logic of agents allows a step-by-step thought process.
- audience participation: you can test out some of what we saw by texting this number! Only pitch of this whole talk–Twilio provides APIs for communication and customer engagement. SMS, in this case, is a means of prompting the model.
- Like $twlo Makes life easier Easier to make apps around LLM
- LangChain allows you to connect LLMs to other sources of data. This is important because LLMs are trained on massive datasets of text and code. However, they can only access the information that is contained in those datasets. By connecting LLMs to other sources of data, you can give them access to a wider range of information.
- We're all in this together, all learning together (click) community
- tutorials
- Docs lol
- Play. have fun. Other libraries.
- Course
- New frontier
- Code