AI and APIs: Peanut Butter and Jelly - Andrew Carlson, Apollo

AI and APIs: Peanut Butter and Jelly - Andrew Carlson, Apollo

• 1.
  AI and APIs: Peanutbutter and jelly Platform Summit, 2024
• 2.
  I champion intent AndrewI. Carlson Principal Field Architect at Apollo linkedin.com/in/andrewicarlson/ andrew@apollographql.com
• 3.
  My name isAndrew Carlson, and I’m an AI skeptic
• 4.
  My name isAndrew Carlson, and I’m an AI skeptic
• 5.
  Blind-faith in AIis risky
• 6.
  My turning point: Alocal LLM + my own data
• 7.
  My turning point: Alocal LLM + my own data
• 8.
  Using data inLLM-driven experiences is straightforward Vector databases Raw SQL queries
• 9.
  But most experiencesneed more than just data Who played Laena in House of the Dragon? When will my order arrive? vs.
• 10.
  But most experiencesneed more than just data Who played Laena in House of the Dragon? When will my order arrive? vs. Nanna Vondell
• 11.
  But most experiencesneed more than just data Who played Laena in House of the Dragon? When will my order arrive? vs. Nanna Vondell User
• 12.
  But most experiencesneed more than just data Who played Laena in House of the Dragon? When will my order arrive? vs. Nanna Vondell User Order
• 13.
  But most experiencesneed more than just data Who played Laena in House of the Dragon? When will my order arrive? vs. Nanna Vondell User Order Inventory
• 14.
  But most experiencesneed more than just data Who played Laena in House of the Dragon? When will my order arrive? vs. Nanna Vondell Inventory User Order Shipper Today at 3pm
• 15.
  Take action on behalfof the user Meaningful experiences need to take action
• 16.
  Take action on behalfof the user Meaningful experiences need to perform business logic Perform business logic
• 17.
  Perform business logic Take actionon behalf of the user Rapidly add capabilities Meaningful experiences need to evolve
• 18.
  Meaningful experiences needAPIs
• 19.
  >30% of theincrease in demand for APIs will come from AI and LLMs by 2026 GARTNER PREDICTS
• 20.
  Supply and demand Producerand consumer API and client
• 21.
  A LLM isjust another client Android Web iOS Watch
• 22.
  A LLM isjust another client LLM Android Web iOS Watch
• 23.
  How? Client API
• 24.
  API Access How? Client API
• 25.
  Today’s approaches don’tsolve this Raw SQL ● LLM generates SQL script ● Direct db access
• 26.
  Trade-offs: Today’s approaches don’tsolve this Raw SQL ● LLM generates SQL script ● Direct db access
• 27.
  Today’s approaches don’tsolve this Simple RAG ● Vector dbs MongoDB, pgVector) ● Often one-step Raw SQL ● LLM generates SQL script ● Direct db access
• 28.
  Today’s approaches don’tsolve this Simple RAG ● Vector dbs MongoDB, pgVector) ● Often one-step Raw SQL ● LLM generates SQL script ● Direct db access Trade-offs: 1. Data consistency concerns when synchronizing data 2. Must maintain data integrity between SoR and vector database 3. Returning static data
• 29.
  Extensions Today’s approaches don’tsolve this ● “Function callingˮ APIs ● Plugins Simple RAG ● Vector dbs MongoDB, pgVector) ● Often one-step Raw SQL ● LLM generates SQL script ● Direct db access
• 30.
  Extensions Today’s approaches don’tsolve this ● “Function callingˮ APIs ● Plugins Simple RAG ● Vector dbs MongoDB, pgVector) ● Often one-step Raw SQL ● LLM generates SQL script ● Direct db access Trade-offs: 1. Client-side API orchestration 2. New backend-for-frontend APIs to write and maintain 3. Re-deploy for each new capability
• 31.
  My favorite approach: alittle bit of everything
• 32.
  For API access:use multiple methods + Extensions: Make an API request Simple RAG Whatʼs available and where
• 33.
  Extensions: Make an API request ForAPI access: use multiple methods Simple RAG Whatʼs available and where GraphQL Query plan across many APIs + +
• 34.
  With federated GraphQL: ManyAPIs from a single endpoint AI / LLM Experience Federated GraphQL User Inventory Product APIs
• 35.
  AI / LLM Experience Federated GraphQL APIs SafelistedPersisted Queries 📄 Contracts 📄 With federated GraphQL: Secure access with contracts and PQs User Inventory Product
• 36.
  AI / LLM Experience Federated GraphQL Reviews APIs SafelistedPersisted Queries 📄 Contracts 📄 With federated GraphQL: Ship new capabilities without a deploy User Inventory Product
• 37.
  Customer LLM query {…} Router Asksa question Products Cart Collections Review Checkout APIs Query-planned API Access
• 38.
  Customer LLM query {…} Router Asksa question Products Cart Collections Review Checkout APIs Safelisted operations Vector DB Similarity search With intentional governance
• 39.
  Demo
• 40.
  Meaningful experiences needAPIs
• 41.
  Using federated GraphQLfor AI and API data access Simple access to APIs Trusted access to data Rapid iteration
• 42.
  Try it foryourself Demo Github Guide
• 43.
  Thank you