SlideShare a Scribd company logo
Architecting AI
in the Enterprise:
APIs and
Applications
Malith Jayasinghe
VP of Research and AI, WSO2
Enterprise AI: Growth Forecast 2022−2030
● Global enterprise AI market was valued at $136.55 billion in 2022 (Forbes,
2022)
● Expected to grow at a CAGR of 36.6% from 2024 to 2030, reaching
$1,811.8 billion by 2030 (Grand View Research, 2024)
● By 2026, over 80% of enterprises are anticipated to have used generative
AI APIs/model and/or have deployed generative AI application in
production (Gartner, 2023)
● More companies are experimenting with generative AI; however, the
percentage of applications entering production is low (5% in 2023) ,
though it is increasing (Gartner, 2023)
2
Value Proposition in AI Development
● Understanding AI's value is key to
unlocking its potential. Strategic
integration of AI can enhance
efficiency, improve user
experience, provide competitive
advantages, and prepare
organizations for the future
3
AI Development Challenges
● Objective: Provide insights into
potential challenges and strategies
for addressing them (Specific Focus:
generative AI applications)
● Challenges: As you develop your AI
application and attempt to move it
into production, you will need to
address many challenges
● UI/UX Considerations
⦿ Initiate design early
⦿ Consider design aspects deeply
4
AI Development Challenges
5
Our Recent Gen AI Features
Selecting the Right Model
● Model selection criteria
⦿ Accuracy (Accuracy vs. speed?)
⦿ Cost (Accuracy vs. cost?)
⦿ Speed (Accuracy vs Speed?)
⦿ Risks
⦿ Other Considerations: Self-hosted
model or Public cloud hosted, Model
availability in specific regions
● Dependency on use case
● Trade-offs in model selection
● Given AI feature can use multiple
models
6
Source: https://informationisbeautiful.net/
Identifying Data Sources
● Model improvements (build a RAG,
Fine tune, pre-train)
● Identify the available data sources
(e.g., internal R&D, product
documentation)
● Data cleaning and preprocessing
⦿ Conduct data cleaning
⦿ Data leakage: LLM accidentally
reveals sensitive information (PIIs,
proprietary algorithms)
● Do I need to collect new data?
7
AI Interaction with APIs
● What are the available APIs?
● APIs can differ in type and have
different representations
⦿ RESTful APIs
⦾ OpenAPI specification
⦿ GraphQL APIs
⦾ GraphQL Schema
● Well-documented APIs perform
better when integrated with AI
● AI can bridge the gaps in missing
documentation
8
Prompt Engineering
● Prompting engineering
⦿ Prompt engineering is guiding
generative AI to produce desired
outputs
⦿ A prompt is a natural language
instruction describing how generative
AI should perform (acts as the
interface between human intent and
machine output)
● Prompting
⦿ Zero-shot prompting, Few-shot
prompting
⦿ Chain-of-Thought (COT), Tree of
Thoughts (TOT)
⦿ ReAct (Reason and Act)
9
https://www.promptingguide.ai/
Prompt Engineering: ReAct (Reason and Act)
● LLM generation provides information, guidance, or suggestions. Their
output doesn't (inherently) drive actions
● ReAct Agents can perform actions (e.g., calling APIs)
● ReAct Agents with the assistance of the model, can execute natural
language commands using a collection of tools registered with them
● Tools can retrieve data or perform tasks, and they come in different forms,
such as functions, API, etc.
10
Use Case: Train Booking System
11
Users
Chat Ul
(Web site)
Train Booking
Service
(Agent Service)
Existing Business APIs
Train API
Email API
Payment API
invokes
Swagger
Specs
(OpenAPI)
Initialization
Evaluating Accuracy
● Create a dataset for evaluation
purposes
● Select a metric (use existing one or
develop new metrics if necessary)
● Automate the evaluation of accuracy
● Re-evaluate after any changes to
confirm that accuracy has not
diminished
● Even minor changes to prompts can
significantly affect accuracy
12
Collecting User Feedback + Improving the Accuracy
● Collect data: Gather feedback on AI
outputs, user interactions, and
engagement metrics
● Compliance: Ensure user consent and
follow privacy laws. Anonymize
sensitive data
● Improve models: Update vector DBs,
fine-tune models, improve prompts
13
CLIENT AGENT
LLM
1
Performance (Latency, Throughput)
14
Enhancing User Experience for Tasks with Long Processing
Times
● Complex Tasks: AI systems execute complex tasks that require multiple
reasoning iterations and involve invoking multiple APIs
● Optimizations: Techniques such as caching are used, but latency can still
be high
● User Satisfaction: Prolonged round-trip times can result in user
dissatisfaction
● Strategies to Improve User Experience:
⦿ Incremental Display: Progressively display partial results as AI systems process
different APIs, thereby reducing wait times for final outcomes
15
CLIENT AGENT
API
1
LLM
1
Enhancing User Experience for Tasks with Long Processing Times
16
Closing Remarks
● Purposeful Beginnings: We start with a clear definition of the problem and a keen
understanding of the application's value, guiding our development journey from the
outset
● User-Centric Design: Prioritizing UI/UX considerations ensures that our applications
are accessible, intuitive, and meet user expectations
● Technical Execution
⦿ Model Selection/Model Improvements
⦿ Dataset Utilization/API Integration
⦿ Prompt Engineering
⦿ Accuracy evaluation/Continuous feedback
⦿ Scalability and Performance
● Forward Outlook: The development of generative AI applications is a dynamic,
ongoing process requiring constant learning and adjustment to meet evolving
technological trends and user demands
17
Question Time!
18
Thank You!

More Related Content

Similar to architecting-ai-in-the-enterprise-apis-and-applications.pdf

Objects.ai Platform Overview
Objects.ai Platform OverviewObjects.ai Platform Overview
Objects.ai Platform Overview
Shekhar Yadav
 
How to Build An AI Based Customer Data Platform: Learn the design patterns fo...
How to Build An AI Based Customer Data Platform: Learn the design patterns fo...How to Build An AI Based Customer Data Platform: Learn the design patterns fo...
How to Build An AI Based Customer Data Platform: Learn the design patterns fo...
TigerGraph
 
Resume
ResumeResume
Using Data Science to Build an End-to-End Recommendation System
Using Data Science to Build an End-to-End Recommendation SystemUsing Data Science to Build an End-to-End Recommendation System
Using Data Science to Build an End-to-End Recommendation System
VMware Tanzu
 
Ai design sprint - Finance - Wealth management
Ai design sprint  - Finance - Wealth managementAi design sprint  - Finance - Wealth management
Ai design sprint - Finance - Wealth management
Chinmay Patel
 
How is ChatGPT and AI Revolutionizing Adaptive Project Portfolio Management
How is ChatGPT and AI Revolutionizing Adaptive Project Portfolio ManagementHow is ChatGPT and AI Revolutionizing Adaptive Project Portfolio Management
How is ChatGPT and AI Revolutionizing Adaptive Project Portfolio Management
OnePlan Solutions
 
Data_and_Analytics_Industry_IESE_v3.pdf
Data_and_Analytics_Industry_IESE_v3.pdfData_and_Analytics_Industry_IESE_v3.pdf
Data_and_Analytics_Industry_IESE_v3.pdf
prevota
 
Accelerate ML Deployment with H2O Driverless AI on AWS
Accelerate ML Deployment with H2O Driverless AI on AWSAccelerate ML Deployment with H2O Driverless AI on AWS
Accelerate ML Deployment with H2O Driverless AI on AWS
Sri Ambati
 
harnessing_the_power_of_artificial_intelligence_for_software_development.pptx
harnessing_the_power_of_artificial_intelligence_for_software_development.pptxharnessing_the_power_of_artificial_intelligence_for_software_development.pptx
harnessing_the_power_of_artificial_intelligence_for_software_development.pptx
sarah david
 
AI in Software Development.pptx
AI in Software Development.pptxAI in Software Development.pptx
AI in Software Development.pptx
Genic Solutions
 
Trouble with Performance Debugging? Not Anymore with Choreo, the AI-Assisted ...
Trouble with Performance Debugging? Not Anymore with Choreo, the AI-Assisted ...Trouble with Performance Debugging? Not Anymore with Choreo, the AI-Assisted ...
Trouble with Performance Debugging? Not Anymore with Choreo, the AI-Assisted ...
WSO2
 
AppliFire Platform
AppliFire PlatformAppliFire Platform
AppliFire Platform
AppliFire Platform
 
Group 3 slide presentation
Group 3 slide presentationGroup 3 slide presentation
Group 3 slide presentation
Michael Young
 
Dev Dives: Unlock automation adoption with UiPath Apps
Dev Dives: Unlock automation adoption with UiPath AppsDev Dives: Unlock automation adoption with UiPath Apps
Dev Dives: Unlock automation adoption with UiPath Apps
Cristina Vidu
 
Mindfields x Workato - The Role of API-based Automation in Enhancing Employee...
Mindfields x Workato - The Role of API-based Automation in Enhancing Employee...Mindfields x Workato - The Role of API-based Automation in Enhancing Employee...
Mindfields x Workato - The Role of API-based Automation in Enhancing Employee...
Mindfields Global
 
ANIn Hyderabad Nov2023|Generative AI for software engineering - Lakshman Peet...
ANIn Hyderabad Nov2023|Generative AI for software engineering - Lakshman Peet...ANIn Hyderabad Nov2023|Generative AI for software engineering - Lakshman Peet...
ANIn Hyderabad Nov2023|Generative AI for software engineering - Lakshman Peet...
AgileNetwork
 
Rsqrd AI: From R&D to ROI of AI
Rsqrd AI: From R&D to ROI of AIRsqrd AI: From R&D to ROI of AI
Rsqrd AI: From R&D to ROI of AI
Sanjana Chowdhury
 
The Future of Software Exploring AI Machine Learning and Beyond
The Future of Software Exploring AI Machine Learning and BeyondThe Future of Software Exploring AI Machine Learning and Beyond
The Future of Software Exploring AI Machine Learning and Beyond
Polyxer Systems
 
Building Intelligent Apps with MongoDB and Google Cloud - Jane Fine
Building Intelligent Apps with MongoDB and Google Cloud - Jane FineBuilding Intelligent Apps with MongoDB and Google Cloud - Jane Fine
Building Intelligent Apps with MongoDB and Google Cloud - Jane Fine
MongoDB
 
Data Science Meets DevOps: GitOps with OpenShift (1).pdf
Data Science Meets DevOps: GitOps with OpenShift (1).pdfData Science Meets DevOps: GitOps with OpenShift (1).pdf
Data Science Meets DevOps: GitOps with OpenShift (1).pdf
HemaVeeradhi1
 

Similar to architecting-ai-in-the-enterprise-apis-and-applications.pdf (20)

Objects.ai Platform Overview
Objects.ai Platform OverviewObjects.ai Platform Overview
Objects.ai Platform Overview
 
How to Build An AI Based Customer Data Platform: Learn the design patterns fo...
How to Build An AI Based Customer Data Platform: Learn the design patterns fo...How to Build An AI Based Customer Data Platform: Learn the design patterns fo...
How to Build An AI Based Customer Data Platform: Learn the design patterns fo...
 
Resume
ResumeResume
Resume
 
Using Data Science to Build an End-to-End Recommendation System
Using Data Science to Build an End-to-End Recommendation SystemUsing Data Science to Build an End-to-End Recommendation System
Using Data Science to Build an End-to-End Recommendation System
 
Ai design sprint - Finance - Wealth management
Ai design sprint  - Finance - Wealth managementAi design sprint  - Finance - Wealth management
Ai design sprint - Finance - Wealth management
 
How is ChatGPT and AI Revolutionizing Adaptive Project Portfolio Management
How is ChatGPT and AI Revolutionizing Adaptive Project Portfolio ManagementHow is ChatGPT and AI Revolutionizing Adaptive Project Portfolio Management
How is ChatGPT and AI Revolutionizing Adaptive Project Portfolio Management
 
Data_and_Analytics_Industry_IESE_v3.pdf
Data_and_Analytics_Industry_IESE_v3.pdfData_and_Analytics_Industry_IESE_v3.pdf
Data_and_Analytics_Industry_IESE_v3.pdf
 
Accelerate ML Deployment with H2O Driverless AI on AWS
Accelerate ML Deployment with H2O Driverless AI on AWSAccelerate ML Deployment with H2O Driverless AI on AWS
Accelerate ML Deployment with H2O Driverless AI on AWS
 
harnessing_the_power_of_artificial_intelligence_for_software_development.pptx
harnessing_the_power_of_artificial_intelligence_for_software_development.pptxharnessing_the_power_of_artificial_intelligence_for_software_development.pptx
harnessing_the_power_of_artificial_intelligence_for_software_development.pptx
 
AI in Software Development.pptx
AI in Software Development.pptxAI in Software Development.pptx
AI in Software Development.pptx
 
Trouble with Performance Debugging? Not Anymore with Choreo, the AI-Assisted ...
Trouble with Performance Debugging? Not Anymore with Choreo, the AI-Assisted ...Trouble with Performance Debugging? Not Anymore with Choreo, the AI-Assisted ...
Trouble with Performance Debugging? Not Anymore with Choreo, the AI-Assisted ...
 
AppliFire Platform
AppliFire PlatformAppliFire Platform
AppliFire Platform
 
Group 3 slide presentation
Group 3 slide presentationGroup 3 slide presentation
Group 3 slide presentation
 
Dev Dives: Unlock automation adoption with UiPath Apps
Dev Dives: Unlock automation adoption with UiPath AppsDev Dives: Unlock automation adoption with UiPath Apps
Dev Dives: Unlock automation adoption with UiPath Apps
 
Mindfields x Workato - The Role of API-based Automation in Enhancing Employee...
Mindfields x Workato - The Role of API-based Automation in Enhancing Employee...Mindfields x Workato - The Role of API-based Automation in Enhancing Employee...
Mindfields x Workato - The Role of API-based Automation in Enhancing Employee...
 
ANIn Hyderabad Nov2023|Generative AI for software engineering - Lakshman Peet...
ANIn Hyderabad Nov2023|Generative AI for software engineering - Lakshman Peet...ANIn Hyderabad Nov2023|Generative AI for software engineering - Lakshman Peet...
ANIn Hyderabad Nov2023|Generative AI for software engineering - Lakshman Peet...
 
Rsqrd AI: From R&D to ROI of AI
Rsqrd AI: From R&D to ROI of AIRsqrd AI: From R&D to ROI of AI
Rsqrd AI: From R&D to ROI of AI
 
The Future of Software Exploring AI Machine Learning and Beyond
The Future of Software Exploring AI Machine Learning and BeyondThe Future of Software Exploring AI Machine Learning and Beyond
The Future of Software Exploring AI Machine Learning and Beyond
 
Building Intelligent Apps with MongoDB and Google Cloud - Jane Fine
Building Intelligent Apps with MongoDB and Google Cloud - Jane FineBuilding Intelligent Apps with MongoDB and Google Cloud - Jane Fine
Building Intelligent Apps with MongoDB and Google Cloud - Jane Fine
 
Data Science Meets DevOps: GitOps with OpenShift (1).pdf
Data Science Meets DevOps: GitOps with OpenShift (1).pdfData Science Meets DevOps: GitOps with OpenShift (1).pdf
Data Science Meets DevOps: GitOps with OpenShift (1).pdf
 

More from WSO2

Accelerate Enterprise Software Engineering with Platformless
Accelerate Enterprise Software Engineering with PlatformlessAccelerate Enterprise Software Engineering with Platformless
Accelerate Enterprise Software Engineering with Platformless
WSO2
 
WSO2Con2024 - WSO2's IAM Vision: Identity-Led Digital Transformation
WSO2Con2024 - WSO2's IAM Vision: Identity-Led Digital TransformationWSO2Con2024 - WSO2's IAM Vision: Identity-Led Digital Transformation
WSO2Con2024 - WSO2's IAM Vision: Identity-Led Digital Transformation
WSO2
 
Driving Innovation: Scania's API Revolution with WSO2
Driving Innovation: Scania's API Revolution with WSO2Driving Innovation: Scania's API Revolution with WSO2
Driving Innovation: Scania's API Revolution with WSO2
WSO2
 
Less Is More: Utilizing Ballerina to Architect a Cloud Data Platform
Less Is More: Utilizing Ballerina to Architect a Cloud Data PlatformLess Is More: Utilizing Ballerina to Architect a Cloud Data Platform
Less Is More: Utilizing Ballerina to Architect a Cloud Data Platform
WSO2
 
Modernizing Legacy Systems Using Ballerina
Modernizing Legacy Systems Using BallerinaModernizing Legacy Systems Using Ballerina
Modernizing Legacy Systems Using Ballerina
WSO2
 
WSO2CON 2024 - Unlocking the Identity: Embracing CIAM 2.0 for a Competitive A...
WSO2CON 2024 - Unlocking the Identity: Embracing CIAM 2.0 for a Competitive A...WSO2CON 2024 - Unlocking the Identity: Embracing CIAM 2.0 for a Competitive A...
WSO2CON 2024 - Unlocking the Identity: Embracing CIAM 2.0 for a Competitive A...
WSO2
 
WSO2CON 2024 Slides - Unlocking Value with AI
WSO2CON 2024 Slides - Unlocking Value with AIWSO2CON 2024 Slides - Unlocking Value with AI
WSO2CON 2024 Slides - Unlocking Value with AI
WSO2
 
Platformless Horizons for Digital Adaptability
Platformless Horizons for Digital AdaptabilityPlatformless Horizons for Digital Adaptability
Platformless Horizons for Digital Adaptability
WSO2
 
Quantum Leap in Next-Generation Computing
Quantum Leap in Next-Generation ComputingQuantum Leap in Next-Generation Computing
Quantum Leap in Next-Generation Computing
WSO2
 
WSO2CON 2024 - Elevating the Integration Game to the Cloud
WSO2CON 2024 - Elevating the Integration Game to the CloudWSO2CON 2024 - Elevating the Integration Game to the Cloud
WSO2CON 2024 - Elevating the Integration Game to the Cloud
WSO2
 
WSO2CON 2024 - OSU & WSO2: A Decade Journey in Integration & Innovation
WSO2CON 2024 - OSU & WSO2: A Decade Journey in Integration & InnovationWSO2CON 2024 - OSU & WSO2: A Decade Journey in Integration & Innovation
WSO2CON 2024 - OSU & WSO2: A Decade Journey in Integration & Innovation
WSO2
 
WSO2CON 2024 - Freedom First—Unleashing Developer Potential with Open Source
WSO2CON 2024 - Freedom First—Unleashing Developer Potential with Open SourceWSO2CON 2024 - Freedom First—Unleashing Developer Potential with Open Source
WSO2CON 2024 - Freedom First—Unleashing Developer Potential with Open Source
WSO2
 
WSO2CON 2024 Slides - Open Source to SaaS
WSO2CON 2024 Slides - Open Source to SaaSWSO2CON 2024 Slides - Open Source to SaaS
WSO2CON 2024 Slides - Open Source to SaaS
WSO2
 
WSO2CON 2024 - Does Open Source Still Matter?
WSO2CON 2024 - Does Open Source Still Matter?WSO2CON 2024 - Does Open Source Still Matter?
WSO2CON 2024 - Does Open Source Still Matter?
WSO2
 
WSO2CON 2024 - IoT Needs CIAM: The Importance of Centralized IAM in a Growing...
WSO2CON 2024 - IoT Needs CIAM: The Importance of Centralized IAM in a Growing...WSO2CON 2024 - IoT Needs CIAM: The Importance of Centralized IAM in a Growing...
WSO2CON 2024 - IoT Needs CIAM: The Importance of Centralized IAM in a Growing...
WSO2
 
WSO2CON 2024 - WSO2's Digital Transformation Journey with Choreo: A Platforml...
WSO2CON 2024 - WSO2's Digital Transformation Journey with Choreo: A Platforml...WSO2CON 2024 - WSO2's Digital Transformation Journey with Choreo: A Platforml...
WSO2CON 2024 - WSO2's Digital Transformation Journey with Choreo: A Platforml...
WSO2
 
WSO2CON 2024 - Software Engineering for Digital Businesses
WSO2CON 2024 - Software Engineering for Digital BusinessesWSO2CON 2024 - Software Engineering for Digital Businesses
WSO2CON 2024 - Software Engineering for Digital Businesses
WSO2
 
WSO2CON 2024 - Navigating API Complexity: REST, GraphQL, gRPC, Websocket, Web...
WSO2CON 2024 - Navigating API Complexity: REST, GraphQL, gRPC, Websocket, Web...WSO2CON 2024 - Navigating API Complexity: REST, GraphQL, gRPC, Websocket, Web...
WSO2CON 2024 - Navigating API Complexity: REST, GraphQL, gRPC, Websocket, Web...
WSO2
 
WSO2CON 2024 - Designing Event-Driven Enterprises: Stories of Transformation
WSO2CON 2024 - Designing Event-Driven Enterprises: Stories of TransformationWSO2CON 2024 - Designing Event-Driven Enterprises: Stories of Transformation
WSO2CON 2024 - Designing Event-Driven Enterprises: Stories of Transformation
WSO2
 
WSO2CON 2024 - Not Just Microservices: Rightsize Your Services!
WSO2CON 2024 - Not Just Microservices: Rightsize Your Services!WSO2CON 2024 - Not Just Microservices: Rightsize Your Services!
WSO2CON 2024 - Not Just Microservices: Rightsize Your Services!
WSO2
 

More from WSO2 (20)

Accelerate Enterprise Software Engineering with Platformless
Accelerate Enterprise Software Engineering with PlatformlessAccelerate Enterprise Software Engineering with Platformless
Accelerate Enterprise Software Engineering with Platformless
 
WSO2Con2024 - WSO2's IAM Vision: Identity-Led Digital Transformation
WSO2Con2024 - WSO2's IAM Vision: Identity-Led Digital TransformationWSO2Con2024 - WSO2's IAM Vision: Identity-Led Digital Transformation
WSO2Con2024 - WSO2's IAM Vision: Identity-Led Digital Transformation
 
Driving Innovation: Scania's API Revolution with WSO2
Driving Innovation: Scania's API Revolution with WSO2Driving Innovation: Scania's API Revolution with WSO2
Driving Innovation: Scania's API Revolution with WSO2
 
Less Is More: Utilizing Ballerina to Architect a Cloud Data Platform
Less Is More: Utilizing Ballerina to Architect a Cloud Data PlatformLess Is More: Utilizing Ballerina to Architect a Cloud Data Platform
Less Is More: Utilizing Ballerina to Architect a Cloud Data Platform
 
Modernizing Legacy Systems Using Ballerina
Modernizing Legacy Systems Using BallerinaModernizing Legacy Systems Using Ballerina
Modernizing Legacy Systems Using Ballerina
 
WSO2CON 2024 - Unlocking the Identity: Embracing CIAM 2.0 for a Competitive A...
WSO2CON 2024 - Unlocking the Identity: Embracing CIAM 2.0 for a Competitive A...WSO2CON 2024 - Unlocking the Identity: Embracing CIAM 2.0 for a Competitive A...
WSO2CON 2024 - Unlocking the Identity: Embracing CIAM 2.0 for a Competitive A...
 
WSO2CON 2024 Slides - Unlocking Value with AI
WSO2CON 2024 Slides - Unlocking Value with AIWSO2CON 2024 Slides - Unlocking Value with AI
WSO2CON 2024 Slides - Unlocking Value with AI
 
Platformless Horizons for Digital Adaptability
Platformless Horizons for Digital AdaptabilityPlatformless Horizons for Digital Adaptability
Platformless Horizons for Digital Adaptability
 
Quantum Leap in Next-Generation Computing
Quantum Leap in Next-Generation ComputingQuantum Leap in Next-Generation Computing
Quantum Leap in Next-Generation Computing
 
WSO2CON 2024 - Elevating the Integration Game to the Cloud
WSO2CON 2024 - Elevating the Integration Game to the CloudWSO2CON 2024 - Elevating the Integration Game to the Cloud
WSO2CON 2024 - Elevating the Integration Game to the Cloud
 
WSO2CON 2024 - OSU & WSO2: A Decade Journey in Integration & Innovation
WSO2CON 2024 - OSU & WSO2: A Decade Journey in Integration & InnovationWSO2CON 2024 - OSU & WSO2: A Decade Journey in Integration & Innovation
WSO2CON 2024 - OSU & WSO2: A Decade Journey in Integration & Innovation
 
WSO2CON 2024 - Freedom First—Unleashing Developer Potential with Open Source
WSO2CON 2024 - Freedom First—Unleashing Developer Potential with Open SourceWSO2CON 2024 - Freedom First—Unleashing Developer Potential with Open Source
WSO2CON 2024 - Freedom First—Unleashing Developer Potential with Open Source
 
WSO2CON 2024 Slides - Open Source to SaaS
WSO2CON 2024 Slides - Open Source to SaaSWSO2CON 2024 Slides - Open Source to SaaS
WSO2CON 2024 Slides - Open Source to SaaS
 
WSO2CON 2024 - Does Open Source Still Matter?
WSO2CON 2024 - Does Open Source Still Matter?WSO2CON 2024 - Does Open Source Still Matter?
WSO2CON 2024 - Does Open Source Still Matter?
 
WSO2CON 2024 - IoT Needs CIAM: The Importance of Centralized IAM in a Growing...
WSO2CON 2024 - IoT Needs CIAM: The Importance of Centralized IAM in a Growing...WSO2CON 2024 - IoT Needs CIAM: The Importance of Centralized IAM in a Growing...
WSO2CON 2024 - IoT Needs CIAM: The Importance of Centralized IAM in a Growing...
 
WSO2CON 2024 - WSO2's Digital Transformation Journey with Choreo: A Platforml...
WSO2CON 2024 - WSO2's Digital Transformation Journey with Choreo: A Platforml...WSO2CON 2024 - WSO2's Digital Transformation Journey with Choreo: A Platforml...
WSO2CON 2024 - WSO2's Digital Transformation Journey with Choreo: A Platforml...
 
WSO2CON 2024 - Software Engineering for Digital Businesses
WSO2CON 2024 - Software Engineering for Digital BusinessesWSO2CON 2024 - Software Engineering for Digital Businesses
WSO2CON 2024 - Software Engineering for Digital Businesses
 
WSO2CON 2024 - Navigating API Complexity: REST, GraphQL, gRPC, Websocket, Web...
WSO2CON 2024 - Navigating API Complexity: REST, GraphQL, gRPC, Websocket, Web...WSO2CON 2024 - Navigating API Complexity: REST, GraphQL, gRPC, Websocket, Web...
WSO2CON 2024 - Navigating API Complexity: REST, GraphQL, gRPC, Websocket, Web...
 
WSO2CON 2024 - Designing Event-Driven Enterprises: Stories of Transformation
WSO2CON 2024 - Designing Event-Driven Enterprises: Stories of TransformationWSO2CON 2024 - Designing Event-Driven Enterprises: Stories of Transformation
WSO2CON 2024 - Designing Event-Driven Enterprises: Stories of Transformation
 
WSO2CON 2024 - Not Just Microservices: Rightsize Your Services!
WSO2CON 2024 - Not Just Microservices: Rightsize Your Services!WSO2CON 2024 - Not Just Microservices: Rightsize Your Services!
WSO2CON 2024 - Not Just Microservices: Rightsize Your Services!
 

Recently uploaded

SQL Accounting Software Brochure Malaysia
SQL Accounting Software Brochure MalaysiaSQL Accounting Software Brochure Malaysia
SQL Accounting Software Brochure Malaysia
GohKiangHock
 
fiscal year variant fiscal year variant.
fiscal year variant fiscal year variant.fiscal year variant fiscal year variant.
fiscal year variant fiscal year variant.
AnkitaPandya11
 
Energy consumption of Database Management - Florina Jonuzi
Energy consumption of Database Management - Florina JonuziEnergy consumption of Database Management - Florina Jonuzi
Energy consumption of Database Management - Florina Jonuzi
Green Software Development
 
Oracle 23c New Features For DBAs and Developers.pptx
Oracle 23c New Features For DBAs and Developers.pptxOracle 23c New Features For DBAs and Developers.pptx
Oracle 23c New Features For DBAs and Developers.pptx
Remote DBA Services
 
14 th Edition of International conference on computer vision
14 th Edition of International conference on computer vision14 th Edition of International conference on computer vision
14 th Edition of International conference on computer vision
ShulagnaSarkar2
 
Measures in SQL (SIGMOD 2024, Santiago, Chile)
Measures in SQL (SIGMOD 2024, Santiago, Chile)Measures in SQL (SIGMOD 2024, Santiago, Chile)
Measures in SQL (SIGMOD 2024, Santiago, Chile)
Julian Hyde
 
WWDC 2024 Keynote Review: For CocoaCoders Austin
WWDC 2024 Keynote Review: For CocoaCoders AustinWWDC 2024 Keynote Review: For CocoaCoders Austin
WWDC 2024 Keynote Review: For CocoaCoders Austin
Patrick Weigel
 
在线购买加拿大英属哥伦比亚大学毕业证本科学位证书原版一模一样
在线购买加拿大英属哥伦比亚大学毕业证本科学位证书原版一模一样在线购买加拿大英属哥伦比亚大学毕业证本科学位证书原版一模一样
在线购买加拿大英属哥伦比亚大学毕业证本科学位证书原版一模一样
mz5nrf0n
 
Malibou Pitch Deck For Its €3M Seed Round
Malibou Pitch Deck For Its €3M Seed RoundMalibou Pitch Deck For Its €3M Seed Round
Malibou Pitch Deck For Its €3M Seed Round
sjcobrien
 
All you need to know about Spring Boot and GraalVM
All you need to know about Spring Boot and GraalVMAll you need to know about Spring Boot and GraalVM
All you need to know about Spring Boot and GraalVM
Alina Yurenko
 
GreenCode-A-VSCode-Plugin--Dario-Jurisic
GreenCode-A-VSCode-Plugin--Dario-JurisicGreenCode-A-VSCode-Plugin--Dario-Jurisic
GreenCode-A-VSCode-Plugin--Dario-Jurisic
Green Software Development
 
How Can Hiring A Mobile App Development Company Help Your Business Grow?
How Can Hiring A Mobile App Development Company Help Your Business Grow?How Can Hiring A Mobile App Development Company Help Your Business Grow?
How Can Hiring A Mobile App Development Company Help Your Business Grow?
ToXSL Technologies
 
Project Management: The Role of Project Dashboards.pdf
Project Management: The Role of Project Dashboards.pdfProject Management: The Role of Project Dashboards.pdf
Project Management: The Role of Project Dashboards.pdf
Karya Keeper
 
Mobile App Development Company In Noida | Drona Infotech
Mobile App Development Company In Noida | Drona InfotechMobile App Development Company In Noida | Drona Infotech
Mobile App Development Company In Noida | Drona Infotech
Drona Infotech
 
Fundamentals of Programming and Language Processors
Fundamentals of Programming and Language ProcessorsFundamentals of Programming and Language Processors
Fundamentals of Programming and Language Processors
Rakesh Kumar R
 
Liberarsi dai framework con i Web Component.pptx
Liberarsi dai framework con i Web Component.pptxLiberarsi dai framework con i Web Component.pptx
Liberarsi dai framework con i Web Component.pptx
Massimo Artizzu
 
UI5con 2024 - Boost Your Development Experience with UI5 Tooling Extensions
UI5con 2024 - Boost Your Development Experience with UI5 Tooling ExtensionsUI5con 2024 - Boost Your Development Experience with UI5 Tooling Extensions
UI5con 2024 - Boost Your Development Experience with UI5 Tooling Extensions
Peter Muessig
 
What next after learning python programming basics
What next after learning python programming basicsWhat next after learning python programming basics
What next after learning python programming basics
Rakesh Kumar R
 
Top Benefits of Using Salesforce Healthcare CRM for Patient Management.pdf
Top Benefits of Using Salesforce Healthcare CRM for Patient Management.pdfTop Benefits of Using Salesforce Healthcare CRM for Patient Management.pdf
Top Benefits of Using Salesforce Healthcare CRM for Patient Management.pdf
VALiNTRY360
 
Modelling Up - DDDEurope 2024 - Amsterdam
Modelling Up - DDDEurope 2024 - AmsterdamModelling Up - DDDEurope 2024 - Amsterdam
Modelling Up - DDDEurope 2024 - Amsterdam
Alberto Brandolini
 

Recently uploaded (20)

SQL Accounting Software Brochure Malaysia
SQL Accounting Software Brochure MalaysiaSQL Accounting Software Brochure Malaysia
SQL Accounting Software Brochure Malaysia
 
fiscal year variant fiscal year variant.
fiscal year variant fiscal year variant.fiscal year variant fiscal year variant.
fiscal year variant fiscal year variant.
 
Energy consumption of Database Management - Florina Jonuzi
Energy consumption of Database Management - Florina JonuziEnergy consumption of Database Management - Florina Jonuzi
Energy consumption of Database Management - Florina Jonuzi
 
Oracle 23c New Features For DBAs and Developers.pptx
Oracle 23c New Features For DBAs and Developers.pptxOracle 23c New Features For DBAs and Developers.pptx
Oracle 23c New Features For DBAs and Developers.pptx
 
14 th Edition of International conference on computer vision
14 th Edition of International conference on computer vision14 th Edition of International conference on computer vision
14 th Edition of International conference on computer vision
 
Measures in SQL (SIGMOD 2024, Santiago, Chile)
Measures in SQL (SIGMOD 2024, Santiago, Chile)Measures in SQL (SIGMOD 2024, Santiago, Chile)
Measures in SQL (SIGMOD 2024, Santiago, Chile)
 
WWDC 2024 Keynote Review: For CocoaCoders Austin
WWDC 2024 Keynote Review: For CocoaCoders AustinWWDC 2024 Keynote Review: For CocoaCoders Austin
WWDC 2024 Keynote Review: For CocoaCoders Austin
 
在线购买加拿大英属哥伦比亚大学毕业证本科学位证书原版一模一样
在线购买加拿大英属哥伦比亚大学毕业证本科学位证书原版一模一样在线购买加拿大英属哥伦比亚大学毕业证本科学位证书原版一模一样
在线购买加拿大英属哥伦比亚大学毕业证本科学位证书原版一模一样
 
Malibou Pitch Deck For Its €3M Seed Round
Malibou Pitch Deck For Its €3M Seed RoundMalibou Pitch Deck For Its €3M Seed Round
Malibou Pitch Deck For Its €3M Seed Round
 
All you need to know about Spring Boot and GraalVM
All you need to know about Spring Boot and GraalVMAll you need to know about Spring Boot and GraalVM
All you need to know about Spring Boot and GraalVM
 
GreenCode-A-VSCode-Plugin--Dario-Jurisic
GreenCode-A-VSCode-Plugin--Dario-JurisicGreenCode-A-VSCode-Plugin--Dario-Jurisic
GreenCode-A-VSCode-Plugin--Dario-Jurisic
 
How Can Hiring A Mobile App Development Company Help Your Business Grow?
How Can Hiring A Mobile App Development Company Help Your Business Grow?How Can Hiring A Mobile App Development Company Help Your Business Grow?
How Can Hiring A Mobile App Development Company Help Your Business Grow?
 
Project Management: The Role of Project Dashboards.pdf
Project Management: The Role of Project Dashboards.pdfProject Management: The Role of Project Dashboards.pdf
Project Management: The Role of Project Dashboards.pdf
 
Mobile App Development Company In Noida | Drona Infotech
Mobile App Development Company In Noida | Drona InfotechMobile App Development Company In Noida | Drona Infotech
Mobile App Development Company In Noida | Drona Infotech
 
Fundamentals of Programming and Language Processors
Fundamentals of Programming and Language ProcessorsFundamentals of Programming and Language Processors
Fundamentals of Programming and Language Processors
 
Liberarsi dai framework con i Web Component.pptx
Liberarsi dai framework con i Web Component.pptxLiberarsi dai framework con i Web Component.pptx
Liberarsi dai framework con i Web Component.pptx
 
UI5con 2024 - Boost Your Development Experience with UI5 Tooling Extensions
UI5con 2024 - Boost Your Development Experience with UI5 Tooling ExtensionsUI5con 2024 - Boost Your Development Experience with UI5 Tooling Extensions
UI5con 2024 - Boost Your Development Experience with UI5 Tooling Extensions
 
What next after learning python programming basics
What next after learning python programming basicsWhat next after learning python programming basics
What next after learning python programming basics
 
Top Benefits of Using Salesforce Healthcare CRM for Patient Management.pdf
Top Benefits of Using Salesforce Healthcare CRM for Patient Management.pdfTop Benefits of Using Salesforce Healthcare CRM for Patient Management.pdf
Top Benefits of Using Salesforce Healthcare CRM for Patient Management.pdf
 
Modelling Up - DDDEurope 2024 - Amsterdam
Modelling Up - DDDEurope 2024 - AmsterdamModelling Up - DDDEurope 2024 - Amsterdam
Modelling Up - DDDEurope 2024 - Amsterdam
 

architecting-ai-in-the-enterprise-apis-and-applications.pdf

  • 1. Architecting AI in the Enterprise: APIs and Applications Malith Jayasinghe VP of Research and AI, WSO2
  • 2. Enterprise AI: Growth Forecast 2022−2030 ● Global enterprise AI market was valued at $136.55 billion in 2022 (Forbes, 2022) ● Expected to grow at a CAGR of 36.6% from 2024 to 2030, reaching $1,811.8 billion by 2030 (Grand View Research, 2024) ● By 2026, over 80% of enterprises are anticipated to have used generative AI APIs/model and/or have deployed generative AI application in production (Gartner, 2023) ● More companies are experimenting with generative AI; however, the percentage of applications entering production is low (5% in 2023) , though it is increasing (Gartner, 2023) 2
  • 3. Value Proposition in AI Development ● Understanding AI's value is key to unlocking its potential. Strategic integration of AI can enhance efficiency, improve user experience, provide competitive advantages, and prepare organizations for the future 3
  • 4. AI Development Challenges ● Objective: Provide insights into potential challenges and strategies for addressing them (Specific Focus: generative AI applications) ● Challenges: As you develop your AI application and attempt to move it into production, you will need to address many challenges ● UI/UX Considerations ⦿ Initiate design early ⦿ Consider design aspects deeply 4
  • 5. AI Development Challenges 5 Our Recent Gen AI Features
  • 6. Selecting the Right Model ● Model selection criteria ⦿ Accuracy (Accuracy vs. speed?) ⦿ Cost (Accuracy vs. cost?) ⦿ Speed (Accuracy vs Speed?) ⦿ Risks ⦿ Other Considerations: Self-hosted model or Public cloud hosted, Model availability in specific regions ● Dependency on use case ● Trade-offs in model selection ● Given AI feature can use multiple models 6 Source: https://informationisbeautiful.net/
  • 7. Identifying Data Sources ● Model improvements (build a RAG, Fine tune, pre-train) ● Identify the available data sources (e.g., internal R&D, product documentation) ● Data cleaning and preprocessing ⦿ Conduct data cleaning ⦿ Data leakage: LLM accidentally reveals sensitive information (PIIs, proprietary algorithms) ● Do I need to collect new data? 7
  • 8. AI Interaction with APIs ● What are the available APIs? ● APIs can differ in type and have different representations ⦿ RESTful APIs ⦾ OpenAPI specification ⦿ GraphQL APIs ⦾ GraphQL Schema ● Well-documented APIs perform better when integrated with AI ● AI can bridge the gaps in missing documentation 8
  • 9. Prompt Engineering ● Prompting engineering ⦿ Prompt engineering is guiding generative AI to produce desired outputs ⦿ A prompt is a natural language instruction describing how generative AI should perform (acts as the interface between human intent and machine output) ● Prompting ⦿ Zero-shot prompting, Few-shot prompting ⦿ Chain-of-Thought (COT), Tree of Thoughts (TOT) ⦿ ReAct (Reason and Act) 9 https://www.promptingguide.ai/
  • 10. Prompt Engineering: ReAct (Reason and Act) ● LLM generation provides information, guidance, or suggestions. Their output doesn't (inherently) drive actions ● ReAct Agents can perform actions (e.g., calling APIs) ● ReAct Agents with the assistance of the model, can execute natural language commands using a collection of tools registered with them ● Tools can retrieve data or perform tasks, and they come in different forms, such as functions, API, etc. 10
  • 11. Use Case: Train Booking System 11 Users Chat Ul (Web site) Train Booking Service (Agent Service) Existing Business APIs Train API Email API Payment API invokes Swagger Specs (OpenAPI) Initialization
  • 12. Evaluating Accuracy ● Create a dataset for evaluation purposes ● Select a metric (use existing one or develop new metrics if necessary) ● Automate the evaluation of accuracy ● Re-evaluate after any changes to confirm that accuracy has not diminished ● Even minor changes to prompts can significantly affect accuracy 12
  • 13. Collecting User Feedback + Improving the Accuracy ● Collect data: Gather feedback on AI outputs, user interactions, and engagement metrics ● Compliance: Ensure user consent and follow privacy laws. Anonymize sensitive data ● Improve models: Update vector DBs, fine-tune models, improve prompts 13
  • 15. Enhancing User Experience for Tasks with Long Processing Times ● Complex Tasks: AI systems execute complex tasks that require multiple reasoning iterations and involve invoking multiple APIs ● Optimizations: Techniques such as caching are used, but latency can still be high ● User Satisfaction: Prolonged round-trip times can result in user dissatisfaction ● Strategies to Improve User Experience: ⦿ Incremental Display: Progressively display partial results as AI systems process different APIs, thereby reducing wait times for final outcomes 15
  • 16. CLIENT AGENT API 1 LLM 1 Enhancing User Experience for Tasks with Long Processing Times 16
  • 17. Closing Remarks ● Purposeful Beginnings: We start with a clear definition of the problem and a keen understanding of the application's value, guiding our development journey from the outset ● User-Centric Design: Prioritizing UI/UX considerations ensures that our applications are accessible, intuitive, and meet user expectations ● Technical Execution ⦿ Model Selection/Model Improvements ⦿ Dataset Utilization/API Integration ⦿ Prompt Engineering ⦿ Accuracy evaluation/Continuous feedback ⦿ Scalability and Performance ● Forward Outlook: The development of generative AI applications is a dynamic, ongoing process requiring constant learning and adjustment to meet evolving technological trends and user demands 17