Isabelle's speech as part of "AI workshop for Enterprise: a real approach to AI driven projects" with Profesia and Linkalab during WSO2 oxygenate Italy 2025 in Milan

1. AI : Technical Introduction
Isabelle MAUNY
Field CTO
ITALIA

2. A bit of vocabulary first…

3. Core building blocks: LLMs / GPTs…
• LLMs (Large Language Models) are trained on a very large
amount of data.
• GPTs (Generative Pre-trained Transformer) are a special type
of LLMs
• Generative: create new text
• Pre-trained
• Transformer: using a transformer architecture.
At a high level: GPTs are good at predicting what the
next token in a response will be based on context.
• GenAI (Generative AI) : AI technology focused on generating
content (text, images, audio, video, etc.)
3

4. …and tokens!
4
https://platform.openai.com/tokenizer

5. Let’s walk an example…

6. Planning a trip to Japan…
6

7. 7
An application that uses Gen AI models to generate
content like text and images

8. 8

9. 9

10. Add more context to the prompt !
• This is a User Experience issue: if you are vacation.com, you can’t ask this information from every
customer, especially since you already have a lot of context from past trips.
• You could use CAG ( Cache Augmented Generation), passing full documents as extended context, but
• Context of LLMs is currently limited by number of input tokens (50k-100k)
• Beware of $$$ !
• CAG is however the right choice in some cases: static data, low latency requirements.
10

11. 11
An application that uses Retrieval Augmented Generation (RAG)

12. Adding more context through RAG
• Can use many sources of data
• Documentation
• Databases
• Customer information
• Emails
• Data must be careful prepared, indexed, chunked =>
Vector Databases
12
https://www.mongodb.com/products/platform/atlas-vector-search

13. 13

14. 14
Ok! Let’s Go !

15. 15

16. So far, we are in GenAI mode,
not in Agent mode.
16

17. We need tools!
17

18. 18
An application that uses an Agent

19. Tools are just functions!
19

20. How do we get an LLM to talk to any tool, in a uniform way ?
20

21. The MCP Trivia!
• Do we need MCPs to have tools ? No.
• Do we need APIs to write MCPs ? No.
• Do we need tools to build enterprise agents ? Yes.
• Do we need to secure MCPs ? Yes. From Day 1.
21

22. WSO2 AI Platform
Isabelle MAUNY
Field CTO
ITALIA

23. AI is fundamentally changing software development
Developers productivity is supercharged
through AI for code across the software
development lifecycle.
Building AI-native applications is
changing code for AI through emerging
programming abstractions and building
blocks
Blending language and code with
natural programming
23

24. 24
Only ~26% of companies
experimenting with AI have scaled
them to production 1
Your Enterprise Architecture
AI
Code for AI
1 IDC Worldwide AI & Generative AI Spending Guide, September 2024, IDC, Doc #US52520724

25. Building applications with generative AI
and AI Agents requires tools and
platforms that natively support:
✓ Generative AI models
✓ Knowledge bases
✓ Agents
✓ Tools
25
Code for AI

26. 26
Building AI Agents: LLMs Governance
Code for AI

27. 27
AI Gateway

28. 28
Guardrail Sample Use Case
Semantic Prompt Guard Stops prompts like “Write my homework” in a student assistant app.
Regex based PII Masking Masks credit card numbers in user prompts for a payment chatbot.
Word Count and Sentence Count Limits AI replies to 50 words in a quick-answer mobile app
JSON Schema Validation Validates API responses for correct format in an e-commerce platform
Regex Validation Verifies user-entered email addresses in a registration form
URL Validation Ensures links in AI responses resolve via DNS for a news aggregator app.
Content Length Caps user inputs at 500 characters in a chat AI to prevent spam
Grounded AI Hallucination Prevents AI from making up facts in product descriptions.
Content Safety Filters hate speech in comments generated by an AI writing assistant.
PII Detection and Masking Detects and hides phone numbers in support chatbot inputs.
Jailbreak detection Stops prompts like “Ignore all rules” in customer service bots.

29. 29
Guardrail Policies
Third-Party Integrations
Integration Guardrail
Build-In Guardrails
Content Safety
AWS Bedrock
Other
Bring you own
Content Safety
PII Detection
Other
Guardrails AI
Advanced Guardrails
Grounded AI Hallucination
Detecting Jailbreak

30. AI Gateway in action
DEMO
AI Gateway

31. 31
Building AI Agents: Building Tools
Code for AI

32. MCP Servers
32
• Create, secure and host MCP Servers
• (Preview soon!) Create MCP server from
APIs
• Secure access to MCP servers
• Host MCP Servers on Choreo
• WSO2 MCP Servers (as of June 1st)
• Asgardeo / IS
• Choreo
• More to come!
• Consuming MCP servers in BI is also on the
roadmap

33. Ballerina Integrator and Agentic AI Demo
33
DEMO

34. 34
Building AI Agents: Agents Identity
Code for AI

35. Zero trust in Agentic AI
35
Traditional Security vs. AI Agent Reality:
● Traditional: "Trust but verify" - humans in the loop
● AI Agents: Autonomous decisions, rapid execution,
broad system access
Image source: AI generated

36. Zero-Trust Principles for AI Agents
● Verify Identity
⦿ Every AI agent must authenticate and maintain identity
⦿ Continuous validation of agent legitimacy
● Just in time / Just enough access:
⦿ Agents receive minimum permissions for their specific tasks
⦿ Dynamic permission adjustment based on context
● Assume Breach
⦿ Design systems expecting agent compromise
⦿ Limit blast radius of potential security incidents
● Continuous Monitoring
⦿ Real-time audit of agent actions and decisions
⦿ Behavioral analysis for anomaly detection
36

37. Gardeo Leisure
Digital Services
A platform that helps customers plan
and book their ideal trip:
Hotel Discovery
Availability Check
Hotel Booking
Concierge Allocation
and More
Source code: in Github
Image source: AI generated

38. 38

39. 39
Application Architecture
Source code: in Github

40. 40

41. 45
Building AI Agents
Code for AI

42. Boosting productivity
AI for code

43. Boosting developer productivity across SDLC
47
AI for code
Coding &
Development
Production &
Delivery
Develop Deploy Manage Reuse
Design Observe Govern
Delivering digital applications from coding and
development to production and delivery
AI boosts productivity at every stage

44. Boosting developer productivity
48
AI for code
Copilots and AI-powered insights
- In all our products
- Across all stages of SDLC
- Directly in the user experience:
IDE or console
Providing
- Natural language interfaces
- Automating and enhancing
cumbersome tasks
- Generating code, docs, tests, and
more

45. Natural programming, continued:
keep requirements, docs and code in sync
49
AI for code
Detect and fix drift between
requirements, docs and
code.

46. 50
AI Your Enterprise Architecture
Your Enterprise Architecture
AI
From AI prototyping To intelligent modernization

47. 51
Visit: https://wso2.ai