Definition
Agentic AI systemsare
designed to pursue
complex goals with a high
level of autonomy and
predictability.
They are productivity
enablers who can
effectively incorporate
humans in the loop via the
use of multi-modality.
Autonomy: takes goal-directed actions with
minimal human oversight
Reasoning: contextual decision-making,
judgment calls & tradeoffs
Adaptable Planning: dynamically adjusts plans
based on changing conditions to complete
processes efficiently
Context Understanding: comprehends and
follows natural language and other modalities
Key Characteristics
Action Enabled: empowered to take action via
access to web services delivering skills
Collaborative Interaction: agents can collaborate
with other agents and humans, leveraging
collective intelligence to achieve shared goals.
Why agents?
Allow themto easily
map AI into existing
human orgs and
workflows
Provide abstractions
that make it easier for
developers to
decompose and
reason about
intelligent apps
Allow multiple models
or compounded
systems to get better
performance out of
AI models
5.
Sends welcome
email
HR Manager
ITManager
Training Manager
New Hire
Manually sets up
user accounts
and provisions
access
Gets new hire
forms
Has questions
Answers
questions
Complete
s
paperwor
k
Confirm
computer
preference
Confirmed
Initiates IT
process
Missing
applications
submits IT
ticket.
Contacts
procureme
nt to
obtain
license.
Initiates
training
process
Schedule
conflict
Reviews
training
schedule and
reschedules
Sends
feedback
survey
Waits for more
data before
analyzing and
optimizing the
entire process.
Completes the
training still
has questions
TODAY
Existing solutions can only automate very specific tasks that have clear inputs and outputs
Initial Interaction Document Verification
Access &
Technology Provisioning
Training
Onboarding Feedback
Loop
Survey is
complete
d
RPA automates the verification of
completion and accuracy
with predefined rules.
RPA automates assigning predefined
training modules but cannot adjust
based on user feedback
6.
• Provides apersonalized
onboarding journey based
on interaction
IT Agent
Training Agent
New Hire
TOMORROW
With AI Agents, these steps can be automated for the first time, while keeping human in the
loop.
HR Agent
• Assesses documents and
learns from interactions
• Analyzes role, experience, and
learning preferences to recommend
training
• Gathers real-time feedback
• Identifies patterns makes
informed decisions
Provides adaptive
training
• Sets up user accounts, adapts to
troubleshoot unexpected issues, and
learns from errors.
Fills out forms and has
no questions
The new hire has
questions during
the training.
Orchestrates the additional
processes
Human in the Loop, Supervisor and Approver
Initial Interaction Document Verification
Access &
Technology Provisioning
Training
Onboarding Feedback
Loop
7.
Evolution of LLM-basedSolutions
document summary
LLM
prompt
LLM
answer
question
Search
Agent
data
query
prompt
+
data
docs
conversation
Search
Retrieva
l
Agent
data
query
User Proxy
Agent
Coordinator
Agent
Analytics
Agent
data
Codin
g
Agent
docker
databases
docs
Memory
history of work
SL
M
S
LM
LLM
LL
M
Web
Service
Service
Agent
output
input
LLM
No Agent
Very narrow one shot task
Ex: log to JSON
Single Agent
Very clearly scoped iterative task
Ex: providing an answer with supporting
evidence to a complex question
Multi-agent Systems
Wide scope complex use case requiring diverse skills
Ex: Propose 2 Instagram marketing campaigns including
assets that would leverage the top 2 recent trends in our
past quarter US Sales to boost our mailing list user base and
predict the impact of each campaign
VALUE
Retrieval Augmented GenerationAgent
Translates questions into a research problem with human in the loop to produce high quality answers
to complex questions within the scope of its domain
Intelligent RAG Agent
Plan
Query
Observe
Update plan
Compile
answer
Knowledge
Graph
Search Tools
Discussion
Final
answer
Questio
n
10.
Session and memorymanagement
Dynamic context look-up
Planning &
tracking
Toolsets & coding interface
Human interaction
Coordinator
Multi-turn reasoning and action (ReAct)
Code Generation Agent
Generates code based on natural language requirements, leveraging existing code base, templates,
guidelines, libraries to match policies and best practices while interacting with humans to clarify,
validate and deliver functionality as intended.
Code
Executor
Constraints
Existing Codebase
Coding Guidelines
(docs)
Internal SDKs
Dev Task
(spec / bug / feature)
Code w/ Tests, Doc,
DevOps code, etc
11.
MS #3
MS #2
MS#1
Multi-Agent System
A complex problem is decomposed into smaller, manageable parts, each addressed by specialized
agents, effectively a micro-service (MS). These agents work together in a coordinated manner within
a workflow to efficiently solve the overall problem.
conversation
Search
Retrieval
Agent
data
query
User Proxy
Agent
Coordinator Agent
Coding
Agent
docker
Coding Guidelines
Memory
history of work
SLM
LLM
LLM
Critical Design Elements
Adaptive planning within scope of
existing tightly scoped skills (agents)
Handles ambiguity by discussing and
refining requirements with human
Memory to handle complex long
running execution of a plan
Effective inter agent communications
Test, monitor, release & maintain each
agent independently to quickly
handle quality & safety issues
12.
Multi-Domain Agents System
Multipledomain-specific agents are orchestrated by an Agent Runner to scale across multiple
domains while appearing as a single agent to users.
Agent pool
Agent Runner
Active Agent
Agent 1
Revaluate
agent
assignment
Run
Transfer
Run
+ Back-off
Role/goals
+ skills
Agent 2
Run
+ Back-off
Role/goals
+ skills
Agent n
Run
+ Back-off
Role/goals
+ skills
Shared context memory
Critical Design Elements
Agents capability descriptors
Scalable Agent Runner able
to manage 10s to 100s of
agents
Ability to manage domain
switching with proper
memory management
Avoid single interceptor
problem as individual agents
maintain direct
communication with user and
can hand off when needed
13.
4 primary considerations
K
Providing
agentswith the
right context
Knowledge Evaluation
Ensuring they
complete their
tasks correctly
Actions
Giving them the
tools to
complete their
tasks
Security
Ensuring they
only have
access to what
they should
Successfully developing AI Agents
requires
14.
So far, buildingagents from scratch has been quite
difficult.
Tool Integration
Creating a cohesive system through complex integration of
various tools and APIs that have different interfaces, data
formats, and requirements.
Interoperability
Achieving interoperability between different tools and platforms
to ensure that data can be shared and understood across
different systems.
Scalability
Handling increased data volumes, more complex computations,
and higher user loads without degrading performance.
Real-time Processing
Ensuring tools can handle real-time requirements without
significant latency.
Maintenance
Making labor–intensive updates to integrated tools for
compatibility with new versions and prevention of obsolescence
and security vulnerabilities.
Flexibility
Modifying or customizing existing tools or developing new ones to
meet unique requirements.
Error Handling
Ensuring errors are handled gracefully and continue functioning
despite tool failures or unexpected inputs is critical for reliability.
Security
Implementing robust encryption, access controls, and compliance
with privacy regulations to protect sensitive data.
15.
Organizations need platformsthat enable rapid
development of performant, secure AI Agents
Current Frameworks
Security and data privacy risks
What’s Needed
Secure, responsible AI that protects
sensitive information and behaves
compliantly
Lack of integrated tools, insecure data
grounding, challenging orchestration
Ineffective deployment of AI across websites,
applications, and production environments
Restrictive pre-defined models that are
challenging to customize
Flexible models that enable processing
and integration of information from
multiple modalities or types of data
Connected complex workflow automation
grounded by seamless connection to
enterprise data
Tools and APIs that seamlessly integrate
across enterprise applications
16.
Visual Studio
Copilot Studio
GitHub
AzureAI
Foundry SDK
Model Catalog
Open-source models
Foundational models Task models Industry models
Azure AI
Content
Safety
Azure
AI Search
Azure AI
Agent Service
Azure
OpenAI Service
Observability
Customization
Evaluations Governance Monitoring
Azure Machine
Learning
Azure AI Foundry
17.
Azure AI AgentService
Public Preview
Empower developers to securely build, deploy, and scale AI agents with
ease
Flexible Model
Selection
Extensive Data
Connections
Enterprise Readiness
Rapid Development
and Automation
AI.Azure.com
18.
The full package
Built-inenterprise
readiness
BYO-file storage
(coming soon)
BYO-search index
Azure AI Agent Service
OBO Authorization Support
Enhanced Observability
Extensive Ecosystem of
Tools Knowledge
Microsoft Fabric*
SharePoint*
Grounding with Bing Search
Azure AI Search
Your own licensed data
Files (local or Azure Blob)
File Search
Code Interpreter
Action
s
Azure Logic Apps*
OpenAPI 3.0 Specified Tools
Azure Functions
Model
Catalog
Azure OpenAI Service
(GPT-4o, GPT-4o mini)
Models-as-a-Service
Llama 3.1-405B-Instruct
Mistral Large
Cohere-Command-R-Plus
19.
Microsoft 365 AgentsSDK
Microsoft 365 Agents SDK
Your agent, your way
AI services of your choice
Channels of your choice
Low and pro-code
20.
Microsoft 365 AgentsSDK
Semantic
Kernel
Azure AI Foundry
Other AI Services
Orchestration
M365
Agents
SDK
+
Customer
Code
M365
Agents
SDK
+
Customer
Code
Copilot
Studio
Copilot Studio
AI services
Dataverse &
Plugins/Actions
Orchestration
Your own custom engine agent
User asks a question
and sends it to the
client
User sees a
response in the
client
21.
Semantic Kernel
Semantic Kernelis lightweight,
open-source, production-
ready, orchestration
middleware that lets you easily
add AI to your apps.
Python
Java .NET
22.
AutoGen
• AutoGen isa framework that enables
development of LLM applications using
multiple agents that can converse with
each other to solve tasks.
• AutoGen is powered by collaborative
research studies from Microsoft, Penn
State University, and University of
Washington.
• AutoGen simplifies the orchestration,
automation, and optimization of a complex
LLM workflow.
23.
•Customizable and conversableagents:
AutoGen uses a generic design of
agents that can leverage LLMs, human
inputs, tools, or a combination of them
•Conversation programming:
• Defining a set of conversable agents
with specific capabilities and roles
• Programming the interaction
behavior between agents via
conversation centric computation
and control.
AutoGen | AutoGen
(microsoft.github.io)
AutoGen Concepts
Deploy agents with
AzureAI Agent Service
Orchestrate them together with
AutoGen and Semantic Kernel
Single-agent Multi-agent
1 2
State-of-the-art
research SDK
Production-ready
and stable SDK
Managed agent
micro-services
Ideation Production
#5 Currently, most AI and automation systems are built to handle very specific tasks with clearly defined inputs and outputs. For example, a process like HR onboarding might be automated, but it's typically limited to simple rule-based tasks, such as sending emails or verifying documents. These systems often fail to adapt when things go off-script or if unexpected situations arise. Here, we see an example of how RPA automates predefined steps, but the flexibility to adjust to evolving needs is limited.
#6 Looking ahead, AI agents offer a transformative approach. Unlike traditional systems, AI agents can manage more complex workflows and adapt in real-time. Notice here how tasks like document verification, IT provisioning, and even training can be handled by AI agents, but the critical distinction is that humans are still involved in key decision-making moments. This hybrid model ensures that while automation is improving efficiency, humans retain oversight and control
Initial Interaction:
Engages with new hires through a virtual assistant, providing a personalized welcome and gathering initial data to tailor the onboarding journey. The AI agent makes decisions based on the information provided by the new hire to customize the onboarding experience immediately.
Document Verification:
Facilitates the submission and verification of documents by assessing completeness and accuracy. It learns from each interaction, recognizing common errors or missing items, and adjusting its feedback to help new hires correct issues quickly.
Training Path Selection:
Analyzes the new hire's role, experience, and learning preferences to recommend a suitable training path. The AI continuously learns from feedback on training effectiveness and adjusts the modules to better suit the new hire's needs, ensuring optimal learning outcomes.
Feedback and Adjustment:
Gathers real-time feedback from new hires regarding their onboarding experience. It processes this feedback to identify patterns and trends, enabling it to make informed decisions on necessary strategy adjustments to improve the onboarding process for future hires.
Ongoing Support:
Continues to support the new hire beyond the initial onboarding phase, answering questions and providing resources as needed. The AI agent utilizes ongoing interactions to refine its support strategies, ensuring continuous improvement in employee engagement and satisfaction.
#7 As AI systems evolve, we're moving from narrow, single-task solutions to more expansive, iterative systems. In the early days, LLMs were used for very specific tasks, like summarizing documents. Now, we're seeing a more sophisticated architecture, where agents can handle more complex tasks by integrating multiple data sources and interacting with humans to refine outcomes. From single-agent solutions to multi-agent systems, we’re unlocking the potential for AI to solve larger, more complicated problems.
Key Technical Enablers
Human level reasoning capabilities
Enables critical thinking, re-planning to establish resolution strategies and incorporate human in the loop.
Larger context windows
Enables reasoning over execution history to re-plan + integrate feedback from multiple agents.
SLM for specialized agents
Enables rapid execution to keep human engaged in the loop and enables cost effective resolution.
Multi-modality support
Widens scope of use-cases and eases access to Agentic AI Systems.
#8 Agentic AI systems follow certain design patterns that ensure they operate effectively and efficiently. Common patterns include Retrieval-Augmented Generation (RAG) agents, which improve the quality of answers by using external knowledge, and multi-agent systems, where specialized agents work together to tackle complex problems. These design patterns are essential in building scalable and robust AI agents that can adapt to a variety of tasks and environments.
#9 Retrieval-Augmented Generation, or RAG, agents are a powerful tool for tackling complex questions. These agents take a question, break it down into smaller parts, and engage in a dynamic research process to gather relevant data, refine their understanding, and deliver high-quality, well-supported answers. This process involves multiple iterations and human-in-the-loop feedback to ensure the final answer is accurate and comprehensive.
#10 Code generation agents represent another breakthrough in Agentic AI. These agents can take natural language requirements and generate code, leveraging existing templates, libraries, and best practices. They engage with human developers for clarification and validation, ensuring that the code produced is both functional and aligned with project needs. This hybrid approach increases productivity and accelerates development cycles.
#11 Multi-agent systems break down complex tasks into smaller, more manageable components, each handled by specialized agents. For example, in a multi-agent system for software development, a coordinator agent manages the overall workflow, while retrieval agents and coding agents focus on specific tasks. This distributed approach allows each agent to focus on what it does best, while coordinating efforts to efficiently solve the larger problem. Memory and inter-agent communication are crucial for keeping the system running smoothly.
#12 Multi-domain agent systems represent a significant leap in how AI can scale and handle diverse tasks. In this model, domain-specific agents are orchestrated by an 'Agent Runner,' which allows them to work across multiple domains while presenting themselves as a single agent to the user. The design is built to handle scalability, ensuring the ability to manage hundreds of agents. A key feature here is the shared context memory, which allows agents to transfer knowledge and coordinate with each other seamlessly. This system is efficient, highly adaptable, and ensures that tasks are handled without bottlenecks, even as the scope of responsibility increases.
#13 When developing AI agents, there are four critical considerations that must be addressed for success: Knowledge, Actions, Security, and Evaluation. First, providing agents with the right context or knowledge ensures they understand the environment they operate in. Second, equipping them with the right tools and capabilities enables them to perform the necessary tasks. Security is paramount—agents must have access only to the resources they need and be protected against misuse. Lastly, evaluation ensures the tasks are completed correctly and allows for performance optimization. Addressing these areas forms the foundation for effective AI agents.
#14 Building AI agents from scratch has traditionally been a complex and challenging task. There are several factors involved, such as integrating diverse tools and APIs that may have different interfaces and data formats. Interoperability is another hurdle, as agents need to work seamlessly across various platforms. Scalability is critical to ensure agents can handle growing data and workloads. Additionally, real-time processing, maintenance, and error handling add to the complexity. These challenges have made it difficult to rapidly deploy and scale AI agents that can be both effective and secure.
#15 Organizations today are facing challenges in developing AI agents due to the limitations of current frameworks. These include a lack of integrated tools, security risks, and challenges in orchestrating complex workflows. What’s truly needed is a platform that connects complex workflow automation to enterprise data and seamlessly integrates with existing tools and applications. Flexible models must enable the processing of multiple types of data and modalities, and security must be built in to ensure compliance and data protection. The need is clear: organizations require platforms that make it easier to build and deploy scalable, secure AI agents rapidly.
#16 OPTION 1: Azure AI Foundry is a trusted, integrated platform designed for Developers and IT Administrators to design, customize, and manage AI applications and agents. It offers a rich set of AI capabilities and tools through a simple portal, unified SDK, and APIs to accelerate the path to production. What sets Azure AI Foundry apart is its accessibility through the world's most loved developer tools: GitHub, Visual Studio, and Copilot Studio. This integration enables developers to work within their preferred environments, enhancing productivity and collaboration. Key tools within Azure AI Foundry include Azure OpenAI Service, Azure AI Search, Azure AI Agent Service, and Azure AI Content Safety. These tools, combined with observability features like evaluations, customization, governance and monitoring, facilitate secure data integration, model customization, app orchestration, and experimentation with built-in trustworthy AI tools and principles. By unifying data, models, and operations into a single platform, Azure AI Foundry enables enterprises, start-ups and software developer companies to fully harness the potential of AI, accelerating innovation and optimizing application quality in production.
OPTION 2: Azure AI Foundry is a trusted, integrated platform designed for Developers and IT Administrators to design, customize, and manage AI applications and agents. It offers a rich set of AI capabilities and tools through a simple portal, unified SDK, and APIs to accelerate the path to production. What sets Azure AI Foundry apart is its accessibility through the world's most loved developer tools: GitHub, Visual Studio, and Copilot Studio. This integration enables developers to work within their preferred environments, enhancing productivity and collaboration. Azure AI Foundry facilitates secure data integration, model customization, app orchestration, evaluation, and experimentation with trustworthy AI tools and principles. It also provides enterprise-grade governance and management, to help ensure AI operations are secure and compliant. By unifying data, models, and operations into a single platform, Azure AI Foundry enables enterprises, start-ups, and software development companies to fully harness the potential of AI, accelerating innovation and optimizing application quality in production.
#17 Azure AI Agent Service is a flexible, use-case-agnostic platform for building, deploying, and managing AI agents. These agents can operate autonomously with human oversight, leveraging contextual data to perform tasks and achieve specified goals.
The service integrates cutting-edge models and tools from Microsoft, OpenAI, and partners like Meta, Mistral, and Cohere, providing an unparalleled platform for AI-driven automation. Through the Azure AI Foundry SDK and an intuitive Azure AI Foundry portal experience, developers can quickly create powerful agents while benefiting from Azure’s enterprise-grade security and performance guarantees.
#18 In our experience talking to hundreds of organizations, we have learned that developing secure, reliable agents rapidly requires four primary ingredients:
Rapidly develop and automate processes: Agents need to seamlessly integrate with the right tools, systems and APIs to perform deterministic or non-deterministic actions.
Integrate with extensive memory and knowledge connectors: Agents need to manage conversation state and connect with internal and external knowledge sources to have the right context to complete a process.
Flexible model choice: Agents built with the appropriate model for its task can enable better integration of information from multiple data types, yield better results for task-specific scenarios, and improve cost efficiencies in scaled agent deployments.
Built-in enterprise readiness: Agents need to be able to support an organization's unique data privacy and compliance needs, scale with an organization's needs, and complete tasks reliably and with high quality.
#21 Semantic Kernel abstracts a way of all these underlying SDKs from different AI providers from Microsoft and other providers that are fully open source, production ready and ready to serve you in Python, Java or .NET
Semantic Kernel is a lightweight, open-source development kit that lets you easily build AI agents and integrate the latest AI models into your C#, Python, or Java codebase. It serves as an efficient middleware that enables rapid delivery of enterprise-grade solutions.
https://learn.microsoft.com/en-us/semantic-kernel/overview/
#22 AutoGen is a framework designed to enable the development of applications using multiple agents that can collaborate to solve tasks. These agents are customizable and conversable, which allows them to interact with each other and external systems. Powered by collaborative research from leading universities, AutoGen simplifies the orchestration of multi-agent systems and helps streamline the process of automating tasks. It enables the creation of complex workflows by coordinating agents that specialize in different tasks, from gathering data to code generation and analysis. This level of customization and collaboration across agents is what makes AutoGen a powerful tool for complex applications.
#23 AutoGen’s primary strength lies in its ability to create customizable, conversable agents. These agents can leverage models, human inputs, and external tools, or a combination of these to work together to solve tasks. The concept of conversation programming allows developers to define a set of agents with specific roles and capabilities. These agents interact with each other via conversation-centric computation, enabling complex workflows to be executed seamlessly. Whether you are building a team of specialized agents or designing a multi-agent conversation, AutoGen allows you to create dynamic, flexible interactions between agents that can be tailored to specific business needs.
#24 Here we see an example of how to construct agents in AutoGen using Python code. We define the roles of the agents, such as a UserProxyAgent, Coder, and Product Manager, and provide them with the necessary instructions to fulfill their tasks. The GroupChatManager coordinates the conversation between the agents, ensuring smooth interaction. This code snippet showcases how you can programmatically create and manage agents in AutoGen, orchestrating them to collaborate, generate responses, and produce desired outcomes such as code snippets or answers to complex queries.
#25 When building a new multi-agent solution, you should always start with Azure AI to get the most reliable, scalable, and secure agents. You can then orchestrate them together using Microsoft’s two multi-agent orchestration libraries: AutoGen and Semantic Kernel.
AutoGen is a library created by Microsoft Research; written in python. It is constantly evolving to find the best collaboration patterns for agents (and humans) to work together.
Semantic Kernel is Microsoft enterprise AI SDK for Python, .NET, and Java. Features that show production value from AutoGen move into Semantic Kernel for customers looking for production support and non-breaking changes.
#26 This slide illustrates how multi-agent orchestration works in AutoGen. Here, Assistant Agents are set up to handle different tasks like searching, writing, and saving, while also leveraging Azure AI Agent Service for integration with external tools such as Bing Search and Code Interpreters. Through this orchestration, multiple agents work together in a group chat environment to solve a problem, each with their specific role and focus area. This setup highlights the flexibility of AutoGen to combine different agents with distinct functionalities into one cohesive workflow, ensuring the system is scalable and efficient.
#28 In this slide, we see the use of Semantic Kernel for multi-agent orchestration. The ChatCompletionAgents work together with a Search Agent and Write Agent, each playing a critical role in the workflow. These agents work within a group chat to share information and collaborate on a task. The Search Agent retrieves information from external sources like Bing, while the Write Agent finalizes the output. This flow exemplifies the power of combining multiple specialized agents, each optimized for a particular task, within a seamless system.
#29 In this sample, we see a web search agent interacting with a site analyzer to extract relevant data. The agent is tasked with parsing HTML, analyzing web pages, and retrieving textual information like weather forecasts. The extracted text undergoes further processing, including image analysis and content filtering, before being passed through the memory module for final output.
This agent's design exemplifies how multi-agent systems can work together seamlessly. The Web Search agent handles the data extraction, while the Verifier agent checks the results against internal documents. The integration of Azure AI services, including content understanding and document search, provides a cohesive and robust workflow.
#30 Moving to a call center example, this system demonstrates how agents can collaborate dynamically to solve customer queries. The system starts with a query parsing agent, which interprets the user's request. The Planner agent then breaks down the task into specific sub-tasks, while agents like Doc and Web Search pull relevant documentation and online resources to assist the customer.
This dynamic, real-time orchestration of tasks by multiple agents enables the handling of complex customer service queries. The process is continuously evaluated by the system, ensuring the relevant data is fetched and presented in an actionable format.
#31 Here, we introduce the concept of AI teammates. The marketing director uses an AI agent configured with a set of skills to perform specific tasks, such as content creation and coding. The agent collaborates within a team by engaging with other agents that perform tasks like writing content or coding.
This model illustrates how AI can act as a collaborative teammate, enhancing productivity by automating routine tasks. The integration with Microsoft Teams and other communication platforms ensures that the AI teammate can collaborate seamlessly with human team members.
![[DSC Europe 24] Lana Malic - AI Agents: The Future of Autonomous Decision-Mak...](https://cdn.slidesharecdn.com/ss_thumbnails/lanamalic-aiagents-250212223710-84219c4c-thumbnail.jpg?width=640&height=640&fit=bounds)
![Vibe Coding vs. Spec-Driven Development [Free Meetup]](https://cdn.slidesharecdn.com/ss_thumbnails/vibecodingvsspecdrivendevelopment-251209105622-43f455e7-thumbnail.jpg?width=640&height=640&fit=bounds)
