Microsoft 365 Copilot is an AI assistant created by Microsoft to be helpful, harmless, and honest. It combines a large language model with data from Microsoft 365 services like Outlook, Teams, and the Microsoft Graph to provide contextual responses and suggestions. Copilot aims to unlock productivity and creativity within Microsoft 365 apps while prioritizing privacy, security, and responsible AI practices like interpretability and oversight.

1. From Prompt to Product:
Microsoft 365 Copilot
Tamara Stankovic
Senior Applied Science Manager
Microsoft Development Center Serbia

2. Microsoft 365 Copilot
Your AI assistant at work

3. Unlock productivity and unleash creativity
Microsoft 365 Copilot
Microsoft Graph
- Your Data -
Large Language
Models
Microsoft 365
Apps
Natural Language
The
Internet

4. Security Compliance Privacy Responsible AI
Microsoft 365 Copilot
Built on Microsoft’s comprehensive approach

5. Copilot for Microsoft 365 | Architecture

6. User prompt
Data flow ( = all requests are encrypted via HTTPS and wss://)
User prompts are sent to Copilot
1
Microsoft 365 Service Boundary
1
Microsoft Copilot for Microsoft 365 architecture
*Represents Microsoft 365 Chat on Bing

7. User prompt
Data flow ( = all requests are encrypted via HTTPS and wss://)
User prompts are sent to Copilot
1
Microsoft 365 Service Boundary
1
Pre-processing
Grounding
2
Microsoft Graph
Your context and content
(emails, files, meetings, chats,
calendars, and contacts)
Customer Microsoft 365 Tenant
Copilot accesses Graph + (optional) Web + Other services for grounding
2
*Represents Microsoft 365 Chat on Bing
Microsoft Copilot for Microsoft 365 architecture
Dataverse + Power
Platform Services
Bing
Plug-ins

8. User prompt
Data flow ( = all requests are encrypted via HTTPS and wss://)
User prompts are sent to Copilot
1
Microsoft 365 Service Boundary
1
Pre-processing
Grounding
2
Microsoft Graph
Your context and content
(emails, files, meetings, chats,
calendars, and contacts)
Customer Microsoft 365 Tenant
Copilot accesses Graph + (optional) Web + Other services for grounding
2
Modified
prompt
3
Large Language
Model
RAI is performed on
input prompt and
output results
RAI
Prompts, responses, and grounding data
aren’t used to train foundation models
Azure
OpenAI
Azure OpenAI instance
is maintained by
Microsoft. OpenAI has
no access to the data
or the model
Copilot sends modified prompt to Large Language Model (LLM)
3
3
*Represents Microsoft 365 Chat on Bing
Microsoft Copilot for Microsoft 365 architecture
Dataverse + Power
Platform Services
Bing
Plug-ins

9. User prompt
Data flow ( = all requests are encrypted via HTTPS and wss://)
User prompts are sent to Copilot
1
Microsoft 365 Service Boundary
1
Pre-processing
Grounding
2
Microsoft Graph
Your context and content
(emails, files, meetings, chats,
calendars, and contacts)
Customer Microsoft 365 Tenant
Copilot accesses Graph + (optional) Web + Other services for grounding
2
Modified
prompt
3
Large Language
Model
RAI is performed on
input prompt and
output results
RAI
Prompts, responses, and grounding data
aren’t used to train foundation models
Azure
OpenAI
Azure OpenAI instance
is maintained by
Microsoft. OpenAI has
no access to the data
or the model
Copilot sends modified prompt to Large Language Model (LLM)
3
LLM
response
4
Copilot receives LLM response
4
3
*Represents Microsoft 365 Chat on Bing
Microsoft Copilot for Microsoft 365 architecture
Dataverse + Power
Platform Services
Bing
Plug-ins

10. User prompt
Data flow ( = all requests are encrypted via HTTPS and wss://)
User prompts are sent to Copilot
1
Microsoft 365 Service Boundary
1
Pre-processing
Grounding
2
Microsoft Graph
Your context and content
(emails, files, meetings, chats,
calendars, and contacts)
Customer Microsoft 365 Tenant
Copilot accesses Graph + (optional) Web + Other services for grounding
2
Modified
prompt
3
Large Language
Model
RAI is performed on
input prompt and
output results
RAI
Prompts, responses, and grounding data
aren’t used to train foundation models
Azure
OpenAI
Azure OpenAI instance
is maintained by
Microsoft. OpenAI has
no access to the data
or the model
Copilot sends modified prompt to Large Language Model (LLM)
3
LLM
response
4
Copilot receives LLM response
4
Copilot accesses Graph for Compliance and Purview
5
Compliance and Purview
Post-processing
5
3
*Represents Microsoft 365 Chat on Bing
Microsoft Copilot for Microsoft 365 architecture
Dataverse + Power
Platform Services
Bing
Plug-ins

11. User prompt
Data flow ( = all requests are encrypted via HTTPS and wss://)
User prompts are sent to Copilot
1
Microsoft 365 Service Boundary
1
Pre-processing
Grounding
2
Dataverse + Power
Platform Services
Bing
Plug-ins
Microsoft Graph
Your context and content
(emails, files, meetings, chats,
calendars, and contacts)
Customer Microsoft 365 Tenant
Copilot accesses Graph + (optional) Web + Other services for grounding
2
Modified
prompt
3
Large Language
Model
RAI is performed on
input prompt and
output results
RAI
Prompts, responses, and grounding data
aren’t used to train foundation models
Azure
OpenAI
Azure OpenAI instance
is maintained by
Microsoft. OpenAI has
no access to the data
or the model
Copilot sends modified prompt to Large Language Model (LLM)
3
LLM
response
4
Copilot receives LLM response
4
Copilot accesses Graph for Compliance and Purview
5
Compliance and Purview
Post-processing
5
3
Copilot sends the response back to the user
6
Copilot response
back to the user
6
Microsoft Copilot for Microsoft 365 architecture

12. Microsoft 365
Copilot stack
AI infrastructure
Foundation models
Hosted fine-tuned foundation models
Hosted foundation models
Grounding
Metaprompt
Prompt & response filtering
Orchestration
Copilot frontend
UX

13. Plugin extensibility
UX
Microsoft 365 Copilot
Retrieval Augmented Generation (RAG)
Semantic Index
People
Devices
Files
Events
Chats & channels
Notifications
Tasks
Groups
Security
Microsoft Graph

14. LLM prompt
 User prompt specifies the task for Copilot
 Relevant data from Graph, Web and other grounding sources, is used to augment the prompt
 Requirement: find relevant data
 Chat session history is also used
 Microsoft 365 has its own default prompt, which gives Copilot responsible rules for interaction
User Prompt + Grounding data + Chat History
+ System Prompt

15. Microsoft’s
Responsible AI
Principles
Privacy &
Security
Inclusiveness
Accountability
Fairness
Reliability &
Safety
Transparency
Building blocks
to enact principles
Tools and processes
Training and practices
Rules
Governance

16. Microsoft’s Responsible AI Standard
Guidance for designing, developing, deploying, and using AI
Identify
Identify and prioritize
potential harms that
could result from your
AI system through
iterative red-teaming,
stress-testing, and
analysis.
Measure
Measure the
frequency and
severity of those
harms by establishing
clear metrics, creating
measurement test
sets, and completing
iterative, systematic
testing (both manual
and automated).
Mitigate
Mitigate harms by
implementing tools
and strategies such as
prompt engineering
and using our content
filters. Repeat
measurement to test
effectiveness after
implementing
mitigations.
Operate
Define and execute
a deployment and
operational readiness
plan.
Review our Responsible AI practices to learn more

17. What’s the difference between ChatGPT and
Microsoft 365 Copilot?
ChatGPT is a general-purpose LLM that works like it a
chatbot. It uses machine learning algorithms to answer
user questions or prompts in a way that mimics human
conversations.
ChatGPT cannot search the internet, and its knowledge
base does not extend beyond September 2021. This means
it can “hallucinate” and provide wrong information.
Popular ChatGPT use cases include text compositions, lists,
coding, and content summaries.
Microsoft 365 Copilot is an AI-powered digital assistant
that combines ChatGPT with data from the Microsoft
Graph and Microsoft 365 apps. It uses data from user
calendars, emails, chats, documents, meetings, and
more to deliver contextual, relevant solutions directly in
Microsoft 365 apps.
This approach only improves the context for a user’s
specific scenario; Microsoft does not leverage user data
to train the Copilot model.
Since Copilot works directly in Microsoft 365 apps, it can
do things like create new content in PowerPoint, suggest
email replies in Outlook, and recap a Teams meeting.
Read more about the difference between ChatGPT and Microsoft 365 Copilot.

18. Thank You!