Microsoft Build 2024 highlighted advancements in AI technology, including updates to Microsoft Copilot, Azure AI, and partnerships to enhance AI capabilities across various industries. Key features include improvements in machine learning models, new AI frameworks, and tools to empower developers and organizations, alongside substantial investments in AI infrastructure. Notable initiatives involve collaborations with educational institutions and enhanced security measures to protect user data and systems.

1. Microsoft Build 2024
Updates
SATO Naoki (Neo)
Senior Software Engineer
Microsoft Corporation

2. The age of AI
transformation

3. Scaling laws
Training compute of notable machine learning systems over time (FLOP)
1e+2
1e+10
1e+18
1e+26
1950 2010 Today
Deep
learning
era
Pre-deep
learning
era
1.4x/year
4.1x/year
Source: Epoch, ‘Parameter, Compute and Data Trends in Machine Learning’

5. Indian teachers are drafting
better lesson plans
India Switzerland
Amid healthcare crunch, AI apps
provide more time for patients
Indonesia
To keep fish and shrimp healthy,
farmers now have a copilot to help
aka.ms/BuildAIStories

7. Microsoft
Copilot
Copilot+ PC
Copilot
stack

9. Windows Copilot Runtime
Applications + Experiences
Windows Copilot Library + On-device models
AI frameworks + Toolchains
Silicon

10. Applications + Experiences
AI frameworks + Toolchains
Windows Copilot Library + On-device models
Coming June
Windows Studio Effects
Live Captions Translations
Recall User Activity
OCR
Phi-Silica

11. Applications + Experiences
AI frameworks + Toolchains
Windows Copilot Library + On-device models
Announcing
Phi-Silica
SOTA SLM included in every Copilot+ PC

12. Applications + Experiences
AI frameworks + Toolchains
Windows Copilot Library + On-device models
Coming later this year
RAG
Vector Embeddings
Text Summarization

13. AI frameworks + Toolchains
Developer preview
PyTorch now native
on Windows powered by DirectML
Developer preview
Web Neural Network
on Windows powered by DirectML
Windows Copilot Library + On-device models

14. udio lets developers responsibly build and deploy custom copilots Azure AI Search features search relevance updates and new integrations N
Service boosts performance and security for web app creation Azure Functions launches Flex Consumption Plan, extensions Azure Static W
Khan Academy and Microsoft announce partnership Azure Compute Fleet now in preview Microsoft Copilot in Azure preview open to all
Azure Logic Apps Announcing custom generative mode in preview soon New Azure virtual machine series optimized for AI and cloud-native
pabilities in Azure Database for PostgreSQL Azure API Center and generative AI capabilities in Azure API Management now available Visual
odel to Pulumi in Azure Deployment Environments Azure Service Bus updates now in preview Microsoft for Startups Founders Hub gains AI c
work 2.0 now in preview SharePoint Embedded now generally available New enhancements for custom app experiences connected to Micro
wered features and enhanced data protection in Microsoft Teams Premium New features in Microsoft Dataverse in preview Power Apps upd
Improvements to the developer experience in Windows Wide array of Windows Developer AI updates Windows Security additions an
or Visual Studio Code now in preview Microsoft adds multimodal Phi-3 model Phi-3-vision Real-time video translation in Microsoft Edge com
50+
new updates

15. Copilot stack
Microsoft Copilot Your copilots
Copilot extensibility
AI safety and security
Your data
Foundation models
AI orchestration and toolchain
AI infrastructure
Microsoft Copilot Your copilots
Copilot extensibility
AI safety and security
Your data
Foundation models
AI orchestration and toolchain
AI infrastructure
Microsoft Copilot Your copilots
Copilot extensibility
AI safety and security
Your data
Foundation models
AI orchestration and toolchain
AI infrastructure
Cloud and Edge

16. Microsoft Copilot Your copilots
Copilot extensibility
AI safety and security
Your data
Foundation models
AI orchestration and toolchain
AI infrastructure
Cloud and Edge

17. 60+
Azure regions

18. France
Largest investment to date to
accelerate the adoption of AI,
skilling, and innovation
Wisconsin
$3.3B investment
to spur AI innovation
and economic growth
Mexico
First hyper-scale
cloud datacenter region
Malaysia
US$2.2B investme
to fuel cloud and
AI transformation

19. 100%
of energy from
zero-carbon sources by 2025

20. Microsoft Azure
Datacenter CPU AI accelerator IO Network
End-to-end systems optimization

21. #1 supercomputer for cloud
Microsoft Azure
561.20 PFlop/s
TOP500, May 2024

22. November 2023
May 2024
30x
Microsoft Azure
supercomputers

23. Best AI performance. Lowest cost.
Microsoft Copilot Your copilot apps
Azure AI

24. March 2023 November 2023 May 2024
GPT-4 GPT-4 Turbo GPT-4o
⅙
⅟₁₂
Cost
Speed
2x
6x

25. Full-stack partnership
aka.ms/NVIDIAPartnership

26. Full-stack partnership
NVIDIA
Blackwell
Platform
NVIDIA
NIM-optimized
models
NVIDIA Omniverse,
NVIDIA DGX
Cloud + Fabric
Windows 365
and Copilot
GPU acceleration
Azure AI Platform Microsoft Apps
NVIDIA on Azure
Microsoft Azure
aka.ms/NVIDIAPartnership

27. Expanded partnership
aka.ms/AMDPartnership

28. General availability
ND MI300X V5
Leading
price/performance
on GPT-4
Optimized for
Microsoft Azure
workloads
aka.ms/AMDPartnership

29. Updates
Azure Maia
Integrated system
of silicon, racks,
and software
Designed for
Azure OpenAI
workloads
aka.ms/AzureMaia

30. Public preview
Azure Cobalt
Up to
40% higher
performance
Most power-efficient
compute offering
in Microsoft Azure
aka.ms/AzureCobalt

31. Azure Cobalt
Customers
aka.ms/AzureCobalt

32. Microsoft Copilot Your copilots
Copilot extensibility
AI safety and security
Your data
Foundation models
AI orchestration and toolchain
AI infrastructure
Cloud and Edge

33. Azure AI

34. Azure AI
50,000+ organizations

35. Microsoft
OpenAI
aka.ms/OpenAI

36. GPT-4o
GPT-4o achieves state-of-the-art performance on visual perception benchmarks
100%
0
50%
MathVista AI2D ChartQA DocVQA ActivityNet EgoSchema
MMMU
GPT-4o
Gemini
1.0
Ultra
Gemini
1.5
Pro
Claude
Opus
GPT-4o
Gemini
1.0
Ultra
Gemini
1.5
Pro
Claude
Opus
GPT-4o
Gemini
1.0
Ultra
Gemini
1.5
Pro
Claude
Opus
GPT-4o
Gemini
1.0
Ultra
Gemini
1.5
Pro
Claude
Opus
GPT-4o
Gemini
1.0
Ultra
Gemini
1.5
Pro
Claude
Opus
GPT-4o
Gemini
1.0
Ultra
Gemini
1.5
Pro
GPT-4o
Gemini
1.0
Ultra
Gemini
1.5
Pro

37. Azure OpenAI Service
GPT-4 Turbo
with Vision
GPT-4
GPT-3.5
Turbo
DALL·E 3 Whisper
aka.ms/AOAIUpdates

38. Azure OpenAI Service
GPT-4o
Generally available
Multimodal
input and output
Real-time
response
Safe
by design
to-speech
models
Preview
aka.ms/AOAIUpdates

39. https://azure.microsoft.com/en-us/blog/openais-fastest-model-gpt-4o-mini-is-now-available-on-azure-ai/

40. Best selection of models
Jan 2024 Feb Mar Apr May
Deci Mistral-large
Databricks
DBRX
Cohere
Command R+
Cohere Embed
Meta Llama 3
Snowflake Arctic
Phi-3-mini
Mistral-small
Phi-3-small
Phi-3-medium
Phi-3-vision

41. New Models-as-a-Service
Core42
JAIS
New
Nixtla
TimeGEN-1
New
Bria AI
Coming soon
Gretel
Coming soon
NTT DATA
Coming soon
Stability AI
Coming soon
AI21
Coming soon
Cohere
Rerank
Coming soon
aka.ms/ModelCatalog

42. https://azure.microsoft.com/en-us/blog/announcing-phi-3-fine-tuning-new-generative-ai-models-
and-other-azure-ai-updates-to-empower-organizations-to-customize-and-scale-ai-applications/
• Serverless fine-tuning for Phi-3-mini and Phi-3-medium models
• Updates to Phi-3-mini including significant improvement
in core quality, instruction-following, and structured output,
• Same day shipping of the latest models from OpenAI (GPT-4o mini),
Meta (Llama 3.1 405B), Mistral (Large 2)

43. Microsoft
Open source

44. Expanded partnership
aka.ms/ExtendedHF

45. aka.ms/ExtendedHF
Hugging Face expanded
partnership
Expanded partnership
Deeper integration
with Azure AI Studio
Additional models
coming to Azure AI
TGI enabled for
optimized runtime

46. Microsoft
SLMs

47. Phi-3
Quality vs. size in SLM
Model quality measured on MMLU benchmark
0 4 8 28
12 16 20 24
Phi-3-small
Phi-3-mini-4k
Phi-3-medium
Llama-3-8B-In
Mixtral 8x7B
Gemma 7B
Mistral 7B
80
75
65
60
70
Size in billions of parameters (active)
Model
quality
Phi-3-mini-128k
Gemma 2

48. Phi-3
Phi-3-mini
3.8B parameters
Available on MaaS
Phi-3-vision
4.2B parameters
New
Phi-3-small
7B parameters
New
Phi-3-medium
14B parameters
Available on MaaS
New
aka.ms/Phi3

49. aka.ms/Phi3
Phi-3
Phi-3-vision
4.2B parameters
New
Multimodal model
with language and
vision capabilities
Reason over
real-world images
Optimized for
chart and diagram
understanding
3-mini
arameters
e on MaaS
Phi-3-s
7B param
New

50. aka.ms/Phi3
Phi-3
Phi-3-vision
4.3B parameters
Optimized for
chart and diagram
understanding
Reason over
real-world images
Multimodal model
with language and
vision capabilities
3-mini
arameters
e on MaaS
Phi-3-s
7B param
New
New

51. Phi-3
Cloud
Local
AI PC • Mobile • Edge
(iOS, Android)

53. Announcing
aka.ms/KhanAcademy

54. Announcing
Building the future of education
with Khan Academy
Providing free access
to Khanmigo for
teachers in the US
Democratizing
AI-powered math
tutoring with Phi-3
Supporting learners
with high-quality
education experiences
aka.ms/KhanAcademy

55. Azure AI Studio
A comprehensive platform to develop
and deploy custom copilots

56. Generally available
Azure AI Studio
API and
model choice
Complete AI
toolchain
Responsible AI
tools & practices
Enterprise-grade
production at scale
AI.Azure.com

58. Announcing
Azure AI Content Safety
Coming soon
Custom
categories
Preview
Prompt
shields
Preview
Groundedness
detection
aka.ms/ContentSafetyUpdates

59. Azure AI custom models
Least
customized
Prompt
engineering
Retrieval-Augmented
Generation
Announcing
Co-innovation
with Microsoft
Most
customized

60. Announcing
Co-innovation with Microsoft
Assisted
fine tuning
Train a custom
language model
Ingest new
knowledge domains
Improve proficiency
for specific languages
Azure AI custom models

61. Microsoft Copilot Your copilots
Copilot extensibility
AI safety and security
Your data
Foundation models
AI orchestration and toolchain
AI infrastructure
Cloud and Edge

62. Microsoft Intelligent Data Platform
Azure
SQL DB
Azure
Cosmos DB
Azure for
PostgreSQL
MySQL Microsoft
Fabric
Azure
Databricks
Azure AI Microsoft
Purview
Databases • Analytics • AI • Governance

63. Microsoft Fabric
Data platform for the era of AI
Complete
analytics platform
Lake-centric
and open
Empower every
business user
AI-powered

64. 11,000+
organizations use Microsoft Fabric

66. Public preview
Real-Time Intelligence
in Microsoft Fabric
Instant actionable insights on all streaming data across clouds
aka.ms/RealTimeIntelligence

67. Public preview
Real-Time Intelligence
in Microsoft Fabric
Accelerated
decisions
AI-powered Intuitive
Unified
data estate
aka.ms/RealTimeIntelligence

68. Public preview
Microsoft Fabric Workload
Development Kit
Seamless user
experience
OneLake
integration
Enhanced customer
discoverability
Marketplace-based
monetization
aka.ms/FabDevKit

69. Public preview
Microsoft Fabric Workload
Development Kit
aka.ms/FabDevKit

71. Azure AI Search
Revolutionary retrieval
State-of-the-art
search
technology
Feature-rich
vector
database
Seamless data
and platform
integrations
Up to 12x
more scale and
performance

73. Microsoft Copilot Your copilots
Copilot extensibility
AI safety and security
Your data
Foundation models
AI orchestration and toolchain
AI infrastructure
Cloud and Edge

74. GitHub Copilot

75. 1.8 million
GitHub Copilot subscribers
The world’s most widely adopted AI developer tool

76. 50,000 organizations

78. Private preview
GitHub Copilot Workspace
Copilot-native developer environment
aka.ms/GitHubCopilotWorkspace

80. Announcing
GitHub Copilot Extensions
Creating an intelligent third-party
ecosystem in GitHub Copilot
aka.ms/GitHubCopilotExtensions

81. GitHub Copilot Extensions
aka.ms/GitHubCopilotExtensions

82. Preview
GitHub Copilot
for Azure
aka.ms/GitHubCopilotAzure

84. AI safety and security
Microsoft Copilot Your copilots
Copilot extensibility
Cloud and Edge
Your data
Foundation models
AI orchestration and toolchain
AI infrastructure

85. Microsoft Copilot

86. 68%
said Copilot helped them
jump-start the creative process
Marketing
70%
of Copilot users said they
were more productive
Knowledge workers
12%
faster at resolving
customer issues
Customer service

87. Copilot Studio
Your Copilot, your way
Customize Build Manage

88. Announcing
Microsoft Copilot connectors
Build and customize your copilot and Microsoft Copilot
using your business data, apps, and workflows
Knowledge
grounding
Real-time actions
and workflows
Q&A over
analytical data
aka.ms/TryCopilotConnectors

89. Microsoft
Copilot
Team
Personal

90. Announcing
Team Copilot
A valuable new member of the team
Meeting
facilitator
Group
collaborator
Project
manager
aka.ms/TeamCopilot

92. Microsoft
Copilot
Agents
Team
Personal

94. Microsoft Copilot Your copilots
Copilot extensibility
AI safety and security
Cloud and Edge
Your data
Foundation models
AI orchestration and toolchain
AI infrastructure

95. Secure Future Initiative
Security above all else
Secure by design Secure by default Secure operations
Protect tenants
and isolate
production systems
Protect
identities
and secrets
Protect
networks
Protect
engineering
systems
Monitor
and detect
threats
Accelerate
response and
remediation

96. Empower every person and every organization
on the planet to achieve more

97. © Copyright Microsoft Corporation. All rights reserved.