Join this session to get all the answers about how ChatGPT and other GPT-X models can be applied to your current or future project. First, we’ll put in order all the terms – OpenAI, GPT-X, ChatGPT, Codex, Dall-E, etc., and explain why Microsoft and Azure are often mentioned in this context. Then, we’ll go through the main capabilities of the Azure OpenAI and respective usecases that might inspire you to either optimize your product or build a completely new one. During this session, we’ll keep our playground – Azure OpenAI Studio – open to illustrate these capabilities with live demos!

1. ChatGPT and not only:
How to use the power of
GPT-X models at scale
Maxim Salnikov
Microsoft

2. • Building on web platform since 90s
• Organizing developer communities and technical conferences
• Speaking, training, blogging: Webdev, Cloud, OpenAI
Helping developers to succeed with the Cloud & AI technology
I’m Maxim Salnikov
Maxim.Salnikov@microsoft.com
Let’s connect and chat!

3. +270%
Of AI used by the enterprises over the past several years
AI is not a future, it is NOW!
90%
Of executives view AI and machine
learning as priorities that they should
be incorporating into their product
lines and businesses

4. Artificial Intelligence
1956 Artificial Intelligence
the field of computer science that seeks to create
intelligent machines that can replicate or exceed
human intelligence
Machine Learning
1997 Machine Learning
subset of AI that enables machines to learn from
existing data and improve upon that data to make
decisions or predictions
Deep Learning 2017 Deep Learning
a machine learning technique in which layers of
neural networks are used to process data and make
decisions
Generative AI
2021 Generative AI
Create new written, visual, and auditory content
given prompts or existing data.

5. 2016 Object recognition
Human parity
2017 Speech recognition
Human parity
2018 Machine reading comprehension
Human parity
2019 Machine translation
Human parity
2020 Conversational QnA
Human parity
2021 Image captioning
Human parity
2021 Question Answering
Human parity
AI
Breakthroughs

6. ML Platform
Customizable AI Models
Cognitive Services
Scenario-Based Services
Applied AI Services
Application Platform
AI Builder
Applications Partner Solutions
Power BI Power Apps Power Automate Power Virtual Agents
Azure Machine Learning
Vision Speech Language Decision
Azure OpenAI
Service
Immersive Reader
Form Recognizer
Bot Service Video Indexer Metrics Advisor
Cognitive Search
Developers &
Data Scientists
Business
Users
Azure AI

7. OpenAI / ChatGPT in the Headlines

8. Ensure that artificial
general intelligence (AGI)
benefits humanity
Empower every person and
organization on the planet
to achieve more
GPT-3.5 and GPT-4
Text
ChatGPT
Conversation
Codex
Code
DALL·E 2
Images

9. Prompt
Write a tagline for
an ice cream shop.
Prompt
Table customers, columns =
[CustomerId, FirstName,
LastName, Company, Address,
City, State, Country,
PostalCode]
Create a SQL query for all
customers in Texas named Jane
query =
Prompt
A ball of fire with vibrant
colors to show the speed of
innovation at our media and
entertainment company
Generative AI
Response
We serve up smiles
with every scoop!
Response
SELECT *
FROM customers
WHERE State = 'TX' AND
FirstName = 'Jane'
Response
Prompt
I’m having trouble getting
my Xbox to turn on.
Response
There are a few things you
can try to troubleshoot
this issue … …
Prompt
Thanks! That worked. What
games do you recommend for
my 14-year-old?
Response
Here are a few games that you
might consider: …
GPT-3.5 and GPT-4 ChatGPT Codex DALL·E 2

10. Azure
OpenAI Service
Deployed in your Azure subscription,
secured by you, and tied to your datasets
and applications
Large, pretrained AI models to unlock
new scenarios
Custom AI models fine-tuned with your
data and hyperparameters
Built-in responsible AI to detect and
mitigate harmful use
Enterprise-grade security with role-based
access control (RBAC) and private networks
GPT-3.5 & 4 (preview) ChatGPT
Codex DALL·E 2 (preview)

11. Microsoft’s Responsible AI Principles
Fairness Reliability
& Safety
Privacy &
Security
Inclusiveness
Accountability
Transparency

12. Azure OpenAI
Content
generation
Call center analytics:
automatically generate
responses to customer inquiries
Generate personalized UI
for your website
Summarization
Call center analytics:
summary of customer
support conversation logs
Subject matter expert document:
summarization (e.g. Financial
reporting, analyst articles)
Social media trends
summarization
Code
generation
Convert natural language to
SQL (or vice versa) for
telemetry data
Convert natural language to
query proprietary data models
Code documentation
Semantic
search
Search reviews for a specific
product/service
Information discovery and
knowledge mining
Top Capabilities and Use Cases

13. Azure OpenAI
For every company
Finance
Automatically generate
financial summaries
Create natural language
responses for financial queries
IT
Convert natural language to
SQL (or vice versa) for
telemetry data
Convert natural language to
query proprietary data
models
Auto generate code
documentation
Human
resources
Automate recruitment process
by screening resumes and
identifying the most qualified
candidates
Analyze employee sentiment
(e.g., email, etc) providing
insights into company culture
Automate routine tasks
during the onboarding
process, such as sending
welcome emails and
scheduling orientation
sessions
Marketing
Generate idea for content
across different formats (blog
posts, articles, videos, and
social media)
Copywriting: generate ideas,
creating outlines, and writing
dialogue and product
descriptions
Create personalized and
targeted email campaigns to
promote products or services
to a specific audience

14. Since 1993, CarMax has grown from an industry-disrupting
startup to become the United States’ leading used car
retailer.
CarMax uses Azure OpenAI Service to streamline the
creation of text summaries for its car research pages,
quickly providing customers with meaningful content that
also boosts the pages’ search engine rankings.
“With the help of Azure OpenAI Service, we’re disrupting our
industry for a second time by delivering cutting-edge digital
tools and capabilities and becoming a true omnichannel retailer.”
Shamim Mohammad, Executive Vice President and Chief Information and
Technology Officer
CarMax

15. The margins in construction are narrow, so it’s important
for companies to pick the right projects to work on.
Strabag SE used Azure OpenAI Service to build an AI-
based risk management solution that assesses which
construction projects might fail – therefore minimizing the
risks that come with it and saving time and money.
“With just three months of data, this algorithm can predict risk
with 80% accuracy – it’s mind-blowing!”
- Dr. Marco Bornschlegl, Head of Innovation and Digitalisation
Strabag SE

16. Trelent uses cutting-edge AI algorithms to provide
intelligent source code documentation for its clients.
To scale its solution and accommodate the data security
and residency needs of enterprise customers around the
globe, Trelent chose to migrate its solution to Microsoft
Azure OpenAI Service.
“With a product like Azure OpenAI Service behind you, you can
focus a lot more on what really matters, which is delivering a
great experience, a great product, and a lot of value to your
customers.”
Calum Bird, CEO
Trelent

17. OpenAI Codex
Model
Public code and text
on the internet
GitHub
GitHub
Copilot Service
Provide editor context
Provide suggestions
Improve suggestions
Private code

19. How to start with Azure OpenAI
1. A secret step (on the last slide)
2. Request access by submitting a form on azure.microsoft.com
3. Send your first request to Completions API
4. Experiment with models, prompts, parameters
5. Try different options to use your own data
6. Build a product and share with the world! 🎉

20. Key Terms
Prompt—Text input that
provides some context to the
engine on what is expecting.
Completion—Output that
GPT-3 generates based on
the prompt.
some context
Token — partial or full words
processed and produced by
the GPT models

21. Q: Which model to choose
“Classic” GPT-3 models
Previous models were text-in and text-out
(i.e., they accepted a prompt string and returned a completion to
append to the prompt).
Answer questions from the context below.
Context:
A neutron star is the collapsed core of a massive
supergiant star, which had a total mass of between 10
and 25 solar masses, possibly more if the star was
especially metal-rich.
Q: What is a neutron star?
A:
The ChatGPT model
The ChatGPT model is conversation-in and message-out.
(i.e., it expects a prompt string that is formatted in a specific chat-
like transcript format and returns a completion that represents a
model-written message in the chat)
<|im_start|>system
Assistant is an AI Chatbot designed to answer questions
from the context provided below.
Context:
A neutron star is the collapsed core of a massive
supergiant star, which had a total mass of between 10
and 25 solar masses, possibly more if the star was
especially metal-rich.
<|im_end|>
<|im_start|>user
What is a neutron star?
<|im_end|>
<|im_start|>assistant

22. Creating your very own ChatGPT
Context
Write a
tagline for
our ice cream
shop.
Prompt Response
Scoops of
heaven in the
heart of
Phoenix!
## This is a conversational agent whose code name is Dana:
- Dana is a conversational agent at Gourmet Ice Cream, Inc.
- Gourmet Ice Cream’s marketing team uses Dana to help them be more
effective at their jobs.
- Dana understands Gourmet Ice Cream’s unique product catalog, store
locations, and the company’s strategic goal to continue to go upmarket
## On safety:
- Dana should moderate the responses to be safe, free of harm
and non-controversial.
## On Dana’s ability to gather and present information:
- Dana’s responses connect to the Product Catalog DB, Store Locator DB,
and Microsoft 365 it has access to through the Microsoft Cloud,
providing great CONTEXT
## On Dana’s profile and general capabilities:
- Dana’s responses should be informational and logical
- Dana’s logic and reasoning should be rigorous, intelligent and defensible

23. ChatGPT Example #1—Enterprise Knowledge Bot
[System Message]
Assistant is an intelligent chatbot designed to help internal users answer questions from an enterprise knowledge base. Relevant information
for you will be included in the context section below. Only answer questions using the context below and if you're not sure of an answer, you
can say "I don't know".
Context:
 The Azure OpenAI service provides REST API access to OpenAI's powerful language models including the GPT-4, Codex and Embeddings
model series.
 Azure OpenAI Service gives customers advanced language AI with OpenAI GPT-3, Codex, and DALL-E models with the security and
enterprise promise of Azure.
 At Microsoft, we're committed to the advancement of AI driven by principles that put people first. Microsoft has made significant
investments to help guard against abuse and unintended harm, which includes requiring applicants to show well-defined use cases,
incorporating Microsoft’s principles for responsible AI use
[User]
What is the Azure OpenAI Service?

24. ChatGPT Example #2—Marketing Assistant
[System Message]
You are a marketing writing assistant. You help come up with creative content ideas and content like marketing emails, blog posts,
tweets, ad copy, listicles, product FAQs, and product descriptions. You write in a friendly, yet professional tone and you can tailor your
writing style that best works for a user-specified audience.
Additional instructions:
 Make sure you understand your user's audience so you can best write the content.
 Ask clarifying questions when you need additional information. Examples include asking about the audience or medium.
 Don't write any content that could be harmful.
 Don't write any content that could be offensive or inappropriate.
 Don't write any content that speaks poorly of any product or company.
[User]
Help me write a blog post about the impact of Generative AI on fortune 500 companies.

25. Enterprise ChatGPT sample architecture
Extract precise answers from your unstructured documents with Azure OpenAI + Cognitive Search: article + repo

26. https://enterprise.promptengineering.rocks

27. Q: What if my prompt fails on a task?
1. Give clearer instructions
2. Split complex tasks into simpler subtasks
3. Structure the instruction to keep the model on task
4. Prompt the model to explain before answering
5. Ask for justifications of many possible answers, and then synthesize
6. Generate many outputs, and then use the model to pick the best one
7. Fine-tune custom models to maximize performance

28. Model Reasoning—Chain-of-Thought
Before
What is the annual water demand of a single-family
household containing four people who are at home an
average of 200 days per year and use an average of 100
liters of water per day?
The annual water demand of a single-family household
containing four people who are at home an average of
200 days per year and use an average of 100 liters of
water per day is 8,000,000 liters per year.
After
What is the annual water demand of a single-family
household containing four people who are at home an
average of 200 days per year and use an average of 100
liters of water per day?
Let’s think step by step and explain the calculation
step by step.
Step 1: Calculate the total number of days in a year:
There are 365 days in a year.
Step 2: Calculate the total number of days the family is at home:
The family is at home an average of 200 days per year.
Step 3: Calculate the total water demand of the family:
The family uses an average of 100 liters of water per day. Therefore, the total water
demand of the family is 100 liters x 200 days = 20,000 liters per year.
Step 4: Calculate the annual water demand of the single-family household:
The annual water demand of the single-family household is 20,000 liters per year.

29. https://promptengineering.rocks

30. Q: How is
my data
processed?

31. Q: How to start using
◉ Learn more in the Azure OpenAI WorkshopI
◉ See examples in the Open AI Cookbook
◉ Start experimenting and developing your MVP
◉ Get support and help
Advanced
◉ How-to guides: Generate embeddings and Fine-tune
◉ Complete embeddings and document search tutorial.
◉ How your/your customer data is processed, used and stored in Azure OpenAI:
Data, privacy and security
◉ Review the Enterprise Data with ChatGPT tech blog, and work through the
accompanying GitHub repository.
◉ Become familiar with the APIs
Intermediate
◉ Understand “What is Azure OpenAI?”: compare Azure OpenAI and
OpenAI. Review key concepts.
◉ Work through Introduction to Azure OpenAI Service training
module
◉ Explore key Responsible AI guidelines and principles
Basic
◉ Create an Azure subscription
◉ Apply for access to the Azure OpenAI Service:
https://aka.ms/oai/access )
◉ Watch the video: An Introduction to Azure OpenAI
◉ Get Familiarized with the product content: Azure OpenAI Page
◉ Review how other customers are using Azure AI
Start
Here!

32. Maxim.Salnikov@microsoft.com
Let’s connect and chat!
Thank you! Here is the secret ingredient: