2. Ken Ross
Senior Consultant
Microsoft ISD
ken@hotkrossbits.com
https://github.com/VeryKross
https://www.linkedin.com/in/techgeek/
https://twitter.com/hotkrossbits
https://www.youtube.com/@imkenross
3. TOPICS COVERED
OpenAI SDK Setup
The price of AI (Tokens)
Basics of OpenAI Chat
Training GPT for Better Outcomes
OpenAI Functions
5. OPENAI SDK SETUP
1. Account Information
1. OpenAI API Key or
1. https://platform.openai.com/
2. Azure OpenAI Key and URL for
your
OpenAI Instance
2. Add the OpenAI SDK to your
Project/Solution
6. OPENAI SDK SETUP
1. Account Information
1. OpenAI API Key or
1. https://platform.openai.com/
2. Azure OpenAI Key and URL for your
OpenAI Instance
2. Add the OpenAI SDK to your
Project/Solution
7. OPENAI SDK SETUP
1. Account Information
1. OpenAI API Key or
1. https://platform.openai.com/
2. Azure OpenAI Key and URL for your
OpenAI Instance
2. Add the OpenAI SDK to your
Project/Solution
8. THE PRICE OF AI (TOKENS)
THE COSTS ASSOCIATED WITH USING OPENAI
9. THE PRICE OF AI (TOKENS)
There are multiple models, each with different capabilities and price points. Prices are per 1,000 tokens.
You can think of tokens as pieces of words, where 1,000 tokens is about 750 words. This paragraph is 35
tokens.
Tokens
GPT-3.5 Turbo
4K context: $0.0015/1K input, $0.002/1K output
16K context: $0.003/1K input, $0.004/1K output
GPT-4
8K context: $0.03/1K input, $0.06/1K output
32K context: $0.06/1K input, $0.12/1K output
https://azure.microsoft.com/en-us/pricing/details/cognitive-services/openai-service/
https://openai.com/pricing
11. BASICS OF OPENAI CHAT
1. Create the OpenAI Client Instance w/ Key
1. If using an Azure OpenAI instance, the URL will also be needed
2. Define Completion Options (temperature etc.)
3. Add your prompt to the Messages collection
4. Call the SDK to get the Chat Completion
5. Retrieve the message value
12. COMPLETION OPTIONS
Temperature
Determines the degree of randomness in the AI's response. A higher temperature corresponds to a higher degree of
randomness. This closely resembles ChatGPT "Conversation Style" where "More Creative" is a low Temperature and "More
Precise" is a high Temperature.
Max Tokens
Limits the length of the AI's response.
Nucleus Sampling Factor
Determines the diversity of the AI's response based on sampling previous tokens. (e.g. 0.1 = 10%)
Note: Don’t set this property if also setting Temperature – results could be unpredictable.
Frequency Penalty
Penalizes the probability of frequently occurring words in the AI's response.
Presence Penalty
Penalizes the probability of generating statements that have already been generated in the response so far.
14. TRAINING GPT FOR BETTER OUTCOMES
CHAT ROLES AND 1-SHOT, 2-SHOT TRAINING
15. TRAINING GPT FOR BETTER OUTCOMES
Chat Roles
System – gives context and instructions
User – the human prompt
Assistant – the reply from the AI
One Shot & Multi-shot Training
Simulated conversation
System: You are a Financial Analyst experienced in
corporate tax systems. You will always provide a tax
impact analysis to any question. You will never use the
word “fungible”.
User: What is the best way to invest $100,000?
Assistant: The best way to invest
$100,000 is to buy a house and rent it
out. The tax impact analysis is that you
will be able to deduct the interest on
the mortgage and the depreciation on
the house, saving you $10,000 in taxes.
18. OPENAI FUNCTIONS
Functions provide a way to generate consistent, structured JSON responses
An OpenAI Function represents the “signature” of a specific Function for a particular type of information
More than one Function can be defined, allowing OpenAI to choose the which function to use based on
the user’s prompt content
OpenAI itself does not call your Function; it will fill out your signature [JSON] so
that you can call the Function with those AI-identified parameters
In some cases, maybe all you want is the JSON so it can be used for other purposes
After calling the Function, the return value(s) can be re-inserted into the conversation,
allowing OpenAI to answer the user’s original prompt and incorporate the data from
the Function call