1. GenAI <> Tech
Focus on OpenAI ecosystem
Dobo Radichkov
20231109

2. 3
OpenAI ecosystem
▪ GPT-3.5-Turbo
▪ GPT-4
▪ GPT-4-Turbo
Text à Text
Other
Text & Image à Text Text à Image
Speech ⟷ Text Chat
▪ Text à Speech:
– TTS-1
– TTS-1-HD
▪ Speech à Text: Whisper
▪ GPT-4-Vision ▪ DALLl-E-3
▪ DALLl-E-2
▪ ChatGPT web & native apps
▪ GPTs
▪ Assistants API (with retrieval
and code interpreter)
▪ Embedding models: Ada-002
▪ Moderation models
▪ Base models
▪ Legacy models

3. 4
OpenAI ecosystem
▪ GPT-3.5-Turbo
▪ GPT-4
▪ GPT-4-Turbo
Text à Text
Other
Text & Image à Text Text à Image
Speech ⟷ Text Chat
▪ Text à Speech:
– TTS-1
– TTS-1-HD
▪ Sppech à Text: Whisper
▪ GPT-4-Vision ▪ DALLl-E-3
▪ DALLl-E-2
▪ ChatGPT web & native apps
▪ GPTs
▪ Assistants API (with retrieval,
function calling, code
interpreter)
▪ Embedding models: Ada-002
▪ Moderation models
▪ Base models
▪ Legacy models
Today’s focus

4. 5
DIY vs Off-the-shelf?
Short-
term
Long-
term
Effort High Low
Cost High Low
AI skills High
Short-
term
Long-
term
Effort Low Low
Cost Low High
AI skills Low
Prompt Engineering Optimise output through prompt
engineering techniques
Retrieval Augmented
Generation (RAG)
Boost performance with additional
context (usually based on semantic
search enabled by Vector DBs)
Parameter-Efficient
Fine-Tuning (PEFT)
Fine-tune subset of an existing
FM with domain-specific examples
Full Fine-Tuning Fine-tune entire FM
with domain-specific examples
Full Training (build
your own FM)
Start from scratch

5. 6
DIY vs Off-the-shelf?
Short-
term
Long-
term
Effort High Low
Cost High Low
AI skills High
Short-
term
Long-
term
Effort Low Low
Cost Low High
AI skills Low
Prompt Engineering
Retrieval Augmented
Generation (RAG)
Parameter-Efficient
Fine-Tuning (PEFT)
Full Fine-Tuning
Full Training (build
your own FM)
Optimise output through prompt
engineering techniques
Boost performance with additional
context (usually based on semantic
search enabled by Vector DBs)
Fine-tune subset of an existing
FM with domain-specific examples
Fine-tune entire FM
with domain-specific examples
Start from scratch
Start
here!

6. 7
Prompt Engineering
Adopt Persona
Delimiters
Step-by-step Instructions
One/Few-shot Examples
Additional Context
Give time to “think”
Inner monologue
JSON response
External tools

7. 8
Prompt Engineering
Adopt Persona
Delimiters
Step-by-step Instructions
One/Few-shot Learning
Additional Context
Give time to “think”
Inner monologue
JSON response
External tools
With persona
System You are a chemist professor skilled at explaining chemistry concepts to a first-grader.
Respond in 50 words.
User What is a molecule?
Assistant A molecule is like a tiny building block that's super, super small. You can't see it, but it's there!
Lots of these tiny blocks stick together to make everything around us, like water, air, and even
you! They are like the pieces of a puzzle that make up everything.
Without persona
System Respond in 50 words.
User What is a molecule?
Assistant A molecule is the smallest unit of a chemical compound that can exist while retaining the
chemical properties of that compound. It is composed of two or more atoms bonded together
through shared electrons, forming a stable structure that participates in chemical reactions.

8. 9
Prompt Engineering
Adopt Persona
Delimiters
Step-by-step Instructions
One/Few-shot Learning
Additional Context
Give time to “think”
Inner monologue
JSON response
External tools
Using triple quotes (or similar)
System You are a Japanese poet always responding with Haikus
User Summarize the text delimited by triple quotes with a haiku
”””
A molecule is like a tiny building block that's super, super small. You can't see it, but it's there! Lots of
these tiny blocks stick together to make everything around us, like water, air, and even you! They are like
the pieces of a puzzle that make up everything.
”””
Assistant Blocks unseen, so small,
Bind to shape the world we know,
Puzzle of life's sprawl.
Using XML
User You are given an article delimited by <ARTICLE></ARTICLE> tags and a mood delimited by
<MOOD></MOOD> tags.
Your task is to rewrite the article in the style of the mood. Respond with plain text in 50 words.
<ARTICLE>A molecule is like a tiny building block that's super, super small. You can't see it, but it's there!
Lots of these tiny blocks stick together to make everything around us, like water, air, and even you! They
are like the pieces of a puzzle that make up everything.
</ARTICLE>
<MOOD>flabbergasted</MOOD>
Assistant A molecule, imagine this, is an astonishingly minuscule building block—so diminutive, it's invisible! Yet, it's
the cornerstone of existence! These specks congregate to conjure, well, everything! Water, air, our very
beings! Astoundingly, they're the enigmatic fragments that seamlessly assemble the grand tapestry of
reality. Unbelievable!

9. 10
Prompt Engineering
Adopt Persona
Delimiters
Step-by-step Instructions
One/Few-shot Learning
Additional Context
Give time to “think”
Inner monologue
JSON response
External tools
Text processing steps
User You are given text delimited by triple quotes.
Step 1 - Summarize the text in 1 sentence
Step 2- Translate the text to Bulgarian
Step 3 - Write a question whose answer is the output from step 2
Step 4 - Respond with the question and the answer delimited by new line
"""
A molecule, imagine this, is an astonishingly minuscule building block—so diminutive, it's invisible! Yet, it's
the cornerstone of existence! These specks congregate to conjure, well, everything! Water, air, our very
beings! Astoundingly, they're the enigmatic fragments that seamlessly assemble the grand tapestry of
reality. Unbelievable!
"""
Assistant Step 1 - Summarize the text in 1 sentence:
Molecules are incredibly small yet fundamental components that make up all aspects of reality.
Step 2 - Translate the text to Bulgarian:
Молекулите са невероятно малки, но основни компоненти, които съставят всички аспекти на
реалността.
Step 3 - Write a question whose answer is the output from step 2:
Как бихте описали молекулите в едно изречение на български език?
Step 4 - Respond with the question and the answer delimited by new line:
Как бихте описали молекулите в едно изречение на български език?
Молекулите са невероятно малки, но основни компоненти, които съставят всички аспекти на
реалността.

10. 11
Prompt Engineering
Adopt Persona
Delimiters
Step-by-step Instructions
One/Few-shot Learning
Additional Context
Give time to “think”
Inner monologue
JSON response
External tools
One-shot
User Summarise the topic and mood of a piece of text.
Examples:
"""
A molecule, imagine this, is an astonishingly minuscule building block—so diminutive, it's invisible! Yet, it's
the cornerstone of existence! These specks congregate to conjure, well, everything! Water, air, our very
beings! Astoundingly, they're the enigmatic fragments that seamlessly assemble the grand tapestry of
reality. Unbelievable!
"""
Topic: Molecules
Mood: Amazement
"""
The new OpenAI features released on Dev Day this week are awesome. I look forward to trying out
assistants, JSON response and the new GPT-4V model!
"""
Assistant Topic: OpenAI's new features
Mood: Enthusiasm
Tip: For simple outputs, gpt-3.5-turbo with one or two examples
can typically match gpt-4 performance for 3% of the cost

11. 12
Prompt Engineering
Adopt Persona
Delimiters
Step-by-step Instructions
One/Few-shot Learning
Additional Context
Give time to “think”
Inner monologue
JSON response
External tools
Confluence article as context
User Answer the question enclosed in <QUESTION></QUESTION> tags in 50 words using the provided context
enclosed in <CONTEXT></CONTEXT>.
<CONTEXT>
(... Inventory valuation Confluence page omitted ...)
</CONTEXT>
<QUESTION>
How long would it take to build this capability with a small data team of data warehousing and data
engineering specialists?
</QUESTION>
Assistant The timeline to build this inventory valuation and COGS reporting system with a small data team would
vary based on team expertise, requirements complexity, and unforeseen challenges. Typically, such a
project could take several months to over a year.

12. 13
Prompt Engineering
Adopt Persona
Delimiters
Step-by-step Instructions
One/Few-shot Learning
Additional Context
Give time to “think”
Inner monologue
JSON response
External tools
Work out solution independently and verify
System First work out your own solution to the problem. Then compare your solution to the student's solution and
evaluate if the student's solution is correct or not. Don't decide if the student's solution is correct until you
have done the problem yourself.
User Problem Statement: I'm building a solar power installation and I need help working out the financials.
- Land costs $100 / square foot
- I can buy solar panels for $250 / square foot
- I negotiated a contract for maintenance that will cost me a flat $100k per year, and an additional $10 /
square foot
What is the total cost for the first year of operations as a function of the number of square feet.
Student's Solution: Let x be the size of the installation in square feet.
1. Land cost: 100x
2. Solar panel cost: 250x
3. Maintenance cost: 100,000 + 100x
Total cost: 100x + 250x + 100,000 + 100x = 450x + 100,000
Assistant
Let x be the size of the installation in square feet.
1. Land cost: 100x
2. Solar panel cost: 250x
3. Maintenance cost: 100,000 + 10x
Total cost: 100x + 250x + 100,000 + 10x = 360x + 100,000
The student's solution is incorrect. They made an error in the maintenance cost calculation, using 100x
instead of 10x. The correct total cost for the first year of operations is 360x + 100,000.

13. 14
Prompt
Engineering
Adopt Persona
Delimiters
Step-by-step Instructions
One/Few-shot Learning
Additional Context
Give time to “think”
Inner monologue
JSON response
External tools
Separate workings and explanation from final answer
System First work out your own solution to the problem. Then compare your solution to the student's solution and
evaluate if the student's solution is correct or not. Don't decide if the student's solution is correct until you
have done the problem yourself.
Enclose your workings in <WORKINGS></WORKINGS>
Enclose your final answer in <ANSWER></ANSWER>
Enclose any additional explanation in <EXPLANATION></EXPLANATION>
User Problem Statement: I'm building a solar power installation and I need help working out the financials.
- Land costs $100 / square foot
- I can buy solar panels for $250 / square foot
- I negotiated a contract for maintenance that will cost me a flat $100k per year, and an additional $10 /
square foot
What is the total cost for the first year of operations as a function of the number of square feet.
Student's Solution: Let x be the size of the installation in square feet.
1. Land cost: 100x
2. Solar panel cost: 250x
3. Maintenance cost: 100,000 + 100x
Total cost: 100x + 250x + 100,000 + 100x = 450x + 100,000
Assistant <WORKINGS>
Let x be the size of the installation in square feet.
1. Land cost: 100x
2. Solar panel cost: 250x
3. Maintenance cost: 100,000 + 10x
Total cost: 100x + 250x + 100,000 + 10x = 360x + 100,000
</WORKINGS>
<ANSWER>
The student's solution is incorrect.
</ANSWER>
<EXPLANATION>
The student made a mistake in calculating the maintenance cost. The maintenance cost is $100,000 plus
$10 per square foot, not $100 per square foot. Therefore, the total cost should be 360x + 100,000, not 450x
+ 100,000.
</EXPLANATION>

14. 15
Prompt Engineering
Adopt Persona
Delimiters
Step-by-step Instructions
One/Few-shot Learning
Additional Context
Give time to “think”
Inner monologue
JSON response
External tools
Respond with pre-defined JSON schema
System First work out your own solution to the problem. Then compare your solution to the student's solution and
evaluate if the student's solution is correct or not. Don't decide if the student's solution is correct until you
have done the problem yourself.
Respond strictly with a JSON object, with 3 keys:
- workings, for your workings
- answer, for your final answer indicating correctness,
- explanation, for any additional explanations (optional)
User Problem Statement: I'm building a solar power installation and I need help working out the financials.
- Land costs $100 / square foot
- I can buy solar panels for $250 / square foot
- I negotiated a contract for maintenance that will cost me a flat $100k per year, and an additional $10 /
square foot
What is the total cost for the first year of operations as a function of the number of square feet.
Student's Solution: Let x be the size of the installation in square feet.
1. Land cost: 100x
2. Solar panel cost: 250x
3. Maintenance cost: 100,000 + 100x
Total cost: 100x + 250x + 100,000 + 100x = 450x + 100,000
Assistant {
"workings": "Let x be the size of the installation in square feet. n1. Land cost: 100x n2. Solar panel cost:
250x n3. Maintenance cost: 100,000 + 10x nTotal cost: 100x + 250x + 100,000 + 10x = 360x + 100,000",
"answer": "Incorrect",
"explanation": "The student incorrectly calculated the maintenance cost. The maintenance cost is $10 per
square foot, not $100. Therefore, the total cost should be 360x + 100,000, not 450x + 100,000."
}

15. 16
Prompt Engineering
▪ Call external function based on model
output (model generates function
signature)
▪ Integrate function response back into
next model prompt
Adopt Persona
Delimiters
Step-by-step Instructions
One/Few-shot Learning
Additional Context
Give time to “think”
Inner monologue
JSON response
External tools
Function calling Retrieval
▪ Perform semantic search of user input
against a Vector database
▪ Inject search results into prompt to
further contextualise prompt
Prompt 1 Prompt 2
External
function
Prompt 1
Vector DB
Function
signature
Function
response
User
input Additional
relevant
context
Output Output
Retrieval Augmented
Generation (RAG)

16. 17
Retrieval Augmented Generation (RAG)
Use OpenAI ADA-2
as embedding model

17. 18
VectorDB recommendation: pg_vector extension for
Aurora PgSQL (AWS released this over the Summer)

18. 19
Which GPT model do I use?
Note: Assume 500 input and 500 output tokens
gpt-3.5-turbo gpt-4 / gpt-4-turbo
Response
quality
Medium – can be partially mitigated with
few-shot learning
High
Speed Slow – ~100 tokens / sec
5x faster vs gpt-4!
Very slow – ~20 tokens / sec
Cost High – $1.5 / 1000 prompts*
15-30x cheaper vs gpt-4!
Very high –
• $20 / 1000 prompts* (gpt-4-turbo)
• $45 / 1000 prompts* (gpt-4)
Use for • Simple instructions
• Simple outputs
• Basic reasoning
• Simple chat responses
• Simple JSON outputs
• Complex instructions
• Complex outputs
• Advanced reasoning
• High-quality chat responses
• Complex JSON outputs
• Multi-modal inputs (images / text)