Mattingly "AI & Prompt Design: Large Language Models"

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

AI and Prompt Design
Large Language Models

1. How machine learning works with texts
2. Text Embeddings
3. Transformer Models
4. Transformer Embeddings
5. What are La...

The _____ ran after the car.

The dog ran after the car.

The elephant ran after the car.

The snowman ran after the car.

Machine Learning and Texts

Tokens
A "token" is defined as a
single, indivisible unit of
text, such as a word,
punctuation mark,
whitespace, or a
numb...

The dog ran after the car.

The dog ran after the car.

The dog ran after the car.

The dog ran after the car.

The boy’s dog ran after the car.

The boy’s dog ran after the car.

Text Embeddings

Text
Embeddings
● Bag-of-Words
● Embeddings

Text
Embeddings
Bag-of-Words
● The apple is in the tree.
○ 1-the
○ 2-apple
○ 3-is
○ 4-in
○ 1-the
○ 5-tree
● [1, 2, 3, 4, 1...

Text
Embeddings
Embeddings
● The apple is in the tree.
○ 1-[0.01234, -0.23456, 0.87654,
0.45678, -0.56123, 0.65432,
0.1234...

Text
Embeddings
Live Demo
https://wjbmattingly.com/word-
embeddings-demo/
https://wjbmattingly.com/text-
embeddings-demo/

Supervised Learning

Example Label

Example Label
You are great! Positive

Example Label
You are great! Positive
This was terrible! Negative

Example Label
You are great! Positive
This was terrible! Negative

Example Label
You are great! Positive
This was terrible! Negative

Example Label
You are great! Positive
This was terrible! Negative
Model

Unsupervised Learning

Corpus

Text
Corpus
Text Text
Text Text Text
Text Text Text
Text Text Text
Text Text Text

Unsupervised
Learning
Unsupervised learning for training a
language model involves training
the model on a large corpus of...

Text
Corpus
Text
Text
Text
Text

Text
Corpus
Text
Text
Text
Text

Text
Corpus
Text
Text
Text
Text

Text
Corpus
Text
Text
Text
Text
Model

Text
Corpus
Text
Text
Text
Text
Model

Text
Corpus
Text
Text
Text
Text
Model
Word Vector
Word Vector
Word Vector
Word Vector
Word Vector

Transformer Models

Transformer
Models
Overview
- Invented in 2017 by Google
- Attention Mechanism
- Masking during Training
- Encoder/Decoder...

Transformer
Models
Attention Mechanism
This allows the model to focus on
different parts of a sentence to
understand how w...

Transformer
Models
Masking
In transformer models, masking is used primarily
in the training phase to help the model learn
...

Transformer
Models
Encoder/Decoder
Transformer models can have an
encoder-decoder structure, where the
encoder processes t...

Transformer
Models
Context Window
A context window in transformer
models, like GPT (Generative
Pretrained Transformer), re...

Transformer Embeddings

Transformer
Embeddings
Overview
Transformer embeddings are
context-dependent, meaning they
can change depending on the
con...

Transformer
Embeddings
Why does this matter?

Transformer
Embeddings
Why does this matter?
She sat by the river bank and
watched the sunset.

Transformer
Embeddings
Why does this matter?
She went to the bank to deposit a
check.

Transformer
Embeddings
Why does this matter?
She sat by the river bank and
watched the sunset.
She went to the bank to dep...

Transformer
Embeddings
Why does this matter?
She sat by the river bank and
watched the sunset.
She went to the bank to dep...

Transformer
Embeddings
Why does this matter?
She sat by the river bank and
watched the sunset.
She went to the bank to dep...

Large Language Models

Large Language
Models
Large Models
Foundational Models
Generative Pre-Trained
Transformers
Can be fine-tuned to specific t...

Large Language
Models
Large
● Size of Training Corpus
○ Trained on terabytes or petabyte of
text data (trillions of words)...

Large Language
Models
Foundational Model
Generate Metadata
for Reddit Posts

Large Language
Models
Foundational Model
Food
Sports
Politics
People
Orgs
Dates
Toxic
Not Toxic

Large Language
Models
Foundational Model
Text
Classification
Model
NER
Model
Toxic
Model
Food
Sports
Politics
People
Orgs
...

Large Language
Models
Foundational Model
Text
Classification
Model
NER
Model
Toxic
Model
Food
Sports
Politics
People
Orgs
...

Large Language
Models
Foundational Model
Text
Classification
Fine-Tuned
NER
Fine-Tuned
Toxic
Fine-Tuned
Food
Sports
Politi...

Large Language
Models
Foundational Model
Text
Classification
Fine-Tuned
NER
Fine-Tuned
Toxic
Fine-Tuned
Food
Sports
Politi...

Large Language
Models
Foundational Model
Text
Classification
Fine-Tuned
NER
Fine-Tuned
Toxic
Fine-Tuned
Food
Sports
Politi...

Large Language
Models
Foundational Model
Text
Classification
Fine-Tuned
NER
Fine-Tuned
Toxic
Fine-Tuned
Food
Sports
Politi...

Large Language
Models
Generative Pre-Trained
Transformer
● Pre-training on Large Text
Corpus
● Transformer Architecture
● ...

The _____ ran after the car.

Fredegis of Tours

Hallucinations

Exercise: Forcing Hallucinations

Share Slideshare

LinkedIn
Facebook
Twitter

Embed

Size (px)

Show related Slideshows at end

WordPress Shortcode

Link

Share
Email

2024 Trend Updates: What Really Wor... by Search Engine Jou... 1036216 views
Storytelling For The Web: Integrate... by Chiara Aliotta 948112 views
Artificial Intelligence, Data and C... by OECD Directorate ... 870219 views
How to Leverage AI to Boost Employe... by SocialHRCamp 365506 views
2024 State of Marketing Report – by... by Marius Sescu 207694 views
Everything You Need To Know About C... by Expeed Software 227895 views

View on Slideshare

1 of 72 Ad

View on Slideshare

1 of 72 Ad