This presentation was provided by William Mattingly of the Smithsonian Institution, during the closing segment of the NISO training series "AI & Prompt Design." Session Eight: Limitations and Potential Solutions, was held on May 23, 2024.

1. Prompt Design
Limitations and Solutions with LLMs

2. 1. Introduction to Open Source Machine Learning
2. ChatGPT vs. API
3. Assistants API
4. Summarize the Limitations of LLMs
a. Context Length Limitation
b. Hardware Requirement Limitation
c. Knowledge Limitation
d. Hallucination Limitations
Goals

3. Open Source Machine Learning

4. Open Source
ML
Overview
Open source machine learning, like
open source software (OSS), is
driven by the public. It has several
components: open source datasets,
open source machine learning
models, and open source
applications.
The best resource: HuggingFace

5. Open Source
ML
Datasets
● Datasets for training
task-specific models
○ NER
○ Text Classification
○ Image Classification
○ Object Detection
● Datasets for training language
models
○ Unannotated collections of texts
● Dataset Cards
○ Task
○ Language
○ Biases

6. Open Source
ML
Models
● Trained Machine Learning Models
for specific Tasks
○ NER
○ Text Classification
○ Image Classification
○ Object Detection
○ ASR
○ HTR
○ OCR
● Trained machine learning
language models (including
LLMs)
● Dataset Cards
○ Task
○ Language
○ Biases

7. Open Source
ML
Benefits and Limitations
● Benefits
○ Open, meaning they are freely
available to use (though
sometimes with commercial
limitations)
○ Publicly Critiqued
○ Understanding of the Data
● Limitations
○ Closed models are better in many
cases (BUT!!! That gap is closing).

8. ChatGPT vs. API

9. ChatGPT
vs.
API
What’s the Difference?
1. Nature of Interaction
a. ChatGPT: Facilitates interactive,
conversational interfaces that
mimic human dialogue.
b. Application Programming Interface
(API): Primarily used for
programmatic access to services
and data.
2. Cost
a. ChatGPT is subscription-based
(unlimited usage depending on
model)
b. Service-based payment plan

10. ChatGPT
vs.
API
What’s the Same?
● Model Availability
● Model Features
○ Some features are disabled for
one vs. the other. For example,
GPT-4o accepts audio in ChatGPT
but not API (for now).
● Assistants

11. ChatGPT
vs.
API
When to use them?
When to Use ChatGPT
● Interactivity and Conversation
● General Inquiries and Explorations
● Content Creation
● Education and Training
● Accessibility
When to Use an API
● Specific Data Retrieval
● Integration with External Services
● Automating Workflows
● Scalability

12. Assistants API

13. Limitations of LLMs & Solutions

14. Limitations of
LLMs
1. Context Length
Limitation
2. Hardware Requirement
Limitation
3. Knowledge Limitation
4. Hallucination
Limitations

15. Limitations of
LLMs
Context Length
1. Chunking
a. Token Chunking
b. Sentence Chunking
c. Window-Style
2. Indexing

16. Limitations of
LLMs
Hardware Requirements and
Cost
1. API
2. Open Source Models

17. Limitations of
LLMs
Knowledge Limitation

18. Limitations of
LLMs
Knowledge Limitation
1. RAG
2. Finetuning

19. Limitations of
LLMs
Knowledge Limitation
1. RAG
2. Finetuning

20. Limitations of
LLMs
Hallucinations
1. RAG
2. Training Task-Specific
Models

21. LLMs are Powerful

22. LLMs are Powerful
BUT! they aren’t the only available tool.