This document summarizes research investigating the memory properties of large language models (LLMs) and their similarities to human memory. It finds that both LLMs and human memory demonstrate similar patterns around recall, repetition, and interference effects. However, it also outlines key differences between the two types of memory, such as LLMs lacking biological and emotional aspects.

1. Aspects of Human memory
and Large Models
Author: Romuald A. Janik
Date: November 2023

2. Introduction:
● Overview of Large Language
Models (LLMs) and their
significance in text
generation and language
modeling.
● Purpose of the study:
Investigating the memory
properties of LLMs and their
similarities with human
memory.

3. Large Language Models
● Briefly explain what LLMs are and
their role in modern AI.
● Example: ChatGPT and GPT-J
models.
● Discuss the concept of memory in the
context of LLMs.

4. Memory
Human Memory Characteristics Probing Memory in LLMs
An infographic detailing key aspects of human
memory such as the memory process (encoding,
storage, retrieval), and types of memory (short-term,
long-term, sensory).
Visuals of the experimental setup or flowcharts
illustrating the methodology used to test memory in
LLMs.

5. Similarities
● Recall Patterns:
○ Both LLMs and human memory demonstrate patterns like the primacy and recency effects. This means both remember
the first and last items in a list better than the middle items.
● Effect of Repetition:
○ In both LLMs and humans, repeating information improves recall accuracy. This is akin to the learning process in
humans where repetition aids memory.
● Interference Effects:
○ Both human memory and LLMs show signs of interference. In humans, this is when new information can disrupt the
recall of old information. LLMs display a similar pattern when processing large amounts of data.
● Memory Decay:
○ Over time, both LLMs and human memory tend to forget information. This gradual loss of information is a common
characteristic of both types of memory.

6. Difference
● Biological Basis:
○ Human memory is rooted in biological processes within the brain, involving complex neural networks and chemical processes. LLMs, on the other hand,
rely on artificial neural networks and algorithms, lacking the biological aspect.
● Capacity and Speed:
○ LLMs typically have a higher capacity for data storage and can process information faster than human memory. Humans, however, have limitations in
terms of memory storage and processing speed.
● Contextual and Emotional Understanding:
○ Human memory is strongly influenced by emotions and context. Humans can remember information better if it has emotional significance. LLMs do not
have emotional responses and their understanding of context is based purely on data patterns.
● Adaptability and Flexibility:
○ Human memory is more adaptable and flexible. It can generalize and apply learned information to new, unfamiliar situations. LLMs, while capable of
learning and adapting to some extent, are more limited in applying learned information outside of their training parameters.
● Error Types:
○ The nature of errors differs. Humans might forget due to distractions, emotional states, or cognitive overload. LLMs' errors are more likely due to data
quality, algorithmic limitations, or biases in the training data.
● Conscious Recall:
○ Humans can consciously attempt to recall information, sometimes even struggling to remember. LLMs lack consciousness; their recall is purely a
computational process.

7. Conclusion
A summary graphic that encapsulates the key
findings of the study, possibly using a
combination of imagery and brief text points.