The document discusses artificial intelligence (AI) systems for categorizing articles and the importance of training data. It provides an example of a university building an AI system to categorize articles as related to AI or not and further categorize AI articles into subdomains. The system was trained on tens of thousands of manually tagged records to understand the categories. The document emphasizes that effective AI requires extensive training data and algorithms developed for small datasets, as subject matter experts typically work with small amounts of specialized data, not large public datasets. New techniques like few-shot learning, zero-shot learning, and transfer learning allow building AI that supports experts working with small, specialized datasets.

1. Copyright ©1997
- 20 21Search Technology, Inc. TheVantagePoint.com | 1
Nils Newman | October 5, 2021
AI
Who is in Control and
Why is that Important

2. Copyright ©1997
- 20 21Search Technology, Inc. TheVantagePoint.com | 2
The Magic of AI
• AI is often sold as a magical
tool
• Click a button - - - get an
answer
• Better yet – no clicking
required
• AI just does the work for you…

3. Copyright ©1997
- 20 21Search Technology, Inc. TheVantagePoint.com | 3
The Reality
• AI can work like magic
But…
..
• Behind the scenes, a lot
of work goes into
making these systems
work

4. Copyright ©1997
- 20 21Search Technology, Inc. TheVantagePoint.com | 4
An Example
• A university is tasked to build an AI system which reads articles and
automatically tags them as AI articles or not.
• The system then further tags the AI articles based on seven sub-
domains in AI.
• The team starts with Google BERT (Bidirectional Encoder
Representations from Transformers).
• Then moves to SciBERT – the BERT trained on scientific publications.
• Next they modify SciBERT by using tens of thousands of manually
human tagged AI records so their AI- SciBERT would “understand”
the categories of AI they were trying to find.

5. Copyright ©1997
- 20 21Search Technology, Inc. TheVantagePoint.com | 5
An Example: continued
• Next they build another AI model that
could adapt as terms changed over time
so the system would be robust in the
future.
• Finally, they run the model against millions
of publications from different sources.
• The system produces F- scores above 0.8
for most AI categories with no human
intervention.
• So…
. Magic!
Plus a big budget for manual tagging.

6. Copyright ©1997
- 20 21Search Technology, Inc. TheVantagePoint.com | 6
What people ignore about AI
• The training piece of AI, whether it is
unsupervised, semi- supervised, or
supervised, is the bit that most people
tend to gloss over when talking about AI.
• Training is also one of the biggest barriers
to the wider deployment of AI.
• It is one of the big reasons why people are
suspicious about AI. Things do not work
because the training is poor.
• Ignoring the training issue contributes to
the overpromise and underperformance
of AI.

7. Copyright ©1997
- 20 21Search Technology, Inc. TheVantagePoint.com | 7
The SME’s
• Areas where specialized
subject matter experts
contribute to knowledge work
have been particularly
problematic for AI.
• The problem is not that AI is
not useful.
• AI can be very useful to SME’s,
but previous generations of AI
have not addressed the SME’s
primary issue…
.

8. Copyright ©1997
- 20 21Search Technology, Inc. TheVantagePoint.com | 8
SCALE: Small Data
• SME’s typically operate on Small Data.
• A researcher at a chemical company
might use patent information that has
been categorized using Big Data AI
such as Patent Classifications or
Chemical Tagging.
• But for the impactful work, the SME
usually operates at the Small Data level
– searches of a few thousand records.
• At the Small Data level, the SME has to
be in control of the training process.

9. Copyright ©1997
- 20 21Search Technology, Inc. TheVantagePoint.com | 9
The problem
• Big Data algorithms do not work in a small
data world.
• For example, most Deep Learning
algorithms need training sets in the
10,000’s or 100,000’s records.
• These types of approaches simply will
never work for the SME who uses 100’s or
1,000’s of records.
• Big Data algorithms also mean the SME has
little control over the process.
• They just use the output and have little
control over training.

10. Copyright ©1997
- 20 21Search Technology, Inc. TheVantagePoint.com | 10
Some Learning Solutions…
• Fortunately, this recent round of AI
development has produced some tools
to allow developers to create effective
AI for the SME working with Small Data.
• Few- Shot Learning
 Deployed in VP’s Smart Trainer
• Zero- Shot Learning
 Deployed in VP’s Find Similar Records
• Transfer Learning
 Portable Knowledge Bases in VP

11. Copyright ©1997
- 20 21Search Technology, Inc. TheVantagePoint.com | 11
More Solutions…Composite AI
• In addition to new algorithms that
require only small training sets, the
latest round of AI has produced
improvements in how AI algorithms
work with each other to form
Composite AI.
• In the past, most AI systems relied on
only one approach.
• Now different AI systems can work
together to address different aspects of
a problem.
• For example, VantagePoint uses around
half a dozen different AI approaches.

12. Copyright ©1997
- 20 21Search Technology, Inc. TheVantagePoint.com | 12
The Possible Future: Generative AI
• Generative AI is AI that creates something new from
what it has already learned.
• Current models typically require very large training
sets so are currently of limited utility in a Small Data
world.
• But, we have experimented with using Generative AI
to write a summary abstract about a collection of
documents.
• This opens the possibility of using Generative AI to
create something approaching a proxy for
“conceptual understanding”.
• This is much work to do, but we see some
interesting potential for Generative AI in a Small Data
world.

13. Copyright ©1997
- 20 21Search Technology, Inc. TheVantagePoint.com | 13
AI, Small Data, and SME’s
• Many Big Data AI designers are trying to
replace humans.
• The work of SME’s is too important and
too “human” to be simply replaced by AI.
• AI can make the SME’s life easier and
allow them to be more effective in their
work.
• But the design of the AI systems in the
Small Data space has to start with the
human and the recognition of the value of
human expertise.
• Regardless of which AI approaches are
deployed, in the Small Data space, the
Subject Matter Expert has to be in control.

14. Copyright ©1997
- 20 21Search Technology, Inc. TheVantagePoint.com | 14
Questions?

15. Copyright ©1997
- 20 21Search Technology, Inc. TheVantagePoint.com | 15