The presentation "ITDays_2023_GeorgeBara" discusses challenges in adopting AI large language models (LLMs) in enterprise settings. The presentation covers: 1. **Challenges in AI LLMs adoption**: It highlights the noise in the current AI landscape and questions the practical use of AI in real businesses. 2. **The DNA of an Enterprise**: Defines enterprise sizes and discusses the new solutions adoption process, emphasizing effective integration and minimizing disruption. 3. **Enterprise-Grade**: Lists qualities like robustness, reliability, scalability, performance, security, and support that are essential for enterprise-grade solutions. 4. **What are LLMs?**: Describes the pre-ChatGPT era with BERT, a model used for language understanding, and details its enterprise applications. 5. **LLM use-cases before ChatGPT**: Focuses on data triage, process automation, knowledge management, and the augmentation of business operations. 6. **EU Digital Decade Report**: Points out that AI adoption in Europe is slow and might not meet the 2030 targets. 7. **Adoption Challenges**: Addresses top challenges such as data security, predictability, performance, control, regulatory compliance, ethics, sustainability, and ROI. 8. **Conclusion**: Reflects on the slow adoption of AI in enterprises, suggesting that a surge might occur once the technology matures and is ready for enterprise use. The presenter concludes by stating that despite the hype around technologies like ChatGPT, enterprises are cautious and will adopt new technologies at their own pace. He anticipates a gradual then sudden adoption pattern once LLMs are proven to be enterprise-ready.

1. Challenges in AI LLMs
adoption in the Enterprise
George Bara
Chief Strategist @ Zetta Cloud

2. About me
• Nearly 20 years in ITC, 12 years in AI
(before it was knows as such!)
• RDBMS developer → Web developer
→ Presales Engineer → AI Solutions
Consultant → Business Development &
Partner Management www.zettacloud.ai
“Artificial Intelligence
Solutions for Deep Content
Understanding”

3. … and this presentation
You’ve probably had it with #ChatGPT and #GenAI and #LLMs
What do REAL businesses - ENTERPRISES - do with AI?
Everyone is an AI
expert
Top 10 ChatGPT
prompts for
anything
The media tells
you that AI will
take your job (or
even kill you)
LOTS of NOISE

4. The DNA of an Enterprise
• Small: 1 and 49 employees, annual turnover < 10M EUR
• Medium: 50– 249 employees, annual turnover < 50M EUR
• Large: over 250 employees, annual turnover > 50M EUR
New Solutions Adoption
• Effective Integration
• Minimize Disruption
• Mitigate Risk
• Maximize Benefits
Research & Awareness ↦ Needs Assessment ↦ Evaluation: BUILD or
BUY ↦ Selection ↦ PoC ↦ Business Case Development ↦
Stakeholder Buy-In ↦ Change Management ↦ Implementation ↦
Training & Support ↦ Continuous Monitoring & Optimization ↦
Scalability & Expansion ↦ Regular Updates & Upgrades ↦ Post-
Implementation Review.

5. Enterprise-Grade
Robustness Reliability Scalability Performance
Security Compliance
Support &
Service
IT Management
How many LLMs are Enterprise-Grade?

6. What are LLMs?
Before ChatGPT there was BERT (Bidirectional Encoder Representations from
Transformers), launched in Oct 2018 - 5 YEARS AGO!
> pretrained on unlabelled corpus for language modeling and next-sentence
prediction
> state of the art (still?) in various Natural Language Understanding tasks such
as entity recognition, sentiment analysis, classification and question answering.
There are countless implementations of LLMs in Enterprise before ChatGPT.

7. LLM use-cases for the Enterprise before ChatGPT
Data Triage
Organizing and extracting meaning
from large data sets of multilingual
unstructured data: PII identification
(compliance), data intelligence for
security & cybersecurity, open source
intelligence, reputation management,
competition monitoring.
Process Automation
Combining RPA & AI to achieve
“hyperautomation”: automating
business processes where text &
documents are involved, and where
humans need to read, understand,
summarize and take decision cognitive
tasks are replaced or complemented by
AI.
Knowledge Management
Knowledge bases become intelligent by
organizing
themselves through AI processing,
making interaction with business users
easier through natural conversations :
document management, customer
support, business operations.
Automate/Augment

8. “Goldman Sachs, nearly a year after ChatGPT
was released, put exactly zero generative AI
use cases into production. Instead, the company
is “deeply into experimentation” and has a “high
bar” of expectation before deployment.
[...]
But Goldman Sachs is also far from new to
implementing AI-driven tools — but is still
treading slowly and carefully.”
https://venturebeat.com/ai/goldman-sachs-cio-is-anxious-to-see-results-from-
genai-but-moving-carefully-the-ai-beat/

9. EU Digital Decade Report
https://digital-strategy.ec.europa.eu/en/library/2023-report-state-digital-decade
AI Take-Up in Europe is still slow:
11% from target.
2030 targets are not likely to be
met: 75% of Enterprises using AI.

10. TOP 7 ADOPTION CHALLENGES
(and how to address them)

11. Data Security: Challenge
Most productized LLM (ChatGPT, Bard) are
cloud-only solutions.
Chat history data can become part of the
model’s training set.
Most public sector and regulated
industry organizations run on private-
cloud and on-premise environments.

12. Predictability
• Predictable, consistent outputs.
• High-quality outputs.
• Handling hallucinations.
Robust, fit-for-purpose AI models
designed for very specific tasks
(Sentiment Analysis)
RAG - retrieval augmented
generation to improve prediction
quality
Confidence
Scores

13. Performance
• Current commercially available
public/cloud LLMs are still very slow.
• Impossible to adhere to business SLAs.
• Not fit for fast or large-volume
processing.
By comparison, a specialized engine built on
BERT (like Named Entity Recognition), can
reach 300,000 words per minute on
commodity hardware.
https://zettacloud.ai/throughput-benchmark-ai-factory-
engines-provide-unprecedented-speed-on-commodity-
hardware/

14. Control
In order to obtain the best output quality, the AI models require domain-specific
adaptation
Prompt Engineering
● Provide reference
text
● Split complex tasks
● Use External Tools
https://platform.openai.com/docs/guid
es/prompt-engineering/strategy-split-
complex-tasks-into-simpler-subtasks
RAG
Model Fine-Tunning
● Build & maintain relevant datasets
● Train ↦ Evaluate ↦ Deploy ↦Monitor
● Make it available to non-experts:
NO CODE Machine Learning

15. Regulatory & Compliance
Beyond the cloud SOC 2 and GDPR compliance, there is the
upcoming
EU AI ACT: European Parliament’s first regulation on
artificial intelligence
> AI classification on Risk.
> Mandatory for selling, buying or implementing AI in the EU.
> Audits on training data (copyright)

16. Ethics and Sustainability
Environmental, social, and governance (ESG) issues are important to most large
enterprises (whether we like it or not).
Adoption of AI solutions not only requires Business and IT buy-in, but might
require analysis on:
- CO2 impact, energy, water and other resources usage.
- Ethical use of training data, and ethical inference outputs.
- Culturally- aware AI systems.
- Designated use within ethical boundaries.

17. ROI
Return of Investment: Does the investment match the benefits?
> Investing in extensive IT infrastructure/service and expertise to solve low-
value, low-volume or trivial issues.
ROI = (Total Value Gained from
LLM - Total Cost of LLM)/
Total Cost of LLM
Cost Reduction
Revenue Growth
Improved Customer
Satisfaction
Risk Mitigation Innovation
Quality of
Insights

18. (My) Conclusion
No matter how exciting the technology, how big is the FOMO, Enterprises will
stick to their processes and will adopt new technologies at their own pace.
Adoption of (ChatGPT-style) LLMs is not as high as advertised; but most
enterprises are experimenting, even if no actual projects are yet in production.
Expecting slow adoption, then All-At-Once, once technology matures and
becomes enterprise-ready.

19. Thank you!
george@zettacloud.ai
www.linkedin.com/in/georgebara/