This document discusses the implications of large language models (LLMs) for statistical offices, highlighting their architecture, training, and applications. It emphasizes the need for standardization and ethical considerations regarding AI use, especially concerning copyright and intellectual property issues. Various degrees of AI integration are presented, from assisted to autonomous AI, alongside recommendations for technology adoption and workforce reskilling.

1. An introduction to Large Language Models
and their relevance for Statistical Offices
Dario Buono, Ph.D.
Marius Felecan, MSE
Cristiano Tessitore, Ph.D.
An Eurostat AI paper by
https://ec.europa.eu/eurostat/product?code=KS-TC-24-001

3. Hallucinate

4. Part One
Framing the Context

5. GenAI Technology
Large Language Models whats and hows.

6. AI definition 2018
“Artificial intelligence (AI) systems are software (and possibly also hardware)
systems designed by humans that, given a complex goal, act in the physical or
digital dimension by perceiving their environment through data acquisition,
interpreting the collected structured or unstructured data,
reasoning on the knowledge, or processing the information,
derived from this data and
deciding the best action(s) to take to achieve the given goal.
AI systems can either use symbolic rules or learn a numeric model,
and they can also adapt their behaviour by analysing how
the environment is affected by their previous actions”
Independent High-Level Group on AI (hired by the European Commission), 2018

7. AI definition 2024
‘AI system‘ is a machine-based system designed to operate with
varying levels of autonomy and that may exhibit adaptiveness
after deployment and that, for explicit or implicit objectives,
infers, from the input it receives, how to generate outputs such
as predictions, content, recommendations, or decisions that can
influence physical or virtual environments”
Art. 3, EU AI Act

8. Large Language Models
Architecture
Domains
Fields
Transformers
NLP Self Driving
Tesla
LLMs:
Gemini
ChatGPT
Deep Learning
Machine Learning
Artificial Intelligence

9. Large Language Models - Training
LLM
(parameters)
Corpus

10. Large Language Models – Fine tuning
LLM
(parameters)

11. Large Language Models – Inference
Mary had a little
LLM lamb
Prompt (Context)

12. Charting LLMs
A Call for Standardization

13. Emerging & Disrupting
Emerging - new and innovative development that significantly alters the
current landscape of business and society.
Disrupting - groundbreaking product or service that fundamentally
changes the market or society.

14. Cars Prediction (1903)
“The automobile is a fad, a novelty. Horses are here to
stay.”
President of Michigan Savings Bank

15. … forecasts about disruptors
PC Prediction (mid-1970s)
“There is no reason an individual would ever want a computer in their home.”
Ken Olsen, founder of DEC
digital photography, mobile computing, smart phone …
Internet is now 35 years old!
still reinventing itself

16. Not mature: Still progress to be made.
Not well understood: Still the expectations are
sometimes unrealistic.
Emergent Disruptive
Huge potential
Chain reactions
How fast?
Hype vs. Reality

17. FOMO (fear of missing out)
FOBO (fear of a better option)
Fear of Being too Early
high risk, high reward

19. Now & Near Future
Some Strategies to Consider
Education and Awareness
Technology Adoption and Integration
Workforce Reskilling and Upskilling
Partnerships and Collaborations
Ethical AI Use and Governance

20. Jeff Bezos on AI: Large language models are
‘not inventions, they’re discoveries’
Source: Jeff Bezos - Amazon and Blue Origin, Lex Fridman Podcast, YouTube

21. Questions ?

22. Part Two
Use Cases

23. LLM4Statistics
Interesting concepts and applications

24. Degrees of AI integration
1. Assisted AI (Supportive): AI systems that enhance human tasks without replacing human decision-
making. Examples include analytical tools that provide insights for humans to interpret.
2. Augmented AI (Collaborative): These AI systems work alongside humans, enhancing their capabilities
through suggestions or automating routine parts of tasks. This is seen in applications like medical
diagnostics, where AI assists in analyzing data.
3. Automated AI (Independent): AI that operates fully autonomously, performing tasks without human
intervention. Common uses include robotic process automation for repetitive tasks such as data entry.
4. Autonomous AI (Self-sufficient): The most advanced form, these systems operate independently in
changing environments, making decisions without human input. Autonomous vehicles and intelligent
drones are key examples.

25. Multitool vs Screwdriver

26. LLM vs SLM
LLM SLM
Parameters Several billions (even
trillion)
Few billions (1 to 3)
Knowledge Wide To be specialized
Languages Several English
Fine tuning Expensive Cheap
Hardware specs High Low
Performances on
benchmark
High variable

27. Mixture of experts
INPUT Router
Expert 1
Expert 2
Expert 3
Expert 4
Expert 5
Additive
combination
OUTPUT
LMSYS Chatbot Arena Leaderboard as of 07/03/24

28. Training Extremely expensive
Fine tuning May be not that cheap
RAG = Retrieval-Augmented Generation
Solution for incorporating knowledge from external databases
Dynamic | Cost-effective | Modular
Training vs Fine tuning vs RAG systems

29. RAG systems
Source: NVIDIA

30. RAG systems
Source: Gao et al (2024)

31. Selected Use cases

32. GPT@JRC
Cloud + Hosted Models
Unified Graphical User Interface
API available
Internal Security and Privacy Rules Compliant
European Commission Joint Research Centre GPTs Platform

33. Coding

34. Improved efficiency
Our Use Case: Old, inherited codebase
Documenting and Commenting

35. Boilerplate code
Our recommendation: Use IDE integrated
plugins, mature and efficient
Comments: careful with prompts
Code Generation

36. Test Case Generation
Our experience: debugging works better for
not so experienced developers
Testing and Debugging

37. (Semi-)Automation
Teams of GPTs
Future developments

38. Research

39. Partner
Assistant
Critic (reviewer)
Personas

40. ChatGPT with plugins
Commercially available solutions
• Writefull - for academic and technical writing
• HeyGPT - Chat with PDFs
• Litmaps - Best Literature Search
• Jenni - Helps you write, edit, and cite with confidence.
In-house custom developed scripts
Literature review

41. (Semi-)Automation
Exploring big context models
Future developments

42. Model tuning
Going beyond prompting

43. EUBERT is a pretrained BERT uncased model that has been trained on a vast corpus of documents registered by the
European Publications Office. These documents span the last 30 years, providing a comprehensive dataset that encompasses
a wide range of topics and domains.
EU-BERT Model
Text Classification, Question Answering, Named entity recognition, part-of-speech tagging, text generation ...
https://huggingface.co/EuropeanParliament/EUBERT
Using the EuroHPC Meluxina cluster, a core component of Europe's high-performance computing landscape:
● Hardware Type: 4 x GPUs 24GB
● GPU Days: 16
Model size: 94M params

44. Questions ?

45. Part Three
IP and Ethics

46. • Generative AI tools are already pre-trained when available for use.
• Most of the main models have been trained vast amounts of data
available on the Internet, not always respecting copyright or intellectual
property.
• The output of the model can infringe copyright
• The proprietorship of the output of GenAI is an issue, as copyright
protection is only available for works created by human beings.
What’s different on GenAI

47. Copyright issue – text

48. Copyright issue - image

49. Practical scenarios
Intellectual Property
1. No intention to use the output in any document;
2. Intention to use the output in an internal document;
3. Intention to use the output in an external document.
Nature of the information
4. Public information;
5. Non-Public information;
Nature of the system
6. Public system;
7. Trusted-cloud provider;
8. Internal LLM.

50. • [Rule n°1] Staff must never share any information that is not already in
the public domain, nor personal data, with an online available generative
AI model.
• [Rule n°2] Staff should always critically assess any response produced
by an online available generative AI model for potential biases and
factually inaccurate information.
European Commission Guidelines

51. • [Rule n°3] Staff should always critically assess whether the outputs of an
online available generative AI model are not violating intellectual property
rights, in particular copyright of third parties.
• [Rule n°4] Staff shall never directly replicate the output of a generative
AI model in public documents, such as the creation of Commission texts,
notably legally binding ones.
• [Rule n°5] Staff should never rely on online available generative AI
models for critical and time-sensitive processes.
European Commission Guidelines

52. Ethics

53. It’s a wrap
Not a magic wand but a useful tool

54. To be continued…
https://cros.ec.europa.eu/dashboard/ntts-2025

55. Thank you
© European Union 2024
CREDITS:
Slide 3: Dictionary.com
Slide 19: Gartner, https://www.gartner.com/en/articles/what-s-new-in-artificial-intelligence-from-the-2023-gartner-hype-cycle
Slide 23: Jeff Bezos - Amazon and Blue Origin, Lex Fridman Podcast, YouTube
Slide 33: Gao et al (2024) Retrieval-Augmented Generation for Large Language Models: A Survey
Slide 53: https://www.theguardian.com/technology/2024/apr/16/techscape-ai-gadgest-humane-ai-pin-chatgpt
Unless otherwise noted the reuse of this presentation is authorised under the CC BY 4.0 license. For any use or reproduction of elements that are
not owned by the EU, permission may need to be sought directly from the respective right holders.