The document discusses the need for AI technology certification to increase trust and regulate AI systems. It proposes a two-level certification approach that evaluates whether an AI system performs as claimed and complies with ethical standards. The document also describes how to implement certification, including agreeing on certifiable AI system classes, standardizing performance metrics, and building prototypes to automatically verify systems. The goal is to take the first technical steps towards establishing an operational certification process for AI technologies.

1. Towards an operational
AI Technology Certification
Marko Grobelnik
(Marko.Grobelnik@ijs.si)
Artificial Intelligence Lab, Jozef Stefan Institute
Digital Champion of Slovenia at EC
UN Legal Week, UN Hq, New York, Oct 28th 2019

2. Three key questions: Why, What and How?
•Why AI Certification is needed?
•What AI Certification should include?
•How AI Certification could be implemented?

3. Why AI Certification is needed?
There seems to be a global consensus on AI is
influencing our personal and societal lives
• AI Technology is for most of the people a mystery
and creates fears
• AI Technology is for its creators also in many ways
a mystery, not really understanding all the
relevant consequences of its use
…therefore it is necessary to regulate AI Technology
to remove fears and to make it trustful

4. What AI Certification should include?
Two levels of certification needed:
(1) Is the AI System actually doing what its creators are claiming?
• …what kind of guarantees and uncertainties does it have?
• E.g., performance, robustness, transparency, and others
(2) Is the AI System compatible with a particular value standard?
• …like Ethical, Social and Legal norms, regulations, principles, criteria
• E.g., UN Human Rights Declaration, OECD AI Principles, EC Ethics
guidelines for trustworthy AI and others

5. How AI Certification could be implemented?
Where to start?
• We need an operational AI definition
• e.g., “AI System” as defined by OECD could serve as a good start
• Check how science developed standards to verify its claims
• …preventing scientists to lie and mislead themselves
• AI Systems need a standardized API backdoor for testing and monitoring
• …allowing an operational technical certification

6. Informal definition of AI
•AI is exactly the opposite from
what is happening in the video…
•…instead of living beings
mimicking machines, AI is
supposed to make machines
imitating living beings.

7. AI System
Environment State
• An environment has its state, not
necessarily fully observable
• An environment can change its
state with or without explicit
actions by the AI System
Environment State Observability
• Environments are real (physical,
social, mental) or artificial (e.g.,
games like chess)
• Real environments are typically
too complex to be observable in its
entirety (like physical or social)
• Artificial environments can be fully
observable (like chess)
• Environments are observable
through ‘percepts’ in a form of raw
data
Environment
Perceiving
(Percepts /
Raw Data)
Sensors: (§1)
• Machine (§2)
• Human (§3)
Actuators: (§13)
• Machine (§14)
• Human (§15) Acting
(Physical or
Informational
Influence)
Model (§8)
(e.g., rules or
analytical
function)
Model
Construction
Algorithm (§4)
(e.g., machine
learning)
Structured Data
Recommendations
Model
Interpretation
Algorithm (§10)
(e.g., classification
or logic reasoning)
Processed by Machine
Processed by Human
Objective(§5)
Human Interpretable
or Uninterpretable
Representation (§9)
Objective (§11)
Historicaldata/
memory(§7)
AI Operational Logic
Performance
Measure (§12)
Performance
Measure(§6)
AI System as defined by OECD

8. AI System
Environment State
• An environment has its state, not
necessarily fully observable
• An environment can change its
state with or without explicit
actions by the AI System
Environment State Observability
• Environments are real (physical,
social, mental) or artificial (e.g.,
games like chess)
• Real environments are typically
too complex to be observable in its
entirety (like physical or social)
• Artificial environments can be fully
observable (like chess)
• Environments are observable
through ‘percepts’ in a form of raw
data
Environment
Perceiving
(Percepts /
Raw Data)
Sensors: (§1)
• Machine (§2)
• Human (§3)
Actuators: (§13)
• Machine (§14)
• Human (§15) Acting
(Physical or
Informational
Influence)
Model (§8)
(e.g., rules or
analytical
function)
Model
Construction
Algorithm (§4)
(e.g., machine
learning)
Structured Data
Recommendations
Model
Interpretation
Algorithm (§10)
(e.g., classification
or logic reasoning)
Processed by Machine
Processed by Human
Objective(§5)
Human Interpretable
or Uninterpretable
Representation (§9)
Objective (§11)
Historicaldata/
memory(§7)
AI Operational Logic
Performance
Measure (§12)
Performance
Measure(§6)
AI System as defined by OECD

9. AI System
Environment State
• An environment has its state, not
necessarily fully observable
• An environment can change its
state with or without explicit
actions by the AI System
Environment State Observability
• Environments are real (physical,
social, mental) or artificial (e.g.,
games like chess)
• Real environments are typically
too complex to be observable in its
entirety (like physical or social)
• Artificial environments can be fully
observable (like chess)
• Environments are observable
through ‘percepts’ in a form of raw
data
Environment
Perceiving
(Percepts /
Raw Data)
Sensors: (§1)
• Machine (§2)
• Human (§3)
Actuators: (§13)
• Machine (§14)
• Human (§15) Acting
(Physical or
Informational
Influence)
Model (§8)
(e.g., rules or
analytical
function)
Model
Construction
Algorithm (§4)
(e.g., machine
learning)
Structured Data
Recommendations
Model
Interpretation
Algorithm (§10)
(e.g., classification
or logic reasoning)
Processed by Machine
Processed by Human
Objective(§5)
Human Interpretable
or Uninterpretable
Representation (§9)
Objective (§11)
Historicaldata/
memory(§7)
AI Operational Logic
Performance
Measure (§12)
Performance
Measure(§6)
AI System as defined by OECD

10. AI System
Environment State
• An environment has its state, not
necessarily fully observable
• An environment can change its
state with or without explicit
actions by the AI System
Environment State Observability
• Environments are real (physical,
social, mental) or artificial (e.g.,
games like chess)
• Real environments are typically
too complex to be observable in its
entirety (like physical or social)
• Artificial environments can be fully
observable (like chess)
• Environments are observable
through ‘percepts’ in a form of raw
data
Environment
Perceiving
(Percepts /
Raw Data)
Sensors: (§1)
• Machine (§2)
• Human (§3)
Actuators: (§13)
• Machine (§14)
• Human (§15) Acting
(Physical or
Informational
Influence)
Model (§8)
(e.g., rules or
analytical
function)
Model
Construction
Algorithm (§4)
(e.g., machine
learning)
Structured Data
Recommendations
Model
Interpretation
Algorithm (§10)
(e.g., classification
or logic reasoning)
Processed by Machine
Processed by Human
Objective(§5)
Human Interpretable
or Uninterpretable
Representation (§9)
Objective (§11)
Historicaldata/
memory(§7)
AI Operational Logic
Performance
Measure (§12)
Performance
Measure(§6)
AI System as defined by OECD

11. Accountability and
responsibility
Risk Management
Human-
centred values
Transparency Robustness
Building human capacity and
preparing for job transition
AI System and relation to higher level principles

12. Perception Bias
Activation Bias
Technical Bias
AI System and sources of various types of biases

13. Modes of certification of an AI System
• Methodological review (manual)
• …detecting possible flaws in construction of a system
• Offline testing review (semi automatic)
• …simulating real-live usage to test response under varying conditions
• Online monitoring review (automatic)
• …real-live observation to monitor behavior
• Review reports expressed in a form of measurable KPIs
• …many of the KPIs are already well established in scientific community
• …some KPIs might need further elaboration to measure relevant aspects (e.g., in
particular in relation to soft concepts like Human Rights)

14. …so, what are the first technical steps to
get an operational AI System certification
1. Agree on classes of AI Systems which are possible to certify
• …some types of systems might be beyond the scope of certification
2. Define and agree on hard and soft KPIs describing an AI System
• …some KPIs could be taken from research, some need development
3. Standardize backdoor API for testing and monitoring
• …enabler to observe AI Systems in action
4. Build a prototype certification system to allow (semi)automatic
verification of AI Systems
• …useful AI certification system could be constructed without a big investment
5. Get a series of success stories to gain trust on certification