@pieroleo
The Era of Artificial Intelligence
Lecture 4
Pietro Leo
IBM Italy Executive Architect and thought leader for Artificial Intelligence
Chief Scientist for IBM Italy Research & Business
IBM Academy of Technology Leadership
Member of ISO/SC42 Artificial Intelligence Standardization Committee
www.pieroleo.com

@pieroleo
Artificial Intelligence
has multiple forms: it is
augmenting human
decision making and is
opening to new forms
of collaboration
between humans and
machines

@pieroleo
TOOL
COACH
COLLABORATOR
ACTIVE
MEDIATOR
AI present and future forms
Number of Use Cases
Today
ASSISTANT

@pieroleo
AI present and future forms
TOOL
COACH
COLLABORATOR
ACTIVE
MEDIATOR
ASSISTANT
Today

@pieroleo
5
Kind of problems where Artificial Intelligence is generating a relevant
business impact

@pieroleo
IBM Shoebox Understands 16 words
zero, one, two, three, four, five, six, seven, eight,
nine, minus, plus, subtotal, total, false, and off.

@pieroleo
IBM Shoebox
Understands
16 words
’60
IBM Speech
Server Series -
Word Recognition
Error Rates -
About 45%-50%
’80
IBM Via Voice –
Error rates 10%-
15% voice
dependent
’00
’90
Understands
20000 words
Today
Error Rate
Around
5%-6%
As good as
humans
Speech
Recognition

@pieroleo
Source: https://arxiv.org/ftp/arxiv/papers/1608/1608.07323.pdf

@pieroleo
9
Source: IBM Research Food Recognition - https://www.ibm.com/blogs/research/2017/05/training-watson-see-whats-plate

@pieroleo
10
er - https://arxiv.org/pdf/1612.00563.pdf
“a blue boat is sitting on the side of a building” “a person holding a giraffe in their hand”
Rennie, Marcheret, Mroueh, Ross & Goel, “Self-Critical Sequence Training for Image Captioning.” CVPR 2017

@pieroleo
11

@pieroleo
AI present and future forms
TOOL
COACH
COLLABORATOR
ACTIVE
MEDIATOR
ASSISTANT
New Forms

@pieroleo
AI present and future forms
TOOL
COACH
COLLABORATOR
ACTIVE
MEDIATOR
ASSISTANT

@pieroleo
Gutman, Codella, Celebi, Helba, Marchetti, Mishra and Halpern, “Skin Lesion Analysis toward Melanoma Detection: A
Challenge”, Int. Symposium on Biomedial Imaging (ISBI) 2016
• Deep Learning for skin lesion image
analysis
• Trained on dermoscopic images of
melanoma and other skin cancers
• Automates analysis of images of
skin lesions
• Extracts clinical features
• Segments lesions
• Predicts disease
• Reports disease score
• Searches for similar lesions
Tool: Skin Lesion Image Analysis for Melanoma Detection

@pieroleo

@pieroleo
AI present and future forms
TOOL
COACH
ACTIVE
MEDIATOR
ASSISTANT
(hyper realistic)
COLLABORATOR

@pieroleo
Source: Soul Machine - https://www.youtube.com/watch?v=fNWjKtVWToc

@pieroleo
19

20
Artificial Interactions – Soul Machine
Soul Machine’s Greg Cross Presentation
https://www.facebook.com/bwnet.fans/videos/10154882117121837/
Soul Machines ‘digital
humans’ have own unique
personality and emotional
intelligence.
• creating a realistic
physiological model,
• the persona, and
• the self-learning from each
human interaction.
The IBM Watson platform
enables these digital humans to
learn an organization's corpus
of knowledge and Soul
Machines’ human computing
engine allows them to embody
the core values of the
organization they will represent.
Soul Machine Digital Humans:
https://www.youtube.com/watch?time_continue=2&v=AzPs7GfOkew
Baby 5.0 - https://www.youtube.com/watch?v=yzFW4-dvFDA

@pieroleo
21
The human face
of Artificial
Intelligence
Air New Zealand Project: https://www.ibm.com/blogs/ibm-anz/digital-humans/
Soul Machines used neural
networks and biological brain
models to bring Sophie to life,
powered by a cloud based
Human Computing Engine.

@pieroleo
@pieroleo
www.pieroleo.com

@pieroleo
AI present and future forms
TOOL
COACH
ACTIVE
MEDIATOR
ASSISTANT
COLLABORATOR

@pieroleo
Personalized diabetes mobile companion
Guardian Connect Cognitive Computing Sugar.IQ
• Real-time, smartphone CGM
with alerts
• Insulin, Meal, Activity, Context
• Standalone, for MDI patients
• Watson Health Cloud & Analytics
• Pattern recognition, insights &
Predictions
• Engagement & Gamification
• Real-time insights, coaching
platform
• Assists in daily diabetes mgmt
• Aggressive capabilities
roadmap
Insights Glucose
Forecasts
Hype-hyper
Predictions Ask
Watson

@pieroleo
Collaborating to have an healthy regime

@pieroleo
Technical R&D today: Disruption opportunity
New
Product
New
Product
Opportunistic
Discovery
by Humans
Simulation
Experiments
Simulation & Inference
Experiments
Comprehensive
Discovery by
Cognitive
Today Cognitive Discovery

@pieroleo
- the car doesn’t need to drive at all, unless this
is part of the story;
- the driving should be peripheral to the story;
- characters in the story should have an
emotional designator, for example a husband
or father over driver or engineer;
- and the use of children helps increase the
emotionality of an advert.
- Additionally, strong facial expressions are
more powerful than strong language;
- ads are most effective where use of the
spoken word is limited;
- use of a midpoint or twist is important to keep
the story moving and to maintain interest;
- and the midpoint should involve an
unexpected event, for example a crash or
near miss.”
AI review of 15 years of
Cannes Lion-winning
ads was the backbone
of the creative process,
it was supplemented by
several other injections
of data and insight.
Emotional intelligence
data from Unruly
helped the machine
learning process
understand which parts
of ads sparked
responses from
viewers.
Creating an Intuitive Car Ad collaborating with AI
Human
Discovery
DATA AI GENERATED SCRIPT HUMAN

@pieroleo

@pieroleo

@pieroleo

@pieroleo

@pieroleo

@pieroleo

@pieroleo

@pieroleo

@pieroleo

@pieroleo

@pieroleo

@pieroleo

@pieroleo

@pieroleo

@pieroleo

@pieroleo
AI Learns the Art of Debate
Source: http://www.research.ibm.com/artificial-intelligence/project-debater/
• Detecting claims in relevant
documents
• Detecting evidence in relevant
documents
• Negating claims
• Synthesizing novel claims
• Assessing argumentation quality
• Relating arguments across texts
• Determining expert opinion stance
• Classifying sentiment of phrases
• Classifying sentiment of idioms
• Understanding Automatic Speech
Recognition
• Predicting phrase breaks
• Emphasizing words and phrases
• Determining concept abstractness
• Identifying related documents
• Detecting argumentative
structures
Challenge: “We should subsidize
space exploration”
Solution Pipeline

@pieroleo
McKinsey estimates
that 23% of legal work
can be automated,
there are many other
aspects of a lawyer’s
working day, like
briefing clients and
appearing in court, that
are beyond the
capabilities of
algorithms, at least for
now.
https://www.lawgeex.com/

@pieroleo
AI present and future forms
TOOL
COACH
COLLABORATOR
ACTIVE
MEDIATOR
ASSISTANT

@pieroleo
ViTA Advisor: it is a
conversational multi-
modal agent to
support older as well
as a tool to collect
meaningful data
about the context of
an individual
ViTA : Virtual Trainer for cognitive impaired patients
Sustain
Independence
and Dignity with
affect and
purpose,
preserve and
reinforce
individuals and
social memories
Vita Memory
Coach: a system
that supports
caregivers to collect
meaningful facts and
memories of an
individual and his
context
Vita
Memories
Leo, D’Onofrio, Sancarlo, Ricciardi, De Petris, Giuliani, Peschiera, Failla, Renzi and Greco, “ViTA:
Virtual Trainer for Aging”- FAAL 2017

@pieroleo
Coaching our memory and triggering affective states
3/9/19Tales

@pieroleo
AI present and future forms
TOOL
COACH
COLLABORATOR
ACTIVE
MEDIATOR
ASSISTANT

@pieroleo
Sense& OrganizeData
& Information
Proposedecisions for us and
our businesses
Big Data
Decision
Lakes
Product & Services that
Proactively mediateusers
Internet
Web/Mobile
Data
Cloud / IoT
Analytics
Artificial Intelligence /Machine
Learning
Natural Interfaces
Multimodal Reasoning
• ProactiveSupportHealthy
decisions
• AugmentProblem Solving and
learning
• ActiveHomes
• ResilientWorkEnvironments&
Manufacturing
Active Intelligence
Active Intelligence

@pieroleo
Source: IBM & Marchesa Cognitive Dress https://www.ibm.com/blogs/internet-of-things/cognitive-marchesa-dress/
& https://www.ibm.com/watson/stories/dress.html
How you perceive see yourself How others see you
A Cognitive Dress mediates you

@pieroleo

@pieroleo
Outright Bans Informing Citizens Active Vehicles

@pieroleo
Source: Shaun Sweeney and others
https://arxiv.org/pdf/1706.00646.pdf
IBM Research & University College Dublin
Cyber-
physics,
pollution
mitigation,
and electric
bikes
A cyber-physical system that mitigates the effect
of urban pollution by indirectly controlling the
minute ventilation (volume of air inhaled per
minute) of cyclists in polluted areas.
Active e-bike

@pieroleo
https://www.wired.com/story/the-wide-eyed-robot/
“Ready to
learn”
Simulate the
natural
interaction
between
baby and
mother or
father.
Active Learning

@pieroleo
Two Pacific Place in downtown Hong Kong
Cyber-physics: AI and ML to Reduce
Energy Use in Cooling Systems
Electricity consumption incurred on
average 30% lower than the
current mode of operation in the
building (green curve).
• Season
• Age of chiller
• Weather Condition
• Outdoor
Temperature
• Model Type
• Building
• Operating Power
• Water Mass Flow
Rate
• Water Temperature
Difference
• Latest Cooling
Load
Features
Source: https://www.ibm.com/blogs/research/2018/07/reduce-energy-
cooling/
Active Building

@pieroleo
Thanks!
Pietro Leo
IBM Italy Executive Architect and thought leader for Artificial Intelligence
Chief Scientist for IBM Italy Research & Business
IBM Academy of Technology Leadership
Member of ISO/SC42 Artificial Intelligence Standardization Committee
www.pieroleo.com