Tutorial helsinki 20180313 v1

Jim Spohrer (IBM)
IBM Finland HQ, Tuesday March 13 2018
http://www.slideshare.net/spohrer/tutorial-Helsinki-20180313-v1
3/13/2018 1
Tutorial: Opentech AI Helsinki

Welcome: Opentech AI Tutorial
3/13/2018
© IBM MAP COG2018
2
Jim SpohrerDaniel Pakkala
Silicon Valley Collaboration
at IBM Research - Almaden
(March 09, 2018)

Tutorial
• Welcome to Opentech AI (Jim & Daniel)
• Opentech AI Architecture (Daniel)
• Break
• Opentech AI Future Trends (Jim)
• 17:00 end
• 17:30 Welcome Reception and Posters
3/13/2018
© IBM MAP COG2018
3

Every 20 years, compute costs are down
by 1000x
• Cost of Digital Workers
– Moore’s Law can be thought of as
lowering costs by a factor of a…
• Thousand times lower
in 20 years
• Million times lower
in 40 years
• Billion times lower
in 60 years
• Smarter Tools (Terascale)
– Terascale (2017) = $3K
– Terascale (2020) = ~$1K
• Narrow Worker (Petascale)
– Recognition (Fast)
– Petascale (2040) = ~$1K
• Broad Worker (Exascale)
– Reasoning (Slow)
– Exascale (2060) = ~$1K
43/13/2018 (c) IBM 2017, Cognitive Opentech Group
2080204020001960
$1K
$1M
$1B
$1T
206020201980
+/- 10 years
$1
Person Average
Annual Salary
(Living Income)
Super Computer
Cost
Mainframe Cost
Smartphone Cost
T
P
E
T P E
AI Progress on Open Leaderboards
Benchmark Roadmap to solve AI/IA

GDP/Employee
3/13/2018 (c) IBM 2017, Cognitive Opentech Group 5
(Source)
Lower compute costs translate into increasing productivity and GDP/employees for nations
Increasing productivity and GDP/employees should translate into wealthier citizens

Measuring AI Progress (MAP)
AI Progress on Open Leaderboards - Benchmark Roadmap
Perceive World Develop Cognition Build Relationships Fill Roles
Pattern
recognition
Video
understanding
Memory Reasoning Social
interactions
Fluent
conversation
Assistant &
Collaborator
Coach &
Mediator
Speech Actions Declarative Deduction Scripts Speech Acts Tasks Institutions
Chime Thumos SQuAD SAT ROC Story ConvAI
Images Context Episodic Induction Plans Intentions Summarizatio
n
Values
ImageNet VQA DSTC RALI General-AI
Translation Narration Dynamic Abductive Goals Cultures Debate Negotiation
WMT DeepVideo Alexa Prize ICCMA AT
Learning from Labeled Training Data and Searching (Optimization)
Learning by Watching and Reading (Education)
Learning by Doing and being Responsible (Exploration)
2015 2018 2021 2024 2027 2030 2033 2036
Which experts would be really surprised if it takes less time… and which experts really surprised if it takes longer?
Approx.
Year
Human
Level ->

Build: 10 million minutes of experience
3/13/2018 Understanding Cognitive Systems 8

Types: Progression of models and
capabilities
Task & World Model/
Planning & Decisions
Self Model/
Capacity & Limits
User Model/
Episodic Memory
Institutions Model/
Trust & Social Acts
Tool + - - -
Assistant ++ + - -
Collaborator +++ ++ + -
Coach ++++ +++ ++ +
Mediator +++++ ++++ +++ ++
Cognitive
Tool
Cognitive
Assistant
Cognitive
Collaborator
Cognitive
Coach
Cognitive
Mediator

3/13/2018
© IBM 2015, IBM Upward University Programs Worldwide
accelerating regional development
11

Opentech AI -
Architecture, Ecosystem
and Roadmap:
Drafting the big picture and future directions of open artificial
intelligence technology13.3.2018 Helsinki, Opentech AI Workshop Tutorials
Daniel Pakkala,
IBM Distinguished Visiting Researcher - IBM Research Almaden.
Principal Scientist - VTT Technical Research Centre of Finland Ltd.
E-mail: Daniel.Pakkala@vtt.fi
LinkedIn: www.linkedin.com/in/dpakkala

Outline0. Introduction
1. Walkthrough on Opentech AI
– Applications & impact
– Process & tools
– Ecosystem & learning
– Platforms & technologies
2. Drafting the big picture of AI
– Is AI more than ML/DL?
– AI Beyond ML/DL
– AI amongst sciences
– Takeaways from the AI related sciences jungle
– AI systems today and in the future
3. A big picture of AI
 Intelligence evolution
 “AI Cube” framework
 No architecture framework and reference
architecture for AI systems – do we need one?
 Towards architecture framework for AI
systems
4. Wrapping it up…
 Catalysing and measuring progress of AI
 AI Architecture, Roadmap and Benchmarks
 Balancing theory and practice:
a deep learning experiment

Introduction (1/2)
• Research exchange co-operation between VTT Technical Research Centre of
Finland and IBM Research – Almaden.
• Co-created blog:
– http://opentechai.blog/

Introduction (2/2)
• Open Source
– Tools
– Platforms
– Code
• Open Datasets
• Open Models
• Open Challenges & Leaderboards
 Science & Education:
– Reproducibility, Progress, Training
 Industries & Society:
– Applications, Impact, Skills
Why?

Walkthrough on Opentech AI
- Applications & impact
- Process & tools
- Ecosystem & learning
- Platforms & technologies

Applications and impact
• Artificial Intelligence ~ intelligence exhibited by
machines
– Wide application potential on different fields of
society and industry
– One technology (in addition to many other
technologies) to solve real problems in real world
• Opentech AI is highly relevant both for companies
and academia
– Rapidly evolving open source platforms, tools,
data sets and community development
– Hard to match the speed of Opentech AI
development with a pure “in-house” AI
technology stack and R&D practices
• Industry specific open AI challenges are emerging
– Tool to measure progress of AI in industry specific
tasks
https://opentechai.blog/applications/

Process and tools
• R&D of Opentech AI:
– no single right way to do it
– plenty of tools available to utilize
• General 3 step learning and discovery process, and
supporting tools, can be identified for Opentech AI R&D
1. READ the state of the art – know and learn from what
others have done in the area (e.g. from arXiv). Check the
performance of the existing solutions to benchmark your
results on leaderboards.
2. REDO an experiment to learn or to improve it – retrieve the
open source code and data sets (e.g. from GitHub) and by
running/reusing/improving those.
3. REPORT the outcome and results of your experiment -
(using e.g. Jupyter notebook). Share modified code as open
source and publish your results also in form of a research
paper (e.g. in arXiv). If you have made an advancement that
can be reported on an AI leaderboard, that is especially
significant to advance AI progress.
https://opentechai.blog/process-tools/

Ecosystem and learning• R&D ecosystem around Opentech AI systems
requires
– open governance by multiple organizations working
to unlock the benefits of AI while mitigating any risks,
or downsides.
– variety of skills, making also learning resources and
their availability an important part of Opentech AI.
– vibrant open technology provider, developer,
contributor and user community
– commercialization friendly licenses
• Ecosystem has emerged and is evolving rapidly
• AI technology development pace is challenging the
education systems
• On-line courses and mini-degrees gaining popularity
in response to rapid skill development needs
– Evolution pace of Opentech AI technologies exceeding
capabilities of the education systems
https://opentechai.blog/ecosystem/

Platforms and technologies
• Growing variety of available open source technologies for
planning and implementing experiments in Opentech AI
R&D, e.g.
– TensorFlow – machine learning framework
– Keras – higher level API for machine learning
– PyTorch – deep learning framework for Python
– Anaconda – Python and R distribution for data science
– Microsoft Cognitive toolkit (CNTK) – ML/DL libraries
• Also commercial MLaaS/PaaS offerings with free trial
possibility
– IBM Watson offering
– Amazon AI Stack offering
– Microsoft Azure ML offering
– Google Cloud ML offering
https://opentechai.blog/platforms-technologies/

Drafting the big picture of AI
- Is AI more than ML/DL?
- AI Beyond ML/DL
- AI amongst sciences
- Takeaways from the AI related sciences jungle
- AI systems today and in the future

Is AI more than Machine/Deep
Learning?• Mainstream of AI R&D today: ML/DL
– Narrow AI applications are reaching or
outperforming human capabilities:
• Jeopardy! (IBM Watson, 2011)
• Go (Google AlphaGo, 2016)
• ”Reading comprehension”/Searching
Wikipedia for answers (SQuAD, 2018)
• What more is needed?:
– Address the real (open) world problems
and tasks!
• common sense & world model
• learning concepts (in addition to
categorizing & mapping),
• multiple/adaptive learning strategies and
• context-awareness, adaptation and
natural interaction
blog: https://opentechai.blog/2017/09/18/ai-is-more-than-machinedeep-learning/

AI beyond machine and deep
learning• The current state and progress of AI
– AI is matching human (task) performance in conceptually
narrow tasks and the quest is towards a wider AI, perhaps
one day matching the general intellectual capability of
humans.
• Beyond ML/DL and ICAT the literature related to AI
“explodes”, and comes from many different disciplines
with different approaches and methodologies
– Fuzzy/chaotic landscape from AI system engineering
viewpoint
• No architecture framework or reference architecture

AI amongst other sciences
 Core:
 Systems science, Computer science
and Mathematics
 Interfaces:
1. AI implementing cognitive architectures within intelligent
agents, multi-agent systems and other intelligent systems for
various applications.
1.1 Artificial neurons and artificial neural networks
1.2 Natural language processing, understanding and
generation. Word vectors.
2. Intelligence is a natural (biological) phenomena, which AI aims
to mimic. Potential new findings related to evolution, cells,
neurons and brain.
3. Availability and applicability of physical materials and energy
for building and operating any construct. Physical environment
and physical laws highly relevant for AI applications in the
physical word.
4. Acceptance, impact, need and use of AI applications by
individuals and organizations. Value of the AI applications is
defined by humans and human organizations in social context.
blog: https://opentechai.blog/2017/11/10/navigating-in-the-ai-sciences-jungle/

Takeaways from the AI
related sciences jungle
1. Progress of AI exploitation is dependent on the progress of R&D on
the related fields
– still active fields of research with potential for new discoveries relevant
for AI R&D.
2. Multiple parallel tracks of AI research and development are
proceeding in parallel
– not necessarily mutually inter-operable from the viewpoint of building
new AI systems and applications.
• ML/DL - Artificial Neural Networks
– “data classification/segmentation engines”
• Cognitive Science - Cognitive Architectures
– “data based processes”
• Symbolic, sub-symbolic, hybrid approaches
84 / 49 active…
FF, RNNs, Spiking,…~25
~1042…complexity

AI systems today and in the future
• AI systems today: Mainly evolution of
smart features of existing systems
– enabled by machine learning, increasing
computing power and availability of data.
• AI systems in future?: Systems
autonomously maintaining themselves,
operating, learning and interacting over
extended periods as part of society and
culture
– Forbus, K. D. (2016). Software social
organisms: implications for measuring AI
progress. AI Magazine, 37(1), 85-90.
~10∞

A big picture of AI
- Intelligence evolution
- AI Cube
- No architecture framework for AI systems – do we need
one?
- Towards architecture framework for AI systems

Intelligence evolution (1/2)• Co-evolution of Natural and Artificial Intelligence
– AI is a product of natural intelligence
– We share time, environment and context
• Time, where to start and how to approach different time scales?
– BB -13.7 Ga, Earth -4.5 Ga, LUA -3.7 Ga or Dartmouth 1956?
– It took about 3.7 billion (109) years for natural intelligence to evolve
and come up with artificial intelligence in 1956!
– Borrowed time concept from Biology:
• Evolutional and generational time dimensions
• Environment
– all life, materials, entities and artifacts
– all biological, physical, social and artificial processes and systems
– in constant change as a result of all the processes and systems
interacting within it

Intelligence evolution (2/2)
• Context
– subset of the environment at given time
– addressing an AI system (or family of AI systems) co-existence and interaction
with the relevant processes, systems and entities in the environment
– enables value, requirements and impact analysis of AI systems in different
phases of the system life-cycle
 Aspects of interest regarding AI
systems
 AI Research & Development,
 AI Craft/Production and
 AI System Implementations
 AI Genome
 AI Genome: codes for co-evolution of
human and artificial intelligence over
generations of both

A big picture of AI – “AI Cube”
~10∞
~1042
~109
see: https://opentechai.blog/2017/12/08/framing-the-big-picture-and-evolution-of-ai/

No architecture framework for AI
systems – do we need one? (1/2)
• Without architecture framework there will be very little knowledge
accumulation on structure and behavior of AI systems
– Encoded only in individual system implementations and their descriptions
Architecture framework:
- Enables architecture descriptions
- Modeliing conventions & practices
- Stakeholders & concerns
- Viewpoints

No architecture framework for AI
systems – do we need one? (2/2)• Agile manifesto and it’s naïve interpretation have hindered progress on
system and software architectures
• Time to rethink prevalent assumptions on architecture as we move towards
autonomous, intelligent cyber-physical systems!!!
– If not: “Who let the dogs out?”
• ISO/IEC/IEEE 42010:2011 Systems and software engineering - Architecture
description
– Sets requirements for defining an architecture framework and architecture
descriptions for systems and software engineering
– Challenge: Agile processes leading to flexible architecture
See, https://philippe.kruchten.com/2013/12/11/agile-architecture/
See, http://wasp-sweden.org/custom/uploads/2017/03/01_Introduction-to-Software-Architecture-for-Autonomous-Systems-
Pellicione.pdf (Picture source)

Towards architecture framework for
AI systems (1/2)
• The transition from current AI systems towards
future AI systems deserves special attention from
the viewpoint of systems architecture:
– Complex autonomic behavior of a system
• safety, security, trust, governance and institutional
control
– Incremental learning and adaptive behavior
• learning from data and past interaction outcomes,
capabilities/knowledge
– System life-cycle and maintainability
• applicable methodologies and processes, quality
– Social, cultural, ethical and environmental
compliance to norms
• acceptance and value
– Interaction and value
• collaborating vs. using
Laird, J. E., Lebiere, C., & Rosenbloom, P. S. (2017). A
Standard Model of the Mind: Toward a Common
Computational Framework Across Artificial Intelligence,
Cognitive Science, Neuroscience, and Robotics. AI
Magazine, 38(4).
Kotseruba, I., Tsotsos, J. K. (2016). A Review of 40 Years
of Cognitive Architecture Research: Core Cognitive
Abilities and Practical Applications, eprint
arXiv:1610.08602.
blog: https://opentechai.blog/2018/01/17/towards-an-architecture-framework-for-opentech-ai-systems/

AI systems
(2/2)
Stakeholders:
• R&D Community, System stakeholders and System Collaborators
(~users)
• Environment as a stakeholder – env.&soc. responsibility
Subsystems:
• Embodiment – physical material structure enabling
implementation of other subsystems and bounding interfaces with
the natural environment.
• Perception and Actuation – all mechanisms related to sensing and
actuation of the system with the environment
• Memory – all different memory mechanisms
• Presence – all spatiotemporal and cyber-temporal presence and
system behavior related mechanisms
• Cognition –all higher cognition related mechanisms accumulating
the internal world model of the system, and possibly adapting the
behavior of the AI System over time.
• Internal world model – The natural environment including human
culture as the AI system comprehends it internally. All knowledge
contained by the AI System as a result of training and learning
while in operation.
• Internal data processing –The internal continuous data processing
and analysis pipeline of the AI System defining the mechanism
converting data flow from perception subsystem into information,
knowledge and wisdom as modifications of the internal world
model and behavior.
Hybrid model driven approach combining agent-orientation and holistic two-
dimensional system orientation, where system thinking is applied both to the
external environment and to the internal organization of the AI system.
https://opentechai.blog/2018/01/17/towards-an-architecture-framework-for-opentech-ai-systems/

Wrapping it up…
- Catalysing and measuring progress of AI
- AI architecture, roadmap and benchmarks
- Balancing theory and practice:

Catalysing and measuring progress
of AI
• Architecture framework might catalyze AI systems development
– Breaking down the complexity involved in AI systems engineering
– Analysis and design framework for new AI systems
– Enabling hybrid/mash-up approaches in applications, when identifying synergies
between different AI systems becomes easier
• Measuring progress of AI
– AI Challenges and Leaderboards
• see: https://opentechai.blog/progress/
– More holistic view to AI progress, performance also in terms of:
• Energy
– Matching human energy consumption and environmental footprint in a task (in addition to just task
outcome performance)
» Brain simulation with current supercomputers: ~20 GW
» Human brain: ~20 W
– New metrics to be considered
» E.g. grams of CO2/task – flops/task – Wh/task - …

AI architecture, roadmap and
benchmarks

Balancing Theory and Practice (1/3):
• Predicting power consumption of a property/building as a
function of weather forecast
• Data
– 2012-2015 hourly power consumption and outside
temperature time series (32784 data points)
• Approach
– Multi-layered dynamic RNN-LSTM network
• # of layers
• # of LSMT units
• feature engineering (3 - 7 input
variables, sliding history windows, …)
• Implementation
– Python 3, TensorFlow 1.2.1, numpy, scikit, matplotlib
Figures a & b: arXiv:1610.09460
a)
b)

• Process – 3Rs
– READ
• Read a few publications on the topic
comparing different ML/DL approaches to
building energy prediction.
• Selected approach for the experiment (LSTM-
RNN).
– REDO
• Did dome LSTM sine wave prediction
examples from GitHub
• Code to be made available to GitHub after
“cleaning and polishing”
– REPORT
• Kind of doing it now
• Let’s see if there are results worth publishing
as a research paper
• Maybe a Jupyter notebook at some point

• Preliminary results:
– cumulative prediction error (%):
• 24h ahead: ~30% -> 70% accuracy achievable?
• 48h ahead: ~20% -> 80% accuracy achievable?
• What did I learn?
– Basic process and building a deep learning pipeline
– Python 3, TensorFlow, Spyder, numpy, scikit, matplotlib,
setting up Linux virtual environments, …
– To appreciate (remotely accessible) computing power
(CPU+GPU)
• Thanks IBM & VTT!
– The border between science and alchemy is fuzzy in field
of deep learning
• Hypothesis first and then experiment, or
• Experiment and try to explain results and figure out
why something works or doesn’t

Thank you!
Questions
or
Comments?

IBM: How many know IBM is very
active in open source communities?
3/13/2018
© IBM MAP COG2018
43
OPEN CONTAINER
PROJECT
Fact: An award program exists for open source, as well as patents.

GitHub: How many have an account?
3/13/2018
© IBM MAP COG2018
44
Technical Eminence: Explore, Read, Pull Request, Contributor, Committer, Governance

Kaggle: How many know about
leaderboards?
3/13/2018
© IBM MAP COG2018
45

3/13/2018
© IBM 2015, IBM Upward University
Programs Worldwide accelerating regional
development
46
Cognitive Mediators
for all people in all roles

Occupations = Many Tasks
3/13/2018
development
47

Watson Discovery Advisor
3/13/2018
development
48
Simonite, T. 2014. Software Mines Science Papers to Make New Discoveries. MIT. November 25, 2014.
URL: http://m.technologyreview.com/news/520461/software-mines-science-papers-to-make-new-discoveries/

AI Trends
3/13/2018
© IBM Cognitive Opentech Group (COG)
49
Dota 2
“Deep Learning” for
“AI Pattern Recognition”
depends on massive
amounts of “labeled data”
and computing power
available since ~2012;
Labeled data is simply
input and output pairs,
such as a sound and word,
or image and word, or
English sentence and French
sentence, or road scene
and car control settings –
labeled data means having
both input and output data
in massive quantities.
For example, 100K images
of skin, half with skin
cancer and half without to
learn to recognize presence
of skin cancer.

TED Arai Todai Robot
… when will
your smartphone
be smart enough to
pass a university
entrance exam?

Other Technologies: Bigger impact?
Yes.
• Augmented Reality (AR)/
Virtual Reality (VR)
– Game worlds
grow-up
• Blockchain/
Security Systems
– Trust and security
immutable
• Advanced Materials/
Energy Systems
– Manufacturing as cheap,
local recycling service
(utility fog, artificial leaf, etc.)

Industries Transformed
Digital Natives Transportation Water Manufacturing
Energy Construction ICT Retail
Finance Healthcare Education Government
“The best way to predict the future is to inspire the next generation of students to build it better”

3/13/2018 53
1955 1975 1995 2015 2035 2055
Better Building Blocks

In Sum: Prepare by…
• Participating
– Opentech AI (GitHub)
– Leaderboard Challenges (Kaggle)
• Making
– Smartphone apps become low-cost digital workers
(expertise economy)
– Return of mini-local factories and farms (manufacturing
and agricultural economy)
• Learning
– Service science and knowledge science study the evolution
– As well as influence the evolution of industries and
professions
3/13/2018
© IBM MAP COG2018
54

Jim Spohrer (IBM)
For C Mohan
Wednesday February 21, 2018
http://www.slideshare.net/spohrer/nsf-20180124-v18
3/13/2018 55
The Future of AI:
Measuring Progress and Preparing

Jim Spohrer & Opentech AI group?
3/13/2018
© IBM UPWard 2016
56
IBM Products and Services to Customers
IBM Research
(Patents Science
Eminence &
Differentiation)
Open Technologies
(Open Source
Technical Eminence &
Industry Standards)
Cognitive Opentech Group (COG) = Open Source AI + Data

3/13/2018
© IBM UPWard 2016
57
AI (Artificial Intelligence) is popular again… you see it mentioned on billboards in SF
However, pattern recognition does not equal AI
Deep learning works if you have lots of data and compute power
We finally have lots of data and compute power – hurray!!!
So finally, deep learning for pattern recognition is working pretty well
However, AI is more than deep learning for pattern recognition…
AI requires commonsense reasoning – that will take another 5-10 years of research
How do we know this? Look at the AI leaderboards – we will get to that…

IBM Research, Patents, Data, Cognitive
3/13/2018
© IBM UPWard 2016
58

TED Arai Todai Robot
… when will
your smartphone
be smart enough to
pass a university
entrance exam?

Questions
• What is the timeline for solving AI and IA?
• Who are the leaders driving AI progress?
• What will the biggest benefits from AI be?
• What are the biggest risks associated with AI, and
are they real?
• What other technologies may have a bigger
impact than AI?
• What are the implications for stakeholders?
• How should we prepare to get the benefits and
avoid the risks?

Every 20 years, compute costs are down
by 1000x
• Cost of Digital Workers
– Moore’s Law can be thought of as
lowering costs by a factor of a…
• Thousand times lower
in 20 years
• Million times lower
in 40 years
• Billion times lower
in 60 years
• Smarter Tools (Terascale)
– Terascale (2017) = $3K
– Terascale (2020) = ~$1K
• Narrow Worker (Petascale)
– Recognition (Fast)
– Petascale (2040) = ~$1K
• Broad Worker (Exascale)
– Reasoning (Slow)
– Exascale (2060) = ~$1K
2080204020001960
$1K
$1M
$1B
$1T
206020201980
+/- 10 years
$1
Person Average
Annual Salary
(Living Income)
Super Computer
Cost
Mainframe Cost
Smartphone Cost
T
P
E
T P E

GDP/Employee
(Source)
Lower compute costs translate into increasing productivity and GDP/employees for nations
Increasing productivity and GDP/employees should translate into wealthier citizens

Leaderboards Framework
AI Progress on Open Leaderboards - Benchmark Roadmap
Perceive World Develop Cognition Build Relationships Fill Roles
Pattern
recognition
Video
understanding
Memory Reasoning Social
interactions
Fluent
conversation
Assistant &
Collaborator
Coach &
Mediator
Speech Actions Declarative Deduction Scripts Speech Acts Tasks Institutions
Chime Thumos SQuAD SAT ROC Story ConvAI
Images Context Episodic Induction Plans Intentions Summarizatio
n
Values
ImageNet VQA DSTC RALI General-AI
Translation Narration Dynamic Abductive Goals Cultures Debate Negotiation
WMT DeepVideo Alexa Prize ICCMA AT
Learning from Labeled Training Data and Searching (Optimization)
Learning by Watching and Reading (Education)
Learning by Doing and being Responsible (Exploration)
2015 2018 2021 2024 2027 2030 2033 2036
Which experts would be really surprised if it takes less time… and which experts really surprised if it takes longer?
Approx.
Year
Human
Level ->

Icons of AI
Progress
• 1956: Dartmouth Conference
organized by:
– John McCarthy (Dartmouth, later
Stanford)
– Marvin Minsky (MIT)
– and two senior scientists:
• Claude Shannon (Bell Labs)
• Nathan Rochester (IBM)
• 1997: Deep Blue (IBM) - Chess
• 2011: Watson Jeopardy! (IBM)
• 2016: AlphaGo (Google DeepMinds)

AI Trends
3/13/2018
© IBM Cognitive Opentech Group (COG)
66
Dota 2
“Deep Learning” for
“AI Pattern Recognition”
depends on massive
amounts of “labeled data”
and computing power
available since ~2012;
Labeled data is simply
input and output pairs,
such as a sound and word,
or image and word, or
English sentence and French
sentence, or road scene
and car control settings –
labeled data means having
both input and output data
in massive quantities.
For example, 100K images
of skin, half with skin
cancer and half without to
learn to recognize presence
of skin cancer.

AI Leaders
• Who is winning?
– Regions China vs USA vs EU vs ROW
– Companies Microsoft vs Google vs IBM
• Leaderboards
– SQuAD – Question Answering
– EFF Measuring AI Progress

AI to IA Timeline: Hard unsolved AI
problems
• 2012-2017 AI Pattern Recognition and
Learning from Massive Labeled Data
– Speech, image, translation, driverless, games
– Chatbots as digital assistants
• 2018 Video Understanding
• 2021 Episodic Memory
• 2022 Learning from Watching
• 2024 Commonsense Reasoning
• 2026 Learning from Reading
• 2028 Learning from Doing
• 2030 Fluent Conversation
• 2031-2039 Cognitive Collaborator and
Mediator; Intelligence Augmentation (IA)

AI Benefits
• Access to expertise
– “Insanely great” labor productivity for trusted
service providers
– Digital workers for healthcare, education, finance,
etc.
• Better choices
– ”Insanely great” collaborations with others on
what matters most
– AI for IA = Augmented Intelligence and higher
value co-creation interactions

AI Risks
• Job Loss
– Shorter term
bigger risk
= de-skilling
• Super-intelligence
– Shorter term
bigger risk
= bad actors

Other Technologies: Bigger impact?
Yes.
• Augmented Reality (AR)/
Virtual Reality (VR)
– Game worlds
grow-up
• Blockchain/
Security Systems
– Trust and security
immutable
• Advanced Materials/
Energy Systems
– Manufacturing as cheap,
local recycling service
(utility fog, artificial leaf, etc.)

Stakeholders
• Individuals
• Families
• Businesses and
other Organizations
• Industry Groups
• Regional
Governments:
– Cities
– States
– Nations

Be Prepared
• Understand open AI code + data +
models + stacks + communities
– Leaderboards
– Ethical conduct
• Learn 3 R’s of IBM’s Cognitive
Opentech Group (COG)
– Read arXiv
– Redo with Github
– Report with Jupyter notebooks on
DSX and/or leaderboards
• Improve your team’s skills of rapidly
rebuilding from scratch
– Build your open code eminence
– Understand open innovation
– Communities + Leaderboards
1972 used
Punch cards
2016 used
IBM Watson
Open APIs to win…

Cupertino Teens
• IBM Watson on Bluemix
AI for NLP
entity identification

Courses
• 2015
– “How to build a cognitive system for Q&A task.”
– 9 months to 40% question answering accuracy
– 1-2 years for 90% accuracy, which questions to reject
• 2025
– “How to use a cognitive system to be a better professional X.”
– Tools to build a student level Q&A from textbook in 1 week
• 2035
– “How to use your cognitive mediator to build a startup.”
– Tools to build faculty level Q&A for textbook in one day
– Cognitive mediator knows a person better than they know themselves
• 2055
– “How to manage your workforce of digital workers.”
– Most people have 100 digital workers.
3/13/2018 75
Take free online cognitive classes today at cognitiveclass.ai

Headlines
• 2017 Popular
– “AI vs People”
– “X-Y team up to invest big in AI”
• 2025 Commonplace
– “People using AI to become better at their
professions, serving others.”
– “Teenagers using AI to solve challenges,
and improve their communities.”
• 2085 Resilience
– “Teams competing to rapidly rebuild
socio-economic-technical systems (wise
service systems) from scratch”
– “U.N. Pluto-base makes major discovery
about nature of universe. U.F.P.
established.”
IEEE 2017

IBM-MIT $240M over 10 year AI
mission

3/13/2018
development
78
I have…
Have you noticed how the building
blocks just keep getting better?

Learning to program:
My first program
3/13/2018
development
79
Early Computer Science Class:
Watson Center at Columbia 1945
Jim Spohrer’s
First Program 1972

3/13/2018
© IBM UPWard 2016
80
Fast Forward 2016:
Consider this…

Microsoft CaptionBot June 19, 2016
3/13/2018
© IBM UPWard 2016
81

Microsoft CaptionBot June 20, 2016
3/13/2018
© IBM UPWard 2016
82

IBM Image Tagging
3/13/2018
© IBM UPWard 2016
83

Today: November 10, 2017
3/13/2018
© IBM DBG COG 2017
84
IBM

3/13/2018
development
85
Cognitive Mediators
for all people in all roles

Occupations = Many Tasks
3/13/2018
development
86

Watson Discovery Advisor
3/13/2018
development
87
Simonite, T. 2014. Software Mines Science Papers to Make New Discoveries. MIT. November 25, 2014.
URL: http://m.technologyreview.com/news/520461/software-mines-science-papers-to-make-new-discoveries/

Measuring AI Progress
3/13/2018
© IBM COG 2017
88

AI Trends
3/13/2018
© IBM UPWard 2016
89

3/13/2018 90
1955 1975 1995 2015 2035 2055
Can better service help us be wiser?
Cognitive Mediator (2035): Tool, Assistant, Collaborator, Coach

Courses
2015
– “How to build a cognitive system for Q&A task.”
– 9 months to 40% question answering accuracy
– 1-2 years for 90% accuracy, which questions to reject
2025
– “How to use a cognitive system to be a better professional X.”
– Tools to build a student level Q&A from textbook in 1 week
2035
– “How to use your cognitive mediator to build a startup.”
– Tools to build faculty level Q&A for textbook in one day
– Cognitive mediator knows a person better than they know themselves
2055
– “How to manage your workforce of digital workers.”
– Most people have 100 digital workers.
3/13/2018 91

Computing: Then, Now, Projected
3/13/2018 92
2035
2055

3/13/2018
development
93

Build:
Hardware < Software < Data < Experience < Transformation

What exists in 2016?
3/13/2018
development
97
360,000 100,000 120,000 60,000 150,000

“The best way to predict the future is to inspire the
next generation of students to build it better”
Digital Natives Transportation Water Manufacturing
Energy Construction ICT Retail
Finance Healthcare Education Government

User Models
3/13/2018
development
99

$5M Prize
3/13/2018
© IBM UPWard 2016
100

“AI will change the world?
Who will change AI?”

“These amazing technologies must be
able to help people like myself…”

3/13/2018
development
103
Future of Skills

Future-Ready T-Shapes
3/13/2018
© IBM UPWard 2016
104

In Summary
3/13/2018
development
105
“A service science
perspective considers
the evolving ecology of
service system entities,
their value co-creation and
capability co-elevation
interactions, and their
capabilities, constraints,
rights, and responsibilities.”
Cognitive Systems
Entities
Service
Systems
Entities With
Cognitive
Mediators
Add Rights &
Responsibilities

Good science = Reproducibility

In Conclusion
• Hundreds of AI Challenge Leaderboards
exist and the number is growing
• Can one system be built to do them all?
See: https://www.youtube.com/watch?v=8FpdEmySsuc

What types of digital cognitive
systems?
• Cognitive Build: Outthink Challenge (250K people)
– Imagine a digital cognitive system to help you do
something important in your personal or professional
lives
– Team to design it and advocate for it, and then
everyone votes
– Winners: reduce waste and human suffering, screen
for health issues and safety threats, learn life skills and
make better choices, find what you are looking for,
move around more effectively, provide emotional
support, provide IT support, learn about important
public policy goals and make better choices
• Types: Tool, Assistant, Collaborator, Coach, Mediator

Types
• Tool
• Assistant
•
Collaborat
or
• Coach
• Mediator

capabilities
Task & World Model/
Self Model/
Capacity & Limits
User Model/
Episodic Memory
Institutions Model/
Trust & Social Acts
Tool + - - -
Assistant ++ + - -
Coach ++++ +++ ++ +
Mediator +++++ ++++ +++ ++
Cognitive
Tool
Cognitive
Assistant
Cognitive
Collaborator
Cognitive
Coach
Cognitive
Mediator

Build: Hardware < Software < Data <
Experience

3/13/2018
117

3/13/2018
118

Prepare for AI Future
• Do you have a GitHub account?
– Yes: proceed
– No: sign up
• Do you program?
– Yes: Learn and do 3 R’s (read, redo, report)
– No: Learn to read and execute code (cheat T2T)
• Do you have favorite AI leaderboards?
– Yes: Learn and do 3 R’s
– No: Find a mentor with favorites, do together
• AI prepared = Favorites you can 3 R together
• … until you find the one favorite that can do them all
3/13/2018
© IBM Cognitive Opentech Group 2018
119

3/13/2018
© IBM UPWard 2016
120

You say you want a
revolution?
October 8, 2017
https://www.slideshare.net/spohrer/revolution-20171008-v23
121

By 2035, T-Shaped Makers with great
Building Blocks and Cognitive Mediators
3/13/2018 122
Empathy & Teamwork
sector
region/culture
discipline
Depth
Breadth
STEM
Liberal Arts

In Summary
3/13/2018
development
123
Cognitive Systems
Entities
Service
Systems
Entities With
Cognitive
Mediators
Add Rights &
Responsibilities

3/13/2018
development
124

Machine
Learning
Natural
Language
Processing
High
Performance
Computing
Knowledge
Representation
and Reasoning
Question
Answering
Unstructured
Information

Brief History
of AI
• 1956 – Dartmouth Conference
• 1956 – 1981 Micro-Worlds
• 1981 – Japanese 5th Generation
• 1988 – Expert Systems Peak
• 1990 – AI Winter
• 1997 – Deep Blue
• 1997 – 2011 Real-World
• 2011 – Jeopardy! & SIRI
• 2013 – Cognitive Systems Institute
• 2014 – Watson Business Unit &
• True North Brain Chip
• 2015 – “Cognition as a Service”
on IBM Bluemix
3/13/2018
development
126

Definitions: AI vs IA
3/13/2018
© IBM UPWard 2016
127
AI is Artificial Intelligence, or
intelligence in machines (smart machines)
IA is Intelligence Augmentation, or
people thinking and working together with smart machines.
IA is what IBM calls “Cognitive Computing” and
the smart machines are called “Watson Solutions” or
more generally “Digital Cognitive Systems (Cogs)”
Cognition as a Service (CaaS):
AI building blocks for IA solutions

Augmenting Workers

Startup
Companies

3/13/2018
development
130
I have…
Have you noticed how the building
blocks just keep getting better?

IBM Cloud Bluemix:
Watson APIs are growing…
3/13/2018
© IBM UPWard 2016
131
So far (June 2016), 100,000 faculty and students globally given access

Societal Grand Challenges
3/13/2018
© IBM UPWard 2016
132
Solving these would mean we are smarter,
but would they mean we are wiser?

3/13/2018
© IBM UPWard 2016
133

High School Senior: Anish Krishnan
3/13/2018
© IBM UPWard 2016
134

IBM in Silicon Valley:
From Punch Cards….
3/13/2018
development
135
On August 22, 1943, 105 men, women and children,
among them 43 IBM employees, alighted from a special
train that carried them across the continent to establish
new homes and the new IBM Card Manufacturing Plant
Number 5 at 16th and St. John Streets, San Jose, CA.

3/13/2018
development
136

IBM in Silicon Valley:
To Brain Chips….
3/13/2018
development
137

What types of digital cognitive
systems?
• Cognitive Build: Outthink Challenge (250K people)
– Imagine a digital cognitive system to help you do
something important in your personal or professional
lives
– Team to design it and advocate for it, and then
everyone votes
– Winners: reduce waste and human suffering, screen
for health issues and safety threats, learn life skills and
make better choices, find what you are looking for,
move around more effectively, provide emotional
support, provide IT support, learn about important
public policy goals and make better choices
• Types: Tool, Assistant, Collaborator, Coach, Mediator

Types
• Tool
• Assistant
•
Collaborat
or
• Coach
• Mediator

capabilities
Task & World Model/
Self Model/
Self Capacity & Limits
User Model/
User’s Episodic Memory and Identity
Institutions Model/
Trust & Social Acts
Tool + - - -
Assistant ++ + - -
Coach ++++ +++ ++ +
Mediator +++++ ++++ +++ ++
tool assistant collaborator coach mediator

Partnership for AI formed

3/13/2018
© IBM UPWard 2016
143

3/13/2018
© IBM UPWard 2016
144
What might Reality 2.0 look like?

3/13/2018
development
145

How fast is Reality 2.0 approaching?
3/13/2018 146
What might it look like?

Reality 2.0 Service Platform:
polite cognitive mediators (CM1,CM2)
do not interrupt people (P1,P2)
3/13/2018
© IBM UPWard 2016
147
Service Platform
CM2
P2
CM1
P1
100x
100x 100x

3/13/2018
development
148
Wise Service System:
All entities’ in network
use cognitive mediators
to enhance
value co-creation interactions
Cognitive Mediators:
Cognitive systems
with deep knowledge of both
customer (user) and provider (expert)
as co-creators of win-win value
Entity augmentation boosts both creativity and productivity of interactions

Two assertions
• Machines seem to be getting smarter fast.
– Agree/disagree?
• People do not seem to be getting wiser fast.
– Agree/disagree?
3/13/2018
© IBM UPWard 2016
149
Homo Sapiens means Wise Man: True/False?

IBM Cloud Bluemix:
3/13/2018
© IBM UPWard 2016
150

What is Industry 4.0?
3/13/2018
© IBM UPWard 2016
151

History of the idea: Mirror Worlds
3/13/2018
© IBM UPWard 2016
152

3/13/2018
© IBM UPWard 2016
153
Why is this relevant?

I am still very skeptical…
but tell me more….
3/13/2018
© IBM UPWard 2016
154

3/13/2018
© IBM UPWard 2016
155

Managers: Courage Required….
3/13/2018
© IBM UPWard 2016
156

Talent required, but…
3/13/2018
© IBM UPWard 2016
157

Industry 4.0
3/13/2018
© IBM UPWard 2016
158

CyberPhysical Systems?
3/13/2018
© IBM UPWard 2016
159

Yesterday: Servitization
• Rolls Royce: “Power By The Hour”

Tomorrow: Servitization
• Start with any traditional product that is sold to customers
• Make the product part of a smart/wise service system
– Instrument it (sensors) – Internet of Things/Everything
– Set-up an intelligent operation center to monitor all products’
performance across their life-cycles
– Use big data analytics to determine how to improve product
performance, efficiency, maintenance, etc.
– Offer customer the “product-performance-as-a-service” with
financing/Internet of Service
– Customer benefits from cost-savings, predictability
– Provider benefits margin-improvements, predictability
• Every product becomes a platform technology (a vehicle for
service innovation) for innovative university startups

Vision: MMaaRRSS
• Modular Manufacturing as a Regional
Recirculation Service System
– “I am the stuff that will be made into product X for
customer Y.”
– Stuff = Material, Energy, and Information Flows
– Minimize transport costs (for products and waste)
• The Vision: Circular Economy (~4 minutes)

3/13/2018
development
166
Explain external
phenomena

3/13/2018
development
167
Explain internal
phenomena

3/13/2018
development
168
Explain
value co-creation
phenomena

3/13/2018
development
169
Physics Chemistry Biology
Neuroscience Psychology Artificial
Intelligence
Engineering Management Public
Policy
Education Design Humanities
Natural Systems
Cognitive Systems
Service Systems

Sciences provide…
• Frameworks for people to ask and answer
questions systematically
• Explanations with instructions on “how to re-do”
3/13/2018
development
170
Kline: Conceptual Foundation of Multidisciplinary Thinking -
“To our children and children’s children,
to whom we elders owe an explanation of the world
that is understandable, realistic, forward-looking, and whole.”
Proenneke:
Alone in the Wilderness -
To do a thorough testing,
should each generation
be required to rapidly rebuild
from scratch?
A re-makers movement?

Some paths to becoming 64x smarter:
Improving learning and performance
• 2x from Learning sciences (methods)
– Better models of concepts
– Better models of learners
• 2x from Learning technology (tools)
– Guided learning paths
– Elimination of “thrashing”
• 2x from Quantity effect (overlaps)
– More you know, faster you go
– Advanced organizers
• 2x from Lifelong learning (time)
– Longer lives and longer careers
– Keeps “learning-mode” activated
• 2x from Early learning (time)
– Start earlier: Challenged-based approach
– STEM-2D in K-12 (SSME+DAPP Design of Smart Service Systems)
• 2x from Cognitive systems (performance support)
– Technology & Infrastructure Interactions
– Organizations & Others Interactions

Next Generation:
Future-Ready T-Shaped Adaptive Innovators
Many disciplines
Many sectors
Many regions/cultures
(understanding & communications)
Deepinonesector
Deepinoneregion/culture
Deepinonediscipline

Brief History
of AI
1956 – Dartmouth Conference
1956 – 1981 Micro-Worlds
1981 – Japanese 5th Generation
1988 – Expert Systems Peak
1990 – AI Winter
1997 – Deep Blue
1997 – 2011 Real-World
2011 – Jeopardy! & SIRI
2013 – Cognitive Systems Institute
2014 – Watson Business Unit &
True North Brain Chip
2015 – “Cognition as a Service”
on IBM Bluemix
3/13/2018
development
174

Cognitive Assistants for all occupations
are beginning to appear
3/13/2018
development
175

The Maker Movement &
Open Source Ecology
3/13/2018
development
176

3/13/2018
development
177
Future of Skills

Assisting individuals and organizations
to close their service innovation skills gap
and co-create wiser service systems
empowering employees, customers, citizens
with cognitive mediators
in the collaborative service economy

Dedication: Doug Engelbart
Father of the mouse and
augmentation theory
3/13/2018
development
181

3/13/2018 182
1955 1975 1995 2015 2035 2055
Can better service help us be wiser?
Cognitive Mediator (2035): Tool, Assistant, Collaborator, Coach

3/13/2018
development
185

Cyber-Social-Learning-Systems
Jim Spohrer (IBM)
August 29, 2016
http://www.slideshare.net/spohrer/csls-20160821-v1
3/13/2018 Understanding Cognitive Systems
186

Today’s Talk: Cyber-Social-Learning-
Systems• What is the impact of Artificial Intelligence (AI) on CSLS?
– Augmented Intelligence (IA) via digital cognitive systems
– White House OSTP Response(s)
• Other topics to think about:
– What are more of the implications of digital cognitive systems?
• Tool > Assistant > Collaborator > Coach > Mediator
• Transformation > Experience > Data > Software > Hardware
– What does social intelligence require? Episodic Memory?
– What is the impact of augmented reality on CSLS?
– What are possible connections to service systems science
(SSME+DAPP)?
– What type of adaptive innovator with growth mindset needed
(T-shapes)?

Augmented Intelligence
• Tool
• Assistant
•
Collaborat
or
• Coach
• Mediator

White House OSTP Response(s)
• AI for public good
• Social & economic implications
• Education to harness AI
• Research questions and gaps
• Data sets and model sets
• Multidisciplinary research
• Role of incentives and prizes
• Safety and control protocols
• Legal and governance issues

In Summary
3/13/2018
development
190
Cognitive Systems
Entities
Service
Systems
Entities With
Cognitive
Mediators
Add Rights &
Responsibilities

Backup Slides
• Understanding Cognitive Systems

Understanding
Cognitive Systems
Jim Spohrer (IBM), August 25, 2016
CSIG (Cognitive Systems Institute Group) Speaker
Series
http://www.slideshare.net/spohrer/understanding_20
160825_v3
3/13/2018 Understanding Cognitive Systems
192

Today’s Talk: Understanding Cognitive
Systems• What is a cognitive system (entity)?
– biological
– technological
– types of digital cognitive systems
• How to…
– build them?
– understand them?
– work with them?
• Steps toward a next generation cognitive
curriculum…

But first…. Cognitive Science, a young
field
• Society
– cognitivesciencesociety.org
• People
– Founders: Roger Schank, Donald Norman,
Allan Collins
– Others: David Rumelhart, Herbert Simon,
Allen Newell
– Today: Patrick Langley, Wayne Gray,
Kenneth Forbus, Ashok Goel, Paul Maglio,
etc.
• Systems Conference
– cogsys.org
– (JCS wishes this was part of HICSS)

Advances in Cognitive Systems -
cogsys.org

Google Search: August 26, 2016

Today’s Talk: Understanding Cognitive
Systems• What is a cognitive system (entity)?
– biological
– technological
– types of digital cognitive systems
• How to…
– build them?
– understand them?
– work with them?
• Steps toward a next generation cognitive
curriculum…

What is a cognitive system (entity)?

What is a digital cognitive system
(entity)?

Understand them…

Work with…

Next generation cognitive curriculum

Backup slides
• Service systems - http://service-
science.info/archives/3368

What is service science?
• IBM initiated effort to establish a
multidisciplinary field to study
service systems … with a focus on
people-centered, IT-enabled service
innovations for business and society
– based on service-dominant logic
– service = value co-creation
– IT-enabled service architectures
– service systems (socio-technical
systems for win-win value co-creation)
• IBM helped establish
– computer science (1945-present)
– service science (2005-present)
Service systems are dynamic configurations of
resources (people, technology, organizations,
and information) interconnected by
value propositions, internally and externally.
Examples:
- macro: cities, states, nations
- meso: hospitals, universities, businesses
- micro: households, families, individuals
Reference:
Spohrer J, Maglio P, Bailey J, Gruhl D (2007)
Steps toward a science of service systems.
IEEE Computer Society. 40(3):71-77(January).

What is service science?
• Now over 500 universities globally teach a more
multidisciplinary approach to service innovation,
including:
– Service management and marketing
– Service engineering and operations
– Service design and arts
– Service public policy and economics
– Service computing and informatics
• SSME + DAPP =
Service Science Management Engineering +
Design Arts Public Policy
– People, technology, organizations, information
interconnected by value propositions.
Reference:
IfM & IBM (2008). Succeeding through service
innovation: A service perspective for education,
research, business and government.
University of Cambridge Institute for Manufacturing,
Cambridge, UK. 2008.

How to get involved?
• Weekly speaker series
– Service innovation
– Service education & research
– Smart service/cognitive systems
• Discovery summits & book series
• Opportunities
– Institutional memberships
– Leadership & ambassadors
– Volunteer opportunities
– Awards & sponsored
conferences
ISSIP.org is a non-profit society
International Society of Service Innovation Professionals
Membership:
Over 1000 professionals and students from 40+ countries,
50+ companies and 50+ universities.

How to get involved?
• Journals (INFORMS,
etc.)
• Conferences (HICSS,
etc.)
• Courses (MIT, etc.)
• Funding (NSF, etc.)
• Society (ISSIP, etc.)

What are the hot topics?
• Smart Service Systems: Intelligence Augmentation
– AI + AR UX (Artificial Intelligence + Augmented Reality User
Experience)
– Smartphones (mobile, social, secure, etc.)
• Collaborative Economy: Servitization
– From assets to co-creation (e.g., Uber, AirBnB, etc.)
– From product to capability/outcome-as-a-service
– Manufacturing as a local recycling service
• Digital Transformation: Trust and Identity
– Blockchain: Don Tapscott’s TED Talk & book
– Big Data: Service Analytics & HAT (Hub of All Things)

Backups
• T-shaped people

Cognitive Assistants for all occupations
are beginning to appear
3/13/2018
development
213

My Quest
• What is the best bot that anyone can easily
install on their smartphone and laptop today?
3/13/2018
© IBM UPWard 2016
219

3/13/2018
226

Tutorial helsinki 20180313 v1

More Related Content

What's hot

Similar to Tutorial helsinki 20180313 v1

More from home

Recently uploaded

Tutorial helsinki 20180313 v1