This document discusses cognitive computing from the Institute of Cognitive Science. It describes how cognitive computing uses social media analysis, data science methods, and IBM's Watson AI to better predict disease spread, such as influenza. By fusing real-time social media data with slower but more reliable CDC data, cognitive systems can improve predictions. The institute also researches neuromorphic hardware and reservoir computing techniques inspired by the brain to enable new kinds of fault-tolerant computing.

1. Cognitive Computing
Prof. Dr. Gordon Pipa,
University Osnabrück
23 Oct. 2016
Prof. Dr. Gordon Pipa, University Osnabrück
Prof. Dr. Kai-Uwe Kühnberger, University Osnabrück
Prof. Dr. Dr. Bertram Scheller, University Hospital Frankfurt

2. Institute of
Cognitive Science
At the core of
Cognitive Computing
since 15 years
Cognitive Computing
Technology
Interaction
Humans
10 Full professors
~ 600 BSc
~ 200 MSc
~ 45 PhD students
Symbiotic fusion of the intelligent system,
the user, and the expert.
Dialog between machine and human
(natural language, intuitive graphics, and
gestures)
Neuro-inspired hardware for fault-tolerant
new computing devices
The machine is the super assistant that
enables the human to make truly
intelligent decisions in complex scenarios.

3. Institute of
Cognitive Science
Influenza
matters
Prediction is
important
Delayed and
too few data
Why Cognitive Computing?
A one year project
by a core team of
three master’s
students

4. Institute of
Cognitive Science
Influenza
matters
Prediction is
important
Delayed and
too few data
Why Cognitive Computing?
Social media
analysis
Data science
methods
Watson as
medical expert

5. Institute of
Cognitive Science
Influenza
matters
Prediction is
important
Delayed and
too few data
Why Cognitive Computing?
Social media
analysis
Data science
methods
Watson as
medical expert
Fully informed
user
Better
prediction
Cognitive Computing

6. Institute of
Cognitive Science
Influenza
matters
Prediction is
important
Delayed and
too few data
Why Cognitive Computing?
Social media
analysis
Data science
methods
Watson as
medical expert
Fully informed
user
Better
prediction

7. Institute of
Cognitive Science
Structured Causal Models
• Disease spreads locally and via transportation hubs
• Weather, vaccination, and seasonal events change spreading

8. Institute of
Cognitive Science
Structured Causal Models
• Disease spreads locally and via transportation hubs
• Weather, vaccination, and seasonal events change spreading

9. Institute of
Cognitive Science
Structured Causal Models
• Disease spreads locally and via transportation hubs
• Weather, vaccination, and seasonal events change spreading

10. Institute of
Cognitive Science
Structured Causal Models
Direction and speed of spread NEEDS to be identified from data

11. Institute of
Cognitive Science
Identify Causal Interactions
A driver influences and
thereby leaves a trace that can be reconstructed
B
• Schumacher et al. (2015) - A Statistical Framework to Infer Delay and Direction of Information …
• Sugihara et al. (2012) - Detecting Causality in Complex Ecosystems
A
C

12. Institute of
Cognitive Science
Identify Causal Interactions
The model can be analyzed:
When is New York going to be hit?
• Schumacher et al. (2015) - A Statistical Framework to Infer Delay and Direction of Information …
• Sugihara et al. (2012) - Detecting Causality in Complex Ecosystems
B
A
C

13. Institute of
Cognitive Science
Identify Causal Interactions
The model can be analyzed:
Can vaccinations in Chicago stop the wave?
• Schumacher et al. (2015) - A Statistical Framework to Infer Delay and Direction of Information …
• Sugihara et al. (2012) - Detecting Causality in Complex Ecosystems
B
A
C

14. Institute of
Cognitive Science
IBM Blue Mix & Watson at work
Supported by:

15. Institute of
Cognitive Science
IBM Blue Mix & Watson at work
Supported by:

16. Institute of
Cognitive Science
Influenza
matters
Prediction is
important
Delayed and
too few data
Why Cognitive Computing?
Social media
analysis
Data science
methods
Watson as
medical expert
Fully informed
user
Better
prediction

17. Institute of
Cognitive Science
Social Media
Twitter gives realtime and anytime available data
IBM Insights contains tweets since 2014
Twitter activity
(geo tag + tweet)

18. Institute of
Cognitive Science
Sample Tweets from 29/09/16
himself
worried
herself
sick
familiy is
sick
friends are
sick

19. Institute of
Cognitive Science
Close the Gap by Fusing Data
Realtime fuzzy
social media
+
Slow but reliable
CDC data
Twitter activity
(geo tag + tweet)
CDC – delayed
influenza data
Use the best from both worlds to improve prediction

20. Institute of
Cognitive Science
IBM Blue Mix & Watson at Work
Supported by:

21. Institute of
Cognitive Science
IBM Blue Mix & Watson at Work
Supported by:

22. Institute of
Cognitive Science
Influenza
matters
Prediction is
important
Delayed and
too few data
Why Cognitive Computing?
Social media
analysis
Data science
methods
Watson as
medical expert
Fully informed
user
Better
prediction

23. Institute of
Cognitive Science
Why Cognitive Computing?
...
?
?
?
?

24. Institute of
Cognitive Science
Watson at Work
Supported by:

25. Institute of
Cognitive Science
Watson at Work
Supported by:

26. Institute of
Cognitive Science
Performance – One Region USA

27. Institute of
Cognitive Science
Performance: Twitter + CDC for 6 Regions of US
target
CDC
CDC+Tw.

28. Institute of
Cognitive Science
Performance – One Region USA

29. Institute of
Cognitive Science
Summary
Social media
analysis
Data science
methods
Watson as expert
● Data science allows identification of very complex causal relations
● Efficient use of BlueMix and Watson services
● Combine social media with other conventional data to get the best of both
worlds  realtime and reliable
● Use large corpora to identify structure and relationships in your problem
● Use natural language interface for easy to use HCI

30. Institute of
Cognitive ScienceLEARNING TO FLY – LIKE A BIRD
• Bioinspired motor control
• Use of real-time recurrent
networks
• Reservoir computing system
learns complex flight
dynamics

31. Institute of
Cognitive Science
Neuromorphic Devices – A new Era of Computing
• Millions of spiking
neurons at ultra low
power (<mW)
• Super fast, super parallel
brain-like computation
Nieters, Leugering, Pipa, „Neuromorphic computation in multi-delay coupled models“, IBM Journal of Research (accepted)

32. Institute of
Cognitive Science
Computing with Random Recurrent Networks
• Fading memory
• Causal changes
• A model

33. Institute of
Cognitive ScienceReservoir Computing
•Herbert. Jaeger, Harald Haas. "Harnessing nonlinearity: Predicting chaotic systems and saving
energy in wireless communication." Science 304.5667 (2004): 78-80.
•Jaeger, H., Lukoševičius, M., Popovici, D., & Siewert, U. (2007). Optimization and applications of
echo state networks with leaky-integrator neurons. Neural Networks, 20(3), 335-352.
•Yildiz, Izzet B., H. Jaeger, and S. J. Kiebel. "Re-visiting the echo state property." Neural Networks
(2012).
•Waegeman, Schrauwen, Jaeger. "Technical report on hierarchical reservoir computing
architectures." (2012).
•W. Maass, T. Natschläger, H. Markram. "Real-time computing without stable states: A new
framework for neural computation based on perturbations." Neural computation 14.11 (2002):
2531-2560.
•Buesing, Lars, et al. "Neural dynamics as sampling: A model for stochastic computation in recurrent
networks of spiking neurons." PLoS computational biology 7.11 (2011): e1002211.
•Buonomano, D. V., & Maass, W. (2009). State-dependent computations: spatiotemporal processing
in cortical networks. Nature Reviews Neuroscience, 10(2), 113-125.
Prof. Herbert Jäger
Prof. Wolfgang Maass

34. Institute of
Cognitive ScienceReservoir Computing
y(t)
Perturbation u(t)
W. Maass, T. Natschläger, H. Markram. "Real-time computing without stable states: A new framework for neural computation based on
perturbations." Neural computation 14.11 (2002): 2531-2560.
Task-specific mapping
In mathematical terms, this liquid state is simply the
current output of some operator or filter LM that maps
input functions u(.) onto a liquid state xM(t) :
( ) ( )( )M M
x t L u t
The second component is a memory-less
readout map fM that transforms, at every time t ,
the current liquid state xM(t) into the output
( ) ( ( ))M M
y t f x t

35. Institute of
Cognitive ScienceNEURO-INSPIRED HARDWARE
Spatiotemporal computations of an excitable and plastic brain: neuronal plasticity
leads to noise-robust and noise-constructive computations, H. Toutounji, G. Pipa -
PLOS Comput Biol, 2014

36. Institute of
Cognitive Science
Neuro-Inspired Self-Learning Systems

37. Institute of
Cognitive Science
Mobile Crowd EEG
• Less than $99
• 8 channel, LED stimulation, gyroscope
• Mobile system with Bluetooth LE, encrypted data transmission
• Allows for nothing less than a new ERA of crowd EEG experiments

38. Institute of
Cognitive Science
IoT: The Future of ALP Crowd Bio-Signals
You design
your analysis
that we perform
for you.
Pay for the
service and the
data you get.
We store
and process
the data.
We rent
devices.

39. Institute of
Cognitive Science
At the core of
Cognitive Computing
since 15 years
The Cognitive Era
Technology
Interaction
Humans
10 Full professors
~ 600 BSc
~ 200 MSc
~ 45 PhD students
Symbiotic fusion of the intelligent
system, the user, and the expert.
Dialog between machine and human
(natural language, intuitive graphics,
and gestures)
Neuro-inspired hardware for fault
tolerant new computing devices
The machine is the super assistant
that enables the human to make
truly intelligent decisions in complex
scenarios.

40. Institute of
Cognitive Science
The Cognitive ERA
JOIN THE ADVENTURE
OF COGNITIVE COMPUTING
Supported by:
Prof. Dr. Gordon Pipa
gpipa@uos.de