More Related Content
Similar to Proactive eth talk (20)
More from Opher Etzion (20)
Proactive eth talk
- 1. Proactive
event-driven
computing
Towards Proactive Event-Driven Computing
Talk in ETH – March 2012
Opher Etzion (opher@il.ibm.com)
IBM Haifa
Research Lab © 2011 IBM Corporation
- 2. Proactive event-driven computing
(The source code movie (spoiler
The hero of the story is sent to the occupy the
Body of a dead person during the last 8 minutes of
His life, trying to find out who put dirty bomb inside
-A train so he‘ll be stopped from doing the next attack
In an unexpected turn of events he succeeds to
Eliminate the attack and change the past
IBM Haifa Research Lab 2 © 2012 IBM Corporation
- 3. Proactive event-driven computing
The proactive event-driven principle
Forecast
Proactive action
Real-time decision
Detect
now time
IBM Haifa Research Lab 3 © 2012 IBM Corporation
- 5. Proactive event-driven computing
The proactive pattern
Forecast
Detect Decide Act
(RT) (proactive)
Taking proactive actions in
setting up entry and exit
traffic lights durations and
Forecasting that at some speed limit in highway
point in 15 minute a traffic segments
congestion of certain size
will occur in probability of
Monitoring streams of events 0.6
from sensor in highway and
leading ways, from mobile
devices, and from accidents
reports
IBM Haifa Research Lab 5 © 2012 IBM Corporation
- 7. Proactive event-driven computing
The proactive pattern
Forecast
Detect Decide Act
(RT) (proactive)
Using RT optimization –
schedule appliances use and
apply through actuators
Forecasting that in the
morning hours the household
will not produce any energy
and the power grid’s price
Monitoring sun, wind and will be high
demand on power grid
IBM Haifa Research Lab 7 © 2012 IBM Corporation
- 9. Proactive event-driven computing
The proactive pattern
Forecast
Detect Decide Act
(RT) (proactive)
Taking proactive actions in
notifying and performing
actions like – close roads,
Forecasting that at some reduce speed of trains, turn
point in the next hour there off gas and water supply…
is going to be a a potential
damage in a certain location
Monitoring earthquake,
spread by sensors, and
citizen reports
IBM Haifa Research Lab 9 © 2012 IBM Corporation
- 10. Proactive event-driven computing
The evolution towards proactive computing
Typically people employ
computing in responsive The initiative remains in
way: the person makes human hands;
decisions and the computer most persons are not
assists in data, knowledge, proactive by nature
advice
The initiative moves to
Recently, there is more the computer;
employment of computers reactions to events that
in reactive way: events already occurred
drive decisions (Detect-
Derive-Decide-Do)
The initiative moves to
the computer;
The vision is to move to
proactive computing:
(Detect-Derive-Predict-
X actions to events before
they occur
Decide now-Do)
IBM Haifa Research Lab 10 © 2012 IBM Corporation
- 11. Proactive event-driven computing
Why it is difficult to create proactive
?solutions now
A way of thinking
Multiple skills are
needed
Incompatible programming model of the
moving parts, and gaps in each of them
within the current product
Predictive analytics
Decision models
Optimization
Event Processing
IBM Haifa Research Lab 11 © 2012 IBM Corporation
- 12. Proactive event-driven computing
Proactive computing as cultural change
The culture in many organizations and
personal behavior advocates a routine
behavior governed by fixed set of rules
Many people are deterred from ad-hoc
behavior even if it has relative benefit in
specific case and prefer statistical
.metrics
Current analytics tools are geared
towards improving the ”fixed set of
”rules
Proactive thinking is different – it provides
exception behavior to mitigate or eliminate
problems when current rules will not work
IBM Haifa Research Lab 12 © 2012 IBM Corporation
- 13. Proactive event-driven computing
The Proactive pillars
Proactive event-driven computing
Integrative platform and validation
Scalable platform
Event processing Event recognition Event-based Human computer
foundations and forecasting optimization interaction
uncertain event adaptive real
events, future recognition, time human
events, expert optimization for interaction in
correctness forecasting proactive proactive
issues ,models decisions systems
goal driven
supervised
learning
Paradigm: methodology, seamless programming model
IBM Haifa Research Lab 13 © 2012 IBM Corporation
- 14. Proactive event-driven computing
Event-flow programming model: the EPN
Agent 1 Agent 2
Event Event
Producer 1 Consumer 1
State
Event
Consumer 2
Agent 3
Event Channel
Producer 2
Event
Consumer 3
IBM Haifa Research Lab 14 © 2012 IBM Corporation
- 15. Proactive event-driven computing
PRA can be
Context defined per
context segment,
and receive
events only from
e1 d3 EPAs in the same
Producer EPA EPA context
PRA can send
events to
d1 d4 EPAs, e.g.,
e3 ”emergency
Messages to and ”generator fix
from EPAs are
)potentially
A1
uncertain( events, EPA PRA OK
with present or d4 Actuator
future time
d2
interval State
Actuator may respond
e2 immediately, or send
Producer EPA acknowledgement via
an event
Enrich
from dB
State Consumer
EPAs may need to consult
the current state of the
dB PRA
IBM Haifa Research Lab 15 © 2012 IBM Corporation
- 16. Proactive event-driven computing
(Proactive Agent (PRA
PRA
Input: forecasted events +
state information
Output: Action –
recommendation, activation,
command to actuator
Process: real-time decision
making
Real time decision making
Spectrum from
Trivial: decision tree
Basic: basic conditions - MDP
Advanced: simulation based
optimization, advanced
modeling
IBM Haifa Research Lab 16 © 2012 IBM Corporation
- 17. Proactive event-driven computing
Enhancing the current event processing technology
,Events may be uncertain: uncertainty about their occurrence
occurrence time, and any of their attribute values; furthermore
there may be uncertainty about relation between derived event and
Situation, and propagation of uncertain values to derived events
Derived event may occur in the future
– (using predictive models)
Running future time window in the present
Furthermore the semantics of derived event
Changes from virtual event to raw event
Applying event processing abstractions to
states – and use hybrid model
IBM Haifa Research Lab 17 © 2012 IBM Corporation
- 18. Proactive event-driven computing
By 2015, 80% of all available data will be uncertain
By 2015 the number of networked devices will
be double the entire global population. All
9000
sensor data has uncertainty.
8000 100
Global Data Volume in Exabytes
90 The total number of social media
7000
accounts exceeds the entire global
Aggregate Uncertainty %
80 population. This data is highly uncertain
6000
in both its expression and content.
70
)
gs
5000
s
or
hin
60
ns
fT
Data quality solutions exist for
Se
o
4000 50
et
rn
enterprise data like customer,
te
(In
3000 40 product, and address data, but
this is only a fraction of the ia )
M ed d text
2000
30 total enterprise data. i al a n
S ,oc audio
20 eo P
1000 (vid VoI
10
0 Enterprise Data
Multiple sources: IDC,Cisco
2005 2010 2015
IBM Haifa Research Lab 18 © 2012 IBM Corporation
- 19. Proactive event-driven computing
“The dimensions of “BIG DATA
Data has to be processed in higher
Velocity
Data has high variability: poly-
structured. Many sources:
sensors, social media, multi-
media…
Volumes of data are
constantly growing
Veracity: Data has
inherent uncertainty
associated with it
IBM Haifa Research Lab 19 © 2012 IBM Corporation
- 20. Proactive event-driven computing
Uncertainty aspects
Meta-data representation
Real-time decision under
of uncertainty
uncertainty
Removal
of uncertainty
Propagation of uncertainty
By
Thresholds
Bayesian Nets
By
Robust determination Semantic propagation
Monte-Carlo methods
IBM Haifa Research Lab 20 © 2012 IBM Corporation
- 21. Proactive event-driven computing
Adding canonic representation for
:uncertainty handling
Uncertain about the level of
Uncertain whether an causality between a car
reported event has occurred heading towards highway
(e.g. accident) and a car getting into the
highway
Uncertain about the accuracy
of a sensor input: count of
Uncertain what really cars, velocity of cars…
happened. What is the type
and magnitude of the
accident (vehicles involved,
casualties) Uncertain about the validity
The pattern: more of a forecasting pattern
than 100 cars
approach an area
Uncertain when an event within 5 minutes after
occurred (will occur): timing an accident derives a
of forecasted congestion congestion forecasting
Uncertain where an event
occurred (will occur):
location of forecasted Uncertain about the quality of
congestion the decision about traffic
lights setting
IBM Haifa Research Lab 21 © 2012 IBM Corporation
- 22. Proactive event-driven computing
Real-time decision under uncertainty
Robust RT
Optimization
Stochastic RT
Optimization
Simulation-based Stochastic
RT optimization RT
optimization
Simulation-
base RT
Simulation-
optimization
base RT
optimization
IBM Haifa Research Lab 22 © 2012 IBM Corporation
- 23. Proactive event-driven computing
Learning patterns and causalities
Event
Patterns This is a direction to
reduce the complexity
of application
development
Pattern and causality acquisition
“There are challenges in doing it – since “detected situations
are “inferred events“ and may not be reflected in past events
IBM Haifa Research Lab 23 © 2012 IBM Corporation
- 24. Proactive event-driven computing
Summary
Proactive event driven computing
is a new paradigm with potential
big impact on society as well as
future IT
There is an ecosystem of
external collaborators
mainly working on proposed
EU project
The aim is to combine science
and engineering to create a
generic software platform
IBM Haifa Research Lab 24 © 2012 IBM Corporation