1.
RISELab:
Enabling Intelligent Real-time
Decisions
Ion Stoica
February 8, 2017

2.
Berkeley’s AMPLab (2011-2016)
2
Algorithms
Machines People
Goal: Nextgeneration of open source
data analytics stack for industry & academia
BerkeleyDataAnalytics Stack (BDAS)

3.
Berkeley’s AMPLab (2011-2016)
3
Algorithms
Machines People
Goal: Nextgeneration of open source
data analytics stack for industry & academia
BerkeleyDataAnalytics Stack (BDAS)
…

4.
RISE: Real-time Intelligent
Secure Execution

5.
From batch data to advanced analytics
AMPLab
5
From live data to real-time decisions
RISELab

6.
RISE Lab (2017-2022)
12 faculty across AI, systems, security, and architectures
11 Founding sponsors
6

7.
Why?
Data only as valuable as the decisions it enables
7

8.
Why?
What does this mean?
• Faster decisions better than slower decisions
• Decisions on fresh data better than decisions on stale data
• Decisions on personalized data better than on aggregate data
8
Data only as valuable as the decisions it enables

9.
Goal
Real-time decisions
on live data
with strong security
9
decide in ms
the current stateof theenvironment
privacy, confidentiality, integrity

10.
Typical decision system
Decision System
Query
Decision
Environment
+
sensors &
actuators
Observations, Feedback
Preprocess Intermediate
data
Decision
Engine

11.
Decision
Query
Decision
Engine
Preprocess Intermediate
data
Environment
+
sensors &
actuators
Typical decision system
Decision System
Observations, Feedback
Live
Update latency
(e.g., ~1 seconds)

12.
Decision
Query
Decision
Engine
Preprocess Intermediate
data
Environment
+
sensors &
actuators
Typical decision system
Decision System
Observations, Feedback
Live
Update latency
(e.g., ~1 seconds)
Secure
Real-time
decision latency
(e.g., ~10 ms)

13.
Example of decision systems
Decision System
Query
Decision
Training
Models
(diff.tradeoffs
complexity/
accuracy)
Model
Serving
Feedback
Observations, Feedback
ML Pipeline
(e.g., Clipper +
Spark/Tensorflow)
Decision System
Obs.
Action
Update
Policy
Policy
obs à
action
Query
Policy
Observations, Rewards
Reinforcement
Learning Systems
(e.g., Ray)
Pre-
process
Interm-
ediate
Data
Decision
Engine

14.
What else do we want from decisions?
Intelligent:complexdecisions in uncertain environments
Robust: handle complexnoise, unforeseeninputs, failures
Explainable:ability to explainnon-obvious decisions

15.
Goal
Develop open source
platforms, tools, and algorithms for
intelligent real-time decisions on live-data

16.
Some Proposed Research
Secure Real-time Decisions Stack (SRDS)
• Open source platform to develop of RISE apps
• Secure from ground up
• Reinforcement Learning (RL) as one of key app patterns
Learning control hierarchies: speeduplearning, training
Shared learning: learn over confidential data
16

17.
Secure Real-time Decisions Stack (SRDS)
• Open source platform to develop of RISE apps
• Secure from ground up
• Reinforcement Learning (RL) as one of key app patterns
Learning control hierarchies: speeduplearning, training
Shared learning: learn over confidential data
17
Some Proposed Research

18.
Secure Real-time Decision Stack (SRDS)
scheduler object store
RISE μkernel
Ray Clipper …
Ground(data contextservice)
Time
Machine

19.
scheduler object store
RISE μkernel
Ray Clipper …
Ground(data contextservice)
Time
Machine
Minimalist executionengine:
• Support both data flow and task-parallel execution models
• High-throughput, low-latency: ~ 1M tasks/sec @ ms latency
Secure Real-time Decision Stack (SRDS)

20.
scheduler object store
RISE μkernel
Ray Clipper …
Ground(data contextservice)
Time
Machine
Central repositoryfor models,APIs to capture the context
in whichdata gets used and produced
Status:ongoing project with industry partners
Secure Real-time Decision Stack (SRDS)

21.
scheduler object store
RISE μkernel
Ray Clipper …
Ground(data contextservice)
Time
Machine
Replaying of apps at fine granularity
• Simplify development, debugging
• Robustness: replay against perturbed inputs
• Explainability: identify inputs causing decision
• Security: confirm vulnerabilities, test security
patches, compliance auditing
Secure Real-time Decision Stack (SRDS)

22.
scheduler object store
RISE μkernel
Ray Clipper …
Ground(data contextservice)
Time
Machine
Dramatically simplify development of RISE applications
• Apache Spark: improve latency and security
• Clipper: model serving for Apache Spark, Scikit learn, etc
• Ray: framework for RL applications
Secure Real-time Decision Stack (SRDS)

23.
ImprovingApache Spark
Drizzle
• Decrease latency of Structured Streaming and ML algorithms by ~10x
• Techniques: group scheduling, shared variables
23

24.
StreamingLatency:YCSB benchmark
24
0
500
1000
1500
2000
1 2 4 8 12 16 20 24
MedianEventLatency
(ms)
Throughput (Million events/s)
Spark Drizzle Flink Drizzle-Opt
Drizzle-Opt: Reduce-by on mapper side

25.
StreamingLatency:YCSB benchmark
25
0
500
1000
1500
2000
1 2 4 8 12 16 20 24
MedianEventLatency
(ms)
Throughput (Million events/s)
Spark Drizzle Flink Drizzle-Opt
Drizzle-Opt: Reduce-by on mapper side

26.
StreamingLatency:YCSB benchmark
26
0
500
1000
1500
2000
1 2 4 8 12 16 20 24
MedianEventLatency
(ms)
Throughput (Million events/s)
Spark Drizzle Flink Drizzle-Opt
Drizzle-Opt: Reduce-by on mapper side

27.
StreamingLatency:YCSB benchmark
27
0
500
1000
1500
2000
1 2 4 8 12 16 20 24
MedianEventLatency
(ms)
Throughput (Million events/s)
Spark Drizzle Flink Drizzle-Opt
Drizzle-Opt: Reduce-by on mapper side

28.
StreamingLatency:YCSB benchmark
28
0
500
1000
1500
2000
1 2 4 8 12 16 20 24
MedianEventLatency
(ms)
Throughput (Million events/s)
Spark Drizzle Flink Drizzle-Opt
Drizzle-Opt: Reduce-by on mapper side
15x

29.
MLlib: SGD Performance
29
0
20
40
60
4 8 16 32 64 128
Time/iter(ms)
Machines
Spark Drizzle

30.
MLlib: SGD Performance
30
0
20
40
60
4 8 16 32 64 128
Time/iter(ms)
Machines
Spark Drizzle
6x

31.
ImprovingApache Spark
Drizzle
• Decrease latency of Structured Streaming and ML algorithms by ~10x
• Techniques: group scheduling, shared variables
• Some of these techniques will make their way to Apache Spark
Opaque
• Full data encryption, authentication, and verification (Intel’s SGX)
• Oblivious mode: hide data access pattern
• Support most SparkSQL functionality
• See Wenting’s talk later
31

32.
RISELab
Already promising results
Expect muchmore over the next five years!
32
Goal: Develop open source platforms,tools, and
algorithms for intelligent real-timedecisions on live-
data

33.
Thank you