1. RISELab:
Enabling Intelligent Real-time
Decisions
Ion Stoica
February 8, 2017

2. Berkeley’s AMPLab (2011-2016)
2
Algorithms
Machines People
Goal: Nextgeneration of open source
data analytics stack for industry & academia
BerkeleyDataAnalytics Stack (BDAS)

3. Berkeley’s AMPLab (2011-2016)
3
Algorithms
Machines People
Goal: Nextgeneration of open source
data analytics stack for industry & academia
BerkeleyDataAnalytics Stack (BDAS)
…

4. RISE: Real-time Intelligent
Secure Execution

5. From batch data to advanced analytics
AMPLab
5
From live data to real-time decisions
RISELab

6. RISE Lab (2017-2022)
12 faculty across AI, systems, security, and architectures
11 Founding sponsors
6

7. Why?
Data only as valuable as the decisions it enables
7

8. Why?
What does this mean?
• Faster decisions better than slower decisions
• Decisions on fresh data better than decisions on stale data
• Decisions on personalized data better than on aggregate data
8
Data only as valuable as the decisions it enables

9. Goal
Real-time decisions
on live data
with strong security
9
decide in ms
the current stateof theenvironment
privacy, confidentiality, integrity

10. Typical decision system
Decision System
Query
Decision
Environment
+
sensors &
actuators
Observations, Feedback
Preprocess Intermediate
data
Decision
Engine

11. Decision
Query
Decision
Engine
Preprocess Intermediate
data
Environment
+
sensors &
actuators
Typical decision system
Decision System
Observations, Feedback
Live
Update latency
(e.g., ~1 seconds)

12. Decision
Query
Decision
Engine
Preprocess Intermediate
data
Environment
+
sensors &
actuators
Typical decision system
Decision System
Observations, Feedback
Live
Update latency
(e.g., ~1 seconds)
Secure
Real-time
decision latency
(e.g., ~10 ms)

13. Example of decision systems
Decision System
Query
Decision
Training
Models
(diff.tradeoffs
complexity/
accuracy)
Model
Serving
Feedback
Observations, Feedback
ML Pipeline
(e.g., Clipper +
Spark/Tensorflow)
Decision System
Obs.
Action
Update
Policy
Policy
obs à
action
Query
Policy
Observations, Rewards
Reinforcement
Learning Systems
(e.g., Ray)
Pre-
process
Interm-
ediate
Data
Decision
Engine

14. What else do we want from decisions?
Intelligent:complexdecisions in uncertain environments
Robust: handle complexnoise, unforeseeninputs, failures
Explainable:ability to explainnon-obvious decisions

15. Goal
Develop open source
platforms, tools, and algorithms for
intelligent real-time decisions on live-data

16. Some Proposed Research
Secure Real-time Decisions Stack (SRDS)
• Open source platform to develop of RISE apps
• Secure from ground up
• Reinforcement Learning (RL) as one of key app patterns
Learning control hierarchies: speeduplearning, training
Shared learning: learn over confidential data
16

17. Secure Real-time Decisions Stack (SRDS)
• Open source platform to develop of RISE apps
• Secure from ground up
• Reinforcement Learning (RL) as one of key app patterns
Learning control hierarchies: speeduplearning, training
Shared learning: learn over confidential data
17
Some Proposed Research

18. Secure Real-time Decision Stack (SRDS)
scheduler object store
RISE μkernel
Ray Clipper …
Ground(data contextservice)
Time
Machine

19. scheduler object store
RISE μkernel
Ray Clipper …
Ground(data contextservice)
Time
Machine
Minimalist executionengine:
• Support both data flow and task-parallel execution models
• High-throughput, low-latency: ~ 1M tasks/sec @ ms latency
Secure Real-time Decision Stack (SRDS)

20. scheduler object store
RISE μkernel
Ray Clipper …
Ground(data contextservice)
Time
Machine
Central repositoryfor models,APIs to capture the context
in whichdata gets used and produced
Status:ongoing project with industry partners
Secure Real-time Decision Stack (SRDS)

21. scheduler object store
RISE μkernel
Ray Clipper …
Ground(data contextservice)
Time
Machine
Replaying of apps at fine granularity
• Simplify development, debugging
• Robustness: replay against perturbed inputs
• Explainability: identify inputs causing decision
• Security: confirm vulnerabilities, test security
patches, compliance auditing
Secure Real-time Decision Stack (SRDS)

22. scheduler object store
RISE μkernel
Ray Clipper …
Ground(data contextservice)
Time
Machine
Dramatically simplify development of RISE applications
• Apache Spark: improve latency and security
• Clipper: model serving for Apache Spark, Scikit learn, etc
• Ray: framework for RL applications
Secure Real-time Decision Stack (SRDS)

23. ImprovingApache Spark
Drizzle
• Decrease latency of Structured Streaming and ML algorithms by ~10x
• Techniques: group scheduling, shared variables
23

24. StreamingLatency:YCSB benchmark
24
0
500
1000
1500
2000
1 2 4 8 12 16 20 24
MedianEventLatency
(ms)
Throughput (Million events/s)
Spark Drizzle Flink Drizzle-Opt
Drizzle-Opt: Reduce-by on mapper side

25. StreamingLatency:YCSB benchmark
25
0
500
1000
1500
2000
1 2 4 8 12 16 20 24
MedianEventLatency
(ms)
Throughput (Million events/s)
Spark Drizzle Flink Drizzle-Opt
Drizzle-Opt: Reduce-by on mapper side

26. StreamingLatency:YCSB benchmark
26
0
500
1000
1500
2000
1 2 4 8 12 16 20 24
MedianEventLatency
(ms)
Throughput (Million events/s)
Spark Drizzle Flink Drizzle-Opt
Drizzle-Opt: Reduce-by on mapper side

27. StreamingLatency:YCSB benchmark
27
0
500
1000
1500
2000
1 2 4 8 12 16 20 24
MedianEventLatency
(ms)
Throughput (Million events/s)
Spark Drizzle Flink Drizzle-Opt
Drizzle-Opt: Reduce-by on mapper side

28. StreamingLatency:YCSB benchmark
28
0
500
1000
1500
2000
1 2 4 8 12 16 20 24
MedianEventLatency
(ms)
Throughput (Million events/s)
Spark Drizzle Flink Drizzle-Opt
Drizzle-Opt: Reduce-by on mapper side
15x

29. MLlib: SGD Performance
29
0
20
40
60
4 8 16 32 64 128
Time/iter(ms)
Machines
Spark Drizzle

30. MLlib: SGD Performance
30
0
20
40
60
4 8 16 32 64 128
Time/iter(ms)
Machines
Spark Drizzle
6x

31. ImprovingApache Spark
Drizzle
• Decrease latency of Structured Streaming and ML algorithms by ~10x
• Techniques: group scheduling, shared variables
• Some of these techniques will make their way to Apache Spark
Opaque
• Full data encryption, authentication, and verification (Intel’s SGX)
• Oblivious mode: hide data access pattern
• Support most SparkSQL functionality
• See Wenting’s talk later
31

32. RISELab
Already promising results
Expect muchmore over the next five years!
32
Goal: Develop open source platforms,tools, and
algorithms for intelligent real-timedecisions on live-
data

33. Thank you