The benefits of fine-grained synchronization in deterministic and efficient stream processing

The benefits of fine-grained synchronization in
deterministic and efficient stream processing
Vincenzo Gulisano, Yiannis Nikolakopoulos
2015-10-31 Vincenzo Gulisano - Yiannis Nikolakopoulos 1
Chalmers University
of technology

Agenda
• Who we are
• Motivation and System Model
• ScaleJoin: a Deterministic, Disjoint-Parallel and
Skew-Resilient Stream Join
• The ScaleGate data structure
• Fine-grained synchronization and other
use-cases
• Research in progress
• Conclusions

Agenda
• Who we are
use-cases
• Conclusions

Who we are
Chalmers university
Computer Science and Engineering
department
Distributed Computing and Systems
research group

Who we are
• 12 PhD degrees awarded
• ~20 researchers, 3 Postdocs and 10 PhD
students.
• acknowledged internationally as leading
group in practical multicore synchronization
algorithms and programming (results
adopted by Java JDK, C++, Intel, NVIDIA)
• extensive network of academic and
industrial collaborations and has been
continuously supported by national and
international projects
Distributed Computing and
Systems research group

Who we are
Vincenzo Gulisano (PhD)
Assistant Professor at
Chalmers University of Technology
Research focus:
• scalable big data analysis in cyber-physical systems
(Smart Grids, Advanced Metering Infrastructures and
Vehicular Networks).
• streaming operators’ parallelization, load balancing,
provisioning, decommissioning and fault tolerance
protocols.
• enhancement of parallel stream operators by means of
streaming- and energy-aware concurrent data
structures that boost analysis performance (both in
throughput and latency).
• applied use of the streaming paradigm in data
validation, intrusion detection systems, privacy-
preserving online analysis, distributed embedded
networks and cloud infrastructures.

Who we are
Yiannis Nikolakopoulos
PhD Student
Research interests:
• Parallel and distributed computing
• Shared memory concurrent data structures
• Multicore/Manycore systems
• Consistency problems
• Synchronization
• Data Streaming:
operator parallelism & scalability

Agenda
• Who we are
use-cases
• Conclusions

Motivation
Applications in sensor networks, cyber-physical systems:
• large and fluctuating volumes of data generated
continuously
demand for:
• Continuous processing of data streams
• In a real-time fashion
Store-then-process is not feasible
Main Memory
1 Data
Query Processing
3 Query
results
2 Query
Main Memory
Query Processing
Disk Main Memory
Query Processing
Continuous
Query
Data
Query
results

System Model
• Data Stream: unbounded sequence of tuples
– Example: Call Description Record (CDR)
time
Field Field
Caller text
Callee text
Time (secs) int
Price (€) double
A B 8:00 3 C D 8:20 7 A E 8:35 6
102015-10-31 Vincenzo Gulisano - Yiannis Nikolakopoulos

System Model
• Operators:
OP
Stateless
1 input tuple
1 output tuple
OP
Stateful
1+ input tuple(s)
1 output tuple

System Model
• Continuous Query: graph operators/streams
Convert
€  $
Only
> 10$
Count calls
made by each
Caller number
Map Filter Agg
Count the number of calls whose price is more than 10
dollars made by each caller

System Model
• Infinite sequence of tuples / bounded memory
 windows
• Example: 1 hour windows
time
[8:00,9:00)
[8:20,9:20)
[8:40,9:40)

System Model
• Infinite sequence of tuples / bounded memory
 windows
• Example: count tuples - 1 hour windows
time
[8:00,9:00)
8:05 8:15 8:22 8:45 9:05
Output: 4
14
[8:20,9:20)

The evolution of SPEs
Centralized
SPEs
Distributed
SPEs
Parallel-Distributed
SPEs
Elastic
SPEs
Inter-operator parallelism … …
Intra-operator parallelism
… …
+/- +/-
Scale up / down

Agenda
• Who we are
use-cases
• Conclusions

What is a stream join?
2015-10-31 17
t1
t2
t3
t4
t1
t2
t3
t4
R S
Sliding
window Window
size WS
WSWR
Predicate P

Why parallel stream joins?
• WS = 600 seconds
• R receives 500 tuples/second
• S receives 500 tuples/second
• WR will contain 300,000 tuples
• WS will contain 300,000 tuples
• Each new tuple from R gets compared with
all the tuples in WS
• Each new tuple from S gets compared with
all the tuples in WR
… 300,000,000 comparisons/second!
t1
t2
t3
t4
t1
t2
t3
t4
R S
WSWR
2015-10-31 18

Which are the challenges of a parallel stream join?
Scalability
High
throughput
Low latency
Disjoint
parallelism
Skew
resilience
Determinism
2015-10-31 19

The 3-step procedure (sequential stream join)
For each incoming tuple t:
1. compare t with all tuples in opposite window given predicate P
2. add t to its window
3. remove stale tuples from t’s window
Add tuples to S
Add tuples to R
Prod
R
Prod
S
Consume resultsConsPU
2015-10-31 20
We assume each
producer delivers tuples
in timestamp order

The 3-step procedure, is it enough?
Scalability
High
throughput
Low latency
Disjoint
parallelism
Skew
resilience
Determinism
2015-10-31 21
t1
t2
t1
t2
R S
WSWR
t3
t1
t2
t1
t2
R S
WSWR
t4
t3

Enforcing determinism in sequential stream joins
• Next tuple to process = earliest(tS,tR)
• The earliest(tS,tR) tuple is referred to as the next ready tuple
• Process ready tuples in timestamp order  Determinism
PU
tS tR
2015-10-31 22

Deterministic 3-step procedure
Pick the next ready tuple t:
1. compare t with all tuples in opposite window given predicate P
Add tuples to S
Add tuples to R
Prod
R
Prod
S
Consume resultsConsPU
2015-10-31 23

Shared-nothing parallel stream join
(state-of-the-art)
Prod
R
Prod
S
PU1
PU2
PUN
… Cons
Add tuple to PUi S
Add tuple to PUi R
Consume results
Pick the next ready tuple t:
1. compare t with all tuples in opposite window given P
Chose a PU
Chose a PU
Take the next
ready output tuple
Scalability
High
throughput
Low latency
Disjoint
parallelism
Skew
resilience
Determinism
2015-10-31 24
Merge

Shared-nothing parallel stream join
(state-of-the-art)
Prod
R
Prod
S
PU1
PU2
PUN
…
2015-10-31 25
enqueue()
dequeue()
ConsMerge

From coarse-grained to fine-grained synchronization
Prod
R
Prod
S
PU1
PU2
PUN
…
Cons
2015-10-31 26

ScaleGate
2015-10-31 27
addTuple(tuple,sourceID)
allows a tuple from sourceID to be merged by ScaleGate in the
resulting timestamp-sorted stream of ready tuples.
getNextReadyTuple(readerID)
provides to readerID the next earliest ready tuple that has not been
yet consumed by the former.
https://github.com/dcs-chalmers/ScaleGate_Java

ScaleJoin
Prod
R
Prod
S
PU1
PU2
PUN
…
Cons
Add tuple SGin
Add tuple SGin
Get next ready
output tuple
from SGout
Get next ready input tuple from SGin
2. add t to its window in a round-robin fashion
2015-10-31 28
SGin SGout
Steps for PU

2015-10-31 29
t1
t2
R S
WR
t3
t4
R S
t4
t1
WR
R S
t4
t2
WR
R S
t4
WR
t3
Sequential stream join:
ScaleJoin with 3 PUs:
ScaleJoin (example)

ScaleJoin
Prod
R
Prod
S
PU1
PU2
PUN
… Cons
Add tuple SGin
Add tuple SGin
Get next ready
output tuple
from SGout
2015-10-31 30
SGin SGout
Scalability
High
throughput
Low latency
Disjoint
parallelism
Skew
resilience
Determinism
Prod
S
Prod
S
Prod
R Get next ready input tuple from SGin
2. add t to its window in a round robin fashion
Steps for PUi

ScaleJoin Scalability – comparisons/second
2015-10-31 32
Number of PUs
4 billion comparison / second!!!

Agenda
• Who we are
use-cases
• Conclusions

2015-10-31 36
addTuple(tuple,sourceID)
allows a tuple from sourceID to be merged by ScaleGate in the
resulting timestamp-sorted stream of ready tuples.
getNextReadyTuple(readerID)
provides to readerID the next earliest ready tuple that has not been
yet consumed by the former.

ScaleGate: Main Functionality
• addTuple(tuple,sourceID)
– Insert tuple from sourceID
effectively in ts order
Vincenzo Gulisano - Yiannis Nikolakopoulos
<6,…>
<1,…>
<3,…>
Insert
concurrently

ScaleGate: Main Functionality
• getNextReadyTuple(readerID)
– readerID gets the next ready tuple t in ts order
i.e. no tuple with ts<t.ts will arrive afterwards
– return null if no ready tuple
<1,…>
Get ready
tuples
<6,…>
<3,…>
<7,…>
<9,…>
<8,…>

Implementation?
• getNextReadyTuple(readerID)
– we get tuples in ts order,
so maybe an extractMin()? (priority queue)
– still no guarantee that t is ready
• addTuple(tuple, sourceID)
– sorted insertion maybe could help
<6,…> <1,…><3,…>
Is there a concurrent data structure with
cheap extractMin() + insert()
Binary Search Trees,
Heaps:
Expensive rebalancing
and heap property

ScaleGate Anatomy (1)
• Inspired from
lock-free skip lists
– randomized height of nodes
– expected cost for search/insertion O(logN)
• Search by traversing from higher to lower levels
BigData’15
[GNPT]

ScaleGate Anatomy (2)
• Reader-local view of "head"
(also has minimum ts for that reader)
• Flagging mechanism:
– If "head" is not flagged can be safely returned
– Flag the last written tuple of each source
• Nodes free to be garbage-collected after every reader passes
(almost...)
head0 head1
BigData’15
[GNPT]

ScaleGate Properties
• Safety: Linearizable
– Every operation appears to take effect
instantaneously
• Progress: Lock-free
– At least one thread makes progress,
independently of the state of others
• Basic building block for providing
deterministic processing
(through the ready semantics)

Why not a Skip List?

Agenda
• Who we are
• ScaleJoin: a Deterministic, Disjoint-Parallel and Skew-
Resilient Stream Join
use-cases
– Streaming Aggregation
– Towards real-time analytics
– Some ongoing work
• Conclusions

Timestamp sorted
Multi-way Streaming Aggregation
1 5
Aggregate F
3 7
5 9
<6,…>
<1,…>
<3,…>
When to
process a tuple?
Tuples not in TS order
across streams;
not ready for
processing at arrival…
Merge & sort
streams
Timestamp
sorted tuples
per stream

Multi-way Streaming Aggregation
(baseline [Streamcloud, Borealis])
Aggregate F:
Update
windows and
output<6,…>
<1,…>
<3,…>
When to
process a tuple?
Keep checking
all buffers!!!
Merge & sort
streamsCan Data
Structures do
more than insert
and extract?

Data Structures to the Rescue
<6,…>
<1,…>
<3,…>
Insert
concurrently
Concurrently
update
windows
Get ready
tuples
Manage
sorted
windows
Output
tuples
ScaleGate objects allow concurrent access with
- consistency/safety , progress guarantees
- determinism
- appropriate interface  parallelization of pipeline stages
Contribution:
SPAA’14 [CGNPT]
“Brief announcement:
concurrent data structures for
efficient streaming aggregation”

Aggregate Scaling

Latency with Increasing
# Input Streams

Best Solution Award 
Analyze taxi trip reports from NYC and compute:
ACM DEBS 2015 [GNWPT]
“Deterministic Real-Time
Analytics of Geospatial Data
Streams through ScaleGate
Objects”

System Architecture Overview

• Networks on chip (NoC)
• Short distance
between cores
• Message passing
model support
• Shared memory support
Can we have Data Structures:
Fast
Scalable
Good progress guarantees
Cache Cache
IA Core
Shared Local
• Eliminated
cache coherency
• Limited support for
synchronization
primitives
What’s happening in hardware?

Off-chip DRAM
Adapteva’s Epiphany Architecture:
16 or 64 cores
core
core
On-chip
32kb per
core
core
core core
core
Cores
communicate
through mesh
by reading
and writing
the on-chip
memory
Distributed
shared memory

Ongoing work with Ericsson:
ScaleGate on Epiphany
• Task-graph based processing
• (Baseband) signal processing applications
Can ScaleGate help?

Agenda
• Who we are
• Fine-grained synchronization and other use-cases
• Conclusions

Source: http://storm.apache.org/
… …
Intra-operator parallelism
Storm is a parallel-distributed
Stream Processing Engine
• Data streaming operators
 Spout / Bolt
• Splitting of work into tasks / workers
 Intra-operator parallelism

Parallelization
• General approach
LB: Load Balancer
IM: Input Merger
OPA OPB
OPA LBIM
Node 1
OPA LBIM
Node m
…
Subcluster A
OPB LBIM
Node 1
OPB LBIM
Node n
…
Subcluster B

Parallelization
• Stateful operators: Semantic awareness
– Aggregate: count within last hour, group-by caller number
Previous Subcluster
LB…
LB…
IM Agg1
IM Agg2
IM Agg3
…
…
…
Caller A
A B 8:00 3
58
A C 8:30 5
A D 9:05 2

Parallelization
Previous Subcluster
LB…
LB…
IM Agg1
IM Agg2
IM Agg3
…
…
…
Fields grouping
User defined

On-going research
Source: http://www.michael-noll.com/blog/2013/06/21/understanding-storm-internal-message-buffers/
Coarse-grained
synchronization!!!
• Individual queues for
executors
• 2 threads per executor
• Dedicated Receive and Send
thread queues…
… What about fine-grained
synchronization?

Agenda
• Who we are
use-cases
• Conclusions

1.Parallel stream joins
2.Parallel stream aggregation
3.“Ad-hoc” streaming applications
4.Determinism in Storm
…
https://github.com/dcs-chalmers/ScaleGate_Java

The benefits of fine-grained synchronization in
deterministic and efficient stream processing
Vincenzo Gulisano, Yiannis Nikolakopoulos
Chalmers University
of technology
Thank you! Questions?

References
• Vincenzo Gulisano, Yiannis Nikolakopoulos, Marina Papatriantafilou, and
Philippas Tsigas, “ScaleJoin: a Deterministic, Disjoint-Parallel and Skew-
Resilient Stream Join”, to appear in IEEE BigData 2015
• Vincenzo Gulisano, Yiannis Nikolakopoulos, Ivan Walulya, Marina
Papatriantafilou, Philippas Tsigas, “Deterministic Real-time Analytics of
Geospatial Data Streams Through ScaleGate Objects”, 9th ACM
International Conference on Distributed Event-Based Systems (DEBS '15)
• Yiannis Nikolakopoulos, Anders Gidenstam, Marina Papatriantafilou,
Philippas Tsigas “A Consistency Framework for Iteration Operations in
Concurrent Data Structures”, 2015 IEEE 29th International Symposium on
Parallel & Distributed Processing (IPDPS)
• Daniel Cederman, Vincenzo Gulisano, Yiannis Nikolakopoulos, Marina
Papatriantafilou, and Philippas Tsigas “Brief announcement: concurrent
data structures for efficient streaming aggregation”, 2014 26th ACM
symposium on Parallelism in algorithms and architectures (SPAA '14)

The benefits of fine-grained synchronization in deterministic and efficient stream processing

The benefits of fine-grained synchronization in deterministic and efficient stream processing

Recommended

More Related Content

Similar to The benefits of fine-grained synchronization in deterministic and efficient stream processing

Similar to The benefits of fine-grained synchronization in deterministic and efficient stream processing(20)

Recently uploaded

Recently uploaded(20)

The benefits of fine-grained synchronization in deterministic and efficient stream processing

Editor's Notes