Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Play With Streams
chentianjian@baidu.com
Jun, 2014
About Me
• Principal Architect
@Baidu.com
• Contract Programmer
– C/C++/Python
• Post Engineering
Disorder Therapist
Post Engineering Disorder
Murphy’s Law
• Everything that Can
Go Wrong, Goes
Wrong
– Unstable Servers
– Unstable Network
– Unstable Data Source
– Uns...
Agenda
• Section I: A Scratch
– Brief Intro to Streams
• Section II: Build From Ground Up
– Modern Stream Processing Archi...
Section I: A Scratch
Brief Intro to Streams
Highlights
• Natural Beauty of Streams
• Stream Processing Design Language(SDL)
• Scratching Stream Applications
Streams are Everywhere
Definition of Stream
• A Series Of Data Packs
• Data is Structured or Semi-Structured
• With Internal Topologies
– DAG in ...
Traffic Cam Network
Wireless Sensor Network
Recommender System
Stream Description Language
• Normalize
• Accumulate
• Join
• Partition
• Filtering
Normalize
NORMMessage
Type A
Message
Type B
Standard
Message
Accumulate
ACCU
BUFFER
Message Accumulated
Buffers
Common Accumulator Types
• Time Window
• Volume
• Condition Trigger
Join
JOIN
Indexed Info.
Message
With a key
Joined Message
Common Join Rules
• Internal Mode
– Indexing Only Input Messages
– Limited Time Window
• External Mode
– Indexing External...
Partition
PARTMessage
With a key
Common Partition Rules
• Feature Based
– Application Related
• Random
– Balancing Load
• Hash
– Aggregating Data by User D...
Filtering
FILT
Drop
Pass ThroughInput Messages
Pit Stop
• Normalize
• Accumulate
• Join
• Partition
• Filtering
Distinct Count
A Simple Hello World:
Word Count
Text Parser ACCU
An Improved Version
Text Parser PART ACCU
ACCU
ACCU
NORM
Sum Up!
• Streams are everywhere
• We have a modeling tool to describe stream
processing flows(SDL)
• Proceed to build rea...
See U In Next Hour
Section II: Build From Ground Up
Modern Stream Processing
Architecture
Highlights
• Flow Means Non-Stopping
• Reliability Matters
• Workload Fluctuation Handling
Message Relay System Basics
• Message Queuing
• Message Passing
• Hybrid Queuing
With Message Queuing
Operator 1
Queue A Queue B Queue C
Operator 2
Message Passing
Operator 1
Operator 2
Operator 3
Hybrid Queuing
Operator 1
Queue A Queue B
Operator 2
Operator 3
Pit Stop
• Message relay is the foundation of stream
processing
• 3 basic methods of message relay
Why Reliability Matters?
• We got mission critical applications
– Real time stock exchange analysis
– Ads network click mo...
Reliability Solutions
• Upstream Backup & Replay
• Source Backup & Replay
• Processing Status Backup & Replay
Upstream Backup & Replay
Upstream Downstream Upstream Downstream
ACK ACK
Source Backup & Replay
Upstream Downstream Upstream Downstream
ACK
ACK
Processing Status Backup & Replay
Upstream
Downstream Upstream
Downstream
Stream Operator
Downstream Upstream
Stream Opera...
While Reliability Hurts Performance
• All Reliability Solutions are Based on
– Indexing
– Snapshot
– Replay
• A Club Of Pe...
Tuning Strategies
• State Operators VS. Stateless Operators
– Independent State Storage
– White Board Programming Model
– ...
Micro Batching
Batch Size Time Window Throughput Snapshot Cost Restore Cost
1 1ms 1x very high very low
10 10ms 10x high l...
Pit Stop
• Reliability is based on
– Message Backup & Replay
– Status Snapshot
• When tuning, think of
– How to handle ope...
Workload Fluctuation
How Does Fluctuation Happen?
• Data Source Fluctuation
• Fault Tolerance Operations
Fluctuation Handling
• Technologies To Obtain
– High Performance RPC Framework
– Auto Partitioning
– Dynamic Resource Allo...
High Performance RPC Framework
• Indication of High Performance
– Over 20k QPS/sec, with 1byte payload
– On commodity serv...
Auto Partitioning
• Adjust Sub-Stream Num.
• Tricks:
– Consistent Hashing
– Exponential Allocation
Dynamic Resource Allocation
• Tree Model
• Evaluation
Function
• Network
Constrains
IDC
A
IDC
B
PHY
1
PHY
2
PHY
3
PHY
4
VM...
Global Flow Control
Op1
Op2
Op3
Op4
Flow Control
Service
Pit Stop
• Workload Fluctuation Handling
– Very Fast RPC Framework
– Auto Partitioning
– Dynamic Resource Allocation
– Glo...
Sum Up!
• Basic Architecture of Stream Processing
System
• Reliability of Stream Processing
• Ways to Handle Workload Fluc...
See U in the Afternoon
Section III:
Could Be Much Sexier
Stream Evolution in Progress
Highlights
• Challenges in Real World Applications
• High Level Stream Programming
• Optimization Inside Hardware
Applications
• Stream Web Crawler
• Co Serving with Hadoop M/R
Stream Web Crawling
OP1
OP2
Redis Cluster
Web Page Cache
OP3 OP4
Log Filter
Data Join
Feature
Extraction
Logging API
Crawl...
Co-Serving
• Distribute RPC
• Dynamic
Routing
OP1
OP2
OP5
OP6
Online Web Services
Query
Preprocess
Query
Transform
Result
...
Pit Stop
• Programming Complexity
• Network Management Complexity
• Scale-out Difficulty
High Level Programming
• Stream DataBase & StreamSQL
• Stream Computing Description Language
• Stream Programming Framework
StreamSQL
Stream DML Stack(Naiad)
Stream Framework
Pit Stop
Programming Interface Complexity Flexibility
Stream SQL Low Low
Stream DML Medium High
Stream Framework High Very...
Network Management Challenges
• Things Are App. Dependent
– QoS Request
– Relay Priority
– Security Strategy
– Resource Al...
Network Management Solution
SDN
Integration
Stream App Flow Control
Network Configuration
SDN Integration
Stream Flow
Control Service
Monitors on
Streams
Open Flow
Supported
Network
Devices
Pit Stop
• Network Management is Crucial
– Stream Processing is Bandwidth Consuming
– Stream Processing may be Latency Sen...
Scale-Out Difficulty
• Application Constrains
• Algorithm Constrains
• Network Hardware Constrains
So Scale-Up !
• Co-Process Device
• Re-Programmable Co-Processors
FPGA Computing Card
• Cheaper than GPGPU
• Flexible than GPGPU
• Higher POE than GPGPU
New Re-Configurable Xeon CPU
See Also
• A Reconfigurable Fabric for Accelerating Large-
Scale Datacenter Services
– 20x Performance
– ISCA 2014 by Micr...
Pit Stop
• Scale-out is difficult, think of scale-up
• Reconfigurable CPU has got significant
performance improvements
Conclusion
• Stream Processing System can be Well
Modeled by SDL
• Trade Off between Reliability & Performance
• High leve...
References
• Stonebraker, Michael, Uǧur Çetintemel, and Stan Zdonik. "The 8 requirements of real-time
stream processing." ...
Thank U!
• G+: chentianjian@gmail.com
• Skype: tianjian_chen
• Linkedin: http://lnkd.in/bRN6xsh
Play With Streams
Upcoming SlideShare
Loading in …5
×

Play With Streams

405 views

Published on

This tutorial gives out an brief and interesting introduction to modern stream computing technologies. The participants can learn the essential concepts and methodologies for designing and building a advanced stream processing system. The tutorial unveils the key fundamentals behind various kinds of design choices. Some forecast of technology developments in this domain is also introduced at the last section of this tutorial.

Published in: Engineering, Technology
  • Be the first to comment

  • Be the first to like this

Play With Streams

  1. 1. Play With Streams chentianjian@baidu.com Jun, 2014
  2. 2. About Me • Principal Architect @Baidu.com • Contract Programmer – C/C++/Python • Post Engineering Disorder Therapist
  3. 3. Post Engineering Disorder
  4. 4. Murphy’s Law • Everything that Can Go Wrong, Goes Wrong – Unstable Servers – Unstable Network – Unstable Data Source – Unstable Managers – Unstable…
  5. 5. Agenda • Section I: A Scratch – Brief Intro to Streams • Section II: Build From Ground Up – Modern Stream Processing Architecture • Section III: Could Be Much Sexier – Stream Evolution in Progress
  6. 6. Section I: A Scratch Brief Intro to Streams
  7. 7. Highlights • Natural Beauty of Streams • Stream Processing Design Language(SDL) • Scratching Stream Applications
  8. 8. Streams are Everywhere
  9. 9. Definition of Stream • A Series Of Data Packs • Data is Structured or Semi-Structured • With Internal Topologies – DAG in most cases
  10. 10. Traffic Cam Network
  11. 11. Wireless Sensor Network
  12. 12. Recommender System
  13. 13. Stream Description Language • Normalize • Accumulate • Join • Partition • Filtering
  14. 14. Normalize NORMMessage Type A Message Type B Standard Message
  15. 15. Accumulate ACCU BUFFER Message Accumulated Buffers
  16. 16. Common Accumulator Types • Time Window • Volume • Condition Trigger
  17. 17. Join JOIN Indexed Info. Message With a key Joined Message
  18. 18. Common Join Rules • Internal Mode – Indexing Only Input Messages – Limited Time Window • External Mode – Indexing External Data – Extreme Useful In Integrating Asynchronous Components
  19. 19. Partition PARTMessage With a key
  20. 20. Common Partition Rules • Feature Based – Application Related • Random – Balancing Load • Hash – Aggregating Data by User Defined Key • Replication – For Improving Availability
  21. 21. Filtering FILT Drop Pass ThroughInput Messages
  22. 22. Pit Stop • Normalize • Accumulate • Join • Partition • Filtering
  23. 23. Distinct Count
  24. 24. A Simple Hello World: Word Count Text Parser ACCU
  25. 25. An Improved Version Text Parser PART ACCU ACCU ACCU NORM
  26. 26. Sum Up! • Streams are everywhere • We have a modeling tool to describe stream processing flows(SDL) • Proceed to build real systems
  27. 27. See U In Next Hour
  28. 28. Section II: Build From Ground Up Modern Stream Processing Architecture
  29. 29. Highlights • Flow Means Non-Stopping • Reliability Matters • Workload Fluctuation Handling
  30. 30. Message Relay System Basics • Message Queuing • Message Passing • Hybrid Queuing
  31. 31. With Message Queuing Operator 1 Queue A Queue B Queue C Operator 2
  32. 32. Message Passing Operator 1 Operator 2 Operator 3
  33. 33. Hybrid Queuing Operator 1 Queue A Queue B Operator 2 Operator 3
  34. 34. Pit Stop • Message relay is the foundation of stream processing • 3 basic methods of message relay
  35. 35. Why Reliability Matters? • We got mission critical applications – Real time stock exchange analysis – Ads network click monitoring & billing
  36. 36. Reliability Solutions • Upstream Backup & Replay • Source Backup & Replay • Processing Status Backup & Replay
  37. 37. Upstream Backup & Replay Upstream Downstream Upstream Downstream ACK ACK
  38. 38. Source Backup & Replay Upstream Downstream Upstream Downstream ACK ACK
  39. 39. Processing Status Backup & Replay Upstream Downstream Upstream Downstream Stream Operator Downstream Upstream Stream Operator Shadow Status Synchronize Messages Redirect Redirect
  40. 40. While Reliability Hurts Performance • All Reliability Solutions are Based on – Indexing – Snapshot – Replay • A Club Of Performance Penalties
  41. 41. Tuning Strategies • State Operators VS. Stateless Operators – Independent State Storage – White Board Programming Model – Lazy State Synchronization • Micro Batching Snapshot
  42. 42. Micro Batching Batch Size Time Window Throughput Snapshot Cost Restore Cost 1 1ms 1x very high very low 10 10ms 10x high low 100 100ms 100x medium low 1000 100ms 1000x medium low 10000 1s 5000x low high Most Systems Are Here May Constrained By Network Configuration
  43. 43. Pit Stop • Reliability is based on – Message Backup & Replay – Status Snapshot • When tuning, think of – How to handle operator status – Micro Batching
  44. 44. Workload Fluctuation
  45. 45. How Does Fluctuation Happen? • Data Source Fluctuation • Fault Tolerance Operations
  46. 46. Fluctuation Handling • Technologies To Obtain – High Performance RPC Framework – Auto Partitioning – Dynamic Resource Allocation – Global Flow Control
  47. 47. High Performance RPC Framework • Indication of High Performance – Over 20k QPS/sec, with 1byte payload – On commodity server with 2 6-core CPU and Giga Ethernet • See Also – SOFA Framework from Baidu.com – https://github.com/BaiduPS/sofa-pbrpc
  48. 48. Auto Partitioning • Adjust Sub-Stream Num. • Tricks: – Consistent Hashing – Exponential Allocation
  49. 49. Dynamic Resource Allocation • Tree Model • Evaluation Function • Network Constrains IDC A IDC B PHY 1 PHY 2 PHY 3 PHY 4 VM 1 VM 2 VM 3
  50. 50. Global Flow Control Op1 Op2 Op3 Op4 Flow Control Service
  51. 51. Pit Stop • Workload Fluctuation Handling – Very Fast RPC Framework – Auto Partitioning – Dynamic Resource Allocation – Global Flow Control
  52. 52. Sum Up! • Basic Architecture of Stream Processing System • Reliability of Stream Processing • Ways to Handle Workload Fluctuations
  53. 53. See U in the Afternoon
  54. 54. Section III: Could Be Much Sexier Stream Evolution in Progress
  55. 55. Highlights • Challenges in Real World Applications • High Level Stream Programming • Optimization Inside Hardware
  56. 56. Applications • Stream Web Crawler • Co Serving with Hadoop M/R
  57. 57. Stream Web Crawling OP1 OP2 Redis Cluster Web Page Cache OP3 OP4 Log Filter Data Join Feature Extraction Logging API Crawling Job Updater OP5 User-Model Updater OP6HBase Cluster Web DataBase Crawling Job Generator OP7 Crawling Bot OP8 Image Crawling Job Generator OP9 Cache Synchronizer OP 10 Image Crawling Cluster Status DB
  58. 58. Co-Serving • Distribute RPC • Dynamic Routing OP1 OP2 OP5 OP6 Online Web Services Query Preprocess Query Transform Result Merge Intent Extraction Online Query Log OP3 OP4 User Intent Mining Hadoop Initiating DRPC from Mapper Mapper Receiving DRPC result
  59. 59. Pit Stop • Programming Complexity • Network Management Complexity • Scale-out Difficulty
  60. 60. High Level Programming • Stream DataBase & StreamSQL • Stream Computing Description Language • Stream Programming Framework
  61. 61. StreamSQL
  62. 62. Stream DML Stack(Naiad)
  63. 63. Stream Framework
  64. 64. Pit Stop Programming Interface Complexity Flexibility Stream SQL Low Low Stream DML Medium High Stream Framework High Very High
  65. 65. Network Management Challenges • Things Are App. Dependent – QoS Request – Relay Priority – Security Strategy – Resource Allocation
  66. 66. Network Management Solution SDN Integration Stream App Flow Control Network Configuration
  67. 67. SDN Integration Stream Flow Control Service Monitors on Streams Open Flow Supported Network Devices
  68. 68. Pit Stop • Network Management is Crucial – Stream Processing is Bandwidth Consuming – Stream Processing may be Latency Sensitive • Solution is Simple – Software Defined Network Integration (SDN)
  69. 69. Scale-Out Difficulty • Application Constrains • Algorithm Constrains • Network Hardware Constrains
  70. 70. So Scale-Up ! • Co-Process Device • Re-Programmable Co-Processors
  71. 71. FPGA Computing Card • Cheaper than GPGPU • Flexible than GPGPU • Higher POE than GPGPU
  72. 72. New Re-Configurable Xeon CPU
  73. 73. See Also • A Reconfigurable Fabric for Accelerating Large- Scale Datacenter Services – 20x Performance – ISCA 2014 by Microsoft Research – http://research.microsoft.com/pubs/212001/Cata pult_ISCA_2014.pdf
  74. 74. Pit Stop • Scale-out is difficult, think of scale-up • Reconfigurable CPU has got significant performance improvements
  75. 75. Conclusion • Stream Processing System can be Well Modeled by SDL • Trade Off between Reliability & Performance • High level programming & Scale-Up are Future Trends
  76. 76. References • Stonebraker, Michael, Uǧur Çetintemel, and Stan Zdonik. "The 8 requirements of real-time stream processing." ACM SIGMOD Record 34, no. 4 (2005): 42-47. • Zaharia, Matei, Tathagata Das, Haoyuan Li, Timothy Hunter, Scott Shenker, and Ion Stoica. "Discretized streams: Fault-tolerant streaming computation at scale." In Proceedings of the Twenty-Fourth ACM Symposium on Operating Systems Principles, pp. 423-438. ACM, 2013. • Murray, Derek G., Frank McSherry, Rebecca Isaacs, Michael Isard, Paul Barham, and Martin Abadi. "Naiad: a timely dataflow system." In Proceedings of the Twenty-Fourth ACM Symposium on Operating Systems Principles, pp. 439-455. ACM, 2013. • Castro Fernandez, Raul, Matteo Migliavacca, Evangelia Kalyvianaki, and Peter Pietzuch. "Integrating scale out and fault tolerance in stream processing using operator state management." In Proceedings of the 2013 international conference on Management of data, pp. 725-736. ACM, 2013.
  77. 77. Thank U! • G+: chentianjian@gmail.com • Skype: tianjian_chen • Linkedin: http://lnkd.in/bRN6xsh

×