Tutorial - Modern Real Time Streaming Architectures
•
22 likes•2,788 views
Across diverse segments in industry, there has been a shift in focus from big data to fast data, stemming, in part, from the deluge of high-velocity data streams as well as the need for instant data-driven insights, and there has been a proliferation of messaging and streaming frameworks that enterprises utilize to satisfy the needs of various applications. Drawing on their experience operating streaming systems at Twitter scale, Karthik Ramasamy, Sanjeev Kulkarni, Arun Kejariwal, and Sijie Guo walk you through state-of-the-art streaming architectures, streaming frameworks, and streaming algorithms, covering the typical challenges in modern real-time big data platforms and offering insights on how to address them. They also discuss how advances in technology might impact the streaming architectures and applications of the future. Along the way, they explore the interplay between storage and stream processing and speculate about future developments. Topics include: Basic requirements of stream processing Streaming and one-pass algorithms Different types of streaming architectures An in-depth review of streaming frameworks Deploying and operating stream processing applications Lessons learned from building a real-time stack using Apache Pulsar and Apache Heron at Twitter Scale
Recommended
More Related Content
What's hot
What's hot (20)
Similar to Tutorial - Modern Real Time Streaming Architectures
Similar to Tutorial - Modern Real Time Streaming Architectures (20)
More from Karthik Ramasamy
More from Karthik Ramasamy (10)
Recently uploaded
Recently uploaded (20)
Tutorial - Modern Real Time Streaming Architectures
- 1. A Tutorial Modern Real-Time Streaming Architectures Karthik Ramasamy*, Sanjeev Kulkarni*, Arun Kejariwal^ & Sijie Guo* *Streamlio, ^MZ
- 2. 2 MESSAGING STREAMING DATA SKETECHES LAMBDA, KAPPA UNIFICATION OUTLINE
- 4. 4 Increasingly Connected World Internet of Things 30 B connected devices by 2020 Health Care 153 Exabytes (2013) -> 2314 Exabytes (2020) Machine Data 40% of digital universe by 2020 Connected Vehicles Data transferred per vehicle per month 4 MB -> 5 GB Digital Assistants (Predictive Analytics) $2B (2012) -> $6.5B (2019) [1] Siri/Cortana/Google Now Augmented/Virtual Reality $150B by 2020 [2] Oculus/HoloLens/Magic Leap Ñ !+ >
- 7. 7 The New Era: Streaming Data/Fast Data Events are analyzed and processed in real-<me as they drive Decisions are <mely, contextual and based on fresh data Decision latency is eliminated Data in mo<on
- 8. Use Cases
- 10. 10 Product Safety - System Overview KV store for clustering Messaging System (Event Bus) Similarity Clustering (Heron)
- 13. 13 Real Time Use Cases Algorithmic trading Online fraud detec<on Geo fencing Proximity/loca<on tracking Intrusion detec<on systems Traffic management Real <me recommenda<ons Churn detec<on Internet of things Social media/data analy<cs Gaming data feed
- 16. 16 State of the World Aggregation Systems Messaging Systems Result Engine HDFS Queryable Engines
- 17. 17 Towards Unification and Simplification Interactive Querying Storm API Streamlets SQL Application Builder Pulsar API BK/ HDFS API Kubernetes Metadata Management Operational Monitoring Chargeback Security Authentication Quota Management Kafka API
- 18. 18 Real Time Stack REAL TIME STACK Collectors s Compute J Messaging a Storage b
- 19. 19 of MESSAGING FRAMEWORKS An In-Depth
- 20. Apache Pulsar
- 21. 21 Current Messaging Systems 01 02 03 04 05 06 07 08 ActiveMQ RabbitMQ Pulsar RocketMQ Azure Event Hub Google Pub-Sub Satori Kafka
- 24. 24 Pulsar Producer PulsarClient client = PulsarClient.create( “http://broker.usw.example.com:8080”); Producer producer = client.createProducer( “persistent://my-property/us-west/my-namespace/my-topic”); // handles retries in case of failure producer.send("my-message".getBytes()); // Async version: producer.sendAsync("my-message".getBytes()).thenRun(() -> { // Message was persisted });
- 25. 25 Pulsar Consumer PulsarClient client = PulsarClient.create( "http://broker.usw.example.com:8080"); Consumer consumer = client.subscribe( "persistent://my-property/us-west/my-namespace/my-topic", "my-subscription-name"); while (true) { // Wait for a message Message msg = consumer.receive(); System.out.println("Received message: " + msg.getData()); // Acknowledge the message so that it can be deleted by broker consumer.acknowledge(msg); }
- 26. 26 Pulsar Architecture Separa@on of Storage and Serving SERVING Brokers can be added independently Traffic can be shifted quickly across brokers STORAGE Bookies can be added independently New bookies will ramp up traffic quickly
- 28. 28 Broker Failure Recovery • Topic is reassigned to an available broker based on load • Can reconstruct the previous state consistently • No data needs to be copied • Failover handled transparently by client library
- 29. 29 Bookie Failure Recovery • After a write failure, BookKeeper will immediately switch write to a new bookie, within the same segment.
- 30. 30 Bookie Failure Recovery • In background, starts a many-to-many recovery process to regain the configured replication factor
- 31. 31 Pulsar Architecture Geo Replica@on GEO REPLICATION Asynchronous replication Integrated in the broker message flow Simple configuration to add/remove regions Topic (T1) Topic (T1) Topic (T1) Subscrip<on (S1) Subscrip<on (S1) Producer (P1) Consumer (C1) Producer (P3) Producer (P2) Consumer (C2) Data Center A Data Center B Data Center C
- 32. 32 Pulsar Use Cases - Message Queue Online Events Topic (T) Worker 1 Worker 2 Decouple Online/Offline Topic (T) Worker 3 MESSAGE QUEUES Decouple online or background High availability Reliable data transport No@fica@ons Long running tasks Low latency publish
- 33. 33 Pulsar Use Cases - Feedback System Event Topic (T) Propagate States Controller Topic (T) Serving System Serving System Serving System FEEDBACK SYSTEM Coordinate large number of machines Propagate states Examples State propagation Personalization Ad-systems Feedback Updates
- 36. 36 of STREAMING FRAMEWORKS An In-Depth
- 37. 37 Current Streaming Frameworks 01 02 03 04 05 06 07 08 Beam S-Store Spark Flink Heron Storm Apex KAFKA STREAMS
- 38. Apache Heron
- 40. 40 Heron Topology % % % % % Spout 1 Spout 2 Bolt 1 Bolt 2 Bolt 3 Bolt 4 Bolt 5
- 41. 41 Heron Topology - Physical Execution % % % % % Spout 1 Spout 2 Bolt 1 Bolt 2 Bolt 3 Bolt 4 Bolt 5 %% %% %% %% %%
- 42. 42 Heron Groupings 01 02 03 04 Shuffle Grouping Random distribution of tuples Fields Grouping Group tuples by a field or multiple fields All Grouping Replicates tuples to all tasks Global Grouping Send the entire stream to one task / . - ,
- 43. 43 Heron Topology - Physical Execution % % % % % Spout 1 Spout 2 Bolt 1 Bolt 2 Bolt 3 Bolt 4 Bolt 5 %% %% %% %% %% Shuffle Grouping Shuffle Grouping Fields Grouping Fields Grouping Fields Grouping Fields Grouping
- 44. 44 Writing Heron Topologies Procedural - Low Level API Directly write your spouts and bolts Functional - Mid Level API Use of maps, flat maps, transform, windows Declarative - SQL (coming) Use of declara<ve language - specify what you want, system will figure it out. , %
- 45. 45 Streamlet Functional API Builder.newBuilder() .newSource(() -> StreamletUtils.randomFromList(SENTENCES)) .flatMap(sentence -> Arrays.asList(sentence.toLowerCase().split("s+"))) .reduceByKeyAndWindow(word -> word, word -> 1, WindowConfig.TumblingCountWindow(50), (x, y) -> x + y);
- 47. 47 Heron Architecture Scheduler Topology 1 Topology 2 Topology N Topology Submission
- 48. 48 Topology Master Monitoring of containers Gateway for metrics Assigns role
- 49. 49 Topology Architecture Topology Master ZK Cluster Stream Manager I1 I2 I3 I4 Stream Manager I1 I2 I3 I4 Logical Plan, Physical Plan and Execution State Sync Physical Plan DATA CONTAINER DATA CONTAINER Metrics Manager Metrics Manager Metrics Manager Health Manager MASTER CONTAINER
- 50. 50 Stream Manager Routes tuples Implements backpressure Ack management
- 51. 51 Stream Manager Sample topology % % S1 B2 B3 % B4
- 52. 52 Stream Manager Physical execu@on S1 B2 B3 Stream Manager Stream Manager Stream Manager Stream Manager S1 B2 B3 B4 S1 B2 B3 S1 B2 B3 B4 B4
- 53. 53 Stream Manager Backpressure Spout based back pressureTCP backpressure Stage by stage back pressure
- 54. 54 Stream Manager Backpressure TCP based backpressure Slows upstream and downstream instances S1 B2 B3 Stream Manager Stream Manager Stream Manager Stream Manager S1 B2 B3 B4 S1 B2 B3 S1 B2 B3 B4 B4
- 55. 55 Stream Manager Backpressure Spout based backpressure S1 S1 S1S1S1 S1 S1S1 B2 B3 Stream Manager Stream Manager Stream Manager Stream Manager B2 B3 B4 B2 B3 B2 B3 B4 B4
- 56. 56 Heron Instance Runs only one task (spout/bolt) Exposes Heron API Collects several metrics API G
- 57. 57 Heron Instance Stream Manager Metrics Manager Gateway Thread Task Execution Thread data-in queue data-out queue metrics-out queue
- 58. 58 Heron Deployment Topology 1 Topology 2 Topology N Heron Tracker Heron VIZ Heron Web ZK Cluster Aurora Services Observability
- 60. 60 Heron @Twitter LARGEST CLUSTER 100’s of TOPOLOGIES BILLIONS OF MESSAGES100’s OF TERABYTESREDUCED INCIDENTS GOOD NIGHT SLEEP 3X - 5X reduction in resource usage
- 62. 62 AND of STREAM PROCESSING APPLICATIONS
- 64. 64 Reacting to Failures - Brokers Brokers don’t have durable state Easily replaceable Topics are immediately reassigned to healthy brokers Expanding capacity Simply add new broker node If other brokers are overloaded, traffic will be automa<cally assigned Load manager Monitor traffic load on all brokers (CPU, memory, network, topics) Ini<ally place topics to least loaded brokers Reassign topics when a broker is overloaded
- 65. 65 Reacting to Failures - Bookies When a bookie fails, brokers will immediately con<nue on other bookies Auto-Recovery mechanism will re-establish the replica<on factor in background If a bookie keeps giving errors or <meouts, it will be “quaran<ned” Not considered for new ledgers for some period of <me
- 70. 70 Heron Topology Scale CONTAINERS - 1 TO 600 INSTANCES - 10 TO 6000
- 71. 71 Heron Happy Facts :) ! No more pages during midnight for Heron team " Very rare incidents for Heron customer teams # Easy to debug during incident for quick turn around $ Reduced resource u<liza<on saving cost
- 72. 72 Heron Developer Issues 01 02 Container resource alloca@on Parallelism tuning
- 73. 73 Heron Operational Issues 01 02 03 Slow Hosts Network Issues Data Skew / . - 04 Load Variations , 05 SLA Violations /
- 74. 74 Slow Hosts Memory Parity Errors Impending Disk Failures Lower GHZ
- 75. 75 Network Issues Network Slowness Network Partitioning G
- 76. 76 Network Slowness 01 02 03 Delays processing Data is accumula@ng Timelines of results Is affected
- 77. 77 Data Skew Multiple Keys Several keys map into single instance and their count is high Single Key Single key maps into a instance and its count is high H C
- 79. 79 Self Regulating Streaming Systems Automate Tuning SLO Maintenance Self Regula@ng Streaming Systems Tuning Manual, <me-consuming and error- prone task of tuning various systems knobs to achieve SLOs SLO Maintenance of SLOs in the face of unpredictable load varia<ons and hardware or sobware performance degrada<on Self Regula@ng Streaming Systems System that adjusts itself to the environmental changes and con<nue to produce results
- 80. 80 Self Regulating Streaming Systems Self tuning Self stabilizing Self healing G "g Several tuning knobs Time consuming tuning phase The system should take as input an SLO and automatically configure the knobs. The system should react to external shocks a n d a u t o m a t i c a l l y reconfigure itself Stream jobs are long running Load variations are common The system should identify internal faults and attempt to recover from them System performance affected by hardware or software delivering degraded quality of service
- 81. 81 Enter Dhalion Dhalion periodically executes well-specified policies that optimize execution based on some objective. We created policies that dynamically provision resources in the presence of load variations and auto-tune streaming applications so that a throughput SLO is met. Dhalion is a policy based framework integrated into Heron
- 82. 82 Dhalion Policy Framework Symptom Detector 1 Symptom Detector 2 Symptom Detector 3 Symptom Detector N .... Diagnoser 1 Diagnoser 2 Diagnoser M .... Resolver Invocation D iagnosis 1 Diagnosis 2 D iagnosis M Symptom 1 Symptom 2 Symptom 3 Symptom N Symptom Detection Diagnosis Generation Resolution Resolver 1 Resolver 2 Resolver M .... Resolver Selection Metrics
- 83. 83 Dynamic Resource Provisioning Policy This policy reacts to unexpected load varia<ons (workload spikes) Goal Goal is to scale up and scale down the topology resources as needed - while keeping the topology in a steady state where back pressure is not observed H C Policy
- 84. 84 Dynamic Resource Provisioning Pending Tuples Detector Backpressure Detector Processing Rate Skew Detector Resource Over provisioning Diagnoser Resource Under Provisioning Diagnoser Data Skew Diagnoser Resolver Invocation Diagnosis Symptoms Symptom Detection Diagnosis Generation Resolution Metrics Slow Instances Diagnoser Bolt Scale Down Resolver Bolt Scale Up Resolver Data Skew Resolver Restart Instances Resolver Implementa@on
- 90. HIGH VELOCITY DATA 90 KEY CHARACTERISTICS In-Order O(1) Storage O(n) Time ONE PASS
- 91. 91 DISTRIBUTED COMPUTATION SCALE OUT ROBUST CHARACTERISTICS FAULT TOLERANCE KEY
- 92. 92 DATA SKETCHES Early work The space complexity of approximating the frequency moments Counting Frequent Elements [Misra and Gries, 1982] Flajolet and Martin 1985] Computing on Data Streams [Henzinger et al. 1998] [Alon et al. 1996] Counting [Morris, 1977] Median of a sequence [Munro and Paterson, 1980] Membership [Bloom, 1970]
- 95. 95 1 2 3 5 4 SAMPLING A/B Tes<ng FILTERING Set membership CORRELATION Fraud Detec<on QUANTILES Network Analysis CARDINALITY Site Audience Analysis Applications
- 96. 96 6 7 8 10 9 MOMENTS Database FREQUENT ELEMENTS Trending Hashtags CLUSTERING Medical Imaging ANOMALY DETECTION Sensor Networks SUBSEQUENCES Traffic Analysis Applications ! ! " "
- 98. 98 Sampling Sliding window approach (sample size k, window width n) Sequence-based Replace expired element with newly arrived element Disadvantage: highly periodic Chain-sample approach Select element ith with probability Min(i,n)/n Select uniformly at random an index from [i+1, i+n] of the element which will replace the ith item Maintain k independent chain samples Timestamp-based # elements in a moving window may vary over <me Priority-sample approach 3 5 1 4 6 2 8 5 2 3 5 4 2 2 5 0 9 8 4 6 7 3 3 5 1 4 6 2 8 5 2 3 5 4 2 2 5 0 9 8 4 6 7 3 3 5 1 4 6 2 8 5 2 3 5 4 2 2 5 0 9 8 4 6 7 3 3 5 1 4 6 2 8 5 2 3 5 4 2 2 5 0 9 8 4 6 7 3 [1] B. Babcock. Sampling From a Moving Window Over Streaming Data. In Proceedings of SODA, 2002.
- 105. 105 Filtering Set Membership [1] R. Pagh and F. Rodler. Cuckoo hashing. Journal of Algorithms, 51(2):122-144, 2004. [2] Illustra<on borrowed from “Fan et al., Cuckoo Filter: Prac<cally Beger Than Bloom. In Proceedings of the 10th ACM Interna<onal on Conference on Emerging Networking Experiments and Technologies, 2014.” [2] Illustra<on of Cuckoo hashing [2] Cuckoo Hashing [1] High space occupancy Prac<cal implementa<ons: mul<ple items/bucket Example uses: Sobware-based Ethernet switches Cuckoo Filter [2] Uses a mul<-way associa<ve Cuckoo hash table Employs par<al-key cuckoo hashing Store fingerprint of an item Relocate exis<ng fingerprints to their alterna<ve loca<ons [2]
- 106. Dele<on Item must have been previously inserted 106 Filtering Set Membership Cuckoo Filter Par<al-key cuckoo hashing Fingerprint hashing ensures uniform distribu<on of items in the table Length of fingerprint << Size of h1 or h2 Possible to have mul<ple entries of a fingerprint in a bucket Alternate bucket Significantly shorter than h1 and h2
- 107. 107 Filtering Set Membership Comparison k ➛ # hash functions, d ➛ # partitions
- 121. Count-min Sketch [1] A two-dimensional array counts with w columns and d rows Each entry of the array is ini<ally zero d hash func<ons are chosen uniformly at random from a pairwise independent family Update For a new element i, for each row j and k = hj(i), increment the kth column by one Point query where, sketch is the table Parameters 121 ),( δε }1{}1{:,,1 wnhh d ……… → ! ! " # # $ = ε e w ! ! " # # $ = δ 1 lnd [1] Cormode, Graham; S. Muthukrishnan (2005). "An Improved Data Stream Summary: The Count-Min Sketch and its Applica<ons". J. Algorithms 55: 29–38. Frequent Elements
- 123. 123 OPEN SOURCE TWITTER YAHOO! HUAWEI streamDM^ SGD Learner and Perceptron Naive Bayes CluStream Hoeffding Decision Trees Bagging Stream KM++ DATA SKETCHES* Unique Quan<le Histogram Sampling Theta Sketches Tuple Sketches Most Frequent ALGEBIRD# Filtering Unique Histogram Most Frequent * hgps://datasketches.github.io/ # hgps://github.com/twiger/algebird ^ hgp://huawei-noah.github.io/streamDM/ ** hgps://github.com/jiecchen/StreamLib StreamLib**
- 125. 125 Very rich - over 150 yrs - history Anomalies are contextual in nature “DISCORDANT observations may be defined as those which present the appearance of differing in respect of their law of frequency from other observations with which they ale combined. In the treatment of such observations there is great diversity between authorities ; but this discordance of methods may be reduced by the following reflection. Different methods are adapted to different hypotheses about the cause of a discordant observation; and different hypotheses are true, or appropriate, according as the subject-matter, or the degree of accuracy required, is different.” F. Y. Edgeworth, “On Discordant Observations”, 1887. Anomaly Detection
- 127. 127 Anomaly Detection COMMON APPROACHES DOMAINS STATS MFG OPS NOT VALID in real-life Moving Averages SMA, EWMA, PEWMA Assumption Normal Distribution PARAMS WIDTH DECAY Rule Based µ ± σ Stone 1868 Glaisher 1872 Edgeworth 1887 Stewart 1920 Irwin 1925 Jeffreys 1932 Rider 1933
- 128. 128 Anomaly Detection ROBUST MEASURES MEDIAN MAD [1] MCD [2] MVEE [3,4] Median Absolute Deviation Minimum Covariance Determinant Minimum Volume Enclosing Ellipsoid [1]P. J. Rousseeuw and C. Croux, “Alterna<ves to the Median Absolute Devia<on”, 1993. [2] hgp://onlinelibrary.wiley.com/wol1/doi/10.1002/wics.61/abstract [3] P. J. Rousseeuw and A. M. Leroy.,“Robust Regression and Outlier Detec<on”, 1987. [4] M. J.Todda and E. A. Yıldırım , “On Khachiyan's algorithm for the computa<on of minimum-volume enclosing ellipsoids”, 2007.
- 129. 129 Anomaly Detection Challenges NOISE STATIONARITY SEASONALITY TREND BREAKOUT
- 140. 140 Unifica<on
- 143. 143 Apache BookKeeper - Stream Storage A storage for log streams Replicated, durable storage of log streams Provide fast tailing/streaming facility Op<mized for immutable data Low-latency durability Simple repeatable read consistency High write and read availability
- 150. 150 Tying it all together A typical installa@on of Apache BookKeeper
- 153. 153 What do we really mean by Compute? Compute Unifica@on Aim is to react to events as they happen in real-<me Where do Events happen/arrive? Message Bus Whats a reac<on An ac<on/transforma<on/func<on
- 155. 155 Traditional Compute representation DAG % % % % % Source 1 Source 2 Action Action Action Sink 1 Sink 2
- 156. 156 Traditional Compute API Func@onal Builder.newBuilder() .newSource(() -> StreamletUtils.randomFromList(SENTENCES)) .flatMap(sentence -> Arrays.asList(sentence.toLowerCase().split("s+"))) .reduceByKeyAndWindow(word -> word, word -> 1, WindowConfig.TumblingCountWindow(50), (x, y) -> x + y);
- 157. 157 Traditional Compute Runtime Separate Messaging Compute
- 161. 161 Whats needed:- Stream-Native Compute Insight Simplest possible API Method/Procedure/Func<on Mul< Language API Scale developers Message bus na<ve concepts Input/Output/Log as topics Flexible run<me Simple standalone applica<ons vs system managed applica<ons
- 163. 163 Pulsar Functions API SDK less API import java.util.function.Function; public class ExclamationFunction implements Function<String, String> { @Override public String apply(String input) { return input + "!"; } }
- 164. 164 Pulsar Functions API SDK API import org.apache.pulsar.functions.api.PulsarFunction; import org.apache.pulsar.functions.api.Context; public class ExclamationFunction implements PulsarFunction<String, String> { @Override public String process(String input, Context context) { return input + "!"; } }
- 168. 168 Pulsar Functions WordCount Topology import org.apache.pulsar.functions.api.Context; import org.apache.pulsar.functions.api.PulsarFunction; public class CounterFunction implements PulsarFunction<String, Void> { @Override public Void process(String input, Context context) throws Exception { for (String word : input.split(".")) { context.incrCounter(word, 1); } return null; } }
- 169. 169 Pulsar Functions Running as a standalone applica@on bin/pulsar-admin functions localrun --input persistent://sample/standalone/ns1/test_input --output persistent://sample/standalone/ns1/test_result --className org.mycompany.ExclamationFunction --jar myjar.jar Runs as a standalone process Run as many instances as you want. Framework automa<cally balances data Run and manage via Mesos/K8/Nomad/your favorite tool
- 171. 171 Pulsar Functions Running inside a Pulsar Cluster bin/pulsar-admin functions create --input persistent://sample/standalone/ns1/test_input --output persistent://sample/standalone/ns1/test_result --className org.mycompany.ExclamationFunction --jar myjar.jar bin/pulsar-admin functions gestatus --name ExclamationFunction --tenant sample —-namespace ns1 bin/pulsar-admin functions delete --name ExclamationFunction --tenant sample —-namespace ns1
- 176. 176 Real Time is Messy and Unpredictable Aggregation Systems Messaging Systems Result Engine HDFS Queryable Engines
- 177. 177 Streamlio - Unified Architecture Interactive Querying Storm API Trident/Apache Beam SQL Application Builder Pulsar API BK/ HDFS API Kubernetes Metadata Management Operational Monitoring Chargeback Security Authentication Quota Management Rules Engine Kafka API
- 178. 178 BookKeeper in Real-Time Solution Durable Messaging, Scalable Compute and Stream Storage
- 179. 179 BookKeeper - Use Cases Combine messaging and storage Stream Storage combines the func<onality of streaming and storage WAL - Write Ahead Log Message Store Object Store SnapshotsStream Processing
- 180. 180 Readings FOCS’00 Clustering Data Streams SIGMOD’02 Querying and mining data streams: You only get one look SIAM Journal of Computing’09 Stream Order and Order Statistics: Quantile Estimation in Random-Order Streams PODS’02 Models and Issues in Data Stream Systems SIGMOD’07 Statistical Analysis of Sketch Estimators PODS’10 An optimal algorithm for the distinct elements problem
- 181. 181 Readings SODA’10 Coresets and Sketches for high dimensional subspace approximation problems SIGMOD’16 Time Adaptive Sketches (Ada-Sketches) for Summarizing Data Streams SOSR’17 Heavy-Hitter Detection Entirely in the Data Plane PODS’12 Graph Sketches: Sparsification, Spanners, and Subgraphs Arxiv’16 Coresets and Sketches ACM Queue’17 Data Sketching: The approximate approach is often faster and more efficient
- 183. 183 GET IN TOUCH C O N T A C T U S @arun_kejariwal @kramasamy, @sanjeerk @sijieg, @merlimat @nlu90 karthik@stremlio.io arun_kejariwal@acm.org
- 185. E N J O Y T H E P R E S E N T A T I O N The End