Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Stream Processing with Pipelines and Stored Procedures

327 views

Published on

This talk will discuss an upcoming feature in MemSQL 6.5 showing how advanced stream processing use cases can be tackled with a combination of stored procedures (new in 6.0) and MemSQL's pipelines feature.

Published in: Data & Analytics
  • Be the first to comment

Stream Processing with Pipelines and Stored Procedures

  1. 1. Stream Processing with Pipelines and Stored Procedures Tewei Luo Software Engineer
  2. 2. CREATE PIPELINE AS LOAD DATA [KAFKA | S3 | FS] INTO TABLE tbl
  3. 3. Stream into more tables?
  4. 4. Stream into more tables? Example: - 2 fields in source - 2 tables, each takes one column
  5. 5. CREATE PIPELINE AS LOAD DATA [KAFKA | S3 | FS] INTO PROCEDURE split
  6. 6. CREATE PROCEDURE split(batch Query(c1 int, c2 text)) AS BEGIN INSERT INTO tb1 SELECT c1 from batch; INSERT INTO tb2 SELECT c2 from batch; … END
  7. 7. 7 DEMO
  8. 8. - Split stream to multiple tables - Update statistics - Complex logic using existing tables - Delete obsolete data automatically
  9. 9. 9 Questions?
  10. 10. Thank you memsql.com

×