The document discusses the integration of Spring Cloud Data Flow with Apache Geode for managing stream and batch data processes. It emphasizes the need for low latency, high throughput, and efficient data handling in data pipelines. Key features mentioned include a drag-and-drop UI, security, and various configurations for processing data in real-time.

1. Spring Cloud Data Flow + Geode
Sabby Anandan | Product Manager | @sabbyanandan

2. Stream Batch
Spring Cloud Data Flow
Spring Cloud Stream Spring Cloud Task
Shell; DSL; REST-
APIs
Drag & Drop UI Security
OOTB
Connectors
Reactive Data Science

3. Dataflow
Server
Admin / Flo UI
Shell
CURL
??X
Stream/Task Spring Boot Apps
YARN

4. Why are we here?
Data Pipelines requiring:
• Low latency and in-memory processing
• High Throughput SLAs
• Correlation between reference-data and data-
in-flight
• Frequent data-shuffling

5. http | transform | log

6. | = ?
http | transform | log

7. | = Binder

8. Binders

9. Region Data Buckets
http | transform | log
Geode Cluster
transform-processor.jar
PARTITION
transform-processor.jar
PARTITION
log-sink.jar
PARTITION
log-sink.jar
PARTITION
http-source.jar
PARTITION_PROXY
http-source.jar
PARTITION_PROXY

10. What’s next?
•+/- scaling and automatic re-partitioning
•Stream / Task metadata-repository
•Key-Value store for OOTB Counters
•Partition level local-state and SQL-like
stream processing

11. 11
Join the Apache Geode Community!
• Check out http://geode.incubator.apache.org
• Subscribe: user-subscribe@geode.incubator.apache.org
• Download: http://geode.incubator.apache.org/releases/