Talk delivered at Spring IO in Barcelona (May 20, 2016) The talk introduces Spring Cloud Stream as a framework for developing event-driven microservices, as well as Spring Cloud Data Flow as an orchestration and deployment layer.

1. CLOUD-NATIVE STREAMING AND
EVENT-DRIVEN MICROSERVICES
Marius Bogoevici
@mariusbogoevici

2. CLOUD-NATIVE STREAMING AND EVENT-
DRIVEN MICROSERVICES
• Use cases:
• predictive maintenance, fraud detection, QoS, IoT
• enterprise integration meets ‘Big Data’
• Specific requirements
• High throughput, low latency
• Grouping, ordering, windowing
• Middleware adaptations: beyond JMS, Rabbit MQ
• Kafka, Amazon Kinesis, Google Pub/Sub, Azure
Event Hubs
• Brokerless: Aeron, ZeroMQ
Spring I/O
2016

3. CLOUD-NATIVE STREAMING AND EVENT-
DRIVEN MICROSERVICES
• Architectural style advantages:
• bounded context, development agility
• Process-specific resource tuning:
• memory, CPU, instance count
• Messaging vs. HTTP
• Discoverability
• Availability
• Consistency across heterogenous resources
Spring I/O
2016

4. CLOUD-NATIVE STREAMING AND EVENT-
DRIVEN MICROSERVICES
• Modern platforms
• Resource management
• memory, CPU, instance count
• Scaling up/down
• Health monitoring and failover
• Routing and load balancing
Spring I/O
2016
Apache YARN
Apache Mesos Kubernetes

5. Spring I/O
2016
dataflow:> stream create demo --definition “http | file”
Goal

6. Spring Cloud Stream
Spring Cloud Data Flow
Spring I/O
2016
Spring Cloud Data Flow
Spring Cloud Stream
Spring Cloud Task
2015

7. Spring Cloud Stream
• Event-driven microservice framework
• Built on battle-tested components
• Spring Boot: full-stack standalone
applications
• Spring Integration: Messaging, EIP
patterns, connectors
• Opinionated primitives
• Pluggable middleware abstractions
Spring I/O
2016

8. Spring Cloud Stream in a nutshell
Spring I/O
2016

9. …. 10000 ft nutshell
Spring I/O
2016

10. Durable publish-subscribe
• Fits both streaming and event-driven use cases
Spring I/O
2016

11. Consumer Groups
• Borrowed from Kafka, applied to all binders
• Groups of competing consumers within the pub-sub
destination
• Used for scaling and partitioning
Spring I/O
2016

12. Partitioning
• Required for stateful processing
• Outputs specify a partitioning strategy
• Inputs consuming from specific partition
Spring I/O
2016

13. Programming model
@EnableBinding + Binder implementation
Apache Kafka
Gemfire

14. Spring I/O 2016
DEMO

15. Spring Cloud Data Flow
• Orchestration:
• DSL for Stream topologies
• REST API
• Shell
• UI
• Portable Deployment SPI
• OOTB apps for common integration use-
cases
Spring I/O
2016

16. Stream DSL: pipes and filters
Stream definition
Boot Apps build
on top of Spring
Ecosystem
httpfile = http | file
|
Spring I/O
2016
‘pipe’, or how to communicate
Rabbit, Kafka, Gemfire

17. Spring Cloud Deployer
• SPI for deploying applications to modern runtimes
• Local (for testing)
• Cloud Foundry
• YARN
• Kubernetes
• Mesos + Marathon
• Different resources supported
• Spring Boot Uberjars
• Docker Images
• Different locations
• File system, HDFS, HTTP, Maven, Docker Hub
Spring I/O
2016

18. Spring Cloud Data Flow
Pla$orm
Spring I/O
2016

19. Spring I/O 2016
DEMO

20. Timeline
• Spring Cloud Stream
• 1.0.0.GA May 10, 2016
• 1.1.0.M1 July 2016
• 1.1.0.GA Q3 2016
• Spring Cloud Data Flow
• 1.0.0.M3 May 7, 2016
• 1.0.0.RC1 End of May, 2016
Spring I/O
2016

21. Spring Cloud Stream 1.1.0.M1
• Reactive APIs: Reactor, RxJava
• Kafka 0.9
• Schema evolution support

22. Spring I/O 2016
Q&A
Marius Bogoevici
twitter: @mariusbogoevici