Flowable BPM has a new event bus capability that allows it to hook into event streams, Kafka, ActiveMQ, RabbitMQ and others

1. Using Kafka: the Flowable Event Registry
Joram Barrez (@jbarrez)
Filip Hrisafov (@filiphr)

2. ©2019FlowableAG. All rights reserved.
Event-Driven Architectures
• In the Flowable community, we have seen
• People experimenting with event-driven/event-sourcing
architectures
• (or more often architectures enriched with events)
• Demand for out-of-the-box support
2

3. ©2019FlowableAG. All rights reserved.
Event-Driven Architectures
• A process/case fits naturally
• Can be the ‘Command’ of an event
(e.g. the C in CQRS)
• Have state to react on the change expressed in the event
(state is often the hard part of event-sourcing)
3

4. ©2019FlowableAG. All rights reserved.
Quick Demo
4

5. ©2019FlowableAG. All rights reserved.
Implementation – The event registry
5
Flowable
Engine
EventRegistry
EventBus
EventDefinitionEventDefinitionEventDefinition
Consumers
Central repository of event metadata
Dispatches events to consumers
(default process/case behaviors are also a consumer)
Description of event structure:
- Payload
- Correlation

6. ©2019FlowableAG. All rights reserved.
Implementation – Channels and Pipelines
6
Flowable
Engine
EventRegistry
EventBus
Channel Adapter
Event Consumer
Inbound Channel
Inbound Pipeline
Deserialization
Event key detection
* All shown here on the slide are pluggable
Payload extraction
Event transformation

7. ©2019FlowableAG. All rights reserved.
Implementation – Channels and Pipelines
7
Flowable
Engine
EventRegistry
EventBus
Channel Adapter
Event Producer
Outbound Channel
Outbound Pipeline
Serialization
Event transformation

8. Demo
https://github.com/flowable/flowable-examples/tree/master/devoxx2019

9. ©2019FlowableAG. All rights reserved.
• Fictional electrical kick-
scooter startup
• Customer satisfaction is
priority #1
Use Case
9

10. ©2019FlowableAG. All rights reserved.
• Customer can provide a review
• Review à Kafka
• A customer case
• Groups all reviews
• Kicks of a sentiment analysis process
• Analysis is done using a process that
calls out to AWS Comprehend
• Bad review à A task for an
employee is created to try to fix the
relationship
Use Case
10

11. ©2019FlowableAG. All rights reserved.
Architecture
11
* (5th service (Spring Cloud API gateway) not shown to simplify things)
UI for customers
Review Service
Customer
Relationship
Service
Flowable Task
Sentiment Analysis
service
UI for employees
AWS Comprehend
Review Event
Review Event Analysis
Result
Analysis
Result
Analysis
Request
Analysis
Request

12. ©2019FlowableAG. All rights reserved.
Demo: The models explained
12

13. ©2019FlowableAG. All rights reserved.
Demo
• Simulating reviews
• Stepping through a bad user review
13

14. ©2019FlowableAG. All rights reserved.
Demo: Review Service
• Simple Spring Webflux REST API
• Reactive pushing of events from UI into Kafka
14

15. ©2019FlowableAG. All rights reserved.
Demo: Customer Case Service
• Spring Boot application
• Flowable Spring Boot starters
• Autodeploy cases and process models
• Register event channels and definitions
• Postgres DB
15

16. ©2019FlowableAG. All rights reserved.
Demo: Customer Case Service
• The sentiment process doesn’t use the ‘old way’
• Send-and-wait-for-response pattern
• Race condition problem
16

17. ©2019FlowableAG. All rights reserved.
Demo: Customer Case Service
• The sentiment process uses a triggerable service task
17
• During execution (tx1)
• Create event subscription
• Create async job data (to send event)
• Post-commit (tx2)
• Trigger job executor (or other node)
• Send event asynchronously
• Go into wait state
• Event receival (tx3)
• Trigger for continuation

18. ©2019FlowableAG. All rights reserved.
Demo: Customer Case Service
• Can this also be made reactive?
• Yes!
• Nicely scaling of message stream handling depending on load
• (insertion of async Flowable job at event receival using R2DBC)
18

19. ©2019FlowableAG. All rights reserved.
Demo: The Sentiment Analysis Service
• Spring Boot application
• Flowable Spring Boot starters
• Autodeploy process model
• Uses AWS SDK to call out to AWS Comprehend
19

20. ©2019FlowableAG. All rights reserved.
Demo: Flowable Task
• Bad review à employee form
20

21. ©2019FlowableAG. All rights reserved.
Demo: The Sentiment Analysis Service
• This service is effectively a function
• Can be run serverless
• With all the benefits of using Flowable
• Visual models, audit/history, etc.
21

22. ©2019FlowableAG. All rights reserved.
Serverless Core Concepts
• Ephemeral (short living)
• Only booted up when needed
• Cold starts
• Auto scaling
• Heavy load à boot up more instances
• No Server management
• Logic is deployed to a ‘serverless provider’
• Don’t care, up to the cloud vendor
• Pay as you use
22

23. ©2019FlowableAG. All rights reserved.
What is serverless?
• A better name would be ‘on-demand/event server’
• (it’s still running on a server)
23

24. ©2019FlowableAG. All rights reserved.
Flowable Serverless (Experimental)
• Challenges
• Cold start
• State
• Solutions
• In memory (no Database)
• No wait states (straight-through processes)
24

25. ©2019FlowableAG. All rights reserved.
What is serverless?
• Example: Amazon AWS Lambda
• AWS Lamba in/out can be chained to the whole AWS ecosystem:
S3/SQS/Alexa/DynamoDB/…
25

26. ©2019FlowableAG. All rights reserved.
Architecture
26
* (5th service (Spring Cloud API gateway) not shown to simplify things)
UI for customers
Review Service
Customer
Relationship
Service
Flowable Task
Sentiment Analysis
service
UI for employees
AWS Comprehend
Review Event
Review Event Analysis
Result
Analysis
Result
Analysis
Request
Analysis
Request

27. ©2019FlowableAG. All rights reserved.
AWS Architecture
27
AWS Cloud
API
Gateway
AWS SQS
Review Event
Frontend
(S3)
Review Event
Analysis
Request Analysis
Result
Analysis
Request
Analysis
Result
Customer Review
Application (EC2)
Sentiment
Analysis
Browser
Comprehe
nd

28. ©2019FlowableAG. All rights reserved.
Review Service Differences
• Doesn’t exist J
• API Gateway pushes events directly to Amazon SQS
28

29. ©2019FlowableAG. All rights reserved.
Customer Case Service Differences
• Listens from Amazon SQS instead of Kafka
• The channel/adapter concept makes this very flexible
• Without impact on using the events in the model
29

30. ©2019FlowableAG. All rights reserved.
Sentiment Analysis Service Differences
• Flowable with in memory DB (no SQL)
• Triggers via Amazon SQS as a Lambda
• Timing (with 256MB Lambda)
• Cold start billed 30s / Engine start 6s / Deploy 1.2s
• Preheat start: billed 1s / Engine start 60ms / Deploy 80ms
• Sends analysis results synchronously to Amazon SQS
30

31. ©2019FlowableAG. All rights reserved.
Serverless: No Free Lunch
• Vendor lock-in
• Security?
• Debugging?
31

32. ©2019FlowableAG. All rights reserved.
Future Ideas
• Aurora serverless
• Passing state around with event payload
32

33. ©2019FlowableAG. All rights reserved.
Conclusions
• We’ve shown that
• using Flowable in event-driven architectures is easy, yet very
powerful
• Flowable is extremely flexible when it comes to
• switching queue/event stream choice
• deploying your Flowable applications in different environments
• running a process as a serverless function
33

34. Thank you.