The document discusses how workflow engines and event streaming brokers can work together. It describes some gaps in using only event streaming at scale for complex business flows and long-running tasks. It then provides an overview of how a workflow engine like Temporal could be integrated to address these gaps, including examples of implementing workflows and activities. Specific opportunities discussed for using Temporal at Wix include long-running processes, cron jobs, and internal microservice tasks.
Dealing with Cultural Dispersion — Stefano Lambiase — ICSE-SEIS 2024
Workflow Engines & Event Streaming Brokers - Can they work together? [Current 2023]
1. September 2023
Workflow Engines &
Event Streaming Brokers
Can they work together?
Natan Silnitsky, Backend Infra TL @Wix
natansil.com twitter@NSilnitsky linkedin/natansilnitsky github.com/natansil
2. Workflow Engines & Event Streaming Brokers @NSilnitsky
Would you
implement a
complex business
flow with Kafka and
Event Driven
Architecture?
A B
C
D
E
5. Workflow Engines & Event Streaming Brokers @NSilnitsky
~1B
Unique visitors
use Wix platform
every month
~3.5B
Daily HTTP
Transactions
~70B
Kafka messages
a day
>600
GAs every day
3000
Microservices
in production
6. Workflow Engines & Event Streaming Brokers @NSilnitsky
@Wix Is great for
● Event Streaming @Scale
● Asynchronous Messaging Pub/Sub
● Decoupling & Extensibility
● Optimize Queries - Materialized views
11. Workflow Engines & Event Streaming Brokers @NSilnitsky
What’s now Complex business flow
Temporal Overview
Long running tasks
Integrating Workflow Engines into Wix
→
→
→
→
12. Workflow Engines & Event Streaming Brokers @NSilnitsky
Complex business flow
Inventory
Service
Payments
Service
Order
Service
Create Order
Reserve Inventory
process payment
Cart
Service
13. Workflow Engines & Event Streaming Brokers @NSilnitsky
Complex business flow
Inventory
Service
Payments
Service
Order
Service
Create Order
Reserve Inventory
process payment
Cart
Service
14. Workflow Engines & Event Streaming Brokers @NSilnitsky
Complex business flow
Inventory
Service
Payments
Service
Order
Service
Create Order
Reserve Inventory
process payment
Cancel reservation
Cart
Service
15. Workflow Engines & Event Streaming Brokers @NSilnitsky
Complex business flow
Payments
Service
Order
Service
Create Order
Reserve Inventory
process payment
Cancel order
Cancel reservation
Inventory
Service
Cart
Service
17. Workflow Engines & Event Streaming Brokers @NSilnitsky
How would you do it with Kafka events
Inventory
Service
Payments
Service
Order
Service
Checkout Started
Order Created
Inventory reserved
Cart
Service
18. Workflow Engines & Event Streaming Brokers @NSilnitsky
Inventory
Service
Payments
Service
Order
Service
Checkout Started
Order Created
Inventory reserved
Cart
Service
Reservation
canceled
Payment failed
Order canceled
How would you do it with Kafka events
19. Workflow Engines & Event Streaming Brokers @NSilnitsky
Order
Service
Cart
Service
function checkout():
summarize cart
// publish CheckoutStarted event
listen to checkoutStarted event:
create order
publish orderCreated event
listen to inventoryCancelled event:
cancel order
publish orderCancelled event
How would you do it with Kafka / EDA
20. Workflow Engines & Event Streaming Brokers @NSilnitsky
listen to orderCreated event:
reserve inventory
if reservation successful:
publish inventoryReserved event
else:
publish inventoryCancelled event
listen to paymentCancelled event:
release reserved inventory
listen to inventoryReserved event:
process payment
if payment successful:
publish paymentProcessed event
else:
publish paymentFailed event
Inventory
Service
Payments
Service
How would you do it with Kafka events
* only once, changing order
23. Workflow Engines & Event Streaming Brokers @NSilnitsky
How would you do it with Workflow engine?
Inventory
Service
Payments
Service
Order
Service
Cart
Service
function checkout():
cart = summarizeCart()
order = createOrder(cart)
result = reserveInventory(order)
if result != "Success":
cancelOrder(order) # Rollback order
creation
return "Inventory reservation failed"
result = processPayment(order)
if result != "Success":
releaseInventory(order)
cancelOrder(order)
return "Payment processing failed"
completeOrder(order)
return "Order placed successfully"
24. Workflow Engines & Event Streaming Brokers @NSilnitsky
How would you do it with Workflow engine?
25. Workflow Engines & Event Streaming Brokers @NSilnitsky
Complex business flow
Seeing the Big Picture
beats*
Production Monitoring
and Debugging
Changing the sequence of
the business flow
26. Workflow Engines & Event Streaming Brokers @NSilnitsky
What’s now Complex business flow
Temporal Overview
Long running tasks
Integrating Workflow Engines into Wix
→
→
→
→
27. Workflow Engines & Event Streaming Brokers @NSilnitsky
Temporal Workflow
Order
Service
Cart
Service
Summarize Cart
Activity
Checkout
Workflow
Order creation
Activity
Logical definition
28. Workflow Engines & Event Streaming Brokers @NSilnitsky
public interface CartActivity {
@ActivityMethod
CartSummary summarizeCart(Cart cart);
}
public interface CheckoutWorkflow {
@WorkflowMethod
void processCheckout(Cart cart);
}
public interface OrderActivity {
@ActivityMethod
Order createOrder(CartSummary cartSummary);
}
29. Workflow Engines & Event Streaming Brokers @NSilnitsky
public class CheckoutWorkflowImpl implements CheckoutWorkflow {
CartActivity cartActivity = Workflow.newActivityStub(CartActivity.class);
@Override
public void processCheckout(Cart cart) {
// Summarize the cart
CartSummary summary = cartActivity.summarizeCart(cart);
// rest of the checkout process here...
}
}
30. Workflow Engines & Event Streaming Brokers @NSilnitsky
Temporal Workflow
Temporal
Server
Cart
Service
Summarize Cart
Activity
Worker
Checkout
Workflow
Worker
Workers
Order creation
Activity
Worker
Order
Service
31. Workflow Engines & Event Streaming Brokers @NSilnitsky
Temporal Workflow
Temporal Server
Cart
Service
Summarize Cart
Activity
Worker
Checkout
Workflow
Worker
Task Queues
Order creation
Activity
Worker
Order
Service
32. Workflow Engines & Event Streaming Brokers @NSilnitsky
public class CartServiceCompositionLayer {
public static void init(String[] args) {
// Create a new worker factory
WorkerFactory factory = WorkerFactory.newInstance(... WorkflowClient ...);
// Create a new worker that listens on the specified task queue
Worker worker = factory.newWorker("MY_TASK_QUEUE");
// Register your workflow and activity implementations
worker.registerWorkflowImplementationTypes(CheckoutWorkflowImpl.class);
worker.registerActivitiesImplementations(new CartActivityImpl());
// Start listening for tasks
factory.start();
}
}
33. Workflow Engines & Event Streaming Brokers @NSilnitsky
Temporal Server
Temporal Workflow
Cart
Service
Summarize Cart
Activity
Worker
Checkout
Workflow
Worker
Retries
Order creation
Activity
Worker
Order
Service
* Wix infra
35. Workflow Engines & Event Streaming Brokers @NSilnitsky
Temporal Supports
Temporal
Server
Cart
Service
Summarize Cart
Activity
Worker
Checkout
Workflow
Worker
Major languages
Order creation
Activity
Worker
Order
Service
Service written with Python
Service written with JS&TS
* Wix infra
38. Workflow Engines & Event Streaming Brokers @NSilnitsky
Temporal
disadvantages
● workflow code has to be deterministic
● Activity Input and Output must be
serializable
● No support for Very low Latency
39. Workflow Engines & Event Streaming Brokers @NSilnitsky
public class CheckoutWorkflowImpl implements CheckoutWorkflow {
public String processCheckout() {
Cart cart = cartActivity.summarizeCart();
Order order = orderActivity.createOrder(cart);
String result = inventoryActivity.reserveInventory(order);
if (!"Success".equals(result)) {
orderActivity.cancelOrder(order);
return "Inventory reservation failed";
}
result = paymentActivity.processPayment(order);
if (!"Success".equals(result)) {
inventoryActivity.releaseInventory(order);
orderActivity.cancelOrder(order);
return "Payment processing failed";
}
orderActivity.completeOrder(order);
return "Order placed successfully";
}
}
Non-deterministic code has to be
inside activity.
Parameters must be serializable!
40. Workflow Engines & Event Streaming Brokers @NSilnitsky
Temporal Server
Temporal Workflow
Cart
Service
Summarize Cart
Activity
Worker
Checkout
Workflow
Worker
Serializable
Order creation
Activity
Worker
Order
Service
Order must be serializable
Cart must be serializable
41. Workflow Engines & Event Streaming Brokers @NSilnitsky
What’s now Complex business flow
Temporal Overview
Long running tasks
Integrating Workflow Engines into Wix
→
→
→
→
45. Workflow Engines & Event Streaming Brokers @NSilnitsky
Temporal Server
Cart
Service
Summarize Cart
Activity
Worker
Checkout
Workflow
Worker
Product Info
Activity
Worker
Order
Service
Long Running tasks
46. Workflow Engines & Event Streaming Brokers @NSilnitsky
public class PaymentActivityImpl implements PaymentActivity {
public processPayment(order) {
...
ExecutionContext executionContext = Activity.getExecutionContext;
executionContext.heartbeat("waiting for bank.");
...
}
}
47. Workflow Engines & Event Streaming Brokers @NSilnitsky
What’s now Complex business flow
Temporal Overview
Long running tasks
Integrating Workflow Engines into Wix
→
→
→
→
48. Workflow Engines & Event Streaming Brokers @NSilnitsky
Wix has built an Open Platform
Wix Orders service
Wix Inventory service
3rd party
Checkout app
3rd party
Analytics app
3rd party Tax
Calculator
SPI
API
Produce order created
49. Workflow Engines & Event Streaming Brokers @NSilnitsky
Can Temporal do Pub/Sub messaging?
Checkout Started
json
Cart
Service
Temporal Signal
Checkout workflow1
Checkout workflow2
Checkout workflow3
Order
Service
* Not design high throu, for same
exec, or for distributing work
51. Workflow Engines & Event Streaming Brokers @NSilnitsky
public class CheckoutWorkflowImpl implements CheckoutWorkflow {
public String processCheckout() {
Cart cart = cartActivity.summarizeCart();
Order order = orderActivity.createOrder(cart);
String result = inventoryActivity.reserveInventory(order);
if (!"Success".equals(result)) {
orderActivity.cancelOrder(order);
return "Inventory reservation failed";
}
result = paymentActivity.processPayment(order);
if (!"Success".equals(result)) {
inventoryActivity.releaseInventory(order);
orderActivity.cancelOrder(order);
return "Payment processing failed";
}
orderActivity.completeOrder(order);
return "Order placed successfully";
}
}
Coupled orchestration
52. Workflow Engines & Event Streaming Brokers @NSilnitsky
Where is Wix thinking
of utilizing Temporal
53. Workflow Engines & Event Streaming Brokers @NSilnitsky
Where Wix is thinking of utilizing Temporal
long running processes
Cron jobs
Internal microservice jobs
Dual use
54. Workflow Engines & Event Streaming Brokers @NSilnitsky
Fitting Temporal in Wix- cron jobs
Cron Job
56. Workflow Engines & Event Streaming Brokers @NSilnitsky
Where Wix is thinking of utilizing Temporal
long running processes
Cron jobs
Internal microservice jobs
Dual use
57. Workflow Engines & Event Streaming Brokers @NSilnitsky
Cart Service Order Service
3rd party
HTTP WebHook
Fitting Temporal in Wix
Option 1 - intra service jobs
gRPC
HTTP
58. Workflow Engines & Event Streaming Brokers @NSilnitsky
Option 2 - dual use
Cart Service Order Service
3rd party
HTTP WebHook
Fitting Temporal in Wix
gRPC
HTTP
59. Workflow Engines & Event Streaming Brokers @NSilnitsky
Low
Latency
Long
Running
Tasks
vs
Customized
Decoupled
Logic
Standalone
Business
Processes
Summary - Microservices Coordination
60. Workflow Engines & Event Streaming Brokers @NSilnitsky
Summary - Wix plans
Wix has a rich and diverse distributed
system and open platform mindset
Kafka and Wix Infra allow us to have
flexibility, extensibility and resiliency
built into the system
Wix is considering to inject Temporal
in strategic places in order to increase
dev velocity
* things to consider. Some companies. Can together?
62. Workflow Engines & Event Streaming Brokers @NSilnitsky
Lessons Learned From Working with 2000
Event-Driven Microservices
The Next Step
https://www.youtube.com/watch?v=
H4OSmOYTCkM
Scan the code