In Cassandra Lunch #88, CEO of Anant, Rahul Singh, will discuss how Cadence works on top of Cassandra to provide workflow management at scale and Cadence architecture in the context of SAGA Patterns Accompanying Blog: Coming Soon! Accompanying YouTube: https://youtu.be/YPPPM0F0xw0 Sign Up For Our Newsletter: http://eepurl.com/grdMkn Join Cassandra Lunch Weekly at 12 PM EST Every Wednesday: https://www.meetup.com/Cassandra-DataStax-DC/events/ Cassandra.Link: https://cassandra.link/ Follow Us and Reach Us At: Anant: https://www.anant.us/ Awesome Cassandra: https://github.com/Anant/awesome-cassandra Cassandra.Lunch: https://github.com/Anant/Cassandra.Lunch Email: solutions@anant.us LinkedIn: https://www.linkedin.com/company/anant/ Twitter: https://twitter.com/anantcorp Eventbrite: https://www.eventbrite.com/o/anant-1072927283 Facebook: https://www.facebook.com/AnantCorp/ Join The Anant Team: https://www.careers.anant.us

1. Cadence / Temporal
Workflows
What is distributed transaction processing and how to
run complex workflows at scale without reinventing the
wheel.

2. Data Pipelines come in
many varieties and meet
different purposes. They
are not all created equal.
Knowing when to use what
is the key to success.

3. Types
● Transaction Processing
● Distributed Processing
● Realtime Processing
● Batch Processing
● Multiprocessing
Framework Workflow
● Apache Spark / Flink
● Apache Kafka Streams
/ Pulsar IO
● Snowflake / BigQuery
● Apache Hadoop MR
● Apache Storm
● Spring Batch
● Apache Camel
● Apache Beam
● Apache Airflow
● Fivetran
● Talend
● Mulesoft
● Spring Cloud Data Flow
● CloudFlow
● AWS Glue
● Azure Data Factory
● Pentaho
Data Processing :

4. Processing Types & Use Cases
1. Transaction Processing
a. Two Phase Commit
b. Saga
2. Distributed [Batch]Processing
3. Realtime Processing
4. Batch Processing
5. Multiprocessing
1. Applications
2. Data Processing
a. Ingestion
b. transformation / cleaning
c. Enhancement
d. Transformation / logic
e. Load
3. Machine Learning
a. Training
b. Evaluation

5. Architecture(s)
Realtime Application &
Data at Scale

6. CRUD
● Updates & Selects
from one Model
● Synchronous
● Breaks at Scale
● Databases Can’t
Handle Both

7. CQRS
● Command Query
Responsibility
Segregation
● Commands are
Processed one way
● Queries are Processed
another way.
● Components can Scale
Independently.
● Databases Can be
Scaled Separately
https://martinfowler.com/bliki/CQRS.html

8. Event Driven Architecture
● Events are Stored
● Events are Processed
into Records, etc.
● Records are Retrieved
https://hazelcast.com/glossary/event-driven-architecture/

9. Kafka CQRS/ED
● User Interactions Send
Events
● Events Processed into
Different Sinks
● Events Pushed to Other
Users

10. Unified Log - A LinkedIN Story

11. Unified Log - A LinkedIN Story
https://engineering.linkedin.com/distributed-systems/log-what-every-software-engineer-should-know-about-real-time-datas-unifying

12. Lambda Architecture
● Batch vs. Speed Layer
● Batch Reconciles Speed
https://hazelcast.com/glossary/lambda-architecture/

13. Kappa Architecture
● One Technology for Batch & Stream
https://hazelcast.com/glossary/kappa-architecture/

14. Microservices
Overview of REST vs. Microservice
vs. Event Driven Architecture

15. Monolith to
Microservices
https://www.infoworld.com/article/3236291/how-to-choose-a-database-for-your-microservices.html

17. REST: Quick Introduction
● REST = REpresentational State Transfer
● Transfer the “State” of the Database at any
Given time.
● Used to wrap other APIs as well.
● REST is Stateless so nothing is assumed.
● Infinitely scalable layer if done right.
● All over HTTP using GET, PUT, etc.
● Can be extended e.g. HATEOAS, GraphQL
https://medium.com/faun/consuming-rest-
apis-with-python-eb86c6b724c5
https://www.parallels.com/blogs/ras/rest-api/

18. Microservice: Quick Introduction
● Loosely Coupled with other services
● Independently Deployable
● Can be implemented Sync or Async
○ E.g. HTTP/REST = Sync
○ E.g. AMQP, Kafka = Async
● Maintained by small teams
● Each service has its own DB
● Consistency Achieved via Saga Pattern ,
Two Phase Commit, or Eventual
Consistency
● Automated deployments
https://www.baeldung.com/transactions-across-microservices

19. Event Driven Architecture
● Pub / Sub Model for Communication
● Event Sourcing - Everything is sourced from
Events in a Queue or a Database
○ Events are created by API into a Channel
○ Channel being a Queue, DB, etc.
○ Events are processed by a Processing
Engine
● Implemented sometimes on top of Message
Driven Architecture
https://medium.com/@javier.ramos1/devops-
microservices-part-5-streaming-platforms-
e9220dc06ed8

20. Command Query
Separation (aka CQRS)

21. Pipeline Design /
Processing Patterns

22. Processing Types & Use Cases
1. Transaction Processing
a. Two Phase Commit
b. Saga
2. Distributed [Batch]Processing
3. Realtime Processing
4. Batch Processing
5. Multiprocessing
1. Applications
2. Data Processing
a. Extraction
b. Ingestion
c. transformation / cleaning
d. Enhancement
e. Transformation / logic
f. Load
3. Machine Learning
a. Training
b. Evaluation

23. Distributed Transactions
Two Phase Commit
- Prepare
- Ready?
- Commit
- Commit!
SAGA (Coordination methods)
- Choreography
- messages are sent
between services
- Orchestration
- orchestrator tells
participants what
transactions to do.
https://microservices.io/patterns/data/saga.html
https://www.baeldung.com/cs/saga-pattern-microservices

24. Two Phase Commit (2PC)
● Two Phase Commit Process
○ Prepare
■ Each Participant Says “Prepare”
is Done
■ If Not ready, aborts.
○ Commit
■ Each Participant Says “Commit”
is Done
■ If Not done, aborts.
● Benefits
○ Consistent and Available
○ Data is always synchronized
● Drawbacks
○ Blocking protocol.
○ Failure of participantscan block processing.
○ Failure of coordinator can cause
inconsistency.
● Framework for Two Phase Commit
○ Seata

25. SAGA - Choreography Pattern
● Benefits
○ Good for simple workflows
○ Doesn't require additional service
○ Doesn't introduce a single point of
failure,
● Drawbacks
○ Workflow can become confusing
when adding new steps
○ There's a risk of cyclic dependency
○ Integration testing is difficult
● Frameworks for ChoreographyPattern
○ Axon Saga
○ Eclipse MicroProfile LR
○ Eventuate Tram Saga
○ Seata
https://docs.microsoft.com/en-us/azure/architecture/reference-architectures/saga/saga

26. SAGA - Choreography
https://www.baeldung.com/cs/saga-pattern-microservices

27. SAGA - Orchestration Pattern
● Benefits
○ Good for complex
workflows
○ Suitable when control is
available
○ Does not have cyclic
dependencies
○ Saga participantsonly
listen to orchestrator
● Drawbacks
○ Additional design
complexity
○ Additional point of
failure.
● Framework for Orchestration
Pattern
○ Camunda
○ Activiti?
○ Apache Camel
○ Cadence
○ Temporal

28. SAGA - Orchestration
https://www.baeldung.com/cs/saga-pattern-microservices

29. Cadence on Cassandra

30. SAGA - Cadence (Orchestration)
https://eng.uber.com/open-source-orchestration-tool-cadence-overview/
https://github.com/uber/cadence

31. ● Front End (FE):
● History Service (HS)
● Matching Service (MS)
● Worker Service (WS)
● Workers: (User created workflow/activities)
Cadence Architecture
https://medium.com/stashaway-engineering/building-
your-first-cadence-workflow-e61a0b29785
https://cadenceworkflow.io/docs/concepts/topology/#overview

32. Cadence Architecture
https://cadenceworkflow.io/blog/2021/10/19/moving-to-
grpc/

33. Cadence Example
https://cadenceworkflow.io/docs/concepts/workflows/#example

34. Banzai Cloud - Pipeline
https://banzaicloud.com/blog/introduction-to-cadence/
https://github.com/banzaicloud/pipeline

35. Cadence at Uber
https://eng.uber.com/customer-obsession-ticket-routing-workflow-and-orchestration-engine/

36. Cadence at Doordash Drive
(Delivery as a Service)
https://doordash.engineering/2020/08/14/workflows-cadence-event-driven-processing/

37. Cadence at Doordash Drive
(Delivery as a Service)
https://doordash.engineering/2020/08/14/workflows-cadence-event-driven-processing/

38. Temporal on Cassandra

39. SAGA - Temporal (Orchestration)
https://github.com/temporalio/temporal
https://temporal.io/
https://docs.temporal.io/docs/tempo
ral-explained/introduction/

40. Temporal Workflows
https://manuel.bernhardt.io/2021/04/12/to
ur-of-temporal-welcome-to-the-workflow

41. Temporal GUI
https://manuel.bernhardt.io/2021/04/12/to
ur-of-temporal-welcome-to-the-workflow

42. Difference between Temporal/Cadence
1. Cadence uses Thrift/TChannel |
Temporal uses Protobuf/gRPC
2. Entities are Different
3. Workflow Timeout are different in
Temporal
4. Workflow retries are different in
Temporal
5. Activity retries are different in Temporal
6. Temporal dependencies are different
7. Temporal Server configuration files are
different

43. Resources
● https://eng.uber.com/cadence-multi-tenant-task-processing/
● https://softwareengineeringdaily.com/wp-content/uploads/2020/04/SED1043-Cadence-Workflow-
Orchestration.pdf
● https://doordash.engineering/2020/08/14/workflows-cadence-event-driven-processing/
● https://medium.com/stashaway-engineering/building-your-first-cadence-workflow-e61a0b29785
● https://doc.neonkube.com/index.html
● https://docs.temporal.io/
● https://cadenceworkflow.io/