Uber Cadence Fault Tolerant Actor Framework

1. UBER CADENCE FAULT TOLERANT ACTOR FRAMEWORK MAXIM FATEEV STAFF ENGINEER @ UBER OCTOBER 5TH, 2018

2. Use Case

3. uDebitReward ● Driver signs up if qualifies ● Eligibility is checked every 30 days ● Participation is lost if doesn’t meet the rewards requirements when checked

4. uDebitReward ● Naive Implementation

5. Availability, Durability and Scalability ● Highly available ● Data loss is not acceptable ● Tens of millions of drivers

6. uDebitReward ● Cadence Based Implementation

7. Cadence

8. Cadence Service ● Hosted ● High Scalability ● High Availability ● Task Queuing ● State Persistence (Cassandra/SQL) ● Visibility APIs ● Written in Go

9. CADENCE CLIENT LIBRARIES ● Go and Java ● Stateless ● Hide most of the complexity ● Distributed business logic programmed as fault tolerant actors.

10. CADENCE: DEPLOYMENT TOPOLOGY Persistence Workflow Worker Activity Worker Activity Worker Activity Worker Workflow Worker Workflow Worker Workflow Starter Workflow Starter Cadence Workflow Starter

11. Workflow Execution

12. Cadence

13. Cadence StartWorkflow DriverRewards

14. Cadence Driver1 StartWorkflow DebitRewards

15. Cadence Driver1 Rewards StartWorkflow DebitRewards

16. Rewards Cadence Driver1

17. Rewards Cadence Driver1 Workflow Worker Workflow Worker Workflow Worker

21. Rewards Cadence Eligibility Driver1 Workflow Worker Workflow Worker Workflow Worker Start Timer: 30 days

22. Rewards Cadence Eligibility Driver1 Workflow Worker Workflow Worker Workflow Worker

23. Rewards Cadence Eligibility Driver1 Workflow Worker Workflow Worker Workflow Worker

24. Rewards Cadence Eligibility Workflow Worker Workflow Worker Workflow Worker Driver1

27. Rewards Cadence Activation Workflow Worker Workflow Worker Workflow Worker Driver1 Schedule ActivateRewards

28. Rewards Cadence Activation Workflow Worker Workflow Worker Workflow Worker Driver1

29. Rewards Cadence Activation Workflow Worker Workflow Worker Workflow Worker Driver1 Activity Worker Activity Worker Activity Worker Start Activate

30. Rewards Cadence Activation Workflow Worker Workflow Worker Workflow Worker Driver1 Activity Worker Activity Worker Activity Worker

31. Rewards Cadence Activation Workflow Worker Workflow Worker Workflow Worker Driver1 Activity Worker Activity Worker Activity Worker

32. Rewards Cadence Activation Workflow Worker Workflow Worker Workflow Worker Driver1 Activity Worker Activity Worker Activity Worker Complete Activate Activation

33. Rewards Cadence Activation Workflow Worker Workflow Worker Workflow Worker Driver1 Activity Worker Activity Worker Activity Worker Activation

34. Rewards Cadence Workflow Worker Workflow Worker Workflow Worker Activity Worker Activity Worker Activity Worker Activation Driver1

37. Rewards Cadence Eligibility Driver1 Workflow Worker Workflow Worker Workflow Worker Start Timer: 30 days Activity Worker Activity Worker Activity Worker Activation

38. Rewards Cadence Eligibility Driver1 Workflow Worker Workflow Worker Workflow Worker Activity Worker Activity Worker Activity Worker Activation

39. Execution Recovery

40. Execution History 1. WorkflowExecutionStarted 2. DecisionTaskScheduled 3. DecisionTaskStarted 4. TimerStarted 30 days 5. TimerFired 6. DecisionTaskScheduled 7. DecisionTaskStarted 8. ActivityTaskScheduled ActivateRewards 9. ActivityTaskStarted ActivateRewards 10. ActivityTaskCompleted ActivateRewards 11. DecisionTaskScheduled 12. DecisionTaskStarted

45. Execution History 1. WorkflowExecutionStarted 2. DecisionTaskScheduled 3. DecisionTaskStarted 4. TimerStarted 30 days 5. TimerFired 6. DecisionTaskScheduled 7. DecisionTaskStarted 8. ActivityTaskScheduled ActivateRewards 9. ActivityTaskStarted ActivateRewards 10. ActivityTaskCompleted ActivateRewards 11. DecisionTaskScheduled 12. DecisionTaskStarted Decisions: 1. StartTimer 30 days

50. Execution History 1. WorkflowExecutionStarted 2. DecisionTaskScheduled 3. DecisionTaskStarted 4. TimerStarted 30 days 5. TimerFired 6. DecisionTaskScheduled 7. DecisionTaskStarted 8. ActivityTaskScheduled ActivateRewards 9. ActivityTaskStarted ActivateRewards 10. ActivityTaskCompleted ActivateRewards 11. DecisionTaskScheduled 12. DecisionTaskStarted Decisions:

54. Execution History 1. WorkflowExecutionStarted 2. DecisionTaskScheduled 3. DecisionTaskStarted 4. TimerStarted 30 days 5. TimerFired 6. DecisionTaskScheduled 7. DecisionTaskStarted 8. ActivityTaskScheduled ActivateRewards 9. ActivityTaskStarted ActivateRewards 10. ActivityTaskCompleted ActivateRewards 11. DecisionTaskScheduled 12. DecisionTaskStarted Decisions: 1. ScheduleActivity ActivateRewards

66. Rewards Cadence Eligibility Driver1 Workflow Worker Workflow Worker Workflow Worker Start Timer: 30 days Activity Worker Activity Worker Activity Worker Activation

67. SOME NOTABLE FEATURES • Cross Datacenter Replication • Signals • Child workflows • Task routing • Heartbeating • Query • Visibility • Debuggability • Workflow Reset • Archival

68. SOME CADENCE USE CASES ● Freight load workflow ● Driver loyalty program ● Customer support workflows ● CI/CD/Deployment infrastructure ● End of month statement generation for each u4b customer ● Recalculate every hexagon on the city map every 1 minute for every city ● Tip processing in microservices architecture ● Managing Flink and Spark Jobs in Mesos or Yarn ● Customer loyalty Program ● Marketing email campaign management ● New datacenter provisioning ● Numerous other periodic jobs

69. CADENCE SUMMARY • Task Execution Service • Fault tolerant actors as a programming model • Perfect to support DSL • Inspired by AWS Simple Workflow (SWF) API • Scalable and fault tolerant • Open sourced

70. QUESTIONS? ● Website: cadenceworkflow.io ● Server: github.com/uber/cadence ● Go Client: github.com/uber-go/cadence-client ● Java Client: github.com/uber/cadence-java-client

Uber Cadence Fault Tolerant Actor Framework

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Similar to Uber Cadence Fault Tolerant Actor Framework

Similar to Uber Cadence Fault Tolerant Actor Framework (20)

Recently uploaded

Recently uploaded (20)

Uber Cadence Fault Tolerant Actor Framework

Editor's Notes