The document provides an overview of AWS Step Functions, an orchestration service that enables users to model workflows as state machines using JSON. It discusses various state types, error handling techniques, design patterns, and real-world case studies showcasing its application in improving efficiency and error management. Additionally, it covers the differences between orchestration and choreography, the cost considerations, and scenarios best suited for implementing Step Functions.

1. Deliver customer value
FASTER with Step Functions
Yan Cui @theburningmonk

2. What is step functions?
How it works?
When to use it?
Orchestration vs Choreography
Real-world case studies
Design patterns
Agenda

3. Step Functions

4. orchestration service that allows you to
model workflows as state machines

5. design with JSON
https://states-language.net/spec.html

7. Step Functions OOP
class
instance
execution
input arguments

8. start a state machine via..
StepFunctions
.startExecution(req)
.promise()

9. start a state machine via..
API Gateway
StepFunctions
.startExecution(req)
.promise()

10. start a state machine via..
EventBridge
including cron
StepFunctions
.startExecution(req)
.promise()
API Gateway

15. state transitions

17. state transitions

18. $25 PER MILLION

19. $25 PER MILLION
15X LAMBDA PRICING!

20. https://aws.amazon.com/about-aws/whats-new/2019/12/introducing-aws-step-functions-express-workflows

22. Yan Cui
http://theburningmonk.com
@theburningmonk
AWS user for 10 years

23. http://bit.ly/yubl-serverless

24. Yan Cui
http://theburningmonk.com
@theburningmonk
Developer Advocate @

26. Yan Cui
http://theburningmonk.com
@theburningmonk
Independent Consultant
advise
training development

27. theburningmonk.com/courses

28. homeschool.dev

29. designing state machines

30. types of states

31. "TaskState": {
"Type": "Task",
"Resource": "arn:aws:lambda:us-east-1:1234556788:function:hello-world",
"Next": "NextState",
"TimeoutSeconds": 300
}
Task
Performs a task.

32. "TaskState": {
"Type": "Task",
"Resource": "arn:aws:lambda:us-east-1:1234556788:function:hello-world",
"Next": "NextState",
"TimeoutSeconds": 300
}
Task
Performs a task.

33. "TaskState": {
"Type": "Task",
"Resource": "arn:aws:lambda:us-east-1:1234556788:function:hello-world",
"Next": "NextState",
"TimeoutSeconds": 300
}
Task
Performs a task.

34. "TaskState": {
"Type": "Task",
"Resource": "arn:aws:lambda:us-east-1:1234556788:function:hello-world",
"Next": "NextState",
"TimeoutSeconds": 300
}
Task
Performs a task.

35. "TaskState": {
"Type": "Task",
"Resource": "arn:aws:lambda:us-east-1:1234556788:function:hello-world",
"Next": "NextState",
"TimeoutSeconds": 300
}
Task
Defaults to 60s, even if function has longer timeout
Performs a task.

36. "TaskState": {
"Type": "Task",
"Resource": "arn:aws:lambda:us-east-1:1234556788:function:hello-world",
"Next": "NextState",
"TimeoutSeconds": 300
}
Task
Defaults to 60s, even if function has longer timeout
Set this to match your function’s timeout
Performs a task.

37. "TaskState": {
"Type": "Task",
"Resource": "arn:aws:lambda:us-east-1:1234556788:function:hello-world",
"Next": "NextState",
"TimeoutSeconds": 300
}
Task
Doesn’t have to be Lambda function.
Performs a task.

38. "TaskState": {
"Type": "Task",
"Resource": "arn:aws:lambda:us-east-1:1234556788:function:hello-world",
"Next": "NextState",
"TimeoutSeconds": 300
}
Task
Doesn’t have to be Lambda function.
Performs a task.
Activity, AWS Batch, ECS task, DynamoDB,
SNS, SQS, AWS Glue, SageMaker

39. { “x”: 42, “y”: 13 }
$ =>
{
“x”: 42,
“y”: 13
}
"choose": {
"Type": "Choice",
"Choices": [
{
"And": [
{
"Variable": "$.x",
"NumericGreaterThanEquals": 42
},
{
"Variable": "$.y",
"NumericLessThan": 42
}
],
"Next": "subtract"
}
],
"Default": "add"
},

40. { “x”: 42, “y”: 13 }
$ =>
{
“x”: 42,
“y”: 13
}
"subtract": {
"Type": "Task",
“Resource": "arn:aws:lambda:…",
"Next": "double",
"ResultPath": "$.n"
},

41. { “x”: 42, “y”: 13 }
$ =>
{
“x”: 42,
“y”: 13
}
module.exports.handler = async (input, context) => {
return input.x - input.y
}
$.n

42. { “x”: 42, “y”: 13 }
$ =>
{
“x”: 42,
“y”: 13,
“n”: 29
}

43. { “x”: 42, “y”: 13 }
$ =>
{
“x”: 42,
“y”: 13,
“n”: 29
}
"double": {
“Type": "Task",
“Resource”: ”arn:aws:lambda:...",
“InputPath": "$.n",
“End": true
}

44. { “x”: 42, “y”: 13 }
$ =>
{
“x”: 42,
“y”: 13,
“n”: 29
}
module.exports.handler = async (input, context) => {
return input * 2;
}
$.n
$

45. { “x”: 42, “y”: 13 }
$ => 58
"double": {
“Type": "Task",
“Resource”: ”arn:aws:lambda:...",
“InputPath": "$.n",
“End": true
}

46. { “x”: 42, “y”: 13 }
{ “output”: 58 }

47. "NoOp": {
"Type": "Pass",
"Result": {
"is": 42
},
"ResultPath": "$.the_answer_to_the_question_of_life_the_universe_and_everything",
"Next": "NextState"
}
Pass
Passes input to output without doing any work.

48. "NoOp": {
"Type": "Pass",
"Result": {
"is": 42
},
"ResultPath": "$.the_answer_to_the_question_of_life_the_universe_and_everything",
"Next": "NextState"
}
Pass
Passes input to output without doing any work.

49. Pass
Passes input to output without doing any work.
{ }
{
“the_answer_to_the_question_of_life_the_universe_and_everything”: {
“is”: 42
}
}

50. "WaitTenSeconds" : {
"Type" : "Wait",
"Seconds" : 10,
"Next": "NextState"
}
Wait
Wait before transitioning to next state.
"WaitTenSeconds" : {
"Type" : "Wait",
“Timestamp": "2018-08-08T01:59:00Z",
"Next": "NextState"
}

51. "WaitTenSeconds" : {
"Type" : "Wait",
"Seconds" : 10,
"Next": "NextState"
}
Wait
Wait before transitioning to next state.
"WaitTenSeconds" : {
"Type" : "Wait",
“Timestamp": "2018-08-08T01:59:00Z",
"Next": "NextState"
}

52. "WaitTenSeconds" : {
"Type" : "Wait",
"SecondsPath" : "$.waitTime",
"Next": "NextState"
}
Wait
Wait before transitioning to next state.
"WaitTenSeconds" : {
"Type" : "Wait",
“TimestampPath": “$.waitUntil”,
"Next": "NextState"
}

53. "ChoiceState": {
"Type" : "Choice",
"Choices": [
{
"Variable": "$.name",
"StringEquals": "Neo"
"Next": "RedPill"
}
],
"Default": "BluePill"
}
Choice
Adds branching logic to the state machine.

54. "ChoiceState": {
"Type" : "Choice",
"Choices": [
{
"Variable": "$.name",
"StringEquals": "Neo"
"Next": "RedPill"
}
],
"Default": "BluePill"
}
Choice
Adds branching logic to the state machine.

55. "ChoiceState": {
"Type" : "Choice",
"Choices": [
{
"Variable": "$.name",
"StringEquals": "Neo"
"Next": "RedPill"
}
],
"Default": "BluePill"
}
Choice
Adds branching logic to the state machine.

56. "ChoiceState": {
"Type" : "Choice",
"Choices": [
{
"Variable": "$.name",
"StringEquals": "Neo"
"Next": "RedPill"
}
],
"Default": "BluePill"
}
Choice
Adds branching logic to the state machine.
{
“And”: [
{
"Variable": "$.name",
"StringEquals": “Cypher"
},
{
"Variable": "$.afterNeoIsRescued",
"BooleanEquals": true
},
],
"Next": "BluePill"
}

57. "FunWithMath": {
"Type": "Parallel",
"Branches": [
{
"StartAt": "Add",
"States": {
"Add": {
"Type": "Task",
"Resource": "arn:aws:lambda:us-east-1:1234556788:function:add",
"End": true
}
}
},
…
],
"Next": "NextState"
}
Parallel
Performs tasks in parallel.

58. "FunWithMath": {
"Type": "Map",
"Iterator": [
{
"StartAt": "DoSomething",
"States": {
"Add": {
"Type": "Task",
"Resource": “arn:aws:lambda:us-east-1:1234556788:function:doSomething",
"End": true
}
}
},
…
],
"Next": "NextState"
}
Map
Dynamic parallelism!

59. "FunWithMath": {
"Type": "Map",
"Iterator": [
{
"StartAt": "DoSomething",
"States": {
"Add": {
"Type": "Task",
"Resource": "arn:aws:lambda:us-east-1:1234556788:function:doSomething",
"End": true
}
}
},
…
],
"Next": "NextState"
}
Map
Dynamic parallelism!
e.g. [ { … }, { … }, { … } ]

60. "SuccessState" : {
"Type" : "Succeed"
}
Succeed
Terminates the state machine successfully.

61. "FailState" : {
"Type" : “Fail",
"Error" : "TypeA",
"Cause" : "Kaiju Attack",
}
Fail
Terminates the state machine and mark it as failure.

62. https://aws.amazon.com/about-aws/whats-new/2019/08/aws-step-function-adds-support-for-nested-workflows

63. WHEN TO USE STEP FUNCTIONS?

64. $25 PER MILLION
15X LAMBDA PRICING!

65. another moving part to manage

66. what’s the return-on-investment?

67. Pros Cons
$$$

68. https://aws.amazon.com/about-aws/whats-new/2019/12/introducing-aws-step-functions-express-workflows

69. Pros Cons
visual
$$$

70. Great for collaboration with team
members who are not engineers

71. Makes it easy for anyone to identify
and debug application errors.

72. Pros Cons
visual
$$$
error handling

73. Makes deciding on timeout setting for Lambda
functions easier when you don’t have to consider
retry and exponential backoff, etc.

74. Surfaces error handling for everyone to see, makes it
easy for others to see without digging into the code.

75. Pros Cons
visual
$$$
error handling
audit

81. Pros Cons
visual
$$$
error handling
audit

82. business critical workflows
what: stuff that makes money, e.g. payment and
subscription flows.
why: more robust error handling worth the premium.

83. complex workflows
what: complex workflows that involves many states,
branching logic, etc.
why: visual workflow is a powerful design (for product)
and diagnostic tool (for customer support).

84. long running workflows
what: workflows that cannot complete in 15 minutes
(Lambda limit).
why: AWS discourages recursive Lambda functions,
Step Functions gives you explicit branching checks,
and can timeout at workflow level.

85. https://aws.amazon.com/about-aws/whats-new/2019/12/introducing-aws-step-functions-express-workflows

87. https://docs.aws.amazon.com/step-functions/latest/dg/concepts-standard-vs-express.html

88. https://docs.aws.amazon.com/step-functions/latest/dg/concepts-standard-vs-express.html

89. https://docs.aws.amazon.com/step-functions/latest/dg/concepts-standard-vs-express.html

90. https://docs.aws.amazon.com/step-functions/latest/dg/concepts-standard-vs-express.html

93. use Express Workflows for high-throughput,
short-lived workflows (OLTP)

94. Pros Cons
visual
$$$
error handling
audit

96. Orchestration Choreography

97. Orchestration Choreography

98. orchestration within a bounded-context
choreography between bounded-contexts
Rule of Thumb

99. bounded context
fits within my head
high cohesion
same ownership

100. bounded context
the workflow doesn’t exist
as a standalone concept,
but as the sum of a series of
loosely connected parts
Lambda
Lambda
Lambda
SQS
SQS
API Gateway

101. bounded context A bounded context B bounded context C
EventBridge SNS

102. https://lumigo.io/blog/5-reasons-why-you-should-use-eventbridge-instead-of-sns

103. don’t forget to
emit events from
the workflow

104. orchestration within a bounded-context
choreography between bounded-contexts
Rule of Thumb

105. Step Functions in the wild

106. Backend system was slow and had
timing issue, so they needed to add a
90s delay before processing payment.
Step Functions was the most cost-
efficient and scalable way to
implement this wait.

107. Update nutritional info on over 100
brands to comply with FDA regulations.
Reduced processing time from 36 hours
to 10 seconds.

108. Transcode video segments in parallel.
Reduced processing time from ~20 mins
to ~2 mins.

109. Manages their food delivery experience.

110. Automates subscription fulfilments with
Step Functions.

111. Automates subscription billing system
with Step Functions.

112. Implements payment processing, and
subscription fulfillment systems with Step
Functions, and many more.

114. sagas

115. managing failures in a distributed transaction

119. Success path!

120. Failure paths…

121. "BookFlight": {
"Type": “Task",
"Resource": “arn:aws:lambda:…”,
"Catch": [
{
"ErrorEquals": [ “States.ALL" ],
"ResultPath": "$.BookFlightError",
"Next": “CancelFlight"
}
],
"ResultPath": "$.BookFlightResult",
"Next": "BookRental"
},

122. "BookFlight": {
"Type": “Task",
"Resource": “arn:aws:lambda:…”,
"Catch": [
{
"ErrorEquals": [ “States.ALL" ],
"ResultPath": "$.BookFlightError",
"Next": “CancelFlight"
}
],
"ResultPath": "$.BookFlightResult",
"Next": "BookRental"
},

123. "BookFlight": {
"Type": “Task",
"Resource": “arn:aws:lambda:…”,
"Catch": [
{
"ErrorEquals": [ “States.ALL" ],
"ResultPath": "$.BookFlightError",
"Next": “CancelFlight"
}
],
"ResultPath": "$.BookFlightResult",
"Next": "BookRental"
},

124. "CancelFlight": {
"Type": “Task",
"Resource": “arn:aws:lambda:…”,
"Catch": [
{
"ErrorEquals": [ “States.ALL" ],
"ResultPath": "$.CancelFlightError",
"Next": “CancelFlight"
}
],
"ResultPath": "$.CancelFlightResult",
"Next": “CancelHotel"
},

125. "CancelFlight": {
"Type": “Task",
"Resource": “arn:aws:lambda:…”,
"Catch": [
{
"ErrorEquals": [ “States.ALL" ],
"ResultPath": "$.CancelFlightError",
"Next": “CancelFlight"
}
],
"ResultPath": "$.CancelFlightResult",
"Next": “CancelHotel"
},

126. https://github.com/theburningmonk/lambda-saga-pattern

127. callbacks

128. https://docs.aws.amazon.com/step-functions/latest/dg/connect-to-resource.html#connect-wait-token

130. TaskToken

132. TaskToken
?

133. ?

134. sendTaskSuccess(TaskToken)
?

136. "Publish SQS message": {
"Type": "Task",
"Resource": "arn:aws:states:::sqs:sendMessage.waitForTaskToken",
"Parameters": {
"QueueUrl": !Ref MyQueue,
"MessageBody": {
"Token.$": "$$.Task.Token"
}
},
"Next": "NextState"
}

137. "Publish SQS message": {
"Type": "Task",
"Resource": "arn:aws:states:::sqs:sendMessage.waitForTaskToken",
"Parameters": {
"QueueUrl": !Ref MyQueue,
"MessageBody": {
"Token.$": "$$.Task.Token"
}
},
"Next": "NextState"
}

138. "Publish SQS message": {
"Type": "Task",
"Resource": "arn:aws:states:::sqs:sendMessage.waitForTaskToken",
"Parameters": {
"QueueUrl": !Ref MyQueue,
"MessageBody": {
"Token.$": "$$.Task.Token"
}
},
"Next": "NextState"
}

139. "Publish SQS message": {
"Type": "Task",
"Resource": "arn:aws:states:::sqs:sendMessage.waitForTaskToken",
"Parameters": {
"QueueUrl": !Ref MyQueue,
"MessageBody": {
"Token.$": "$$.Task.Token"
}
},
"Next": "NextState"
}

140. https://docs.aws.amazon.com/step-functions/latest/dg/input-output-contextobject.html

141. "Publish SQS message": {
"Type": "Task",
"Resource": "arn:aws:states:::sqs:sendMessage.waitForTaskToken",
"Parameters": {
"QueueUrl": !Ref MyQueue,
"MessageBody": {
"Token.$": "$$.Task.Token",
"ExecutionId.$": "$$.Execution.Id",
"StateMachineId.$": "$$.StateMachine.Id"
}
},
"Next": "NextState"
}

142. sendTaskSuccess(TaskToken)
?
needs access to
Step Functions

143. HTTP POST
?
sendTaskSuccess

144. https://go.aws/38KynD1

145. TaskToken

146. map-reduce

147. Map

148. Map
…

149. Map
…
{ … }
{ … }
{ … }
{ … }
{ … }

150. Map
…
{ … }
{ … }
{ … }
{ … }
{ … }
[{ … }, { … } … ]

151. Map
…
{ … }
{ … }
{ … }
{ … }
{ … }
[{ … }, { … } … ] Reduce

152. https://github.com/awsdocs/aws-step-functions-developer-guide/blob/master/doc_source/limits.md

153. https://github.com/awsdocs/aws-step-functions-developer-guide/blob/master/doc_source/limits.md
use S3 to store
large payloads

155. theburningmonk.com/hire-me
Advise
Training Delivery
“Fundamentally, Yan has improved our team by increasing our
ability to derive value from AWS and Lambda in particular.”
Nick Blair
Tech Lead

156. productionreadyserverless.com

157. @theburningmonk
theburningmonk.com
github.com/theburningmonk