This document discusses AWS Step Functions and how it can be used to coordinate serverless applications and functions. It provides examples of how Step Functions allows developers to sequence functions, select functions based on data, retry functions, run functions in parallel, and add error handling with try/catch/finally. It also discusses how Step Functions supports long-running workflows through activities that can run for hours. Overall, Step Functions allows developers to build serverless workflows and step-by-step applications through a visual interface and JSON definitions.

1. ©2017, Amazon Web Services, Inc. or its affiliates. All rights reserved
Pop-up Loft
Serverless Apps with
AWS Step Functions
Trevor Roberts Jr – Solutions Architect

2. λλ
λ
DBMS
λ
λ
λ
λ
λ
λ λ
λ
λ
Queue
Modern
Serverless
app

3. Modern
Serverless
app

4. “I want to sequence functions”
“I want to select functions based on data”
“I want to retry functions”
“I want try/catch/finally”
Functions into apps
“I have code that runs for hours”
“I want to run functions in parallel”

6. Coordination by method call
λ λ
λ
λ
λ
λ
λ

7. Coordination by function chaining
λ
λ
λ
λ
λλ
λ

8. Coordination by database
λλ
λ
λ
λλ
λ

9. Coordination by queues
λ
λ
λ
λ λλ
λ

10. Coordination must-haves
• Scales out
• Doesn’t lose state
• Deals with errors/timeouts
• Easy to build & operate
• Auditable

12. “State Machine”(noun)
1. A concept used by CompSci
profs for torturing
undergrads, full of arcane
math.
2. A practical way to build and
manage modern Serverless
Cloud apps.
Dictionary

13. Application Lifecycle in AWS Step Functions
Visualize in the
Console
Define in JSON Monitor
Executions

14. Define in JSON and Then Visualize in the Console
{
”Comment”: “Hello World Example",
"StartAt” : "HelloWorld”,
"States” : {
"HelloWorld” : {
"Type” : "Task",
"Resource” :
"arn:aws:lambda:REGION:ACCOUNT_ID:function:FUNCTI
ON_NAME”,
"End” : true
}
}
}

15. Execute One or One Million
Start
End
HelloWorld
Start
End
HelloWorld
Start
End
HelloWorld
Start
End
HelloWorld
Start
End
HelloWorld
Start
End
HelloWorld
Start
End
HelloWorld
Start
End
HelloWorld

16. Monitor Executions from the Console

17. Seven State Types
Task A single unit of work
Choice Adds branching logic
Parallel Fork and join the data across tasks
Wait Delay for a specified time
Fail Stops an execution and marks it as a failure
Succeed Stops an execution successfully
Pass Passes its input to its output

18. Build Visual Workflows from State Types
Task
Choice
Fail
Parallel

20. How it works – API-level
Create State Machine – input is a machine spec
in a JSON DSL
Run Machine – input JSON blob, returns
Execution ID
List Executions
Describe Execution
Stop Execution

21. Integrate with other AWS services
• Create state machines and Activities with AWS
CloudFormation
• Call AWS Step Functions with Amazon API Gateway
• Start state machines in response to events, or on a
schedule, with CloudWatch Events
• Monitor state machine executions with Amazon CloudWatch
• Log API calls with AWS CloudTrail

22. Serverless Human Approval Tasks
https://aws.amazon.com/blogs/compute/
implementing-serverless-manual-approval-steps-in-aws-step-functions-and-amazon-api-gateway/

23. Image Recognition and Processing Backend
Task
Choice
Fail
Parallel
https://github.com/awslabs/lambda-refarch-imagerecognition

24. EBS Snapshot Management
State Machine State Machine
create-snapshot
complete event
copy-snapshot
complete event
Primary Region DR Region
Parallel State
Choice
Tag Snapshot
Count Snapshots
CopyToDRRegion
Success
State
Delete
Snapshots
Choice
Tag SnapshotCopy
Count Snapshots
Pass State
Delete
Snapshots
Notification
Topic
Errors Sent
to SNS
Notification
Topic
Errors Sent
to SNS
https://github.com/awslabs/aws-step-functions-ebs-snapshot-mgmt
1. Tag
2. Count
3. Copy to DR region
4. Delete Expired
Whenever a new EBS
snapshot completes:
State Machines
invoked by Amazon
CloudWatch Events

25. “I want to sequence functions”
AWS Step Functions, we can easily change and iterate on
the application workflow of our food delivery service in
order to optimize operations and continually improve
delivery times. AWS Step Functions lets us dynamically
scale the steps in our food delivery algorithm so we can
manage spikes in customer orders and meet demand.
Mathias Nitzsche, CTO, foodpanda
“

28. var Twit = require('twit');
var T = new Twit(require('botfiles/config.js'));
exports.handler = function myBot(event, context, callback) {
var list = event.inputList;
var textToTweet = list.shift();
var output = { inputList: list }
T.post('statuses/update',
{ status: textToTweet }, function(err, reply) {
if (err) {
console.log('error:', err); context.fail();
} else {
console.log('tweet:', reply); callback(null, output);
}
});
};

29. “I want to select functions based on data”
“With AWS Step Functions, it was easy to build a multi-step product
updating system to ensure our database and website always have the
latest price and availability information.
AWS Step Functions let us replace a manual updating process with an
automated series of steps, including built-in retry conditions and error
handling, so we can reliably scale before a big show, and keep pace
with rapidly changing fashions.”
Jared Browarnik, CTO, TheTake
“

31. "ChoiceState": {
"Type" : "Choice",
"Choices": [
{
"Variable": "$.productSource",
"StringEquals": "screen-scrape",
"Next": "ScreenScrapeState"
},{
"Variable": "$.productSource",
"StringEquals": "vendor-a",
"Next": "VendorA"
},{
"Variable": "$.productSource",
"StringEquals": "vendor-b",
"Next": "VendorB"
},
{
"Variable": "$.productSource",
"StringEquals": "vendor-c",
"Next":"VendorC"
},{
"Variable": "$.productSource",
"StringEquals": "updateProduct",
"Next":"UpdateProduct"
}
],
"Default": "ScreenScrapeState”
}

32. “I want to retry functions”
We get transient errors from a RESTful
service we depend on, once every four
or five times we call it. But if we keep
retrying, it eventually works.
“

33. {
"Comment": "Call out to a RESTful service",
"StartAt": "Call out",
"States": {
"Call out": {
"Type": "Task",
"Resource":
"arn:aws:lambda:eu-central-1:123456789012:function:RestCallout",
"Retry": [
{ "ErrorEquals": [ "HandledError" ], "MaxAttempts": 10 }
],
"End": true
}
}
}

34. “I want to run functions in parallel”
We want to send the captured image to
three OCR providers and take the result
with the highest confidence value.“

36. "Send for OCR": {
"Type": "Parallel",
"Next": "Pick result",
"Branches": [
{
"StartAt": "Prep1",
"States": {
"Prep1": {
"Type": "Pass",
"Result": { "inputList": [ "OCR Provider 1" ] },
"Next": "Go1"
},
"Go1": {
"Type": "Task",
"Resource": "arn:aws:lambda:eu-central-1:123456789012:function:StatesBot",
"End": true
}
}

37. Where does transient application
state live?
In the machine, in JSON texts
passing from state to state.A:
Q:

38. Input processing
{
"title": "Numbers to add",
"numbers": [ 3, 4 ]
}
{
"Type": "Task",
"InputPath": "$.numbers",
"Resource": "arn:aws:lambda…"
…
[ 3, 4 ]
Raw input:
State spec:
Task input:

39. Input processing
Q: InputPath not provided?
A: State gets raw input as-is.
Q: InputPath is null?
A: State gets an empty JSON object: {}
Q: InputPath produces plural output?
A: State gets it wrapped in a JSON array.

40. Result placement
{
"title": "Numbers to add",
"numbers": [ 3, 4 ]
}
{
"Type": "Task",
"InputPath": "$.numbers",
"ResultPath": "$.sum”,
…
Raw input:
State spec:
Output: {
"title": "Numbers to add",
"numbers": [ 3, 4 ],
”sum": 7
}

41. Result placement
Q: ResultPath not provided?
A: Input discarded, raw output used.
Q: ResultPath is null?
A: State input is state output.
Q: ResultPath produces plural output?
A: Not allowed, validator won’t accept.

42. “I want try/catch/finally”
AWS Step Functions makes it simple to coordinate information
from many different infrastructure systems using easy to design
workflows and create a more intelligent monitoring system for our
Platform as a Service (PaaS).
With AWS Step Functions, we can reliably automate monitoring
decisions and actions in order to reduce human intervention by
over 60%, which improves infrastructure operation productivity and
customer application availability on our platform.
Pedro Pimenta, VP R&D, OutSystems
“

43. 13 AWS Lambda Task States
6 Choice States
1 Fail State
“I want try/catch/finally”

45. "Access Media": {
"Type": "Task",
"Resource": "arn:aws:lambda:eu-central-1:123456789012:function:FindMedia",
"TimeoutSeconds": 2,
"Next": "Graceful Exit",
"Retry": [
{
"ErrorEquals": [ "States.Timeout" ],
"IntervalSeconds": 2, "MaxAttempts": 2, "BackoffRate": 1.5
}
],
"Catch": [
{ "ErrorEquals": [ "States.ALL" ], "Next": "Clean Up" }
]
},

46. “I have code that runs for hours”
We need to gather data from our
production line, in units of 8-hour shifts.
“

47. More APIs
Register Activity Task - Returns ARN
Poll For task (by ARN)
Report Success
Report Failure
Report Heartbeat

48. "NextShift": {
"Type": "Wait",
"TimestampPath": "$.ShiftStart",
"Next": "Gather Plant Data"
},
"Gather Plant Data": {
"Type": "Task",
"Resource":
"arn:aws:states:ap-northeast-1:123456789012:activity:PlWatch",
"TimeoutSeconds": 30000,
"HeartBeatSeconds": 120,
"Next": "Clean up"
}

49. “I want to sequence functions”
“I want to select functions based on data”
“I want to retry functions”
“I want try/catch/finally”
Is this you?
“I have code that runs for hours”
“I want to run functions in parallel”

50. (Demo)

52. https://states-language.net/spec.html

54. 2.5¢
How much?
per thousand
state transitions
4,000 free
transitions/month
Free tier:

55. ©2017, Amazon Web Services, Inc. or its affiliates. All rights reserved
Pop-up Loft
Thank You