AWS Step Functions 実践
AWS Premeir Night #3
クラスメソッド株式会社
菊池 修治
• 菊池 修治(きくち しゅうじ)
– クラスメソッド AWS事業
– Solutions Architect
– AWS Certified in All 5!
– SIer à 製造業 à クラスメソッド
– 好きなAWSサービス
• VPC、S3、Step Functions
⾃⼰紹介
Agenda
• AWS Step Functions とは
• Amazon State Language
• Step Functions の実⾏
– State Machine の実⾏⽅法
– Activity Task の実⾏
Agenda
• AWS Step Functions とは
• Amazon State Language
• Step Functions の実⾏
– State Machine の実⾏⽅法
– Activity Task の実⾏
AWS Step Functions
re:Invent 2016 にて発表!
AWS Step Functions
• 「Step Functionsやってみた」ブログ公開中
– http://dev.classmethod.jp/referencecat/aws-step-functions/
AWS Step Functions
• 分散されたアプリケーションをビジュアルに
コーディネート
– Lambdaなどの複数アプリケーションを⼀連の
ワークフローとして定義・実⾏
– 処理の流れ・ステップをビジュアル化
AWS Step Functions
AWS Step Functions
• ユースケース
– シーケンシャルなアプリケーションの実⾏
– タスクの並列化
– 処理の条件分岐
– エラーハンドリング
– ⻑時間の実⾏
AWS Step Functions
• 利点
– アプリケーション間を疎結合に
– 汎⽤性・再利⽤性の向上
– 実⾏結果の管理
AWS Step Functions
• コンポーネント
– State Machine:定義された⼀連のフロー
– State Language:State Machineを定義する⾔語
Agenda
• AWS Step Functions とは
• Amazon State Language
• Step Functions の実⾏
– State Machine の実⾏⽅法
– Activity Task の実⾏
Amazon State Language
• JSONで記述
{
"Comment": "A Retry example of the Amazon States Language using an AWS Lambda Function",
"StartAt": "HelloWorld",
"States": {
"HelloWorld": {
"Type": "Task",
"Resource": "arn:aws:lambda:REGION:ACCOUNT_ID:function:FUNCTION_NAME",
"Retry": [
{
"ErrorEquals": ["HandledError"],
"IntervalSeconds": 1,
"MaxAttempts": 2,
"BackoffRate": 2.0
},
{
"ErrorEquals": ["States.TaskFailed"],
"IntervalSeconds": 30,
"MaxAttempts": 2,
"BackoffRate": 2.0
},
{
"ErrorEquals": ["States.ALL"],
"IntervalSeconds": 5,
"MaxAttempts": 5,
"BackoffRate": 2.0
}
],
"End": true
}
}
}
Amazon State Language
• State (= Step)をつなげてフロー(State Machine)
を作成
• ⼀度作成したState Machineは変更・修正できない
Amazon State Language
• 7つの State
– Task :アプリケーションの実⾏
– Choice :分岐条件
– Parallel :Taskの並列実⾏
– Wait :待ち
– Fail :異常終了
– Succeed :正常終了
– Pass :なにもしない
Amazon State Language
• 7つの State
– Task :アプリケーションの実⾏
– Choice :分岐条件
– Parallel :Taskの並列実⾏
– Wait :待ち
– Fail :異常終了
– Succeed :正常終了
– Pass :なにもしない
Amazon State Language
• Task
"TaskState": {
"Comment": "Task State example",
"Type": "Task",
"Resource": "arn:aws:lambda:us-east-1:123456789012:function:HelloFunctio
"Next": "NextState",
"TimeoutSeconds": 300,
"Retry": [
{
"ErrorEquals": [ "ErrorA", "ErrorB" ],
"IntervalSeconds": 1,
"BackoffRate": 2,
"MaxAttempts": 2
}
]
}
Amazon State Language
• Task
"TaskState": {
"Comment": "Task State example",
"Type": "Task",
"Resource": "arn:aws:lambda:us-east-1:123456789012:function:HelloFunctio
"Next": "NextState",
"TimeoutSeconds": 300,
"Retry": [
{
"ErrorEquals": [ "ErrorA", "ErrorB" ],
"IntervalSeconds": 1,
"BackoffRate": 2,
"MaxAttempts": 2
}
]
}
Amazon State Language
• Task
"TaskState": {
"Comment": "Task State example",
"Type": "Task",
"Resource": "arn:aws:lambda:us-east-1:123456789012:function:HelloFunctio
"Next": "NextState",
"TimeoutSeconds": 300,
"Retry": [
{
"ErrorEquals": [ "ErrorA", "ErrorB" ],
"IntervalSeconds": 1,
"BackoffRate": 2,
"MaxAttempts": 2
}
]
}
Amazon State Language
• Task
"TaskState": {
"Comment": "Task State example",
"Type": "Task",
"Resource": "arn:aws:lambda:us-east-1:123456789012:function:HelloFunctio
"Next": "NextState",
"TimeoutSeconds": 300,
"Retry": [
{
"ErrorEquals": [ "ErrorA", "ErrorB" ],
"IntervalSeconds": 1,
"BackoffRate": 2,
"MaxAttempts": 2
}
]
}
Amazon State Language
• Choice
"ChoiceState": {
"Type" : "Choice",
"Choices": [
{
"Variable": "$.foo",
"NumericEquals": 1,
"Next": "FirstMatchState"
},
{
"Variable": "$.foo",
"NumericEquals": 2,
"Next": "SecondMatchState”
}
],
"Default": "DefaultState"
},
• StringEquals
• StringLessThan
• StringGreaterThan
• StringLessThanEquals
• StringGreaterThanEquals
• NumericEquals
• NumericLessThan
• NumericGreaterThan
• NumericLessThanEquals
• NumericGreaterThanEquals
• BooleanEquals
• TimestampEquals
• TimestampLessThan
• TimestampGreaterThan
• TimestampLessThanEquals
• TimestampGreaterThanEquals
Amazon State Language
• Choice "ChoiceStateX": {
"Type" : "Choice",
"Choices": [
{
"Not": {
"Variable": "$.type",
"StringEquals": "Private”
},
"Next": "Public”
},
{
"And": [
{
"Variable": "$.value",
"NumericGreaterThanEquals": 20
},
{
"Variable": "$.value",
"NumericLessThan": 30
}
],
"Next": "ValueInTwenties”
}
],
"Default": "DefaultState”
},
• Not
• Or
• And
Amazon State Language
• Choice "ChoiceStateX": {
"Type" : "Choice",
"Choices": [
{
"Not": {
"Variable": "$.type",
"StringEquals": "Private”
},
"Next": "Public”
},
{
"And": [
{
"Variable": "$.value",
"NumericGreaterThanEquals": 20
},
{
"Variable": "$.value",
"NumericLessThan": 30
}
],
"Next": "ValueInTwenties”
}
],
"Default": "DefaultState”
},
• Not
• Or
• And
Amazon State Language
• Choice "ChoiceStateX": {
"Type" : "Choice",
"Choices": [
{
"Not": {
"Variable": "$.type",
"StringEquals": "Private”
},
"Next": "Public”
},
{
"And": [
{
"Variable": "$.value",
"NumericGreaterThanEquals": 20
},
{
"Variable": "$.value",
"NumericLessThan": 30
}
],
"Next": "ValueInTwenties”
}
],
"Default": "DefaultState”
},
• Not
• Or
• And
Amazon State Language
• Parallel
– 全ての分岐が完了
してから次のState
へ進む
"Parallel": {
"Type": "Parallel",
"Next": "Final State",
"Branches": [
{
"StartAt": "Wait 20s",
"States": {
"Wait 20s": {
"Type": "Wait",
"Seconds": 20,
"End": true
}
}
},
{
"StartAt": "Wait 10s",
"States": {
"Wait 10s": {
"Type": "Wait",
"Seconds": 10,
"End": true
}
}
}
]
},
Amazon State Language
• Wait
– Seconds
– SecondsPath
– Timestamp
– TimestampPath
"wait_ten_seconds" : {
"Type" : "Wait",
"Seconds" : 10,
"Next": "NextState”
}
"wait_until" : {
"Type": "Wait",
"Timestamp": "2016-03-14T01:59:00Z",
"Next": "NextState”
}
"wait_until" : {
"Type": "Wait",
"TimestampPath": "$.expirydate",
"Next": "NextState”
}
Agenda
• AWS Step Functions とは
• Amazon State Language
• Step Functions の実⾏
– State Machine の実⾏⽅法
– Activity Task の実⾏
State Machine の実⾏
• start-execution API
Step Functions
State Machine
start-execution
State Machine の実⾏
• start-execution API
Lambda Step Functions
State Machine
start-execution
State Machine の実⾏
• start-execution API
API
Gateway
S3
SNS
Cloud Watch
Event
Lambda Step Functions
State Machine
start-execution
State Machine の実⾏
• 実⾏結果
– State Machine ごとに管理
State Machine の実⾏
• 実⾏結果
– State Machine の実⾏毎に管理
State Machine の実⾏
• 実⾏結果
– State 毎に管理
Agenda
• AWS Step Functions とは
• Amazon State Language
• Step Functions の実⾏
– State Machine の実⾏⽅法
– Activity Task の実⾏
Activity Taskの実⾏
• Lambda以外のアプリケーションの実⾏
–EC2
–オンプレミス
• Pull型の実⾏
– アプリケーションからポーリング
– LambdaはPush型
Activity Taskの実⾏
• Activityの作成
Activity Taskの実⾏
• Activityの作成
設定項⽬は名前だけ
ARNが作成される
Activity Taskの実⾏
• State Machineの作成
{
"Comment": "An example using a Task state.",
"StartAt": "getGreeting",
"Version": "1.0",
"TimeoutSeconds": 300,
"States":
{
"getGreeting": {
"Type": "Task",
"Resource": "arn:aws:states:eu_central-1:123456789012:activity:get-greeting",
"End": true
}
}
}
Activity Taskの実⾏
• アプリケーションからポーリング
– Get-Activity-Task API
Polling
Get-Activity-Task
--Activity ARN
--Worker Name (Option)
Worker
State Machine
Activity Taskの実⾏
• State Machine実⾏
Input
TaskToken
In progress
Get-Activity-Task
--Activity ARN
--Worker Name (Option)
Worker
State Machine
Activity Taskの実⾏
• 実⾏完了
Success
Send-Task-Success
--Task-Token
--Output
Worker
State Machine
Activity Taskの実⾏
• 分散アプリケーション
Workers
State Machines
まとめ
• Step Functionsを使うことで分散アプリケーション
による複雑な処理を簡単に定義・実⾏・管理
• マイクロサービス化、サーバレス化
• Lambdaだけでなくサーバ上のアプリケーションも
組み込み可能
classmethod.jp44

AWS Step Functions 実践

  • 1.
    AWS Step Functions実践 AWS Premeir Night #3 クラスメソッド株式会社 菊池 修治
  • 2.
    • 菊池 修治(きくちしゅうじ) – クラスメソッド AWS事業 – Solutions Architect – AWS Certified in All 5! – SIer à 製造業 à クラスメソッド – 好きなAWSサービス • VPC、S3、Step Functions ⾃⼰紹介
  • 3.
    Agenda • AWS StepFunctions とは • Amazon State Language • Step Functions の実⾏ – State Machine の実⾏⽅法 – Activity Task の実⾏
  • 4.
    Agenda • AWS StepFunctions とは • Amazon State Language • Step Functions の実⾏ – State Machine の実⾏⽅法 – Activity Task の実⾏
  • 5.
    AWS Step Functions re:Invent2016 にて発表!
  • 6.
    AWS Step Functions •「Step Functionsやってみた」ブログ公開中 – http://dev.classmethod.jp/referencecat/aws-step-functions/
  • 7.
    AWS Step Functions •分散されたアプリケーションをビジュアルに コーディネート – Lambdaなどの複数アプリケーションを⼀連の ワークフローとして定義・実⾏ – 処理の流れ・ステップをビジュアル化
  • 8.
  • 9.
    AWS Step Functions •ユースケース – シーケンシャルなアプリケーションの実⾏ – タスクの並列化 – 処理の条件分岐 – エラーハンドリング – ⻑時間の実⾏
  • 10.
    AWS Step Functions •利点 – アプリケーション間を疎結合に – 汎⽤性・再利⽤性の向上 – 実⾏結果の管理
  • 11.
    AWS Step Functions •コンポーネント – State Machine:定義された⼀連のフロー – State Language:State Machineを定義する⾔語
  • 12.
    Agenda • AWS StepFunctions とは • Amazon State Language • Step Functions の実⾏ – State Machine の実⾏⽅法 – Activity Task の実⾏
  • 13.
    Amazon State Language •JSONで記述 { "Comment": "A Retry example of the Amazon States Language using an AWS Lambda Function", "StartAt": "HelloWorld", "States": { "HelloWorld": { "Type": "Task", "Resource": "arn:aws:lambda:REGION:ACCOUNT_ID:function:FUNCTION_NAME", "Retry": [ { "ErrorEquals": ["HandledError"], "IntervalSeconds": 1, "MaxAttempts": 2, "BackoffRate": 2.0 }, { "ErrorEquals": ["States.TaskFailed"], "IntervalSeconds": 30, "MaxAttempts": 2, "BackoffRate": 2.0 }, { "ErrorEquals": ["States.ALL"], "IntervalSeconds": 5, "MaxAttempts": 5, "BackoffRate": 2.0 } ], "End": true } } }
  • 14.
    Amazon State Language •State (= Step)をつなげてフロー(State Machine) を作成 • ⼀度作成したState Machineは変更・修正できない
  • 15.
    Amazon State Language •7つの State – Task :アプリケーションの実⾏ – Choice :分岐条件 – Parallel :Taskの並列実⾏ – Wait :待ち – Fail :異常終了 – Succeed :正常終了 – Pass :なにもしない
  • 16.
    Amazon State Language •7つの State – Task :アプリケーションの実⾏ – Choice :分岐条件 – Parallel :Taskの並列実⾏ – Wait :待ち – Fail :異常終了 – Succeed :正常終了 – Pass :なにもしない
  • 17.
    Amazon State Language •Task "TaskState": { "Comment": "Task State example", "Type": "Task", "Resource": "arn:aws:lambda:us-east-1:123456789012:function:HelloFunctio "Next": "NextState", "TimeoutSeconds": 300, "Retry": [ { "ErrorEquals": [ "ErrorA", "ErrorB" ], "IntervalSeconds": 1, "BackoffRate": 2, "MaxAttempts": 2 } ] }
  • 18.
    Amazon State Language •Task "TaskState": { "Comment": "Task State example", "Type": "Task", "Resource": "arn:aws:lambda:us-east-1:123456789012:function:HelloFunctio "Next": "NextState", "TimeoutSeconds": 300, "Retry": [ { "ErrorEquals": [ "ErrorA", "ErrorB" ], "IntervalSeconds": 1, "BackoffRate": 2, "MaxAttempts": 2 } ] }
  • 19.
    Amazon State Language •Task "TaskState": { "Comment": "Task State example", "Type": "Task", "Resource": "arn:aws:lambda:us-east-1:123456789012:function:HelloFunctio "Next": "NextState", "TimeoutSeconds": 300, "Retry": [ { "ErrorEquals": [ "ErrorA", "ErrorB" ], "IntervalSeconds": 1, "BackoffRate": 2, "MaxAttempts": 2 } ] }
  • 20.
    Amazon State Language •Task "TaskState": { "Comment": "Task State example", "Type": "Task", "Resource": "arn:aws:lambda:us-east-1:123456789012:function:HelloFunctio "Next": "NextState", "TimeoutSeconds": 300, "Retry": [ { "ErrorEquals": [ "ErrorA", "ErrorB" ], "IntervalSeconds": 1, "BackoffRate": 2, "MaxAttempts": 2 } ] }
  • 21.
    Amazon State Language •Choice "ChoiceState": { "Type" : "Choice", "Choices": [ { "Variable": "$.foo", "NumericEquals": 1, "Next": "FirstMatchState" }, { "Variable": "$.foo", "NumericEquals": 2, "Next": "SecondMatchState” } ], "Default": "DefaultState" }, • StringEquals • StringLessThan • StringGreaterThan • StringLessThanEquals • StringGreaterThanEquals • NumericEquals • NumericLessThan • NumericGreaterThan • NumericLessThanEquals • NumericGreaterThanEquals • BooleanEquals • TimestampEquals • TimestampLessThan • TimestampGreaterThan • TimestampLessThanEquals • TimestampGreaterThanEquals
  • 22.
    Amazon State Language •Choice "ChoiceStateX": { "Type" : "Choice", "Choices": [ { "Not": { "Variable": "$.type", "StringEquals": "Private” }, "Next": "Public” }, { "And": [ { "Variable": "$.value", "NumericGreaterThanEquals": 20 }, { "Variable": "$.value", "NumericLessThan": 30 } ], "Next": "ValueInTwenties” } ], "Default": "DefaultState” }, • Not • Or • And
  • 23.
    Amazon State Language •Choice "ChoiceStateX": { "Type" : "Choice", "Choices": [ { "Not": { "Variable": "$.type", "StringEquals": "Private” }, "Next": "Public” }, { "And": [ { "Variable": "$.value", "NumericGreaterThanEquals": 20 }, { "Variable": "$.value", "NumericLessThan": 30 } ], "Next": "ValueInTwenties” } ], "Default": "DefaultState” }, • Not • Or • And
  • 24.
    Amazon State Language •Choice "ChoiceStateX": { "Type" : "Choice", "Choices": [ { "Not": { "Variable": "$.type", "StringEquals": "Private” }, "Next": "Public” }, { "And": [ { "Variable": "$.value", "NumericGreaterThanEquals": 20 }, { "Variable": "$.value", "NumericLessThan": 30 } ], "Next": "ValueInTwenties” } ], "Default": "DefaultState” }, • Not • Or • And
  • 25.
    Amazon State Language •Parallel – 全ての分岐が完了 してから次のState へ進む "Parallel": { "Type": "Parallel", "Next": "Final State", "Branches": [ { "StartAt": "Wait 20s", "States": { "Wait 20s": { "Type": "Wait", "Seconds": 20, "End": true } } }, { "StartAt": "Wait 10s", "States": { "Wait 10s": { "Type": "Wait", "Seconds": 10, "End": true } } } ] },
  • 26.
    Amazon State Language •Wait – Seconds – SecondsPath – Timestamp – TimestampPath "wait_ten_seconds" : { "Type" : "Wait", "Seconds" : 10, "Next": "NextState” } "wait_until" : { "Type": "Wait", "Timestamp": "2016-03-14T01:59:00Z", "Next": "NextState” } "wait_until" : { "Type": "Wait", "TimestampPath": "$.expirydate", "Next": "NextState” }
  • 27.
    Agenda • AWS StepFunctions とは • Amazon State Language • Step Functions の実⾏ – State Machine の実⾏⽅法 – Activity Task の実⾏
  • 28.
    State Machine の実⾏ •start-execution API Step Functions State Machine start-execution
  • 29.
    State Machine の実⾏ •start-execution API Lambda Step Functions State Machine start-execution
  • 30.
    State Machine の実⾏ •start-execution API API Gateway S3 SNS Cloud Watch Event Lambda Step Functions State Machine start-execution
  • 31.
    State Machine の実⾏ •実⾏結果 – State Machine ごとに管理
  • 32.
    State Machine の実⾏ •実⾏結果 – State Machine の実⾏毎に管理
  • 33.
    State Machine の実⾏ •実⾏結果 – State 毎に管理
  • 34.
    Agenda • AWS StepFunctions とは • Amazon State Language • Step Functions の実⾏ – State Machine の実⾏⽅法 – Activity Task の実⾏
  • 35.
    Activity Taskの実⾏ • Lambda以外のアプリケーションの実⾏ –EC2 –オンプレミス •Pull型の実⾏ – アプリケーションからポーリング – LambdaはPush型
  • 36.
  • 37.
  • 38.
    Activity Taskの実⾏ • StateMachineの作成 { "Comment": "An example using a Task state.", "StartAt": "getGreeting", "Version": "1.0", "TimeoutSeconds": 300, "States": { "getGreeting": { "Type": "Task", "Resource": "arn:aws:states:eu_central-1:123456789012:activity:get-greeting", "End": true } } }
  • 39.
    Activity Taskの実⾏ • アプリケーションからポーリング –Get-Activity-Task API Polling Get-Activity-Task --Activity ARN --Worker Name (Option) Worker State Machine
  • 40.
    Activity Taskの実⾏ • StateMachine実⾏ Input TaskToken In progress Get-Activity-Task --Activity ARN --Worker Name (Option) Worker State Machine
  • 41.
  • 42.
  • 43.
    まとめ • Step Functionsを使うことで分散アプリケーション による複雑な処理を簡単に定義・実⾏・管理 •マイクロサービス化、サーバレス化 • Lambdaだけでなくサーバ上のアプリケーションも 組み込み可能
  • 44.