Download as PDF, PPTX
![src/main/scala/com/example/Hello.scala
class HelloHandler extends RequestStreamHandler {
@throws(classOf[java.io.IOException])
override
def handleRequest(input: InputStream, output: OutputStream, context: Context): Unit = {
val bytes = toByteArray(input)
output.write(bytes)
}
def toByteArray(input: InputStream) =
Stream.continually(input.read).takeWhile(_ != -1).map(_.toByte).toArray
}](https://image.slidesharecdn.com/scalamatsuri2017-170225083755/85/Scala-Matsuri-2017-22-320.jpg)
![assembly
$ sbt
> assembly
[info] Packaging .../target/scala-2.12/aws-lambda-scala-assembly-0.1.0-SNAPSHOT.jar](https://image.slidesharecdn.com/scalamatsuri2017-170225083755/85/Scala-Matsuri-2017-23-320.jpg)
![Giter8
• https://github.com/yoshiyoshifujii/sbt-aws-serverless-ddd.g8
$ sbt new yoshiyoshifujii/sbt-aws-serverless-ddd.g8
name [My Something Project]:
version [0.1.0-SNAPSHOT]:
organization [com.example]:
package [com.example]:
Giter8でサンプルプロジェクトを公開しました。](https://image.slidesharecdn.com/scalamatsuri2017-170225083755/85/Scala-Matsuri-2017-36-320.jpg)
![Account
case class AccountId(value: String) extends ValueObject[String]
case class Account(
id: AccountId,
version: Option[Version],
email: String,
password: String,
name: String) extends Entity[AccountId]](https://image.slidesharecdn.com/scalamatsuri2017-170225083755/85/Scala-Matsuri-2017-45-320.jpg)
![AccountRepository
trait AccountRepository {
def save(account: Account): Either[DomainError, Account]
def get(id: AccountId): Either[DomainError, Option[Account]]
def findBy(email: String): Either[DomainError, Option[Account]]
def findAll: Either[DomainError, Seq[Account]]
}](https://image.slidesharecdn.com/scalamatsuri2017-170225083755/85/Scala-Matsuri-2017-46-320.jpg)
![AccountEventPublisher
case class AccountModified(
accountId: AccountId,
email: String,
password: String,
name: String)
trait AccountEventPublisher {
def publish(modified: AccountModified): Either[DomainError, AccountModified]
}](https://image.slidesharecdn.com/scalamatsuri2017-170225083755/85/Scala-Matsuri-2017-47-320.jpg)
![AccountRepositoryOnDynamoDB
trait AccountRepositoryOnDynamoDB {
def save(account: Account): Either[DomainError, Account] = ???
def get(id: AccountId): Either[DomainError, Option[Account]] = ???
def findBy(email: String): Either[DomainError, Option[Account]] = ???
def findAll: Either[DomainError, Seq[Account]] = ???
}](https://image.slidesharecdn.com/scalamatsuri2017-170225083755/85/Scala-Matsuri-2017-49-320.jpg)
![AccountEventPublisherOnKinesis
trait AccountEventPublisherOnKinesis {
def publish(modified: AccountModified): Either[DomainError, AccountModified] = ???
}](https://image.slidesharecdn.com/scalamatsuri2017-170225083755/85/Scala-Matsuri-2017-50-320.jpg)
![Base
trait Base extends BaseStreamHandler {
val accountRepository: AccountRepository
val accountEventPublisher: AccountEventPublisher
override def handle(input: Input): Try[String] = Try {
JsObject(
"message" -> JsString("hello world!!")
).compactPrint
}
}](https://image.slidesharecdn.com/scalamatsuri2017-170225083755/85/Scala-Matsuri-2017-52-320.jpg)
Uploaded byYoshitaka Fujii
Scala Matsuri 2017
The document discusses building a serverless architecture in Scala using AWS Lambda. It covers serverless architecture concepts, Function as a Service (FaaS) using AWS Lambda, API Gateway, and Kinesis Streams. It then discusses developing and deploying Scala code to AWS Lambda using SBT plugins and templates, and implementing domain-driven design principles with Hexagonal architecture in a multi-module project structure.
More Related Content
Scala Matsuri 2017
- 1. Let's Build a ServerlessArchitecture in Scala! Yoshitaka Fujii (@yoshiyoshifujii) 2017 ScalaMatsuri Saturday, February 25th 16:30 - 17:10 Conference Room 1
- 2. Who am I? • Yoshitaka Fujii (@yoshiyoshifujii) • Joined MOTEX in April, 2014 • Software Engineer • Scala(19 months) • Scala Kansai summit 2016 staff. 自己紹介。エムオーテックス株式会社。Scala歴19ヶ月。Scala関西サミット2016でスタッフさせていただきました。
- 4. Agenda • Serverless Architecture •Development & Deployment • DDD
- 5. Serverless Architecture • usingFaaS • a significant reduction in costs • scalability • the lack of operating and maintaining infrastructure サーバレスアーキテクチャとは、FaaSを活用したシステム。コストの大幅な削減。スケーラビリティ。 インフラの運用管理が不要。
- 6. Function as aService • AWS Lambda • Google Cloud Function • Microsoft Azure Functions
- 7. AWS Lambda • AWSLambda is a compute service that lets you run code without provisioning or managing servers. • executes your code only when needed. • scales automatically, a few requests per day to thousands per second. • Node.js, Java, C# and Python AWS Lambda はサーバーをプロビジョニングしたり管理しなくてもコードを実行できるコンピューティングサービスです。必要な ときにコードを実行し、自動的にスケールします。
- 8. AWS Lambda Events •API Gateway (HTTP Requests) • Kinesis Streams • DynamoDB • S3 • Schedule (CloudWatch) • SNS • IoT これらのAWSサービスのイベントに応じてAWS Lambdaを実行できます。今回は、主にAPI GatewayとKinesis Streamsについて 取り上げます。
- 9. Amazon API Gateway •Amazon API Gateway is a fully managed service that makes it easy for developers to create, publish, maintain, monitor, and secure APIs at any scale. • Low-Cost and Efficient. (only for calls made to your APIs and data transfer out) • Performance at Any Scale • Run Your APIs Without Servers フルマネージドサービスで、開発者はどのようなスケールであっても、簡単に API の作成、配布、保守、監視、保護が行えます。
- 10.
- 11. Amazon Kinesis Streams •Use Amazon Kinesis Streams to collect and process large streams of data records in real time. • rapid and continuous data intake and aggregation. • Accelerated log and data feed intake and processing. • Real-time metrics and reporting. • Real-time data analytics. • Complex stream processing. Amazon Kinesis Streams を使用して、データレコードの大量のストリームをリアルタイムで収集し、処理します。 高速かつ継続的 にデータの取り込みと集約を行うことができます。
- 12.
- 13. Amazon API Gatewayclient AWS Lambda Amazon S3 Amazon DynamoDB Amazon Kinesis AWS Lambda AWS Lambda Amazon Elasticsearch Service Context+ Token Principal + Policy Policy is cached Denied 403 Allowed Auth function Consumers function System diagrams
- 14. Amazon API Gateway client AWS Lambda Amazon S3 Amazon DynamoDB Amazon Kinesis AWS Lambda Amazon Elasticsearch Service Policyis cached Large scale system client client client client client AWS Lambda AWS Lambda AWS Lambda AWS Lambda AWS Lambda AWS Lambda AWS Lambda AWS Lambda AWS Lambda AWS Lambda AWS Lambda 大規模なシステムを想定しています。
- 15. Large scale system •More functional requirements. • AWS Lambda wants to be a simple function. • 1 Lambda per method request. GET:/hello => getHello POST:/hello => postHello • develop and deploy efficiently 大規模なシステムは、機能要件が多い。AWS LambdaはシンプルなFunctionを保ちたい。1メソッドリクエストにつきLambdaを1 つ。また効率良く開発とデプロイをしたい。
- 16.
- 17. How to makesimple AWS Lambda. 単純なAWS Lambdaの作り方を紹介します。
- 18. RequestStreamHandler.java public interface RequestStreamHandler{ /** * Handles a Lambda Function request * @param input The Lambda Function input stream * @param output The Lambda function output stream * @param context The Lambda execution environment context object. * @throws IOException */ public void handleRequest(InputStream input, OutputStream output, Context context) throws IOException; } AWS Lambdaの公開インタフェースです。この中身を実装すると任意の処理が実行できます。
- 19. RequestHandler.java public interface RequestHandler<I,O> { /** * Handles a Lambda Function request * @param input The Lambda Function input * @param context The Lambda execution environment context object. * @return The Lambda Function output */ public O handleRequest(I input, Context context); } I/Oを任意の型で実装できますが、Java Beanなクラスもしくは、プリミティブな型のみになっており、Javaの制約を受けたりするの で、あまりオススメしません。
- 20. build.sbt lazy val root= (project in file(".")). settings( name := "aws-lambda-scala", organization := "com.example", scalaVersion := "2.12.1", libraryDependencies ++= Seq( "com.amazonaws" % "aws-lambda-java-core" % "1.1.0" ) )
- 21.
- 22. src/main/scala/com/example/Hello.scala class HelloHandler extendsRequestStreamHandler { @throws(classOf[java.io.IOException]) override def handleRequest(input: InputStream, output: OutputStream, context: Context): Unit = { val bytes = toByteArray(input) output.write(bytes) } def toByteArray(input: InputStream) = Stream.continually(input.read).takeWhile(_ != -1).map(_.toByte).toArray }
- 23. assembly $ sbt > assembly [info]Packaging .../target/scala-2.12/aws-lambda-scala-assembly-0.1.0-SNAPSHOT.jar
- 24. AWS CLI -lambda create function $ aws lambda create-function --region us-east-1 --function-name aws-lambda-scala --zip-file fileb://aws-lambda-scala-assembly-0.1.0-SNAPSHOT.jar --role arn:aws:iam::${AWS Account ID}:role/lambda_basic_execution --handler com.example.HelloHandler::handleRequest --runtime java8 --timeout 15 --memory-size 512
- 25. AWS CLI -lambda create function { "LastModified": "2017-01-02T12:34:56.789+0000", "FunctionName": "aws-lambda-scala", "Runtime": "java8", "Version": "$LATEST", "Role": "arn:aws:iam::${AWS Account ID}:role/lambda_basic_execution", "CodeSha256": "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx", "Handler": "com.example.HelloHandler::handleRequest", "Timeout": 15, "Description": "", "MemorySize": 512, "FunctionArn": "arn:aws:lambda:us-east-1:${AWS Account ID}:function:aws-lambda-scala", "CodeSize": 5264477 }
- 26. AWS CLI -lambda invoke $ aws lambda invoke --region us-east-1 --function-name aws-lambda-scala --payload 123 --invocation-type RequestResponse /tmp/response { "StatusCode": 200 } 123
- 27. Amazon API Gatewayclient AWS Lambda Amazon S3 Amazon DynamoDB Amazon Kinesis AWS Lambda AWS Lambda Amazon Elasticsearch Service Context+ Token Principal + Policy Policy is cached Denied 403 Allowed Auth function Consumers function System diagrams
- 28. Labor … • Itis hard to run Lambda. • I will `assembly` one by one and upload it. • It is troublesome to configure the API Gateway on the console. Lambdaで実行するまでが大変。assemblyしてアップロードしてを繰り返さないとうまく動くか分からない。コンソール上でAPI Gatewayの設定を行うのは面倒。
- 29. sbt-aws-serverless plugin • sbtplugin to deploy code to Amazon API Gateway and AWS Lambda. • https://github.com/yoshiyoshifujii/sbt-aws-serverless 自前で開発したsbt-pluginを使います。API GatewayやLambdaにデプロイするpluginです。
- 30. serverless framework • Tobe honest this way is better. • However, blue green deployment is difficult. 正直、serverless frameworkを使うのが良いと思います。ただ、ブルーグリーンデプロイができない印象です。
- 31.
- 32. Amazon API Gateway Stage StageVariables test env : test prod env : prod Deployment Resources Lambda ARN 1 /hello hello:${stageVariables.env}_1 AWS Lambda Alias Publish Versions dev $LATEST test_1 1 prod_1 1 Blue-Green deployment - 1st release Amazon DynamoDB Table account-test account-prod
- 33. Amazon API Gateway Stage StageVariables test env : test prod env : prod Deployment Resources Lambda ARN 1 /hello hello:${stageVariables.env}_1 2 /hello hello:${stageVariables.env}_2 AWS Lambda Alias Publish Versions dev $LATEST test_1 1 prod_1 1 test_2 2 Blue-Green deployment - 2nd release test Amazon DynamoDB Table account-test account-prod
- 34. Amazon API Gateway Stage StageVariables test env : test prod env : prod Deployment Resources Lambda ARN 1 /hello hello:${stageVariables.env}_1 2 /hello hello:${stageVariables.env}_2 AWS Lambda Alias Publish Versions dev $LATEST test_1 1 prod_1 1 test_2 2 prod_2 2 Amazon DynamoDB Table account-test account-production Blue-Green deployment - 2nd release
- 35. in this case •Release the test environment as it is. • Quickly return to the previous version. • A/B Testing. • Canary Release. https://martinfowler.com/bliki/CanaryRelease.html この方式なら、テストした環境をそのままリリースできる。すぐに前のバージョンに戻すことができる。A/Bテストや、カナリア・リ リースに使える。
- 36. Giter8 • https://github.com/yoshiyoshifujii/sbt-aws-serverless-ddd.g8 $ sbtnew yoshiyoshifujii/sbt-aws-serverless-ddd.g8 name [My Something Project]: version [0.1.0-SNAPSHOT]: organization [com.example]: package [com.example]: Giter8でサンプルプロジェクトを公開しました。
- 37.
- 38. DDD - Hexagonalarchitecture • Implementing Domain-Driven Design - 2013/2/16 • Vaughn Vernon • Mitigate vendor lock-in. DDDのヘキサゴナルアーキテクチャを採用する。ベンダーロックインを緩和できる。
- 39.
- 40.
- 41. Amazon API Gatewayclient AWS Lambda Amazon S3 Amazon DynamoDB Amazon Kinesis AWS Lambda AWS Lambda Amazon Elasticsearch Service Context+ Token Principal + Policy Policy is cached Denied 403 Allowed Auth function Consumers function System diagrams Make this part
- 42. build.sbt - dependsOn lazyval domain = (project in file("./modules/domain")). lazy val infraDynamo = (project in file("./modules/infrastructure/dynamodb")). dependsOn(domain). lazy val infraKinesis = (project in file("./modules/infrastructure/kinesis")). dependsOn(domain). lazy val appHello = (project in file("./modules/application/hello")). dependsOn(infraLambda, infraDynamo, infraKinesis). マルチプロジェクトの依存関係を依存モデルの通りに設定。
- 43. Project tree • Sourcesunder modules. • application is a module of Lambda. • Make the Infrastructure as finely as possible. • Reduce module size. プロジェクト構成。ソースはmodulesディレクトリの配下に置く。applicationがLambdaのモジュールになる。Infrastructureは限 りなく小さく作る。Lambdaのモジュールサイズを小さくするため。
- 44. domain • POSO (Plainold Scala object) • Do not specify anything for libraryDependencies. domain層。POSOで作る。libraryDependenciesに何も指定しない。
- 45. Account case class AccountId(value:String) extends ValueObject[String] case class Account( id: AccountId, version: Option[Version], email: String, password: String, name: String) extends Entity[AccountId]
- 46. AccountRepository trait AccountRepository { defsave(account: Account): Either[DomainError, Account] def get(id: AccountId): Either[DomainError, Option[Account]] def findBy(email: String): Either[DomainError, Option[Account]] def findAll: Either[DomainError, Seq[Account]] }
- 47. AccountEventPublisher case class AccountModified( accountId:AccountId, email: String, password: String, name: String) trait AccountEventPublisher { def publish(modified: AccountModified): Either[DomainError, AccountModified] }
- 48. infrastructure • implementation ofRepository or Domain Event. • DI with Cake pattern. • It has no direct inheritance relation. infrastructure層。RepositoryやDomainEventの実装を持つ。Cake patternでDIされる。直接の継承関係を持たない。
- 49. AccountRepositoryOnDynamoDB trait AccountRepositoryOnDynamoDB { defsave(account: Account): Either[DomainError, Account] = ??? def get(id: AccountId): Either[DomainError, Option[Account]] = ??? def findBy(email: String): Either[DomainError, Option[Account]] = ??? def findAll: Either[DomainError, Seq[Account]] = ??? }
- 50. AccountEventPublisherOnKinesis trait AccountEventPublisherOnKinesis { defpublish(modified: AccountModified): Either[DomainError, AccountModified] = ??? }
- 51. application • Generate Lambda'smodule. • Dependency injection. application層。Lambdaのモジュールを生成する。依存性が注入する。
- 52. Base trait Base extendsBaseStreamHandler { val accountRepository: AccountRepository val accountEventPublisher: AccountEventPublisher override def handle(input: Input): Try[String] = Try { JsObject( "message" -> JsString("hello world!!") ).compactPrint } }
- 53. App class App extendsBase { override val accountRepository = new AccountRepository with AccountRepositoryOnDynamoDB override val accountEventPublisher = new AccountEventPublisher with AccountEventPublisherOnKinesis }
- 54. Amazon API Gatewayclient AWS Lambda Amazon S3 Amazon DynamoDB Amazon Kinesis AWS Lambda AWS Lambda Amazon Elasticsearch Service Context+ Token Principal + Policy Policy is cached Denied 403 Allowed Auth function Consumers function System diagrams Make this part
- 55. build.sbt - dependsOn lazyval domain = (project in file("./modules/domain")). lazy val infraDomain = (project in file("./modules/infrastructure/domain")). dependsOn(domain). lazy val appAccountModified = (project in file(“./modules/application/accountmodified”)). dependsOn(infraLambdaConsumer, infraDomain). 依存関係は、API Gatewayとほぼ同じ。
- 56.
- 57. sbt deploy prod $sbt -DAWS_ACCOUNT_ID=<AWS Account ID> -DAWS_ROLE_ARN=arn:aws:iam::<AWS Account ID>:role/<Role NAME> -DAWS_BUCKET_NAME=<BUCKET NAME> > deploy v1 API Gateway created: xxxxxxxxxx API Gateway put: xxxxxxxxxx Lambda deployed: arn:aws:lambda:us-east-1:aaaaaaaaaaaa:function:$name-auth Lambda published: arn:aws:lambda:us-east-1:aaaaaaaaaaaa:function:$name-auth:1 Lambda Alias: arn:aws:lambda:us-east-1:aaaaaaaaaaaa:function:$name-auth:prod Authorizer: auauau Lambda deployed: arn:aws:lambda:us-east-1:aaaaaaaaaaaa:function:$name-app-hello Lambda published: arn:aws:lambda:us-east-1:aaaaaaaaaaaa:function:$name-app-hello:1 Lambda Alias: arn:aws:lambda:us-east-1:aaaaaaaaaaaa:function:$name-app-hello:prod_1 Create Deployment: {Id: dddddd,Description: 0.1.0-SNAPSHOT,CreatedDate: Sat Feb 25 12:34:56 JST 2017,}
- 58. Amazon API Gateway Stage StageVariables test env : test prod env : prod Deployment Resources Lambda ARN 1 /hello hello:${stageVariables.env}_1 AWS Lambda Alias Publish Versions dev $LATEST test_1 1 prod_1 1 Blue-Green deployment - 1st release Amazon DynamoDB Table account-test account-prod
- 59. sbt deploy test $sbt -DAWS_ACCOUNT_ID=<AWS Account ID> -DAWS_ROLE_ARN=arn:aws:iam::<AWS Account ID>:role/<Role NAME> -DAWS_BUCKET_NAME=<BUCKET NAME> > deploy test API Gateway created: xxxxxxxxxx API Gateway put: xxxxxxxxxx Lambda deployed: arn:aws:lambda:us-east-1:aaaaaaaaaaaa:function:$name-auth Lambda published: arn:aws:lambda:us-east-1:aaaaaaaaaaaa:function:$name-auth:2 Lambda Alias: arn:aws:lambda:us-east-1:aaaaaaaaaaaa:function:$name-auth:test Authorizer: auauau Lambda deployed: arn:aws:lambda:us-east-1:aaaaaaaaaaaa:function:$name-app-hello Lambda published: arn:aws:lambda:us-east-1:aaaaaaaaaaaa:function:$name-app-hello:2 Lambda Alias: arn:aws:lambda:us-east-1:aaaaaaaaaaaa:function:$name-app-hello:test_2 Create Deployment: {Id: dddddd,Description: 0.1.0-SNAPSHOT,CreatedDate: Sat Feb 25 12:34:56 JST 2017,}
- 60. Amazon API Gateway Stage StageVariables test env : test prod env : prod Deployment Resources Lambda ARN 1 /hello hello:${stageVariables.env}_1 2 /hello hello:${stageVariables.env}_2 AWS Lambda Alias Publish Versions dev $LATEST test_1 1 prod_1 1 test_2 2 Blue-Green deployment - 2nd release test Amazon DynamoDB Table account-test account-prod
- 61. sbt deployCopy testto prod $ sbt -DAWS_ACCOUNT_ID=<AWS Account ID> -DAWS_ROLE_ARN=arn:aws:iam::<AWS Account ID>:role/<Role NAME> -DAWS_BUCKET_NAME=<BUCKET NAME> > deployCopy test prod Lambda Alias: arn:aws:lambda:us-east-1:aaaaaaaaaaaa:function:$name-auth:prod Lambda Alias: arn:aws:lambda:us-east-1:aaaaaaaaaaaa:function:$name-app-hello:prod_2 Stage: prod
- 62. Amazon API Gateway Stage StageVariables test env : test prod env : prod Deployment Resources Lambda ARN 1 /hello hello:${stageVariables.env}_1 2 /hello hello:${stageVariables.env}_2 AWS Lambda Alias Publish Versions dev $LATEST test_1 1 prod_1 1 test_2 2 prod_2 2 Amazon DynamoDB Table account-test account-production Blue-Green deployment - 2nd release
- 63. sbt deployDev $ sbt -DAWS_ACCOUNT_ID=<AWS Account ID> -DAWS_ROLE_ARN=arn:aws:iam::<AWS Account ID>:role/<Role NAME> -DAWS_BUCKET_NAME=<BUCKET NAME> > deployDev dev API Gateway created: xxxxxxxxxx API Gateway put: xxxxxxxxxx Lambda deployed: arn:aws:lambda:us-east-1:aaaaaaaaaaaa:function:$name-auth Lambda Alias: arn:aws:lambda:us-east-1:aaaaaaaaaaaa:function:$name-auth:dev Authorizer: auauau Lambda deployed: arn:aws:lambda:us-east-1:aaaaaaaaaaaa:function:$name-app-hello Lambda Alias: arn:aws:lambda:us-east-1:aaaaaaaaaaaa:function:$name-app-hello:dev Create Deployment: {Id: ddddd,Description: 0.1.0-SNAPSHOT,CreatedDate: Sat Feb 25 12:34:56 JST 2017,}
- 64. sbt deployList $ sbt -DAWS_ACCOUNT_ID=<AWS Account ID> -DAWS_ROLE_ARN=arn:aws:iam::<AWS Account ID>:role/<Role NAME> -DAWS_BUCKET_NAME=<BUCKET NAME> > deployList dev ============================================================================================================ xxxxxxxxxx ============================================================================================================ | Stage Name | Last Updated Date | Deployment Id | Description | |----------------------|--------------------------------|-----------------|--------------------------------| | dev | Mon Feb 20 21:30:07 JST 2017 | qirn6v | null | | v1 | Mon Feb 20 21:23:41 JST 2017 | 8tb1zr | null | ===================================================================================== xxxxxxxxxx ===================================================================================== | Created Date | Deployment Id | Description | |--------------------------------|-----------------|--------------------------------| | Mon Feb 20 21:30:07 JST 2017 | ddddd2 | 0.1.0-SNAPSHOT | | Mon Feb 20 21:23:41 JST 2017 | ddddd1 | 0.1.0-SNAPSHOT |
- 65. sbt invoke $ sbt -DAWS_ACCOUNT_ID=<AWS Account ID> -DAWS_ROLE_ARN=arn:aws:iam::<AWS Account ID>:role/<Role NAME> -DAWS_BUCKET_NAME=<BUCKET NAME> > invoke prod ============================================================ GET:https://xxxxxxxxxx.execute-api.us-east-1.amazonaws.com/prod/hellos ============================================================ 200 {"message":"hello world!!"}
- 66. Summary • Scala iseasy to combine DDD and Serverless. • Easy to share processing with multiple projects. • Cheap, scalable and easy to operate. ScalaはDDDとServerlessを組み合わせるとやりやすい。マルチプロジェクトで処理を共通化しやすい。安くてスケーラブルで運用 が楽。