© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Modern Applications Development
and Serverless Computing
Boaz Ziniman
Technical Evangelist - Amazon Web Services
@ziniman
boaz.ziniman.aws
ziniman
© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
“The Monolith”
© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Challenges with monolithic software
Long Build/Test/Release
Cycles
(who broke the build?)
Operations
is a nightmare
(module X is failing,
who’s the owner?)
Difficult to
scale
New releases
take months
Long time to add
new features
Architecture is
hard to maintain
and evolve
Lack of innovation
Frustrated customers
Lack of agility
© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Monolith development lifecycle
releasetestbuild
delivery pipeline
App (aka the “monolith”)developers
© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Too much software coupling
Shared libraries
Shared data
© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Development transformation at Amazon: 2001–2002
monolithic application
+ teams
2001
Lesson learned: decompose for agility
2002
microservices
+ 2 pizza teams
© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Full ownership
Full accountability
“DevOps”
Focused innovation
Two-pizza teams
© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Monolith development lifecycle
monitorreleasetestbuild
developers
delivery pipelines
services
© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Microservice development lifecycle
???
developers
delivery pipelines
services
© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Microservice development lifecycle
developers services
monitorreleasetestbuild
delivery pipelines
monitorreleasetestbuild
monitorreleasetestbuild
monitorreleasetestbuild
monitorreleasetestbuild
monitorreleasetestbuild
© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Approaches to modern application development
• Simplify environment management
• Reduce the impact of code changes
• Automate operations
• Accelerate the delivery of new, high-quality services
• Gain insight across resources and applications
• Protect customers and the business
© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Approaches to modern application development
• Simplify environment management with serverless technologies
• Reduce the impact of code changes with microservice architectures
• Automate operations by modeling applications & infrastructure as code
• Accelerate the delivery of new, high-quality services with CI/CD
• Gain insight across resources and applications by enabling observability
• Protect customers and the business with end-to-end security & compliance
© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
What changes do you need to make to adopt these best practices?
Serverless
No provisioning/management
Automatic scaling
Pay for value billing
Availability and resiliency
Microservices
Componentization
Business capabilities
Products not projects
Infrastructure automation
DevOps
Cultural philosophies
Cross-disciplinary teams
CI/CD
Automation tools
DEV OPS
Architectural
patterns
Operational
Model
Software
Delivery
© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Microservices on AWS
AWS Lambda
Serverless functions
AWS Fargate
Serverless containers
ECS/EKS
Container-management as a service
EC2
Infrastructure-as-a-Service
More opinionated
Less opinionated
AWS manages Customer manages
• Data source integrations
• Physical hardware, software, networking,
and facilities
• Provisioning
• Application code
• Container orchestration, provisioning
• Cluster scaling
• Physical hardware, host OS/kernel,
networking, and facilities
• Application code
• Data source integrations
• Security config and updates, network config,
management tasks
• Container orchestration control plane
• Physical hardware software,
networking, and facilities
• Application code
• Data source integrations
• Work clusters
• Security config and updates, network config,
firewall, management tasks
• Physical hardware software,
networking, and facilities
• Application code
• Data source integrations
• Scaling
• Security config and updates, network config,
management tasks
• Provisioning, managing scaling and
patching of servers
© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
What is serverless?
No infrastructure provisioning,
no management
Automatic scaling
Pay for value Highly available and secure
© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Deliver on demand, never pay for idle
EVENT DRIVEN CONTINUOUS SCALING PAY BY USAGE
© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Using AWS Lambda
Simple resource model
• Select power rating from
128 MB to 3 GB
• CPU and network
allocated proportionately
Flexible use
• Synchronous or
asynchronous
• Integrated with other
AWS services
Flexible authorization
• Securely grant access to
resources and VPCs
• Fine-grained control for
invoking your functions
Bring your own code
• Node.js, Java, Python, C#,
.Net Core 2.0, GO, Ruby
• NEW - Your own Runtime
• Bring your own libraries
• Layers
© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Using AWS Lambda
Authoring functions
• WYSIWYG editor or
upload packaged .zip
• Third-party plugins
(Eclipse, Visual Studio,
PyCharm)
Monitoring and logging
• Metrics for requests,
errors, and throttles
• Built-in logs to Amazon
CloudWatch Logs
Programming model
• Use processes, threads,
/tmp, sockets normally
• AWS SDK built in (Python
and Node.js)
Stateless
• Persist data using
external storage
• No affinity or access to
underlying infrastructure
© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Serverless architectures
1. File put into bucket
2. Lambda invoked 2. Lambda invoked
1. Data published to a topic
Data
1. Message inserted into to a
queue
3. Function
removes
message from
queue
2. Lambda polls queue
and invokes function
Object Message
Amazon S3
Lambda
Function
Amazon Simple Queue
Service
Amazon Simple
Notification Service
Lambda
Function
Lambda
Function
© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Web application
Data stored in Amazon
DynamoDB
Dynamic content in
AWS Lambda
Amazon API
Gateway
Browser
Amazon
CloudFront
Amazon S3
© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Streaming data ingestion
Amazon S3:
Buffered files
Kinesis
Agent
Record
Producers Amazon Redshift:
Table loads
Amazon Elasticsearch Service:
Domain loads
Amazon S3:
Source record backup
AWS Lambda:
Transformations &
enrichment
Amazon DynamoDB:
Lookup tables
Raw records
Lookup
Transformed records
Transformed recordsRaw records
Amazon Kinesis Firehose:
Delivery stream
© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Image recognition and processing
Web App
Amazon DynamoDB:
Image meta-data & tags
Amazon Cognito:
User authentication
Amazon S3:
Image uploads
Start state machine execution
1
Amazon Rekognition:
Object detection
Invoke Amazon Rekognition
Generate image thumbnail
3
3
Extract image meta-data
2
Store meta-data and tags
4
https://github.com/awslabs/lambda-refarch-imagerecognition
AWS Step Functions:
Workflow orchestration
© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Cron
events
DATA STORES ENDPOINTS
CONFIGURATION REPOSITORIES EVENT/MESSAGE SERVICES
Example event sources that trigger AWS Lambda
… and a few more with more on the way!
Amazon S3 Amazon API
Gateway
Amazon
DynamoDB
AWS Step
Functions
Amazon
SNS
Amazon
SES
Amazon
Kinesis
Amazon
Kinesis
AWS IoT
Core
Alexa for
Business
Amazon
CloudWatch
AWS
CodeCommit
AWS
CloudTrail
AWS
CloudFormation
© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
FINRA
© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Serverless is an operational model that spans many
different categories of services
AWS
Lambda
AWS
Fargate
Amazon
API Gateway
Amazon
SNS
Amazon
SQS
AWS
Step Functions
COMPUTE
DATA STORES
INTEGRATION
Amazon Aurora
Serverless
Amazon
S3
Amazon
DynamoDB
AWS
AppSync
© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
AWS SERVERLESS PORTFOLIO
APPLICATION PRIMITIVES – COMPUTE AND DATASTORES
AWS
Lambda
AWS
Fargate
Amazon
API Gateway
Amazon
SNS
Amazon
MQ
Amazon
SQS
AWS
Step Functions
APPLICATION INTEGRATION
DEVELOPER TOOLS
SECURITY AND ADMINISTRATION
Amazon Aurora
Serverless
Amazon
S3
Amazon
DynamoDB
AWS
IAM
Amazon
Cognito
Amazon
Inspector
Amazon
VPC
Amazon
GuardDuty
AWS
CloudFormation
AWS
Cloud9
AWS
CloudTrail
Amazon
CloudWatch
AWS
X-Ray
AWS
CodePipeline
AWS
Config
AWS
SSO
AWS
Shield
AWS
WAF
Amazon
Kinesis
AWS Serverless
Application
Repository
AWS
AppSync
Thank you!
© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Boaz Ziniman
Technical Evangelist, Amazon Web Services
@ziniman
boaz.ziniman.aws
ziniman

Modern Applications Development on AWS

  • 1.
    © 2019, AmazonWeb Services, Inc. or its affiliates. All rights reserved. Modern Applications Development and Serverless Computing Boaz Ziniman Technical Evangelist - Amazon Web Services @ziniman boaz.ziniman.aws ziniman
  • 2.
    © 2019, AmazonWeb Services, Inc. or its affiliates. All rights reserved.
  • 3.
    © 2019, AmazonWeb Services, Inc. or its affiliates. All rights reserved. “The Monolith”
  • 4.
    © 2019, AmazonWeb Services, Inc. or its affiliates. All rights reserved. Challenges with monolithic software Long Build/Test/Release Cycles (who broke the build?) Operations is a nightmare (module X is failing, who’s the owner?) Difficult to scale New releases take months Long time to add new features Architecture is hard to maintain and evolve Lack of innovation Frustrated customers Lack of agility
  • 5.
    © 2019, AmazonWeb Services, Inc. or its affiliates. All rights reserved. Monolith development lifecycle releasetestbuild delivery pipeline App (aka the “monolith”)developers
  • 6.
    © 2019, AmazonWeb Services, Inc. or its affiliates. All rights reserved. Too much software coupling Shared libraries Shared data
  • 7.
    © 2019, AmazonWeb Services, Inc. or its affiliates. All rights reserved. Development transformation at Amazon: 2001–2002 monolithic application + teams 2001 Lesson learned: decompose for agility 2002 microservices + 2 pizza teams
  • 8.
    © 2019, AmazonWeb Services, Inc. or its affiliates. All rights reserved. Full ownership Full accountability “DevOps” Focused innovation Two-pizza teams
  • 9.
    © 2019, AmazonWeb Services, Inc. or its affiliates. All rights reserved. Monolith development lifecycle monitorreleasetestbuild developers delivery pipelines services
  • 10.
    © 2019, AmazonWeb Services, Inc. or its affiliates. All rights reserved. Microservice development lifecycle ??? developers delivery pipelines services
  • 11.
    © 2019, AmazonWeb Services, Inc. or its affiliates. All rights reserved. Microservice development lifecycle developers services monitorreleasetestbuild delivery pipelines monitorreleasetestbuild monitorreleasetestbuild monitorreleasetestbuild monitorreleasetestbuild monitorreleasetestbuild
  • 12.
    © 2019, AmazonWeb Services, Inc. or its affiliates. All rights reserved. Approaches to modern application development • Simplify environment management • Reduce the impact of code changes • Automate operations • Accelerate the delivery of new, high-quality services • Gain insight across resources and applications • Protect customers and the business
  • 13.
    © 2019, AmazonWeb Services, Inc. or its affiliates. All rights reserved. Approaches to modern application development • Simplify environment management with serverless technologies • Reduce the impact of code changes with microservice architectures • Automate operations by modeling applications & infrastructure as code • Accelerate the delivery of new, high-quality services with CI/CD • Gain insight across resources and applications by enabling observability • Protect customers and the business with end-to-end security & compliance
  • 14.
    © 2019, AmazonWeb Services, Inc. or its affiliates. All rights reserved. What changes do you need to make to adopt these best practices? Serverless No provisioning/management Automatic scaling Pay for value billing Availability and resiliency Microservices Componentization Business capabilities Products not projects Infrastructure automation DevOps Cultural philosophies Cross-disciplinary teams CI/CD Automation tools DEV OPS Architectural patterns Operational Model Software Delivery
  • 15.
    © 2019, AmazonWeb Services, Inc. or its affiliates. All rights reserved. Microservices on AWS AWS Lambda Serverless functions AWS Fargate Serverless containers ECS/EKS Container-management as a service EC2 Infrastructure-as-a-Service More opinionated Less opinionated AWS manages Customer manages • Data source integrations • Physical hardware, software, networking, and facilities • Provisioning • Application code • Container orchestration, provisioning • Cluster scaling • Physical hardware, host OS/kernel, networking, and facilities • Application code • Data source integrations • Security config and updates, network config, management tasks • Container orchestration control plane • Physical hardware software, networking, and facilities • Application code • Data source integrations • Work clusters • Security config and updates, network config, firewall, management tasks • Physical hardware software, networking, and facilities • Application code • Data source integrations • Scaling • Security config and updates, network config, management tasks • Provisioning, managing scaling and patching of servers
  • 16.
    © 2019, AmazonWeb Services, Inc. or its affiliates. All rights reserved. What is serverless? No infrastructure provisioning, no management Automatic scaling Pay for value Highly available and secure
  • 17.
    © 2019, AmazonWeb Services, Inc. or its affiliates. All rights reserved.
  • 18.
    © 2019, AmazonWeb Services, Inc. or its affiliates. All rights reserved. Deliver on demand, never pay for idle EVENT DRIVEN CONTINUOUS SCALING PAY BY USAGE
  • 19.
    © 2019, AmazonWeb Services, Inc. or its affiliates. All rights reserved.
  • 20.
    © 2019, AmazonWeb Services, Inc. or its affiliates. All rights reserved.
  • 21.
    © 2019, AmazonWeb Services, Inc. or its affiliates. All rights reserved. Using AWS Lambda Simple resource model • Select power rating from 128 MB to 3 GB • CPU and network allocated proportionately Flexible use • Synchronous or asynchronous • Integrated with other AWS services Flexible authorization • Securely grant access to resources and VPCs • Fine-grained control for invoking your functions Bring your own code • Node.js, Java, Python, C#, .Net Core 2.0, GO, Ruby • NEW - Your own Runtime • Bring your own libraries • Layers
  • 22.
    © 2019, AmazonWeb Services, Inc. or its affiliates. All rights reserved. Using AWS Lambda Authoring functions • WYSIWYG editor or upload packaged .zip • Third-party plugins (Eclipse, Visual Studio, PyCharm) Monitoring and logging • Metrics for requests, errors, and throttles • Built-in logs to Amazon CloudWatch Logs Programming model • Use processes, threads, /tmp, sockets normally • AWS SDK built in (Python and Node.js) Stateless • Persist data using external storage • No affinity or access to underlying infrastructure
  • 23.
    © 2019, AmazonWeb Services, Inc. or its affiliates. All rights reserved. Serverless architectures 1. File put into bucket 2. Lambda invoked 2. Lambda invoked 1. Data published to a topic Data 1. Message inserted into to a queue 3. Function removes message from queue 2. Lambda polls queue and invokes function Object Message Amazon S3 Lambda Function Amazon Simple Queue Service Amazon Simple Notification Service Lambda Function Lambda Function
  • 24.
    © 2019, AmazonWeb Services, Inc. or its affiliates. All rights reserved. Web application Data stored in Amazon DynamoDB Dynamic content in AWS Lambda Amazon API Gateway Browser Amazon CloudFront Amazon S3
  • 25.
    © 2019, AmazonWeb Services, Inc. or its affiliates. All rights reserved. Streaming data ingestion Amazon S3: Buffered files Kinesis Agent Record Producers Amazon Redshift: Table loads Amazon Elasticsearch Service: Domain loads Amazon S3: Source record backup AWS Lambda: Transformations & enrichment Amazon DynamoDB: Lookup tables Raw records Lookup Transformed records Transformed recordsRaw records Amazon Kinesis Firehose: Delivery stream
  • 26.
    © 2019, AmazonWeb Services, Inc. or its affiliates. All rights reserved. Image recognition and processing Web App Amazon DynamoDB: Image meta-data & tags Amazon Cognito: User authentication Amazon S3: Image uploads Start state machine execution 1 Amazon Rekognition: Object detection Invoke Amazon Rekognition Generate image thumbnail 3 3 Extract image meta-data 2 Store meta-data and tags 4 https://github.com/awslabs/lambda-refarch-imagerecognition AWS Step Functions: Workflow orchestration
  • 27.
    © 2019, AmazonWeb Services, Inc. or its affiliates. All rights reserved. Cron events DATA STORES ENDPOINTS CONFIGURATION REPOSITORIES EVENT/MESSAGE SERVICES Example event sources that trigger AWS Lambda … and a few more with more on the way! Amazon S3 Amazon API Gateway Amazon DynamoDB AWS Step Functions Amazon SNS Amazon SES Amazon Kinesis Amazon Kinesis AWS IoT Core Alexa for Business Amazon CloudWatch AWS CodeCommit AWS CloudTrail AWS CloudFormation
  • 28.
    © 2019, AmazonWeb Services, Inc. or its affiliates. All rights reserved. FINRA
  • 29.
    © 2019, AmazonWeb Services, Inc. or its affiliates. All rights reserved.
  • 30.
    © 2019, AmazonWeb Services, Inc. or its affiliates. All rights reserved. Serverless is an operational model that spans many different categories of services AWS Lambda AWS Fargate Amazon API Gateway Amazon SNS Amazon SQS AWS Step Functions COMPUTE DATA STORES INTEGRATION Amazon Aurora Serverless Amazon S3 Amazon DynamoDB AWS AppSync
  • 31.
    © 2019, AmazonWeb Services, Inc. or its affiliates. All rights reserved. AWS SERVERLESS PORTFOLIO APPLICATION PRIMITIVES – COMPUTE AND DATASTORES AWS Lambda AWS Fargate Amazon API Gateway Amazon SNS Amazon MQ Amazon SQS AWS Step Functions APPLICATION INTEGRATION DEVELOPER TOOLS SECURITY AND ADMINISTRATION Amazon Aurora Serverless Amazon S3 Amazon DynamoDB AWS IAM Amazon Cognito Amazon Inspector Amazon VPC Amazon GuardDuty AWS CloudFormation AWS Cloud9 AWS CloudTrail Amazon CloudWatch AWS X-Ray AWS CodePipeline AWS Config AWS SSO AWS Shield AWS WAF Amazon Kinesis AWS Serverless Application Repository AWS AppSync
  • 32.
    Thank you! © 2019,Amazon Web Services, Inc. or its affiliates. All rights reserved. Boaz Ziniman Technical Evangelist, Amazon Web Services @ziniman boaz.ziniman.aws ziniman