Presented at Tech in Asia PDC 2019 in Jakarta. Most developers today are adopting a microservices based application design. Microservices can provide higher system reliability, fine-grained scalability, and faster development cycles. At hyperscale (thousands to millions of requests per second), however, additional thought, careful design, and greater operational rigor are required. In this session, learn fundamental design principles and best practices for hyperscale applications.

1. Operating Microservices at Hyperscale
Donnie Prakoso, MSc
Senior Technical Evangelist, ASEAN
Amazon Web Services
@donnieprakoso
donnieprakoso

2. > hello, world
Donnie Prakoso, MSc
Senior Technical Evangelist, ASEAN
@donnieprakoso
donnieprakoso
• Speak in Go and Python
• I talk a lot about microservices and machine learning
• I AWS User Groups
• Self-proclaimed Barista and Café Racer enthusiasts

3. https://bit.ly/aws-cloud-day

4. ListenIterate
Experiment
Innovation
flywheel
Experiments power the engine of rapid innovation

5. Development transformation at Amazon: 2001–2002
Monolithic application
+ teams
2001
Lesson learned: Decompose for agility
2002
Microservices
+ two pizza teams

6. Changes to the team structure

7. Full ownership
Full accountability
DevOps
Focused innovation
Two-pizza teams

8. Changes to the architectural patterns

9. When the impact of change is small,
release velocity can increase
Monolith
Does everything
Microservices
Does one thing

10. When a small team can make a change,
release velocity can increase
Multiple teams
Loosely connected
One team
Focus and ownership

11. New services and features released per year
0
500
1000
1500
2000
2500
2011 2012 2013 2014 2015 2016 2017 2018

12. Amazon S3
Object storage built to store and retrieve any amount of data from anywhere

13. Deployment at scale
Thousands of
teams
Microservices CI/CD Serverless
> 60 million deployments a year*

14. Journey to Hyperscale Your Microservices
1. Understand important components in Microservices
architecture
2. Utilizing Event-Driven Architectures
3. Increase software release velocity — CI/CD
4. Changing operational model — Serverless

15. APIs are the front door of microservices

16. Microservice
API
API
Microservice
Microservice
Event
API
Microservice
Event
API
Microservice
Application
Mobile client
Client
Internet of
Things (IoT)
Persistence Persistence

17. APIs are the “hardened contract”
between teams

18. Manage APIs with Amazon API Gateway
Mobile apps
Websites
Services
Internet Amazon
CloudFront
Amazon CloudWatch
monitoring
API
Gateway
cache
Any other
AWS service
All publicly
accessible
endpoints
Lambda
functions
Endpoints
in your VPC
Regional API endpoints
AWS Lambda
functions
Endpoints on
Amazon Elastic
Compute Cloud
(Amazon EC2)
Your virtual private
cloud (VPC)
Amazon Web Services
(AWS)

19. AWS AppSync
Managed serverless
GraphQL service
Connect to data
sources in your account
Add data sync, real-time, and
offline capabilities for any data
source or API
GraphQL facade for any
AWS service
Conflict detection and
resolution in the cloud
Enterprise security features:
IAM, Amazon Cognito, OIDC,
API keys

20. Event-driven architectures

21. Decouple state from code using
messaging

22. Decouple state from code using messaging
Queues
Simple
Fully managed
Any volume
Pub/sub
Simple
Fully managed
Flexible
Amazon Simple
Queue Service
(Amazon SQS)
Amazon Simple
Notification
Service (Amazon
SNS)
Messaging
Synchronization
Rapid
Fully managed
Real time
Amazon
CloudWatch
Events

23. Microservice
Event
Event
Microservice
Microservice
Event
API
Microservice
Event
API
Microservice
Application
Mobile client
Client
IoT
Persistence Persistence
Queue
Topic

24. Amazon SNS
“CheckoutEventsTopic”
Event storage and backup
pipeline
Event search and analytics
pipeline
Event replay pipeline
Checkout event processing pipeline
filtered orders (amount >= $100)
Orders to replay
All
orders
AWS Cloud
Ecommerce
users
Internet
Lambda
“CheckoutFunction”
Amazon SQS
“CheckoutQueue”
Lambda
“CheckoutFunction”
Amazon
DynamoDB
“CheckoutTable”
Event-driven architecture

25. NASA Image and Video Library
Amazon SNS
Amazon SQS

26. Data-stream-based architectures

27. Stream processing
Ingest
Data streams
Data processing
Real time
Data store
Microservices
Performance at scale
Fast and flexible
Amazon Kinesis
data streams
DynamoDB
Data-stream capture

28. Track status of data
and execution
Remove
redundant code
Build workflows to orchestrate everything

29. AWS Step Functions
With serverless
polling
With new
service integration
Start
End
Lambda
functions
Start
End
No
Lambda
functions

30. Object storage
Durable and scalable
Comprehensive security
Query-in-place
AWS serverless storage and database options
Non-relational
Performance at scale
Fully managed
Enterprise-ready
Relational
Fully managed
High performance
Cost effective

31. Cloud-native architectures are small
pieces, loosely joined

32. Changes to the operational model

33. AWS operational responsibility models
On premises Cloud
Less More
Compute Virtual machine
Amazon EC2 AWS Elastic Beanstalk LambdaAWS Fargate
Databases MySQL MySQL on Amazon EC2
Amazon RDS MySQL Amazon RDS Aurora Aurora Serverless DynamoDB
Storage Storage
Amazon S3
MessagingEnterprise service
buses (ESBs) Amazon MQ Kinesis Amazon SQS/Amazon SNS
Analytics
Apache Hadoop Hadoop on
Amazon EC2
Amazon EMR Amazon Elasticsearch Service Amazon Athena

34. More granular control also means more moving
parts which will increase complexity use
serverless to help you

35. What is serverless?
No infrastructure management Automatic scaling
Pay for value Highly available and secure

36. Lambda Fargate
API Gateway
Amazon
SNS
Amazon
SQS
AWS
Step Functions
Compute
Data stores
Integration
Aurora ServerlessAmazon S3 DynamoDB
AWS
AppSync

37. Serverless compute engine
for containers
Long-running
Bring existing code
Fully managed orchestration
Fargate
Let’s focus on compute for now
Serverless event-driven
code execution
Short-lived
All language runtimes
Data-source integrations
Lambda

38. Making development easier with Lambda
Accessible for
all developers
Enable new
application patterns
Greater
productivity
Support for all runtimes
with Lambda Layers and Runtime API
ISO, PCI, HIPAA, SOC, GDPR,
and FedRamp compliances
15-minute functions
Amazon SQS for Lambda
Automatic load balancing for Lambda
Support for Kinesis Data Streams Enhanced
Fan-Out and HTTP/2
Toolkits for popular integrated
development environments (IDEs):
VSCode, IntelliJ, and PyCharm
Simplified deployment
with nested apps

39. Author and debug Lambda applications on AWS
using your favorite IDEs
Python, Node Python Java
Developer
preview
.NET, Node

40. Serverless containers with Fargate
Bring existing code Containers as first-class
primitive
Production ready
No changes required of existing
code, works with existing
workflows and microservices built
on Amazon ECS
Time- and event-based scheduling,
network integration, individually
metered and billed; native service
discovery
ISO-, PCI-, HIPAA-, SOC-
compliant. Launch tens or tens of
thousands of containers in
seconds in
nine global regions
(+ seven in 2018)

41. Changes to the delivery of software

42. Monolith development lifecycle
MonitorReleaseTestBuild
Developers
Delivery pipelines
Services

43. Microservice development lifecycle
Developers Services
MonitorReleaseTestBuild
Delivery pipelines
MonitorReleaseTestBuild
MonitorReleaseTestBuild
MonitorReleaseTestBuild
MonitorReleaseTestBuild
MonitorReleaseTestBuild

44. How Amazon
does DevOps
(microservices, two pizza teams)
(governance, templates)

45. We released the AWS developer tools for continuous
integration and continuous delivery (CI/CD)
AWS CodeBuild +
third party
AWS CodeCommit AWS CodeBuild AWS CodeDeploy AWS X-Ray
Source Build Test Deploy Monitor
AWS CodePipeline

46. CI/CD – Don’t Only Run It, Own it
Source
MyAppSource
CodeCommit
Build
MyAppBuild
Build
Deploy
safetyCheck
Approval
ProductionDeploy
CodeDeploy
Lambda Function
safetyCheck
New update!
SNS Topic

47. X-Ray is built for modern applications
Identify customer
impact
Support for
serverless
Map all services and
ephemeral resources
Follow end-to-end
interactions

48. Conclusion

49. Increase
customer
value
Build better
products
Innovate
more often
Release
features
faster
Focus on
business
logic
Decouple
software
systems
Our goal is to automate and abstract away as much as is possible so customers can
focus on building applications for their business
We are witnessing a paradigm shift

50. Thank you!
Donnie Prakoso
Senior Technical Evangelist, ASEAN
Amazon Web Services
@donnieprakoso
donnieprakoso