How to encourage App Team to adopt DevOps flow on AWS AWS 如何協助客戶建立 DevOps 流程 Level: 200, 中文演講 講師: Bob Yeh, Solution Architect, AWS 開發團隊在面對各式不同的工作流程，應該如何與原有的工作流程銜接？藉由 AWS 與客戶的合作案例中帶您看到如何在舊有的開發流程與雲上的服務工具緊密的結合。 聯繫銷售: https://aws.amazon.com/tw/contact-us/ 與銷售線上聊天: https://pages.awscloud.com/tw-hkt-sales-chat.html

1. S U M M I T
Taipei 2019

2. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
How we encourage App Team
adopt DevOps Flow on AWS
Bob Yeh,
Startups Solutions Architect
AWS Hongkong Taiwan
A W S S u m m i t T a i p e i | 2 0 1 9

3. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Overview
What is DevOps?
Real Customer Case:
How does it look like AFTER DevOps?
How does it look like BEFORE DevOps?
How to make it looks like DevOps?
Wrap up

4. S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.

5. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
What we see into two different domain
Developers Operations

6. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
What we usually do on software development..
Plan
Code
Test
ReleaseBuild
Package
Deploy
Configuration
ValidationFeedback
Operation

7. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Real Customer Case:
Customer migrated to AWS.
CTO expect their Web App team will
moving faster, but they still work in
slow pace…

8. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
5xMore Frequent
Deployments
1/12Shorter Lead Times
1/12Faster Recovery
How does it looks like AFTER DevOps?

9. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
How does it looks like AFTER DevOps?
Daily
Deploy
More Frequent
Deployments
40mShorter Lead Times
10mFaster Recovery

10. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
How does it looks like BEFORE DevOps?
Monthly
Deploy
More Frequent
Deployments
8hShorter Lead Times
2hFaster Recovery

11. How did they work before?

12. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Monthly Release
- Manually pull code from Github
- Manually configure environment after pull code
Before Release….
- Merge Code At Code Freeze Date
- Deploy To Test Env
- If QA Passed, deploy To Prod Env

13. How do they work now?

14. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Daily Deploy
Biweekly Release
- Daily deploy to Dev Env
- Weekly to QA Env
- Biweekly to Production Env

15. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Improved version control flow
- Branch rule for development and automation
- Leverage Git Merge
Auto Deployment in QA/Beta, Semi-Auto in Prod
- Create Temp Env For Daily Build
- Bundle full-round test in Daily Build
Artifact in Central Pool
Configuration With Tags

16. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
How did we start?

17. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
New tool? New flow? New effort?
Github
Jenkins

18. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Every developer own a branch, and
we need to re-configure every release
In general case, the integration tool will require a specific
code repository & branch to build up an automation flow.
!

19. BEFORE AFTER
master
Deb John Amy Sam
master
Release1
Release2
Release1
Release2
feature1
feature2
feature3
feature4

20. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
We keep branches for tracking
We rotate release owner in every
sprint, he/she will fix branch conflict

21. BEFORE AFTER
master
Dev John Amy Sam
master
Release
Release
Release
Release
R0401
R0402
feature1
feature2
feature3
feature4
Tag=R0401
Tag=R0402

22. BEFORE AFTER
master
Release
Release
Release
Release
R0401
R0402 Tag=R0401
HotFix
Release
R0403
? HotFix
Hotfix-for0401
Release
Tag=R0403
Merge on local branch
before you merge back.
Tag=R0402
Unit Test Passed
Local End2End Passed
Scripts provided by QA Teams…

23. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
We do manual configure in Env
everytime after deploy
Pull Code take 5~10+ minutes….

24. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
BEFORE AFTER
push
code
Release
Env
QA
Beta
Build & Pack
automation
Prodconfig
push
code
Release
hook
download
& deploy

25. S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
BEFORE AFTER
> git push r0401 origin
Password: ********
> Remote to xxxxxxx
> cd c:xxxxxxx
> git fetch all
> git checkout r0401
> git pull origin r0401
> copy beta.conf.xml conf.xml
> restart app
> git push origin
Password: ********
Remote powershell + awscli

26. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
QA Beta Prod
Deploy to Beta Deploy to Prod
Deploy
QA Test
Deploy Deploy
QA Test Pass Beta Test Pass
Beta Test
Daily Trigger Weekly Trigger Manual Trigger

27. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
S3://company/RD1/Web1/QA/Release/R0401.zip
S3://company/RD1/Web1/QA/Release/R0402.zip
...
S3://company/RD1/Web1/QA/Release/R0409.zip
S3://company/RD1/Web1/QA/Release/R0410.zip
S3://company/RD1/Web1/Beta/Release/R0402.zip
S3://company/RD1/Web1/Beta/Release/R0410.zip
S3://company/RD1/Web1/Prod/Release/R0410.zip

28. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Env
D:Web1
S3://company/$Dept/$Proj/$Env/Release/R0401.zip
S3://company/$Dept/$Proj/$Env/Release/latest.zip
static filename
Download to Env
Extract to target Directory
tag: Dept=RD1
tag: Proj=Web1
tag: Env=QA

29. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Download
from S3?
Download
from S3?

30. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
aws s3 cp <S3Uri> <LocalPath>
S3://company/$Dept/$Proj/$Env/Release/latest.zip D:/packages/$Proj/

31. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Plan
Code
Test
Release
Package
ValidationFeedback
Operation
So, what do they have till now?
Deploy
Env
artifact
Configuration
Env
config
Build

32. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
We built AMI after every release
(manually…..)
Looks good
on static Env

33. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Dev QA Prod
One AMI + Different Tag = Different Flow in Env(s)

34. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Plan
Release
Package
Deploy
Configuration
Validation
Operation
Feedback
artifact
config
Env
Test
Code
Build
?

35. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
We have Auto-Scaling-Group
on production, it will roll up
empty machine!!!
Looks good
on Dev/Beta

36. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Dev QA Prod
Trigger deploy script
at launch time
userdata Auto scaling group

37. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
We need Roll-Back plan, and
need to be fast!
(Roll-Back < 20min)
What’s plan B?

38. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Env
S3://company/$Dept/$Proj/$Env/Release/latest.zip
Download to Env
D:Web1
D:Web1-recover
Prepare rescue pack before deploy
D:Web1
D:Web1-recover
Recover from local drive
Extract zip to D:Web1
tag: Dept=RD1
tag: Proj=Web1
tag: Env=QA

39. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Plan
Release
Package
Deploy
Configuration
ValidationFeedback
artifact
config
Env
Test
Code
Build
Operation

40. S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.

41. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
What is DevOps in this case?
• Cultural philosophies
• Practices
• Tools

42. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
DevOps Culture
• Development & Operations coming together
• Shared responsibility
• Ownership
• Visibility and communication

43. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
DevOps Practices
• Find the road block
• Mitigate the risk from manual process
• Minimal the change to exist flow

44. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
DevOps Tools
• There is no best tool, find your best combination.
AWS
CodeCommit
AWS
CodeDeploy
AWS
CodeBuild
AWS
CodePipeline
Github
Jenkins PowerShell Scripts

45. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
DevOps Practice on AWS
Infrastructure
as code
Microservices Logging and
monitoring
Continuous
integration/
continuous
delivery
AWS provides on-demand infrastructure resources and tooling built to enable common
DevOps practices

46. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Building our own tools
for automating
software release
https://secure.flickr.com/photos/lindseygee/5894617854/

47. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
• # of deployments at Amazon
• Thousands of service teams across Amazon
• Building microservices
• Practicing continuous delivery
• Many environments (staging, beta, production)
50 million

48. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Additional Resources
AWS DevOps Workshop:
https://s3.amazonaws.com/aws-devops-workshop/site/index.html
Integrating Git with AWS CodePipeline:
https://aws.amazon.com/blogs/devops/integrating-git-with-aws-codepipeline/
AWS CodeStar: https://aws.amazon.com/codestar
AWS CodeBuild: https://aws.amazon.com/codebuild
AWS CodeCommit: https://aws.amazon.com/codecommit
AWS CodeDeploy: https://aws.amazon.com/codedeploy
AWS CodePipeline: https://aws.amazon.com/codepipeline
AWS Cloud9: https://aws.amazon.com/cloud9

49. S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.

50. Thank you!
S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
bobyeh@amazon.com
ryanlao@amazon.com