EC2 to VPC with Chefand some other CI stuff...
+
About
● Leading online
grocery store in
Singapore.
● Proprietary
Ecommerce platform
● `redmart10` for 10%
off. :)
EC2 to VPC
EC2 to VPC - Goals
● Automate our infrastructure.
● Security
● Maintain (or improve) workflow
● Migrate with zero downtime
Security ☐
VPC
● Public Subnets
● Private Subnets
● Gateway
● NATs
VPC Infra
Public Subnet
10.0.0.0/24
Private
10.0.1.0/24
Frontend,
ELBs
Backend,
Api, Services
etc.
Gateway
Internet. Mostl...
Why VPC - without
● Security
o Security groups
Node
Why VPC - With
● Security
o Security groups
o Network ACLs
o Route Tables
o VPN
● Easier to manage
o Intra VPC
o External
...
Security ☑
Maintain (or improve) Workflow ☐
Workflow 1 - Deployment
1. Code
2. Push to git
3. Relax
Developer
1. Transfer Artifacts to
S3
Push
Build
Fail
Pass
Emails, Slack
SSH
Chef Server
`sudo chef-client`
Download
2. Ge...
Workflow 2 - Bootstrapping
1. Clone boilerplate project repo
2. Clone boilerplate cookbook & role
3. knife create server -...
1. Transfer Artifacts to
S3
Chef ServerDevOps
`sudo chef-client`
Bootstrap
Download
Developer
1. Transfer Artifacts to
S3
Push
Build
Fail
Pass
Send Emails
SSH
Chef ServerDevOps
`sudo chef-client`
Bootstrap
...
Travis process
1. Tests and Builds your project
2. If success:
o Uploads project to S3
o Finds nodes relevant to your proj...
Chef Process
1. Bootstrapping a node `knife bootstrap`
a. creates a server with specified role & environment
2. Converging...
Everyone deploys the same way.
Security ☑
Maintain (or improve) Workflow ☑
Migrate with zero downtime ☐
Stateful services
● Mongo (superfun)
● Redis
● Rabbitmq (not really stateful)
Mongo with six hosts
Apps
private
Apps
VPC Internet
public
Redis Daisy chaining
● Crucial to ensure
user sessions not
lost
o slaveof IPADDRESS
o slave-read-only no
private
VPC Inter...
Rabbitmq
● Keep alive till queues
die out.
Apps
private
Apps
VPC Internet
public
Non stateful apps
● sudo chef-client
● Hooray for chef!
DDay
● Duplicate version of entire redmart.com in
VPC.
o chef.redmart.com
o chefapi.redmart.com
● Change DNS of everything...
Security ☑
Maintain (or improve) Workflow ☑
Migrate with zero downtime ☑
… Cache busted!
● Full chef-client takes - 1min
● 3 nodes deployed sequentially - 3 mins
● Deploying one node at a time no...
What’s worked for us
● Infra as code is awesome
● One command to deploy is awesome
● Chef search is awesome
● Chef for ent...
Thank you
We’re hiring! - redmart.recruiterbox.com
We now have bread! - redmart.com
Slides - slideshare.net/riteshangural
Upcoming SlideShare
Loading in …5
×

RedMart Migrating from EC2 to VPC with Chef

987 views

Published on

We at RedMart migrated our platform from EC2 to VPC on AWS. This is an outline of how we went about doing it.

Published in: Technology, Business
0 Comments
2 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
987
On SlideShare
0
From Embeds
0
Number of Embeds
22
Actions
Shares
0
Downloads
19
Comments
0
Likes
2
Embeds 0
No embeds

No notes for slide

RedMart Migrating from EC2 to VPC with Chef

  1. 1. EC2 to VPC with Chefand some other CI stuff... +
  2. 2. About ● Leading online grocery store in Singapore. ● Proprietary Ecommerce platform ● `redmart10` for 10% off. :)
  3. 3. EC2 to VPC
  4. 4. EC2 to VPC - Goals ● Automate our infrastructure. ● Security ● Maintain (or improve) workflow ● Migrate with zero downtime
  5. 5. Security ☐
  6. 6. VPC ● Public Subnets ● Private Subnets ● Gateway ● NATs
  7. 7. VPC Infra Public Subnet 10.0.0.0/24 Private 10.0.1.0/24 Frontend, ELBs Backend, Api, Services etc. Gateway Internet. Mostly cats. HOP Mongo (Ecom) Mongo (Fulfill ment) RedMart VPC (10.0.0.0/16) Scale/ERP Scale DB Private (10.0.2.0/24)
  8. 8. Why VPC - without ● Security o Security groups Node
  9. 9. Why VPC - With ● Security o Security groups o Network ACLs o Route Tables o VPN ● Easier to manage o Intra VPC o External Node Node Node Node Gateway or NAT
  10. 10. Security ☑ Maintain (or improve) Workflow ☐
  11. 11. Workflow 1 - Deployment 1. Code 2. Push to git 3. Relax
  12. 12. Developer 1. Transfer Artifacts to S3 Push Build Fail Pass Emails, Slack SSH Chef Server `sudo chef-client` Download 2. Get nodes, SSH
  13. 13. Workflow 2 - Bootstrapping 1. Clone boilerplate project repo 2. Clone boilerplate cookbook & role 3. knife create server -r “role[shiny_new_app]” 4. Code!
  14. 14. 1. Transfer Artifacts to S3 Chef ServerDevOps `sudo chef-client` Bootstrap Download
  15. 15. Developer 1. Transfer Artifacts to S3 Push Build Fail Pass Send Emails SSH Chef ServerDevOps `sudo chef-client` Bootstrap Download 2. Get nodes, SSH
  16. 16. Travis process 1. Tests and Builds your project 2. If success: o Uploads project to S3 o Finds nodes relevant to your project (chef search) o Triggers `chef-client` on relevant nodes
  17. 17. Chef Process 1. Bootstrapping a node `knife bootstrap` a. creates a server with specified role & environment 2. Converging a node `sudo chef-client` a. role based recipe eg. recipe[golden_admin] b. redmart deployment recipe eg. recipe[base_redmart::deploy] i. download artifacts based from s3 ii. extract iii. run start script eg. eg. `sh /ci/start.sh` iv. symlink release
  18. 18. Everyone deploys the same way. Security ☑ Maintain (or improve) Workflow ☑ Migrate with zero downtime ☐
  19. 19. Stateful services ● Mongo (superfun) ● Redis ● Rabbitmq (not really stateful)
  20. 20. Mongo with six hosts Apps private Apps VPC Internet public
  21. 21. Redis Daisy chaining ● Crucial to ensure user sessions not lost o slaveof IPADDRESS o slave-read-only no private VPC Internet public
  22. 22. Rabbitmq ● Keep alive till queues die out. Apps private Apps VPC Internet public
  23. 23. Non stateful apps ● sudo chef-client ● Hooray for chef!
  24. 24. DDay ● Duplicate version of entire redmart.com in VPC. o chef.redmart.com o chefapi.redmart.com ● Change DNS of everything in public subnet ● TTL!
  25. 25. Security ☑ Maintain (or improve) Workflow ☑ Migrate with zero downtime ☑
  26. 26. … Cache busted! ● Full chef-client takes - 1min ● 3 nodes deployed sequentially - 3 mins ● Deploying one node at a time not okay. ● Cache busting on Backbone.js apps Solution: parallel triggering of chef & specific deployment recipe - 5 secs
  27. 27. What’s worked for us ● Infra as code is awesome ● One command to deploy is awesome ● Chef search is awesome ● Chef for entire lifecycle (bootstrap + deployment) is awesome
  28. 28. Thank you We’re hiring! - redmart.recruiterbox.com We now have bread! - redmart.com Slides - slideshare.net/riteshangural

×