Edmunds.com on AWS
John Martin
November 14, 2013

© 2013 Amazon.com, Inc. and its affiliates. All rights reserved. May not...
Moving isn’t easy
Moving Isn’t Easy
• A move isn’t easy
• Taking something your familiar with elsewhere
• How does that work again?
Today’s Agenda
• Technology Overview
• The Business Case
• The Approach
• Challenges
• What’s Next?
John Martin
@tekbuddha

WHO
AM
I?
John Martin
@tekbuddha
• 15+ years in .com
• 10+ years of Java
• Old School Ops
• New School Cultures

WHO
AM
I?
The Company
The Company
• Founded in 1966
• First online in 1994 as a gopher
• First website in 1996
The Environment
The Environment
• 30+ web applications across 300+ hosts.
• Java on Redhat Linux
• Tomcat, Solr, Coherence, Mongo, ActiveM...
The Environment
The Environment
• Chef + Cloudstack/UCS
• Perforce, Jenkins, Nexus, Selenium, JMeter
• AppDynamics, Splunk, RTview, Zenoss
The Deployment Pipeline
The Deployment Pipeline
• OSS + Homegrown Tooling
• All artifacts flow through the pipeline
• Release Cycles: 1 Month > 3 ...
The
Business
Case
The Business Case
• Simple: disaster recovery
• Bad business not to evaluate cloud services
The Approach
The Approach
• How to move 30+ apps and 300+ servers?
• Make it run like it already does
• Avoid biting off too much at on...
Objectives
• Minimize change / leverage existing toolchain
• Manage cost
• Provide initial design patterns for future buil...
Challenges
Source: http://is.gd/YmewdR
Challenges
• A move isn’t easy.
• Making the things we were familiar with work
elsewhere wasn’t easy

Source: http://is.gd...
CloudFormation

Source: http://is.gd/rayzL1
CloudFormation
• Necessary from Day 1
• Familiar concepts
• Infrastructure Is Code

Source: http://is.gd/rayzL1
CloudStack
CloudStack JSON
• Different structure, similar logic
• Defining single hosts, not groups of services
• No definitions of n...
CloudFormation
CloudFormation JSON
• Define network and services, not hosts
• Live by cf-validate
• Automate creation / avoid manual edit...
NFS
Source: http://is.gd/rayzL1
NFS
• Still a core dependency
• No great options in EC2 without refactor
• Move static content to S3

Source: http://is.gd...
Load
Balancing
Source: http://is.gd/rayzL1
Load Balancing
• No physical load balancers
• ELBs, HAproxy, and Chef were the key
• Learn how Public/Private ELBs work wi...
[auto-]scaling

Source: http://is.gd/rayzL1
[auto-]scaling
• Worked but can leave behind a mess
• Tooling must be prepared for ephermal nodes
• Helped survive outages...
Testing…
1!
2!
3!
WHAT’S NEXT?
Source: http://is.gd/xKdI6E
What Next?
• Additional live-traffic tests
• Deployment of internal services
• Prepare for full move out of data center

S...
REFACTOR

Source: http://is.gd/D8bVaC
Refactor
• Greater adoption of SOA principles
• Full refactor of data pipelines (in and out)
• Refactor for fragility of c...
THE BIG MOVE
The Big Move
• Work thus far completed by a small team
• The Big Move will be all hands on deck
• A two year goal to be fu...
Please give us your feedback on this
presentation

DMG205
As a thank you, we will select prize
winners daily for completed...
Upcoming SlideShare
Loading in...5
×

Migrating Edmunds.com to AWS (re:Invent 2013 DMG205)

425

Published on

Taking a stack composed of 30 web applications and their service dependencies to the cloud is no easy feat. Do you take the entirety of the stack or go the hybrid path? How transparent should the end result be to your technology teams? Does it look exactly the same in the cloud as it does in your data center? These are not rhetorical questions; they were very real for those tasked with the challenge of taking Edmunds.com to the AWS Cloud. This talk addresses these questions and many more, examining the challenges, successes, and lessons learned as the team took their first steps out of their own data centers.

Accompanying video is available on YouTube: http://youtu.be/itbNET2dc3c

Published in: Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
425
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
3
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Migrating Edmunds.com to AWS (re:Invent 2013 DMG205)

  1. 1. Edmunds.com on AWS John Martin November 14, 2013 © 2013 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified, or distributed in whole or in part without the express consent of Amazon.com, Inc.
  2. 2. Moving isn’t easy
  3. 3. Moving Isn’t Easy • A move isn’t easy • Taking something your familiar with elsewhere • How does that work again?
  4. 4. Today’s Agenda • Technology Overview • The Business Case • The Approach • Challenges • What’s Next?
  5. 5. John Martin @tekbuddha WHO AM I?
  6. 6. John Martin @tekbuddha • 15+ years in .com • 10+ years of Java • Old School Ops • New School Cultures WHO AM I?
  7. 7. The Company
  8. 8. The Company • Founded in 1966 • First online in 1994 as a gopher • First website in 1996
  9. 9. The Environment
  10. 10. The Environment • 30+ web applications across 300+ hosts. • Java on Redhat Linux • Tomcat, Solr, Coherence, Mongo, ActiveMQ
  11. 11. The Environment
  12. 12. The Environment • Chef + Cloudstack/UCS • Perforce, Jenkins, Nexus, Selenium, JMeter • AppDynamics, Splunk, RTview, Zenoss
  13. 13. The Deployment Pipeline
  14. 14. The Deployment Pipeline • OSS + Homegrown Tooling • All artifacts flow through the pipeline • Release Cycles: 1 Month > 3 Weeks > 1 Week
  15. 15. The Business Case
  16. 16. The Business Case • Simple: disaster recovery • Bad business not to evaluate cloud services
  17. 17. The Approach
  18. 18. The Approach • How to move 30+ apps and 300+ servers? • Make it run like it already does • Avoid biting off too much at once
  19. 19. Objectives • Minimize change / leverage existing toolchain • Manage cost • Provide initial design patterns for future builds
  20. 20. Challenges Source: http://is.gd/YmewdR
  21. 21. Challenges • A move isn’t easy. • Making the things we were familiar with work elsewhere wasn’t easy Source: http://is.gd/YmewdR
  22. 22. CloudFormation Source: http://is.gd/rayzL1
  23. 23. CloudFormation • Necessary from Day 1 • Familiar concepts • Infrastructure Is Code Source: http://is.gd/rayzL1
  24. 24. CloudStack
  25. 25. CloudStack JSON • Different structure, similar logic • Defining single hosts, not groups of services • No definitions of network resources
  26. 26. CloudFormation
  27. 27. CloudFormation JSON • Define network and services, not hosts • Live by cf-validate • Automate creation / avoid manual editing
  28. 28. NFS Source: http://is.gd/rayzL1
  29. 29. NFS • Still a core dependency • No great options in EC2 without refactor • Move static content to S3 Source: http://is.gd/rayzL1
  30. 30. Load Balancing Source: http://is.gd/rayzL1
  31. 31. Load Balancing • No physical load balancers • ELBs, HAproxy, and Chef were the key • Learn how Public/Private ELBs work with VPC Source: http://is.gd/rayzL1
  32. 32. [auto-]scaling Source: http://is.gd/rayzL1
  33. 33. [auto-]scaling • Worked but can leave behind a mess • Tooling must be prepared for ephermal nodes • Helped survive outages in US-EAST Source: http://is.gd/rayzL1
  34. 34. Testing… 1! 2! 3!
  35. 35. WHAT’S NEXT? Source: http://is.gd/xKdI6E
  36. 36. What Next? • Additional live-traffic tests • Deployment of internal services • Prepare for full move out of data center Source: http://is.gd/xKdI6E
  37. 37. REFACTOR Source: http://is.gd/D8bVaC
  38. 38. Refactor • Greater adoption of SOA principles • Full refactor of data pipelines (in and out) • Refactor for fragility of cloud environments Source: http://is.gd/D8bVaC
  39. 39. THE BIG MOVE
  40. 40. The Big Move • Work thus far completed by a small team • The Big Move will be all hands on deck • A two year goal to be fully cloud-based
  41. 41. Please give us your feedback on this presentation DMG205 As a thank you, we will select prize winners daily for completed surveys! Thank You
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×