Downloaded 70 times
Uploaded byRodrigo Campos
Velocity Conference NYC 2014 - Real World DevOps
Rodrigo Campos presented on the challenges faced and benefits of adopting a DevOps culture at Walmart's Latin American e-commerce division. The key challenges included deploying a new platform before Black Friday, expanding infrastructure for traffic spikes, developing an agile mindset, and rebuilding trust between teams. Adopting DevOps practices like improved communication, transparency and collaboration helped address these challenges. It resulted in the successful deployment of the new platform with 100% uptime on Black Friday and increased deployment frequency, success rates and adoption of agile methodologies.
More Related Content
Similar to Velocity Conference NYC 2014 - Real World DevOps
Velocity Conference NYC 2014 - Real World DevOps
- 1. Real World DevOps Rodrigo Campos (@xinu) Velocity NY 2014
- 2. Hi I’m Rodrigo! • Proudly crimping cables since 1992 • System Administrator • Operations Director at walmart.com in Latin America
- 3. I’m not goingto talk about • Silver Bullets • Unicorns • Tools and Utilities
- 4. Instead I’ll focuson • The challenges we faced • How DevOps has helped us • Bumps and bruises along the way
- 5.
- 6. Our challenges •Deploy a whole new platform before Black Friday
- 7. Our challenges •Deploy a whole new platform before Black Friday • Expand our private infrastructure to withstand web traffic spikes
- 8. Our challenges •Deploy a whole new platform before Black Friday • Expand our private infrastructure to withstand web traffic spikes • Develop a company-wide agile mindset
- 9. Our challenges •Deploy a whole new platform before Black Friday • Expand our private infrastructure to withstand web traffic spikes • Develop a company-wide agile mindset • Rebuild trust between tech and business
- 10. Different companies, same problems
- 11. Different companies, same problems • Silos not only between Developers and Operations, but within departments as well
- 12. Different companies, same problems • Silos not only between Developers and Operations, but within departments as well • BOFH attitude started to kick-in
- 13. Different companies, same problems • Silos not only between Developers and Operations, but within departments as well • BOFH attitude started to kick-in • Over the wall production deployments
- 14. Different companies, same problems • Silos not only between Developers and Operations, but within departments as well • BOFH attitude started to kick-in • Over the wall production deployments • Tug of war between Dev and Ops
- 15. Different companies, same problems • Silos not only between Developers and Operations, but within departments as well • BOFH attitude started to kick-in • Over the wall production deployments • Tug of war between Dev and Ops • Lots of bureaucracy and red-tape
- 16. Some phrases (roughly translated)
- 17. Some phrases (roughly translated) • “I'll go to the 9th floor to help you kick everyone’s a**”
- 18. Some phrases (roughly translated) • “I'll go to the 9th floor to help you kick everyone’s a**” • “I don’t care if it’s stable as long as it passed QA testing”
- 19. Some phrases (roughly translated) • “I'll go to the 9th floor to help you kick everyone’s a**” • “I don’t care if it’s stable as long as it passed QA testing” • “This is operations’ job”
- 20. Some phrases (roughly translated) • “I'll go to the 9th floor to help you kick everyone’s a**” • “I don’t care if it’s stable as long as it passed QA testing” • “This is operations’ job” • “Consider this a favor”
- 21. How DevOps wasseen
- 22. So what isDevOps anyway?
- 23. So what isDevOps anyway? • Being loosely defined can actually be a good thing!
- 24. So what isDevOps anyway? • Being loosely defined can actually be a good thing! • For us: “DevOps is a culture that fosters communication, transparency and collaboration"
- 25. So what isDevOps anyway?
- 26. So what isDevOps anyway? • Communication: I’ll stand up and go to your desk if I need to talk to you, you’re more than welcome to do the same!
- 27. So what isDevOps anyway? • Communication: I’ll stand up and go to your desk if I need to talk to you, you’re more than welcome to do the same! • Transparency: I want to know your problems, I’ll make sure you know mine!
- 28. So what isDevOps anyway? • Communication: I’ll stand up and go to your desk if I need to talk to you, you’re more than welcome to do the same! • Transparency: I want to know your problems, I’ll make sure you know mine! • Collaboration: We’re in the same boat and I really mean it!
- 29. We needed tochange people's mindset Operations Developers Us Them InfoSec Works on my Machine
- 30. We needed tochange people's mindset Source: http://youtu.be/Fx8OBeNmaWw
- 31. We needed tochange people's mindset
- 32. We needed tochange people's mindset • Presentations to technology, business and human capital teams
- 33. We needed tochange people's mindset • Presentations to technology, business and human capital teams • Reinforcing an inclusive and welcoming culture was essential
- 34. We needed tochange people's mindset • Presentations to technology, business and human capital teams • Reinforcing an inclusive and welcoming culture was essential • Everyone felt prepared and willing to change
- 35. Typical Scenario Business Developers Operations Infrastructure { } • Things go over the wall • No alignment between tech and business • One-way communications • IT as a necessary evil (aka cost center)
- 36. We were alreadya little bit different… Business Developers Operations Infrastructure { } DevOps Team
- 37. 1st step Business Developers Operations Infrastructure { } Production Engineering DevOps
- 38. 2nd step Business Infrastructure Production { } Developers Management Production Engineering + { } DevOps
- 39.
- 40. InfoSec • Firstand foremost: end the dossier culture
- 41. InfoSec • Firstand foremost: end the dossier culture • Communication + collaboration + transparency = safer systems
- 42. InfoSec • Firstand foremost: end the dossier culture • Communication + collaboration + transparency = safer systems • Integrate security tests into continuous integration
- 43. InfoSec • Firstand foremost: end the dossier culture • Communication + collaboration + transparency = safer systems • Integrate security tests into continuous integration • Rugged DevOps: http://slidesha.re/1r87Vs7
- 44. InfoSec • Nomore FUD • What is riskier to the business? • Lack of speed is a risk too!
- 45. Compliance • ITIL:“You keep using that word, I don’t think it means what you think it means” • People often blame ITIL as an excuse for badly designed processes • Surprisingly ITIL compliance was not our biggest problem
- 46. Compliance • Auditsare still our biggest challenge • In our case, we have a sponsor (the CTO) • Someone (who happens to be me) will be held accountable if something goes wrong
- 47. The bad apples • We’ve adopted the “No asshole rule" • There will be obstructive people that won’t embrace change • If you can't get rid of them, simply let them fade into oblivion
- 48.
- 49. KPIs • DeploymentFrequency Source: Puppet Labs Blog (http://bit.ly/Z3gRpu)
- 50. KPIs • DeploymentFrequency • Speed of Deployment Source: Puppet Labs Blog (http://bit.ly/Z3gRpu)
- 51. KPIs • DeploymentFrequency • Speed of Deployment • Deployment Success Rate Source: Puppet Labs Blog (http://bit.ly/Z3gRpu)
- 52. KPIs • DeploymentFrequency • Speed of Deployment • Deployment Success Rate • Service Recovery Source: Puppet Labs Blog (http://bit.ly/Z3gRpu)
- 53. KPIs • DeploymentFrequency • Speed of Deployment • Deployment Success Rate • Service Recovery • Culture Source: Puppet Labs Blog (http://bit.ly/Z3gRpu)
- 54.
- 55. Bumps & Bruises • DO:
- 56. Bumps & Bruises • DO: • Understand beforehand what you want to achieve
- 57. Bumps & Bruises • DO: • Understand beforehand what you want to achieve • Use measurable KPIs
- 58. Bumps & Bruises • DO: • Understand beforehand what you want to achieve • Use measurable KPIs • Explain to business areas what you’re doing
- 59. Bumps & Bruises • DO: • Understand beforehand what you want to achieve • Use measurable KPIs • Explain to business areas what you’re doing • Align with InfoSec and Compliance ASAP
- 60. Bumps & Bruises • DO: • Understand beforehand what you want to achieve • Use measurable KPIs • Explain to business areas what you’re doing • Align with InfoSec and Compliance ASAP • Work hard on a sponsor
- 61.
- 62. Bumps & Bruises • DON'T:
- 63. Bumps & Bruises • DON'T: • Create a “Ninja Superstar team”
- 64. Bumps & Bruises • DON'T: • Create a “Ninja Superstar team” • Postpone communication
- 65. Bumps & Bruises • DON'T: • Create a “Ninja Superstar team” • Postpone communication • Delay role changes
- 66. Bumps & Bruises • DON'T: • Create a “Ninja Superstar team” • Postpone communication • Delay role changes • Underestimate the managers' ability to influence behavior
- 67. Keep in mind • In the beginning it will be a lonely fight • Some people will label you as the “hippie-anarchist”
- 68.
- 69. Results • NeweCommerce platform successfully deployed
- 70. Results • NeweCommerce platform successfully deployed • Only major eCommerce website in Brazil to have 100% uptime during Black Friday
- 71. Results • NeweCommerce platform successfully deployed • Only major eCommerce website in Brazil to have 100% uptime during Black Friday • Increase in deployment frequency and success rate
- 72. Results • NeweCommerce platform successfully deployed • Only major eCommerce website in Brazil to have 100% uptime during Black Friday • Increase in deployment frequency and success rate • Business is incorporating agile methodologies
- 73. Any questions? •Special thanks to: https://www.flickr.com/photos/51321237@N07/ https://www.flickr.com/photos/kh-67/ https://www.flickr.com/photos/jrparis/ https://www.flickr.com/photos/andrewdubock/ https://www.flickr.com/photos/rachelyoung/ ! https://bitly.com/bundles/o_4p62vc3lid/5