DevTernity - DevOps with smell
•
3 likes•1,252 views
My last talk at DevTernity conference. It was about anti patterns captured consulting several projects on their DevOps adoption. There are few common pitfalls we can see repeating again and again over DevOps culture discovery. This talk is my experience summary there
Recommended
More Related Content
What's hot
What's hot (20)
Viewers also liked
Viewers also liked (10)
Similar to DevTernity - DevOps with smell
Similar to DevTernity - DevOps with smell (20)
Recently uploaded
Recently uploaded (20)
DevTernity - DevOps with smell
- 1. DEVOPS WITH SMELL ANTONS KRANGA
- 2. ~ WHOAMI ▸ Full stack developer ~ 15years ▸ Cloud Architect ▸ DevOps evangelist ▸ Innovation Center of Accenture Cloud Platform ▸ Speaker ▸ Marathon runner ANTONS KRANGA
- 4. DRIVERS FOR SOFTWARE DEVELOPMENT ▸ development price BEFORE AFTER ▸ innovations speed
- 5. DRIVERS FOR SOFTWARE DEVELOPMENT ▸ development price BEFORE AFTER ▸ innovations speed ▸ specialization silos ▸ ops comes first ▸ cross functional ▸ software defined data centers ▸ complex communication ▸ complexity theory
- 6. DEVOPS IS AN ENVIRONMENT WHERE PEOPLE TALK me :) DEFINITION OF DEVOPS
- 8. ANTIPATTERN # 0 UNICORNS VS HORSES You cannot be a unicorn when all others are horses
- 9. ANTIPATTERN # 0.1 DEVOPS IN A BOX You cannot buy culture! ▸ Buy a Golden DevOps LVL5 ▸ Become certified DevOps master ▸ Give me DevOps compliance checklist ▸ Give me 5 key DevOps control metrics
- 10. ANTIPATTERN # 0.2 REBRANDING DevOps != Configuration Management DevOps != Release Management| DevOps != Product Management … DevOps != (*) Management DEVOPS IS THE CULTURE YOU CANNOT FIND IN IN ITIL CATALOG
- 11. ANTIPATTERN # 1 DEVOPS AS A BLACK BOX What is the value in DevOps? ! Developer ! Sys-Op
- 12. ANTIPATTERN # 2 SORRY, NOT MY DEPARTMENT Bread ownership and specialization with autonomous teams vs http://martinfowler.com/bliki/DevOpsCulture.html
- 13. ANTIPATTERN # 2.1 DEVOPS AS A SILO DevOps teams build their own Silo ▸ You cannot talk to DevOps ▸ Use Jira instead !!!! ▸ RTFM Architecture
- 14. ANTIPATTERN # 3 DEFINITION OF DONE ▸ DoD fails with “ilities” ▸ Services can be easily “undone”
- 15. ANTIPATTERN # 4 FEAR OF RELEASE RELEASE === RISK NO RELEAE NO REVENUE vs
- 16. ANTIPATTERN # 5 SNOWFLAKE SERVER Applying changes to Server Instance manually leads to unique and distinct server configuration footprint (TECHNICAL DEBT) http://martinfowler.com/bliki/SnowflakeServer.html
- 19. VITAMINS INFRASTRUCTURE AS CODE DEV Code is a Contract OPS
- 20. VITAMINS INFRASTRUCTURE AS CODE DEV Code is a Contract OPS Put infrastructure into SCM
- 21. VITAMINS INFRASTRUCTURE AS CODE DEV Code is a Contract OPS` Make infrastructure part of app baseline
- 22. VITAMINS WE LIKE CI/CD ▸ We need feedback not to be afraid ▸ Feedback != SPAM ▸ It’s about size of release not frequency ▸ Don’t judge for broken builds ▸ Go home when build is green
- 24. ANTIPATTERN # 6 “JENKINS” DRIVEN DEVELOPMENT ` ` ` Pipelines are easily becomes your single point of failure CI
- 25. ANTIPATTERN # 6: PROPOSED SOLUTION SHIFTING RESPONSIBILITY CI users (DEVs or OPs) are best for managing their jobs ▸ Use DSL to build CI/CD pipelines ▸ Store CI/CD pipelines in git if possible make it part of app baseline ▸ Generate and bootsrap pipelines with API
- 26. ANTIPATTERN # 7 SLOW PIPELINES ` ` ` Over-engineered Pipelines Pipeline execution takes too long Leads to delayed feedback
- 27. ANTIPATTERN # 6: PROPOSED SOLUTION SLOW PIPELINES ` ` ` Parallelize where you can!
- 28. ANTIPATTERN # 7 MANUAL PROMOTION Engineer PRODUATIntegr TestsDEV System Tests ENVENVENV ENV
- 29. ANTIPATTERN # 7: PROPOSED SOLUTION KILL SWITCH FOR MANUAL TESTING PRODUATIntegr TestsDEV System Tests ENVENVENV ENV TEST
- 30. ANTIPATTERN # 7: PROPOSED SOLUTION AB TESTING PROD A Integr TestsDEV System Tests ENV ENVENV TEST A PROD B ENV TEST B feedback feedback measure
- 31. ANTIPATTERN # 7.1 CODEREVIEW PRODUATIntegr Tests System Tests ENVENVENV ENV DEV B feature branch DEV C code review
- 32. ANTIPATTERN # 7.1: PROPOSED SOLUTION REACTIVE CODEREVIEW ReleaseIntegr Tests System Tests ENVENV ENV Full Regression Calc technical debt ENV Nightly DEV A Standup code review …
- 33. ANTIPATTERN # 7.1 CODEREVIEW PRODUATIntegr TestsDEV A System Tests ENVENVENV ENV master DEV B feature branch DEV C code review
- 34. ANTIPATTERN # 8 DEV DEPLOYMENT VIA CI DEV ENV DEV ENV DEV ENV …DEV CI
- 35. ANTIPATTERN # 8: PROPOSED SOLUTION DEV DEPLOYMENT VIA CI DEV …NO CI DEV ENV DEV ENV DEV ENV
- 36. ANTIPATTERN # 8: PROPOSED SOLUTION DEV DEPLOYMENT VIA CI ENV ENV DEV ENV …DEV NO CI WANT TO HACK? SURE!
- 37. ANTIPATTERN # 8: PROPOSED SOLUTION DEV DEPLOYMENT VIA CI ENV ENV DEV ENV … YOU BROKE IT? YOU FIX IT! DEV NO CI
- 39. ANTIPATTERN # 9 GOLDEN IMAGE VM OS Problems ▸ Maintained manually ▸ No collaboration ▸ Hard to distribute ▸ Non versioning Chnorr Service
- 40. ANTIPATTERN # 9: PROPOSED SOLUTION PROVISIONING OS Chnorr Service
- 41. ANTIPATTERN # 9: PROPOSED SOLUTION PROVISIONING OS Configure Infrastructure code Harden Download Install Chnorr Service
- 42. ANTIPATTERN # 9: PROPOSED SOLUTION PROVISIONING OS Configure Infrastructure code Harden Download Install cmdb Chnorr Service
- 43. ANTIPATTERN # A FRAGILE PROVISIONING OS Configure Infrastructure code Harden Download Install ▸ Provisioning success < 100% ▸ Time to provision Chnorr Service
- 44. ANTIPATTERN # A FRAGILE PROVISIONING OS Configure Infrastructure code Harden Download Install ▸ Provisioning success < 100% ▸ Time to provision Chnorr Service
- 45. ANTIPATTERN # A FRAGILE PROVISIONING OS Configure Infrastructure code Harden Download Install ▸ Provisioning success < 100% ▸ Time to provision Chnorr Service
- 46. ANTIPATTERN # A FRAGILE PROVISIONING OS Configure Infrastructure code Harden Download Install ▸ Provisioning success < 100% ▸ Time to provision Chnorr Service
- 47. ANTIPATTERN # A.1 PRIVATE DATA CENTER PROVISIONING PRIVATE DATA CENTER: NO INTERNET!!! OS ConfigureHarden Download Install Chnorr Service
- 48. ANTIPATTERN # A.1 PRIVATE DATA CENTER PROVISIONING PRIVATE DATA CENTER: NO INTERNET!!! OS ConfigureHarden Download Install … apt yum gem Chnorr Service
- 49. ANTIPATTERN # A.1 PRIVATE DATA CENTER PROVISIONING PRIVATE DATA CENTER: NO INTERNET!!! OS ConfigureHarden Download Install … apt yum gem complicated? Chnorr Service
- 50. ANTIPATTERN # A.1: PROPOSED SOLUTION STEM CELLS static dynamic OS ConfigureHarden Download Install Chnorr Service
- 51. ANTIPATTERN # A.1: PROPOSED SOLUTION STEM CELLS OS ConfigureHarden Download Install LAB PRIVATE DATA CENTER Chnorr Service
- 52. ANTIPATTERN # A.1: PROPOSED SOLUTION STEM CELLS OS ConfigureHarden Download Install LAB PRIVATE DATA CENTER code packer PROVISIONSNAPSHOT docker Chnorr Service
- 53. ANTIPATTERN # B DEPENDENCY HELL OS Configure Infrastructure code Harden Download Install ▸ Version of libraries ▸ Version of packages ▸ Maintain dependencies Chnorr Service
- 54. ANTIPATTERN # B DEPENDENCY HELL OS Configure Infrastructure code Harden Download Install ▸ Version of libraries ▸ Version of packages ▸ Maintain dependencies ▸ Version of your infra code ▸ Maintain dependencies Chnorr Service
- 55. ANTIPATTERN # B: PROPOSED SOLUTION CANARY BUILDS Accept Tests CI Unit Test Provision … PRECISE version libs Accept Tests CI Unit Test Provision … LATEST version libs Chnorr Service Chnorr Service
- 56. ANTIPATTERN # C INFRASTRUCTURE PET Attributes of Pet ▸ Have meaningful names ▸ Long living instance ▸ Often needs manual nursing ▸ Requires scary patching ▸ Leads to snowflakes ▸ PaaS is modern pet
- 57. ANTIPATTERN # C: PROPOSED SOLUTION INFRASTRUCTURE CATTLE Attributes of Pet ▸ Have numbers in its name ▸ Short living instance ▸ Immutable configuration ▸ Recreate instead of patching ▸ Requires careful planning
- 58. ANTIPATTERN # C: PROPOSED SOLUTION PET TO TRANSFORMATION CATTLE EXAMPLE Chnorr Service Database
- 59. ANTIPATTERN # C: PROPOSED SOLUTION PET TO TRANSFORMATION CATTLE EXAMPLE Chnorr Service Database
- 60. ANTIPATTERN # C: PROPOSED SOLUTION PET TO TRANSFORMATION CATTLE EXAMPLE Chnorr Service Database Cattle Pet
- 61. ANTIPATTERN # C: PROPOSED SOLUTION PET TO TRANSFORMATION CATTLE EXAMPLE Chnorr Service Database Cattle Pet User Data
- 62. ANTIPATTERN # D SECRETS LEAK OS Configure Infrastructure code Install cmdb wrong place for your secrets wrong place for your secrets Chnorr Service
- 63. ANTIPATTERN # D: PROPOSED SOLUTION SECRETS LEAK ▸ Don’t store secrets with code ▸ Don’t store secrets with configuration ▸ Don’t leave secrets in service Secrets DON’Ts
- 64. ANTIPATTERN # D: PROPOSED SOLUTION SECRETS LEAK ▸ Secret can be leased and rotated ▸ tmpfs is your fiend Secrets DOs
- 65. ANTIPATTERN # D: PROPOSED SOLUTION SECURITY LEASING EXAMPLE Chnorr Service vault consul IAM Database api x hours leasing AWS
- 67. TAKEAWAYS LEARN PATTERNS BEFORE TOOLS Patterns Tools vs
- 68. TAKEAWAYS EVERYTHING MUST HAVE AN API REST DSL CLI
- 69. TAKEAWAYS SELF TESTABLE CODE ▸ Use assertions for infrastructure code ▸ Use acceptance test frameworks Tests improves your confidence
- 70. TAKEAWAYS READING ▸ Book: A Human Error Approach to Aviation Accident Analysis ▸ Author: Douglas A. Wiegmann Scott A. Shappell ▸ ISBN: 978-0754618737
- 71. TAKEAWAYS READING ▸ Book: Clean Code ▸ Author: Robert C Martin ▸ ISBN: 978-0132350884
- 72. THANK YOU