Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Progressive Deployment & NoDeploy

13,798 views

Published on

從 Progressive Deployment 面向敘談 DevOps 領域中的 Continuous Integration 及 Continuous Deployment,並提出 NoDeploy 瘋狂想法的可行性。

Published in: Technology

Progressive Deployment & NoDeploy

  1. 1. Progressive Deployment & NoDeploy 曾義峰 (Ant) yftzeng@gmail.com 2019-08-28
  2. 2. 2/81 Introduction & Research interest 13 年網際網路開發經驗, 4 年顧問資歷。 現任書亞集成股份有限公司 CTO 現任香港商帕格數碼媒體股份有限公司 CTO 現任希幔數位股份有限公司總工程師 現任台灣資安社群 CHROOT 成員 現任 TGO Networks 台北分會會籍委員 從事軟體架構研究數十年載,對資訊安全及智慧財產權略有研究。
  3. 3. 3/81 ~Dark 1980s 2008 2014 ITIL Design for Anti-FragilityDesign for Robustness Design for Resilience DevOps 堅實性設計 抗脆性設計 復原性設計 精心設計。 嚴格管控。 故障被設計為需要異常 處理。 將開發及維運為一體。 視故障為正常狀態。 為故障發生設計隔離。 將維運視為工程。 最小化人為錯誤。 為故障復原至正常狀態。 轉型 轉型 進化 SRE Partial Credit: https://noops.connpass.com 進化
  4. 4. 4/81Image Credit: https://www.freepik.com/free-icon/silo_869863.htm Sales Marketing Dev Ops Compliance (Security) Value DevOps Cost Cost
  5. 5. 5/81Image Credit: https://www.freepik.com/free-icon/silo_869863.htm Sales Marketing Dev Ops Compliance (Security) Value DevOps Dev(Sec)Ops
  6. 6. 6/81Image Credit: DevOps-Agile-Perspective_Oct-18-Appleton.pdf Silo ( 穀倉 / 壁壘 ) DevOps
  7. 7. 7/81Image Credit: DevOps-Agile-Perspective_Oct-18-Appleton.pdf DevOps
  8. 8. 8/81 Code Test Acceptance Deploy Operate Continuous Integration / 持續整合 Continuous Delivery / 持續交付 Continuous Deployment / 持續部署 DevOps / SRE Commit Dev Test Staging Test Deploy Prod Continuous Deployment Local Dev Staging Prod Environments Chaos
  9. 9. 9/81 為什麼今天建立一個應用程式,我們需要學習 Git 、 Kubernetes 、 Docker 、 SQL 、 NoSQL 、 DNS 、 gRPC 、 GraphQL 、數個 AWS 雲服務, 以及其餘無止盡的工具和技術清單列表。
  10. 10. 10/81 大膽的想法
  11. 11. 11/81 No Acceptance
  12. 12. 12/81 Code Test Acceptance Deploy Operate Continuous Integration / 持續整合 Continuous Delivery / 持續交付 Continuous Deployment / 持續部署 DevOps / SRE Commit Dev Test Staging Test Deploy Prod Continuous Deployment Local Dev Staging Prod Environments Chaos
  13. 13. 13/81 Code Test Deploy Operate Continuous Integration / 持續整合 Continuous Delivery / 持續交付 Continuous Deployment / 持續部署 DevOps / SRE Commit Dev Test Staging Test Deploy Prod Continuous Deployment Local Dev Staging Prod Environments Chaos
  14. 14. 14/81 No Acceptance No Deploy (artifact)
  15. 15. 15/81 Code Test Deploy Operate Continuous Integration / 持續整合 Continuous Delivery / 持續交付 Continuous Deployment / 持續部署 DevOps / SRE Commit Dev Test Staging Test Deploy Prod Continuous Deployment Local Dev Staging Prod Environments Chaos
  16. 16. 16/81 Code Test Operate Continuous Integration / 持續整合 Continuous Delivery / 持續交付 Continuous Deployment / 持續部署 DevOps / SRE Commit Dev Test Staging Test Deploy Prod Continuous Deployment Local Dev Staging Prod Environments Chaos
  17. 17. 17/81 Deploy (artifact) 隨時皆可部署任意版本至任意環境
  18. 18. 18/81 Deploy (artifact) 隨時皆可部署任意版本至任意環境 No Local No Staging No Deploy
  19. 19. 19/81 Deploy (artifact) 隨時皆可部署任意版本至任意環境 No Local No Staging No Deploy
  20. 20. 20/81 No Deploy (artifact) No Staging No Acceptance
  21. 21. 21/81 Code Test Operate Continuous Integration / 持續整合 Continuous Delivery / 持續交付 Continuous Deployment / 持續部署 DevOps / SRE Commit Dev Test Staging Test Deploy Prod Continuous Deployment Local Dev Staging Prod Environments Chaos
  22. 22. 22/81 Code Test Operate Continuous Integration / 持續整合 Continuous Delivery / 持續交付 Continuous Deployment / 持續部署 DevOps / SRE Commit Dev Test Staging Test Deploy Prod Continuous Deployment Local Dev Staging Prod Environments Chaos
  23. 23. 23/81 Staging Staging 環境必須與 Production 環境幾乎 相同,這十分困難,否則測試就不準了。 Trying to mirror your staging environment to production is a fool’s errand. Just give up. 嘗試將 Staging 視為 Production 是一個愚蠢的事。放棄吧。 Partial Credit: https://thenewstack.io/honeycombs-charity-majors-go-ahead-test-in-production/
  24. 24. 24/81 Staging Staging 環境必須與 Production 環境幾乎 相同,這十分困難,否則測試就不準了。 Trying to mirror your staging environment to production is a fool’s errand. Just give up. 嘗試將 Staging 視為 Production 是一個愚蠢的事。放棄吧。 Partial Credit: https://thenewstack.io/honeycombs-charity-majors-go-ahead-test-in-production/
  25. 25. 25/81 Staging Credit: https://www.infoq.cn/article/b9lG4O8oJ8bWcFFQfFKE
  26. 26. 26/81 No Deploy (artifact) No Staging No QA / CI No Acceptance
  27. 27. 27/81 Code Test Operate Continuous Integration / 持續整合 Continuous Delivery / 持續交付 Continuous Deployment / 持續部署 DevOps / SRE Commit Dev Test Staging Test Deploy Prod Continuous Deployment Local Dev Staging Prod Environments Chaos
  28. 28. 28/81 Code Operate Continuous Integration / 持續整合 Continuous Delivery / 持續交付 Continuous Deployment / 持續部署 DevOps / SRE Commit Dev Test Staging Test Deploy Prod Continuous Deployment Local Dev Staging Prod Environments Chaos
  29. 29. 29/81 No Deploy (artifact) No Staging No QA / CI No Visible container No Acceptance
  30. 30. 30/81 No Deploy (artifact) No Staging No QA / CI No Visible container No Test Locally No Acceptance
  31. 31. 31/81 Code Operate Continuous Integration / 持續整合 Continuous Delivery / 持續交付 Continuous Deployment / 持續部署 DevOps / SRE Commit Dev Test Staging Test Deploy Prod Continuous Deployment Local Dev Staging Prod Environments Chaos
  32. 32. 32/81 Code Operate Continuous Integration / 持續整合 Continuous Delivery / 持續交付 Continuous Deployment / 持續部署 DevOps / SRE Commit Dev Test Staging Test Deploy Prod Continuous Deployment Local Dev Staging Prod Environments Chaos
  33. 33. 33/81 Test Locally 在 Microservices 環境中, 幾乎不可避免要在分佈式環境中進行部署和測試。
  34. 34. 34/81 Test Locally 在 Microservices 環境中, 幾乎不可避免要在分佈式環境中進行部署和測試。
  35. 35. 35/81 No Deploy (artifact) No Staging No QA / CI No Visible container No Test Locally Only Code CommitNo Acceptance
  36. 36. 36/81 Code Operate Continuous Integration / 持續整合 Continuous Delivery / 持續交付 Continuous Deployment / 持續部署 DevOps / SRE Commit Dev Test Staging Test Deploy Prod Continuous Deployment Local Dev Staging Prod Environments Chaos
  37. 37. 37/81 Only Feature FlagNo Deploy (artifact) No Staging No QA / CI No Visible container No Test Locally Only Code CommitNo Acceptance
  38. 38. Cost in Time hours mins ~ hour secs ~ mins Applicability 2% 8% 90% Toggle Feature Flag ReDeploy Previous Version Rollback Revert Commit Incident Recovery
  39. 39. 39/81 Feature Flag 01 所有程式皆為部署,並可控制開關,減少整合問題 All code is deployed, but control exposure, reduces integration issue. 02 控制細緻度可至特定人層級或是開發分支 Flags provide runtime control down to individual user. 03 開發者可以增加或刪除功能,無需再次部署 Users can be added or removed with no redeployment 04 暗啟動 Enables dark launch Credit: Progressive Deployment, Experimentation, Multitenancy, No Downtime, Cloud Security (2018-02-23).pdf Image Credit: https://opensource.com/article/18/2/feature-flags-ring-deployment-model
  40. 40. 40/81 Feature Flag 01 提交程式至主幹可能會影響其他功能 Commit code on trunk may affects other features. 02 未完成的功能可能也會部署,如果配置有誤可能將未完成的功能開啟 Unfinished features may be deployed, and turned on if the configuration is incorrect. Image Credit: https://opensource.com/article/18/2/feature-flags-ring-deployment-model
  41. 41. 41/81 Only Feature Flag Only Progressive Deployment No Deploy (artifact) No Staging No QA / CI No Visible container No Test Locally Only Code CommitNo Acceptance
  42. 42. 42/81 Only Feature Flag Only Progressive Deployment Only Observability No Deploy (artifact) No Staging No QA / CI No Visible container No Test Locally Only Code CommitNo Acceptance
  43. 43. 43/81 Only Feature Flag Only Progressive Deployment And Test in Production Only Observability No Deploy (artifact) No Staging No QA / CI No Visible container No Test Locally Only Code CommitNo Acceptance
  44. 44. 44/81 Test in Production 在 Microservices 環境中,尤其是 FaaS 環境中, Test in production 是一種實際的需要。
  45. 45. 45/81 Test in Production Chaos Engineering @ Netflix 這些是需要整合測試的故障模式示例,因為它們涉及服務之間的互動。在某 些情況下,可能只能在 Production 進行完全整合測試。在 Netflix ,根本不 可能完全重現整個架構並運行端到端測試。 當可以在 Testing 環境中重現整個系統時,我們仍然相信需要在 Production 運行實驗,因為在 Testing 環境中永遠不可能完全重現系統的所有方面。總 是存在差異,例如合併客戶端與真實客戶端相比的行為方式或 DNS 配置問 題。 Credit: Chaos Engineering IEEE Software (2016-05-05).pdf
  46. 46. 46/81Image Credit: Progressive Deployment, Experimentation, Multitenancy, No Downtime, Cloud Security (2018-02-23).pdf Microsoft/Azure
  47. 47. 47/81Image Credit: Progressive Deployment, Experimentation, Multitenancy, No Downtime, Cloud Security (2018-02-23).pdf Microsoft/Azure
  48. 48. 48/81Image Credit: Progressive Deployment, Experimentation, Multitenancy, No Downtime, Cloud Security (2018-02-23).pdf Microsoft/Azure
  49. 49. 49/81 Code Operate Continuous Integration / 持續整合 Continuous Delivery / 持續交付 Continuous Deployment / 持續部署 DevOps / SRE Commit Dev Test Staging Test Deploy Prod Continuous Deployment Local Dev Staging Prod Environments Chaos
  50. 50. 50/81 Code Operate Continuous Integration / 持續整合 Continuous Delivery / 持續交付 Continuous Deployment / 持續部署 DevOps / SRE Prod Continuous Deployment Prod Environments ChaosCommit
  51. 51. 51/81 Only Feature Flag Only Progressive Deployment And Test in Production Only Observability No ACCEPTANCE No Deploy (artifact) No Staging No QA / CI No Visible container No Test Locally Only Code Commit
  52. 52. 52/81 ~Dark 1980s 2008 2014 ITIL Design for Anti-FragilityDesign for Robustness Design for Resilience DevOps 堅實性設計 抗脆性設計 復原性設計 精心設計。 嚴格管控。 故障被設計為需要異常 處理。 將開發及維運為一體。 視故障為正常狀態。 為故障發生設計隔離。 將維運視為工程。 最小化人為錯誤。 為故障復原至正常狀態。 轉型 轉型 進化 SRE Partial Credit: https://noops.connpass.com 進化
  53. 53. 53/81 ~Dark 1980s 2008 2014 ITIL Design for Anti-FragilityDesign for Robustness Design for Resilience DevOps 堅實性設計 抗脆性設計 復原性設計 精心設計。 嚴格管控。 故障被設計為需要異常 處理。 將開發及維運為一體。 視故障為正常狀態。 為故障發生設計隔離。 將維運視為工程。 最小化人為錯誤。 為故障復原至正常狀態。 轉型 轉型 進化 SRE Dev(ops) Design for Deployless Partial Credit: https://noops.connpass.com NoDeploy 2019 想像 想像進化 無部署設計
  54. 54. 54/81 OpsDev OpsDev Dev(ops)
  55. 55. 55/81 NoDeploy != No Deploy(ment) NoDeploy == No conscious Deploy(ment) 無察覺部署
  56. 56. 56/81 How do you go fast And zero downtime Font Credit: https://www.dafont.com/theme.php?cat=109 (Defused)
  57. 57. 57/81 Your aim won’t be perfect, control over the blast radius. 你的瞄準並不完美, 請控制好爆炸半徑。 Image Credit: https://www.behance.net/gallery/57558081/FREE-BUSINESS-ICONS Progressive Deployment
  58. 58. 58/81Credit: https://thenewstack.io/the-rise-of-progressive-delivery-for-systems-resilience/ 01 User segmentation 使用者細分 02 Traffic management 流量管理 03 Observability 可觀測性 04 Automation 自動化 Progressive Deployment
  59. 59. 59/81 Progressive Deployment Observability != Monitoring Observability == Logs + Traces + Metrics Monitoring is what. Observability is why. 可觀測性 監控
  60. 60. 60/81 人力 資源能力 時間 Progressive Deployment
  61. 61. 61/81 Credit: https://thenewstack.io/the-rise-of-progressive-delivery-for-systems-resilience/ Image Credit: https://www.trisignup.com/Race/24568/Sponsor/94608 Progressive deployment is continuous deploy with fine-grained control over the blast radius. 漸進式部署是對爆破半徑進行細部控制的連續部署。 Progressive Deployment
  62. 62. 62/81 Credit: https://thenewstack.io/the-rise-of-progressive-delivery-for-systems-resilience/ Image Credit: https://support.google.com/optimize/answer/7012154?hl=en Deploying a service is not that same as activating it for all users. 部署一個服務,未必需要讓所有使用者皆同時生效。 Progressive Deployment
  63. 63. 63/81Credit: https://thenewstack.io/the-rise-of-progressive-delivery-for-systems-resilience/ 01 Feature flag 功能標誌 02 A/B Test A/B 測試 03 Blue-green deployment 藍綠部署 04 Canary deployment 金絲雀部署 05 Service mesh 服務網格 Progressive Deployment
  64. 64. 64/81 Progressive Deployment Chaos Engineering 可協助我們找到 Edge / Boundary / Blast radius 。 Image Credit: https://medium.com/walmartlabs/chaos-engineering-using-concord-gremlin-c922d7dcd629
  65. 65. 65/81 Progressive Deployment Image Credit: https://engineering.fb.com/web/rapid-release-at-massive-scale/ (2017-08-31) Facebook
  66. 66. 66/81 Progressive Deployment Image Credit: Progressive Deployment, Experimentation, Multitenancy, No Downtime, Cloud Security (2018-02-23).pdf Microsoft/Azure
  67. 67. 67/81 Code Operate Continuous Integration / 持續整合 Continuous Delivery / 持續交付 Continuous Deployment / 持續部署 DevOps / SRE Prod Continuous Deployment Prod Environments ChaosCommit
  68. 68. 68/81 SLA, SLI, SLO
  69. 69. 69/81 Code Operate Continuous Integration / 持續整合 Continuous Delivery / 持續交付 Continuous Deployment / 持續部署 DevOps / SRE Prod Continuous Deployment Prod Environments ChaosCommit
  70. 70. 70/81 Code Continuous Integration / 持續整合 Continuous Delivery / 持續交付 Continuous Deployment / 持續部署 DevOps / SRE Prod Continuous Deployment Prod Environments ChaosCommit
  71. 71. 71/81 許多最好的想法都是瘋狂的,直到他們不再瘋狂而是現實。
  72. 72. 72/81 DevOps N Font Credit: https://www.dafont.com/theme.php?cat=109 (Defused) DevNops
  73. 73. 73/81 我不知道何時會發生,但這是我對下一步 ( 未來 ) 的想像
  74. 74. 74/81 DevOps O O DevOoops Font Credit: https://www.dafont.com/theme.php?cat=109 (Defused)
  75. 75. 75/81Image Credit: https://medium.com/darklang/the-design-of-dark-59f5d38e52d2
  76. 76. 76/81Credit: https://medium.com/darklang/how-dark-deploys-code-in-50ms-771c6dd60671 Dark
  77. 77. 77/81Image Credit: https://medium.com/darklang/how-dark-deploys-code-in-50ms-771c6dd60671 Dark
  78. 78. 78/81Credit: https://darklang.com Dark
  79. 79. 79/81 Only Feature Flag Only Progressive Deployment And Test in Production Only Observability No ACCEPTANCE No Deploy (artifact) No Staging No QA / CI No Visible container No Test Locally Only Code Commit
  80. 80. 80/81 OpsDev OpsDev Dev(ops)
  81. 81. 81/81 yftzeng@gmail.com https://www.facebook.com/yftzeng.tw https://twitter.com/yftzeng 曾義峰 (Ant)

×