0
楽天が挑むDevOps
July,29th 2013
YoheiSasaki
PaaS Development& Operations Section
Architecture CommitteeOffice
DevelopmentUnit
R...
About me
• Yohei Sasaki
• Lead of Rakuten Platform as a Service.
• Technology Background:
• IBM Tivoli / Ruby on Rails / A...
About us
• Rakuten, Inc.
• Internet Service Company
• Many businesses
• 1000+ developers
• 70+ teams
• 1000+ projects
インター...
Go Global
• Englishnization 英語公用語化
Agenda for Today
• Why/What is RPaaS.
• Practices in RPaaS.
• Messages for DevOps people
WHY/WHAT IS RPAAS
RPaaSについて
Business Objectives
• Expand business services more rapidly
• Reduce the cost
早い + 安い + 旨い
Rakuten Platform as a Service
• Everything should be automated.
• Developer’s IN Rakuten should focus on
business implemen...
Traditional Service Building Steps
• Too many human to
human workflows
• Too many approvals.
• Too many meetings.
• Too ma...
Steps in RPaaS
Developer
rpaas push myapp
Grab a coffee
• Make it simple
• Deploy
• rpaas push myapp
• URL mapping
• rpaas...
Tricks – DevOps Layering
Service Devs
System Operators
Focus on Stable Infrastructure
Focus on Development
Ask
Response
依頼
Tricks – DevOps Layering
RPaaS API Service
Service DevOps
RPaaS DevOps
Focus on Infrastructure Features
Focus on Business ...
PaaS
Reference: RPaaS Architecture
IaaS
Network
Hardware
Operating System
PaaS Kernel
Data
Services
Application
Runtime
Op...
Key Concepts
• Standard Document => Programmable API
• Standard Infrastructure => Infrastructure with Library
Key Concepts
• Standard Document => Programmable Standard
• Standard Infrastructure => Infrastructure with Library
標準化 道具
Key Concepts
• Standard Document => Programmable Standard
• Standard Infrastructure => Infrastructure with Library
標準化 道具
...
Key Concepts
• Standard Document => Programmable Standard
• Standard Infrastructure => Infrastructure with Library
標準化 道具
...
Service Team DevOps
rpaas stats
rpaas instances
rpaas log-storage
git push
rpaas push
rpaas map
デプロイサイクルが全部自動化
デバッグ用オペレーショ...
PRACTICES IN RPAAS
どうやってRPaaSを作っているのか
Change the business
RPaaS DevOps
Focus on Infrastructure Features
System Operators
Focus on Stable Infrastructure
やり方を変える
...
Internal Competition
• Objectives:
• Provide multiple options for users
PaaS
IaaS
Internal Customers Internal Market
ユーザーを...
Member Roles Definition
• Objectives:
• Clarify the responsibilities for each member.
• A member has more than one role.
R...
KPI Definition
• Objectives:
• Clarify the mission.
• Evaluate the mission result.
Role In charge of:
System
Administrator...
Release Note
• Objectives:
• Clarify impacts
• Clarify who are customers of what you develop.
リリースノートを
全員で最初に作る
Communication in Code
• Objectives:
• Do not use manual ops.
• Anyone can do any ops
紙だけでコミュニケーションしない
Git-flow as Communication Protocol
• Objectives:
• Design & implementations in code repositories, not in meeting
rooms.
• ...
Standard Release Procedure
• Objectives:
• Reduce manual operations.
$ cd ~/rpaas-commander
$ cap setup-summary
$ cap setu...
CI for infrastructure
• Objectives:
• Reduce the release tasks of which 80% are ‘task confirmation.
確認作業もコードで指示
Rolling Update
• Objectives:
• Easy to rollback
• A/B test for infrastructure
L7 Software LoadBalancer
Virtual Machine (v1...
Eat our own dog food
• Objectives:
• Stands on customer side.
• Understand pros and cons for our weapons.
自分の道具を自分で使う
Off topic?: Englishnization
• Team members from 7 countries
• Finding Specialists in the world
• Source Code / Diagram > E...
MESSAGES FOR DEVOPS
PEOPLE
Key Factors for DevOps
• Everyone should have ‘developer role’.
• Tool is a tool.
• Everything should be automated.
Key Factors for DevOps
• Everyone should have ‘administration role’.
• Tool is a tool.
• Everything should be automated.
Everyone should have ‘developer role’
• De-facto standard communication method.
• Read source
• Find a bug
• Fix the bug
•...
Tool is a tool
• Chef does not solve the barrier between Devs and Ops.
@@ -1,4 +1,4 @@
-package "apache" do
+package "ngin...
Everything should be automated
• Everything / Uncompromising
• Automate after manual operations
“手作業” を使っていいのは1ヶ月に1度
手順書が必...
Thank you!
And we are hiring!
楽天 PaaS開発エンジニア
• Global Team
• Emerging Technology
• Trend words
• Challenge Everyday
Upcoming SlideShare
Loading in...5
×

楽天が挑むDevOps

6,992

Published on

Systems Management Forum 2013 (http://ac.nikkeibp.co.jp/nc/smf2013s/) の基調講演資料です。楽天では2012年4月から社内でPaaSを展開しており、その基盤の上でさまざまなサービスが開発、運用されています。講演では、楽天PaaSでのDevOpsの実践方法について具体的に紹介しました。

Published in: Technology
2 Comments
34 Likes
Statistics
Notes
No Downloads
Views
Total Views
6,992
On Slideshare
0
From Embeds
0
Number of Embeds
10
Actions
Shares
0
Downloads
77
Comments
2
Likes
34
Embeds 0
No embeds

No notes for slide

Transcript of "楽天が挑むDevOps"

  1. 1. 楽天が挑むDevOps July,29th 2013 YoheiSasaki PaaS Development& Operations Section Architecture CommitteeOffice DevelopmentUnit Rakuten,Inc.
  2. 2. About me • Yohei Sasaki • Lead of Rakuten Platform as a Service. • Technology Background: • IBM Tivoli / Ruby on Rails / Apache Hadoop / Apache CouchDB / Node.js / CloudFoundry / … • Job Background: • Technical Support Engineer / Application Developer / System Developer / System Administrator / Project Manager / … 万屋
  3. 3. About us • Rakuten, Inc. • Internet Service Company • Many businesses • 1000+ developers • 70+ teams • 1000+ projects インターネット企業
  4. 4. Go Global • Englishnization 英語公用語化
  5. 5. Agenda for Today • Why/What is RPaaS. • Practices in RPaaS. • Messages for DevOps people
  6. 6. WHY/WHAT IS RPAAS RPaaSについて
  7. 7. Business Objectives • Expand business services more rapidly • Reduce the cost 早い + 安い + 旨い
  8. 8. Rakuten Platform as a Service • Everything should be automated. • Developer’s IN Rakuten should focus on business implementation in the latest technologies. • Platform as a Service infrastructure for Developers in Rakuten, Inc. • Like Google App Engine, Heroku, ..etc • Abstraction Layer of On-Premise/Off-Premise プライベート クラウド?
  9. 9. Traditional Service Building Steps • Too many human to human workflows • Too many approvals. • Too many meetings. • Too many documents. Request Servers Wait for hardware/VM creation Setup Operating System Install Software Setup monitoring Deploy website Check / Test Add to Load Balancer ... Developer Operators 複雑 ステークホルダー多い 申請書たくさん
  10. 10. Steps in RPaaS Developer rpaas push myapp Grab a coffee • Make it simple • Deploy • rpaas push myapp • URL mapping • rpaas map myapp xxx.rakuten.co.jp • Manage scaling • rpaas instances myapp +10 • Bind database • rpaas bind-service db1 myapp • Set Monitoring • rpaas monitor myapp –cpu 90 –mem 80 コマンド1発
  11. 11. Tricks – DevOps Layering Service Devs System Operators Focus on Stable Infrastructure Focus on Development Ask Response 依頼
  12. 12. Tricks – DevOps Layering RPaaS API Service Service DevOps RPaaS DevOps Focus on Infrastructure Features Focus on Business App Ask Response Consult Consult 自動化された標準 一緒に 考える
  13. 13. PaaS Reference: RPaaS Architecture IaaS Network Hardware Operating System PaaS Kernel Data Services Application Runtime Operation API SDK RPaaS API Service 一緒に 考える場所
  14. 14. Key Concepts • Standard Document => Programmable API • Standard Infrastructure => Infrastructure with Library
  15. 15. Key Concepts • Standard Document => Programmable Standard • Standard Infrastructure => Infrastructure with Library 標準化 道具
  16. 16. Key Concepts • Standard Document => Programmable Standard • Standard Infrastructure => Infrastructure with Library 標準化 道具 早い 旨い
  17. 17. Key Concepts • Standard Document => Programmable Standard • Standard Infrastructure => Infrastructure with Library 標準化 道具 早い 旨い安い
  18. 18. Service Team DevOps rpaas stats rpaas instances rpaas log-storage git push rpaas push rpaas map デプロイサイクルが全部自動化 デバッグ用オペレーションもAPIで提供 サーバーのことは気にしない
  19. 19. PRACTICES IN RPAAS どうやってRPaaSを作っているのか
  20. 20. Change the business RPaaS DevOps Focus on Infrastructure Features System Operators Focus on Stable Infrastructure やり方を変える 安定だけではない 旨さを求める
  21. 21. Internal Competition • Objectives: • Provide multiple options for users PaaS IaaS Internal Customers Internal Market ユーザーを確保する ところからが仕事
  22. 22. Member Roles Definition • Objectives: • Clarify the responsibilities for each member. • A member has more than one role. Role In charge of: Release Manager • Conducting the release operations by cooperation with administrator. System Administrator • Handling Daily Operation. • Quick Fix on troubles. • Reporting system stats to Team Leader. System Developer • RPaaS functions from planning to operations. • Negotiation with Team Leader and Managers to decide the release line and scope. Team Leader • Coordination of system developers and related people • Evaluation for RPaaS. Manager • Budget Control • HR related things such as evaluation “兼任できる” ロールを明確にする
  23. 23. KPI Definition • Objectives: • Clarify the mission. • Evaluate the mission result. Role In charge of: System Administrator • # of alerts • Time to solve the alerts Release Manager • # of releases. • Time to release. System Developer • # of features newly implemented • # of solved issues Team Leader • # of launched business services ロールに目標をつけ、評価する
  24. 24. Release Note • Objectives: • Clarify impacts • Clarify who are customers of what you develop. リリースノートを 全員で最初に作る
  25. 25. Communication in Code • Objectives: • Do not use manual ops. • Anyone can do any ops 紙だけでコミュニケーションしない
  26. 26. Git-flow as Communication Protocol • Objectives: • Design & implementations in code repositories, not in meeting rooms. • Isolate each feature feature/* develop release/* master hotfix/* Administrator Release Manager Developer Admin/RM 独自のルールは極力作らない
  27. 27. Standard Release Procedure • Objectives: • Reduce manual operations. $ cd ~/rpaas-commander $ cap setup-summary $ cap setup-role -n $ cap setup-role $ cap service-in 手順書 = 動くコード
  28. 28. CI for infrastructure • Objectives: • Reduce the release tasks of which 80% are ‘task confirmation. 確認作業もコードで指示
  29. 29. Rolling Update • Objectives: • Easy to rollback • A/B test for infrastructure L7 Software LoadBalancer Virtual Machine (v1) Virtual Machine (v1) Virtual Machine (v1) Virtual Machine (v2) Virtual Machine (v2) Virtual Machine (v2) アップデートではなく新規構築
  30. 30. Eat our own dog food • Objectives: • Stands on customer side. • Understand pros and cons for our weapons. 自分の道具を自分で使う
  31. 31. Off topic?: Englishnization • Team members from 7 countries • Finding Specialists in the world • Source Code / Diagram > English?
  32. 32. MESSAGES FOR DEVOPS PEOPLE
  33. 33. Key Factors for DevOps • Everyone should have ‘developer role’. • Tool is a tool. • Everything should be automated.
  34. 34. Key Factors for DevOps • Everyone should have ‘administration role’. • Tool is a tool. • Everything should be automated.
  35. 35. Everyone should have ‘developer role’ • De-facto standard communication method. • Read source • Find a bug • Fix the bug • Create a new application. #!/usr/bin/env ruby puts “Hello Developers!” ソースコードいじりましょう
  36. 36. Tool is a tool • Chef does not solve the barrier between Devs and Ops. @@ -1,4 +1,4 @@ -package "apache" do +package "nginx" do action :install end I wanna use Nginx! Unacceptable. Pull Request Chef 使ってるかどうかは 問題ではない
  37. 37. Everything should be automated • Everything / Uncompromising • Automate after manual operations “手作業” を使っていいのは1ヶ月に1度 手順書が必要な作業を実施したら、 自動化するまでが仕事
  38. 38. Thank you!
  39. 39. And we are hiring! 楽天 PaaS開発エンジニア • Global Team • Emerging Technology • Trend words • Challenge Everyday
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×