Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Shake up the Culture
with Automation!
Jul/06/2016
Hiroyuki Ito / 伊藤 宏幸
2
Hiroyuki Ito (The Hiro)
@hageyahhoo
Yahoo Japan Corporation
 Automation Coach
 Agile Coach
Who I Am
3
Speaker of Agile2014 
4
Japanese #1 Internet Provider
Web Services -> Smartphone Apps
Aiming to become a Tech Giant
About Yahoo Japan Corporation
5
What is
DevOps?
6
CALMS / CLAMS
General Definitions of DevOps
Culture
Automation
Lean
Measurement
Sharing
http://itrevolution.com/devops-c...
7
"DevOps" means:
The Hiro Says
Total Optimization
Whole Process Improvement
Based on Technical Foundation
8
Successful DevOps requires
the cultural change
in your organization.
9
Version
Control
Commit
Stage
Acceptance
Test Stage
Release
Stage
Manual
Test Stage
Performance
Testing Stage
Artifact
Re...
10
Version
Control
Commit
Stage
Acceptance
Test Stage
Release
Stage
Manual
Test Stage
Performance
Testing Stage
Artifact
R...
11
4 Teams with 3 Products
Our Target
We coaches supported them
Lots of Immature Engineers
Few Automated Tests
Lots of Int...
12
Agenda
1. Test Automation
3. Evaluation of Measures
2. Continuous Integration
4. Conclusion
13
1. Test Automation
3. Evaluation of Measures
2. Continuous Integration
4. Conclusion
14
Make product teams
write test scripts voluntarily
Self-running Team
Cross-functional Team
Higher Quality
Higher Product...
15
Background
16
Prerequisite
PHP
Cloud-based Infrastructure
Brownfield
17
Legacy Code: No test scripts!
Brownfield: Our Common Challenges
Test Automation
as a Technical Foundation
Improve desig...
18
Lots of Immature Engineers
Engineers
Test Scripts
How to Write
Test Scripts
The Importance of
Test Automation
How to Te...
19
Passive Culture
Front-end
Engineers
Database
Engineers
API
Engineers
Waiting for
Instructions
Silos Based on
Component ...
20
Concrete Actions
21
Scope of Test Automation
Unit Testing
Acceptance Testing
Performance Testing
Security Testing
Other ility Testing
22
Stages of Adapting to Unit Testing
1. Write simple test scripts
2. Adapt to TDD
3. Use Test Double (Mock)
4. Write char...
23
Schedule & Actions
Jan Feb Mar
Preparation
• Investigate products
• Create sample scripts
Execution
• Lectures
• Worksh...
24
A powerful tool
to learn
Unit Testing & TDD!
Cyber Dojo: Our Weapon
http://cyber-dojo.org/
25
1. Test Automation
3. Evaluation of Measures
2. Continuous Integration
4. Conclusion
26
Make product teams set up CI
processes
Our Goal
Nurture product teams
Avoid integration/deployment troubles
Achieve the...
27
Background
28
e.g.) During integration & deployment process
Tons of Failures were There
Needed to avoid these troubles fundamentally
...
29
Voice of CTO
Let’s do CI Companywide!
Top-down Approach
Bottom-up Approach
CTO
30
Concrete Actions
31
Accustomed to CI
Working
Samples
Team Members Coach
Worked with Samples
Configuration
of CI Server
CONFIG File
Did Pair...
32
Always added Automated Tests
Test! Test! Test!
Pull the real power of CI
Adapt to Test Automation more
Nurture product ...
33
Simplified CI Processes
Version
Control
AP
Server
CONFIG
File
DB
Server
Web
Server
Version
Control
AP
Server
CONFIG
Fil...
34
1. Test Automation
3. Evaluation of Measures
2. Continuous Integration
4. Conclusion
35
1. Results
36
Updated on Mar/31/2016
(during about 2 months)
Metrics
Item Quantity
Unit Tests Added/Updated 27
Software Bugs Detected...
37
Self-running Actions by Product Team Members
PO added “writing unit tests” as DoD
Started to write unit tests voluntari...
38
2. Lessons Learned
39
Coaches should NOT do everything
Role of Coaches
For retaining learning opportunity
For nurturing voluntary actions
Giv...
40
Should Create Unavoidable Situation
CI:
Unavoidable from Test Automation
Top-down Approach:
Unavoidable to achieve obje...
41
Based on the production code
Sample Test Scripts Are Very Useful
Easy to adapt to Test Automation
Can use them as initi...
42
Need to Solve Architectural Bottlenecks
Unable to apply mocks
Side effects
by static mocks
Hard to identify
web compone...
43
/**
* @test
*/
public function exampleOfTest() {
$sut = new Foo();
$result = $sut->bar('abc');
$this->assertEquals('hug...
44
/**
* @test
*/
public function exampleOfTest() {
$sut = Foo::new();
$result = …? /* Cannot write logics  */
$result = ...
45
Detected failures
up-front with Automated Tests
Prevented Troubles
Unexpected Environmental Changes
Failures of Merging...
46
Detected Misconfigurations on Servers
Only PHP
Package Hell 
Don’t need to compileInterpreter
Unused Packages Leakages...
47
Detected Misconfigurations on Servers
Can solve Package Hell 
PHP + Automated Test + CI
CI
48
Made team members
learn a lot based on
automated working processes
Nurtured Product Teams
Fast Feedback
Learn from Fail...
49
Technical Foundation
Technology-Driven Development
-> Presentation at Agile2014
CI/CDTest Automation
Teams
Members
Orga...
50
3. Next Actions
51
Next Challenges
Skills of other languages
Acceptance Test Automation
Gather other useful metrics
52
Example of Additional Metrics
Test &
Test Automation
Code Coverage
Cyclomatic Complexity
# of Bugs Detected
# of Member...
53
1. Test Automation
3. Evaluation of Measures
2. Continuous Integration
4. Conclusion
54
What is
DevOps?
55
"DevOps" means:
The Hiro Says Again
Total Optimization
Whole Process Improvement
Based on Technical Foundation
56
We are challenging
the cultural change
in our organization.
57
Test Automation with CI
is sufficiently valuable
Adapting to Test Automation & CI in DevOps Context
Iterative & Increme...
58
Adapting to Automation
is a long way
It’s Not Easy to Continue…
It takes 6 month
to do the same thing
for 1 team
in Fac...
59
But necessary to continue
for achieving
total optimization
60
Shake up
• product teams
• working processes
• ourselves
for
• total optimization
• huge success
Always Improve, Always...
61
With DevOps!
Upcoming SlideShare
Loading in …5
×

Shake up the Culture with Automation!

15,078 views

Published on

Keynote presentation at DevOps Summit 2016 in Taipei.
http://devopssummit.ithome.com.tw/

I explained the importance of Test Automation and Continuous Integration for cultural change in DevOps context.
DevOpsのコンテキストでの「文化の変化」に対する、テスト自動化とCIの重要性を説明した資料です。

Published in: Technology
  • Hi there! I just wanted to share a list of sites that helped me a lot during my studies: .................................................................................................................................... www.EssayWrite.best - Write an essay .................................................................................................................................... www.LitReview.xyz - Summary of books .................................................................................................................................... www.Coursework.best - Online coursework .................................................................................................................................... www.Dissertations.me - proquest dissertations .................................................................................................................................... www.ReMovie.club - Movies reviews .................................................................................................................................... www.WebSlides.vip - Best powerpoint presentations .................................................................................................................................... www.WritePaper.info - Write a research paper .................................................................................................................................... www.EddyHelp.com - Homework help online .................................................................................................................................... www.MyResumeHelp.net - Professional resume writing service .................................................................................................................................. www.HelpWriting.net - Help with writing any papers ......................................................................................................................................... Save so as not to lose
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • Sex in your area is here: ❶❶❶ http://bit.ly/39mQKz3 ❶❶❶
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • Dating for everyone is here: ♥♥♥ http://bit.ly/39mQKz3 ♥♥♥
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here

Shake up the Culture with Automation!

  1. 1. Shake up the Culture with Automation! Jul/06/2016 Hiroyuki Ito / 伊藤 宏幸
  2. 2. 2 Hiroyuki Ito (The Hiro) @hageyahhoo Yahoo Japan Corporation  Automation Coach  Agile Coach Who I Am
  3. 3. 3 Speaker of Agile2014 
  4. 4. 4 Japanese #1 Internet Provider Web Services -> Smartphone Apps Aiming to become a Tech Giant About Yahoo Japan Corporation
  5. 5. 5 What is DevOps?
  6. 6. 6 CALMS / CLAMS General Definitions of DevOps Culture Automation Lean Measurement Sharing http://itrevolution.com/devops-culture-part-1/
  7. 7. 7 "DevOps" means: The Hiro Says Total Optimization Whole Process Improvement Based on Technical Foundation
  8. 8. 8 Successful DevOps requires the cultural change in your organization.
  9. 9. 9 Version Control Commit Stage Acceptance Test Stage Release Stage Manual Test Stage Performance Testing Stage Artifact Repository Humble, Jez, and Farley David, Continuous Delivery, Addison-Wesley, 2010 Check-in The Hiro’s Scope
  10. 10. 10 Version Control Commit Stage Acceptance Test Stage Release Stage Manual Test Stage Performance Testing Stage Artifact Repository Humble, Jez, and Farley David, Continuous Delivery, Addison-Wesley, 2010 Check-in The Hiro’s Scope Cultural Change with Test Automation & CI
  11. 11. 11 4 Teams with 3 Products Our Target We coaches supported them Lots of Immature Engineers Few Automated Tests Lots of Integration Troubles
  12. 12. 12 Agenda 1. Test Automation 3. Evaluation of Measures 2. Continuous Integration 4. Conclusion
  13. 13. 13 1. Test Automation 3. Evaluation of Measures 2. Continuous Integration 4. Conclusion
  14. 14. 14 Make product teams write test scripts voluntarily Self-running Team Cross-functional Team Higher Quality Higher Productivity Our Goal
  15. 15. 15 Background
  16. 16. 16 Prerequisite PHP Cloud-based Infrastructure Brownfield
  17. 17. 17 Legacy Code: No test scripts! Brownfield: Our Common Challenges Test Automation as a Technical Foundation Improve design & program Refactoring Prevent regressions
  18. 18. 18 Lots of Immature Engineers Engineers Test Scripts How to Write Test Scripts The Importance of Test Automation How to Test Programs & Products What should I do?
  19. 19. 19 Passive Culture Front-end Engineers Database Engineers API Engineers Waiting for Instructions Silos Based on Component TeamsLeader
  20. 20. 20 Concrete Actions
  21. 21. 21 Scope of Test Automation Unit Testing Acceptance Testing Performance Testing Security Testing Other ility Testing
  22. 22. 22 Stages of Adapting to Unit Testing 1. Write simple test scripts 2. Adapt to TDD 3. Use Test Double (Mock) 4. Write characterization test if necessary 5. Refactor production code & architecture
  23. 23. 23 Schedule & Actions Jan Feb Mar Preparation • Investigate products • Create sample scripts Execution • Lectures • Workshops • Pair Programming Execution Remove architectural impediments
  24. 24. 24 A powerful tool to learn Unit Testing & TDD! Cyber Dojo: Our Weapon http://cyber-dojo.org/
  25. 25. 25 1. Test Automation 3. Evaluation of Measures 2. Continuous Integration 4. Conclusion
  26. 26. 26 Make product teams set up CI processes Our Goal Nurture product teams Avoid integration/deployment troubles Achieve the target from CTO Improve product teams
  27. 27. 27 Background
  28. 28. 28 e.g.) During integration & deployment process Tons of Failures were There Needed to avoid these troubles fundamentally Staging ENV Production ENV CONFIG File for Staging Troubles!
  29. 29. 29 Voice of CTO Let’s do CI Companywide! Top-down Approach Bottom-up Approach CTO
  30. 30. 30 Concrete Actions
  31. 31. 31 Accustomed to CI Working Samples Team Members Coach Worked with Samples Configuration of CI Server CONFIG File Did Pair Setting (like Pair Programming) Team Members Coach
  32. 32. 32 Always added Automated Tests Test! Test! Test! Pull the real power of CI Adapt to Test Automation more Nurture product teams
  33. 33. 33 Simplified CI Processes Version Control AP Server CONFIG File DB Server Web Server Version Control AP Server CONFIG File DB Server Web Server
  34. 34. 34 1. Test Automation 3. Evaluation of Measures 2. Continuous Integration 4. Conclusion
  35. 35. 35 1. Results
  36. 36. 36 Updated on Mar/31/2016 (during about 2 months) Metrics Item Quantity Unit Tests Added/Updated 27 Software Bugs Detected 10 Infrastructure Bugs Detected 24 Prevented Troubles 3 It may be insufficient, but it’s REAL.
  37. 37. 37 Self-running Actions by Product Team Members PO added “writing unit tests” as DoD Started to write unit tests voluntarily Established their own CI/CD strategy Established their own testing rules
  38. 38. 38 2. Lessons Learned
  39. 39. 39 Coaches should NOT do everything Role of Coaches For retaining learning opportunity For nurturing voluntary actions Giving hints is sometimes appropriate Total
  40. 40. 40 Should Create Unavoidable Situation CI: Unavoidable from Test Automation Top-down Approach: Unavoidable to achieve objectives Bottom-up Approach: Unavoidable to scale improvements Total
  41. 41. 41 Based on the production code Sample Test Scripts Are Very Useful Easy to adapt to Test Automation Can use them as initial test scripts Can learn their products & problems Team Members Coaches Test Automation
  42. 42. 42 Need to Solve Architectural Bottlenecks Unable to apply mocks Side effects by static mocks Hard to identify web components (Selenium WebDriver) Modified interfaces for injecting mocks Introduced Phake (Mock FW for PHP) Defined locators to each component (like “id” or “name”) Test Automation
  43. 43. 43 /** * @test */ public function exampleOfTest() { $sut = new Foo(); $result = $sut->bar('abc'); $this->assertEquals('huga', $result); } Pair Programming Clarifies Real SkillsTest Automation
  44. 44. 44 /** * @test */ public function exampleOfTest() { $sut = Foo::new(); $result = …? /* Cannot write logics  */ $result = Foo::bar('abc'); } Pair Programming Clarifies Real SkillsTest Automation
  45. 45. 45 Detected failures up-front with Automated Tests Prevented Troubles Unexpected Environmental Changes Failures of Merging Branches Leakages of Merging Branches CI
  46. 46. 46 Detected Misconfigurations on Servers Only PHP Package Hell  Don’t need to compileInterpreter Unused Packages Leakages of Packages CI
  47. 47. 47 Detected Misconfigurations on Servers Can solve Package Hell  PHP + Automated Test + CI CI
  48. 48. 48 Made team members learn a lot based on automated working processes Nurtured Product Teams Fast Feedback Learn from Failures Lots of Confidence to Go ahead CI
  49. 49. 49 Technical Foundation Technology-Driven Development -> Presentation at Agile2014 CI/CDTest Automation Teams Members Organization Products
  50. 50. 50 3. Next Actions
  51. 51. 51 Next Challenges Skills of other languages Acceptance Test Automation Gather other useful metrics
  52. 52. 52 Example of Additional Metrics Test & Test Automation Code Coverage Cyclomatic Complexity # of Bugs Detected # of Members Who Can Write Tests Update Times of Test Scripts (Lots of Updates = Valuable Test) CI/CD Cycle Time MTBF MTTR # of Releases per Week # of Members Who Can Operate CI/CD Servers
  53. 53. 53 1. Test Automation 3. Evaluation of Measures 2. Continuous Integration 4. Conclusion
  54. 54. 54 What is DevOps?
  55. 55. 55 "DevOps" means: The Hiro Says Again Total Optimization Whole Process Improvement Based on Technical Foundation
  56. 56. 56 We are challenging the cultural change in our organization.
  57. 57. 57 Test Automation with CI is sufficiently valuable Adapting to Test Automation & CI in DevOps Context Iterative & Incremental Improvement Do Kaizen / 改善
  58. 58. 58 Adapting to Automation is a long way It’s Not Easy to Continue… It takes 6 month to do the same thing for 1 team in Facebook Photo by Improve It - Kent Beck no Workshop Mapping XP.(2006) / CC BY-SA 2.0
  59. 59. 59 But necessary to continue for achieving total optimization
  60. 60. 60 Shake up • product teams • working processes • ourselves for • total optimization • huge success Always Improve, Always Advance
  61. 61. 61 With DevOps!

×