Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Coolblue - Behind the Scenes Continuous Integration & Deployment

1,053 views

Published on

Do you want to know what our process looks like from code to production? Or do you want to learn how we envision the future of deployment at Coolblue? During this evening, our Engineers will show you a peek Behind the Scenes and tell you everything about our challenges with Continuous Integration and Deployment.

Published in: Technology
  • Be the first to comment

Coolblue - Behind the Scenes Continuous Integration & Deployment

  1. 1. July 2016 | Behind the Scenes
  2. 2. Agenda. ● Welcome & introduction ● From Code to Node ● Short Break ● A look to the future of deployment ● Tour of our office ● Drinks ● Fin
  3. 3. July 2016 | Behind the Scenes Flynn Bundy <@bundyfx> & Matthew Hodgkins <@matthodge>
  4. 4. “I was trying to sleep last night but all I could think about was how to make our pipelines better.” Matt
  5. 5. PLAN.
  6. 6. CODE. [sensu]Get(){...} [void]Set(){...} [boolean]Test(){...} static [void] XMLConversion () {...} static [void] InstallService () {...} static [void] ConfigureAgentTransport([Transport]$Transport) {...}
  7. 7. CODE. node $AllNodes.Where{$_.Role -eq "our_app"}.role {...} node $AllNodes.Where{$_.Role -eq "our_app_1"}.role {...} node $AllNodes.Where{$_.Role -eq "our_app_2"}.role {...} node $AllNodes.Where{$_.Role -eq "our_app_3"}.role {...} node $AllNodes.Where{$_.Role -eq "our_app_4"}.role {...}
  8. 8. BUILD.
  9. 9. BUILD.
  10. 10. TEST. Context Download Agent Method [+] Should download the Sensu client 7.58s [+] Should be the full file 101ms Context Install Agent Method [+] Should Install the Sensu client 38.49s Context ConfigureAgentClient Method [+] Should be correctly formatted into JSON 1.77s [+] Should contain the Correct Information in the JSON | Subscriptions 20ms [+] Should contain the Correct Information in the JSON | Name 19ms [+] Should contain the Correct Information in the JSON | Private IPv4 Address 15ms Context ConfigureAgentTransport Method [+] Should be formatted correctly into JSON 154ms [+] Should contain the Correct Information in the JSON | Name 16ms [+] Should contain the Correct Information in the JSON | Reconnect on Error 13ms Context InstallService Method [+] Should deploy and install the Sensu client service 256ms Context Conversion XML Method [+] Should correctly create an valid XML configuration file 119ms Context RemoveAgent Method [+] It should remove the agent 239.16s Describing DSC Based methods Context Testing DSC based methods [+] Test() Method should return a Boolean Type 283ms [+] Get() Method should return a Sensu Type 31ms
  11. 11. TEST.
  12. 12. Move fast and break things. Unless you are breaking stuff, you are not moving fast enough. Mark Zuckerberg
  13. 13. DEPLOY.
  14. 14. DEPLOY.
  15. 15. WHAT IS DSC?.
  16. 16. WHAT IS DSC?.
  17. 17. WHAT IS DSC?.
  18. 18. OPERATE.
  19. 19. PILLARS OF MONITORING.
  20. 20. WHY MONITORING IS HARD. ▪Automation is an after thought ▪Forgetting to add new services ▪Checks are hard to write ▪Don’t want to do annoying things
  21. 21. ENTER SENSU. IIS /purchases Sensu Agent Sensu Server Rabbit MQ
  22. 22. STEP 1 - DEPLOY AGENT WITH DSC. Sensu Install { Ensure = "Present" DownloadURL = 'https://sensu.global.net/msi/sensu-0.24.0-1.msi' RabbitMQServer = 'coolblue-rabbitmq.internal.fake' Transport = 'rabbitmq' vHost = 'sensu' User = 'sensu' Port = '5672' Password = 'some_password' #Encrypted of course Subscription = 'windows_base' }
  23. 23. IIS /purchases Sensu Agent Sensu Server Rabbit MQ Deploy Agent
  24. 24. STEP 2 - CREATE SOME CHECKS. { "checks": { "disk_usage": { "command": "check-windows-disk.rb.bat -w 85 -c 95", "interval": 600, "subscribers": [ "windows" ] }, "windows_update": { "command": "powershell.exe -file C:/coolblue_checks/win_last_updated.ps1 -w 30 -c 60", "interval": 86400, "subscribers": [ "windows" ] } } }
  25. 25. SENSU CHECK DEFINITIONS. IIS /purchases Sensu Agent Sensu Server Rabbit MQ Add Check Definitions
  26. 26. STEP 3 - UPDATE THE CONFIG. Sensu Install { Ensure = "Present" DownloadURL = 'https://sensu.global.net/msi/sensu-0.24.0-1.msi' RabbitMQServer = 'coolblue-rabbitmq.internal.fake' Transport = 'rabbitmq' vHost = 'sensu' User = 'sensu' Port = '5672' Password = 'some_password' #Encrypted of course Subscription = 'windows_base','my_iis_app' }
  27. 27. STEP 4 - ADD APP SPECIFIC CHECKS. { "checks": { "iis_purchases": { "command": "check-http.rb http://localhost/purchases", "interval": 60, "playbook": "https://github.com/devTeam/readme.md", "subscribers": [ "my_iis_app" ] } } }
  28. 28. WHY MONITORING IS HARD. ▪Automation is an after thought ▪Forgetting to add new services ▪Checks are hard to write ▪Don’t want to do annoying things
  29. 29. THE SENSU SOCKET INPUT. IIS /purchases Sensu Agent Sensu Server Rabbit MQ 3030 TCP/ UDP My App
  30. 30. JUST SEND IN JSON. { "name": "ravendb_connectivity", "output": "Could not connect to RavenDB!", "status": 2, "team": "TeamName", "application": "My App", "playbook": "https://github.com/devTeam/readme.md" }
  31. 31. USING A DEAD-MAN SWITCH. { "name": "daily_sql_backup", "output": "Backup of SQL databases performed", "status": 0, "team": "OpsTeamName", "application": "sql_backups", "playbook": "https://github.com/opsTeam/readme.md", "ttl": 86400 }
  32. 32. MONITORING CAN BE EASY. ▪Automation is an after thought ▪Forgetting to add new services ▪Checks are hard to write ▪Don’t want to do annoying things
  33. 33. FIN. Matthew Hodgkins @matthodge hodgkins.io Flynn Bundy @bundyfx flynnbundy.com
  34. 34. BREAK TIME.
  35. 35. July 2016 | Behind the Scenes Flynn Bundy <@bundyfx> & Chris Thorp <@thorpgeek>
  36. 36. Challenges. ● OS Base Memory Usage ● Large HDD Usage ● Build Times for Servers ● Further increase the development speed of Coolblue
  37. 37. SAD (Server And a Desktop). Install web server feature
  38. 38. Server Core.
  39. 39. WE
  40. 40. Nano Server.
  41. 41. Nano Server Features.
  42. 42. Results. Server Core Nano Server Build Time: 6 min ~40 Seconds Start Time: 25 Seconds 2 Seconds Restart Time: 15 Seconds 1 Seconds Hard Drive Space used: 8 GB 600 MB Basic Memory Usage 250 MB 160 MB
  43. 43. Nano Server Thoughts. ● No 32 Bit Support ● MSI is gone ● Not all server core features available ● No Full .Net Framework Support
  44. 44. Challenges. ● OS base Memory Usage ● Large HDD Usage ● Build and Deployment Times for Servers ● Further increase the development speed of Coolblue
  45. 45. Summing up challenges. ● OS Base Memory Usage ● Large HDD Usage ● Build Deployment Times for Servers ● Further increase the development speed of Coolblue
  46. 46. FIN. Chris Thorp @thorpgeek Flynn Bundy @bundyfx flynnbundy.com
  47. 47. Events: coolblue.nl/behindthescenes DevBlog: devblog.coolblue.nl

×