Sergey Dzyuban ''Cooking the Cake for Nuget packages"
•Download as PPTX, PDF•
0 likes•476 views
How to manage CI\CD process for many git repositories (400+), when each of repository needs to be published as NuGet package? Combining Cake and Jenkins features allow building the generic pipeline for packages creation, versioning and publish. Based on C# code, source controlled and compiled in runtime, Cake gives perfect flexibility to build advanced CI\CD process. Large scale also required some tricks inside CI infrastructure and related services.
Recommended
Recommended
More Related Content
What's hot
What's hot (20)
Similar to Sergey Dzyuban ''Cooking the Cake for Nuget packages"
Similar to Sergey Dzyuban ''Cooking the Cake for Nuget packages" (20)
More from Fwdays
More from Fwdays (20)
Recently uploaded
Recently uploaded (20)
Sergey Dzyuban ''Cooking the Cake for Nuget packages"
- 1. Sergey Dzyuban, SBTech Cooking the Cake for Nuget packages
- 3. Consider regular repository CICD DevOps Repository build tests deploy
- 4. … but what if company has more repos ?
- 6. … much more
- 9. … ok, actually 400+ repos and most of them are .NET solutions
- 10. The common issues • Lot of repositories, managed separately • Repository-specific logic placed in each repository as scrips • 1++ jobs in Jenkins to process each repository OR none jobs at all • Common logic (unit test, nuget publish) need to be applied per each repository … as result • 90 % of all DevOps resources are taken to fix existing jobs and adding new one • Impossible to make common changes (adding Sonar, notifications) for all jobs at once • “Lost” and “phantom” jobs: build_xxx_repo_dev_temp_1
- 11. Analyzing existing repositories Common Unique C# Solutions count File structure Scripts (even language) VS solution Projects type Git controlled Deploy type • Manage CICD logic in version control system • Add ability to minimize Jenkins jobs number by adding common jobs, which can process multiple repositories • Allow to easy use CICD scripts outside Jenkins • Allow to make centralized changes in whole CICD process as simply as possible. • Add support for create Nuget Packages
- 12. The main idea – Repository as Interface Each repository should contain some part of internal scripts, which will have common interface to interact with repository in same way. Common Interface members • REPO HELP • REPO BUILD • REPO UNITTEST • REPO PACKAGE REPOSITORY help build test pack
- 13. How it was
- 14. How it should be build test deploy
- 15. Part 2 Cake
- 17. Why we are using cake ? • Cake is not Powershell • Many build-in integrations • Cake is C# and Roslyn • Cake can use PowerShell • Cake can use tools and add-ins as NuGet packages • Cake can reference another *.cake as Tasks • Simpsons already did it
- 18. Adding cake to existing repository Repository build.cake src cake build.ps1 repo.sln tools Tasks deploy.cake tests.cake
- 19. Best practices for single repository • Place cake in separate folder • Use separate cake tasks • Powershell is the best start
- 20. … but take a look at real call example ./cake/build.ps1 -ScriptArgs “- solution='.srclog4net.sln' -source='src'" -Target build -Verbosity Verbose
- 21. … but take a look at real call example ./cake/build.ps1 -ScriptArgs “- solution='.srclog4net.sln' -source='src'" -Target build -Verbosity Verbose
- 23. Adding batch file Repository build.cake src cake build.ps1 repo.sln tools Tasks deploy.cake tests.cake cake.bat Batch script: • build.ps1 check-download • define solution file(s) name • hides default settings • makes call clear REPOSITORY help build test pack
- 24. Adding batch file Repository build.cake src cake build.ps1 repo.sln tools Tasks deploy.cake tests.cake cake.bat Batch script: • build.ps1 check-download • define solution file(s) name • hides default settings • makes call clear REPOSITORY help build test pack
- 25. Adding batch file Repository build.cake src cake build.ps1 repo.sln tools Tasks deploy.cake tests.cake cake.bat Batch script: • build.ps1 check-download • define solution file(s) name • hides default settings • makes call clear REPOSITORY help build test pack cake build cake cake test cake package
- 26. What about another repository ? … we have lot of them
- 27. Using contract Repository build.cake src cake build.ps1 repo.sln tools Tasks deploy.cake tests.cake cake.bat F# build.cake src cake build.ps1 repo.sln tools Tasks fsharp.cake gatling.cake cake.bat > cake > cake build > cake test > cake package
- 28. Adding Jenkins jobs build test deploy git clone cake build notify
- 29. Some issues on this stage • Lot of cake tasks are duplicated • Hard to make changes in cake tasks • Adding new repo by copy-paste • Bug fixes in cake requires changes of all repos
- 30. Adding Master repository Master Repo build.cake cake build.ps1 Repository custom.cake src cake repo.sln cake.bat build.cake build.ps1 copy tools
- 31. Master Repository allows • Keep all logic in one place • Fix bugs and implement features quickly • Avoid code duplication • Decrease cake script size committed in each repo • Easy to add a new repository
- 32. Deep dive into the build.cake
- 33. Dependency based programming package init clean set new version build pack publish
- 34. Dependency based programming package init clean set new version build pack publish Task("package") .IsDependentOn("init") .IsDependentOn("clean") .IsDependentOn("setnewversion") .IsDependentOn("build") .IsDependentOn("pack") .IsDependentOn("publish") .Does(() => { });
- 35. Dependency based programming package init clean set new version build pack publish before build msbuild unit testsonar begin sonar end clean nuget restore nunit ncover
- 36. … what about customization? Master Repo build.cake cake build.ps1 Repository custom.cake src cake repo.sln cake.bat
- 37. Tasks inheritance base.package init clean set new version build pack publish custom.cake package build.cake base.init base.clean base.version base.build base.pack base.publish
- 38. Tasks inheritance base.package init clean set new version build pack publish custom.cake package build.cake base.init base.clean base.version base.build base.pack base.publish Task("Information") .IsDependentOn("Base.Information") .Does(() => {}); Task("Clean") .IsDependentOn("Base.Clean") .Does(() => {}); Task("Build") .IsDependentOn("Base.Build") .Does(() => {}); Task("BeforeBuild") .Does(() => {}); Task("UnitTest") .IsDependentOn("Base.UnitTest") .Does(() => {});
- 39. Tasks inheritance base.package init clean set new version build pack publish custom.cake package build.cake base.init base.clean base.version base.build base.pack base.publish
- 40. Tasks inheritance base.package init clean set new version build pack publish custom.cake package build.cake base.init base.clean base.version base.build base.pack base.publish Task("Information") .IsDependentOn("Base.Information") .Does(() => {}); Task("Clean") //.IsDependentOn("Base.Clean") .Does(() => { StartProcess("rm -rf /"); }); Task("Build") .IsDependentOn("Base.Build") .Does(() => {}); Task("BeforeBuild") .Does(() => {}); Task("UnitTest") .IsDependentOn("Base.UnitTest") .Does(() => {});
- 41. Out of the box features We provide some list of features, available for all repositories by default
- 42. Build-in features • Run UnitTests • NUnit • XUnit • Code coverage • Sonar code analysis • Build .NET solutions • .NET Framework • .NET Core • Restore nuget packages • Create nuget packages • By csproj • By nuspec • By core csproj • Make commitstags to git • Publish artifacts
- 43. How we maintain scripts 5% 95% CODE custom.cake build.cake 5% 55% 40% CODE custom.cake build.cake addons
- 44. How we maintain scripts Master Repo build.cake cake build.ps1 Repository custom.cake src cake repo.sln cake.bat build.cake build.ps1 tools src addon Class1.cs addon.dll
- 45. How we test scripts
- 46. How we test scripts Master repo master Repo 0 Repo 60 master master cake
- 47. How we test scripts Master repo master Repo 0 Repo 60 master master cake dev feature/_temp F# Repo master dev
- 48. How we test scripts Master repo master Repo 0 Repo 60 master master cake dev feature/_temp F# Repo master dev
- 52. Jenkins by itself is good enough … but let’s add some additional features for static code analysis
- 55. Slack for build notifications
- 58. Elastic for great CI telemetry In high scale CI process single build makes no lot of sense, but Jenkins telemetry allows to show the whole picture.
- 61. Custom Dashboard … allows to aggregate events from all system parts and visualize status snapshot, actual on the current moment of time. It provide only the required information from large amount of data.
- 65. Hosting for Dashboard Having DCOS cluster setup and running, it’s easy to add and maintain all required components.
- 67. Q & A