Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

C# development workflow @ criteo

339 views

Published on

When you have more than one hundred developers working on a large C# codebase that spans among many git repositories, it is hard to keep an agile delivery pipeline. Criteo solved this by using a trunk based development approach. Trunk based development is powerful but comes at cost and must be implemented carefully to work properly. In this talk, we will present the workflow and tools we have put in place to allow our developers to efficiently and safely interact with our C# codebase.

Published in: Engineering
  • Be the first to comment

C# development workflow @ criteo

  1. 1. Copyright © 2016 Criteo C# development workflow @ Criteo Patrick Bruneton Lead of DevTools 2016-06-08
  2. 2. Copyright © 2016 Criteo C# at Criteo • 1000 .NET projects (C# and SQL server) • 100 Git repositories • 200 developers touching the C# codebase • Complex dependency graph Creating an efficient development workflow has been a challenge.
  3. 3. Copyright © 2016 Criteo Early days: open source model • Code splited into many C# Nugets • Each team responsible of a few Nugets (/ repositories) Team A Repo A A 1.0 build Repo B build B 1.1 1.2 Team BMerge Requests 1.1
  4. 4. Copyright © 2016 Criteo Issue #1: change propagation A B C • Changes in C are long to propagate • High “Commit to Prod” lag • Many versions of each component to maintain • High integration cost • Late feedback 1.1 1.3
  5. 5. Copyright © 2016 Criteo Issue #2: Nuget Dependency Hell A B D C 1.0 1.0 1.1 1.2
  6. 6. Copyright © 2016 Criteo New workflow Goal: integrate early (Continous Delivery) A B D C HEAD HEAD HEAD HEAD (for internal components)
  7. 7. Copyright © 2016 Criteo Trunk Based Development All commits in master branch No feature branches, no internal versioning Requires a good test coverage Benefits: • Very early integration of commits • Allows true Continuous Delivery Drawbacks: • A commit can break all builds • Harder to do large changes (you’re not allowed to break anything)
  8. 8. Copyright © 2016 Criteo Trunk Based Development with C# at Criteo Challenges • Multiple git repositories • Lots of projects • Use cases • Build on the CI servers • Build on the dev machines • Pre-merge tests
  9. 9. Copyright © 2016 Criteo Trunk Based Development with C# at Criteo Jenkins CBS+ + +
  10. 10. Copyright © 2016 Criteo Build in Jenkins: the MOAB A job that continuously builds all C# code from latest commits Lib1 Lib2 App MOAB #11 MOAB #10
  11. 11. Copyright © 2016 Criteo Build in Jenkins: the MOAB $ cbs checkout $ cbs build $ cbs test $ cbs export Full build: - Uses Microsoft.Build API - Manages the build queue itself - 24 builds in parallel - CreateHardLinksFor* = true A full build every 20 minutes
  12. 12. Copyright © 2016 Criteo Build in Jenkins: the MOAB $ cbs checkout $ cbs build $ cbs test $ cbs export Full build: Runs Unit tests in //
  13. 13. Copyright © 2016 Criteo Build in Jenkins: the MOAB $ cbs checkout $ cbs build $ cbs test $ cbs export Full build: List of SHA1 used Dependency graph Deployable packages Assemblies (dll + pdb) Filer
  14. 14. Copyright © 2016 Criteo Build on the Dev machine: using all sources $ cbs checkout app --with-dependencies $ ls app/app.csproj lib1/lib1.csproj lib2/lib2.csproj default.sln $ cbs build App Lib1 Lib2
  15. 15. Copyright © 2016 Criteo Build on the Dev machine: using MOAB snapshots $ cbs checkout app $ ls app/app.csproj default.sln bin/moab/lib1.dll bin/moab/lib2.dll <Reference Include="Lib1"> <Reference Include="Lib2"> <Import Project="cbs.targets"> App Lib1 Lib2
  16. 16. Copyright © 2016 Criteo Pre-submit tests cbs checkout changed-repo --with-clients cherry-pick change cbs build cbs test [--with-clients] M O A B
  17. 17. Copyright © 2016 Criteo Conclusion • Trunk Based Development is powerful • It’s a pre-requisite for Continuous Delivery, which allows agility • TBD can be implemented with C#, but we needed some tooling
  18. 18. Copyright © 2016 Criteo Questions ?

×