Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

NuGet 3.0 - Transitioning from OData to JSON-LD


Published on

From Oredev 2014

The NuGet team has spent most of 2014 working on NuGet 3.0. You’ll get a tour of NuGet 3.0, its features, and the direction we’re headed. But then we’ll get to the fun stuff we can all learn from: the challenges we’ve faced, and how we’re overcoming them. You’ll hear chilling tales about how interfaces, LINQ, and general-purpose server APIs have caused us so much grief. But you’ll also see how we broke a cycle of endless design and started making progress when it seemed like none could be made, allowing us to transition from OData to JSON-LD.

Published in: Technology
  • Dating direct: ❶❶❶ ❶❶❶
    Are you sure you want to  Yes  No
    Your message goes here

NuGet 3.0 - Transitioning from OData to JSON-LD

  1. 1. NuGet 3.0 Transitioning from OData to JSON-LD Jeff Handley @JeffHandley
  2. 2. NuGet 3.0 New UI for Visual Studio • Lots of new UI features New client/server API • Built for availability, based on usage patterns Lots of code refactoring • Easier maintenance going forward
  3. 3. NuGet 3.0 Plans Aligned with Visual Studio “14” release cycle • Preview in the next release • Shipping for VS 2012 and VS 2013 • Not shipping for VS 2010 MSBuild will consume package references Solution Explorer integration Details at
  4. 4. Transitioning to the new client/server API Design retrospective
  5. 5. package repository remote server local folder package metadata from API unpacked into memory Visual Studio project system C# F# Windows Phone … file system /packages folder project system
  6. 6. Guiding principles BEFORE NUGET 3.0 Bolt onto Visual Studio Heavy use of abstractions Comprehensive unit tests Generic client/server API (OData)
  7. 7. Visual Studio bolt-on Use Visual Studio API Same result as manually referencing libraries Integrate with most Visual Studio project systems Support VS-integrated source control
  8. 8. Abstractions IPackageRepository • • Other remote servers • Folder on disk or network IPackage/IPackageMetadata • Remote • In-memory • On disk IPackageManager IProjectSystem • C# • F# • Windows Phone IFileSystem • Actual file system • Project systems • Source control LOOSE COUPLING & UNIT TESTING more than 80 other interfaces
  9. 9. Generic client/server API ANOTHER FORM OF LOOSE COUPLING Don’t couple the client to a server implementation Avoid versioning the server to add client features LINQ programming model for local and remote
  10. 10. NuGet.VisualStudio NuGet.Core PowerShell console package manager window VS IDE integration source control integration command extensions nuget.exe
  11. 11. NuGet.VisualStudio Visual Studio IDE integration • Output window logging • Settings (package sources) • Automatic package restore upon build • Garbage-collecting packages on VS restart • Extension update banner Visual Studio APIs • Project system integration • Actual package installation and uninstallation
  12. 12. NuGet.Core • Package authoring (including package analysis rules) • Package reading (OPC/Zip, data model w/validation, file conventions) • Semantic Version implementation • Target framework compatibility • Dependency resolver • Local and remote sources (API, HTTP, proxies, authentication) • NuGet.Config Processing (including hierarchical and extensible features) • 80+ interfaces
  13. 13. /revised-dumping-ordinance-allows.html
  14. 14. Classes fulfilling multiple interfaces Project systems are also file systems • Add/remove files • Enforces VS project system and source control integration /packages folder is also a package repository • GetPackages() • AddPackage() • RemovePackage() Extension methods on interfaces to simulate multiple inheritance • 17 “extensions” classes that act as base classes 33 “utility” classes to contain business logic
  15. 15. Extreme loose coupling Magic values trigger special handling several layers deep • FindPackage(id, version: null); Granular interfaces led to unmanageable parameter counts • ProjectSystemExtensions.DeleteFiles FxCop suppressions Events during operations allowed decoupled reactions • InstallWalker needs to report progress to the VS UI
  16. 16.
  17. 17. No change was easy • 18 classes touched adding HTTP headers for downloads • Dozens of classes affected to add a new field to nuspec • Mocks require more work than the actual implementation • Extension methods are not compatible with mocking •We became afraid to touch the dependency resolver
  18. 18. Big changes needed Replacing API v2 with API v3
  19. 19. SQL Azure database Azure Blob Storage Gallery / API v2 Azure web role (3 instances) EntityFramework code-first WCF Data Services and ASP.NET MVC and Razor OData Search and feed requests Content Delivery Network (CDN) Package downloads
  20. 20. Azure Blob Storage Search Service Azure worker role (3+ instances) Lucene.NET Full index loaded into memory Content Delivery Network (CDN) Metrics Service Azure website (3+ instances) SQL Azure database Search queries Package metadata requests Package downloads
  21. 21. JSON-LD { "@context": "", "@id": "", "name": "John Lennon", "born": "1940-10-09", "spouse": "" }
  22. 22. Creating the v3 client For the new UI
  23. 23. API v2 support The UI must support both v2 and v3 clients No clean surface area to re-implement • UI  install walker  API v2  install walker  API v2 • IQueryable<IPackage> exposed throughout code
  24. 24. Infinite surface area Finite scenarios
  25. 25. Scenarios and clients 1. Search 2. Install specific version 3. Install latest version 4. Install with dependencies 5. Install from cache 6. Install into another project 7. Check for updates 8. Update and update all 9. Restore 1. Package manager UI 2. PowerShell console 3. nuget.exe
  26. 26. p-5-reasons-youre-unemployed.html
  27. 27. API v3 search (feed) Search Service Azure worker role (3+ instances) Lucene.NET Full Index Loaded into Memory Gallery / API v2 Azure web role (3+ instances) Content Delivery Network (CDN) API v3 Package Metadata Package Downloads Metrics Service Azure website (3+ instances) API v2 OData SQL Azure database Azure Blob Storage
  28. 28. WCF Data Services to the rescue!? NuGet Core WCF Data Services API v2 Server OData XML WCF Data Services Entity Framework API v3 Services JSON-LD Convert JSON-LD to XML
  29. 29. .br/2013/10/bigode-grosso.html
  30. 30. Introducing NuGet.Client Handle the client/server scenarios Support API v2 and API v3 Scenario-focused object model Prevent new dependencies on NuGet.Core
  31. 31. Wrapping NuGet.Core Package Manager UI NuGet.Core NuGet.Client
  32. 32. Dealing with deep API v2 assumptions • Implemented v2 interfaces in v3 classes • Threw NotImplementedException on all members • Ran a scenario, identified code paths required • Identified more concrete API v3 requirements • Discovered data needed for each scenario • Influenced API v3 JSON-LD views
  33. 33. Shrinking NuGet.Core Package Manager UI NuGet.Core Package Authoring Package Reading Semantic Versioning Target FX Compat Dependency Resolver Local Package Sources NuGet Config Processing Project and NuGet.Core NuGet.Client NuGet.Client File Systems
  34. 34. Compromises API v3 isn’t completely “clean” yet • It exposes the API v2 interfaces Chatty • API v3 uses more HTTP requests than a pure v3 server/client would • API v2 interop performs more OData requests than we used to Out of scope • PowerShell console • nuget.exe
  35. 35. New guiding principles Our new rules
  36. 36. Guiding principle #1 Technology is not a substitute for understanding our scenarios.
  37. 37. Guiding principle #2 Don’t expose IQueryable on public API. * this applies to our RESTful services too
  38. 38. Guiding principle #3 Every client/server scenario needs an explicit endpoint.
  39. 39. Guiding principle #4 Shipping is more valuable than code purity.
  40. 40. Guiding principle #5 Don’t implement interfaces just because the members match.
  41. 41. Guiding principle #6 Side-effecting event handlers hurt in unpredictable ways.
  42. 42. Guiding principle #7 Avoid business logic in extension methods. They can’t be mocked.
  43. 43. Guiding principle #8 If adding new business logic is hard, refactor right away.
  44. 44. Thanks Come get NuGet stickers @jeffhandley | | |