Developing NuGet
Upcoming SlideShare
Loading in...5
×
 

Developing NuGet

on

  • 523 views

Learn about how the core NuGet team the NuGet Client project. The project has challenges related to: ...

Learn about how the core NuGet team the NuGet Client project. The project has challenges related to:

- Open Source
- Hard Deadlines
- Very Large Backlog
- Many Conflicting Requirements
- Git

We'll talk about our strategies for:

- Issue tracking - Branching
- Continuous integration
- Testing
- Release sign-off

Statistics

Views

Total Views
523
Views on SlideShare
512
Embed Views
11

Actions

Likes
0
Downloads
3
Comments
0

1 Embed 11

https://twitter.com 11

Accessibility

Categories

Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

    Developing NuGet Developing NuGet Presentation Transcript

    • Developing NuGet JEFF HANDLEY | DEVELOPMENT LEAD | NUGET | MICROSOFT @JEFFHANDLEY JEFFHANDLEY.COM 1
    • Airborne (1993) 2SOURCE: WWW.IMDB.COM
    • 3
    • 4
    • 5
    • NuGet Landscape WHAT MAKES UP THE NUGET PROJECT 6
    • NuGet Project Open Source (owned by Outercurve Foundation) ◦ Releases shipped by Microsoft Included in all Visual Studio installations Millions of users (some knowingly, some not) around the world Dozens of high-profile dependent projects at Microsoft International contributors ◦ Only a few external contributors are in the US Very large backlog of issues Conflicting requirements Backward compatibility Mixture of hard and soft deadlines Extremely costly test passes 7
    • Outercurve and Microsoft NUGET CLIENT Outercurve’s NuGet Project Microsoft’s NuGet-Based Package Manager In the Box with Visual Studio NUGET GALLERY Outercurve’s NuGet Gallery Project www.nuget.org Azure Hosting provided by Microsoft Operated by Microsoft on behalf of Outercurve 8
    • NuGet’s Core Team NUGET CLIENT Visual Studio Extension WebMatrix Extension NuGet.exe Command-Line NuGet Packages 3 Developers 2 Testers NUGET GALLERY www.nuget.org staging.nuget.org docs.nuget.org build.nuget.org 3 Developers 2 Testers Always 1 member on ops/support rotation 9
    • NuGet’s Core Team NUGET CLIENT Visual Studio Extension WebMatrix Extension NuGet.exe Command-Line NuGet Packages 3 Developers 2 Testers NUGET GALLERY www.nuget.org staging.nuget.org docs.nuget.org build.nuget.org 3 Developers 2 Testers Always 1 member on ops/support rotation 10
    • Release Management NUGET CLIENT 11
    • NuGet Releases New release every 10 weeks since NuGet 1.5 12 2/ 12/ 2011 1.1 3/ 30/ 2011 1.2 4/ 25/ 2011 1.3 6/ 17/ 2011 1.4 8/ 30/ 2011 1.5 12/ 13/ 2011 1.6 4/ 4/ 2012 1.7 5/ 23/ 2012 1.8 6/ 19/ 2012 2.0 10/ 4/ 2012 2.1 12/ 12/ 2012 2.2 2/ 15/ 2013 2.2.1 4/ 25/ 2013 2.5 6/ 26/ 2013 2.6 8/ 22/ 2013 2.7 10/ 7/ 2013 2.7.1 1/ 14/ 2011 1.0
    • NuGet in Visual Studio Releases 13 2/ 12/ 2011 1.1 3/ 30/ 2011 1.2 4/ 25/ 2011 1.3 6/ 17/ 2011 1.4 8/ 30/ 2011 1.5 12/ 13/ 2011 1.6 Introduced with ASP.NET MVC3 and WebPages v1 4/ 4/ 2012 1.7 5/ 23/ 2012 1.8 6/ 19/ 2012 2.0 10/ 4/ 2012 2.1 12/ 12/ 2012 2.2 2/ 15/ 2013 2.2.1 4/ 25/ 2013 2.5 6/ 26/ 2013 2.6 8/ 22/ 2012 Visual Studio 2012 RTM 4/ 4/ 2013 Visual Studio 2012 Update 2 6/ 26/ 2013 Visual Studio 2013 Preview 9/ 9/ 2013 Visual Studio 2013 RC 1/ 14/ 2011 1.0 5/ 31/ 2012 Visual Studio 2012 RC 2/ 28/ 2012 Visual Studio 2012 Beta 8/ 22/ 2013 2.7 10/ 7/ 2013 2.7.1 Visual Studio 2013 RTM
    • Release Tiers CI build from Outercurve Foundation always available at http://build.nuget.org ◦ Usually 100 or fewer users ◦ Build created from every commit Beta/RC builds from Microsoft occasionally available at http://nuget.codeplex.com ◦ 1,000 – 2,500 users ◦ Published after we hit code-complete ◦ Real-world testing of newly introduced features or significant design changes Final builds from Microsoft published to the Visual Studio Extension Gallery ◦ 500,000 – 750,000 users ◦ Released after our test team signs off, a few weeks before Visual Studio locks down for its next release ◦ Last stage real-world testing from users that are knowingly using NuGet and are a bit more forgiving In the box with Visual Studio ◦ Millions of users ◦ Many of these users don’t know what NuGet is and will file Visual Studio bugs if we break them 14
    • Backlog CodePlex: 858 open issues as of 10/4/2013 Microsoft Internal Issues ◦ TFS: Fluctuates from 0 to 25 ◦ Dr. Watson reports ◦ Microsoft Connect ◦ Microsoft shipping requirements ◦ Future Visual Studio release issues Strategic Product Features ◦ Schedule alignment for release within Visual Studio and Visual Studio Update 15
    • Strategic Product Features Package Manager for the Microsoft Development Platform ◦ Not just ASP.NET ◦ Not just .NET ◦ Includes C++, Windows Store, and Windows Phone ◦ Cross-platform support with Command-Line, Xamarin Studio, MonoDevelop, and SharpDevelop Improving Package Discovery (through collaboration with the NuGet Gallery crew) Avoiding Package Version Hell Platform Multi-Targeting and Re-Targeting New Usage Workflows for Improved Application Maintenance and Ecosystem Growth Helping Improve Gallery Reliability 16
    • Triaging Issues 1. Is it a high-priority issue (crash, hang, regression)? ◦ Include it in the current release 2. Is the bug fix or feature needed for the current release? ◦ Include it in the current release 3. Is it a high value item that fits in the next release? ◦ Include it in the next release 4. Do we agree with the concept? ◦ Put it in the “Up For Grabs” release ◦ Ideally picked up by the community ◦ High-voted items are reviewed when planning releases and some items are pulled in 5. Do we disagree with the concept? ◦ Close it 17
    • Priorities 1. Strategic roadmap that integrates with the Microsoft development platform ◦ Features planned for 2-3 releases at a time ◦ Align with Visual Studio releases: Preview, Beta, RC, RTM, Visual Studio Update 2. Community-reported high-value bug fixes or features ◦ Including those that come with external pull requests 3. Highly-voted community feature requests 4. Community-reported high-value bug fixes 5. Microsoft partner team bug reports 6. Microsoft partner team feature requests 7. Community-reported low-value bug fixes 18
    • Philosophy Small and easy bug fixes are low priority ◦ Otherwise, we’d spend 100% of our time making small changes that affect only a handful of users Don’t prematurely act upon cool ideas ◦ Does the idea spark new strategic concepts? ◦ Do other new ideas relate? ◦ Can the feature be implemented with backwards compatibility? ◦ Could this feature become obsolete based on other potential strategic directions? Favor community-requested features over partner team features ◦ Avoids scenario-specific features that conflict with broad usage ◦ Encourages partner teams to find workarounds with existing features ◦ Those workarounds often surface pain points others might also be experiencing We’d rather delight 1,000,000 external users than gain 1 more internal partner team 19
    • Conflicting Requirements Microsoft’s Customers vs. NuGet’s Ecosystem ◦ Group Policy control over package sources Bold New Features vs. Backwards Compatibility ◦ Package Restore ◦ Build-time reference resolution NuGet User Experience vs. Microsoft’s shipping requirements ◦ Online consent ◦ Loading only Microsoft-signed assemblies ◦ Modified content files Favor user desires as much as possible Work within the constraints of Microsoft’s shipping requirements Negotiate and compromise with privacy, security, and legal representatives to arrive at acceptable implementations 20
    • Source Control NUGET CLIENT 21
    • master merge Feature A Feature B Branch: 2.7 Branch: 2.7.1 mergemerge Branch: 2.8 Tag: 2.7 Tag: 2.7.1 Git Branch per feature Branch per version Tag for release 22
    • Continuous Integration Outercurve Builds ◦ TeamCity – running on Virtual Machines in Windows Azure ◦ 1 VM build agent with .NET 4.0 for building the VS Extension for VS 2010 and VS 2012 ◦ 1 VM build agent with .NET 4.5 for building the VS Extension for VS 2013 ◦ Hosted at http://build.nuget.org ◦ Produces non-localized, unsigned builds with Outercurve branding and licensing ◦ “Developer Branches” build configuration builds all “dev-” prefixed branches Microsoft Builds ◦ TeamCity – running on Virtual Machines in a group lab ◦ 1 VM build agent with .NET 4.0 for building the VS Extension for VS 2010 and VS 2012 ◦ 1 VM build agent with .NET 4.5 for building the VS Extension for VS 2013 ◦ Only accessible on the corporate network ◦ Produces localized, signed builds with Microsoft branding and licensing 23
    • Going Dark Microsoft-confidential work cannot go to CodePlex ◦ Integration with new versions of Visual Studio ◦ Support for new platforms (e.g. Windows Phone 8) Team Foundation Service private project ◦ Fork the NuGet Git repo into that project ◦ Move confidential development into that fork ◦ Keep all other development on CodePlex Merge commits into TFS daily Configure our Internal Builds to use the TFS fork instead of CodePlex 24
    • Testing NUGET CLIENT 25
    • Release Timing Release sign-off takes about 4 weeks after code-complete ◦ 40% of our 10-week release cadence is spent testing and fixing bugs Snap releases to the Visual Studio schedule Release to the gallery first, and then integrate into Visual Studio’s build Between Visual Studio Beta and RTM, we have 2 or 3 tight quality-driven releases ◦ Meet Microsoft’s release requirements ◦ Address any partner team blocking issues ◦ Focus on extremely broad use of the product After Visual Studio hits RTM, we go big with features and aim for a 3-month release 26
    • Test Matrix Dimensions Visual Studio SKUS ◦ VS 2010 ◦ Express for Web ◦ Express for Phone ◦ Pro+ ◦ VS 2012+ - All SKUs Project Types ◦ Virtually all Microsoft Project Types ◦ Lightswitch ◦ Windows Store ◦ Windows Phone ◦ Portable Class Libraries Languages: C#, VB, F#, JavaScript Operating Systems ◦ Windows XP ◦ Windows Server 2003+ ◦ Windows Vista and Windows 7 ◦ Windows 8 ◦ Windows 8.1 Different types of NuGet Packages Different types of NuGet Package Sources ◦ Gallery ◦ NuGet.Server ◦ Local Disk ◦ UNC Share Source Control Integration ◦ TFS ◦ Git, Mercurial ◦ None  NuGet Clients ◦ VS Extension ◦ WebMatrix Extension ◦ NuGet.exe Command-Line Plus, all of the combinations of actual features 27
    • Achieving Release Sign-Off Unit Tests managed and run by developers End-to-end Functional tests managed and run by developers and through automation ◦ Based on a PowerShell test system included within our project ◦ Runs within Visual Studio, automating VS through the DTE QA Tests for interactive testing ◦ Mostly automated, but exploratory testing always finds more bugs ◦ Run across a farm of machines in our group’s test lab, covering the machine configuration matrix ◦ A representative set of combinations give us confidence ◦ Uses an internal VS automation library to simulate user actions in VS Group-wide bug bashes Company-wide partner team bug bashes Public Beta testing from CodePlex builds Reach 0 bugs in CodePlex for the current release 28
    • Throughput 109 Weeks • Total time working on those 10 releases 10 Releases • 816 work items have been completed 3 Developers • Averaging 27 work items per release 2 Testers • Each testing 3.7 bug fixes or features per week 29
    • Success Never defined by throughput No regressions No minor updates necessary Users delighted by new features Took steps toward our strategic roadmap Addressed highly-voted community bugs and features Accepted a level of failure in smaller user base releases 30
    • Success Rate NuGet 1.6 ◦ Required a 1.6.1 update ◦ Regressions ◦ Missed bugs in new features NuGet 1.7 ◦ Success NuGet 1.8 ◦ Success NuGet 2.0 ◦ Success NuGet 2.1 ◦ Success NuGet 2.2 ◦ Success NuGet 2.5 ◦ Success NuGet 2.6 ◦ Success NuGet 2.7 ◦ Required a 2.7.1 update ◦ Regression ◦ Missed bugs in new features 31
    • Tiered Releases are Key Our tiered release approach worked for 2.7 2.7.1 update in place in time for integrating into Visual Studio Thousands of users affected instead of millions As Visual Studio’s broad user base jumps from NuGet 2.2 to NuGet 2.7, they’ll have no idea we ever introduced regressions You wouldn’t have noticed the camera in Airborne if I hadn’t pointed it out to you 32
    • Q & A Get a NuGet Sticker! Twitter: @jeffhandley Website: jeffhandley.com 33