2013 Perforce Collaboration Tour - MathWorks


Published on

By Marc Ullman, Senior Systems Architect at MathWorks

See how MathWorks is using version management to increase the speed and predictability of their product releases.

Published in: Technology
  • Be the first to comment

  • Be the first to like this

No Downloads
Total Views
On Slideshare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

2013 Perforce Collaboration Tour - MathWorks

  1. 1. MathWorks & PERFORCE! New York and Boston Roadshow Nov 5th & 7th 2013 Marc Ullman / Senior Systems Architect / MathWorks, Inc. © 2013 MathWorks, Inc. 1
  2. 2. MATHWORKS OVERVIEW §  §  MathWorks is a 2000+ person company dedicated to accelerating the pace of engineering and science We have ~90 products based upon our core platforms MATLAB, the language of technical computing –  Simulink, for simulation and model-based design –  §  They serve a wide array of markets –  §  §  Aero, Auto, Bio, Communications, Finance, Medical, etc. Full product family is released twice a year from a unified code base Products are used to develop safety-critical systems –  Quality and correctness are paramount 2
  3. 3. TECHNICAL CHALLENGES §  Managing an almost 1 million file code base §  Partitioning it into usable subsets §  Integrating changes from ~1000 developers §  Supporting multiple platforms (Windows, Mac, Linux) §  Hosting development teams on three continents §  Dealing with bugs and scalability issues in underlying infrastructure Operating systems (e.g., Windows) –  Tools (e.g., Perforce, Visual Studio) –  3
  4. 4. MATHWORKS DEVELOPMENT ENVIRONMENT §  Developed in-house continuous integration system almost 20 years ago §  Originally based on RCS, then CVS §  Built significant infrastructure on top of CVS to support –  Change sets –  Merge history tracking –  Lightweight (sparse) private developer branches –  Post-commit (a.k.a. pre-flight) queue-based operation –  Hierarchical (promotion-based) code flow 4
  5. 5. NEED FOR A NEW SCM SYSTEM CVS-based System Suffered Significant Drawbacks §  Poor performance –  –  §  Poor reliability –  §  Code to augment feature set was maintenance headache Poor usability –  §  Branching full code base took ~24 hours—hence done rarely Repository operations painfully slow for remote developers Lacked GUI and graphical merge tools Poor scalability –  Used overly strict partitioning to keep branches small Net Effect: Lost productivity due to tooling limitations 5
  6. 6. SEARCH FOR A NEW SCM SYSTEM Hard to Find a Solution That Met All Our Requirements §  Must-haves –  Atomic change sets –  Merge history tracking –  Flexible partitioning of large code base –  Hierarchical branching –  Rename support –  Private developer branches §  Most tools are not designed to handle a unified code base the size of ours –  Accurev and Git have very efficient branching but lack good mechanisms for partitioning a large codebase §  e.g., “Repo” used with Git for Android development 6
  7. 7. PERFORCE ADDRESSED 3 OF OUR “MUST HAVES” ü  Atomic change sets ü  Merge history tracking ü  Scalability with partitioning –  Via client and branch views But it lacked other key features like: –  Hierarchical branching –  Rename (with live/dead merging and resolve) –  Lightweight (sparse) private developer branches 7
  8. 8. PERFORCE REDUX—STREAMS With streams, Perforce became a better fit for our needs §  Streams are true hierarchical branches –  Stream (branch) hierarchies are clearly defined and captured within Perforce rather than via customer conventions –  Their evolution is recorded as part of depot history –  With StreamAtChange, stream shape can be referenced in a time-consistent manner §  Integration engine was beefed up to handle move/ rename §  Task streams approximate light-weight branches 8
  9. 9. EXCEPT THAT… The out-of-the box streams inheritance model of did not mesh well with our desired workflows We wanted: §  §  §  Wide-open client views The ability to widen a branch (stream) from below Stream (shape) changes to be atomic with content changes 9
  10. 10. COLLABORATION WAS THE KEY TO SUCCESS Perforce helped us come up with an elegant solution §  Virtual streams are used in conjunction with the Perforce broker to narrow p4 sync and merge / copy / integrate +  Permits users to operate with wide open views +  Transparent to use of CLI, P4V, and plug-ins §  A change trigger is used to modify behavior of p4 submit +  Limits scope of submits to active components +  Updates the virtual stream paths if pending change includes modified component definitions +  Results in changes to the stream content and shape being atomic But Collaboration didn’t stop there—it went both ways §  §  We helped Perforce refine behavior of StreamAtChange We provided use cases that Perforce incorporated into their regression test suite 10
  11. 11. BENEFITS WE SEE WITH PERFORCE STREAMS §  Simple user model Developers no longer edit client views or branch specs –  Stream specs are updated automatically by submit trigger –  Perforce generates client and branch views as needed –  §  Dynamically partitioned codebase –  §  Developers can define the width (shape) of a branch by specifying the set of components they need Improved productivity Removed major barrier to using narrow branches –  Developers and release engineers no longer struggle to make componentization changes –  –  Able to largely automate our hierarchical code motion (what Perforce likes to call merge down/copy up) 11
  12. 12. LOOKING AHEAD §  Continuing to work closely with Perforce §  Considering use of Task Streams §  Interested in gaining better insight into development projects and processes –  Excited to try Perforce Insights 12
  13. 13. Questions? © 2013 MathWorks, Inc. 13
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.