By Marc Ullman, Senior Systems Architect at MathWorks See how MathWorks is using version management to increase the speed and predictability of their product releases.

1. MathWorks &
PERFORCE!
New York and Boston Roadshow
Nov 5th & 7th 2013
Marc Ullman / Senior Systems Architect / MathWorks, Inc.
© 2013 MathWorks, Inc.
1

2. MATHWORKS OVERVIEW
§
§
MathWorks is a 2000+ person company dedicated to
accelerating the pace of engineering and science
We have ~90 products based upon our
core platforms
MATLAB, the language of technical computing
– Simulink, for simulation and model-based design
–
§
They serve a wide array of markets
–
§
§
Aero, Auto, Bio, Communications,
Finance, Medical, etc.
Full product family is released twice a year from a
unified code base
Products are used to develop safety-critical systems
–
Quality and correctness are paramount
2

3. TECHNICAL CHALLENGES
§
Managing an almost
1 million file code base
§
Partitioning it into usable subsets
§
Integrating changes from ~1000 developers
§
Supporting multiple platforms
(Windows, Mac, Linux)
§
Hosting development teams on
three continents
§
Dealing with bugs and scalability issues
in underlying infrastructure
Operating systems (e.g., Windows)
– Tools (e.g., Perforce, Visual Studio)
–
3

4. MATHWORKS DEVELOPMENT ENVIRONMENT
§
Developed in-house continuous
integration system almost 20 years ago
§
Originally based on RCS, then CVS
§
Built significant infrastructure on top
of CVS to support
– Change sets
– Merge history tracking
– Lightweight (sparse) private
developer branches
– Post-commit (a.k.a. pre-flight)
queue-based operation
– Hierarchical (promotion-based) code flow
4

5. NEED FOR A NEW SCM SYSTEM
CVS-based System Suffered
Significant Drawbacks
§
Poor performance
–
–
§
Poor reliability
–
§
Code to augment feature set was maintenance headache
Poor usability
–
§
Branching full code base took ~24 hours—hence
done rarely
Repository operations painfully slow for remote developers
Lacked GUI and graphical merge tools
Poor scalability
–
Used overly strict partitioning to keep branches small
Net Effect: Lost productivity due to tooling limitations
5

6. SEARCH FOR A NEW SCM SYSTEM
Hard to Find a Solution That Met All Our Requirements
§
Must-haves
– Atomic change sets
– Merge history tracking
– Flexible partitioning of large code base
– Hierarchical branching
– Rename support
– Private developer branches
§
Most tools are not designed to handle a unified code base the size of ours
– Accurev and Git have very efficient branching but lack good mechanisms for
partitioning a large codebase
§
e.g., “Repo” used with Git for Android development
6

7. PERFORCE ADDRESSED 3 OF OUR “MUST HAVES”
ü Atomic change sets
ü Merge history tracking
ü Scalability with partitioning
– Via client and branch views
But it lacked other key features like:
– Hierarchical branching
– Rename (with live/dead merging and resolve)
– Lightweight (sparse) private developer branches
7

8. PERFORCE REDUX—STREAMS
With streams, Perforce became
a better fit for our needs
§
Streams are true hierarchical branches
– Stream (branch) hierarchies are clearly defined and
captured within Perforce rather than via customer
conventions
– Their evolution is recorded as part of depot history
– With StreamAtChange, stream shape can be referenced
in a time-consistent manner
§
Integration engine was beefed up to handle move/
rename
§
Task streams approximate light-weight branches
8

9. EXCEPT THAT…
The out-of-the box streams inheritance
model of did not mesh well with our
desired workflows
We wanted:
§
§
§
Wide-open client views
The ability to widen a branch (stream)
from below
Stream (shape) changes to be atomic with
content changes
9

10. COLLABORATION WAS THE KEY TO SUCCESS
Perforce helped us come up with an elegant solution
§
Virtual streams are used in conjunction with the Perforce
broker to narrow p4 sync and merge / copy / integrate
+ Permits users to operate with wide open views
+ Transparent to use of CLI, P4V, and plug-ins
§
A change trigger is used to modify behavior of p4 submit
+ Limits scope of submits to active components
+ Updates the virtual stream paths if pending change includes modified
component definitions
+ Results in changes to the stream content and shape being atomic
But Collaboration didn’t stop there—it went both ways
§
§
We helped Perforce refine behavior of StreamAtChange
We provided use cases that Perforce incorporated into their
regression test suite
10

11. BENEFITS WE SEE WITH PERFORCE STREAMS
§
Simple user model
Developers no longer edit client views or branch specs
– Stream specs are updated automatically by submit trigger
– Perforce generates client and branch views as needed
–
§
Dynamically partitioned codebase
–
§
Developers can define the width (shape) of a branch by
specifying the set of components they need
Improved productivity
Removed major barrier to using narrow branches
– Developers and release engineers no longer struggle to
make componentization changes
–
–
Able to largely automate our hierarchical code motion
(what Perforce likes to call merge down/copy up)
11

12. LOOKING AHEAD
§
Continuing to work closely with Perforce
§
Considering use of Task Streams
§
Interested in gaining better insight into
development projects and processes
– Excited to try Perforce Insights
12

13. Questions?
© 2013 MathWorks, Inc.
13