This document discusses edge cases and challenges that can occur when merging code changes between component-based software development streams. It outlines several types of complex merge scenarios, such as renames that cross stream views and "shadowed deletes" not caught by integration tools. The key lessons are to consider the big picture problem rather than symptoms, have a simple managed workflow, and continuously test upgrades. An ideal solution would involve source control at the file object level rather than filenames to more easily handle renames and component changes.

1. Outsmarting Merge Edge Cases in
Component Based Development
Mike Hobbs
Software Engineer

2. 2
Overview
 Background
 Edge Cases, Pitfalls, and Solutions
 Conclusion / Q&A

3. Background

4. 4
Background – MathWorks
 We are a 3500+ person company dedicated to accelerating
the pace of engineering and science
 We have ~90 products based upon our
core platforms
• MATLAB – The Language of Technical Computing
• Simulink – Simulation and Model-Based Design

5. 5
Background - Definitions
 Product
• MATLAB
• Parallel Computing Toolbox
• Signal Processing Toolbox
 Component
• Set of strongly-related files

6. 6
Background - Componentization
 ~90 products
• 5,000+ components, acyclical dependencies
• 100,000s of tests and test points
• ~36 hours to build and test from the ground up
• Managing an over 1 million file code base
 Benefits of componentization
• Develop in isolation
• Identify problems ahead of time
• Run only a subset of building and testing

7. 7
Background – Solution Chosen
 Single stream per product, mostly
• //mw/Bparallel
• CTB (“Components to Build”) list, can change
• Virtual stream enforcing a view
 How to manage getting changes from one stream to
another?
• Tools (What I work on, automatically merging between streams)
 More background: “Moving 1000 Users and 100 Branches
Into Streams” – John LoVerso, MERGE 2014

8. 8
Background - Architectural Issues
 Renames with regards to crossing the
stream view boundary
 Submitting both a delete and an add onto
the same filename
 Filename-based versus file object-based source control
system

9. 9
Background – Filename / File Object
 Filename
• //mw/productA/file1
• //mw/productA/file2

10. 10
Background – Filename / File Object
 Filename
• //mw/productA/file1
• //mw/productA/file2
 File Object
• p4 add file1
• p4 submit

11. Edge Cases

12. 12
Edge Cases – Complex Merge
 What is it?
• Multiple changes to merge to a destination
• Cannot be represented in one single change
 Classic example
 Easy solution, no?
• “Just merge the left half first.”
 Can’t ask the developer to manually apply the solution to
each issue we find

13. 13
Edge Cases – Complex Merge
 How do you identify one?
• First pitfall: Problem definition
 Incorrect: Counting endpoints
• Correct in many cases (95% or so)
• False positives and false negatives

14. 14
Edge Cases – Complex Merge
 How do you identify one?
• First pitfall: Problem definition
 Incorrect: Set of changes that cannot
be represented in just one change
• How do you know you are correct?

15. 15
Edge Cases – Complex Merge
 How do you identify one?
• First pitfall: Problem definition
 Correct: Identify the actual, big-ticket item
• For us, this was the fact that people were renaming files often
• Much easier to identify “filenames which need both a delete, and then an
add applied to them” (i.e. complex merge on filename A)

16. 16
Edge Cases – Complex Merge
 How do you resolve one?
• Second pitfall: Merging at an earlier change
• Third pitfall: Asking the user to do it
 Incorrect: Complex merge identified at
c123456, so run the command at c123455
• Or at c123000, and then at c123100,
and then at c123150, and then…
• Manually resolve!

17. 17
Edge Cases – Complex Merge
 How do you resolve one?
• Second pitfall: Merging at an earlier change
• Third pitfall: Asking the user to do it
 Correct: Automatically submit

18. 18
Edge Cases – Deadwood
 What is it?
• Stream has fileA in view
• Stream CTB changes, fileA no longer in view
• fileA continues to exist, no changes are received
Files in view
“Dead wood”
Files out of view

19. 19
Edge Cases – Deadwood
 Why is it bad?
Files in view
“Dead wood”
Files out of view

20. 20
Edge Cases – Deadwood
 First pitfall: Leave it alone
Files in view
“Dead wood”
Files out of view

21. 21
Edge Cases – Deadwood
 Second pitfall: Merge everything from the beginning of time
Files in view
“Dead wood”
Files out of view

22. 22
Edge Cases – Deadwood
 Implementation we settled on: Iteratively merge everything
in a smaller chunk of changes
• Has downsides, manual resolves scheduled when they’re not needed
Files in view
“Dead wood”
Files out of view

23. 23
Edge Cases – Deadwood
 Implementation we want to eventually get to: Sparse
Branches (John LoVerso’s MERGE 2016 presentation)
Files in view
“Dead wood”
Files out of view

24. 24
Edge Cases – Renames Across View
Files in view
“Dead wood”
Files out of view
 What is it?
• FileA -> FileB -> FileC on the source
• FileB is not in view

25. 25
Edge Cases – Rename Across View
What should happen: What actually happens:

26. 26
Edge Cases – Rename Across View
What should happen: What actually happens:

27. 27
Edge Cases – Rename Across View
What should happen: What actually happens:

28. 28
Edge Cases – Rename Across View
What should happen: What actually happens:

29. 29
Edge Cases – Rename Across View
What should happen: What actually happens:

30. 30
Edge Cases – Renames Across View
 Pitfalls:
• Only include the first and last names in view
• Only consider one rename, not multiple renames
• Add too many things to the view of a virtual stream

31. 31
Edge Cases – Renames Across View
 Solution:
• Merge everything in a given range, selectively revert edits and branches
• Same solution we use for keeping deadwood relatively managed

32. 32
Edge Cases – Shadowed Delete
 What is it?
• Delete that does not show up when you merge
• Not the head revision
• Integration engine thinks it’s done at the move/add

33. 33
Edge Cases – Shadowed Delete
What should happen: What actually happens:

34. 34
Edge Cases – Shadowed Delete
What should happen: What actually happens:

35. 35
Edge Cases – Shadowed Delete
What should happen: What actually happens:

36. 36
Edge Cases – Shadowed Delete
What should happen: What actually happens:

37. 37
Edge Cases – Shadowed Delete
 How to identify it?
• Look for deletes not at the head revision
in the range you are merging
 How to resolve it?
• Merge at a specific revision for the problem file

38. 38
Edge Cases – Shadowed Delete
 How to identify it?
• Look for deletes not at the head revision
in the range you are merging
 How to resolve it?
• Merge at a specific revision for the problem file
 Pitfalls?
• Merging multiple times without resolving

39. Conclusion

40. 40
Conclusion – Lessons Learned
 Things to do:
• Look at the big picture, solve a problem instead of its symptoms
• Ask users to follow a simple, managed workflow
• Test everything across releases and decide to upgrade or not
 Things not to do:
• Assume you’ve found all cases of a problem
• Assume Perforce behavior will stay the same

41. 41
Conclusion – What We’d Like
 Source control on an object-by-object basis
• Reconcile would work in most / all cases
• Renames, view, and componentization would be easier to handle and
define by file object content rather than specific filenames
 Truly sparse streams
• Prevent issues from ever happening, if the files never actually exist
on a stream

42. Questions?
Mike Hobbs
Mike dot Hobbs at mathworks dot com