2. TRAINING GOALS
Describe problems and
risks emerging during
merge operations in
VCSs
Show how to mitigate
those risks and avoid
problems related to
merging
2
3. TRAINING PLAN
1. What is merging?
2. Why do we need to manage merging?
3. Merging machinery
4. Branch types
5. Merging operation – maturity testing
6. Merging in action
7. Advanced merge management
8. Criteria of successful merge
3
5. WHAT IS MERGING?
• In general sense:
combination of two (or more) different objects into
the whole in the form of new object with unified
properties
• Applicably to Version Control Systems:
fundamental operation that reconciles multiple
changes made to a revision-controlled collection
of files
5
6. WHY DO WE NEED TO MANAGE
MERGING?
1. Result of merging often might be unpredictable
2. Nobody can guarantee that result of merging is
the one you really wanted to achieve
3. Few people know the machinery of merging
process
4. If something unpredictable happened during
merging, almost nobody can explain why.
5. This is all because merging is the one of the most
non-trivial operations of version control systems
6
9. COMMON BUZZWORDS OF
MERGING MACHINERY
diff patch
branch merge
There are two diffs:
diff and diff3
conflict cherrypicking
9
10. DIFF (TWO-WAY MERGE)
Based on the longest common subsequence (LCS)
algorithm
Not to confuse with longest common substring algorithm
Complexity of LCS algorithm is O(mn); m, n – lengths of
the strings
Therefore, diff works using line by line comparison
Uses four formats: normal, context, unified, ed
Can compare directories and their content recursively
But does not track directories move/rename
10
12. DIFF3 (THREE-WAY MERGE)
Diff3 utility is the basis for:
Manual comparison of three files
Automatic merging
Generation of conflict situations
Textual representation of the conflicts
12
13. DIFF3 (THREE-WAY MERGE)
You can think of diff3 as subtracting older
from theirs and adding the result to mine …
or
… as merging into mine the changes that
would turn older into theirs
13
15. COMMON BUZZWORDS OF
MERGING MACHINERY
diff patch
branch merge
patch is often
complementary to the
diff
conflict cherrypicking
15
16. PATCH
You can think about patch – as a:
Useful utility for applying specific source code change
set to the working copy
Reversed diff
Imitation of distributed version control in centralized
VCSs
16
17. PATCH
Interesting facts about patch:
Merging functionality in DVCSs is implemented via
series of subsequent patches applied.
There is no patch format extension for directories
structure modification. Beware about move, rename and
delete operations
svn merge command is like patch, but with additional
special abilities such as: tree structure changes tracking
and mergeinfo tracking
Result of svn diff command is the fully functioning
patch file, but with little bit more limited representation
abilities (subset of patch format)
Patch is also often called changeset 17
19. COMMON BUZZWORDS OF
MERGING MACHINERY
diff merge has no sense
patch
without branches
branch merge
conflict cherrypicking
19
20. BRANCH
No merge without branch
No branch without merge
20
21. BRANCH
You can think about branch as:
Separate isolated parallel
development line (codebase)
Operation of separate development
line creation (branching)
Set of files having common ancestor
with another set of files
Another folder in repository (usually in
/branches parent directory) of
centralized VCS
Dedicated private place in distributed
21
VCS
22. BRANCH TYPES
svn merge sourceURL1[@N] sourceURL2[@M] [WCPATH]
svn merge -r N:M SOURCE [PATH]
Merging operation is highly dependent on source
and target branches
Purpose of the branches, their contents and types
directly affect the merging operation workflow and
its results 22
24. BRANCH TYPES
Trunk Experimental
Branch
types
Release Support 24
25. COMMON BUZZWORDS OF
MERGING MACHINERY
diff patch
branch merge
merge is the most
advanced operation
conflict
VCSs have cherrypicking
25
26. MERGE
You can think about merge as:
Reconciliation of multiple changes made to a revision-
controlled collection of files
Application of the differences between two sources to a
working copy path (svn merge, svn update)
Operation of joining two or more development histories
together (git merge)
Reintegration of the whole branch (or its part) into
another branch
Process of two or three files comparison: 2-way merge
and 3-way merge
Automatically generated result of 3-way merge:
26
automerge
29. COMMON BUZZWORDS OF
MERGING MACHINERY
diff patch
conflicts happen
when there are difficulties
branch
with merging
merge
conflict cherrypicking
29
30. CONFLICT
It is difficult to avoid conflict situation
during team work
30
31. CONFLICT
Conflict – is a state of one or more working copy
artifacts assuming that:
The same source code lines have been modified by different
users
Artifact properties have been modified by different users
VCS cannot resolve modifications of directory structure have
been made by different users
Conflict situations are potentially produced by:
svn update
svn merge
31
32. GENERAL CONFLICT RESOLUTION
WORKFLOW
Or edit files
Update manually saving
working copy final version and
with running svn
conflicting resolved
changes command
Use one of Commit
the conflict resolved
resolution changes
options
suggested by
VCS (svn 32
resolve)
33. CONFLICT RESOLUTION
OPTIONS
p • postpone
df • diff-full
e • edit
r • resolved
mc • mine-conflict
tc • theirs-conflict
33
s • show all options
34. COMMON BUZZWORDS OF
MERGING MACHINERY
diff patch
cherrypicking means
replication of single
change
branch merge
conflict cherrypicking
34
40. MERGING TWO TREES
>: svn merge
svn://localhost/ParseCmdLine/branches/experimental/experimentalOptions
svn://localhost/ParseCmdLine/trunk --ignore-ancestry
40
41. DIFF3 VS MERGE
diff3 always ignores ancestry
merge takes into account ancestry by default
Therefore, repository should store information about:
Branching (svn copy operations)
Merging (svn merge operations, revisions, baselines, etc)
Tree modifications (files copying, deletion, movement, etc)
Operations sequence
Before the release of Subversion 1.5 this functionality
was implemented in svnmerge.py
In Subversion 1.5 it is called merge tracking
41
42. MERGE TRACKING
Reduces the bookkeeping overhead for branch maintenance
Allows to avoid common cases of the "repeated merge" problem
Allows cherry-picking of changes
Allows outputting list of revisions available for merging
Support of bidirectional merges
Support of blocking changes
Support of merge rollbacks 42
48. MERGE. MATURITY TESTING
Merging operation maturity test cases:
Standard conflict handling
BOF/EOF Stingers
Identical Twins sure same source code
Making
lines have been modified by
X-Tuplets different users
Hungry Blobs
Unique Diff3
anchors Subversion Git
Works as expected Works as expected Works as expected 48
49. MERGE. MATURITY TESTING
Merging operation maturity test cases:
Standard conflict handling
BOF/EOF Stingers
Identical Twins
Sometimes problems emerge
X-Tuplets at the beginning and ending of
the file
Hungry Blobs
Unique Diff3
anchors Subversion Git
Works as expected Works as expected Works as expected 49
50. MERGE. MATURITY TESTING
Merging operation maturity test cases:
Standard conflict handling
BOF/EOF Stingers
Identical Twins
X-Tuplets handling identical files from
Hungry Blobsdifferent working copies
Unique Diff3
anchors Subversion Git
Fails Works as expected Works as expected 50
51. MERGE. MATURITY TESTING
Merging operation maturity test cases:
Diff3 Subversion Git
Standard conflict Works as expected
Fails handling Fails
BOF/EOF Stingers
handling identical files from different working
Identical Twins with minor adjacent changes
copies, but
X-Tuplets
Hungry Blobs
Unique anchors
51
52. MERGE. MATURITY TESTING
Merging operation maturity test cases:
Diff3 Subversion Git
Standard expected handling
Works as conflict Fails Works as expected
BOF/EOF Stingers
adjacent changes from
Identical Twins
different working copies
X-Tuplets
Hungry Blobs
Unique anchors
52
53. MERGE. MATURITY TESTING
Merging operation maturity test cases:
Diff3 Subversion Git
Standard conflict handling
Fails Fails Fails
BOF/EOF Stingers
Identical Twins additional heuristics
using
during diff for covering special
X-Tuplets cases of merging
Hungry Blobs
Unique anchors
53
56. BRANCHES INHERITANCE
There are certain branch creation and inheritance
restrictions which will be described by filling in table
shown below
RELATION child
BRANCH TYPE trunk support release experimental
trunk
parent support
release
experimental
57
57. BRANCHES INHERITANCE
Trunk cannot be the child of any other branch
RELATION child
BRANCH TYPE trunk support release experimental
trunk
parent support
release
experimental
58
58. BRANCHES INHERITANCE
Support branch can be inherited only from trunk
RELATION child
BRANCH TYPE trunk support release experimental
trunk +
parent support -
release -
experimental -
59
59. BRANCHES INHERITANCE
Release branches can be inherited both from trunk
and support branches
RELATION child
BRANCH TYPE trunk support release experimental
trunk + +
parent support - +
release - -
experimental - -
60
60. BRANCHES INHERITANCE
Experimental branches can be inherited from any
type of branch except release branch
RELATION child
BRANCH TYPE trunk support release experimental
trunk + + +
parent support - + +
release - - -
experimental - - +
61
63. TO MERGE OR NOT TO MERGE?
There are certain merging restrictions which will be
described by filling in table shown below
We will fill in it step by step in order to understand all
aspects of the advanced merge management.
DESTINATION to
BRANCH TYPE trunk support release experimental
RELATION parent parent child parent child parent child
trunk parent
parent
support
from child
parent
release
child 64
parent
experimental
child
64. TO MERGE OR NOT TO MERGE?
Final result will look as follows
Logically excluded cases of merging
Cases excluded by branch inheritance and creation rules
+ Allowed case of merging
- Disallowed case of merging operation
DESTINATION to
BRANCH TYPE trunk support release experimental
RELATION parent parent child parent child parent child
trunk parent - - +
parent - +
support
from child - - -
parent
release
child + + - - 65
parent +
experimental
child + + - - + +
65. TO MERGE OR NOT TO MERGE?
Trunk can be only parent branch
RELATION child
BRANCH TYPE trunk support release experimental
trunk + + +
parent support - + +
release - - -
experimental - - +
DESTINATION to
BRANCH TYPE trunk support release experimental
RELATION parent parent child parent child parent child
trunk parent
parent
support
from child
parent
release
child 66
parent
experimental
child
66. TO MERGE OR NOT TO MERGE?
Merging from parent to parent of the same branch
seems to be merged into itself. It makes sense only
as a conflict which is not the case of merge we're
considering
DESTINATION to
BRANCH TYPE trunk support release experimental
RELATION parent parent child parent child parent child
trunk parent
parent
support
from child
parent
release
child 67
parent
experimental
child
67. TO MERGE OR NOT TO MERGE?
Branch cannot be parent to itself
DESTINATION to
BRANCH TYPE trunk support release experimental
RELATION parent parent child parent child parent child
trunk parent
parent
support
from child
parent
release
child 68
parent
experimental
child
68. TO MERGE OR NOT TO MERGE?
There are cases automatically excluded by branch
creation and inheritance rules
RELATION child
BRANCH TYPE trunk support release experimental
trunk + + +
parent support - + +
release - - -
experimental - - +
DESTINATION to
BRANCH TYPE trunk support release experimental
RELATION parent parent child parent child parent child
trunk parent
parent -
support
from child - -
parent -
release
child - - 69
parent +
experimental
child + +
69. TO MERGE OR NOT TO MERGE?
There are cases automatically excluded by branch
creation and inheritance rules
RELATION child
BRANCH TYPE trunk support release experimental
trunk + + +
parent support - + +
release - - -
experimental - - +
DESTINATION to
BRANCH TYPE trunk support release experimental
RELATION parent parent child parent child parent child
trunk parent
parent
support
from child - -
parent - -
release
child - 70
parent - -
experimental
child -
70. TO MERGE OR NOT TO MERGE?
This is the result of exclusion merging cases by
logic and branches creation and inheritance rules.
We will be filling in empty cells
DESTINATION to
BRANCH TYPE trunk support release experimental
RELATION parent parent child parent child parent child
trunk parent
parent
support
from child
parent
release
child 71
parent
experimental
child
71. TO MERGE OR NOT TO MERGE?
Support branches are fully independent, they are
not assumed to be merged with trunk or other
support branches
/trunk
/1.x.x
DESTINATION to
BRANCH TYPE trunk support release experimental
RELATION parent parent child parent child parent child
trunk parent -
parent
support
from child -
parent
release
child 72
parent
experimental
child
72. TO MERGE OR NOT TO MERGE?
Purpose of release branch is maturing the release
It is expected that only bugfixes will be merged to
parent branch and no changes will be merged from
parent branch
DESTINATION to
BRANCH TYPE trunk support release experimental
RELATION parent parent child parent child parent child
trunk parent - -
parent -
support
from child -
parent
release
child + + 73
parent
experimental
child
73. TO MERGE OR NOT TO MERGE?
Purpose of release branch is maturing the release
/0.2.x /1.0.x
/trunk /1.x.x
DESTINATION to
BRANCH TYPE trunk support release experimental
RELATION parent parent child parent child parent child
trunk parent - -
parent -
support
from child -
parent
release
child + + 74
parent
experimental
child
74. TO MERGE OR NOT TO MERGE?
Child to child relation represents sibling branches
No merge between sibling release and support
branches is allowed
DESTINATION to
BRANCH TYPE trunk support release experimental
RELATION parent parent child parent child parent child
trunk parent - -
parent -
support
from child - -
parent
release
child + + - 75
parent
experimental
child
75. TO MERGE OR NOT TO MERGE?
Child to child relation represents sibling branches
/trunk
/1.0.x
/1.x.x
DESTINATION to
BRANCH TYPE trunk support release experimental
RELATION parent parent child parent child parent child
trunk parent - -
parent -
support
from child - -
parent
release
child + + - 76
parent
experimental
child
76. TO MERGE OR NOT TO MERGE?
It is allowed to merge experimental branches with
any type of parent branch
/trunk
/UI_integration
/Layout_rework
/3-column
DESTINATION to
BRANCH TYPE trunk support release experimental
RELATION parent parent child parent child parent child
trunk parent - - +
parent - +
support
from child - -
parent
release
child + + - 77
parent +
experimental
child + + +
77. TO MERGE OR NOT TO MERGE?
No merge allowed between sibling experimental and
any type of other branch (except experimental)
/trunk /Layout_rework
/1.x.x /UI_integration
/1.0.x /3-column
/UI_integration
DESTINATION to
BRANCH TYPE trunk support release experimental
RELATION parent parent child parent child parent child
trunk parent - - +
parent - +
support
from child - - -
parent
release
child + + - - 78
parent +
experimental
child + + - - + +
78. TO MERGE OR NOT TO MERGE?
Final picture
49 Total number of cells
33 Number of cells with logically excluded cases
20 Number of cells with cases excluded logically and using
branch inheritance rules
9 Allowed case of merging
73% Disallowed cases introduced by merging rules
DESTINATION to
BRANCH TYPE trunk support release experimental
RELATION parent parent child parent child parent child
trunk parent - - +
parent - +
support
from child - - -
parent
release
child + + - - 79
parent +
experimental
child + + - - + +
79. SUPPORT AND RELEASE
BRANCHES
/1.1.x /2.0.x
/trunk
/1.2.x
/1.0.x
/1.x.x
release branch allowed merge
operation 80
support branch disallowed merge
operation
85. HOW WOULD YOU KNOW THE MERGE IS
SUCCESSFUL?
Don’t leave merges unfinished
87
86. HOW WOULD YOU KNOW THE MERGE IS
SUCCESSFUL?
Run unit-tests and inspections afterwards
88
87. HOW WOULD YOU KNOW THE MERGE IS
SUCCESSFUL?
Formal criteria:
There are no conflicts
Code compiles
Unit-tests run successfully
Conceptual criteria:
Implemented features do not have logical and architectural
contradictions
Whole codebase correspond to the introduced concepts
In case of intersection between merged features
functionality it should lead to additional development
89
93. USEFUL LINKS
http://www.misuse.org/science/2007/02/24/3-way-merging/ -
three way merging
http://svnbook.red-
bean.com/en/1.5/svn.branchmerge.advanced.html - advanced
merging in subversion
http://codereflect.com/2010/09/21/versioning-systems-and-3-
way-merge-process/ - simple way to understand 3 way merge
process
http://book.git-
scm.com/3_basic_branching_and_merging.html - basic
branching and merging
http://book.git-
scm.com/5_advanced_branching_and_merging.html -
advanced branching and merging
http://hgbook.red-bean.com/read/a-tour-of-mercurial-merging-
work.html - how mercurial merging works
http://www.collab.net/community/subversion/articles/merge- 95
info.html - detailed description of merge tracking features
Editor's Notes
У сегодняшнего тренинга две цели.Первая - ?Вторая – ?
Мы рассмотрим следующие вопросы:Что такое CIЗачем это вообще надоБез чего CI невозможенСамый общий workflow – действия, входящие в число стандартных при организации процесса непрерывной интеграции Как CI влияет на весь процесс разработкиИнструменты и их особенностиОписание случаев, когда CI не нужен или неэффективенНовая грань непрерывной интеграции – чего не хватает в функциональности и базовых идеях существующих инструментовСвязь непрерывной интеграции и модели зрелости процессов.
Начнем с базовых идей. Опрос на тему использования практики CI
Google (сущ.) – гуглить (гл.)Diff – диффовать(глагол), дифф (существительное),Патч, ПатчитьBranch, branchingMerge – процесс слияния, результат слиянияConflict – состояние рабочей копии (файлов), процесс конфликтования измененийCherrypicking – процесс точечного выбора изменений для проведения слияния
История слияний начинается с появления команды diff, которая стала базисом для операций, связанных со сравнением текстовых файлов. Всё, что в последствии стало надстройкой и дополнением к операции сравнения превратилось в то, что мы сейчас знаем как слияние.
….6. 7. Directories structure evolution tracking has not been implemented yet. Even in subversionMention unified diff
Демонстрация двунаправленного слияния: unified format, normal, ed (diffman)Example: properties file
Демонстрация тринаправленного слияния
Патч – это сравнение наоборот, применение результата сравнения с целью изменить файл или группу файлов. Rsyncиспользует концепцию патчей для минимизации трафика.
Демонстрация работы патчей
There is a simple fact: no merge without branch, no branch without merge.Even when there are no branches, sure you do merging: either automatic (during svn update) or manual (during conflict resolution)
What branch features influence merge operations?
Демонстрация создания ветки
2-way merge
Наиболее оптимальный подход – это комбинированая модель версионирования. Как в subversion. Copy-modify-merge – для исходных и текстовых файлов, lock-unlock – для бинарных файлов.
Google (сущ.) – гуглить (гл.)Diff – диффовать(глагол), дифф (существительное),Патч, ПатчитьBranch, branchingMerge – процесс слияния, результат слиянияConflict – состояние рабочей копии (файлов), процесс конфликтования измененийCherrypicking – процесс точечного выбора изменений для проведения слияния
directory structure conflicts happen usually when merged files has been deleted. There are several cases: Local delete, incoming edit upon updateLocal edit, incoming delete upon updateLocal delete, incoming delete upon updateLocal missing, incoming edit upon mergeLocal edit, incoming delete upon mergeLocal delete, incoming delete upon merge
С разбором основных понятий, касающихся слияний мы закончили. К cherrypickingи объяснению того, что это такое, мы вернемся чуть позже. Следующая секция практически полностью посвящена описанию и демонстрации этой концепции.
Давайте потестируем операции слияния на зрелость и непротиворечивость. Нас разработчики систем контроля версий всегда уверяют в том, что автоматические слияния безопасны, не вносят существенных изменений в исходный код, а когда не могут определить результат слияния, помечают его как конфликт. Но на практике оказывается, что так происходит далеко не всегда. В теории одно, на практике – другое. И мы сейчас в этом убедимся.
Наш тест покажет, что автоматическим слияниям нельзя полностью доверять. «Доверяй, но проверяй». Причем будем показано, что распределенные системы контроля версий также допускают досадные ошибки при слияниях.
Есть несколько простых тест-кейсов для анализа алгоритма слияния и их недостатков.
Демонстрация случая
Демонстрация случая
Демонстрация случая
Демонстрация случая
Демонстрация случая
Демонстрация случая
All the automerged code should be covered with unit test cases.Distributed version control has the possibility of review/approve source code changeset (Mercurial and Kiln)Conslusion. You cannot fully trust automerge and therefore, you cannot more complex merging results as well.
Нам нужно знать, какие существуют инструменты для организации процесса непрерывной интеграции и какими специфическими особенностями они обладают.
Это были основы. Сейчас я буду углубляться в тонкости всего того, что может быть связано с управлением слияниями. C помощью организационных ограничений мы попытаемся решить большинство проблем, возникающих при проведении слияний. Мы разберем большинство случаев слияний, взяв за основу типизацию веток, а также подход Agile SCM, который эту типизацию вводит.
Попросить лучших студентов заполнить эту таблицу
Если мы выпускаем релиз, нам не нужны никакие эксперименты
Как уже было сказано, появляется еще один тип ветки – ветка поддержки, которая на диаграммах потока разработки будет изображаться другим цветом.
Еще один тип веток – экспериментальные. На экспериментальные ветки не накладывается практически никаких ограничений: ни на количество дочерних веток (1), ни на именование самих веток, ни на слияния. Два единственных ограничения – это:(2) Номер сборок, выполняющихся из экспериментальной ветки. Должна соблюдаться сквозная нумерация, принятая для родительской ветки.Время создания. Экспериментальные ветки могут быть дочерними только для веток поддержки (и НЕ для веток релиза)Вот это тот случай, когда должны вступать в бой возможности DVCS. Причем, в этом месте у людей начинает образовываться полная каша в голове. Если вы ничего не поняли, не переживайте!
Если мы выпускаем релиз, нам не нужны никакие эксперименты
Если мы выпускаем релиз, нам не нужны никакие эксперименты
Если мы выпускаем релиз, нам не нужны никакие эксперименты
Если мы выпускаем релиз, нам не нужны никакие эксперименты
Если мы выпускаем релиз, нам не нужны никакие эксперименты
Если мы выпускаем релиз, нам не нужны никакие эксперименты
Если мы выпускаем релиз, нам не нужны никакие эксперименты
Если мы выпускаем релиз, нам не нужны никакие эксперименты
Если мы выпускаем релиз, нам не нужны никакие эксперименты
Если мы выпускаем релиз, нам не нужны никакие эксперименты
Если мы выпускаем релиз, нам не нужны никакие эксперименты
Если мы выпускаем релиз, нам не нужны никакие эксперименты
Если мы выпускаем релиз, нам не нужны никакие эксперименты
Если мы выпускаем релиз, нам не нужны никакие эксперименты
Еще один тип веток – экспериментальные. На экспериментальные ветки не накладывается практически никаких ограничений: ни на количество дочерних веток (1), ни на именование самих веток, ни на слияния. Два единственных ограничения – это:(2) Номер сборок, выполняющихся из экспериментальной ветки. Должна соблюдаться сквозная нумерация, принятая для родительской ветки.Время создания. Экспериментальные ветки могут быть дочерними только для веток поддержки (и НЕ для веток релиза)Вот это тот случай, когда должны вступать в бой возможности DVCS. Причем, в этом месте у людей начинает образовываться полная каша в голове. Если вы ничего не поняли, не переживайте!
Еще один тип веток – экспериментальные. На экспериментальные ветки не накладывается практически никаких ограничений: ни на количество дочерних веток (1), ни на именование самих веток, ни на слияния. Два единственных ограничения – это:(2) Номер сборок, выполняющихся из экспериментальной ветки. Должна соблюдаться сквозная нумерация, принятая для родительской ветки.Время создания. Экспериментальные ветки могут быть дочерними только для веток поддержки (и НЕ для веток релиза)Вот это тот случай, когда должны вступать в бой возможности DVCS. Причем, в этом месте у людей начинает образовываться полная каша в голове. Если вы ничего не поняли, не переживайте!
Слияний нужно избегать, но если вдруг необходимость в этом возникла – вы теперь с уверенностью сможете это сделать. А после открыть бутылку пива или другого любимого напитка и отпраздновать.