1. A tale about a Big
SVN to Git migration
Max rydahl Andersen
@maxandersen
Senior Principal Engineer, red Hat
Wednesday 27 March 13

2. • Do not use svn2git
• Learn about fast-filter.py
• Scripts/Resources at:
• https://github.com/maxandersen/jbosstools-gitmigration
Wednesday 27 March 13

3. goals for git migration
• Split up Big Repo to smaller ones
• Keep the history
• Get to use GitHub
• Make it fast and fun again!
Wednesday 27 March 13

4. Keeping it simple
common plugins .core
.ui
...
features .feature
.source.fe
ature
...
tests .tests
...
site
Wednesday 27 March 13

5. Keeping it simple
common plugins .core
.ui
...
features .feature
.source.fe
ature
...
tests .tests
Svn
...
site
git svn clone -s
git svn --authors-file=authors.txt
http://svn.example.com/svn-repository/ svn-mirror
svn-mirror.git
Wednesday 27 March 13

6. ...a real project
common plugins .core
.ui
...
features .feature
.source.fe
ature
...
tests .tests
...
site
Wednesday 27 March 13

7. ...a real project
JSF
... plugins .core ... plugins .core
common plugins .core
hibernate plugins .core CDI plugins .core plugins .core
seam plugins .core
.ui
.ui .ui .ui .ui
.ui .ui
...
... ... ... ...
features .feature
... ...
.source.fe features .feature features .feature
ature
features .feature features .feature features .feature
features .feature
... .source.fe
ature .source.fe
ature
.source.fe .source.fe .source.fe
tests .tests .source.fe ature ature ature
... ature
...
...
... ... ...
tests .tests
site ...
tests .tests
... tests .tests
tests .tests tests .tests
tests .tests ...
site
...
site ... ...
...
site
site site
site
... plugins .core ... plugins .core ... plugins .core ... plugins .core ... plugins .core ... plugins .core ... plugins .core ... plugins .core ... plugins .core ... plugins .core ... plugins .core ... plugins .core
.ui .ui .ui .ui .ui .ui .ui .ui .ui .ui .ui .ui
... ... ... ... ... ... ... ... ... ... ... ...
features .feature features .feature features .feature features .feature features .feature features .feature features .feature features .feature features .feature features .feature features .feature features .feature
.source.fe .source.fe .source.fe .source.fe .source.fe .source.fe .source.fe .source.fe .source.fe .source.fe .source.fe .source.fe
ature ature ature ature ature ature ature ature ature ature ature ature
... plugins .core ... plugins .core ... plugins .core ... plugins .core ... plugins .core ... plugins .core ... plugins .core ... plugins .core ... plugins .core ... plugins .core ... plugins .core ... plugins .core
... ... ... ... ... ... ... ... ... ... ... ...
.ui .ui .ui .ui .ui .ui .ui .ui .ui .ui .ui .ui
tests .tests tests .tests tests .tests tests .tests tests .tests tests .tests tests .tests tests .tests tests .tests tests .tests tests .tests tests .tests
... ... ... ... ... ... ... ... ... ... ... ...
... ... ... ... ... ... ... ... ... ... ... ...
features .feature features .feature features .feature features .feature features .feature features .feature features .feature features .feature features .feature features .feature features .feature features .feature
site site site site site site site site site site site site
.source.fe .source.fe .source.fe .source.fe .source.fe .source.fe .source.fe .source.fe .source.fe .source.fe .source.fe .source.fe
ature ature ature ature ature ature ature ature ature ature ature ature
... ... ... ... ... ... ... ... ... ... ... ...
tests .tests tests .tests tests .tests tests .tests tests .tests tests .tests tests .tests tests .tests tests .tests tests .tests tests .tests tests .tests
... plugins .core ... plugins .core ... plugins .core ... plugins .core ... plugins .core ... plugins .core ... plugins .core ... plugins .core ... plugins .core ... plugins .core ... plugins .core ... plugins .core
... ... ... ... ... ... ... ... ... ... ... ...
.ui .ui .ui .ui .ui .ui .ui .ui .ui .ui .ui .ui
site site site site site site site site site site site site
... ... ... ... ... ... ... ... ... ... ... ...
features .feature features .feature features .feature features .feature features .feature features .feature features .feature features .feature features .feature features .feature features .feature features .feature
.source.fe .source.fe .source.fe .source.fe .source.fe .source.fe .source.fe .source.fe .source.fe .source.fe .source.fe .source.fe
ature ature ature ature ature ature ature ature ature ature ature ature
... ... ... ... ... ... ... ... ... ... ... ...
tests .tests tests .tests tests .tests tests .tests tests .tests tests .tests tests .tests tests .tests tests .tests tests .tests tests .tests tests .tests
... ... ... ... ... ... ... ... ... ... ... ...
site site site site site site site site site site site site
Wednesday 27 March 13

8. How hard can it be ?
Svn
git svn clone --prefix=svn/ -s
git svn --authors-file=authors.txt
http://svn.example.com/svn-repository/ svn-mirror
svn-mirror.git
Wednesday 27 March 13

9. Check list for migration
1.Create svn-mirror.git from mirrored svn
2.Cleanup Mirror
1. rename master to trunk, Delete dead branches,
Checkout all branches, tag branches to real tags
3.Create individual Repositories
1. Use fast-filter.py to process content, Remove empty
tags/branches, Garbage Collect
4.Clean up Garbage
1. Prune big files, Fix line endings, Garbage Collect
5. ? (Publish to Github)
6.Profit!
Wednesday 27 March 13

10. SVN Mirroring
svn sync Svn
Svn
Mirror
svnsync init file://svn-mirror http://svn.example.com/repository
while(true)
svnsync sync file://svn-mirror http://svn.example.com/repository
Wednesday 27 March 13

11. SVN Mirroring
svn sync Svn
Svn
Mirror
svnsync init file://svn-mirror http://svn.example.com/repository
while(true) git svn
svnsync sync file://svn-mirror http://svn.example.com/repository
cd svn-mirror
git svn rebase --fetch-all
svn-mirror.git
Wednesday 27 March 13

12. SVN Mirroring
svn sync Svn
Svn
Mirror
svnsync init file://svn-mirror http://svn.example.com/repository
while(true) git svn
svnsync sync file://svn-mirror http://svn.example.com/repository
cd svn-mirror
git svn rebase --fetch-all
svn-mirror.git
Svn and Git now isolated and
always in sync !!!
Wednesday 27 March 13

13. svn mirroring
• Atlassian blogs has excellent material, but...
• Use --fetch-all to get full history!
• Do not remove metadata - it will help you in case
of failures
Wednesday 27 March 13

14. Cleanup Branches/Tags
svn-mirror.git
Wednesday 27 March 13

15. Cleanup Branches/Tags
svn-mirror.git
• Branches:
• trunk
• deadbranch
• tags/jbt-4.0.1
• tags/jbt-4.0.2
• jbt-4.0.x
• jbt-3.03
Wednesday 27 March 13

16. Cleanup Branches/Tags
svn-mirror.git
• Branches: • Branches:
• trunk • master
• jbt-4.0.x
• deadbranch
• jbt-3.03
• tags/jbt-4.0.1
• Tags:
• tags/jbt-4.0.2 • jbt-4.0.1
• jbt-4.0.x • jbt-4.0.2
• jbt-3.03
Wednesday 27 March 13

17. Cleanup branches/tags
svn-mirror.git
# Rename trunk to master
$ git branch -m trunk master
$ git branch -rd origin/trunk
# Delete dead branches
$ git branch -a -r |
grep -E "dead|broken|weird|I_WAS_DRUNK" |
xargs git branch -r -D
Wednesday 27 March 13

18. Cleanup branches/tags
svn-mirror.git
# Convert svn tag-branches into git-tags
$ git branch | grep tags |
sed -e "s/tags///g" |
xargs -n 1 -I {}
git tag -m "svn branch tag" {} tags/{}
# delete svn tag-branches
$ git branch | grep tags |
xargs -n 1 git branch -d
Wednesday 27 March 13

19. To split or not to split ?
svn-mirror.git
Wednesday 27 March 13

20. To split or not to split ?
svn-mirror.git
Wednesday 27 March 13

21. Wednesday 27 March 13

22. split/filter process
svn-mirror.git
• /modulex
• plugina
• pluginb
• /moduley
• plugina
Wednesday 27 March 13

23. split/filter process
svn-mirror.git fastfilter
• /modulex
• /modulex
• plugina
• plugina
• pluginb
• pluginb
• /moduley
• plugina
Wednesday 27 March 13

24. split/filter process
svn-mirror.git fastfilter filter-branch
• /modulex
• /modulex
• plugina
• plugina
• plugina
• pluginb
• pluginb
• pluginb
• /moduley
• plugina
Wednesday 27 March 13

25. fast-filter+filter-branch
$ python filter_repo.py
jbosstools-svn-mirror jbosstools-openshift "^openshift.*"
Wednesday 27 March 13

26. fast-filter+filter-branch
$ python filter_repo.py
jbosstools-svn-mirror jbosstools-openshift "^openshift.*"
$ git filter-branch --tag-name-filter cat --prune-empty
--subdirectory-filter openshift -f -- --all
Wednesday 27 March 13

27. Clean up branches
(again)
• $ deleteemptybranches.sh openshift-repo
• # delete empty commits
• $ git filter-branch --tag-name-filter cat
--prune-empty -- --all
Wednesday 27 March 13

28. “it’s simple”
Wednesday 27 March 13

29. “it’s simple”
svn sync Svn
Svn
Mirror
Wednesday 27 March 13

30. “it’s simple”
svn sync Svn
Svn
Mirror
git svn
svn-mirror.git
Wednesday 27 March 13

31. “it’s simple”
svn sync Svn
Svn
Mirror
git svn
svn-mirror.git
cleanup
Wednesday 27 March 13

32. “it’s simple”
svn sync Svn
Svn
Mirror
git svn
svn-mirror.git
cleanup
Wednesday 27 March 13

33. “it’s simple”
svn sync Svn
Svn
Mirror
cleanup
git svn
svn-mirror.git
cleanup
Wednesday 27 March 13

34. Check list for migration
1.Create svn-mirror.git from mirrored svn
2.Cleanup Mirror
1. rename master to trunk, Delete dead branches,
Checkout all branches, tag branches to real tags
3.Create individual Repositories
1. Use fast-filter.py to process content, Remove empty
tags/branches, Garbage Collect
4.Clean up Garbage
1. Prune big files, Fix line endings, add .gitignore
Garbage Collect
5. ?
6.Profit!
Wednesday 27 March 13

35. Tips’n’Tricks
Wednesday 27 March 13

36. Train your team
• http://try.github.com
• O'Reilly Videos
• Understand the internals: https://vimeo.com/49444883
• Learn the CLI before using the UI’s
• Ban git push --force
• Green Button Considered Harmful
Wednesday 27 March 13

37. guardians
vs
Kings
Wednesday 27 March 13

38. Git Hub api is your friend
Wednesday 27 March 13

39. Hub
“github cli”
http://defunkt.io/hub/
• $ hub clone maxandersen/autolinkjboss
• $ hub fork
• $ hub pull-request
• “Fixed this bad presentation”
Wednesday 27 March 13

40. Questions ?
https://github.com/maxandersen/jbosstools-gitmigration
Wednesday 27 March 13