Five Real-World Strategies for Perforce Streams
•Download as PPTX, PDF•
0 likes•8,853 views
Before you deploy Perforce Streams in your organization, you should have a plan in place. Get advice and hear the five strategies for using Streams and how to handle integration exceptions gracefully.
Recommended
Recommended
More Related Content
What's hot
What's hot (20)
Similar to Five Real-World Strategies for Perforce Streams
Similar to Five Real-World Strategies for Perforce Streams (20)
More from Perforce
More from Perforce (20)
Recently uploaded
Recently uploaded (20)
Five Real-World Strategies for Perforce Streams
- 1. # Erik van Nooijen Technical Lead at TomTom
- 2. # 10 years as contractor • (senior) developer 5 years in TomTom - Netherlands • 3 years senior developer • 2 years technical lead
- 3. #
- 4. # TomTom total Perforce installation size 780 Gb database 3.2 TB versioned files #proxies 10 #replicas 4 #users ~700 #workspaces ~19000 #changes ~1.000.000 Main server 32 core 284 Gb ram 4* Fusion IO (ssd) 8.6 Tb Streamed depot size 52 Gb #streams 360 #mainlines 1 #release streams 20 #users ~150 #changes ~80000
- 5. #
- 6. # • Parent / child stream relationship – predefined integration flows • Merge-down / copy-up – resolve-on-child paradigm • Stream specification vs branch mapping • Moving workspace • Naming conventions – Renaming a stream does not rename the underlying branch • Deleting streams
- 7. #
- 8. # • Integrate 1 (or few) specific changelists up, without doing a full copy-up • Examples: – Quick propagation of a bug fix – Selective moving finished work to prep stream
- 9. #
- 10. # • Integrations that are not following pre-defined flow • Using ‘p4 integrate //source/…@11,11 //dest/…’ • Example: integrate specific changelist into existing release. Most often happens for bug fixes.
- 11. # • Integrations that are not following pre-defined flow • Using ‘p4 integrate //source/…@11,11 //dest/…’ • Example: integrate specific changelist across development streams, without going via main. Usually for inter-dependencies between developments. • Alternative; use stream specification with ‘import’ – Defined imports with ‘import @’ available with p4d 2014.1 – Writable imports become available with p4d 2014.2
- 12. #
- 13. # Stream: //depot/main Type: mainline Paths: share //… Stream: //depot/dev-X Type: development Parent: //depot/main Paths: share //… import versions.txt //depot/main/versions.txt Client: depot_dev_ws View: //depot/dev-X/… // depot_dev_ws/… !! No ‘Stream:’ but using branch mappings $ p4 edit versions.txt $ p4 submit –d “my own development identifiers in versions.txt” $ p4 copy //depot/dev-X/… //depot/main/… $ p4 submit -d “copy up from dev stream” !! No –S <stream> for copy argument, but branch mappings Result: versions.txt changed on main! Final solution: 1. RW permission group on main for this file 2. educating users on p4 copy
- 14. #
- 15. # • Most benefit on server side, less so for end-users • Can be confusing when used with branch mapping • When to use regular development stream vs task stream ? – Task streams can be converted to type development, but conversion back is not possible • We decided not to use them, instead we use development streams only
- 16. # • Reduce the amount of data that gets pulled into a workspace • Compose product combinations with only imports – One mainline with all imports for ‘mainlines’ – Many virtual streams based on the mainline that overrule the imports with various ‘release’ imports – Easy and cheap to make many combinations
- 17. # canary-main (type: mainline) import os/… //depot-os/main/… import engine/… //depot-engine/main/… import ui/… //depot-ui/main/… import engine/ui-tweaks/… //depot-ui/main/tweaks/… canary-ui-rel1 (type: virtual) share … import ui/… //depot-ui/rel-1/… import engine/ui-tweaks/… //depot-ui/rel-1/tweaks/… canary-ui-rel2 (type: virtual) share … import ui/… //depot-engine/rel-2/… import engine/ui-tweaks/… //depot-ui/rel-2/tweaks/…
- 18. #
- 19. # • ‘p4 integrate –f’ is evil ! – alternative is ‘p4 copy’ • Use ‘integ.engine = 3’ – engine 3 is default, but maybe you have engine 2 specifically configured • Import with stream specification not (yet) supported – ‘import //… -S //depot/stream’
- 20. # • Educate your user base • Cherry picking integrations • Cross integrations • Stream specification versus branch mapping • Task and virtual streams
- 21. # Erik van Nooijen eric.vannooijen@tomtom.com #idonttweet
Editor's Notes
- 10 years contractor on and off for TomTom, some other jobs in between
- To give some indication of the size of Perforce in TomTom Streamed depot ~2.5 years in existence (since May 2012)
- Parent/child stream relationship: explain the ‘levels of stability: from dev -> dev -> main -> release’ “resolve on child paradigm”: or “Copy-up is trivial” This is the true power of perforce streams! Put focus here. Stream specification vs branch mapping: p4 commands support the “–S <stream>” option, which maps the stream view, while the same commands also allow for direct branch mapping /… notation (p4 integrate, copy, merge, …) Moving workspace: Very useful, but not all users like it. We have seen on numerous occasions submits in the wrong stream because the user forget the workspace was moved before. Use or don’t use per user-preference. Naming convention: we have: One mainline only, called ‘main’ All release stream names start with ‘rel-’ All development stream names start with ‘dev-’ All streams are named ‘<parentname>-<extension>’ to show clearly the parent/child structure, which is very convenient most of the time. It gives issues when reparenting (and would call for renaming, but that confuses again because of the streamname/branchname differences) Exception is of course all dev streams below main (they don’t start with ‘main-’) Exception is development stream below release stream (starts with ‘dev-’ and not ‘rel-’) Renaming a stream will be confusing for users and is not advised, based on experience. Deleting streams as to keep the streams overview clean. Note that it does not remove them from the left-hand side of p4v (@perforce; can this become a nested list?) TODO: add “use p4 <command> –S <stream>” option for all p4 commands that support it (see also later ‘streams vs branch mapping’)
- Cherry-picking concept is following the parent/child relationship, but does not do the full sets of changelists. Instead it does one or selected few sets of changelists.
- Cherrypicking happens, for example, when a specific change (say bugfix) needs to be propagated, but a full copy-up (or merge-down) is not wishful yet (because the full work on some task has not been completed). As we have streams for each stream below the mainline we have this behavior often. We have such setup because the teams like ‘working in their own area and setup’. Therefore teams cherrypick from their real development stream into a prep(eration) stream and from there copy-up into the mainline. TODO explain other situations where this might occur Call out to Perforce for official support and a whitepaper on this behavior: “Hey Perforce representatives; maybe you can support this officially with a ‘p4 –S <stream>’ command and describe the desired workflow in a nice whitepaper ?”
- Because there is no flow enabled from main into the release stream (by design, but can be enabled) it is not possible to normally integrate a fix from the mainline environment into the release. You could enable the flow in the stream specification of the release stream, but you’d have to copy a full set of changes, which is still not what you’d want. Making the edits twice manually, one in development stream and one in the release stream, is not a wishful situation as it is/can be lot of duplicate work and it opens the door for typos and (minor) differences. Call out to Perforce representatives again for another official support and whitepaper.
- Typically the same situation as before; consider a situation where 1 team/workpackage is depending on another that is not ready to deliver fully into the parent/mainline it happens that there is a crossintegration done to get the preliminary work quickly from one development stream into the other. Another option here is to be creative with import statements in the stream specification of dev-Y, but that can be cumbersome to maintain and gives issues if dev-Y wants to edit the imported files. Note: writable imports will be available in p4d 2014.2
- Tell the story of our user who did this and wasn’t even aware he did wrong. “but I used p4 copy” was his actual first response when asked about how he managed to copy versions.txt into main. Another option here is to be creative with import statements in the stream specification of dev-Y, but that can be cumbersome to maintain and gives issues if dev-Y wants to edit the imported files. Side effect is every stream gets the update via import whenever it changes on main, this is undesirable Note: writable imports will be available in 2014.2 P4D Educate your users: Use “p4 <command> –S <stream>”
- Task streams can be converted into development stream, but cannot convert back
- You can also do this with real (release) streams, but using virtual streams makes it very clear there isn’t any special branch, there isn’t any data stored specifically for these canaries and cleanup is easier, just delete the stream specification, no trace is left behind.
- ‘p4 integrate –f’ resets your integration history and causes unexpected results further down the line. We used it as our ‘go to hammer to hit any nail’ and since have moved away, preferring ‘p4 copy’ commands instead. TODO: can we make the road go behind the template footer ?
- Educate your user base: It seems so obvious, but instilling the true reasons and power of streams in combination with the appropriate commands with prevent you from some common mistakes Cherry picking integrations: Sometimes it is needed. Think about it, be prepared for the questions. Done with regular integrations that is not ‘streamy’ but works good. Cross integrations: Similar to cherry picking integrations, where it also ignores the common direction of merge and copy flows. @Perforce; can you support this use cases better ? Stream specification versus branch mapping: Use streamy commands wherever and whenever possible. When you use branch mapping; make sure you thought about its consequences. Task and virtual streams: Task streams have limited use, hard to define when you should choose for task rather than regular development type. @Perforce; maybe you can comment on this ? Virtual streams can be useful indeed, especially to limit the size of your workspaces, but some other creative use exists.
- Please pay attention to the mismatch in my email address, but really, my name is spelled correctly (that’s how my mum writes it, and she’s always right, as usual for mums) and my email is also correct.