If you work with large volumes of data—multimedia assets, video game art, or firmware designs—you understand the pain of trying to quickly get a copy of source and build assets. But if you have the right storage system, you can be up and running with a new Perforce workspace in minutes instead of hours. See a simple procedure for fast workspace cloning using a few Perforce commands and NetApp FlexClone.

1. 1
Managing Big Workspaces with
Storage Magic
Agnes Jacob, Netapp
Randy DeFauw, Perforce
Randy DeFauw
Senior Product Manager
Agnes Jacob

2. 2
What’s a big workspace?
§ Working with art or media for
games
§ Working with EDA data
§ Large amount of build artifacts
§ 80+ GB, 90 minute build time
is not unusual

3. 3
What’s the problem?
§ Storage is cheap! CPUs are fast!
§ But…
§ A personal workstation still has limits
§ You may lose more than an hour
every time you need a new
workspace fully populated with
source and build data

4. 4
A little trick: lazy copies
§ What Perforce does for branches…
§ Storage solutions can do for workspaces
Branch
Edit
New
revisions

5. 5
Step One: Make a reference workspace
§ Create a volume on the filer
§ Make the new workspace
§ Sync the data
§ Build
§ Snapshot
§ This is our baseline

6. 6
Step Two: Prepare a new workspace
§ Clone the snapshot
§ Mount the cloned volume
§ Assign cloned volume ownership

7. 7
Step Three: Register a new workspace
§ Define workspace in Perforce
§ Flush metadata: make Perforce think you have
the same data as the reference workspace
§ Sync to head (optional)

8. 8
Step Four: Keep the template up to date
§ Run a build daemon
§ Sync and build reference workspace

9. 9
A success story
Standard
workspace
methods
80
minutes
Hard
links
40
minutes
Workspace
cloning
2-­‐3
minutes
No
build
necessary
Minimal
data
transfer
work
Saves
millions
of
$$$

10. 10
NetApp FlexClone
Behind the Magic

11. 11
NetApp Snapshot™
Technology
• Take snapshot 1
• Copy pointers only
• No data movement
11
"A
"B
"C
A
B
C
Snapshot
Blocks
in
LUN
or
File
Blocks
on
the
Disk
A
B
C

12. 12
NetApp Snapshot™
Technology
12
"A
"B
"C
Snapshot
Blocks
on
the
Disk
A
B
C
A
B
C
B1
B1
¡ Take
snapshot
¡ ConMnue
wriMng
data
– Write
data
anywhere
– Does
not
affect
the
snapshot
Blocks
in
LUN
or
File

13. 13
FlexClone
13
"A
"B
"C
Snapshot
Blocks
on
the
Disk
A
B
C
A
B
C
B1
B1
Cloned
Data
A
B
C
¡ Clone
from
snapshot
Blocks
in
LUN
or
File

14. 14
FlexClone™
14
"A
"B1
"C
Snapshot
Blocks
on
the
Disk
A
B
C
A
B
C
B1
B1
A
B1
C
Results:
– Both
parent
and
clone
can
change
independently
– Both
file
systems
share
common
blocks
– Disk
space
is
only
allocated
when
there
are
updates
and/or
addiMons
to
either
the
parent
or
the
clone
volume.
C3
C3
C4
C4
Cloned
Data
Blocks
in
LUN
or
File
¡ Clone
from
Snapshot

15. 15
FlexClone: Rapid thin-provisioned copies
15
6 TB
Database
30 TB Storage
5 full copies
6 TB
Database
Without
FlexClone™
With
FlexClone
Production
Storage
Test & Dev
Storage
6 TB Storage
1 copy, 4 clones

16. 16
Considerations
§ Do not leave abandoned workspaces (CLEAN
UP)
§ If using bisect workflow, use “p4 sync/flush –p” to
prevent spamming the server
§ Limited to 32,767 FlexClones per volume for
Data ONTAP 8.1 and for prior releases it is 255.
§ Use SAN protocols (FCP or iSCSI) for Perforce
Database

17. 17
Compounding Effect:
Snapshot + Dedupe + Snapmirror + Flexclone
Production Site
Full Copy
DR or Remote Site
Full Copy
FlexClone
Copies
Develop
Disk
Backup
DR
Test
Data
Mine
Test
Patches

18. 18
• Quick and easy to set up and take down
• Use your remote site to create workspaces for distributed development.
• Very space efficient
• Reduces bandwidth utilization
• Provide replicated (read-only) copy of Perforce File Depots
• Other uses for FlexClone:
• Performance Testing
• Product Testing
• Diagnosing Issues

19. 19
Demo!
Get your stopwatches out…

20. 20
Other uses for read-write snapshots
§ Basic backup and recovery
§ HA/DR planning (conference session)
§ ‘Shelving’ a workspace

21. 21
Alternatives
§ ICManage Views (conference session)
§ ZFS snapshots
§ Using hard links to reference workspace

22. 22
P4Ideax
Forums
Contact
Randy DeFauw: rdefauw@perforce.com, @rdefauw