Year in Review: 
Perforce 2014 Product Updates 
Charles McLouth 
Director of Solutions Engineering
Presented By: 
Charlie McLouth 
Director of Solutions Engineering 
Charlie has over 15 years of 
experience with technologies such 
as RDBMS, web applications, back 
office applications, and web services. 
He has held positions ranging from 
Developer to Product Manager 
before joining Perforce 8 years ago.
Agenda 
• Server Optimizations 
• Features for End Users 
• Collaboration Features with Swarm
SERVER 
OPTIMIZATIONS
Server Performance and Scalability 
• Concurrency Optimizations – aka Lockless 
Reads 
• Federated Architecture for Distributed Teams 
• Clustering for Horizontal Scalability 
• Clustering for High Availability 
• Sync Performance 
SERVER OPTIMIZATIONS
Concurrency – aka Lockless Reads 
SERVER OPTIMIZATIONS 
Writer 
Reader 
Database 
p4 submit 
Lockless Reads 
Writer does not block 
Reader 
p4 changes 
✗
VMware 
Before After 
Total Commands 3,518,462 3,211,786 
Total write-wait-time 58,468s 
Total read-wait-time 1,033,957s 
SERVER OPTIMIZATIONS 
VMware with 5,000 users 
(24 hour period) 
3,430s 
992s 1000X 
17X
VMware – Command Delays 
Commands affected, before Lockless 
Commands affected, after Lockless 
SERVER OPTIMIZATIONS 
1,749 
0 
Commands affected by delays 
3,000 
2,500 
2,000 
1,500 
1,000 
500 
0 
Seconds of 
delay 
1 2 4 8 16 32 64 128 256
Federated Architecture for Distributed Teams 
• Commit/Edge modes for server 
• Local metadata with edge 
services 
• Submits propagated to commit 
service 
• Other commands execute locally 
Commit 
Edge Edge 
SERVER OPTIMIZATIONS
Performance Gains with Federation 
56.6 
8.7 
61 
9.8 
0.9 0.7 1.8 0.2 
70 
60 
50 
40 
30 
20 
10 
0 
Sync Revert Integrate Edit 
Direct Edge 
Perforce Lab Benchmarks SERVER OPTIMIZATIONS
Clustering for Horizontal Scalability 
• Distribute workload to multiple nodes 
• Better performance and experience for all users 
From This… 
Activity 
100% 
P4D 
To This… 
User Activity System Activity User Activity User Activity 
2% 
Commit 
34% 33% 33% 
Workspace 
Server 
Workspace 
Server 
Workspace 
Server 
SERVER OPTIMIZATIONS
Horizontal Scalability Results 
Commit 
WS WS WS WS 
WS WS WS WS 
• 100,000 simulated developers 
• 742 commands/sec 
• 8 node cluster 
SERVER OPTIMIZATIONS
Clustering – Failover 
Router 
Depot 
Master 
P4D P4D 
SERVER OPTIMIZATIONS 
NAS 
P4D 
Depot 
Master 
Standby 
LAN
Sync Performance 
• Sync Performance Increased 
– Network window size optimization 
– Transfer multiple times at a time – aka Parallel Sync 
SERVER OPTIMIZATIONS
Network Window Size Optimization 
SERVER OPTIMIZATIONS
Network Window Size Optimization 
SERVER OPTIMIZATIONS
Parallel Sync 
SERVER OPTIMIZATIONS
Sync Performance Results 
• Network window size optimization 
– 1/16th the amount of network round trips 
• Parallel Sync 
– 1/4th the amount of time 
• Results of both 
– Syncs up to 20 times faster 
SERVER OPTIMIZATIONS
LDAP For User Authentication 
• Perforce provides Authentication Trigger entry points for lookup into 
third party user database (including LDAP) 
• auth_check trigger works well, but ... 
– Needs to be installed separately 
– No standard (Python, Perl, C++ implementations) 
– One more headache for administrators 
• Most common request on P4Ideax: 
– Perforce should provide built-in LDAP integration 
SERVER OPTIMIZATIONS
Native LDAP support 
LDAP 
SERVER OPTIMIZATIONS 
p4 login 
user-login 
client-Prompt 
Enter Password: 
<password> dm-login 
LDAP Query 
<accepted> 
client-SetPassword 
User logged in.
Native LDAP support 
• Increases Security 
– Consistent Authentication and Authorization 
• Simplifies Policy Management 
– Access granted/revoked by LDAP group membership 
SERVER OPTIMIZATIONS
P4Search 
SERVER OPTIMIZATIONS 
??? 
File names, 
Changes ... 
File content? 
C# 
.h 
JAVA 
PPTX 
PDF
Solution: External Index 
P4 Search 
SERVER OPTIMIZATIONS 
store 
search 
Search engine indexes content 
Stores it in its own database 
Users search the index first 
Index returns a depot path 
Permissions applied to index 
results 
Returns results to user
P4Search 
• File content 
• P4 Metadata 
• P4 attributes 
• And the common 
metadata if 
desired 
SERVER OPTIMIZATIONS
END USERS
P4V – 2014 New Features 
• User interface refresh 
• Remove excess baggage 
• Simplify experience 
• Faster rendering 
• Faster for large file sets 
• “Snappier” responsiveness 
END USERS
P4V – 2014 New Features 
END USERS
P4V – 2014 New Features 
END USERS
P4V – 2014 New Features 
END USERS
Streams 
• Model the flow of change for 
the entire product lifecycle 
• Visual roadmap for users 
• Remove the fear/overhead 
of branching and merging 
END USERS
Streams – 2014 Features 
• Streams allows “import” 
paths to include 
components from one 
Stream into another. 
– Import is by reference (not a 
END USERS 
copy) 
– Import path is Read Only
Streams – 2014 Features 
Top Requests of Streams to 
facilitate managing Components 
with Streams: 
– “I want to lock an import a 
specific version of my 
component.” 
– “I want to submit changes to a 
referenced component.” 
• Makes managing components 
END USERS 
simpler
Streams – Frozen Imports 
Paths: 
import import/... //depot/import/…@1000 
• Freezes the import path (component) at a point in time. 
• p4 sync import/…#head behaves like p4 sync import/…@1000 
END USERS
Streams – Writeable Imports 
Paths: 
import+ import/... //depot/import/… 
• Functions like standard import, enabling you to map a path or 
Component from outside the stream. 
• But unlike a standard import path, you can submit changes to the 
files in an import+ path. 
END USERS
Streams – 2014 Features 
• Frozen Imports and Writeable 
Imports 
– Simplifies the management of 
Components within a Stream 
35END USERS
SWARM 
COLLABORATION
Swarm – 2014 Features 
• Review Votes 
• Required Reviewers 
• Lightweight Tasks 
• Attachments 
• API 
SWARM COLLABORATION
Swarm – 2014 Features 
SWARM COLLABORATION
Swarm – 2014 Features 
SWARM COLLABORATION
Swarm – 2014 Features 
SWARM COLLABORATION
Swarm – 2014 Features 
SWARM COLLABORATION
Swarm - Comment Attachments 
SWARM COLLABORATION
Swarm – Rest API 
SWARM COLLABORATION
Questions? 
Charlie McLouth 
cmclouth@perforce.com 
• Resources 
• Videos of all MERGE 2014 presentations: 
perforce.com/MERGE 
• Sign up for Monthly Product Updates: 
perforce.com/communication-center 
• Blog posts from Perforce product teams: 
perforce.com/blog

Year in Review: Perforce 2014 Product Updates

  • 1.
    Year in Review: Perforce 2014 Product Updates Charles McLouth Director of Solutions Engineering
  • 2.
    Presented By: CharlieMcLouth Director of Solutions Engineering Charlie has over 15 years of experience with technologies such as RDBMS, web applications, back office applications, and web services. He has held positions ranging from Developer to Product Manager before joining Perforce 8 years ago.
  • 3.
    Agenda • ServerOptimizations • Features for End Users • Collaboration Features with Swarm
  • 4.
  • 5.
    Server Performance andScalability • Concurrency Optimizations – aka Lockless Reads • Federated Architecture for Distributed Teams • Clustering for Horizontal Scalability • Clustering for High Availability • Sync Performance SERVER OPTIMIZATIONS
  • 6.
    Concurrency – akaLockless Reads SERVER OPTIMIZATIONS Writer Reader Database p4 submit Lockless Reads Writer does not block Reader p4 changes ✗
  • 7.
    VMware Before After Total Commands 3,518,462 3,211,786 Total write-wait-time 58,468s Total read-wait-time 1,033,957s SERVER OPTIMIZATIONS VMware with 5,000 users (24 hour period) 3,430s 992s 1000X 17X
  • 8.
    VMware – CommandDelays Commands affected, before Lockless Commands affected, after Lockless SERVER OPTIMIZATIONS 1,749 0 Commands affected by delays 3,000 2,500 2,000 1,500 1,000 500 0 Seconds of delay 1 2 4 8 16 32 64 128 256
  • 9.
    Federated Architecture forDistributed Teams • Commit/Edge modes for server • Local metadata with edge services • Submits propagated to commit service • Other commands execute locally Commit Edge Edge SERVER OPTIMIZATIONS
  • 10.
    Performance Gains withFederation 56.6 8.7 61 9.8 0.9 0.7 1.8 0.2 70 60 50 40 30 20 10 0 Sync Revert Integrate Edit Direct Edge Perforce Lab Benchmarks SERVER OPTIMIZATIONS
  • 11.
    Clustering for HorizontalScalability • Distribute workload to multiple nodes • Better performance and experience for all users From This… Activity 100% P4D To This… User Activity System Activity User Activity User Activity 2% Commit 34% 33% 33% Workspace Server Workspace Server Workspace Server SERVER OPTIMIZATIONS
  • 12.
    Horizontal Scalability Results Commit WS WS WS WS WS WS WS WS • 100,000 simulated developers • 742 commands/sec • 8 node cluster SERVER OPTIMIZATIONS
  • 13.
    Clustering – Failover Router Depot Master P4D P4D SERVER OPTIMIZATIONS NAS P4D Depot Master Standby LAN
  • 14.
    Sync Performance •Sync Performance Increased – Network window size optimization – Transfer multiple times at a time – aka Parallel Sync SERVER OPTIMIZATIONS
  • 15.
    Network Window SizeOptimization SERVER OPTIMIZATIONS
  • 16.
    Network Window SizeOptimization SERVER OPTIMIZATIONS
  • 17.
    Parallel Sync SERVEROPTIMIZATIONS
  • 18.
    Sync Performance Results • Network window size optimization – 1/16th the amount of network round trips • Parallel Sync – 1/4th the amount of time • Results of both – Syncs up to 20 times faster SERVER OPTIMIZATIONS
  • 19.
    LDAP For UserAuthentication • Perforce provides Authentication Trigger entry points for lookup into third party user database (including LDAP) • auth_check trigger works well, but ... – Needs to be installed separately – No standard (Python, Perl, C++ implementations) – One more headache for administrators • Most common request on P4Ideax: – Perforce should provide built-in LDAP integration SERVER OPTIMIZATIONS
  • 20.
    Native LDAP support LDAP SERVER OPTIMIZATIONS p4 login user-login client-Prompt Enter Password: <password> dm-login LDAP Query <accepted> client-SetPassword User logged in.
  • 21.
    Native LDAP support • Increases Security – Consistent Authentication and Authorization • Simplifies Policy Management – Access granted/revoked by LDAP group membership SERVER OPTIMIZATIONS
  • 22.
    P4Search SERVER OPTIMIZATIONS ??? File names, Changes ... File content? C# .h JAVA PPTX PDF
  • 23.
    Solution: External Index P4 Search SERVER OPTIMIZATIONS store search Search engine indexes content Stores it in its own database Users search the index first Index returns a depot path Permissions applied to index results Returns results to user
  • 24.
    P4Search • Filecontent • P4 Metadata • P4 attributes • And the common metadata if desired SERVER OPTIMIZATIONS
  • 25.
  • 26.
    P4V – 2014New Features • User interface refresh • Remove excess baggage • Simplify experience • Faster rendering • Faster for large file sets • “Snappier” responsiveness END USERS
  • 27.
    P4V – 2014New Features END USERS
  • 28.
    P4V – 2014New Features END USERS
  • 29.
    P4V – 2014New Features END USERS
  • 30.
    Streams • Modelthe flow of change for the entire product lifecycle • Visual roadmap for users • Remove the fear/overhead of branching and merging END USERS
  • 31.
    Streams – 2014Features • Streams allows “import” paths to include components from one Stream into another. – Import is by reference (not a END USERS copy) – Import path is Read Only
  • 32.
    Streams – 2014Features Top Requests of Streams to facilitate managing Components with Streams: – “I want to lock an import a specific version of my component.” – “I want to submit changes to a referenced component.” • Makes managing components END USERS simpler
  • 33.
    Streams – FrozenImports Paths: import import/... //depot/import/…@1000 • Freezes the import path (component) at a point in time. • p4 sync import/…#head behaves like p4 sync import/…@1000 END USERS
  • 34.
    Streams – WriteableImports Paths: import+ import/... //depot/import/… • Functions like standard import, enabling you to map a path or Component from outside the stream. • But unlike a standard import path, you can submit changes to the files in an import+ path. END USERS
  • 35.
    Streams – 2014Features • Frozen Imports and Writeable Imports – Simplifies the management of Components within a Stream 35END USERS
  • 36.
  • 37.
    Swarm – 2014Features • Review Votes • Required Reviewers • Lightweight Tasks • Attachments • API SWARM COLLABORATION
  • 38.
    Swarm – 2014Features SWARM COLLABORATION
  • 39.
    Swarm – 2014Features SWARM COLLABORATION
  • 40.
    Swarm – 2014Features SWARM COLLABORATION
  • 41.
    Swarm – 2014Features SWARM COLLABORATION
  • 42.
    Swarm - CommentAttachments SWARM COLLABORATION
  • 43.
    Swarm – RestAPI SWARM COLLABORATION
  • 44.
    Questions? Charlie McLouth cmclouth@perforce.com • Resources • Videos of all MERGE 2014 presentations: perforce.com/MERGE • Sign up for Monthly Product Updates: perforce.com/communication-center • Blog posts from Perforce product teams: perforce.com/blog