How To Make Git Scale Effectively

How to Make Git Scale
Effectively

2© Perforce Software Inc. All Rights Reserved.
What We’re Talking About
1
Git Overview
2
Perforce Technology
3
Solution Mapping

• Popular
• Local operations can be fast
• Lots of publicly clone-able code
• Distributed
• Great for small projects that
are mostly code files
What’s So Great About Git?

• Large binaries are problematic
• Repo sprawl
• Mixed-team composition
• Build + CI
• LFS
• Dependencies
• Distance
Git at Large Scale

• New depot type (graph) for storing
tree-based git commits, objects, etc.
• Simple grant-based permission model for
single repos and collections
• Supports many “classic” style p4
commands: sync, edit, submit, merge, etc.
• Includes git-lfs support
• Git users connect to Helix Core Server
(P4D) via Git Connector
• Depot/repo/project management and code
review through Helix TeamHub
What Is Helix4Git?

• Supports SSH/HTTP(S) authentication
• Authorizations are all handled by central
Helix Core Server (P4D)
• Translates between user’s git pull/push
to depot
• Keeps local cache of native git repo
• Git developers don’t even know it’s there
What Is Git Connector?

• Git Connector and Helix4Git
have built-in mirroring
• Mirrored Helix4Git repos are
protected from normal commits
• Easily integrates with
Gerrit workflow
• Includes GitHub/GitLab
mirroring capabilities
Mirroring Repos and CI

• Directly supported in the
Helix4Git repo
• Same local git-lfs configuration
• Stored in archive files on the
Helix Core Server (P4D)
• Fully accessible by p4 clients
(sync and edit)
Git LFS

• Clients – a view into your
repo data
• Helix Core Server’s clients
are a lot like git shallow
clones
• Client views can span:
• Depots
• Repos
• Depot types (classic,
streams, and graph)
Graph Clients

CI: 40-80% Faster Builds; 18% Less Storage
All tests performed with shallow clone of Linux kernel on a 1 Gbps link, and four
parallel threads of p4 sync. For WAN test, a 200 ms round-trip latency was added
between client and server.
Sync Linux kernel head
revision over WAN
Sync Linux kernel head
revision over LAN
Resulting sync sizes
0
10
20
30
40
50
Git Clone P4 Sync
796
MB
650
MB
43 (s)
13 (s)
47 (s)
25 (s)

• Faster build sync times
with parallel sync enabled
• Dedicated build replicas to
reduce master overhead
• Can pull all project
components in single client
• Integration with GitLab,
GitHub, and Gerrit
Build + CI

• A single Helix4Git depot can
contain all repos for a single
project
• Centralized management of
repos and authorization
• Centralized management of
externally sourced project
components
• Helix TeamHub for
management of projects
Repo Sprawl

• Keep all assets for your mobile/web
app in a single project in TeamHub
Multi-Repo Example — Helix TeamHub
Open source app
framework code
My mobile app
source code
Graphics for my UI
Dependencies for
Android platform
Project docs

• Code developers just want to pull
and develop
• Use git local and pull/push to
Helix4Git repo
• Digital asset editors don’t need
to build
• They need to find assets and edit them,
but have it pull-able through LFS
• Build team pulls it all together
• Multiple repos, some external,
controllable inputs to the build system
Mixed-Team Composition

• Any significant project using git
for VCS is using multiple repos
• Client/server
• Frontend/backend
• Microservices
• Libraries and versions
• Philosophy of component
dev = be backwards/forwards
compatible
Project Dependencies

Multi-Repo Merge

• Helix TeamHub has the only
multi-repo code review in market
• Synchronize atomic commits
across multiple repos
• Keep frontend and backend
changes aligned
• Stop intermittent build failures
• Pre-flight changes to production
Multi-Repo Merge

• p4 sync is your
best friend
• Custom clients can map
and re-map how you want
• Allows commits that are
fully git-compatible
• Git developers keep using
the tool that makes them
efficient
Large Projects

• Co-locate Git Connectors
with your teams
• Set up pull-polling to reduce
developer hang time
• Multiple Git Connectors to
spread the load
• Build replica servers to
decrease master load
Geography

Recap
Problem Solution
Large binaries are problematic LFS/p4 sync
Repo sprawl Centralized commit server + mirroring
Mixed-team composition Git+ LFS + p4 sync
Build Parallel sync, replicas
CI Mirroring
LFS Out-of-the-box support + p4 sync
Dependencies Multi-repo code review + merge
Distance
Inherent Git Connector replication,
federated servers

Questions?
Thanks for attending!

Follow us for news and insights!
Visit www.perforce.com

How To Make Git Scale Effectively

Recommended

Recommended

More Related Content

More from Perforce

More from Perforce (20)

Recently uploaded

Recently uploaded (20)

How To Make Git Scale Effectively