Git: The Lean, Mean, Distributed Machine

Git: The Lean, Mean, Distributed Machine - Document Transcript

Chris Wanstrath http://defunkt.github.com hi everyone, i’m chris wanstrath. how many people here use git?

i play guitar i have a schecter classic similar to this. mine is prettier.

i’m from cincinnati ohio

but live in san francisco

i started as a lowly paid consultant

Then I worked at CNET then worked at CNET for a few years

Then I worked at CNET (which is now owned by CBS)

Then I was a highly paid consultant after that i became a highly paid consultant

before co-founding github

git The Lean Mean Distributed Machine By Chris Wanstrath anyway, i want to talk a bit about git today

if we’re going to talk about git, we need to start by talking about source control management (SCM) or version control

basically version control is like wikipedia

for your code

you use it to see what changes others made

inspect those changes

and contribute your own

git? who uses token promo slide

these companies use git

and so do these open source projects

let’s brieﬂy go through the history of open source SCMs (at least the ones we care about)

Revision Control System (RCS) was written in the early 80s and used to store history on a file by file basis. A directory of source code could contain many RCS repositories, each concerning itself with a single file. it’s like a hut - very basic, primitive even, but works great when all you need is shelter Later that decade, a professor began working with two grad students on a C compiler (in the name of scholarly pursuits). As they began using RCS, the professor noted a number of limitations. It was difficult to share files, and even more difficult to share entire projects.

So, they wrote CVS - the Concurrent Versioned System - and released it as open source in the early 1990s. Concurrent because it allowed multiple individuals to collaborate on a project together, without stepping on each other's toes, and versioned system because it was initially a collection of RCS repositories with network awareness. CVS was like a cabin. better than a hut, but still pretty crappy it worked well for a while, but there were limitations. Dealing with directories was difficult, and much different than the way one would normally deal with directories in Unix.

Ten years later (see a pattern?) a new revision control system was released, called Subversion (or SVN). Subversion was intended to replace CVS by improving on CVS. History, directories, deletions, and other CVS warts were ﬁxed. mod_dav integration was included, as well as anonymous checkout. (Anonymous checkout in CVS was literally a hack added by the OpenBSD.) Subversion was not subversive, but it did work well enough. Many felt it a welcome relief and hurriedly switched. Big, lumbering organizations spent years converting their repositories to SVN. IDEs and editors included Subversion integration. it was like a house, same idea but much better than a cabin

Server Committer Committer Committer this is the rcs / cvs / svn model.

Server Committer Committer Committer someone commits to the server

Server Committer Committer Committer and everyone else pulls down the changes

this is bad why?

Server ﬁrst off, the server is the babysitter

“The Subversion ser ver’s down” SVN’s down! you can’t do anything without the server’s permission

second, low visibility into your coworkers and subordinates’ activity.

at cnet, we used bugzilla. it was awesome (as you can see)

we’d also get diffs emailed to us after each commit

because our group was large, and ﬂuid, i’d often get commits emailed to me i didnt care about

or understand

this meant i spent extra time throwing away junk

emails that come to me should be for me

Trac if you’re lucky, you use trac to watch what everyone is doing. it has rss and is a bit smarter. usually you have to set it up yourself

another problem: your subversion workflow is single threaded it’s hard to stop working on a feature to quickly fix a bug without losing your feature’s changes changed files are either committed or discarded well, not entirely true...

you could always use a branch but that sucks, and big changes are usually disasters i once worked on a project for 3 months where we worked on a massive new feature

as it neared completion, we had a big meeting

a 2 hour meeting

there we decided how to merge in the changes from our branch to trunk it was one of the worst meetings ever

and i’ve been in some bad meetings

one person was assigned with the task of merging the branch and trunk

he was merging and ﬁxing bugs in code he did not write

or understand

it did not go well

another problem with the ‘babysitter’ model is that experimentation is difficult

all your experiments are public

everyone sees everything you commit

solution? don’t commit at least, that was my solution

and that’s just work stuff the centralized model, when applied to open source, is a huge pain it’s difficult to maintain patched versions of open source independently if a project dies on the internet, does anyone care?

what’s the answer? dvcs! git! happy!

git was started by linus torvalds a DVCS built as an efficient content addressable ﬁle system

the guy who started the linux kernel

if rcs is a hut

cvs is a cabin

and subversion is a house

git is a castle

or a ninja

or shaq

Server Committer Committer Committer the thing that makes git, and all distributed version control systems like it different, is the idea that it’s “distributed” take this centralized model

Server Committer Committer Committer and make every copy of the code its own, full ﬂedged repository any copy can accept or create commits. anyone can pull commits from any copy.

Server Committer Committer Committer now you can push to the server

Server Committer Committer Committer or committers can push and pull from each other

Server Committer instead of “checking out” code

Server Committer you “clone” or copy a repository

if github explodes, you don’t lose any code

Server Server Server Committer in fact, because you have a full copy of your repository at all times, you don’t need to tie yourself to a single remote repository

you can push to as many servers as you want

another word for ‘clone’ or ‘copy’ is ‘fork’

it may seem like anarchy at ﬁrst but sane and useful workﬂows have evolved

#1 in fact, the ﬁrst workﬂow i want to talk about is called Anarchy

Server Committer Committer Committer you remove the server

Committer Committer Committer then make everyone a peer

Committer Committer Committer then take away commit access from each other

Coder Coder Coder and you end up with repositories ﬂoating in the void this is how the internet works, or how git works by default

Coder Coder Coder everyone pushes and pulls from each other, managing their own version of the code for small projects with very few contributors, it works ﬁne and it would work great on small, experimental projects inside of any organization (you just need a place to publish your changes)

an example of this, let’s say i was on github and i wanted to add a patch to schacon’s ticgit

i’d click the fork button

i now have a copy of schacon’s ticgit called defunkt’s ticgit now i can make changes and ask scott to check them out. if he likes them, he’ll merge them in if he doesn’t like them, oh well. i can still use them in my project and keep up to date with his changes. someone can come along and fork from me if they want, too

#2 blessed but anarchy doesn’t scale the second workﬂow is called Blessed

Coder Coder Coder Coder the blessed workﬂow has the same basic idea as Anarchy

Blessed Coder Coder Coder but one of the repositories is the Blessed repo

Blessed Coder Coder Coder everyone takes their cues from the blessed repository its development is considered the mainline, or trunk deploys and packages are pushed from the blessed repo

Blessed Coder Coder Coder others can still push and pull from each other, remember

in the business world this works great for dealing with contractors

Blessed Coder Coder Coder don’t give them push access, just pull access

Blessed Coder they pull down your code, make their changes, then tell you when the changes are ready

Blessed Coder if you like what you see, you merge in the contractor’s changes the contractor never has direct write access to your company’s code

if there were a bunch of us working on ticgit, scott’s may be the Blessed repository he started the project and is in charge of merging in all changes. we all watch his changes

this is how rails works rails/rails is the Blessed repository, from which the gems are built and david controls. we all follow this repo’s development and treat it as “official” by convention only

this is also how rentzsch’s click to flash works click to flash is an amazing safari plugin that disables flash, similar to the firefox extension

it was forked from google code and has been given a life of its own on github, under rentzsch’s guidance. contributors fork his repository and he merges in good changes. the plugin’s development has been a perfect example of how distributed version control puts the power in the hands of the developer, not the server

#3 lieutenant the next workﬂow is called ‘lieutenant’ - great for massive projects, like the kernel

Blessed Lieutenant Lieutenant Coder Coder Coder Coder in this model, there is a blessed repository and a few designated lieutenants the lieutenants are people trusted by the blessed repository

Blessed Lieutenant Lieutenant Coder Coder Coder Coder coders will pull from a lieutenant, make their changes, then request the lieutenant merge in their changes

Blessed Lieutenant Lieutenant Coder Coder Coder Coder lieutenants are usually in charge of a speciﬁc subsystem or part of the large system if they like the change, they will pull it in

Blessed Lieutenant Lieutenant Coder Coder Coder Coder they’ll then inform the blessed repository that they have changes which need to be merged in the blessed repo, trusting the lieutenant, pulls in the changes

Blessed Lieutenant Lieutenant Coder Coder Coder Coder this is typically coordinated over a mailing list

this is the kernel’s model

#4 centralized ﬁnally the centralized model, one repository acts as the ‘server’

Server Committer Committer Committer this mimics the traditional babysitter model but you only need your babysitter to pull and push changes - branches and commits can still be created locally whenever

Server Committer Committer Committer the server being down does not dramatically hamper your work

Server Committer Committer Committer yet the ﬂow stays mostly the same

Deploy Server Committer Committer Committer the central server, as in the old model, can also be used to deploy

Staging Production Committer Committer Committer and staging servers can easily be setup

git isn’t all about distributed servers in fact, one of the best parts about git is its branching support

branches are local, incredibly lightweight, and easy to switch between it’s easy to devote each branch to a single feature or bug we call these ‘topic branches’

$ git checkout -b bug_2342 from your working directory you just made a new branch

buckets of different things because branches are so cheap, you can keep around buckets ﬁlled with experiments, new ideas, or new features no one will ever see them unless you want them to be seen

Staging Production Committer Committer Committer this changes the staging server idea

Feature A Production Feature B Committer Committer Committer you may start to have topic staging servers where you boot up staging for a single branch and test out a new feature, no more generic ‘staging’ branch - each person may even have their own staging server

Coder because every copy is its own repository, we are given the freedom to structure our workﬂows socially rather than technically

Team A Team B it’s easy to have multiple small teams, move people between projects, and monitor multiple projects no need for one monolithic subversion server - git repositories are a breeze to setup

with something like github, watching your team’s development is trivial. do it with rss...

or with a service you’re already comfortable using integration with campﬁre, email, fogbugz, lighthouse, friendfeed, twitter, etc

the site also lets you comment on commits, providing dead simple and effective code review git and github are what we use in our private client work and on our own websites, as well as for our open source

as far as git IDE support, the textmate ProjectPlus extension shows you the status of tracked ﬁles right in the drawer

there’s also a git textmate bundle available on github

if you’re an eclipse user, the egit plugin lets you commit to, manage, and track git repositories from within eclipse it’s written using jgit, a pure-java implementation of git

emacs people can use DVC which aims to provide a common interface for all distributed version control systems there’s also a git mode

or my person favorite, magit

if you use os x, an open source program called GitX is under active development

which is based on the cross platform Git-GUI

for OS X there’s also GitNub which isn’t as actively developed

as far as libraries go, a search for ‘git’ on github returns almost 3000 unique repositories darcs or hg to git converters, git vim projects, git in .NET, even blogs and wikis based on git

remember when i said git was a content addressable ﬁle system? well, it’s true this is gist. it’s a git powered pastie

you paste in code and share it with coworkers or friends

but the best part are these clone URLs i can check out a pastie i made, make changes, then push a new version

these are the revisions you can’t tell the difference between changes i made on the web and changes i made locally then pushed

this kind of stuff is the future imagine a distributed, versioned wiki or documentation project or book a distributed, versioned bug tracker a distributed, versioned chat application

in fact, a number of books are already being written on github

scott’s book is being translated right now

a distributed, versioned everything

http://git-scm.com this has been a fairly basic overview of git for more information, check out git-scm.com

thanks questions?

http://flickr.com/photos/baonguyen/2557970434/ http://flickr.com/photos/dooleymtv/2660521051/ http://flickr.com/photos/bocavermelha/66759796/ http://flickr.com/photos/shankrad/219945665/ http://flickr.com/photos/dark-o/408532292/ http://flickr.com/photos/thomashawk/268524287/ http://flickr.com/photos/vividbreeze/480057824/ http://flickr.com/photos/drp/41370809/ http://flickr.com/photos/12693492@N04/1338136415/ http://flickr.com/photos/keithmarshall/432924465/ http://flickr.com/photos/theducks/2236111019/ http://flickr.com/photos/ivviphotography/2555026221/ http://flickr.com/photos/pfenwick/2229585929/ http://flickr.com/photos/41232123@N00/457119628/ http://flickr.com/photos/raybyrne/43604789/ http://flickr.com/photos/pedacitosdemi/2346980097/ http://flickr.com/photos/robertlz/481261885/ http://flickr.com/photos/aleks/2759584309/ http://flickr.com/photos/petervanallen/774522321/ http://flickr.com/photos/bigfrank/71691236/ http://flickr.com/photos/seantubridy/389310649/ http://flickr.com/photos/chinapix/2757393280/ http://flickr.com/photos/millzero/705902956/ http://flickr.com/photos/28960190@N03/2766239909/ http://flickr.com/photos/larskflem/95757299/ http://flickr.com/photos/68226666@N00/444253460/ http://flickr.com/photos/dc5dugg/2330760034/ http://flickr.com/photos/chilledsalad/2335688523/ http://flickr.com/photos/will-lion/2629598996/ http://flickr.com/photos/thefuzz90/444183705/ http://flickr.com/photos/probablykat/377911954/ http://flickr.com/photos/domk/291808114/ http://flickr.com/photos/pixelbuffer/5375185/ http://flickr.com/photos/csb13/66558459/ http://flickr.com/photos/gordonhamilton/2096653379/ http://flickr.com/photos/kalhusoru/1346688882/ http://flickr.com/photos/poolie/2250698836/ http://flickr.com/photos/heather_shade/254728862/ http://flickr.com/photos/ostromentsky/479147675/ http://flickr.com/photos/doergn/425407884/ http://flickr.com/photos/flopper/1336613530/ http://flickr.com/photos/miscellaneous/531864821/ http://flickr.com/photos/jagelado/16631508/ http://flickr.com/photos/hchalkley/30724738/ http://flickr.com/photos/ansy/2533871446/ http://flickr.com/photos/absence-is-steel/419515663/ http://flickr.com/photos/cdm/54246114/ http://flickr.com/photos/samiksha/408007916/ http://flickr.com/photos/chibnall/2278573838/ http://flickr.com/photos/keithallison/3230928864/ flickr

http://www.flickr.com/photos/andrer69/246847221/ http://www.flickr.com/photos/willstotler/758104340/ flickr