CI User Experience Spec

Continuous Integration — User Experience Specification

Continuous Integration is a software development practice
where members of a team integrate their work frequently,
usually each person integrates at least daily - leading to
multiple integrations per day. Each integration is verified by
an automated build (including test) to detect integration
errors as quickly as possible. Many teams find that this
approach leads to significantly reduced integration
problems and allows a team to develop cohesive software
more rapidly. — Martin Fowler

Continuous

Program Manager User Education
Jim Lamb Sirkku Willie

Development Lead User Experience
Buck Hodges Kristoffer Schultz

Test Lead International
Jason Barile Aldo Donetti

Microsoft Visual Studio – Team Foundation Server | Team Build
May 15, 2007 1/19


Microsoft Corporation Technical Documentation License Agreement (Standard)
READ THIS! THIS IS A LEGAL AGREEMENT BETWEEN MICROSOFT CORPORATION (quot;MICROSOFTquot;) 3. These Materials may contain preliminary information or inaccuracies, and may not correctly
AND THE RECIPIENT OF THESE MATERIALS, WHETHER AN INDIVIDUAL OR AN ENTITY (quot;YOUquot;). IF represent any associated Microsoft Product as commercially released. All Materials are provided
YOU HAVE ACCESSED THIS AGREEMENT IN THE PROCESS OF DOWNLOADING MATERIALS entirely quot;AS IS.quot; To the extent permitted by law, MICROSOFT MAKES NO WARRANTY OF ANY KIND,
(quot;MATERIALSquot;) FROM A MICROSOFT WEB SITE, BY CLICKING quot;I ACCEPTquot;, DOWNLOADING, USING OR DISCLAIMS ALL EXPRESS, IMPLIED AND STATUTORY WARRANTIES, AND ASSUMES NO LIABILITY TO
PROVIDING FEEDBACK ON THE MATERIALS, YOU AGREE TO THESE TERMS. IF THIS AGREEMENT IS YOU FOR ANY DAMAGES OF ANY TYPE IN CONNECTION WITH THESE MATERIALS OR ANY
ATTACHED TO MATERIALS, BY ACCESSING, USING OR PROVIDING FEEDBACK ON THE ATTACHED INTELLECTUAL PROPERTY IN THEM.
MATERIALS, YOU AGREE TO THESE TERMS.
4. If You are an entity and (a) merge into another entity or (b) a controlling ownership interest in You
1. For good and valuable consideration, the receipt and sufficiency of which are acknowledged, You changes, Your right to use these Materials automatically terminates and You must destroy them.
and Microsoft agree as follows:
5. You have no obligation to give Microsoft any suggestions, comments or other feedback
(a) If You are an authorized representative of the corporation or other entity designated below (quot;Feedbackquot;) relating to these Materials. However, any Feedback you voluntarily provide may be used
(quot;Companyquot;), and such Company has executed a Microsoft Corporation Non-Disclosure Agreement in Microsoft Products and related specifications or other documentation (collectively, quot;Microsoft
that is not limited to a specific subject matter or event (quot;Microsoft NDAquot;), You represent that You Offeringsquot;) which in turn may be relied upon by other third parties to develop their own Products.
have authority to act on behalf of Company and agree that the Confidential Information, as defined in Accordingly, if You do give Microsoft Feedback on any version of these Materials or the Microsoft
the Microsoft NDA, is subject to the terms and conditions of the Microsoft NDA and that Company Offerings to which they apply, You agree: (a) Microsoft may freely use, reproduce, license, distribute,
will treat the Confidential Information accordingly; and otherwise commercialize Your Feedback in any Microsoft Offering; (b) You also grant third
parties, without charge, only those patent rights necessary to enable other Products to use or
(b) If You are an individual, and have executed a Microsoft NDA, You agree that the Confidential
interface with any specific parts of a Microsoft Product that incorporate Your Feedback; and (c) You
Information, as defined in the Microsoft NDA, is subject to the terms and conditions of the Microsoft
will not give Microsoft any Feedback (i) that You have reason to believe is subject to any patent,
NDA and that You will treat the Confidential Information accordingly; or
copyright or other intellectual property claim or right of any third party; or (ii) subject to license terms
(c) If a Microsoft NDA has not been executed, You (if You are an individual), or Company (if You which seek to require any Microsoft Offering incorporating or derived from such Feedback, or other
are an authorized representative of Company), as applicable, agrees: (a) to refrain from disclosing or Microsoft intellectual property, to be licensed to or otherwise shared with any third party.
distributing the Confidential Information to any third party for five (5) years from the date of
6. Microsoft has no obligation to maintain confidentiality of any Microsoft Offering, but otherwise the
disclosure of the Confidential Information by Microsoft to Company/You; (b) to refrain from
confidentiality of Your Feedback, including Your identity as the source of such Feedback, is governed
reproducing or summarizing the Confidential Information; and (c) to take reasonable security
by Your NDA.
precautions, at least as great as the precautions it takes to protect its own confidential information,
but no less than reasonable care, to keep confidential the Confidential Information. You/Company, 7. This agreement is governed by the laws of the State of Washington. Any dispute involving it must
however, may disclose Confidential Information in accordance with a judicial or other governmental be brought in the federal or state superior courts located in King County, Washington, and You waive
order, provided You/Company either (i) gives Microsoft reasonable notice prior to such disclosure and any defenses allowing the dispute to be litigated elsewhere. If there is litigation, the losing party must
to allow Microsoft a reasonable opportunity to seek a protective order or equivalent, or (ii) obtains pay the other party’s reasonable attorneys’ fees, costs and other expenses. If any part of this
written assurance from the applicable judicial or governmental entity that it will afford the agreement is unenforceable, it will be considered modified to the extent necessary to make it
Confidential Information the highest level of protection afforded under applicable law or regulation. enforceable, and the remainder shall continue in effect. This agreement is the entire agreement
Confidential Information shall not include any information, however designated, that: (i) is or between You and Microsoft concerning these Materials; it may be changed only by a written
subsequently becomes publicly available without Your/Company’s breach of any obligation owed to document signed by both You and Microsoft.
Microsoft; (ii) became known to You/Company prior to Microsoft’s disclosure of such information to
You/Company pursuant to the terms of this Agreement; (iii) became known to You/Company from a
source other than Microsoft other than by the breach of an obligation of confidentiality owed to
Microsoft; or (iv) is independently developed by You/Company. For purposes of this paragraph,
quot;Confidential Informationquot; means nonpublic information that Microsoft designates as being
confidential or which, under the circumstances surrounding disclosure ought to be treated as
confidential by Recipient. quot;Confidential Informationquot; includes, without limitation, information in
tangible or intangible form relating to and/or including released or unreleased Microsoft software or
hardware products, the marketing or promotion of any Microsoft product, Microsoft's business
policies or practices, and information received from others that Microsoft is obligated to treat as
confidential.
2. You may review these Materials only (a) as a reference to assist You in planning and designing Your
product, service or technology (quot;Productquot;) to interface with a Microsoft Product as described in these
Materials; and (b) to provide feedback on these Materials to Microsoft. All other rights are retained by
Microsoft; this agreement does not give You rights under any Microsoft patents. You may not (i)
duplicate any part of these Materials, (ii) remove this agreement or any notices from these Materials,
or (iii) give any part of these Materials, or assign or otherwise provide Your rights under this
agreement, to anyone else.

May 15, 2007 2/19


Team Explorer
Builds

tfs01
My Favorites
Orcas
+
View Builds
VSTS Dogfood
– Queue New Build…
New Build Definition…
Work Items
+
Documents
+ Manage Build Agents…
Reports
+ Manage Build Qualities...
Builds
–
Properties
All Builds
Rolling
Nightly
Default action
View Builds
On-demand
Queue New Build...
Source Control
Edit Build Definition…

Delete

Add to My Favorites

Properties

May 15, 2007 3/19


Accessible From
New Build Definition The selected tab is drawn using
Team Build Explorer
the selected item background
General and text colors for contrast. Permissions Required
Administer a build

Build Definition — Nightly

Build definition name:
General
Rolling
Workspace
Description (optional):
Project File
This is our continuous integration build triggered automatically by anyone’s
check-in.
Retention Policy

Build Defaults

This description is used in
Trigger
contexts where the user needs
to select a build definition

Disable this build definition (no new builds will be queued)

OK Cancel
This icon indicates that the tab requires input.

Warning icon is used to Clicking the OK button causes
indicate a tab that requires all input across all tabs to be
further input validated prior to submission.

Navigating the New Build Definition UI Enabling and disabling a build definition

This is effectively a tabbed dialog with the tabs running The check box at the bottom of the panel is used to
vertically down the left-hand side. The user can use the enable/disable the build definition. New builds of disabled
Ctrl+Tab and Ctrl+Shift+Tab to navigate forward and build definitions cannot be queued. Disabled build
backward through the tabs, respectively. Alternatively, definitions are also filtered out of the related combo box
they can tab to the list of tabs and use the arrow keys to in the Queue Build dialog.
navigate them (and space or enter to activate a tab).

May 15, 2007 4/19


New Build Definition
All workspace paths are
Workspace relative to this root directory
which may vary across build
agents. The default maps the
entire team project to a
Build Definition — Nightly corresponding folder on the
build agent.
Working Folders:
General
Type Version Control Folder Build Agent Folder
Workspace
Active $/Orcas/TFS $(SourceDir)TFS
Cloak $/Orcas/TFS/Qtools
Project File
Cloak $/Orcas/TFS/Setup
Retention Policy

Build Defaults

Trigger

Invokes an existing dialog to
select a workspace mapping to
Copy Existing Workspace… copy. The selected workspace is
normalized to a common root
directory.
OK Cancel

Defining a Build Workspace

The process of defining a build workspace is nearly
identical to defining a client workspace mapping. The
difference is that each path is mapped to a relative
location beneath a root build directory. This build
directory is a property of each distinct build agent.

May 15, 2007 5/19


Project File


Team Foundation Build uses an MSBuild project file and response file to
General Invokes a pop-up dialog
perform your build. Specify the location in version control to store these files.
to select a folder from
Workspace
version control.
Version control folder:
Project File
$/Orcas/TFS/BuildAll Browse…
Retention Policy
Warning: no TFSBuild.proj file was found in the version control folder
Build Defaults These controls are
specified above. To create a new project file, click Create to run the Build
hidden when the user
Project File Creation Wizard.
Trigger
selects a version control
Create…
path that contains a
TFSBuild.proj file.

Invokes the new build
project file wizard shown
on the next page

OK Cancel

May 15, 2007 6/19


New Build Project File
Selections

The user can no longer
MSBuild Project File Creation Wizard
select a workspace since
they have already defined
the mapping for the build
n

Select and order solutions to build
i co

definition.

Selections
Solutions are listed based on the specified workspace. Selected solutions will be built
sequentially in the order specified below.
Configurations

Options Select and order solutions to build:

(Select All)
HelloWorld ($/HelloWorld)
HelloWorld2 ($/HelloWorld2)

< Back Next > Finish Cancel

May 15, 2007 7/19


Configurations

MSBuild Project File Creation Wizard How is the user supposed to
remove a configuration?

What are the validation
n

Select configurations to build
i co

rules for this page?

Selections
Solutions will be built in the listed configurations
Configurations
Which configurations would you like to build?
Options Configuration Platform
Release Any CPU


May 15, 2007 8/19


Options

MSBuild Project File Creation Wizard

This will still be limited to
n

Select build options
i co

selecting a test metadata
file and a corresponding
set of test lists

Selections
Which build options would you like to include in the build process?
Configurations Run test (e.g. run BVTs, etc.)

Options Test metadata file:
On-demand

Test list to run:
This edit box is disabled if
Build Verification Test
the corresponding
checkbox is unchecked.
When empty, this edit box
will display the following
text italicized and in the
system “disabled text”
color: “For example:
$(OutDir)*test.dll”.
Automatically detect and run tests in the following assemblies:

Semi-colon delimited list of file specifications:
$(OutDir)*test.dll

Returns to the new build
Perform code analysis according to project settings definition wizard


May 15, 2007 9/19


Project File


Team Foundation Build uses an MSBuild project file and response file to
General Invokes a pop-up dialog
perform your build. Specify the location in version control to store these files.
to select a folder from
Workspace
version control.
Version control folder:
Project File
$/Orcas/TFS/BuildAll Browse…
Retention Policy
Found MSBuild project file: TFSBuild.proj
Build Defaults

Trigger

OK Cancel

May 15, 2007 10/19


Retention

Invokes a simple input
box where the user can
specify the number of
builds to retain
Specify how builds should be retained:
General
Build Outcome Retention Policy
Workspace
Keep All
Failed Keep Latest Only
Keep None
Stopped Keep None
Project File
Keep Latest Only
Completed - Test(s) Failed Keep 5 Latest
Succeeded - Test(s) Passed Keep All
Retention Policy Keep 2 Latest
Keep 5 Latest
Build Defaults
Keep 7 Latest
Keep 10 Latest
Trigger
Specify Count to Keep…

Users exempt builds from
Note: Completed builds may be exempted from their associated retention
the build explorer view
policy in the view of builds by selecting Retain Indefinitely from their context
menu.

OK Cancel

Icons are consistent with
those used in the team
build explorer view to
Default Retention Policies
indicate build status.
The default retention policy for all
outcomes is to keep all builds.

May 15, 2007 11/19


Defaults

Defaults
Default Build Agent: if only
Invokes the new build
Optionally, you may specify the build agent and staging location for this one (1) build agent is defined,
General agent dialog box.
build definition. These selections may be modified by the person queuing it will be the default selection;
Leaving this blank
the build. otherwise this will default to
Workspace causes the Queue Build
<blank>
dialog to default to the
Build Agent:
Project File
first idle build agent (if
Mini-lab: Continuous Integration New… Drop Folder: <blank>
any).
Retention Policy
Description:
Build Defaults Read-only edit box
This build agent is used exclusively by the continuous integration feature
displays description of
crew.
Trigger
the selected agent

Builds will be staged to the following share (for example, servershare):
Auto-completion of
BuildDropsOrcas
UNC drop folder path

OK Cancel

May 15, 2007 12/19


Trigger

Default
Do not trigger a new build

Specify when this build definition should be triggered:
General
Build on request only
Workspace Rolling builds – the edit box
is enabled only when the
Build each check-in (more builds)
Project File corresponding check box is
Accumulate check-ins until the prior build finishes (fewer builds) checked. Valid values range
Retention Policy Build no more often than every minutes from 1 to Int32.MaxValue
Build Defaults
Gated check-in
Accept check-in only if changes build successfully
Trigger Build every week on the following days:
Monday Tuesday Wednesday Thursday
Friday Saturday Sunday
Queue the build on the default build agent at:

11:30 PM Eastern Standard Time (GMT-5:00)

Note: scheduled builds will not occur if no changes have been checked in
since the previous build.

OK Cancel

May 15, 2007 13/19


Queue a New Build

Updating Queue Position

When the dialog is instantiated, the
build definition is set to the last build
definition the user specified or, on first
Queue Build <Team Project Name> use, to the current team project’s first
build definition. The build agent is set to
Lists all build
the default agent specified in the build
Select a build definition to build and specify the agent that
definitions for the
definition (if no default is specified, it
will build perform the build. Displays the description
current team project
defaults to the first idle agent when
of the selected build
Build definition: available). The priority defaults to
definition
Normal. The position in queue is
On-demand
updated asynchronously.
On-demand builds of the continuous integration feature branch
When the user modifies
Defaults to the default
(SelectionChangeCommitted) the build
agent for the selected
agent or priority, the system sets a timer
build definition
500 ms timer. The timer is reset on each
Shows “idle” or number
Build agent:
newly committed selection change. If
of queued builds
Mini-lab: CI the timer elapses before any changes
(including build in
are committed, the position is updated
Auto-complete progress) next to agent
Stage build to share: asynchronously. If there is already a
name.
previous, pending update, the previous
tfdgbbldsvr2dropsVSTFMQlayouts
one is cancelled.
High, Above Normal,
Normal, Below Normal, Priority in queue: Position:
State Persistence
or Low – disabled if
Normal 1
user doesn’t have
The selected build definition is persisted
“Administer a build”
MSBuild command-line arguments (optional): across invocations on a per-team project
privilege
Estimated position in basis. The selected build definition
queue based on agent drives the default build agent and drop
and priority; displays folder. The priority will always default to
“Checking…” while Normal when this dialog is instantiated
Use fixed-pitch font for Queue Cancel waiting for query but it will not reset if they change the
legibility – Associated
results build definition or any other field.
help topic links to
MSBuild command-line
MSBuild command-line arguments will
help
be persisted on a per-build definition
User is taken to the basis. Selecting a build definition will
Team Build Explorer’s populate this field with the previously
Queued builds tab specified arguments (if any).

May 15, 2007 14/19


Team Build Explorer
Queued Builds
Any Build Agent
Synchronized
Lab: Visual Studio

s
ie
Q ts
across both tabs

lit
Lab: Team Foundation

ild en
ua
ly

ag ui d
Bu Ag
te ef y
ite
In Progress

il
le nd alit
Lab: Team System

M age Bu
State Transitions

in

e ld
Re Bui el
De in I Qu
Queued Mini-lab: CI

an ew
nc

an B
ta ld
op e
Ed Ca

N
Postponed Mini-lab: Drop Mgmt
St pon
sh

The Queued tab contains a grid with all

ue
/
re

st

ue
it
f

builds that are currently queued or in
Po
Re

M
Q
progress. This view should poll the
Set Priority application tier periodically to update
itself. Builds that transition to a
Build definition: Status filter: Agent filter: completed state as a result of polling
will not be removed from this view until
<Any Build Definition> <Any Status> <Any Build Agent>
the user manually refreshes the view.
Build Definition Priority Date Queued Requested by Build Agent Builds that transition from a queued
state to a cancelled state will not be
CI - Rolling Normal Thu 10/12/2006 10:27 AM Mary Jones Mini-lab: CI
removed from this view unless they were
CI - Rolling Normal Thu 10/12/2006 10:40 AM Mary Jones Mini-lab: CI
cancelled from this view (as opposed to
CI - Rolling Normal Thu 10/12/2006 8:39 AM Davey Jones Mini-lab: CI transitioning as a result of polling).
Co
nt
ex The default action for completed builds - opens
t State Persistence
Me
the build result view; disabled and grayed for
nu
queued builds and multiple selections The selected build definition is persisted
across invocations on a per-team project
basis. The selected build definition
Open
drives the default build agent and drop
Set Priority High
Postponed builds folder. The priority will always default to
Stop / Cancel Above Normal
are grayed Normal when this dialog is instantiated
Postpone Normal
but it will not reset if they change the
Below Normal
build definition or any other field.
Queue New Build… Low
MSBuild command-line arguments will
Refresh
be persisted on a per-build definition
basis. Selecting a build definition will
populate this field with the previously
specified arguments (if any).
The term “Stop” is used for builds in progress Polling Strategy
while “Cancel” is used for queued builds;
confirmation is required This view should poll every 30 seconds.
When refreshing the view, the user’s
selection and focus should be preserved
Completed
Queued
whenever possible.

May 15, 2007 15/19


Team Build Explorer Any Quality
Unexamined
Completed Builds Initial Test Passed
Lab Test Passed Today
Ready for Deployment Last 24 hours
Synchronized Ready for Initial Test Last 48 hours

s
ie
Q ts
across both tabs

lit
Rejected Last 7 days

ild en
ua
ly

ag ui d
Bu Ag
te ef y
ite
Released

il
le nd alit
Last 14 days

M age Bu
in

e ld
Re Bui el
De in I Qu
UAT Passed Last 28 days

an ew
nc

an B
ta ld
op e
Ed Ca

N
Under Investigation <Any Time>
St pon
sh

ue
/
re

st

ue
it
f

Po
Re

M
Q
Set Priority Filtering

Build definition: Quality filter: Date filter: The build definition filter is synchronized
with the corresponding filter on the
<Any Build Definition> <Any Build Quality> Today
queued builds tab. Build status is
consciously omitted from the filtering
Name Build Definition Build Quality Date Completed Requested By
i
options. The user can, however, sort by
CI - Rolling_20061012.2 CI – Rolling Unexamined Thu 10/12/2006 10:35 AM Mary Jones build status (the second column). Note
Ready for Initial Test that build agent is also consciously
CI - Rolling_20061012.1 CI – Rolling Thu 10/12/2006 9:15 AM John Smith
omitted from the filtering options since
Initial Test Passed
CI - Rolling_20061011.2. CI – Rolling Wed 10/11/2006 1:45 PM Mary Jones
this is significantly less interesting for a
Rejected
CI - Rolling_20061012.1 CI – Rolling Wed 10/11/2006 1:45 PM John Smith
completed build than it is for a queued
build. The default filter is any build
Co
nt definition, any build quality, completed
ex
tM today.
en
u
Exempt from
Polling Strategy
retention policy
Open Disabled for This view should never poll. The user can
Open Drop Folder stopped builds refresh the grid contents manually. This
Retain Indefinitely
view will query for its contents when it is
Edit Build Quality
Requires first activated. It will re-query (refresh)
Delete
confirmation when it is de-activated (not in the
Checked it the selected
foreground) if it has been more than 5
build is exempted Queue New Build…
seconds since the last query.
Edit Retention Policy…
This view only shows completed builds
Retained Indefinitely
Sort By
and will not show builds in progress.
Status
Sort Ascending
Name
Sort Descending
Multiple Selection
Build Definition
Refresh Build Quality
The Open Drop Folder and Edit Build
Date Completed
Quality commands will be disabled for
Requested By
Queued Completed
multiple selections.

May 15, 2007 16/19


Build Agent Manager

Accessible From
Manage Build Agents
Visual Studio main menu’s, Build pull-down

Build agents:
New…
Name | Status | Queued Builds
Lab: Product Unit Main Enabled
Edit…
Lab: Orcas Branch Enabled
Mini-lab: Continuous Integration Enabled 2 Remove
Mini-lab: New MSBuild Enabled
Mini-lab: Retired Disabled

Close

The names of disabled build
agents are drawn with the
gray text system color

May 15, 2007 17/19


Build Agent Properties

Build Agent Properties

Display name:
Mini-lab: Continuous Integration

Description:
This build agent is used exclusively by the Continuous Integration Defaults to the same
feature crew. port used in v1.0

Computer name: Communications port:
TKBGITBB03 7490

Working directory:
Defaults to a folder
$(Temp)$(BuildDefinitionPath) beneath the system
temp folder
Agent status:
3 builds in queue
Enabled
If the user disables the build
agent while a build is in
Note: Look in the Build folder on the Team Foundation Server media progress the system will ask if
for the Team Build service installer. You will need the build domain they want to stop the build
account to complete the install, so make sure you have that setup first. first. Any queued builds will be
stalled until the build agent is
enabled.
Default OK Cancel
For new agents, this label
displays the guidance shown.
For existing agents, it displays
The Defaults button lets the
a status message provided by
user revert the communications
the server.
port and working directory to
their default values.

May 15, 2007 18/19


Rosario Age of last build
Status of last build
Rolling Building?
Open drop folder of last good build
View build log of last build
Force (queue) build
Take responsibility/volunteer to fix
Orcas
Coloring
Orcas Green – most recent build was successful
Continuous – Building
Red – most recent build failed
Started 1/15/2007 1:57 PM for Jim Lamb Yellow – currently building
Rosario
Gray – server is unavailable or in a bad state
Nightly – Succeeded
Succeeded
Nightly Open Drop Folder | View Log
Open Drop Folder | View Log
Release – Failed
Volunteer to Fix | View Log | Rebuild
Building
Continuous
Started 1/15/2007 1:57 PM for Jim Lamb Rolling – Queued
Queued 1/15/2007 1:50 PM for Buck Hodges
Failed
Release
Volunteer to Fix | View Log | Rebuild

Succeeded
On-demand
Open Drop Folder | View Log

Team Project
Orcas

| Build Definitions:
Continuous Building
Nightly Succeeded
Release Failed
Rolling Queued

Select definitions to monitor

Queued 1/15/2007 1:50 PM for Buck Hodges

May 15, 2007 19/19

CI User Experience Spec

Recommended

Recommended

More Related Content

Viewers also liked

Viewers also liked (10)

Similar to CI User Experience Spec

Similar to CI User Experience Spec (20)

Recently uploaded

Recently uploaded (20)

CI User Experience Spec