Enterprise CI as-a-Service using Jenkins

1 Copyright ©2014 CollabNet, Inc. All Rights Reserved.
Enterprise CI-As-A-Service
A Case Study in Jenkins and OSS Tools to Support CI at Scale
Enterprise Transformation Manager
bdawson@collab.net
Brian Dawson

Presenter
Brian Dawson, Manager Enterprise Transformation, CollabNet
As a Consultant CollabNet, for 10 years Brian Dawson has helped software development organizations transform development processes through the use of TeamForge and development best practices, including Continuous Integration, Continuous Delivery, and Agile.
Prior to CollabNet, Brian spent 12 years in software development most recently at Sony Computer Entertainment where he worked as the Director of Tools and Technology and oversaw the development and deployment of TeamForge as a worldwide collaboration and reuse portal.

•
Review of Continuous Integration
•
CI In the Enterprise
•
Case Study: CI-As-A-Service
•
Conclusion
•
Q & A
Agenda

Review of CI

–
Fail fast so problems can be found and corrected early
–
CI feedback
•
Coding and architectural issues
•
Unit tests
•
Security issues
•
etc
What is CI?
“Continuous integration (CI) is a software engineering practice in which isolated changes are frequently integrated, immediately tested and reported on. The goal of CI is to provide rapid feedback so that if a defect is introduced into the code base, it can be identified and corrected as soon as possible.”

Why CI?
•
Automation of repetitive manual processes
•
More time focusing on adding value
•
No need to wait hours, days, months to know how check-in affects the build
•
Constant availability for the build for testing, demo or release
•
Shared visibility
•
Increased team productivity
•
Higher quality software, faster!

CI In The Enterprise

CI In The Enterprise: Current State
–
Rapid adoption
–
Spanning languages and technologies
–
“Shadow” Implementations (Internal and Cloud)
–
Implemented at team level
–
Implemented and maintained by developers

CI In The Enterprise: Issues and Challenges
–
Server, application sprawl and/or external exposure
–
Developers spending time as admins
–
Lack of shared knowledge
–
Slow project start-up and ramp-up
–
CI disconnected from enterprise development processes
–
Lack of security
–
Lack of governance

Centralize Source Code
•
Reduce ramp-up administration overhead
•
Manage IP
•
Supports traceability, visibility and governance
Manage binaries
•
Reduce risk
•
Promote reuse
Support Security
•Project, Dept. and/or organizationally aligned access controls
Naturally connect the build process to the common platform
•Supports traceability, visibility and governance
CI in The Enterprise: Path to Success

Support Dynamically Scaling
•
Server provisioning
•
Virtual Machines
Establish central build platform and/or eco-system
•
Flexible support for different teams needs
•
Support easy on-boarding
Document process and shared knowledge
•Develop community to support best practices
•Reduce ramp-up and onboarding time
CI in The Enterprise: Path to Success

Case Study: CI-As-A-Service

Case Study: Customer Overiew
Large Financial Services Company
•
4000+ users
•
100’s of applications
•
>100 teams
Globally Distributed
•
US, UK, Eastern Europe, Asia, etc
Multiple Technologies
•Java, .Net, Batch, etc
•Web, Desktop, Mobile,etc

Case Study: Goals and Challenges
Goals
–
Change the culture of development Create open-source development culture
•
Deliver productivity to the developer … for the developer
•
Interested, passionate developer can be 10x more productive than a developer “just doing his(her) job”
–
Provide standard build tool implementations
–
Provide build to release management platform
–
Leverage private cloud to enable scalable solution
–
Provide a support structure
Challenges
–
Multiple technologies
–
Multiple tools and processes
–
Regulatory compliance mandates
–
Scale

Case Study: Solution

Common Platform = TeamForge

SCM = Managed Subversion

CI Server = Jenkins w/Integration

Binary Artifact Management = Sonatype Nexus w/Integration

Quality = SonarQube Sonar w/Integration

Server/VM Provisioning = Lab Management
•
VM Ware and EC2
•
LM Profiles (IAAS)
•Puppet for Config Management

Community Support
•Dedicated Build Engineer
•TeamForge Trackers
•Wikis
Established common CI tooling, with rich integration to a common platform managed by a published process for CI server provisioning and support

Case Study: Process
•
All application development hosted in TeamForge Projects providing role-based access controlled workspaces with common tools
•
Projects use integrated Subversion for centralized administration and management of source code
•
Process and shared knowledge documented in community support project wikis
•
Build resources requested via TeamForge Tracker ticket with immediate notificaion to dedicated build engineer
•
Jenkins server dynamically provisioned from internal Lab Management cloud using common pofiles and Puppet for configuration management
•
Access provided to Jenkins through TeamForge project-level role-based access controls, providing project and role appropriate permissions
•
Access provided to Nexus and Sonar via to TeamForge role-based access controls, providing access to common, department and project-specific binary assets.
•
Build Environment handed off to team within hours with all necessary info provided via ticket and shared documentation
•
Team has ability to configure environment and create jobs on-going support and server configuration is facilitated via Trackers by build engineer with High SLA

17 Copyright ©2014 CollabNet, Copyright © 2008 CollabNet, Inc. All Rights Reserved. Inc. All Rights Reserved.

~29 virtual machines

~20+ Jenkins Servers

1 Nexus Artifact Repository

1 Sonar Server

1 Puppet Server

1 OpenNMS Server

~100 Development Teams

>500 Developers

~150 Applications
…Supported by 1 build engineer
Case Study: Current State

Case Study: Benefits
Benefits
•
Centralized build and test on a cloud platform
•
Centralized support
•
Increased productivity
•
Decreased cost per developer
•
Central instance SaaS and CIaaS
•
Better economies of scale and higher availability
•
Standard tools
•
Lower support costs, fewer integration points, better resource mobility
•
Central integration and “certification”
•
Reduced effort per team, higher quality

Case Study: Reference Architecture
Automated Build/Test
Artifact Publication
Source Code Management
Product / Sprint
Planning
Agile Project Development
Deployment Services
Build/Tag - Jenkins
Unit Tests - XUnit
Functional Tests - XUnit
Static Code Analysis - Sonar
Code Coverage – Sonar
Essential Services
Dependency Management
Governance Process
Artifact Reuse
Runtime Dependencies
Governance Process
Change Management
Test Manager
Artifact Creation / Management
Monitoring and Analysis
Data Consolidation / Reporting
Role Based Access Control
Full Subversion integration as well as future integration and support for Git
Focus on automation especially in the test area
Planning:
TeamForge, SWP
Development:
Eclipse,Visual Studio
Code Review:
ReviewBoard
SCM Management:
Subversion, Git
Prod
UAT
Test
Development
Data Warehouse

Conclusion

CollabNet Solutions: TeamForge
TeamForge provides the essential components needed to drive collaborative software development and deployment: Integrated ALM development and collaboration tools, flexible and repeatable process templates, and a cross-project collaboration architecture.
Reporting & Governance
search
traceability
documents
wikis
discussions
Release
Lab Management
Build & Test
Track
Code
Plan
My page
Reports
Projects
Monitoring
My settings
Open ALM
Collaboration
Architecture
Communities
My workspace
Deploy
GitEye
Orchestrate
Process
Templates
Dashboards & Report Catalog
Desktops

Agile Delivery & DevOps
Agile Development
Enterprise SCM
Training
Training
Implementation
Implementation
Consulting
Consulting
Consulting
Solutions
Product
Services
TeamForge
Enterprise Agility Assessment
Service
Solution Pack
Solution Pack
Blueprint
for Enterprise Agility
DevOps Applied Workshop
Agile Dev Applied Workshop
Custom Continuous
Integration Implementation
Private/Public Agile, CSM and ALM Training
CollabNet Solutions: Products and Services

•
Solution Pack solutions:
•
Enterprise SCM leverages TeamForge SCM (Git and SVN) Integration
•
Agile Development (CI/CD) includes Enterprise SCM
•
Agile Delivery & DevOps Solution includes Agile Development
CollabNet Solutions: TeamForge Solution Packs
Agile Delivery and DevOps
•
Build + Release Workflow (Jenkins and ARA)
•
Auto-build of build and release trackers
•
Nexus integration – binary repos
Agile Development
•
Build Workflow (Jenkins)
•
Auto-build of build trackers
•
TeamForge File Release System integration
Enterprise SCM
•
TeamForge template – enterprise agility template
•
Dashboard template component
•
Alerts component
•
Solution Packs provide Enterprise Visibility and Traceability promote repeatable, solutions for Agile Development and Delivery (Process and Tools)
•
Solution Packs provide integrated solutions (Jenkins, Team City, ARA +)

›
Across user stories, sprints, releases, products, teams
›
Within workgroups—and across your organization
›
Enterprise-grade security with RBAC, permissions management, authentication, encryption, auditing
›
For creation and sharing of common, repeatable processes across your organization
›
Throughout the development lifecycle and across projects, tools, and teams
With CollabNet Solutions, You Gain
›
Organization-wide discovery, collaboration, re-use, and reporting

Questions?
blogs.collab.net
twitter.com/collabnet
www.facebook.com/collabnet
www.linkedin.com/company/collabnet-inc

Thank You

Enterprise CI as-a-Service using Jenkins

More Related Content

What's hot

Viewers also liked

Similar to Enterprise CI as-a-Service using Jenkins

More from CollabNet

Recently uploaded

Enterprise CI as-a-Service using Jenkins