PHP Unconference Continuous Integration
Upcoming SlideShare
Loading in...5
×
 

PHP Unconference Continuous Integration

on

  • 263 views

Presentation for PHP Unconference 2011 - unfortunately I did not get enough votes so never did the talk :(

Presentation for PHP Unconference 2011 - unfortunately I did not get enough votes so never did the talk :(

Statistics

Views

Total Views
263
Views on SlideShare
263
Embed Views
0

Actions

Likes
0
Downloads
1
Comments
0

0 Embeds 0

No embeds

Accessibility

Categories

Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

    PHP Unconference Continuous Integration PHP Unconference Continuous Integration Presentation Transcript

    • Continuous Integration for Heterogeneous Environments Nils Hofmeister, 11.09.2011 PHP Unconference
    • Hello world! • Nils Hofmeister • 5+ Years experience in realtime 3D / C++ • 2+ Years experience in web-based games (php, javascript, Java, C++, C#) • Lead Integration Architect @Bigpoint, Hamburg – “the SVN guy” ;) 2
    • Agenda • What is continuous integration? • Why do you want continuous integration for your project? • How do we implement continuous integration at Bigpoint? • What are our next steps? 3
    • What is continuous integration? 4
    • What is continuous integration? The gossip • A way to automize certain tasks, e.g. • code validation • build automation • packaging of products • A system of cooperating hardware- and software components , e.g. • Source code management system (SCM) • Staging system + production servers • Build cluster with all kinds of compilers and frameworks on it The idea behind this is much more important, so we do the technical stuff later. 5
    • What is continuous integration? The idea • Not brand-new: First known example: IBM OS/360, early 60s • Current ideas and drivers closely related to agile movement and Extreme Programming • Martin Fowler says: • • • • • • • software development practice integrate *…+ frequently multiple integrations per day verified by an automated build detect integration errors significantly reduced integration problems develop cohesive software more rapidly Martin Fowler on Continuous Integration - 2006 version Original article – 2000 6
    • What is continuous integration? The philosophy Jez Humble says in "Continuous Delivery": • • • • • • • • Create a repeatable, reliable process [...] Automate almost everything Keep everything in version control If it hurts, do it more freqeuntly and bring the pain forward Build quality in Done means released Everybody is responsible for the delivery process Continuous improvement “If something hurts, do it more often” 7
    • What is continuous integration? The attitude • Accept that CI is a standard tool. Just as compilers, SCMs or documentation are • Understand that failing builds must have highest priority - always! • Keep in mind that errors happen anyway. CI only makes them visible immediately and reliably. 8
    • Why do you want continuous integration for your project? 9
    • Why do you want continuous integration for your project? Reduce risks • Defects are detected and fixed sooner • Health of software is measurable • Reduce assumptions (environment variables, local setup, etc) 10
    • Why do you want continuous integration for your project? Reduce repetitive manual processes • Process runs the same way every time • The process starts automatically • This frees resources for higher-value work 11
    • Why do you want continuous integration for your project? Generate deployable software • Product is deployable always • Paradigm-shift from crunch time before release to "always ready to deliver" 12
    • Why do you want continuous integration for your project? Better project visibility • Supports effective decisions • Facilitates recognizing trends 13
    • Why do you want continuous integration for your project? Greater confidence in your project • CI provides a safety net to avoid releasing flawed software • Developers get a better feeling for impact of changes 14
    • How do we implement continuous integration at Bigpoint? 15
    • How do we implement continuous integration at Bigpoint? The setting • Websites, backend systems, clients • PHP is one of many technologies required to assemble our games (an important one, don’t worry ;)) 16
    • How do we implement continuous integration at Bigpoint? The requirements • Per artifact: • Check for defects • Measure quality • Per project: • Assemble deliverable packages • Deploy these 17
    • How do we implement continuous integration at Bigpoint? The solution space (subjective! and there is more of course) Microsoft Team Foundation Server Integrates fine with Microsoft Tools. But with nothing else  TeamCity Well-known product by JetBrains. Feature set is fine, but no USP that justifies to pay for it right now. CruiseControl The classic invented by ThoughtWorks. Now an open source project. Delightfully old-school. Bamboo Quite solid, full-featured product by Atlassian. No real USP though that justifies the high price. Go The current CI solution by ThoughtWorks. The theory behind it is cool but it lacks a lot of features and configuration is no fun. 18
    • How do we implement continuous integration at Bigpoint? The actual solution for us • Hudson: • Created by Kohsuke Kawaguchi • "Duke's Choice Award" in 2008 • Comes from the Java world, ships with CVS/SVN, Maven, JUnit support • Jenkins: • Kohsuke was working for Sun till Oracle acquired Sun in 2010 • Conflict between the Hudson Core developers and Oracle => Fork in 2011 19
    • How do we implement continuous integration at Bigpoint? The actual solution for us II • Trends after fork • Hudson => Focus on quality & stability • Jenkins => Core developers, faster pace • Our decision • choose Jenkins as for us features > stability. • to compensate we use Jenkins LTS 20
    • How do we implement continuous integration at Bigpoint? The actual solution for us III We like the Jenkins community! (My colleague Marcelo and Kohsuke on their way to get their hands on some beer) 21
    • How do we implement continuous integration at Bigpoint? Hamburg Jenkins cluster There are more instances, but this is the biggest one… • Setup • 1 fileserver as master (Debian) • 3 blades (Debian) • 4 windows workstations • Toolchains • • • • • Java Flash/Flex Php (+ JS, html, CSS) Unity/Mono C++ • Highly customized • Home-grown housekeeping + backup • Custom plugins / actively develop Jenkins • All the stuff needed to deal with Kerberos… 22
    • How do we implement continuous integration at Bigpoint? Hamburg Jenkins cluster II Under the hood: • Unity3D • Unity/Mono + Ant • Java • Maven + some Ant • Artifactory/Nexus (memo: synergies…) • Flash • Ant/Maven (Not sure who will win yet) • C++ • gcc/make via Ant/Shell • PHP/web apps • All glory to Sebastian Bergmann – we more or less use his template… • …plus some mods, e.g. minifiers, CDN stuff, 23
    • How do we implement continuous integration at Bigpoint? Bigpoint standard game build pipeline Behold: join plugin, downstream builds, copy artifacts, parameterized builds. We do not do deployment from within Jenkins but have a custom deployment tool. Interface is SVN tags. 24
    • How do we implement continuous integration at Bigpoint? Bigpoint standard build pipeline II • Oh noes, this is too slow… • Optimize hardware • Parallelize wherever possible • Where not possible, decompose and create additional pipelines • But… this is a SPOF! • • • • Backup _everything_ automatically Fully automized CI server setup Find some replication/failover setup (todo) Remember: everything the CI server does must run on your machine as well! • Wait a minute… no more Ninja-deployment? But we need that to be agile… • No you don’t  25
    • How do we implement continuous integration at Bigpoint? The obstacles • The vast amount of knowledge needed • Understanding the Bigpoint platform (the Release Engineering team is “brand new”) • The different requirements, setup and skills of about 25 groups of stakeholders • Linux/IT-driven platform vs. Java toolchain • Acceptance (everyone claims to like Software Engineering, but some people seem to have a different understanding of what that is  ) We can handle much of this very well by now. Our biggest problem remaining: • Technical debt (already!) • We need a build framework… 26
    • What are our next steps? 27
    • What are our next steps? Win the SVN + Kerberos battle • Custom SVN sub system for Jenkins + custom Ant task • Allows svn+ssh with Kerberos for our setup. _Massive_ speedup • At some point we will contribute this to the Jenkins community… 28
    • What are our next steps? The Build Framework • Collect learnings and known requirements • Derive a flexible framework • Unify maintenance and interfaces to internal customers Either we automize things further or we need tons of people doing frustrating stuff. We don’t like frustrating stuff. 29
    • What are our next steps? The canonical build pipeline in Jenkins https://wiki.jenkins-ci.org/display/JENKINS/Build+Pipeline+Plugin • Learn from the ThoughtWorks guys! • Interesting: build pipeline plugin – Centrum Systems 30
    • “If something hurts, do it more often” 31
    • Contact us Bigpoint GmbH Nils Hofmeister Lead Integration Architect Drehbahn 47-48 20354 Hamburg Germany Tel +49 40.88 14 13 - 0 Fax +49 40.88 14 13 - 11 info@bigpoint.net www.bigpoint.net Bigpoint Inc. Bigpoint Distribuição de Entretenimento Online Ltda. 500 Howard Street Suite 300 San Francisco, CA 94105 Av. Brig. Faria Lima 3729 cj. 528 04538-905 São Paulo Brazil Bigpoint GmbH Bigpoint International Services Limited Alexanderstraße 5 10178 Berlin Germany 1 Villa Zimmermann Ta’Xbiex Terrace XBX 1035 Ta’Xbiex Malta Find us on 32
    • Bigpoint GmbH Nils Hofmeister Lead Integration Architect Drehbahn 47-48 20354 Hamburg Germany Tel +49 40.88 14 13 - 0 Fax +49 40.88 14 13 - 11 info@bigpoint.net www.bigpoint.net Find us on 33