Continuous Inspection and Integration of PHP Projects

Continuous Integration of PHP Projects Sebastian Bergmann November 3 rd 2010

Hello! My name is Sebastian. Sebas tian

Hello! My name is Sebastian … and this is what I do:  sebastian@thePHP.cc Co-Founder of The PHP Consulting Company  sebastian@phpunit.de Creator and Main Developer of PHPUnit  sebastian@php.net Contributor to PHP  sebastian@apache.org Contributor to Zeta Components  sbergmann@{acm.org | ieee.org} Practitioner of academic computing

Get the help you need from the experts you trust. Consulting | Code Review | Training

Deploy Early, Deploy Often ”Our highest priority is to satisfy the customer through early and continuous delivery of valuable software.” – Agile Manifesto

Deploy Early, Deploy Often ”Our highest priority is to satisfy the customer through early and continuous delivery of valuable software.” – Agile Manifesto ”Software delivers no revenue until it is in the hands of its users. […] For large companies every week of delay between having an idea and releasing the code that implements it can represent millions of dollars in opportunity costs […]” – Jez Humble and David Farley

Deploy Early, Deploy Often ”Our highest priority is to satisfy the customer through early and continuous delivery of valuable software.” – Agile Manifesto ”Software delivers no revenue until it is in the hands of its users. […] For large companies every week of delay between having an idea and releasing the code that implements it can represent millions of dollars in opportunity costs […]” – Jez Humble and David Farley ”We need to get rid off the term release cycle.” – Simon Stewart (over lunch at GTAC 2010)

Agile Environments Scrum is an agile method for project management. http://en.wikipedia.org/wiki/File:Scrum_process.svg

Agile Environments Scrum is not an agile method for software engineering. http://en.wikipedia.org/wiki/File:Scrum_process.svg

Agile Environments Common elements of agile methods for software engineering  Coding  Collective Code Ownership  Test-Driven Development  Continuous Integration & Continuous Inspection  Testing  All code must have unit tests  All code must pass all unit tests before release  When a bug is found, tests are created  DevOps  Continuous Deployment & Continuous Delivery

Avoid ”Big Bang” Deployments

Deploy Early, Deploy Often ”Flickr was last deployed 6 hours ago, including 5 changes by 3 people. In the last week there were 74 deploys of 447 changes by 22 people.” – http://code.flickr.com/

Deploy Early, Deploy Often ”Flickr was last deployed 6 hours ago, including 5 changes by 3 people. In the last week there were 74 deploys of 447 changes by 22 people.” – http://code.flickr.com/ ”Every engineer at Etsy writes automated tests on a daily basis, and our Hudson continuous integration server runs about 2500 tests with every code push. Since we push code around 20 times a day, that means running around 250,000 PHPUnit tests in a typical work week!” – http://codeascraft.etsy.com/

Continuous Integration is about preventing your developers from burning in Integration Hell.

Integration Hell  I've been working on my classes and think they are perfect.  You've been working on yours and I suppose you think they're pretty good, too.  Carl has been working on his, and you know how that goes. This slide contains material by Ron Jeffries

Integration Hell  Now we have to integrate them to build a new system.  Carl's code, as usual, breaks everything.  It looks to me as if you have a few problems, too.  My code is solid, I know that because I worked hard on it. This slide contains material by Ron Jeffries

Integration Hell  What I can't understand is why you think there might be something wrong with my code, and Carl, the idiot, is after both of us.  We're in for a few really unpleasant days.  Maybe next time we shouldn't wait so long to integrate. This slide contains material by Ron Jeffries

Continuous Integration is about improving software quality and reducing project risks.

Continuous Integration Software development practice where members of a team integrate their work frequently  Usually each person integrates at least daily, leading to multiple integrations per day  Each integration is verified by an automated build (including test) to detect integration errors as quickly as possible  Leads to significantly reduced integration problems and allows a team to develop cohesive software more rapidly This slide contains material by Martin Fowler

Continuous Integration What is the Return on Investment?

The Value of Continuous Integration  Reduce risks  Reduce repetitive processes  Generate deployable software  Enable better project visibility  Establish greater project confidence This slide contains material by Paul M. Duvall

The Value of Continuous Integration Risks Continuous Integration helps to reduce  Late discovery of defects  Regression Testing  Test Coverage  Low-Quality Software  Coding Standard Adherence  Architectural Adherence  Duplicate Code  Lack of deployable software  Lack of project visibility This slide contains material by Paul M. Duvall

Static Code Analysis

Static Code Analysis How much code do we have? sb@thinkpad ~ % phploc typo3-4.4.2 phploc 1.5.1 by Sebastian Bergmann. Directories: 303 Files: 1246 Lines of Code (LOC): 365956 Cyclomatic Complexity / Lines of Code: 0.19 Executable Lines of Code (ELOC): 170544 Comment Lines of Code (CLOC): 152534 Non-Comment Lines of Code (NCLOC): 213422 Namespaces: 0 Interfaces: 77 Classes: 1093 Abstract: 30 (2.74%) Concrete: 1063 (97.26%) Average Class Length (NCLOC): 199 Methods: 9176 Scope: Non-Static: 8662 (94.40%) Static: 514 (5.60%) Visibility: Public: 8020 (87.40%) Non-Public: 1156 (12.60%) Average Method Length (NCLOC): 23 Cyclomatic Complexity / Number of Methods: 5.05 Anonymous Functions: 0 Functions: 197 Constants: 536 Global constants: 346 Class constants: 190

Static Code Analysis How is the code structured? sb@thinkpad ~ % phploc typo3-4.4.2 phploc 1.5.1 by Sebastian Bergmann. Directories: 303 Files: 1246 Lines of Code (LOC): 365956 Cyclomatic Complexity / Lines of Code: 0.19 Executable Lines of Code (ELOC): 170544 Comment Lines of Code (CLOC): 152534 Non-Comment Lines of Code (NCLOC): 213422 Namespaces: 0 Interfaces: 77 Classes: 1093 Abstract: 30 (2.74%) Concrete: 1063 (97.26%) Average Class Length (NCLOC): 199 Methods: 9176 Scope: Non-Static: 8662 (94.40%) Static: 514 (5.60%) Visibility: Public: 8020 (87.40%) Non-Public: 1156 (12.60%) Average Method Length (NCLOC): 23 Cyclomatic Complexity / Number of Methods: 5.05 Anonymous Functions: 0 Functions: 197 Constants: 536 Global constants: 346 Class constants: 190

Static Code Analysis How is the code structured? sb@thinkpad ~ % pdepend --overview-pyramid=pyramid.svg typo3-4.4.2 ANDC Average Number of Derived Classes AHH Average Hierarchy Height NOP Number of Packages NOC Number of Classes NOM Number of Methods LOC Lines of Code (non-comment, non-whitespace) CYCLO Cyclomatic Complexity CALLS Number of Operation Calls FANOUT Number of Called Classes

Static Code Analysis How is the code executed? 1 <?php 2 if (TRUE) { 3 print '*'; 4 } 5 ?> sb@thinkpad ~ % bytekit if.php bytekit-cli 1.1.1 by Sebastian Bergmann. Filename: /home/sb/if.php Function: main Number of oplines: 8 line # opcode result operands ----------------------------------------------------------------------------- 2 0 EXT_STMT 1 JMPZ true, ->6 3 2 EXT_STMT 3 PRINT ~0 '*' 4 FREE ~0 4 5 JMP ->6 6 6 EXT_STMT 7 RETURN 1

Static Code Analysis How is the code executed? 1 <?php 2 if (TRUE) { 3 print '*'; 4 } 5 ?> sb@thinkpad ~ % bytekit --graph /tmp --format svg if.php

Static Code Analysis Can we find ”interesting” code using tools? sb@thinkpad ~ % phpcs --standard=CodeReview --extensions=php --report=source typo3-4.4.2 PHP CODE SNIFFER VIOLATION SOURCE SUMMARY -------------------------------------------------------------------------------- STANDARD CATEGORY SNIFF COUNT -------------------------------------------------------------------------------- Generic PHP No silenced errors 1055 Generic Metrics Cyclomatic complexity 746 Generic Code analysis Unused function parameter 624 Generic Metrics Nesting level 299 Generic Code analysis For loop with test function call 76 Generic Code analysis Empty statement 64 Generic Code analysis Useless overriding method 7 Generic Code analysis Jumbled incrementer 3 Generic Code analysis Unconditional if statement 3 Generic Code analysis For loop should be while loop 1 -------------------------------------------------------------------------------- A TOTAL OF 2878 SNIFF VIOLATION(S) WERE FOUND IN 10 SOURCE(S) --------------------------------------------------------------------------------

Static Code Analysis Can we find ”interesting” code using tools? sb@thinkpad ~ % phpcs --standard=CodeReview --extensions=php typo3-4.4.2 FILE: /home/sb/typo3/index.php -------------------------------------------------------------------------------- FOUND 0 ERROR(S) AFFECTING 2 LINE(S) -------------------------------------------------------------------------------- 57 | WARNING | Silencing errors is discouraged 59 | WARNING | Silencing errors is discouraged -------------------------------------------------------------------------------- FILE: .../sb/typo3/t3lib/cache/backend/class.t3lib_cache_backend_filebackend.php -------------------------------------------------------------------------------- FOUND 1 ERROR(S) AFFECTING 4 LINE(S) -------------------------------------------------------------------------------- 89 | ERROR | Empty CATCH statement detected 105 | WARNING | Function's cyclomatic complexity (17) exceeds 10; consider | | refactoring the function 327 | WARNING | Avoid function calls in a FOR loop test part 358 | WARNING | Avoid function calls in a FOR loop test part -------------------------------------------------------------------------------- . . .

Static Code Analysis Can we find ”interesting” code using tools? sb@thinkpad ~ % phpmd typo3-4.4.2 text codesize,design,naming,unusedcode t3lib/class.t3lib_sqlengine.php:104 This class has a bunch of public methods and attributes t3lib/class.t3lib_sqlengine.php:104 This class has too many methods, consider refactoring it. t3lib/class.t3lib_sqlengine.php:104 This class t3lib_sqlengine has an overall complexity of 105 which is very high. The configured complexity threshold is 50. t3lib/class.t3lib_sqlengine.php:126 Avoid unused parameters such as '$config'. t3lib/class.t3lib_sqlengine.php:126 Avoid unused parameters such as '$pObj'. t3lib/class.t3lib_sqlengine.php:150 The method processAccordingToConfig() has a Cyclomatic Complexity of 15. t3lib/class.t3lib_sqlengine.php:227 Avoid unused local variables such as '$options'. t3lib/class.t3lib_sqlengine.php:383 Avoid unused parameters such as '$select_fields'. t3lib/class.t3lib_sqlengine.php:383 Avoid unused parameters such as '$groupBy'. t3lib/class.t3lib_sqlengine.php:383 Avoid unused parameters such as '$orderBy'. t3lib/class.t3lib_sqlengine.php:383 Avoid unused parameters such as '$limit'. t3lib/class.t3lib_sqlengine.php:426 Avoid unused parameters such as '$query'. t3lib/class.t3lib_sqlengine.php:502 Avoid unused parameters such as '$tableName'. t3lib/class.t3lib_sqlengine.php:513 Avoid unused parameters such as '$tableName'. t3lib/class.t3lib_sqlengine.php:629 The method select_evalSingle() has a Cyclomatic Complexity of 38. t3lib/class.t3lib_sqlengine.php:629 The method select_evalSingle() has an NPath complexity of 3980. t3lib/class.t3lib_sqlengine.php:629 Avoid really long methods. t3lib/class.t3lib_sqlengine.php:835 Avoid excessively long variable names like $TYPO3_DBAL_handlerType . . .

Static Code Analysis Do we have duplicated code? sb@thinkpad ~ % phpcpd typo3-4.4.2 phpcpd 1.3.2 by Sebastian Bergmann. Found 80 exact clones with 3415 duplicated lines in 82 files: - t3lib/class.t3lib_fullsearch.php:800-814 t3lib/class.t3lib_querygenerator.php:1409-1423 - t3lib/class.t3lib_tceforms.php:2210-2216 t3lib/class.t3lib_tceforms.php:2322-2328 - typo3/file_edit.php:99-115 typo3/file_rename.php:110-126 . . . 0.93% duplicated lines out of 365956 total lines of code. Time: 01:28, Memory: 359.00Mb

Static Code Analysis Even more tools  HipHop for PHP: http://bit.ly/HipHop4QA  pfff: http://github.com/facebook/pfff  Phantm: http://github.com/colder/phantm  Padawan: http://github.com/mayflowergmbh/padawan/  php-tracer-weaver: http://github.com/troelskn/php-tracer-weaver/

QA Tools for PHP Overview http://phpqatools.org/

Continuous Integration is about feeling the pulse of a project.

Continuous Integration Server A software solution that  automatically fetches new revisions from the version control system  automatically builds and tests the software  keeps software metrics information over time  provides insight into the development progress and code quality throughout the development lifecycle

Continuous Integration Server Available Solutions  CruiseControl  http://cruisecontrol.sf.net/  phpUnderControl  http://phpUnderControl.org/  Hudson  http://hudson-ci.org/  Atlassian Bamboo  http://www.atlassian.com/software/bamboo/  Arbit  http://arbitracker.org/

DevOps http://en.wikipedia.org/wiki/File:Devops.png

The End  Web: http://thePHP.cc/ http://sebastian-bergmann.de/  Mail: sebastian@thePHP.cc  Twitter: @s_bergmann  Slides: http://talks.thePHP.cc/  Vote: http://joind.in/2260

http://www.berejeb.com	41
http://victimofbabylon.com	24
http://www.devetdesign.com	6
http://www.techgig.com	2
http://static.slidesharecdn.com	1

Continuous Inspection and Integration of PHP Projects

by Sebastian Bergmann on Nov 03, 2010

Accessibility

Categories

Tags

Upload Details

Usage Rights

5 Embeds 74

Statistics

Continuous Inspection and Integration of PHP Projects — Presentation Transcript