The document discusses quality metrics and testing approaches beyond unit testing that are important for PHP projects. It describes different types of testing including acceptance testing, integration testing, load testing, usability testing, and security testing. It also discusses metrics for measuring code quality, such as complexity metrics, coding standards, and avoiding anti-patterns. Maintaining high-quality documentation and avoiding overly complex code is emphasized.

1. Quality in PHP projects beyond Unittests
International PHP Conference 2008
October 30th 2008 | Thorsten Rinne

2. Introduction
❙ Thorsten Rinne
❙ 31 years old
❙ Senior Developer and Team Lead at Mayflower GmbH
❙ Reporting applications
❙ Rating applications
❙ PHP consulting for QA and migration problems
❙ PHP software development since 1999
❙ Founder and main developer of open source FAQ
management software phpMyFAQ since 2001
❙ Zend Certified Engineer (PHP 5)
Quality in PHP projects
beyond Unittests
© MAYFLOWER GmbH 2008 2

3. Introduction
❙ Warm up
❙ Quality Criterias
❙ Quality Metrics
❙ Testing
❙ PHP Source Code Quality
❙ Questions and answers
Quality in PHP projects
beyond Unittests
© MAYFLOWER GmbH 2008 3

4. Who are you?
❙ What's your profession?
❙ Software company or agency?
❙ What's your team size?
❙ Who develops using Unittests?
❙ Who uses QA supporting tools?
❙ What's your test coverage?
❙ Even for legacy code? :-)
Quality in PHP projects
beyond Unittests
© MAYFLOWER GmbH 2008 4

5. Quality Criterias
Quality in PHP projects
beyond Unittests
© MAYFLOWER GmbH 2008 5

6. Quality criterias:
Understandability
❙ Availability of documentation
❙ Good documentation quality
❙ Complexity of software
❙ Code commenting and formatting
❙ Understandable naming
Quality in PHP projects
beyond Unittests
© MAYFLOWER GmbH 2008 6

7. Quality criterias:
Completeness
❙ Everything is already there
❙ Your application is fully implemented
❙ No mockups or temporary solutions left
❙ External interfaces are usable
❙ All needed data is available
Quality in PHP projects
beyond Unittests
© MAYFLOWER GmbH 2008 7

8. Quality criterias:
Conciseness
❙ No unneeded, overlong documentation
❙ Especially auto-generated documentation
❙ No unused libraries
❙ No dead code
❙ No duplicate code
❙ No code inside loops that could be outside
Quality in PHP projects
beyond Unittests
© MAYFLOWER GmbH 2008 8

9. Quality criterias:
Portability
❙ Easy to configure in new environments
❙ Few OS and webserver dependencies
❙ Few, documented version dependencies
❙ Language version
❙ Database version
❙ External libraries
❙ Able to survice environment updates
Quality in PHP projects
beyond Unittests
© MAYFLOWER GmbH 2008 9

10. Quality criterias:
Consistency
❙ Predictability
❙ Directory paths and filenames
❙ Class and method names
❙ Variable and constant names
❙ Documentation style and language
❙ Coding style
Quality in PHP projects
beyond Unittests
© MAYFLOWER GmbH 2008 10

11. Quality criterias:
Maintainability
❙ Easy to install and easy to upgrade
❙ Tests and debug logs to help to locate bugs
❙ Few dependencies hiding the bug
❙ No complex dependencies or effects on the platform
❙ Easy to understand and navigate source code
Quality in PHP projects
beyond Unittests
© MAYFLOWER GmbH 2008 11

12. Quality criterias:
Testability
❙ Known acceptence criteria
❙ Adequate separation of concerns
❙ Easy to use interfaces for tests
❙ No limiting dependencies
Quality in PHP projects
beyond Unittests
© MAYFLOWER GmbH 2008 12

13. Quality criterias:
Reliability
❙ The criteria formerly known as „robustness“
❙ Fault tolerance
❙ Graceful degradation
❙ Exception handling exists
❙ Is method input checked?
Quality in PHP projects
beyond Unittests
© MAYFLOWER GmbH 2008 13

14. Quality criterias:
Usability
❙ Learnability: Easyness of first time usage
❙ Efficiency: Overall speed of usage per task
❙ Memorabilty: Time to re-establish usage knowledge after
absence
❙ Errors: Frequency of usage errors
❙ Satisfaction: Fun to use the software
Quality in PHP projects
beyond Unittests
© MAYFLOWER GmbH 2008 14

15. Quality Metrics
Quality in PHP projects
beyond Unittests
© MAYFLOWER GmbH 2008 15

16. Quality Metrics:
Defect Density Metrics
❙ Code-based metric
❙ Defects per KSSI (1000 shipped source instructions) for
product quality
❙ Defects per KCSI (1000 changed source instructions)
for development quality
❙ (Hard to measure)
❙ Statistic needed
❙ easier for products
Quality in PHP projects
beyond Unittests
© MAYFLOWER GmbH 2008 16

17. Quality Metrics:
Customer Problem Metrics
❙ PUM: Problems per user month
❙ Defects per month / number of installations
❙ Maintainance metric: Number of bugs closed / number of
bugs arrived
❙ The customer does not care how big or how complicated
the software is :-)
Quality in PHP projects
beyond Unittests
© MAYFLOWER GmbH 2008 17

18. Quality Metrics:
Customer Satisfaction Metrics
❙ You only get them by survey
❙ for
❙ Functionality
❙ Usability
❙ Reliability
❙ Performance
❙ Maintainability
❙ Service
❙ Levels: Very Satisfied, Satisfied, Neutral, Dissatisfied, Very
Dissatisfied
Quality in PHP projects
beyond Unittests
© MAYFLOWER GmbH 2008 18

19. Quality Metrics:
When are bugs detected?
❙ Number of pre-release bugs found while testing
❙ Number of post-release bugs
❙ found by customer
❙ found by testing
Quality in PHP projects
beyond Unittests
© MAYFLOWER GmbH 2008 19

20. Quality Metrics:
How are defects fixed?
❙ Average Time to fix a bug
❙ Average Time for bugfix release
❙ that‘s the time relevant for the customer
❙ Average engineering hours / fixed defect
Quality in PHP projects
beyond Unittests
© MAYFLOWER GmbH 2008 20

21. Quality Metrics:
Defect Impact
❙ was the defect visible to the user?
❙ has a monetary transaction been touched?
❙ has sensitive data been touched?
❙ are there security implications?
❙ has there been a marketing/image effect
❙ did data / user work get lost?
Quality in PHP projects
beyond Unittests
© MAYFLOWER GmbH 2008 21

22. Quality Metrics:
Testing Efficiency
❙ Found bugs before release / found bugs post release
❙ Found bugs by qa team / found bugs by customer
❙ Number of bugs found by QA Team / time
Quality in PHP projects
beyond Unittests
© MAYFLOWER GmbH 2008 22

23. „We do PHPUnit!“ !== Testing
Quality in PHP projects
beyond Unittests
© MAYFLOWER GmbH 2008 23

24. Unit Testing
❙ For PHP: PHPUnit or SimpleTest
❙ Focussing on testing a single unit of work
❙ solves depencies using mock objects
❙ should not depend on database
❙ perfect and easy to automate
❙ Reason for all refactoring
Quality in PHP projects
beyond Unittests
© MAYFLOWER GmbH 2008 24

25. Acceptence Testing
❙ originally user acceptance testing
❙ one of the final testing stages
❙ change to automated black box testing in agile
environments
❙ used to test user stories
❙ often contractual requirement
Quality in PHP projects
beyond Unittests
© MAYFLOWER GmbH 2008 25

26. SOA / Service Testing
❙ in multi-tier environments every service can have several
consumers
❙ black box test for the service api
❙ can easily be automated
❙ a lot of tools for SOAP
Quality in PHP projects
beyond Unittests
© MAYFLOWER GmbH 2008 26

27. Integration Testing
❙ tests the application with all parts
❙ needed for multi-tier applications
❙ tests the interaction paths
❙ intersection with load testing: user simulation
Quality in PHP projects
beyond Unittests
© MAYFLOWER GmbH 2008 27

28. Load Testing
❙ Simulates high load situations
❙ to detect the current platform abilities
❙ to detect resource bottlenecks (Database, CPU,
Network)
❙ to detect concurreny bottlenecks (locking jams, caching
effects)
❙ Scalability testing: scale up or scale out?
Quality in PHP projects
beyond Unittests
© MAYFLOWER GmbH 2008 28

29. Usability Testing
❙ How does the user work with the application?
❙ Efficiency: time to fulfill a basic task
❙ Accuracy: how many mistakes happened?
❙ Recall: does the user recall the usage after a period of
time?
❙ Emotional response: does the user feel lucky, confident or
stressed about the task?
Quality in PHP projects
beyond Unittests
© MAYFLOWER GmbH 2008 29

30. Regression Testing
❙ originally: if you find a bug, fix it and write a test
❙ PHP itself does it
❙ now often written by the test team to assure the expected
default behavior
❙ modern: automated test generation
Quality in PHP projects
beyond Unittests
© MAYFLOWER GmbH 2008 30

31. Security Testing
❙ Blackbox: Penetration Test done by external auditor just
before release
❙ Graybox: using tools like fuzzers and scanners before
release or while development
❙ Whitebox: internal or external source code audit, static
code analysis while development
Quality in PHP projects
beyond Unittests
© MAYFLOWER GmbH 2008 31

32. A lot more ...
❙ Compability testing for maintainability
❙ Volume testing for bigger amounts of data
❙ Stress testing for degrade and failover processes
❙ Installation testing for maintenance
❙ Recovery testing to check recovery behavior
Quality in PHP projects
beyond Unittests
© MAYFLOWER GmbH 2008 32

33. PHP Source Code Quality
Quality in PHP projects
beyond Unittests
© MAYFLOWER GmbH 2008 33

34. Documentation
❙ We have different forms of documentation:
❙ Inline documentation
❙ API documentation
❙ Developer documentation
❙ Security documentation
❙ Administration documentation
❙ End-user documentation
Quality in PHP projects
beyond Unittests
© MAYFLOWER GmbH 2008 34

35. Coding Style / Formatting
❙ Coding style is defined in the developer documentation
❙ All developers have to write their code formatted in the
given coding style
❙ Can be checked by CodeSniffer tools
❙ Coding style should be based on your main used libraries
❙ If you build your application on top of the Zend
Framework, use the ZF coding style
❙ With IDEs like Eclipse you can format old code
autpmatically
Quality in PHP projects
beyond Unittests
© MAYFLOWER GmbH 2008 35

36. Naming Conventions
❙ Naming conventions are very important
❙ NonamespacesbeforePHP5.3
❙ Easier for __autoload()
❙ Examples
❙ Class names:
❙ class HTML_Template_ITX2
❙ Directory: include/PEAR2/HTML/Template/ITX2.php
❙ Function names in camel caps:
❙ public function blockExists($blockname)
Quality in PHP projects
beyond Unittests
© MAYFLOWER GmbH 2008 36

37. Code Complexity
❙ Complexity costs time and money if you don‘t need it
❙ KISS principle: Keep it simple and stupid
❙ Complexity checks by CodeSniffer metrics
❙ Important for good code coverage and unittests
❙ Examples
❙ Too high number of linearly independent paths through
a program's source code
❙ Too long class and method names
❙ Too many properties
❙ Too many parameters
❙ Too many methods inside a class
Quality in PHP projects
beyond Unittests
© MAYFLOWER GmbH 2008 37

38. Anti Patterns
❙ An antipattern is a design pattern that appears obvious but
is ineffective or far from optimal in practice
❙ Examples
❙ Unnecessary abstraction
❙ God objects
❙ Spaghetti code
❙ Too short or too long variable names
Quality in PHP projects
beyond Unittests
© MAYFLOWER GmbH 2008 38

39. Anti Pattern Examples
❙ if(<EXPR>) {<empty>}
❙ if(<EXPR>) {<EXPR>} else {<empty>}
❙ try(<EXPR>) {<empty>}
❙ try(<EXPR>) {<EXPR>} catch(Exception $e)
{<empty>}
❙ define('SOME_NAME', $var)
❙ include[_once]|require[_once]|readfile|
virtual|file_get_contents|fopen|file|
mysql_query($_GET|$_POST|$_REQUEST)
Quality in PHP projects
beyond Unittests
© MAYFLOWER GmbH 2008 39

40. Questions?
Quality in PHP projects
beyond Unittests
© MAYFLOWER GmbH 2008 40

41. Thank you very much!
Thorsten Rinne
Mayflower GmbH
Mannhardtstr. 6
80538 München
+49 (89) 24 20 54 - 31
thorsten.rinne@mayflower.de