provectus.com
Code quality
Developing code quality process
provectus.com
What is code quality?
• Code style
• Code complexity (size of files/functions, cyclomatic complexity)
• Duplicated code
• Documentation, comments
• Test coverage
provectus.com
Why do we need to control quality of code?
Code quality correlates with technical debt.
Big technical debt leads to bugs and additional efforts required for new functionality.
As result bad code quality means financial loss (transitive).
provectus.com
Java code quality tools
• Code style control: Checkstyle
• Code errors control: PMD, FindBugs
• Test coverage: Jacoco, EMMA, Cobertura
• SonarQube
provectus.com
What does CheckStyle check
• Formatting: indents, braces, etc.
• Unused imports
• Redundant modifiers (e.g. “public” modifier in interfaces)
• Maximum function parameters number
• Magic numbers
• Hidden fields
• Naming conventions
• hashCode() and equals() contract
• Number of lines in classes / functions
• RegExp. checks
provectus.com
Checkstyle not only for java
<module name="Checker">
<module name="RegexpSingleline">
<property name="format"
value="(?i)((VARCHAR2)|(VARCHAR))s*(s*d+s+((char)|(byte)))"/>
<property name="minimum" value="0"/>
<property name="maximum" value="0"/>
<property name="message"value="Don't specify character's size
VARCHAR2(XXX CHAR / BYTE). "/>
</module>
</module>
How we prevented columns declared like following in our SQL changesets.
columnName VARCHAR2(400 BYTE)
provectus.com
What does PMD check
• Double Checked Locking
• return statement in finally block
• Redundant checks, e.g. if (a!=null && method1().equals(a))
• Constructions like new BigInteger()
• Catching of Throwable, NPE, Exception, Error
• Usage implementation types (i.e., HashSet) instead od interface
• Usingusing implementation types (i.e., HashSet); use the interface
• Usage of System.out.println
• Unused parameters, variables, private methods
provectus.com
What does FindBugs check
• Places with defined compareTo() without Object.equals()
• Unclosed streams and Statement objects
• Potential NPE
• Redundant null checks
• Self assignment. Example from our project:
public void setInventoryManager(LocalizingInventoryManager pInventoryManager) {
this.mInventoryManager = mInventoryManager;
}
• Synchronization problems
• Duplicated code in conditional statements
• Dead local variables
provectus.com
FindBugs is really cool
• FindBugs has found that in line 59 null can be potentially passed as parameter. But in 68
line a method called on this object without checking for null.
provectus.com
provectus.com
Merge error found by FindBugs
provectus.com
Redundant check for null
provectus.com
CI build
We have CI job on Jenkins that checks repository every 30 minutes.
If changes were found, CI build runs Checkstyle, PMD, FindBugs and Unit tests.
If build fails, then Jenkins informs dev team about it via email.
provectus.com
SonarQube
First SonarQube was just a web interface for Checkstyle, PMD and FindBugs. But now SonarQube
uses it’s own analyzer and set of rules.
Also SonarQube shows errors diff between analyze runs. And it’s show author for each issue.
provectus.com
provectus.com
What can skilled developer write being in rush or
because of other objective factors
public boolean isXXX(...) {
if ( <condition> ) {
return true;
} else {
return false;
}
}
provectus.com
@Override
public void setPropertyValue(RepositoryItemImpl pItem, Object pValue) {
try {
super.setPropertyValue(pItem, pValue);
}catch(Exception e){
e.printStackTrace();
}
}
Boolean b = <some invocation>;
if (b != null && b.equals(Boolean.TRUE))
provectus.com
What we wanted to do
• Using same coding style on the project
• Prevent new “stupid” problem before code review
• Prevent issues that hardly can be found by human, but can be found automatically
provectus.com
How we started code quality process – steps:
1. Rules filtration
2. Instruction with selected rules
3. Instruction how to use tools and IDE plugins
4. Build script modification in separate branch
5. Merge to master
provectus.com
Selecting of rules for project
ATG doesn’t follow all JCC rules and best practices, that’s why some rules were filtered out, e.g.
ATG defines class version for it’s components like this:
String CLASS_VERSION = "$Id:
//product/DCS/version/9.3/Java/atg/commerce/order/Order.java#3 $$Change: 633147 $";
provectus.com
Code quality tools on our project
• ~60 000 Checkstyle violations
• ~ 2 000 PMD issues
• couldn’t fix all of them
• Rule: threshold value = current # of issues
• build fails if # of violations > threshold
• Rule for merges: threshold value = # of issues after merge
• New ANT task for updating threshold value after merge (temporary and bad solution)
provectus.com
Documentation on wiki
We created wiki pages with detailed information about rules were planning to use. Links pages
were sent to all developer so they could tell their objections.
Also we prepared guide instructions how to install and configure IDE plugins for CheckStyle and
PMD.
provectus.com
How much time has it taken
• Checkstyle:
– selecting rules for our project – 6 hour
– modifying ant script – 8 hours
– creating IDE (Eclipse + Idea) configurations – 2 hours
– Writing instruction on wiki – 2 hours
• PMD:
– selecting rules for our project – 8 hours
– modifying ant script – 2 hours
– creating IDE (Eclipse + Idea) configurations – 2 hours
– writing instruction on wiki – 1 hour
provectus.com
SonarQube on our project
SonarQube can be used to monitor new issues with their authors.
If someone decides to alter threshold value, it will be seen in SonarQube.
provectus.com
Time for developing code quality process
• Checkstyle check was developed in free time and presented as first step of code quality
process
• PMD and FindBugs checks were developed in project time
provectus.com
Managers role in code quality process
• Project manager should understand importance of code quality process, and how negative
growing technical debt is.
• Manager shouldn’t think of code quality process as minor thing that has lowest priority.
• Ideally manager should plan code quality related task as project time.
provectus.com
How to explain the need in code quality to manager
Give to a manager an example how code quality will have to improve situation on project.
E.g. it will reduce a number of bugs related to null-pointer exception.
provectus.com
Formal workflow
Formal process (related to code quality) should be defined:
• Required actions before pushing changes
• What to do if build fails on CI
provectus.com
Workflow on our project
• On our project each developer should perform Checkstyle+PMD check (using Ant task)
before pushing changes.
• FindBugs check is implemented as separate task, and it doesn’t fail build.
• If Checkstyle or PMD fails on CI, developers are informed via email and CCTray.
provectus.com
Refactoring
Code quality process has two goals:
• Don’t increase technical debt by adding new issues
• Get rid of existing problems by refactor code
Many developers afraid of refactoring because it can cause regression.
Part of code that is under refactoring should be covered with unit tests for all cases. It takes
much time, but it’s the only right way.
provectus.com
Human factor
Any rules about coding should be checked automatically. Don’t try to solve anything by
agreement.
On our project most developers ignored emails regarding Checkstyle and PMD, until their build
failed.
provectus.com
Summary
• Fix styling problems as soon as possible. Don’t use violations threshold with Checkstyle
• Both developers and managers should be involved in code quality process: metrics,
refactoring tasks
• Information about code quality process should be delivered to developers in the most
convenient (for them) way: explanation on small meeting, presentation, short article on wiki,
video

Code quality

  • 1.
  • 2.
    provectus.com What is codequality? • Code style • Code complexity (size of files/functions, cyclomatic complexity) • Duplicated code • Documentation, comments • Test coverage
  • 3.
    provectus.com Why do weneed to control quality of code? Code quality correlates with technical debt. Big technical debt leads to bugs and additional efforts required for new functionality. As result bad code quality means financial loss (transitive).
  • 4.
    provectus.com Java code qualitytools • Code style control: Checkstyle • Code errors control: PMD, FindBugs • Test coverage: Jacoco, EMMA, Cobertura • SonarQube
  • 5.
    provectus.com What does CheckStylecheck • Formatting: indents, braces, etc. • Unused imports • Redundant modifiers (e.g. “public” modifier in interfaces) • Maximum function parameters number • Magic numbers • Hidden fields • Naming conventions • hashCode() and equals() contract • Number of lines in classes / functions • RegExp. checks
  • 6.
    provectus.com Checkstyle not onlyfor java <module name="Checker"> <module name="RegexpSingleline"> <property name="format" value="(?i)((VARCHAR2)|(VARCHAR))s*(s*d+s+((char)|(byte)))"/> <property name="minimum" value="0"/> <property name="maximum" value="0"/> <property name="message"value="Don't specify character's size VARCHAR2(XXX CHAR / BYTE). "/> </module> </module> How we prevented columns declared like following in our SQL changesets. columnName VARCHAR2(400 BYTE)
  • 7.
    provectus.com What does PMDcheck • Double Checked Locking • return statement in finally block • Redundant checks, e.g. if (a!=null && method1().equals(a)) • Constructions like new BigInteger() • Catching of Throwable, NPE, Exception, Error • Usage implementation types (i.e., HashSet) instead od interface • Usingusing implementation types (i.e., HashSet); use the interface • Usage of System.out.println • Unused parameters, variables, private methods
  • 8.
    provectus.com What does FindBugscheck • Places with defined compareTo() without Object.equals() • Unclosed streams and Statement objects • Potential NPE • Redundant null checks • Self assignment. Example from our project: public void setInventoryManager(LocalizingInventoryManager pInventoryManager) { this.mInventoryManager = mInventoryManager; } • Synchronization problems • Duplicated code in conditional statements • Dead local variables
  • 9.
    provectus.com FindBugs is reallycool • FindBugs has found that in line 59 null can be potentially passed as parameter. But in 68 line a method called on this object without checking for null.
  • 10.
  • 11.
  • 12.
  • 13.
    provectus.com CI build We haveCI job on Jenkins that checks repository every 30 minutes. If changes were found, CI build runs Checkstyle, PMD, FindBugs and Unit tests. If build fails, then Jenkins informs dev team about it via email.
  • 14.
    provectus.com SonarQube First SonarQube wasjust a web interface for Checkstyle, PMD and FindBugs. But now SonarQube uses it’s own analyzer and set of rules. Also SonarQube shows errors diff between analyze runs. And it’s show author for each issue.
  • 15.
  • 16.
    provectus.com What can skilleddeveloper write being in rush or because of other objective factors public boolean isXXX(...) { if ( <condition> ) { return true; } else { return false; } }
  • 17.
    provectus.com @Override public void setPropertyValue(RepositoryItemImplpItem, Object pValue) { try { super.setPropertyValue(pItem, pValue); }catch(Exception e){ e.printStackTrace(); } } Boolean b = <some invocation>; if (b != null && b.equals(Boolean.TRUE))
  • 18.
    provectus.com What we wantedto do • Using same coding style on the project • Prevent new “stupid” problem before code review • Prevent issues that hardly can be found by human, but can be found automatically
  • 19.
    provectus.com How we startedcode quality process – steps: 1. Rules filtration 2. Instruction with selected rules 3. Instruction how to use tools and IDE plugins 4. Build script modification in separate branch 5. Merge to master
  • 20.
    provectus.com Selecting of rulesfor project ATG doesn’t follow all JCC rules and best practices, that’s why some rules were filtered out, e.g. ATG defines class version for it’s components like this: String CLASS_VERSION = "$Id: //product/DCS/version/9.3/Java/atg/commerce/order/Order.java#3 $$Change: 633147 $";
  • 21.
    provectus.com Code quality toolson our project • ~60 000 Checkstyle violations • ~ 2 000 PMD issues • couldn’t fix all of them • Rule: threshold value = current # of issues • build fails if # of violations > threshold • Rule for merges: threshold value = # of issues after merge • New ANT task for updating threshold value after merge (temporary and bad solution)
  • 22.
    provectus.com Documentation on wiki Wecreated wiki pages with detailed information about rules were planning to use. Links pages were sent to all developer so they could tell their objections. Also we prepared guide instructions how to install and configure IDE plugins for CheckStyle and PMD.
  • 23.
    provectus.com How much timehas it taken • Checkstyle: – selecting rules for our project – 6 hour – modifying ant script – 8 hours – creating IDE (Eclipse + Idea) configurations – 2 hours – Writing instruction on wiki – 2 hours • PMD: – selecting rules for our project – 8 hours – modifying ant script – 2 hours – creating IDE (Eclipse + Idea) configurations – 2 hours – writing instruction on wiki – 1 hour
  • 24.
    provectus.com SonarQube on ourproject SonarQube can be used to monitor new issues with their authors. If someone decides to alter threshold value, it will be seen in SonarQube.
  • 25.
    provectus.com Time for developingcode quality process • Checkstyle check was developed in free time and presented as first step of code quality process • PMD and FindBugs checks were developed in project time
  • 26.
    provectus.com Managers role incode quality process • Project manager should understand importance of code quality process, and how negative growing technical debt is. • Manager shouldn’t think of code quality process as minor thing that has lowest priority. • Ideally manager should plan code quality related task as project time.
  • 27.
    provectus.com How to explainthe need in code quality to manager Give to a manager an example how code quality will have to improve situation on project. E.g. it will reduce a number of bugs related to null-pointer exception.
  • 28.
    provectus.com Formal workflow Formal process(related to code quality) should be defined: • Required actions before pushing changes • What to do if build fails on CI
  • 29.
    provectus.com Workflow on ourproject • On our project each developer should perform Checkstyle+PMD check (using Ant task) before pushing changes. • FindBugs check is implemented as separate task, and it doesn’t fail build. • If Checkstyle or PMD fails on CI, developers are informed via email and CCTray.
  • 30.
    provectus.com Refactoring Code quality processhas two goals: • Don’t increase technical debt by adding new issues • Get rid of existing problems by refactor code Many developers afraid of refactoring because it can cause regression. Part of code that is under refactoring should be covered with unit tests for all cases. It takes much time, but it’s the only right way.
  • 31.
    provectus.com Human factor Any rulesabout coding should be checked automatically. Don’t try to solve anything by agreement. On our project most developers ignored emails regarding Checkstyle and PMD, until their build failed.
  • 32.
    provectus.com Summary • Fix stylingproblems as soon as possible. Don’t use violations threshold with Checkstyle • Both developers and managers should be involved in code quality process: metrics, refactoring tasks • Information about code quality process should be delivered to developers in the most convenient (for them) way: explanation on small meeting, presentation, short article on wiki, video

Editor's Notes

  • #22 We already had about 60 000 Checkstyle violations and about 2 000 PMD issues. We couldn’t fix all of them, so we have defined a threshold value equal to current number of issues. If number of found violations is greater than threshold, then build fails. For merges we created task that sets threshold to current violations number.