QDashboard 1.2

Copyright 2006 Vincent Massol, all rights reserved
QDashboard
Proposal v 1.2
Vincent Massol, 9 Oct. 2006
TODO: 
Find a better name!

The Need
■ Projects are not controlled in term of quality
It’s a best effort which results most often in poor quality and high maintenance
costs
There’s no tool to help monitor the quality over the duration of the project
■ Project health is also not controlled
Results in delays to deliver and cost overrun
Again there’s no tool to help monitor a project’s health over its course
■ Quality and Health are currently monitored manually (when they are
monitored at all)
Lots of guessing
Manual and tedious work
Inaccurate because based on people’s perception of how good and how finished
a project is
■ QDashboard is about extracting tangible information from the project’s
outputs (source code, artifacts, issue tracker, etc)

General goals
■ Develop a Quality and Health Check Dashboard to track quality and
health of development projects
■ 3 main features
Display metrics (historical and by committer)
■ For quality and project health
Rank the projects
■ On quality and health criteria
Suggest recommendations
■ To improve quality and health
■ Reuses existing open source tools for gathering metrics
Checkstyle, PMD, findbugs, Xradar, JavaNCSS, JUnit, Xref, Cobertura, etc
■ Aggregate the results, store them over time, generate trends, generate
reports, set alarm thresholds, correlate them together

Target Audience
■ Project Managers
How is my project doing?
■ Project Directors
How are my different projects doing?
■ Developers
How am I doing in my team?
■ QA Managers/Teams
How are all the projects doing, compared with each
other?

Business value
■ Improve project quality
Less bugs
Easier maintenance
■ Improve respect of deadlines
■ Project audits
Ex: Outsourced projects
■ Healthy comparison between different projects
Leads to emulation and quality/productivity increase (if done right)
■ Increase developer’s responsibility
By providing visibility on his work
■ Reward team members
Factual recognition of work done
Good developers can now be rewarded

Features
The features are presented 
in no particular order

Recommendations & Rankings
■ Ranker to rank the different projects according to metrics
Ranking in quality and in health
Ability to compare projects
Only rank common metrics. Take common metrics between
project for ranking, i.e. ranking against Metric Set
■ Recommendation engine
Adapted for different types of projects
■ ex.: OSS projects, Prototype (build and throw away), Core IS project
(ex: banking project that must last 10 years), etc
Configurable
Well known Recommendation sets

Features (1)
■ Add project from a SCM URL
Points to a Maven POM
Manual by entering information such as location of
sources, tests, etc
■ With automatic detection as much as possible
■ See http://blogs.codehaus.org/people/vmassol/archives/
001369_intellib.html

Features (2)
■ Execute coding rules on project
Checkstyle
JavaNCSS
Findbugs
PMD
CPD
…
Only select rules that everyone agree on by default

Features (3)
■ Measure project activity
# Commits per day/week/month/developer
Release frequency
Nb active committers
■ Count number of tests
■ Measure Test coverage
Need runtime model (see after) or integrate
with a build/CI

Features (4)
■ Automatically discover the build used by the
project and display it
Ant, Maven1, Maven2, etc
Give a negative point to project if no build
■ Provide an instance of QDashboard for open
source projects
Automatically discover M2 projects on ibiblio and
generate dashboard for them
To make advertisement for QDashboard

Features (5)
■ Provide individual stats by committer
For example, how many rules John Doe has violated
in his last commit?
All reports (checkstyle, PMD, CPD, etc) should allow
an individual view
Goal: every developer must be aware of what he’s
doing and be sensitive to quality. By emulation
developers should automatically work towards
improving their code quality

Features (6)
■ Display # of documentation pages (and quantity) modified
by users
So that not only code lines is made visible but also
documentation
Support xdoc, APT, FML, confluence, etc
■ Display individual statistics on:
Modified code – main sources (src/main)
Modified code – test sources (src/test)
Modified documentation (see above) (src/site + wiki)
Modified build files (pom.xml, profiles.xml)
Other

Features (7)
■ Subscribe an Agent to the project mailing lists to measure
project member activity
In order to see who helps with questions and who communicates
in general
Nb answers per day/week/month on the list
■ Run Clirr on releases project versions
To measure how much importance is given to binary compatibility.
Need to take into account non-stable API
■ Must be marked manually by someone
Need to take into account deprecations so that a removed
deprecations does not trigger a bad point
■ Note: Project need to decide how long a deprecation should stay

Features (8)
■ JIRA integration
How long issues are opened on average
Who closes issues
■ Crucible integration
Reviewed code %, % of code not reviewed
Reviews in progress, when was last review done (if too long raise warning), etc
■ Top projects view
List the project will the less violations
■ Need to ensure we compare apples with apples
And worse projects!
Good for the open source instance to list the top 50 projects for example.
Provide emulation. Nobody wants to see his project in the worse category…
Users will be more tempted to use a good quality and healthy project
Provide the same view for internal project portofoli management

Features (9)
■ Indicator: # of mails on the mailing lists
Per day/week/month
Helps evaluate project’s activity and health
Note: need to support forums too for projects not using
mailing lists
■ Offer threshold warnings
Ex: If # of PMD errors goes beyond N
Ex: If no release done in N months
Notify by Mail, IRC, IM

Features (10)
■ Allow projects to compare with each other
Metric by metric
Globally (need some ranking system – Not easy)
■ Integrate with bug tracker to measure # bugs /
release
Warning: more testers will lead to more bugs up to a
level. Thus it’s not the projects with the most
discovered bugs which are less good on quality! Need
to be correlated with # of tests

Features (11)
■ Recommendations: Offer quick wins advice
Provide suggestions to improve score
Does this by comparing current project with other existing projects (open source
ones for example) and find the top few metrics where the project is off the most
Examples
■ If committer repeatedly increase # of checkstyle violations during last 5 commits then
recommend fixing it and raise highlight
■ If no release in past 1 month then recommend release
■ If bug tracker issues pending for more than 1 month raise warning
Use groovy for writing rules by offering Java objects to wrap around. This allows
easy edition of rules and creating new ones by users.
■ Allow comments for each metric on all projects
So that users/developers of project can provide feedback/explain the value in
some metrics.
Provide way for users to provide feedback on something not working
■ Automatically generate a JIRA issue for QDashboard for exemple (as is done by IntelliJ
IDEA)
This is one killer feature.  
TODO: List 40 good 
recommendations rules

Features (12)
■ Scheduler
Take snapshot of dashboard every day/week (+ on demand for releases)
for example for historical trends
■ Display graphs for each metric
■ Support for subprojects
Example: M2 modules
■ Display project versions milestones in metric graphs
■ Ability to create Parsers/Analyzers/Metrics directly from the webapp
Including generic parser, etc
■ Create an online repository of Dashlets
That can also be accessed and discovered from QDashboard directly

Architecture

System Architecture
Dashboard 
Server
Web Browser
IDE plugins
Mail, IRC, IM
Bug Trackers
Mailing lists
Build/CI tools
SCMs
View results Data sources
Crucible/FishEye

Webapp (Dashweb)
Product stack
Plugins & JSR-168 Portlets (Dashlets)
Java API (Dashrock)
■ Dashlet = Portlet displaying a metric
■ + REST-type API for getting data from dashlets

Dashrock
■ 100% Java, strongly typed API
■ Ability for users to add their own Parser,
Analyzer and Metric
■ Stateless architecture (i.e. users of
Dashrock have to instantiate instances of
Parser, Analyzer and Metric)

Dashlets
■ A Dashlet is a plugin for Dashweb
In the form of a JAR
■ A Dashlet provides a JSR-168 compatible
view that can display metrics computed by
Dashrock
■ Allows embedding Dashrock Portlets in
any JSR-88 Portlet container

Technical Architecture
■ (Could be built on top of XWiki/Exo)
■ Generic webapp and platform
Metrics, Dashlets, Analyzers, Parsers, Recos can be added from repositories
• Metrics
• Parsers
• Dashlets
Remote repo
Local
repo
Dashlet 1 Dashlet 2 Dashlet N
Dashweb
Scheduler
Project Mgr
Renderer
Ranker
Reco. Engine
Dashrock

Database
Parser
JUnit 
Parser
Checkstyle 
Parser
Parse and
persist data
Analyzer
Importer
Metric
Monthly 
Aggregator
Test Coverage 
/LOCRead parsed
data and write
analyzed data
Nb of JUnit
tests
Test
Coverage %
Create project
Retrieve metric
Dashrock Architecture
Persistence API
Create composite
metric from
several parsed
information

Open questions
■ Do we execute the quality tools within
QDashboard?
For example do we execute Checkstyle?
■ In which case we need a Project object and need to
be able to build a project. Could be done by
embedding M2 for example.
Or do we only parse the results of an
execution
■ In which case we need to integrate with build tools

Appendix A: Competitors
■ CAST
http://www.castsoftware.com/Products/Index.html
■ JetBrain’s TeamCity
Especially with inspections and code coverage
■ Mergere
■ Atlassian

Appendix B: Directory structure
■ QDashboard/
dashrock/
■ api/
■ parsers/
■ analyzers/
■ metrics/
■ importers/
dashlet/
■ api/
■ dashlets/
dashweb/

QDashboard 1.2

More Related Content

What's hot

Similar to QDashboard 1.2

More from Vincent Massol

Recently uploaded

QDashboard 1.2