Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

QDashboard 1.2

655 views

Published on

See http://massol.myxwiki.org/xwiki/bin/view/Blog/QDashboardSonarQube

Published in: Technology
  • Be the first to comment

  • Be the first to like this

QDashboard 1.2

  1. 1. Copyright 2006 Vincent Massol, all rights reserved QDashboard Proposal v 1.2 Vincent Massol, 9 Oct. 2006 TODO:
 Find a better name!
  2. 2. Copyright 2006 Vincent Massol, all rights reserved The Need ■ Projects are not controlled in term of quality It’s a best effort which results most often in poor quality and high maintenance costs There’s no tool to help monitor the quality over the duration of the project ■ Project health is also not controlled Results in delays to deliver and cost overrun Again there’s no tool to help monitor a project’s health over its course ■ Quality and Health are currently monitored manually (when they are monitored at all) Lots of guessing Manual and tedious work Inaccurate because based on people’s perception of how good and how finished a project is ■ QDashboard is about extracting tangible information from the project’s outputs (source code, artifacts, issue tracker, etc)
  3. 3. Copyright 2006 Vincent Massol, all rights reserved General goals ■ Develop a Quality and Health Check Dashboard to track quality and health of development projects ■ 3 main features Display metrics (historical and by committer) ■ For quality and project health Rank the projects ■ On quality and health criteria Suggest recommendations ■ To improve quality and health ■ Reuses existing open source tools for gathering metrics Checkstyle, PMD, findbugs, Xradar, JavaNCSS, JUnit, Xref, Cobertura, etc ■ Aggregate the results, store them over time, generate trends, generate reports, set alarm thresholds, correlate them together
  4. 4. Copyright 2006 Vincent Massol, all rights reserved Target Audience ■ Project Managers How is my project doing? ■ Project Directors How are my different projects doing? ■ Developers How am I doing in my team? ■ QA Managers/Teams How are all the projects doing, compared with each other?
  5. 5. Copyright 2006 Vincent Massol, all rights reserved Business value ■ Improve project quality Less bugs Easier maintenance ■ Improve respect of deadlines ■ Project audits Ex: Outsourced projects ■ Healthy comparison between different projects Leads to emulation and quality/productivity increase (if done right) ■ Increase developer’s responsibility By providing visibility on his work ■ Reward team members Factual recognition of work done Good developers can now be rewarded
  6. 6. Copyright 2006 Vincent Massol, all rights reserved Features The features are presented
 in no particular order
  7. 7. Copyright 2006 Vincent Massol, all rights reserved Recommendations & Rankings ■ Ranker to rank the different projects according to metrics Ranking in quality and in health Ability to compare projects Only rank common metrics. Take common metrics between project for ranking, i.e. ranking against Metric Set ■ Recommendation engine Adapted for different types of projects ■ ex.: OSS projects, Prototype (build and throw away), Core IS project (ex: banking project that must last 10 years), etc Configurable Well known Recommendation sets
  8. 8. Copyright 2006 Vincent Massol, all rights reserved Features (1) ■ Add project from a SCM URL Points to a Maven POM Manual by entering information such as location of sources, tests, etc ■ With automatic detection as much as possible ■ See http://blogs.codehaus.org/people/vmassol/archives/ 001369_intellib.html
  9. 9. Copyright 2006 Vincent Massol, all rights reserved Features (2) ■ Execute coding rules on project Checkstyle JavaNCSS Findbugs PMD CPD … Only select rules that everyone agree on by default
  10. 10. Copyright 2006 Vincent Massol, all rights reserved Features (3) ■ Measure project activity # Commits per day/week/month/developer Release frequency Nb active committers ■ Count number of tests ■ Measure Test coverage Need runtime model (see after) or integrate with a build/CI
  11. 11. Copyright 2006 Vincent Massol, all rights reserved Features (4) ■ Automatically discover the build used by the project and display it Ant, Maven1, Maven2, etc Give a negative point to project if no build ■ Provide an instance of QDashboard for open source projects Automatically discover M2 projects on ibiblio and generate dashboard for them To make advertisement for QDashboard
  12. 12. Copyright 2006 Vincent Massol, all rights reserved Features (5) ■ Provide individual stats by committer For example, how many rules John Doe has violated in his last commit? All reports (checkstyle, PMD, CPD, etc) should allow an individual view Goal: every developer must be aware of what he’s doing and be sensitive to quality. By emulation developers should automatically work towards improving their code quality
  13. 13. Copyright 2006 Vincent Massol, all rights reserved Features (6) ■ Display # of documentation pages (and quantity) modified by users So that not only code lines is made visible but also documentation Support xdoc, APT, FML, confluence, etc ■ Display individual statistics on: Modified code – main sources (src/main) Modified code – test sources (src/test) Modified documentation (see above) (src/site + wiki) Modified build files (pom.xml, profiles.xml) Other
  14. 14. Copyright 2006 Vincent Massol, all rights reserved Features (7) ■ Subscribe an Agent to the project mailing lists to measure project member activity In order to see who helps with questions and who communicates in general Nb answers per day/week/month on the list ■ Run Clirr on releases project versions To measure how much importance is given to binary compatibility. Need to take into account non-stable API ■ Must be marked manually by someone Need to take into account deprecations so that a removed deprecations does not trigger a bad point ■ Note: Project need to decide how long a deprecation should stay
  15. 15. Copyright 2006 Vincent Massol, all rights reserved Features (8) ■ JIRA integration How long issues are opened on average Who closes issues ■ Crucible integration Reviewed code %, % of code not reviewed Reviews in progress, when was last review done (if too long raise warning), etc ■ Top projects view List the project will the less violations ■ Need to ensure we compare apples with apples And worse projects! Good for the open source instance to list the top 50 projects for example. Provide emulation. Nobody wants to see his project in the worse category… Users will be more tempted to use a good quality and healthy project Provide the same view for internal project portofoli management
  16. 16. Copyright 2006 Vincent Massol, all rights reserved Features (9) ■ Indicator: # of mails on the mailing lists Per day/week/month Helps evaluate project’s activity and health Note: need to support forums too for projects not using mailing lists ■ Offer threshold warnings Ex: If # of PMD errors goes beyond N Ex: If no release done in N months Notify by Mail, IRC, IM
  17. 17. Copyright 2006 Vincent Massol, all rights reserved Features (10) ■ Allow projects to compare with each other Metric by metric Globally (need some ranking system – Not easy) ■ Integrate with bug tracker to measure # bugs / release Warning: more testers will lead to more bugs up to a level. Thus it’s not the projects with the most discovered bugs which are less good on quality! Need to be correlated with # of tests
  18. 18. Copyright 2006 Vincent Massol, all rights reserved Features (11) ■ Recommendations: Offer quick wins advice Provide suggestions to improve score Does this by comparing current project with other existing projects (open source ones for example) and find the top few metrics where the project is off the most Examples ■ If committer repeatedly increase # of checkstyle violations during last 5 commits then recommend fixing it and raise highlight ■ If no release in past 1 month then recommend release ■ If bug tracker issues pending for more than 1 month raise warning Use groovy for writing rules by offering Java objects to wrap around. This allows easy edition of rules and creating new ones by users. ■ Allow comments for each metric on all projects So that users/developers of project can provide feedback/explain the value in some metrics. Provide way for users to provide feedback on something not working ■ Automatically generate a JIRA issue for QDashboard for exemple (as is done by IntelliJ IDEA) This is one killer feature. 
 TODO: List 40 good
 recommendations rules
  19. 19. Copyright 2006 Vincent Massol, all rights reserved Features (12) ■ Scheduler Take snapshot of dashboard every day/week (+ on demand for releases) for example for historical trends ■ Display graphs for each metric ■ Support for subprojects Example: M2 modules ■ Display project versions milestones in metric graphs ■ Ability to create Parsers/Analyzers/Metrics directly from the webapp Including generic parser, etc ■ Create an online repository of Dashlets That can also be accessed and discovered from QDashboard directly
  20. 20. Copyright 2006 Vincent Massol, all rights reserved Architecture
  21. 21. Copyright 2006 Vincent Massol, all rights reserved System Architecture Dashboard
 Server Web Browser IDE plugins Mail, IRC, IM Bug Trackers Mailing lists Build/CI tools SCMs View results Data sources Crucible/FishEye
  22. 22. Copyright 2006 Vincent Massol, all rights reserved Webapp (Dashweb) Product stack Plugins & JSR-168 Portlets (Dashlets) Java API (Dashrock) ■ Dashlet = Portlet displaying a metric ■ + REST-type API for getting data from dashlets
  23. 23. Copyright 2006 Vincent Massol, all rights reserved Dashrock ■ 100% Java, strongly typed API ■ Ability for users to add their own Parser, Analyzer and Metric ■ Stateless architecture (i.e. users of Dashrock have to instantiate instances of Parser, Analyzer and Metric)
  24. 24. Copyright 2006 Vincent Massol, all rights reserved Dashlets ■ A Dashlet is a plugin for Dashweb In the form of a JAR ■ A Dashlet provides a JSR-168 compatible view that can display metrics computed by Dashrock ■ Allows embedding Dashrock Portlets in any JSR-88 Portlet container
  25. 25. Copyright 2006 Vincent Massol, all rights reserved Technical Architecture ■ (Could be built on top of XWiki/Exo) ■ Generic webapp and platform Metrics, Dashlets, Analyzers, Parsers, Recos can be added from repositories • Metrics • Parsers • Dashlets Remote repo Local repo Dashlet 1 Dashlet 2 Dashlet N Dashweb Scheduler Project Mgr Renderer Ranker Reco. Engine Dashrock
  26. 26. Copyright 2006 Vincent Massol, all rights reserved Database Parser JUnit
 Parser Checkstyle
 Parser Parse and persist data Analyzer Importer Metric Monthly
 Aggregator Test Coverage
 /LOCRead parsed data and write analyzed data Nb of JUnit tests Test Coverage % Create project Retrieve metric Dashrock Architecture Persistence API Create composite metric from several parsed information
  27. 27. Copyright 2006 Vincent Massol, all rights reserved Open questions ■ Do we execute the quality tools within QDashboard? For example do we execute Checkstyle? ■ In which case we need a Project object and need to be able to build a project. Could be done by embedding M2 for example. Or do we only parse the results of an execution ■ In which case we need to integrate with build tools
  28. 28. Copyright 2006 Vincent Massol, all rights reserved Appendix A: Competitors ■ CAST http://www.castsoftware.com/Products/Index.html ■ JetBrain’s TeamCity Especially with inspections and code coverage ■ Mergere ■ Atlassian
  29. 29. Copyright 2006 Vincent Massol, all rights reserved Appendix B: Directory structure ■ QDashboard/ dashrock/ ■ api/ ■ parsers/ ■ analyzers/ ■ metrics/ ■ importers/ dashlet/ ■ api/ ■ dashlets/ dashweb/

×