Continuous Code Quality with the sonar ecosystem
•4 likes•2,301 views
Download to read offline
Report
Continuous Code Quality with the SonarEcosystem SonarQube is the leading platform for static code analysis and Continuous Code Quality. In this talk we will look into all three lines of defense of the SonarEcosystem and how they can help to find bugs before they enter your codebase (or at least go into production). After this talk, you’ll have a good overview of the SonarEcosystem as well as actionable starting points for increasing your code quality. Furthermore, we will share learnings from using SonarQube for more than 4 years and pointers to additional resources. About the Speaker: As Chief Technical Officer, Roman Pickl is in charge of technical development at Fluidtime. He has comprehensive experience in project management, the technical coordination of national and international mobility projects and the optimisation of business and development processes.
Recommended
More Related Content
What's hot
What's hot(20)
Similar to Continuous Code Quality with the sonar ecosystem
Similar to Continuous Code Quality with the sonar ecosystem(20)
More from Roman Pickl
More from Roman Pickl(8)
Recently uploaded
Recently uploaded(20)
Continuous Code Quality with the sonar ecosystem
- 1. Continuous Code Quality with the SonarEcosystem Roman Pickl (roman.pickl@fluidtime.com)
- 2. Fluidtime Enabling Smart Mobility. © 2017 Copyright Fluidtime Data Services GmbH At a glance 10+ years’ experience in integrating transport systems 22 mobility Apps in the stores 43+ million requests / month 500.000+ unique users / month qando Vienna, Linz, Graz, Salzburg, Klagenfurt. SMILE Lighthouse project for integrated mobility Wien Mobil Lab Vienna FluidHub Powering the Integrated Mobility and MaaS Ecosystem 2009 avv connect Aachen Region, North Rhine-Westphalia. 2014 NUMO Vienna 2012 2015 2016
- 3. Fluidtime Enabling Smart Mobility. © 2017 Copyright Fluidtime Data Services GmbH How did I end up here? Roman Pickl (@rompic) CTO @ Fluidtime In charge of the technical development Using SonarQube since 2013 Met Dominik at GeeCON Prague 2017 Here to learn
- 4. Fluidtime Enabling Smart Mobility. © 2017 Copyright Fluidtime Data Services GmbH Agenda Code Quality Continuous Inspection with Three Lines of Defense • Pre-Commit Analysis: SonarLint • Pull Request Analysis • SonarQube: Managing the Leak / Quality Gates and more Learnings Summary Demo Additional Resources
- 5. Fluidtime Enabling Smart Mobility. © 2017 Copyright Fluidtime Data Services GmbH What gets measured gets managed I often say that when you can measure what you are speaking about, and express it in numbers, you know something about it; but when you cannot express it in numbers, your knowledge is of a meagre and unsatisfactory kind; it may be the beginning of knowledge, but you have scarcely, in your thoughts, advanced to the stage of science, whatever the matter may be. William Thomson, 1. Baron Kelvin https://athinkingperson.com/2012/12/02/who-said-what-gets- measured-gets-managed/
- 6. Fluidtime Enabling Smart Mobility. © 2017 Copyright Fluidtime Data Services GmbH Software Product Quality > Code Quality www.mif.vu.lt/~sigitas/Quality/04_SQuaRE.ppt ISO software quality model (ISO/IEC 25010)
- 7. Fluidtime Enabling Smart Mobility. © 2017 Copyright Fluidtime Data Services GmbH SQALE Model (Technical Debt Pyramid) Testability Index Reliability Index Changeability Index Efficiency Index Security Index Maintainability Index Portability Index Reusability Index https://en.wikipedia.org/wiki/SQALE
- 8. © 2017 Copyright Fluidtime Data Services GmbH | www.fluidtime.com Fluidtime Enabling Smart Mobility. http://www.osnews.com /story/19266/Smells_m
- 9. Fluidtime Enabling Smart Mobility. © 2017 Copyright Fluidtime Data Services GmbH SonarQube quality model Evolved SQALE model Bugs, Vulnerabilities and Code Smells are 1st class citizens • Bugs: Code that is demonstrably wrong or highly likely to yield unexpected behaviour. • Vulnerabilities: Code that is potentially vulnerable to exploitation by hackers. • Code Smells: Will confuse maintainers or give them pause. Not only ratings, but also approximate remediation efforts. https://blog.sonarsource.com/bugs-and-vulnerabilities-are-1st-class-citizens- in-sonarqube-quality-model-along-with-code-smells/
- 10. Fluidtime Enabling Smart Mobility. © 2017 Copyright Fluidtime Data Services GmbH http://www.sasqag.org/pastmeetings/QualityPlans.pdf
- 11. Fluidtime Enabling Smart Mobility. © 2017 Copyright Fluidtime Data Services GmbH Catch those bugs early in the process September 9, 1947 „At 3:45 p.m., Grace Murray Hopper records the first computer bug in her log book as she worked on the Harvard Mark II“ http://www.computerhistory.org/tdih/September/9/
- 12. Fluidtime Enabling Smart Mobility. © 2017 Copyright Fluidtime Data Services GmbH SonarLint https://blog.sonarsource.com/putting-it-all-together-end- to-end-quality-with-sonarecosystem/ Pull RequestCode Locally Trunk Release1 2 3 Three Lines of Defense
- 13. Fluidtime Enabling Smart Mobility. © 2017 Copyright Fluidtime Data Services GmbH First line of defense : SonarLint (by SonarSource) Extension of your IDE / CL LGPL v3 On-the-fly feedback Pre commit analysis (Fix issues before they exist) Local or connected mode Included languages • Local: Java, JS, PHP, Python • More with connected mode http://www.sonarlint.org/
- 14. Fluidtime Enabling Smart Mobility. © 2017 Copyright Fluidtime Data Services GmbH SonarLint for IntelliJ
- 15. Fluidtime Enabling Smart Mobility. © 2017 Copyright Fluidtime Data Services GmbH SonarLint Connected Mode Bind to project on SonarQube server Use analyzers, quality profiles & settings from your SonarQube server Shared Custom Rule Sets Support for additional languages (not all plugins!)
- 16. Fluidtime Enabling Smart Mobility. © 2017 Copyright Fluidtime Data Services GmbH SonarLint for Command Line
- 17. Fluidtime Enabling Smart Mobility. © 2017 Copyright Fluidtime Data Services GmbH Pull Request AnalysisSonarLint X Pull RequestCode Locally Trunk Release https://blog.sonarsource.com/putting-it-all-together-end- to-end-quality-with-sonarecosystem/ Three Lines of Defense 1 2 3
- 18. Fluidtime Enabling Smart Mobility. © 2017 Copyright Fluidtime Data Services GmbH Pull Request Analysis (GitHub (SonarSource), BitBucket/Stash & GitLab (Community)) SonarQube Server must be up and running. Plugin installed on SonarQube Server Run for each commit / pull|merge request Preview analysis Adds an inline comment for each issue Adds a global summary Updates the status of the analysis Human reviewer can focus on other issues
- 19. © 2017 Copyright Fluidtime Data Services GmbH | www.fluidtime.com Fluidtime Enabling Smart Mobility. https://gitlab.talanlabs.com/gabriel-allaigre/sonar-gitlab-plugin Run: mvn --batch-mode verify sonar:sonar -Dsonar.host.url=$SONAR_URL - Dsonar.analysis.mode=preview -Dsonar.gitlab.commit_sha=$CI_BUILD_REF - Dsonar.gitlab.ref_name=$CI_BUILD_REF_NAME - Dsonar.gitlab.project_id=$CI_PROJECT_ID Setup GitLab in SonarQube Link the project in SonarQube to GitLab
- 20. Fluidtime Enabling Smart Mobility. © 2017 Copyright Fluidtime Data Services GmbH Links to plugins (not all of them are in the SonarQube update center yet) GitHub: https://docs.sonarqube.org/display/PLUG/GitHub+Plugin BitBucket: https://github.com/mibexsoftware/sonar-bitbucket-plugin Stash: https://github.com/AmadeusITGroup/sonar-stash GitLab: https://github.com/gabrie-allaigre/sonar-gitlab-plugin or https://git.johnnei.org/Johnnei/sonar-gitlab-plugin/tree/v0.2.0 (see discussion in https://groups.google.com/forum/#!topic/sonarqube/naLLNDD2JAM)
- 21. Fluidtime Enabling Smart Mobility. © 2017 Copyright Fluidtime Data Services GmbH Pull Request Analysis Quality Gates & Fixing the leak SonarLint X X Pull RequestCode Locally Trunk Release https://blog.sonarsource.com/putting-it-all-together-end- to-end-quality-with-sonarecosystem/ Three Lines of Defense 1 2 3
- 22. Fluidtime Enabling Smart Mobility. © 2017 Copyright Fluidtime Data Services GmbH SonarQube Server (Developed by SonarSource; GNU LGPL v3) 20+ languages Wide range of plugins (Auth, SCM, Language, External Analyzers, …) • external Analyzers like Findbugs/PMD; Most of the functionality already included in SonarJava analyzer (https://blog.sonarsource.com/sonarqube-java-analyzer-the- only-rule-engine-you-need/) 2 Versions • Latest (Always in the middle of a major refactoring; Next LTS forecast mid-2017) • LTS (use this if you apply any community plugins) Also available as a service( free for open source projects) https://blog.sonarsource.com/walking-the-tightrope-balancing- agility-and-stability/
- 23. Fluidtime Enabling Smart Mobility. © 2017 Copyright Fluidtime Data Services GmbH SonarQube 6.3 (latest)
- 24. Fluidtime Enabling Smart Mobility. © 2017 Copyright Fluidtime Data Services GmbH Architecture https://docs.sonarqube.org/display/SONAR/Architecture+and +Integration
- 25. Fluidtime Enabling Smart Mobility. © 2017 Copyright Fluidtime Data Services GmbH Quality Gates I Best way to enforce a quality policy in your organization indicates whether your project is releaseable collection of go/no-go conditions Each gate condition is a combination of : • Measure • period: Value (to date) or Leak (differential value over the Leak period) • comparison operator • warning value (optional) • error value (optional) https://docs.sonarqube.org/display/SONAR/Quality+Gates
- 26. Fluidtime Enabling Smart Mobility. © 2017 Copyright Fluidtime Data Services GmbH Quality Gates II
- 27. Fluidtime Enabling Smart Mobility. © 2017 Copyright Fluidtime Data Services GmbH Fixing the leak Do you reach for the mop? Or do you try to find the source and fix it? Clean up as you update and refactor your code over time https://docs.sonarqube.org/display/HOME/Fixing+the+Water+Leak
- 28. Fluidtime Enabling Smart Mobility. © 2017 Copyright Fluidtime Data Services GmbH Fix the leak
- 29. Fluidtime Enabling Smart Mobility. © 2017 Copyright Fluidtime Data Services GmbH Detailed information about bugs found / commiters / coverage / effort to fix etc.
- 30. Fluidtime Enabling Smart Mobility. © 2017 Copyright Fluidtime Data Services GmbH
- 31. Fluidtime Enabling Smart Mobility. © 2017 Copyright Fluidtime Data Services GmbH
- 32. Fluidtime Enabling Smart Mobility. © 2017 Copyright Fluidtime Data Services GmbH
- 33. Fluidtime Enabling Smart Mobility. © 2017 Copyright Fluidtime Data Services GmbH
- 34. Fluidtime Enabling Smart Mobility. © 2017 Copyright Fluidtime Data Services GmbH Cool stuff in the SonarEcosystem I: „Tricky Bugs are Running Scared“ https://blog.sonarsource.com/sonaranalyzer-for-java-tricky-bugs-are- running-scared/
- 35. Fluidtime Enabling Smart Mobility. © 2017 Copyright Fluidtime Data Services GmbH Cool stuff in the SonarEcosystem II: „Cognitive Complexity“ https://blog.sonarsource.com/cognitive-complexity-because- testability-understandability/
- 36. Fluidtime Enabling Smart Mobility. © 2017 Copyright Fluidtime Data Services GmbH Some more goodies Owasp 10 dependency check plugin: https://github.com/stevespringett/dependency-check-sonar-plugin Scala analysis: http://www.openforce.com/2017/02/sonarqube-with- scala/
- 37. Fluidtime Enabling Smart Mobility. © 2017 Copyright Fluidtime Data Services GmbH Three Lines of Defense Pull Request Analysis Quality Gates & Fixing the leak SonarLint X X X Pull RequestCode Locally Trunk Release https://blog.sonarsource.com/putting-it-all-together-end- to-end-quality-with-sonarecosystem/ 1 2 3
- 38. Fluidtime Enabling Smart Mobility. © 2017 Copyright Fluidtime Data Services GmbH Last Bastion – Break the build (Not always a good idea) Since 5.2 SonarQube analysis is asynchronuous so you have to wait for the result: Alternatives: Make quality gate failures visible (information radiators), Issue notifications https://blog.sonarsource.com/why-you-shouldnt-use-build-breaker/ https://www.sonarsource.com/resources/product-news/2017/02/2017- 02-28-sonarqube-scanner-for-jenkins-2.6-released.html
- 39. Fluidtime Enabling Smart Mobility. © 2017 Copyright Fluidtime Data Services GmbH Learnings Green field projects: Start early, Legacy projects: Don‘t try to fix everything– Fix the leak / exclusions / different quality profiles Cultural change - Not everyone loves transparency Good to have: Objective instance (Best Practices) with detailed explanations (Stick to the default rules as long as possible) File false positives (create trust in the system; or at least make it difficult to blame SonarQube), maybe you‘ll even learn something Restore build in profiles after update (to enable newly added rules) Use LTS if you use any community plugins (or check compatibility)
- 40. Fluidtime Enabling Smart Mobility. © 2017 Copyright Fluidtime Data Services GmbH Just one tool in your belt https://smartbear.com/SmartBear/media/ebooks/State-of-Code- Quality-2016.pdf
- 41. Fluidtime Enabling Smart Mobility. © 2017 Copyright Fluidtime Data Services GmbH Summary Code Quality is just one, important, aspect of software quality Continuously inspect your code. Start with SonarLint, today! Look into Pull Request Analysis and Quality Gates with SonarQube Explore capabilities and extend functionalities with plugins It’s “just” a very powerful tool and no silver bullet solution
- 42. Fluidtime Enabling Smart Mobility. © 2017 Copyright Fluidtime Data Services GmbH Demo 1. Start SonarQube Server locally: docker run -p 9000:9000 -p 9092:9092 sonarqube:6.3 2. git clone https://github.com/SonarSource/sonarlint-intellij.git (or any other project with some issues / tests where SonarQube plugin is set up) 3. Show SonarLint in IntelliJ (Preferences ->Plugin, Add a //TODO/BUG, Show Explaination / Analyse open Files, All Files / Changed Files; Bind to Server Dialog) 4. (Show example for pull request analysis: E.g. https://github.com/SonarSource/sonarqube/pull/1750) 5. Run ./gradlew check buildPlugin sonarqube in the sonarlint-intellij project 6. Increase version in gradle.properties and run again 7. Browse to http://localhost:9000/ (admin/admin) 8. Show SonarQube GUI
- 43. Fluidtime Enabling Smart Mobility. © 2017 Copyright Fluidtime Data Services GmbH Additional Resources Docs: https://docs.sonarqube.org Blog: https://blog.sonarsource.com/ Twitter: https://twitter.com/SonarQube Stackoverflow: http://stackoverflow.com/questions/tagged/sonarqube Mailing List: https://groups.google.com/forum/#!forum/sonarqube Roadmap: https://www.sonarqube.org/roadmap/ Online Server (Free for open source projects): https://sonarqube.com
- 44. Fluidtime Enabling Smart Mobility. © 2017 Copyright Fluidtime Data Services GmbH Contact Roman Pickl (@rompic) roman.pickl@fluidtime.com Fluidtime Data Services GmbH Neubaugasse 12-14/25 A–1070 Wien Tel +43 (0)1 5860 180 www.fluidtime.com
- 45. Fluidtime Enabling Smart Mobility. © 2017 Copyright Fluidtime Data Services GmbH
- 46. Fluidtime Enabling Smart Mobility. © 2017 Copyright Fluidtime Data Services GmbH Additional References (where not indicated on the slide) slide 11,16,20,36: All Images from the noun project (no changes made) Alexander Skowalsky, „Servers“, https://thenounproject.com/search/?q=server&i=573662 Danil Polshin, „Developer“, https://thenounproject.com/search/?q=developer&i=597289 Blake Stevenson, „Bug“, https://thenounproject.com/search/?q=bug&i=781390 Oliviu Stoian, „Bed Bug“, https://thenounproject.com/search/?q=bug&i=902732 All Icons Licensed CC BY 3.0: https://creativecommons.org/licenses/by/3.0/us/
Editor's Notes
- Questions in between / Discussion at the end Question: Raise your hand if you are using it?
- Wikipedia: William Thomson, 1st Baron Kelvin, OM, GCVO, PC, FRS, FRSE (/ˈkɛlvɪn/; 26 June 1824 – 17 December 1907) was a Scots-Irish[1][2] mathematical physicist and engineer who was born in Belfast in 1824. At the University of Glasgow he did important work in the mathematical analysis of electricity and formulation of the first and second laws of thermodynamics, and did much to unify the emerging discipline of physics in its modern form.
- Software Product Quality is a multi dimensional concept External factors that directly influence the customer and internal factors that only have an indirect impact And as as software developer you may only have impact on some of these dimensions (e.g. you could write perfect code, but still no one may need your product; i.e. it doesn‘t meet your customers‘ needs)
- Wikipedia: SQALE (Software Quality Assessment based on Lifecycle Expectations) is a method to support the evaluation of a software application source code. It is a generic method, independent of the language and source code analysis tools, licensed under the Creative Commons Attribution-NonCommercial-NoDerivs 3.0 Unported license.[1] Software editors can freely use and implement the SQALE method. The SQALE method has been developed to answer a general need for assessing the quality of source code. It is meant to answer fundamental questions such as: What is the quality of the source code delivered by the developers? Is the code changeable, maintainable, portable, reusable? What is the design debt stored up by the project?
- SonarSource: SQALE is primarily about maintainability, but the SQALE quality model also encompasses bugs and vulnerabilities. So those important issues get lost in the crowd. The result is that a project can have blocker-level bugs, but still get an A SQALE rating. For us, that was kinda like seeing a green light at the intersection while cross-traffic is still flowing. Yes, it’s recoverable if you’re paying attention, but still dangerous.
- Wikipedia: The Harvard Mark II was an electromechanical computer built under the direction of Howard Aiken and was finished in 1947. It was financed by the United States Navy.
- Code Complete: industry average 15-50 bugs per 1000 lines of code
- Visual Studio: .Net
- http://www.sonarlint.org/intellij/index.html
- http://www.sonarlint.org/intellij/index.html#Connected
- http://www.sonarlint.org/commandline/index.html
- https://blog.sonarsource.com/putting-it-all-together-end-to-end-quality-with-sonarecosystem/ Fast Feedback
- Unfortunately, there are issues that won’t be raised in SonarLint or by pull request analysis. That’s where you start managing the leak.
- you can also use gradle to achieve this
- https://blog.sonarsource.com/putting-it-all-together-end-to-end-quality-with-sonarecosystem/ Fast Feedback
- I‘ll show you sonarqube 6.3 which is the latest version
- https://blog.sonarsource.com/water-leak-changes-the-game-for-technical-debt-management/
- Wikipedia: Thomas J. McCabe introduced Cyclomatic Complexity in 1976 as a way to guide programmers in writing methods that “are both testable and maintainable”. At SonarSource, we believe Cyclomatic Complexity works very well for measuring testability, but not for maintainability. That’s why we’re introducing Cognitive Complexity, which you’ll begin seeing in upcoming versions of our language analyzers. We’ve designed it to give you a good relative measure of how difficult the control flow of a method is to understand.
- https://blog.sonarsource.com/putting-it-all-together-end-to-end-quality-with-sonarecosystem/ Fast Feedback