Fitting SCA into Continuous Integration

1© 2016 Rogue Wave Software, Inc. All Rights Reserved. 1
Verification at scale:
Fitting static code analysis into
continuous integration
Embedded Conference Scandinavia

Evolution of SCA

At first there was the desktop analysis
…
• All started in December 1977, Stephen C Johnson, Bell
Labs
• Simple, structural static analysis on a file-by-file basis
Advantages
Disadvantages
• Developer learning
• Immediate, post compile
• Fast feedback loop
• Low quality due to file scope
• Unregulated platform / build
• No collaborative working

Then server-side analysis …
• Deep, inter-procedural, control- and data-flow
analysis by dedicated servers on a periodic basis
(nightly, weekly, per-release)
Advantages
Disadvantages
• Centralised – It is the simplest option to implement and maintain
• Reliable – The build process and platform are verified
• Accurate – Performs a full analysis and sees the full picture
• Feedback – slow feedback creates rework, lower fix rates and
higher new issue rates
• Developer adoption – not seen as developer task

Now client-server connected desktop
…
• Current state of the art solution: Continuous integration and
static code analysis
• Deep, inter-procedural control- and data-flow analysis
connected to the developer desktop
• Enables pre-flight analysis of new and changed code, in real
time
Advantages
Disadvantages
• Desktop feedback loop and developer focus
• Server accuracy, centralised configuration
• The best of both worlds!
• Management of desktop tools
• Requires desktop build to be possible
• Desktop environments don’t always match

Continuous integration

What is continuous integration (CI)?
• In software engineering, CI is the practice of merging all developer
working copies to a shared mainline several times a day. Grady
Booch first named and proposed CI in his 1991 method, although he did
not advocate integrating several times a day.
• Continuous integration – the practice of frequently integrating
one's new or changed code with the existing code repository –
should occur frequently enough that no intervening window remains
between commit and build, and such that no errors can arise without
developers noticing them and correcting them immediately.

Example CI process

CI best practices
• Automate the build
• Everyone commits to the baseline every day
• Every commit should be built
• Keep the build fast
• Fast feedback

So why CI?
• CI embodies the concept that by
checking the impact of changes more
frequently we’re able to quickly identify
the cause of any problems and
remediate as soon as possible with no
nasty surprises downstream
• CI gives us greater understanding,
earlier, so we can then act upon that
knowledge as we see fit
• It is an early warning system – we are
checking the things that could cause
delays or risk later earlier in the cycle –
we are shifting them to the left

Shift left
• CI shifts to the left our checking of code integration and build
problems
• But it doesn’t stop there, consider:
– Continuous deployment
– Continuous analysis
– Continuous testing
– Continuous reporting
– Continuous compliance

Continuous integration and
static code analysis
– better together

Example CI process with SCA
Klocwork
KlocworkKlocworkKlocwork

Enhanced SCA process with CI
• For some developers, compiling their code on the desktop is not possible,
so desktop analysis is not an option
• In addition, integration issues may still be detected after check in, even
when using desktop analysis
Edit &
Save
Analyze
& Fix
Compile
& Test
Check In
Developer 1
Edit &
Save
Analyze
& Fix
Compile
& Test
Check In
Developer 2
Time
Integrate
Check In
Compile
& Test
k In
New possible
issues found
here!

Continuous static code
analysis

The future: Continuous static code analysis …
• Continuous static code analysis (CSCA) brings all the benefits of centralised server-
side, deep, inter-procedural control- and data-flow analysis to a near-desktop
feedback timescale!
• Central management of development systems fits well with DevOps movement
• Enables continuous reporting and continuous compliance
Advantages
Disadvantages
• Near desktop speed feedback loop
• Server accuracy, centralised configuration
• Visibility of the current status
• Not quite as fast as connected desktop analysis
• Server resource requirements of CI builds
• Tooling must be designed for CI

Designed for CI
To work in a true CI environment CSCA tools must be designed to be:
Automated
Fast(er)
Scalable
Relevant
To reduce feedback time, only
the affected code should be
analyzed
By requiring minimal
resources & deploying
across multiple agents
By reporting only the information
that is required for the given
context (example: only the diffs
since the last build / build X)
Supporting the most important
CI build management systems

Consider before implementing CSCA
• The addition of static code analysis to a CI build system means that an
integration analysis (similar to a server-analysis) is performed on every
check in
– Developers get integration static code analysis results after every
check in or prior to each promotion?
– Less effort spent on changes and less risk to delivery timescales, as
there should be no outstanding issues at release time
– Continuous static code analysis performance is fast if only the new
and changed code is analyzed and reported
• If there are additional tests that must be written due to defects detected,
they can become part of the test plan immediately

Conclusion

So what’s right for you?
• There is no one size fits all and even within one organization. All of these
factors play a part in the decision: :
– The type of projects
– The phase of projects (developing, maintenance, re-opened legacy)
– The development tools and reporting requirements
• New engineers benefit hugely from desktop feedback, where they can
experiment, learn, and improve without the fear of exposure
– SCA in this scenario provides a huge benefit for developer training
– But the issues have to be accurate so connected (client-server) desktops
are better
– Detailed checker documentation, linking back to appropriate industry
standards or compliance guidelines will certainly help

So what’s right for you?
• Rapidly developing, new projects, where large groups of engineers are
committing new code will reap the benefits of regular integration analysis
giving them a public view of the merged code and analysis results
– Here true CI, CSCA is hard to beat if done properly, whereas server only
analysis would result in a fast growing backlog and a mammoth task of
unpicking complex issues
– Connected desktop analysis pre-check in may detect many self
contained issues even earlier keeping integration issues very clearly
highlighted
• Legacy projects or projects entering a maintenance phase with
compliance requirements will work well with results that are public,
reliable (based on certified hardware and tool chains) and clear
– Server only analysis is sufficient and the lowest level of effort to deliver
this

It all boils down to staying Agile…
• Ultimately then, we’re going to need
different capabilities from our static code
analysis tooling at different times for
different projects and with different
teams
• Tool agility is as important as team
agility for making the most of the
productivity opportunities

Questions?
23
Steve Howard
Field technical services, EMEA
Rogue Wave Software
steve.howard@roguewave.com

roguewave.com

Fitting SCA into Continuous Integration

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Similar to Fitting SCA into Continuous Integration

Similar to Fitting SCA into Continuous Integration (20)

More from Rogue Wave Software

More from Rogue Wave Software (20)

Recently uploaded

Recently uploaded (20)

Fitting SCA into Continuous Integration