The document discusses the concept of an ideal static analyzer, outlining characteristics such as 100% error detection, 0% false positives, high performance, and comprehensive support for various IDEs and programming languages. It highlights the inherent challenges in achieving these ideals, including the impossibility of complete error detection and the prevalence of false positives. Additionally, the necessity for ongoing rule refinement and the importance of quality customer support in the development and maintenance of static analysis tools are emphasized.

1. An ideal static analyzer, or why ideals are
unachievable
Author: Evgeniy Ryzhkov
Date: 15.03.2012
Being inspired by Eugene Laspersky's post about an ideal antivirus, I decided to write a similar post
about an ideal static analyzer. And meanwhile think how far from being it our PVS-Studio is.
An ideal static analyzer's characteristics
Those who are not familiar with the notion static code analysis, please follow the link. So let me
enumerate the characteristics right away:
• 100% detection of all the types of programming errors;
• 0% false positives;
• high performance - "whooosh, and the code is analyzed completely almost at the same
instance";
• integration with my favorite (i.e. every) IDE; ability to work under my favorite (i.e. every)
operating system; analysis of code in my favorite (i.e. any) programming language;
• free (freeware, open source);
• high-quality and prompt customer support.
Of course, this ideal can never be achieved but it shows the direction towards which companies
developing solutions in this sphere can head.
100% detection of all the types of programming errors
You should understand that none of the static analyzers will ever provide 100% error detection. Why?
Well, if only because some error types are better detected by dynamic analyzers. And it's ridiculous to
try to compete with them in this area. As well as dynamic analyzers cannot compete with static ones
regarding some rule types.
It's difficult to obtain 100% error detection even for diagnostics characteristic of static analysis. First, any
live programming language is constantly developing acquiring new syntax and therefore new ways of
making an error. Second, even old syntax can be with time used by people in a rather unusual way
analyzer developers did not think of.
Finally, a static analyzer doesn't possess knowledge about what a program SHOULD contain, it doesn't
have AI. If there is a phrase in a program "A is equal to B", while the correct one is "A is not equal to B",
static analysis won't help you with that.
That's why the only real way out is to constantly create new diagnostic rules. It will never give you 100%
error detection but will keep you close to it all the time.

2. 0% false positives
Any static analyzer produces false positives, as, in the long run, only the programmer KNOWS what
exactly the code IS TO do. But an analyzer sees what the code really DOES and tries to UNDERSTAND
what it SHOULD do.
Returning to the previous section about "100% detection of all the errors", one can make a naïve
suggestion: "Why, let's detect everything that moves and we'll be happy!" That is, let's detect everything
that looks like an error in the least bit. But this approach is wrong because the number of false positives
will go overboard. And there is an opinion that when a user sees 10 false positives in a row, he/she
closes the tool not to deal with it anymore.
We have the following ways to reduce the number of false positives:
1. Constantly handling existing rules to refine their formulations. For example, if in a test project a
rule "was triggered" 100 times and 50 of them were false positives, refining the rule can reduce
this number to 10. However, you can lose 1 or 2 real warnings, but it's the eternal issue of
making compromises.
2. Refusing rules which are no more relevant. If you only add new rules and never remove (turn
off) obsolete ones, some of your diagnostics lose their relevance with time.
3. Having useful tools to handle false positives. For instance, PVS-Studio provides a mechanism to
suppress false positives. Once marking a message as a false report, you won't see it next time.
High performance
Everybody wants software to work fast but it's not always possible. Usually the code analysis technology
requires more resources than the compiler - because the compiler checks only very crude errors, while
the analyzer's aim is to perform fine analysis. Of course, it needs more data for that. The more the data,
the deeper analysis is and the more interesting errors can be found.
An obvious solution to enhance performance is to provide support of several processor cores when
analyzing the code. It's rather easy to implement in static analyzers: each file is checked separately and
the results are simply combined then.
Less obvious is an attempt to check just a code fragment instead of a whole compilation unit (a file). This
is a very complicated task and, taken generally, it's quite difficult to solve (for any language). You have to
find and "calculate" data types, analyze classes being used and so on. Costs on "extracting" the part you
need to analyze might be even higher than just analyzing the whole code completely.
Integration with my favorite (i.e. every) IDE; ability to work under my
favorite (i.e. every) operating system; analysis of code in my favorite (i.e.
any) programming language
The issue of providing support for a certain operating system, development environment or analyzable
programming language is important in choosing between static analysis tools. To my great surprise,
programmers, being the main users of static analyzers, often cannot understand the difficulties of
implementing support for the whole zoo of operating systems they want. But let's discuss it in due
order.

3. Supported (analyzable) programming languages
Programming errors detected by code analyzers surely can occur in every programming language and
these errors have common features: in every language programmers forget to initialize variables,
confuse keys when typing a program and so on. But parsing and analysis of a program is VERY different
from language to language.
If some analyzer is announced to support analysis of software in several programming languages, it
means that there are most likely several analysis modules in it too. It can even be hidden from users! I'm
writing this just for people to understand that the phrase: "Why don't you make the SAME but for
C#/PHP/Java?" implies very much work.
Supported operating systems
It's very naïve to think that a code analyzer "just" handles text and therefore can work in any operating
system. Of course, different programming languages are "tied" to the environment to various extents:
some are more, like C++; others are less, like PHP.
Where does this difference come from? The point is that there exist several compilers for large and
powerful languages like C++, considering all their differences and subtleties in the language syntax. The
code written for Windows-based compilers is just a bit yet noticeably different from the code written for
Linux-based compilers. Though this difference is not very crucial from the user code's viewpoint, it might
be important from the viewpoint of a static analyzer - because if the code being analyzed contains key
words that are used in this very compiler, the analyzer needs to be "taught" them. In this sense, support
of one more compiler and support of one more operating system are equal tasks, generally speaking.
Note that this is an easier task for simpler languages than C++.
Thus, supported operating systems include not only platforms an executed file is run on, but the code
for these platforms the analyzer can "understand".
Supported IDE
There are a lot of development tools for different languages. What is important for users is this:
• a static analyzer should be able to integrate with their favorite development environment;
• the tool can be run in automatic mode at night;
• the analyzer should be able to integrate into continuous integration systems;
The last two points are often called "support of command line version" but it has nothing to do with the
command line. No one nowadays actually finds it interesting to watch white letters on the black screen
instead of a conveniently organized report which can be converted into a text file and sent via e-mail or
written into the build system's log.
Support of different IDE's is a difficult, effort-intensive task, as each IDE imposes certain restrictions on
their plugins. These restrictions often vary in different systems.
Free (freeware, open source) and high-quality customer support
I've united two sections into one because they are closely connected.

4. Static code analysis tools refer to the software type for which quality and continuous support are very
important. Yes, there are a few tools distributed for free, but I believe they will never reach the market
leaders (Coverity, Klocwork, Parasoft).
Generally speaking, a static analysis tool can become free and open-source if the developer company is
purchased by some giant like Google, Microsoft or Intel, but this is a special case.
Static analysis tools are usually sold according to the model of annually renewable license. Some users
might not like it, but I will try to explain why this scheme is the best. And please forgive me if you have
entered the "Free" section and now are reading about licensing schemes.
As I've already said, customer support is very important for static analysis tools. In the field of static
analysis, support implies, first of all, cases when the analyzer cannot parse user code (because of
complex C++ templates, non-standard compiler extensions, etc.). In these cases you need to promptly
(during several days) improve the analyzer so that it can parse the customer's code. User support also
includes aid in integrating the tool into their development process. Well, implementation of customer
requests that makes use of the tool more convenient is also necessary.
All this costs money. That's why you cannot sell a license once and support your users for free for the
rest of your life.
One could sell new major-releases, for example, versions v3, v4, v5... What is bad about this scheme is
that it makes the developer "hold" new cool capabilities of the tool till the next major-version instead of
releasing them right away as soon as they are ready.
Thus, it appears that annual license renewal is the best way. Meanwhile, some developer companies set
the renewal price at the 100% of the initial price, while others set a lower price (making a discount for
renewal). Regarding the latter case, it can be explained this way: the first year's price includes additional
costs on teaching the customer to work with the tool.
So, it appears that a quality tool with quality support cannot be free, if only it is not being developed by
a company-giant, but in this case you can forget about targeted individual customer support.
Conclusion
In this article I've tried to show you what characteristics an ideal static code analysis tool should possess;
how users want it to look. And it is users, of course, who decide how much this or that tool really
corresponds to this ideal.