The document evaluates call graph construction algorithms for Java, highlighting their importance in static analyses and the impact of algorithm choice on analysis precision and recall. A comprehensive test suite is developed to assess various implementations, such as Soot and Wala, focusing on aspects like polymorphic calls and reflection. The study identifies sources of unsoundness in call graphs and encourages contributions to expand the test suite for better coverage of Java's evolving features.

1. Systematic Evaluation of the
Unsoundness of Call Graph
Construction Algorithms for Java
Michael Reif, Florian Kübler, Michael Eichberg, and Mira Mezini
Software Technology Group
Technische Universität Darmstadt
Germany
1

2. Why We Shouldn’t Take
Call Graphs for Granted
• Call graphs are a central data-structure for numerous
static analyses
• Call graphs directly impact a client analysis’ result
• The chosen algorithm predetermines an analysis’
precision and recall
• Programming languages evolve (APIs and features are
added) and frameworks might not
!2

3. How We Assess Call-graph
Implementations
• Build a test suite that contains a wide range of minimal,
focused test cases
• Similar tests are kept within the same category (e.g.,
polymorphic calls or the Reflection API)
• Each test case tests a single relevant aspect
• Proof of concept: compare Soot and WALA’s built-in call
graphs
!3

4. Overall Approach
!4
TC1.jar
TC2.jar
Test Cases
Extractor
TC3.jar
Framework
Adapter
currently:
Soot, WALA
Computed
& Serialized
Call Graph
CG3.json
Computed
& Serialized
Call Graph
CG3.json
Computed
& Serialized
Call Graph
TC3.json
<Test Fixtures
Category>.md
Test Case 1(TC1)
…
Test Case 3 (TCN)
Call Graph
Matcher
Report_T
C3.txtReport_T
C3.txtReport
TC3.csv
expected call targets
generate
parse
compile
runanalysis
write
process

5. Exemplary Test Case
• Each category has:
• a description
• multiple test cases
• Each test case has:
• a scenario description
• the test code
• excepted calls
• Available annotations:
• CallSite
• IndirectCall
!5

6. Covered
Test Suite Categories
• Static Initializer
• Polymorphic Calls
• Java 8 Polymorphic Calls
• Lambdas
• Method References
• Trivial Reflection
• Locally Resolve Reflection
• Context-senstive
Reflection
• Unsafe API
• Type casts
• Serialization
• …
!6

7. Reflection
Unsafe
JNI
APIs
…
JVM
Threads
GC
Serialization
…
Features
Lambdas
default methods
…
virtual calls
Static Initializer
Research Questions
vs
RQ1
How do the call graphs of Soot
and WALA compare with each other?
RQ2
What are the main sources of unsoundness
in built-in call-graph implementations?
!7

8. !8
vs vs
Both support
basic features
Java 8 support
only in WALA
Reflection is
barely supported
Serialization is
unsupported

9. Reflection
Unsafe
JNI
APIs
…
JVM
Threads
GC
Serialization
…
Features
Lambdas
default methods
…
virtual calls
Static Initializer
Sources of Unsoundness
• APIs that are not or only partially modeled (e.g. Reflection
or Unsafe)
• new language features, such as Lambdas
• corner cases that must be covered
• the test suite is not complete
!9

10. Call for Contribution
• The test suite is by far not complete w.r.t to all Java
features and Core APIs
• add an adapter for your framework
• contribute under: https://bitbucket.org/delors/jcg
!10

11. What’s next?
• We’ll add further test scenarios for yet missing Language
Features (e.g. Java 9 modules), Core APIs (e.g.
Classloading), and JVM specifics
• Add support for other frameworks (e.g. DOOP and OPAL)
• Quantify the effect of missing features on real-world
projects
!11

12. Summary
!12
Questions?