Crange is a tool to index and cross-reference C/C++ source code. It can be used to generate tags database that can help with: * Identifier definitions * Identifier declaraions * References * Expressions * Operators * Symbols * Source range The source metadata collected by Crange can help with building tools to provide cross-referencing, syntax highlighting, code folding and deep source code search.

1. CRANGE
CROSS-REFERENCING SOURCE CODE WITH CLANG
— —AnuragPatel RedHat github.com/crange/crange
0

2. TABLE OF CONTENTS
1. Tools considered
2. Clangand AST
3. Indexingwith libclang
4. Crange
5. Challenges
6. Summary

3. SOURCE CODE
#include<stdio.h>
intFibonacci(int);
intFibonacci(intn){
if(n==0)
return0;
elseif(n==1)
return1;
else
return(Fibonacci(n-1)+Fibonacci(n-2));
}
intmain(){
intn=12,i=0,c;
for(c=1;c<=n;c++){
printf("%dn",Fibonacci(i));i++;
}
return0;
}
Fibonacciseries

4. TOOLS CONSIDERED

5. CTAGS
+ Fast, comprehensive language support
-Definitions only

6. DOXYGEN
+ X-Ref, XML output
-Slow

7. GNU GLOBAL
+ Fast, definitions, X-Ref

8. ANTLR
+ Seems powerful, LL(*) parser
-Java, grammar files, complicated

9. FLEX+BISON
+ Is powerful
-CS 143 -Compilers

10. CLANG
+ Declarations, definitions, X-Ref, range, statements, operators,
expressions, literals

11. CLANG

12. CLANG
Compiler frontend and static analyzer for C, C++, Objective-C.
Converts programs to ASTand allows ASTmanipulation.

13. AST
[snip]
|-FunctionDecl0x610a090<fibonacci.c:2:1,col:18>Fibonacci'int(int)'
|`-ParmVarDecl0x6109fd0<col:15>'int'
|-FunctionDecl0x610a1d0prev0x610a090<line:3:1,line:10:1>Fibonacci'int(int)'
||-ParmVarDecl0x610a150<line:3:15,col:19>n'int'
|`-CompoundStmt0x6153a70<col:22,line:10:1>
| `-IfStmt0x6153a40<line:4:3,line:9:44>
| |-<<<NULL>>>
| |-BinaryOperator0x610a2d8<line:4:7,col:12>'int''=='
| ||-ImplicitCastExpr0x610a2c0<col:7>'int'<LValueToRValue>
| ||`-DeclRefExpr0x610a278<col:7>'int'lvalueParmVar0x610a150'n''int'
| |`-IntegerLiteral0x610a2a0<col:12>'int'0
| |-ReturnStmt0x610a320<line:5:5,col:12>
| |`-IntegerLiteral0x610a300<col:12>'int'0
| `-IfStmt0x6153a10<line:6:8,line:9:44>
[snip]
$ clang-cc1 -ast-dump fibonacci.c

14. LIBCLANG

15. LIBCLANG
Libraryfor processingsource code.
Translate source to AST
Resolve identifiers and symbols
Expand macros
Syntax highlighting
Code completion

16. INDEXING AND X-REF (0/6)
#include<stdio.h>
intFibonacci(int);
intFibonacci(intn){
if(n==0)
return0;
elseif(n==1)
return1;
else
return(Fibonacci(n-1)+Fibonacci(n-2));
}
intmain(){
intn=12,i=0,c;
for(c=1;c<=n;c++){
printf("%dn",Fibonacci(i));i++;
}
return0;
}
Walk the AST

17. INDEXING AND X-REF (1/6)
intFibonacci(int);
intFibonacci(intn){
if(n==0)
return0;
elseif(n==1)
return1;
else
return(Fibonacci(n-1)+Fibonacci(n-2));
}
intmain(){
intn=12,i=0,c;
for(c=1;c<=n;c++){
printf("%dn",Fibonacci(i));i++;
}
return0;
}
#include<stdio.h>
Macros

18. INDEXING AND X-REF (2/6)
#include<stdio.h>
int (int);
int (int ){
if(n==0)
return0;
elseif(n==1)
return1;
else
return(Fibonacci(n-1)+Fibonacci(n-2));
}
int (){
int =12, =0, ;
for(c=1;c<=n;c++){
printf("%dn",Fibonacci(i));i++;
}
return0;
}
Fibonacci
Fibonacci n
main
n i c
Declarations

19. INDEXING AND X-REF (3/6)
#include<stdio.h>
intFibonacci(int);
intFibonacci(intn){
if( ==0)
return0;
elseif( ==1)
return1;
else
return( (-1)+ (-2));
}
intmain(){
intn=12,i=0,c;
for( =1; <= ; ++){
printf("%dn", ()); ++;
}
return0;
}
n
n
Fibonaccin Fibonaccin
c c n c
Fibonaccii i
References

20. INDEXING AND X-REF (4/6)
#include<stdio.h>
intFibonacci(int);
intFibonacci(intn){
if(n==0)
;
elseif(n==1)
;
else
;
}
intmain(){
intn=12,i=0,c;
;
}
return0
return1
return(Fibonacci(n-1)+Fibonacci(n-2))
for(c=1;c<=n;c++){
printf("%dn",Fibonacci(i));i++;
}
return0
Statements

21. INDEXING AND X-REF (5/6)
#include<stdio.h>
intFibonacci(int);
intFibonacci(intn){
if(n==0)
return0;
elseif(n==1)
return1;
else
return( + );
}
intmain(){
intn=12,i=0,c;
for(c=1;c<=n;c++){
("%dn", );i++;
}
return0;
}
Fibonacci(n-1) Fibonacci(n-2)
printf Fibonacci(i)
Expressions

22. INDEXING AND X-REF (6/6)
#include<stdio.h>
intFibonacci(int);
intFibonacci(intn){
if(n 0)
return0;
elseif(n 1)
return1;
else
return(Fibonacci(n1) Fibonacci(n2));
}
intmain(){
intn=12,i=0,c;
for(c 1;c n;c ){
printf("%dn",Fibonacci(i));i ;
}
return0;
}
==
==
- + -
= <= ++
++
Operators

23. CRANGE

24. FRONTEND
C++
libclang+ Python
libclang+ Python + multiprocessing
crtags: generatingtagdb
crange: query

25. DATASTORE
JSON
HDF5
BerkeleyDB
SQLite

26. MULTIPROCESSING MODEL
1 DBworker
1 multiprocessing.Queue
n-1 ASTworkers*

27. MULTIPROCESSING.POOL()
1 async DBworker, n-1 sync ASTworkers

28. CRTAGS
$cdlinux-3.13.5
$timecrtags-v-j32.
Parsingfs/xfs/xfs_bmap_btree.c(count:1)
Indexingfs/xfs/xfs_bmap_btree.c(nodes:379,qsize:0)
...
Parsingsound/soc/codecs/ak4641.h(count:34348)
Generatingindexes
real 415m10.974s
user 8108m52.241s
sys 72m51.554s
$
Generates 22GBtags file in ~7 hours
16GBRAM &32 CPUs

29. CURL DEMO
$ time crtags -v-j 32 curl-7.35.0/

30. CHALLENGES

31. INCLUDE PATHS
$ clang-I/usr/include -Iinclude -I`llvm-config--cflags`

32. MACROS
-D_DEBUG -D_GNU_SOURCE -D_CONSTANT_MACROS

33. MAKEFILE/CMAKE
[
{"directory":"/home/anurag/llvm/build",
"command":"/usr/bin/clang++-Irelative-DSOMEDEF="Withspaces,quotesand-es."
"file":"MachineVerifier.cpp"},
...
]

34. PYTHON VS C++

35. SUMMARY

36. CRANGE CAN PROVIDE
Definitions
Declarations
References
Expressions
Operators
Symbols
Source range
Fancysearch
Code folding*
Find similar code*

37. FURTHER READING
https://github.com/crange/crange
http://clang.llvm.org/get_started.html
github.com/trolldbois/python-clang/tree/master/tests/cindex
https://docs.python.org/2/library/multiprocessing.html
http://clang.llvm.org/doxygen/group__CINDEX.html

38. END