An OCaml newbie meets Camlp4 parser

An OCaml newbie meets Camlp4
parser
parser
parser
parser
parser
Kiwamu OkabeKiwamu OkabeKiwamu OkabeKiwamu OkabeKiwamu Okabe

Why use Camlp4 parser?Why use Camlp4 parser?Why use Camlp4 parser?Why use Camlp4 parser?Why use Camlp4 parser?
☆ Why not use the Menhir as parser?☆ Why not use the Menhir as parser?☆ Why not use the Menhir as parser?☆ Why not use the Menhir as parser?☆ Why not use the Menhir as parser?
☆ Because VeriFast verifier uses Camlp4 parser.☆ Because VeriFast verifier uses Camlp4 parser.☆ Because VeriFast verifier uses Camlp4 parser.☆ Because VeriFast verifier uses Camlp4 parser.☆ Because VeriFast verifier uses Camlp4 parser.

What is VeriFast verifier? #1What is VeriFast verifier? #1What is VeriFast verifier? #1What is VeriFast verifier? #1What is VeriFast verifier? #1
☆ https://github.com/verifast/verifast☆ https://github.com/verifast/verifast☆ https://github.com/verifast/verifast☆ https://github.com/verifast/verifast☆ https://github.com/verifast/verifast
☆ A verifier for single-threaded and multi-
threaded C and Java language programs
annotated with preconditions and
postconditions written in separation logic.
☆ VeriFast avoids illegal memory accesses such
like buffer overflow.
☆ VeriFast is written by OCaml.☆ VeriFast is written by OCaml.☆ VeriFast is written by OCaml.☆ VeriFast is written by OCaml.☆ VeriFast is written by OCaml.

VeriFast has own C language parserVeriFast has own C language parserVeriFast has own C language parserVeriFast has own C language parserVeriFast has own C language parser
let rec parse_decls ?inGhostHeader =
if match inGhostHeader with None -> false | Some b -> b then
parse_pure_decls
else
parse_decls_core
and
parse_decls_core = parser
[< '((p1, _), Kwd "/*@"); ds = parse_pure_decls; '((_, p2), Kwd "@*/");
ds' = parse_decls_core >] -> ds @ ds'
| [< _ = opt (parser [< '(_, Kwd "public") >] -> ());
abstract = (parser [< '(_, Kwd "abstract") >] -> true | [< >] -> false);
final = (parser [< '(_, Kwd "final") >] -> FinalClass | [< >] ->
ExtensibleClass);
ds = begin parser
parse_pure_decls
else
parse_decls_core
and
ExtensibleClass);
ds = begin parser
parse_pure_decls
else
parse_decls_core
and
ExtensibleClass);
ds = begin parser
parse_pure_decls
else
parse_decls_core
and
ExtensibleClass);
ds = begin parser
parse_pure_decls
else
parse_decls_core
and
ExtensibleClass);
ds = begin parser

Why modify VeriFast parser? #1Why modify VeriFast parser? #1Why modify VeriFast parser? #1Why modify VeriFast parser? #1Why modify VeriFast parser? #1

Why modify VeriFast parser? #2Why modify VeriFast parser? #2Why modify VeriFast parser? #2Why modify VeriFast parser? #2Why modify VeriFast parser? #2
☆ Real usage needs two C language headers.☆ Real usage needs two C language headers.☆ Real usage needs two C language headers.☆ Real usage needs two C language headers.☆ Real usage needs two C language headers.
☆ Original C header is for compiling C code.☆ Original C header is for compiling C code.☆ Original C header is for compiling C code.☆ Original C header is for compiling C code.☆ Original C header is for compiling C code.
☆ Pseud C header is for verifying C code.☆ Pseud C header is for verifying C code.☆ Pseud C header is for verifying C code.☆ Pseud C header is for verifying C code.☆ Pseud C header is for verifying C code.
☆ They may have some semantic gaps, which
cause miss verification.
☆ Because VeriFast's parser is a subset of C99
parser.
parser.
parser.
parser.
parser.

What is a near-term goal?What is a near-term goal?What is a near-term goal?What is a near-term goal?What is a near-term goal?
☆ Getting C99 compatible parser is hard.☆ Getting C99 compatible parser is hard.☆ Getting C99 compatible parser is hard.☆ Getting C99 compatible parser is hard.☆ Getting C99 compatible parser is hard.
☆ Let's verify "kern/subr_optstr.c" file in NetBSD
kernel:
kernel:
kernel:
kernel:
kernel:
$ sh build.sh -U -u -j 4 -T obj/tooldir -m amd64 tools
$ sh build.sh -U -u -j 1 -T obj/tooldir -m amd64 kernel=GENERIC
$ vfide -D __STDC__ -D __GNUC__ -D _KERNEL -D __ELF__ -D NO_KERNEL_RCSIDS -I
sys -I sys/arch/amd64/compile/obj/GENERIC sys/kern/subr_optstr.c
☆ The kernel code is found at following:☆ The kernel code is found at following:☆ The kernel code is found at following:☆ The kernel code is found at following:☆ The kernel code is found at following:
☆ https://github.com/IIJ-NetBSD/netbsd-src☆ https://github.com/IIJ-NetBSD/netbsd-src☆ https://github.com/IIJ-NetBSD/netbsd-src☆ https://github.com/IIJ-NetBSD/netbsd-src☆ https://github.com/IIJ-NetBSD/netbsd-src

Original verifierOriginal verifierOriginal verifierOriginal verifierOriginal verifier

Modified verifierModified verifierModified verifierModified verifierModified verifier

My patches for VeriFast's parserMy patches for VeriFast's parserMy patches for VeriFast's parserMy patches for VeriFast's parserMy patches for VeriFast's parser
* Parse semicolon without any declarations #121
https://github.com/verifast/verifast/pull/121
* Allow hardtab in string literals #120
* Ignore inline keyword at static function declaration #119
* Support operators in macros / Add -D option #116

Knowledge to debug parser #1-1Knowledge to debug parser #1-1Knowledge to debug parser #1-1Knowledge to debug parser #1-1Knowledge to debug parser #1-1
☆ Don't catch exception to get stack trace.☆ Don't catch exception to get stack trace.☆ Don't catch exception to get stack trace.☆ Don't catch exception to get stack trace.☆ Don't catch exception to get stack trace.
$ git diff
diff --git a/src/vfconsole.ml b/src/vfconsole.ml
index 8846dcb5..9204c2a4 100644
--- a/src/vfconsole.ml
+++ b/src/vfconsole.ml
@@ -23,7 +23,6 @@ let _ =
Java_frontend_bridge.unload();
with
PreprocessorDivergence (l, msg) -> print_msg l msg; exit 1
- | ParseException (l, msg) -> print_msg l ("Parse error" ^ ...
| CompilationError(msg) -> print_endline (msg); exit 1
| StaticError (l, msg, url) -> print_msg l msg; exit 1
| SymbolicExecutionError (ctxts, l, msg, url) ->
$ git diff
index 8846dcb5..9204c2a4 100644
@@ -23,7 +23,6 @@ let _ =
with
$ git diff
index 8846dcb5..9204c2a4 100644
@@ -23,7 +23,6 @@ let _ =
with
$ git diff
index 8846dcb5..9204c2a4 100644
@@ -23,7 +23,6 @@ let _ =
with
$ git diff
index 8846dcb5..9204c2a4 100644
@@ -23,7 +23,6 @@ let _ =
with

$ cat tab_in_printf.c
#include <stdio.h>
int main() {
printf("Use hard tabn");
return 0;
}
$ OCAMLRUNPARAM=b verifast -c tab_in_printf.c
tab_in_printf.c
Fatal error: exception Lexer.ParseException(_, "")
Raised at file "parser.ml", line 1627, characters 26-61
Called from file "verifast.ml", line 2966, characters 12-185
Called from file "verifast1.ml", line 470, characters 6-13
Called from file "util.ml" (inlined), line 109, characters 15-18
#include <stdio.h>
int main() {
return 0;
}
tab_in_printf.c
#include <stdio.h>
int main() {
return 0;
}
tab_in_printf.c
#include <stdio.h>
int main() {
return 0;
}
tab_in_printf.c
#include <stdio.h>
int main() {
return 0;
}
tab_in_printf.c

☆ Dump "token" using debug functions.☆ Dump "token" using debug functions.☆ Dump "token" using debug functions.☆ Dump "token" using debug functions.☆ Dump "token" using debug functions.
let rec print_tokens_list tokens =
match tokens with
[(_, tok)] -> Printf.printf "%sn" (string_of_token tok)
| (_, tok) :: xs -> Printf.printf "%s, " (string_of_token tok);
print_tokens_list xs
| [] -> Printf.printf "n"
let rec print_tokens_stream tokens =
Printf.printf "[n";
Stream.iter (fun (_, tok) -> Printf.printf " %sn" (string_of_token tok))
tokens;
Printf.printf "]n"
match tokens with
Printf.printf "[n";
tokens;
Printf.printf "]n"
match tokens with
Printf.printf "[n";
tokens;
Printf.printf "]n"
match tokens with
Printf.printf "[n";
tokens;
Printf.printf "]n"
match tokens with
Printf.printf "[n";
tokens;
Printf.printf "]n"

☆ Exception "Stream.Error" has a string
parameter specified by "??".
$ wc -l src/parser.ml
1801 src/parser.ml
$ vi src/parser.ml
1801 src/parser.ml
$ vi src/parser.ml
1801 src/parser.ml
$ vi src/parser.ml
1801 src/parser.ml
$ vi src/parser.ml
1801 src/parser.ml
$ vi src/parser.ml
let parse_c_file = parser
[< (headers, _) = parse_include_directives ignore_eol verbose
enforceAnnotations dataModel;
ds = parse_decls CLang dataModel enforceAnnotations ~inGhostHeader:false;
_ = Stream.empty ?? "Stream.empty expected" (* <= *) >]
-> (headers, [PackageDecl(dummy_loc,"",[],ds)])
in
parse_c_file token_stream
in
in
in
in

$ cat main.c
;
#include <stdio.h>
$ OCAMLRUNPARAM=b verifast -c main.c
main.c
Fatal error: exception Lexer.Stream.Error("Stream.empty expected") # <=
Called from file "parser.ml", line 1634, characters 6-31
$ cat main.c
;
#include <stdio.h>
main.c
$ cat main.c
;
#include <stdio.h>
main.c
$ cat main.c
;
#include <stdio.h>
main.c
$ cat main.c
;
#include <stdio.h>
main.c

Let's join the VeriFast project!Let's join the VeriFast project!Let's join the VeriFast project!Let's join the VeriFast project!Let's join the VeriFast project!
☆ To verify C language on real world.☆ To verify C language on real world.☆ To verify C language on real world.☆ To verify C language on real world.☆ To verify C language on real world.
☆ To get better parser.☆ To get better parser.☆ To get better parser.☆ To get better parser.☆ To get better parser.

Ads: 静的コード解析の会第7回Ads: 静的コード解析の会第7回Ads: 静的コード解析の会第7回Ads: 静的コード解析の会第7回Ads: 静的コード解析の会第7回
☆ The meeting will be held on 2018-05-12.☆ The meeting will be held on 2018-05-12.☆ The meeting will be held on 2018-05-12.☆ The meeting will be held on 2018-05-12.☆ The meeting will be held on 2018-05-12.
☆ https://metasepi.connpass.com/event/77398/☆ https://metasepi.connpass.com/event/77398/☆ https://metasepi.connpass.com/event/77398/☆ https://metasepi.connpass.com/event/77398/☆ https://metasepi.connpass.com/event/77398/
☆ Discussing about static code analysis such like:☆ Discussing about static code analysis such like:☆ Discussing about static code analysis such like:☆ Discussing about static code analysis such like:☆ Discussing about static code analysis such like:
ATS, B-Method, CBMC, Coq, Coverity Scan, CSP, Dafny, F*, Frama-C,
FreeSafeTy, Infer, Isabelle, SATABS, SPARK, Spin, Uppaal, VDM, VeriFast,
Why3, boogie, cogent, corral, seL4, vcc, etc.
☆ We are calling for presentation.☆ We are calling for presentation.☆ We are calling for presentation.☆ We are calling for presentation.☆ We are calling for presentation.

Ads: Functional Ikamusume bookAds: Functional Ikamusume bookAds: Functional Ikamusume bookAds: Functional Ikamusume bookAds: Functional Ikamusume book
☆ You can buy a latest issue from Toranoana.☆ You can buy a latest issue from Toranoana.☆ You can buy a latest issue from Toranoana.☆ You can buy a latest issue from Toranoana.☆ You can buy a latest issue from Toranoana.
http://www.toranoana.jp/mailorder/article/04/0030/58/73/040030587376.htmlhttp://www.toranoana.jp/mailorder/article/04/0030/58/73/040030587376.htmlhttp://www.toranoana.jp/mailorder/article/04/0030/58/73/040030587376.htmlhttp://www.toranoana.jp/mailorder/article/04/0030/58/73/040030587376.htmlhttp://www.toranoana.jp/mailorder/article/04/0030/58/73/040030587376.html
☆ Translated VeriFast tutorial appears in it!☆ Translated VeriFast tutorial appears in it!☆ Translated VeriFast tutorial appears in it!☆ Translated VeriFast tutorial appears in it!☆ Translated VeriFast tutorial appears in it!
☆ And we are calling for article on Comiket 94.☆ And we are calling for article on Comiket 94.☆ And we are calling for article on Comiket 94.☆ And we are calling for article on Comiket 94.☆ And we are calling for article on Comiket 94.
☆ The abstract deadline is 2018-06-24.☆ The abstract deadline is 2018-06-24.☆ The abstract deadline is 2018-06-24.☆ The abstract deadline is 2018-06-24.☆ The abstract deadline is 2018-06-24.
☆ Please read following about CFA:☆ Please read following about CFA:☆ Please read following about CFA:☆ Please read following about CFA:☆ Please read following about CFA:
☆ https://wiki.haskell.org/Books#Joke☆ https://wiki.haskell.org/Books#Joke☆ https://wiki.haskell.org/Books#Joke☆ https://wiki.haskell.org/Books#Joke☆ https://wiki.haskell.org/Books#Joke

License of photosLicense of photosLicense of photosLicense of photosLicense of photos
* Creative Commons BBB | Flickr
https://www.flickr.com/photos/steren/2732488224/
Copyright: Steren Giannini / License: CC BY 2.0
* Smiling Camel | John Campbell | Flickr
https://www.flickr.com/photos/104346167@N06/16765659629/
Copyright: John Campbell / License: Public domain
* ritchie-thompson | 12 octobre 2011 : Décès de Dennis Ritchie… | Paille |
Flickr
https://www.flickr.com/photos/paille-fr/8070924277/
Copyright: Paille / License: CC BY-SA 2.0
* Spiral - BlakeWalk | Staircase in the Monument BlakeWalk, 7 … | Flickr
https://www.flickr.com/photos/joeflintham/4196385813/
Copyright: Joe Flintham / License: CC BY-SA 2.0
Flickr
Flickr
Flickr
Flickr

An OCaml newbie meets Camlp4 parser

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Similar to An OCaml newbie meets Camlp4 parser

Similar to An OCaml newbie meets Camlp4 parser (20)

Recently uploaded

Recently uploaded (20)

An OCaml newbie meets Camlp4 parser