Upcoming SlideShare
×

# Introduction of bison

59,217 views
58,936 views

Published on

1 Comment
6 Likes
Statistics
Notes
• Full Name
Comment goes here.

Are you sure you want to Yes No
• pls mail me this presentation to manabs21@gmail.com

Are you sure you want to  Yes  No
Views
Total views
59,217
On SlideShare
0
From Embeds
0
Number of Embeds
55,184
Actions
Shares
0
0
1
Likes
6
Embeds 0
No embeds

No notes for slide

### Introduction of bison

1. 1. Introducing of Bison<br />天官<br />2011-09-15<br />1<br />
2. 2. Flex and Bison<br />statement: NAME '=' expression<br />expression: NUMBER '+' NUMBER<br /> | NUMBER '-' NUMBER<br />Flex:<br />recognizes regular expressions.<br />divides the input stream into pieces(token)<br />terminal symbol:<br /> Symbols produced by the lexer are called terminal symbols or tokens<br />nonterminal symbol:<br />Those that are defined on the left-hand side of rules are called nonterminal symbols or nonterminals.<br />VS<br />Bison<br /><ul><li>for building programs that handle structure input.
3. 3. takes these pieces and groups them together logically.</li></li></ul><li>Shift/Reduce Parsing<br />Shift<br />As the parser reads tokens, each time it reads a token that doesn't complete a rule, it pushes the token on an internal stack and switchs to a new state reflecting the token it just read. This action is called a shift.<br />Reduce<br />When it has found all the symbols that constitute the right-hand side of a rule, it pops the right-hand side symbols off the stack, pushes the left-hand side symbol onto the stack, and switches to a new state reflecting the new symbol on the stack. This action is called a reduction.<br />
4. 4. Parsing methods<br />Bison parsers can use either of two parsing methods, known as LALR(1) and GLR<br />LALR(1) (Look Ahead Left to Right with a one-token lookahead), which is less powerful but considerably faster and easier to use than GLR.<br />GLR (Generalized Left to Right).<br />The most common kind of language that computer parsers handle is a context-free grammar(CFG)<br />The standard form to write down a CFG is Baskus-Naur Form (BNF)<br />
5. 5. LR parser<br />LR parser is a parser that reads input from Left to right and produces a Rightmost derivation.<br /> The term LR(k) parser is also used; where the k refers to the number of unconsumed "look ahead" input symbols that are used in making parsing decisions.<br />Usually k is 1 and the term LR parser is often intended to refer to this case. (LALR(1))<br />
6. 6. look ahead<br />LALR(1) cannot deal with grammars that need more than one token of lookahead to tell whether it has matched a rule.<br />phrase: cart_animal AND CART<br /> | work_animal AND PLOW<br />cart_animal: HORSE | GOAT<br />work_animal: HORSE | OX<br />phrase: cart_animalCART<br /> | work_animalPLOW<br />cart_animal: HORSE | GOAT<br />work_animal: HORSE | OX<br />Not support!<br />OR<br />phrase: cart_animal AND CART<br /> | work_animal AND PLOW<br />cart_animal: HORSE | GOAT<br />work_animal: OX<br />
7. 7. Rightmost Derivation<br />Rule 1<br />expr  expr – digit<br />exprexpr – digit<br />exprexpr + digit<br />expr digit<br />digit 0|1|2|…|9<br />Example input:<br /> 3 + 8 - 2<br />The rightmost non-terminal is replaced in each step<br />Rule 4<br />expr – digit  expr – 2<br />Rule 2<br />expr – 2 expr + digit - 2<br />Rule 4<br />expr + digit - 2  expr + 8-2<br />Rule 3<br />expr + 8-2 digit + 8-2<br />Rule 4<br />digit + 8-23+8 -2<br />
8. 8. Leftmost Derivation<br />Rule 1<br />expr  expr – digit<br />The leftmost non-terminal is replaced in each step<br />expr<br />1<br />1<br />Rule 2<br />expr – digit  expr + digit – digit<br />2<br />2<br />3<br />expr<br />-<br />digit<br />Rule 3<br />expr + digit – digit  digit + digit – digit<br />3<br />5<br />4<br />expr<br />digit<br />+<br />Rule 4<br />4<br />digit + digit – digit3 + digit – digit<br />2<br />Rule 4<br />3 + digit – digit 3 + 8 – digit<br />5<br />6<br />digit<br />8<br />Rule 4<br />3 + 8 – digit 3 + 8 – 2<br />6<br />3<br />
9. 9. Leftmost Derivation<br />Rule 1<br />expr  expr – digit<br />expr  expr – digit<br />expr  expr + digit<br />expr  digit<br />digit 0|1|2|…|9<br />Example input:<br /> 3 + 8 - 2<br />The leftmost non-terminal is replaced in each step<br />Rule 2<br />expr – digit  expr + digit – digit<br />Rule 3<br />expr + digit – digit  digit + digit – digit<br />Rule 4<br />digit + digit – digit3 + digit – digit<br />Rule 4<br />3 + digit – digit 3 + 8 – digit<br />Rule 4<br />3 + 8 – digit 3 + 8 – 2<br />
10. 10. Leftmost Derivation<br />Rule 1<br />expr  expr – digit<br />The leftmost non-terminal is replaced in each step<br />expr<br />1<br />1<br />Rule 2<br />expr – digit  expr + digit – digit<br />6<br />2<br />2<br />expr<br />-<br />digit<br />Rule 3<br />expr + digit – digit  digit + digit – digit<br />3<br />3<br />5<br />expr<br />digit<br />+<br />Rule 4<br />4<br />digit + digit – digit3 + digit – digit<br />2<br />Rule 4<br />3 + digit – digit 3 + 8 – digit<br />5<br />4<br />digit<br />8<br />Rule 4<br />3 + 8 – digit 3 + 8 – 2<br />6<br />3<br />
11. 11. Context-Free Grammars<br />A context-free grammar G is defined by the 4-tuple:<br />G = (V, ∑, R, S) where<br />V is a finite set; each element v ϵ V is called a non-terminal character or a variable. Each variable represents a different type of phrase or clause in the sentence. Variables are also sometimes called syntactic categories. Each variable defines a sub-language of the language defined by .<br />∑ is a finite set of terminals, disjoint from V, which make up the actual content of the sentence. The set of terminals is the alphabet of the language defined by the grammar G.<br />R is a finite relation from V to (V U ∑)*. The members of R are called the (rewrite) rules or productions of the grammar.<br />S is the start variable (or start symbol), used to represent the whole sentence (or program). It must be an element of V.<br />The asterisk represents the Kleene star operation.<br />
12. 12. Context-free language<br />The language of grammar G = (V, ∑, R, S) is the set<br /> L(G) = { ωϵ ∑* : S ωω }<br /> A language L is said to be context-free languange(CFL), if there exists a CFG G, such that L = L(G).<br />
13. 13. Context-Free Grammars<br />Comprised of<br />A set of tokens or terminal symbols<br />A set of non-terminal symbols<br />A set of rules or productions which express the legal relationships between symbols<br />A start or goal symbol<br />Example:<br />exprexpr – digit<br />exprexpr + digit<br />expr digit<br />digit 0|1|2|…|9<br /><ul><li>Tokens: -,+,0,1,2,…,9
14. 14. Non-terminals: expr, digit
15. 15. Start symbol: expr</li></li></ul><li>A Bison Parser<br />A bison specification has the same three-part structure as a flex specification.<br />... definition section ...<br />%%<br />... rules section ...<br />%% a bison example<br />... user subroutines ...<br />The first section, the definition section, handles control information for the parser and generally sets up the execution environment in which the parser will operate.<br />The second section contains the rules for the parser.<br />The third section is C code copied verbatim into the generated C program.<br />
16. 16. Terms<br />Symbols are strings of letters, digits, periods, and underscores that do not start with a digit.<br />error is reserved for error recovery.<br />Do not use C reserved words or bison's own symbols such as yyparse.<br />Symbols produced by the lexer are called terminal symbols or tokens<br />Those that are defined on the left-hand side of rules are called nonterminal symbols or nonterminals.<br />
17. 17. Structure of a Bison Specification<br />... definition section ...<br /> %%<br /> ... rules section ...<br /> %%<br /> ... user subroutines ...<br />
18. 18. Literal Block<br /> %{<br /> ... C code and declarations ...<br /> %}<br />The contents of the literal block are copied verbatim to the generated C source file near the beginning, before the beginning of yypare().<br />Usually contains declarations of variables and functions, as well as #include.<br />Bison also provides an experimental %code POS { ... } where POS is a keyword to suggest where in the generated parser the code should go.<br />
19. 19. Delaration<br />%parse-param<br />%require "2.4“<br /> declare the minimum version of bison needed to compile it<br />%start<br />identifies the top-level rule (Named the first rule.)<br />%union<br />%token<br />%type<br />%left<br />%right<br />%nonassoc<br />%expect<br />
20. 20. Token<br />Define the ternimators.<br />Bison treats a character in single quotes as a token<br />Bison also allows you to decalre strings as aliases for tokens<br />This defines the token NE and lets you use NE and != interchangeably in the parser. The lexer must still return the internal token values for NE when the token is read, not a string.<br />expr: '(' expr ')';<br />%token NE "!="<br />%%<br />...<br />expr: expr "!=" exp;<br />
21. 21. Parse-param<br />Normally, you call yyparse() with no arguments, if you need, youcan add parameters to its definition:<br />%parse-param { char *modulename }<br /> %parse-param { int intensity }<br />This allows you to call yyparse("mymodule", 42)<br />
22. 22. Type<br />The %union declaration specifies the entire list of possible types<br />%token is used for declaring token types<br />%type is used for declaring nonterminal symbols<br />%{<br />#include "calc.h“ /* Contains definition of `symrec' */ <br />%} <br />%union { <br /> double val; /* For returning numbers. */ <br />symrec *tptr; /* For returning symbol-table pointers */<br /> } <br />%token <tptr> VAR FNCT /* Variable and Function */ <br />%type <val> exp <br />%%<br />
23. 23. Structure of a Bison Specification<br /> ... definition section ...<br /> %%<br />... rules section ...<br /> %%<br /> ... user subroutines ...<br />
24. 24. Actions<br />An action is C code executed when bison matches a rule in the grammar.<br />The action can refer to the values associated with the symbols in the rule by using a dollar sign followed by a number.<br />The name \$\$ refers to the value for the left-hand side (LHS) symbol.<br />For rules with no action, bison uses a default of the following<br />date: month '/' day '/' year { printf("date %d-%d-%d found", \$1, \$3, \$5); } ;<br />{ \$\$ = \$1; }<br />
25. 25. Rules<br />Recursive Rules<br />The action can refer to the values associated with the symbols in the rule by using a dollar sign followed by a number.<br />In most cases, Bison handles left recursion much more efficiently than right recursion.<br />numberlist : /* empty */<br /> | numberlist NUMBER<br /> ;<br />exprlist: exprlist ',' expr; /* left recursion */<br />or<br />exprlist: expr ',' exprlist; /* right recursion */<br />
26. 26. Special Characters<br />% All of the declarations in the definition section start with %.<br />\$ In actions, a dollar sign introduces a value reference.<br />@ In actions, an @ sign introduces a location reference, such as @2 for the location of the second symbol in the RHS.<br />' Literal tokens are enclosed in single quotes.<br />" Bison lets you declare quoted string as parser alias for tokens.<br /><> In a value reference in an action, you can override the value's default type by enclosing the type name in angle brackets.<br />{} The C code in actions is enclosed in curly braces.<br />; Each rule in the rules section should end with a semicolon.<br />| or syntax for multi-rules with same LHS.<br />: separate left-hand side and right-hand side<br />- Symbols may include underscores along with letters, digits, and periods.<br />. Symbols may include periods along with letters, digits, and underscores.<br />
27. 27. Reserved<br />YYABORT<br />In an action makes the parser routine yyparse() return immediately with a nonzero value, indicating failure.<br />YYACCEPT<br />In an action makes the parser routine yyparse() return immediately with a value 0, indicating success.<br />YYBACKUP<br />The macro YYBACKUP lets you unshift the current token and replace it with something else.<br />sym: TOKEN { YYBACKUP(newtok, newval); }<br />It is extremely difficult to use YYBACKUP() correctly, so you're best off not using it.<br />
28. 28. Reserved<br />yyclearin<br />The macro yyclearin in an action discards a lookahead token if one has been read. It is most oftern useful in error recovery in an interactive parser to put the paarser into a known state after an error:<br />YYDEBUG<br />To include the trace code, either use the -t flag on the bison command line or else define the C preprocessor symbol YYDEBUG to be nonzero either on the C compiler command line or by inlcuding something like this in the definition section:<br />stmtlist : stmt | stmtlist stmt;<br />stmt : error { reset_input(); yyclearin; };<br />%{<br />#define YYDEBUG 1<br />%}<br />
29. 29. Ambiguity and Conflicts<br />The grammar is truly ambiguous<br />Shift/Reduce Conflicts<br />Reduce/Reduce Conflicts<br />The grammar is unambiguous, but the standard parsing technique that bison uses is not powerful enough to parse the grammar. (need to look more than one token ahead)<br />We have already told about it of LALR(1).<br />
30. 30. Reduce/Reduce Conflicts<br />A reduce/reduce conflict occurs when the same token could complete two different rules.<br /> %%<br /> prog: proga | progb;<br /> proga: 'X';<br /> progb: 'X';<br />
31. 31. Shift/Reduce Conflicts<br />%type <a> exp<br />...<br />%%<br />...<br />expr: expr '+' exp<br /> { \$\$ = newast('+', \$1, \$3); }<br /> | expr '-' exp <br /> { \$\$ = newast('-', \$1, \$3); }<br /> | expr '*' exp <br /> { \$\$ = newast('*', \$1, \$3); }<br /> | expr '/' exp <br /> { \$\$ = newast('/', \$1, \$3); }<br /> | '|' exp <br /> { \$\$ = newast('|', \$2, NULL); }<br /> | '(' exp ')' <br /> { \$\$ = \$2); }<br /> | '-' exp <br /> { \$\$ = newast('M', \$2, NULL); }<br /> | NUMBER { \$\$ = newnum(\$1); }<br /> ;<br />%%<br />Example 2+3*4<br />
32. 32. Problem<br />At this point, the parser looks at the * and could either reduce 2+3 using;<br />to an expression or shift the *, expecting to be able to reduce:<br /> later on.<br />2 shift NUMBER<br />E reduce E->NUMBER<br />E + shift +<br />E + 3 shift NUMBER<br />E + E reduce E->NUMBER<br />Example 2+3 * 4<br />expr: expr '+' exp<br />expr: expr ‘*' exp<br />
33. 33. Analysis<br />The problem is that we haven't told bison about the precedence and associativity of the operators.<br />Precedence controls which operators execute first in an expression.<br />In and expression grammar, operators are grouped into levels of precedence from lowest to highest.The total number of levels depends on the language. The C language is notorious for having too many precedence levels, a total of 15 levels.<br />Associativity controls the grouping of operators at the same precedence level.<br />
34. 34. Implicitly Solution<br />%type <a> exp exp1 exp2<br />...<br />%%<br />...<br />expr : expr1 '+' exp1 { \$\$ = newast('+', \$1, \$3); }<br /> | expr1 '-' exp1 { \$\$ = newast('-', \$1, \$3); }<br /> | expr1 { \$\$ = \$1; }<br />expr1: expr2 '*' exp2 { \$\$ = newast('*', \$1, \$3); }<br /> | expr2 '/' exp2 { \$\$ = newast('/', \$1, \$3); }<br /> | expr2 { \$\$ = \$1; }<br />expr2: '|' exp { \$\$ = newast('|', \$2, NULL); }<br /> | '(' exp ')' { \$\$ = \$2); }<br /> | '-' exp { \$\$ = newast('M', \$2, NULL); }<br /> | NUMBER { \$\$ = newnum(\$1); }<br /> ;<br />%%<br />
35. 35. Explicitly Solution<br />%left '+' '-’<br />%left '*' '/’<br />%nonassoc '|' NMINUS<br />%type <a> exp exp1 exp2<br />...<br />%%<br />...<br />expr: expr '+' exp { \$\$ = newast('+', \$1, \$3); }<br /> | expr '-' exp { \$\$ = newast('-', \$1, \$3); }<br /> | expr '*' exp { \$\$ = newast('*', \$1, \$3); }<br /> | expr '/' exp { \$\$ = newast('/', \$1, \$3); }<br /> | '|' exp { \$\$ = newast('|', \$2, NULL); }<br /> | '(' exp ')' { \$\$ = \$2); }<br /> | '-' exp %prec UMINUS { \$\$ = newast('M', \$2, NULL); }<br /> | NUMBER { \$\$ = newnum(\$1); }<br /> ;<br />%%<br />
36. 36. Explicitly Solution<br />%left, %right, and %nonassoc declarations defining the order of precedence from lowest to highest.<br />%left, left associative<br />%right, right associative<br />%nonaccoc, no associativity<br />UMINUS, pseudo token standing fro unary minus<br />%prec UMINUS, %prec tells bison to use the precedence of UMINUS for this rule.<br />
37. 37. IF/THEN/ELSE conflict<br />When Not to Use Precedence Rules<br />In expression grammars and to resolve the "dangling else" conflict in grammars for if/then/else language constructs, it is easy to understand.<br />But in other situations, it can be extremely difficult to understand.<br />stmt: IF '(' cond ')' stmt<br /> | IF '(' cond ')' stmt ELSE stmt<br /> | TERMINAL<br />cond: TERMINAL<br />Ambiguous!!!<br />IF ( cond ) IF ( cond ) stmt ELSE stmt<br />Which one?<br />IF ( cond ) { IF ( cond ) stmt } ELSE stmt<br />IF ( cond ) { IF ( cond ) stmt ELSE stmt }<br />
38. 38. Implicitly Solution<br />stmt :matched<br /> | unmatched<br /> ;<br />matched :other_stmt<br /> | IF expr THEN matched ELSE matched<br /> ;<br />unmatched : IF expr THEN stmt<br /> | IF expr THEN matched ELSE unmatched<br /> ;<br />other_stmt: /* rules for other kinds of statement */<br />...<br />IF ( cond ) { IF ( cond ) stmt ELSE stmt }<br />
39. 39. Explicitly Solution<br />%nonassoc THEN<br />%nonassoc ELSE<br />%%<br />stmt : IF expr THEN stmt<br /> | IF expr stmt ELSE stmt<br /> ;<br />Equal to:<br />%nonassoc LOWER_THAN_ELSE<br />%nonassoc ELSE<br />%%<br />stmt : IF expr stmt %prec LOWER_THAN_ELSE<br /> | IF expr stmt ELSE stmt<br /> ;<br />IF ( cond ) { IF ( cond ) stmt ELSE stmt }<br />
40. 40. expect<br />Occasionally you may have a grammar that has a few conflicts, you are confident that bison will resolve them the way you want, and it's too much hassle to rewrite the grammar to get rid of them.<br />%expect N tells bison that your parser should have N shift/reduce conflicts.<br />%expect-rr N to tell it how many reduce/reduce conflicts to expect.<br />
41. 41. Common Bugs In Bison Programs<br />Infinite Recursion<br />%%<br />xlist: xlist ‘X’ ;<br />should be ==><br />%%<br />xlist : 'X'<br />| xlist 'X’<br /> ;<br />
42. 42. Common Bugs In Bison Programs<br />Interchanging Precedence<br />%token NUMBER<br />%left PLUS<br />%left MUL<br />%%<br />expr : expr PLUS expr %prec MUL<br /> | expr MUL expr %prec PLUS<br /> | NUMBER<br /> ;<br />
43. 43. Lexical Feedback<br />Parsers can sometimes feed information back to the lexer to handle otherwise difficult situations. <br />E.g. syntax like this:<br />message ( any characters )<br />/* parser */<br />%{<br /> init parenstring = 0;<br />}%<br />...<br />%%<br />statement: MESSAGE { parenstring = 1; } '(' STRING ')';<br />
44. 44. Lexical Feedback<br />/* lexer */<br />%{<br /> extern int parenstring;<br />%}<br />%s PSTRING<br />%%<br />"message" return MESSAGE;<br />"(" {<br /> if(parenstring) BEGIN PSTRING;<br /> return '(';<br /> }<br /><PSTRING>[^)]* {<br /> yylval.svalue = strdup(yytext); <br /> BEGIN INITIAL;<br /> return STRING;<br /> }<br />
45. 45. Structure of a Bison Specification<br /> ... definition section ...<br /> %%<br /> ... rules section ...<br /> %%<br />... user subroutines ...<br />
46. 46. User subroutines Section<br />This section typically includes routines called from the actions.<br />Nothing special.<br />
47. 47. Discuss Everything<br />