Pushdown AutomataChapter 12Recognizing Context-F.docx

Pushdown Automata
Chapter 12
Recognizing Context-Free Languages
Two notions of recognition:
(1) Say yes or no, just like with FSMs
(2) Say yes or no, AND
if yes, describe the structure
a + b * c
Just Recognizing
We need a device similar to an FSM except that it needs more
power.
The insight: Precisely what it needs is a stack, which gives it
an unlimited amount of memory with a restricted structure.
Example: Bal (the balanced parentheses language)
(((()))

Definition of a Pushdown Automaton
K is a finite set of states
ion relation. It is a finite subset of
symbols symbols
to pop to push
from top on top
of stackof stack
Definition of a Pushdown Automaton
Manipulating the Stack
c will be written as cab
a
b
If c1c2…cn is pushed onto the stack:

c1
c2
cn
c
a
b
c1c2…cncab
Yields
Then:
-
Let |-M* be the reflexive, transitive closure of |-M.
C1 yields configuration C2 iff C1 |-M* C2
Computations
A computation by M is a finite sequence of configurations C0,
● C0 is an initial configuration,

● Cn is of the
● C0 |-M C1 |-M C2 |-M … |-M Cn.
Nondeterminism
Accepting
A computation C of M is an accepting computation iff:
-
M accepts a string w iff at least one of its computations accepts.
Other paths may:
● Read all the input and halt in a nonaccepting state,
● Read all the input and halt in an accepting state with the
stack not
empty,
● Loop forever and never finish reading the input, or
● Reach a dead end where no more input can be read.
The language accepted by M, denoted L(M), is the set of all
strings accepted by M.

Rejecting
A computation C of M is a rejecting computation iff:
-
● C is not an accepting computation, and
M rejects a string w iff all of its computations reject.
So note that it is possible that, on input w, M neither accepts
nor rejects.
A PDA for Balanced Parentheses
A PDA for Balanced Parentheses
K = {s}the states
A = {s}
**Important: This does not mean that the stack is empty

K = {s, f}the states
stack alphabet
A = {f}the accepting states
Exploiting Nondeterminism

A PDA M is deterministic iff:
other, and
● Whenever M is in an accepting configuration it has no
available moves.
But many useful PDAs are not deterministic.
A PDA:
A PDA:

More on Nondeterminism
Accepting Mismatches
Start with the case where n = m:
1
2
Start with the case where n = m:
● If stack and input are empty, halt and reject.
● If input is empty but stack is not (m > n) (accept):
● If stack is empty but input is not (m < n) (accept):
1
2

● If input is empty but stack is not (m < n) (accept):
1
2
2
1
3
● If stack is empty but input is not (m > n) (accept):
/a
1
2
2
1
4

Putting It Together
● Jumping to the input clearing state 4:
Need to detect bottom of stack.
● Jumping to the stack clearing state 3:
Need to detect end of input.
The Power of Nondeterminism
PDA for it?
The Power of Nondeterminism
der}, and
unequal numbers of a’s, b’s, and c’s).

Are the Context-Free Languages Closed Under Complement?
If the CF languages were closed under complement, then
would also be context-free.
But we will prove that it is not.
s

a's
Reducing Nondeterminism
Need to detect bottom of stack, so push # onto the
stack before we start.
Need to detect end of input. Add to L a termination
character (e.g., $)
More on PDAs
What about a PDA to accept {
PDAs and Context-Free Grammars
Theorem: The class of languages accepted by PDAs is exactly
the class of context-free languages.
Recall: context-free languages are languages that
can be defined with context-free grammars.

Restate theorem:
Can describe with context-free grammar
Can accept by PDA
Going One Way
Lemma: Each context-free language is accepted by some PDA.
Proof (by construction):
The idea: Let the stack do the work.
Two approaches:
Top down
Bottom up
Top Down
The idea: Let the stack keep track of expectations.
Example: Arithmetic expressions

A Top-Down Parser
The outline of M is:
● The start-
Example of the Construction
L = {anb*an}
0 (p,

transstate unread inputstack
0 q a a b b a aS
3 q a a b b a aaSa
6 q a b b a aSa
3 q a b b a aaSaa
6 q b b a aSaa
2 q b b a aBaa
5 q b b a abBaa
7 q b a aBaa
5 q b a abBaa
7 q a aBaa
4 q a aaa
6 q aa
Another Example
L = {anbmcpdq : m + n = p + q}
Another Example

input = a a b c d d
Another Example
(
(
transstate unread input stack
The Other Way to Build a PDA - Directly

input = a a b c d d
The Other Way to Build a PDA - Directly
c
input = a a b c d d
1
2
3
4

Notice Nondeterminism
Machines constructed with the algorithm are often
nondeterministic, even when they needn't be. This happens
even with trivial languages.
A grammar for AnBn is:A PDA M for AnBn is:
(4)
But transitions 1 and 2 make M nondeterministic.
A directly constructed machine for AnBn:
Bottom-Up

Reduce Transitions:
(2) (p
Shift Transitions
The idea: Let the stack keep track of what has been found.
A Bottom-Up Parser
The outline of M is:
● The reduce transi
each rule

Going The Other Way
Lemma: If a language is accepted by a pushdown automaton M,
it is context-free (i.e., it can be described by a context-free
grammar).
Proof (by construction):
Step 1: Convert M to restricted normal form:
special
symbol # onto the stack and then transfer to a state s from
which
the rest of the computation begins. There must be no
transitions
● M has a single accepting state a. All transitions into a pop
# and
read no input.
exactly one
symbol from the stack.
Converting to Restricted Normal Form
Example:

Pop no more than one symbol:
M in Restricted Normal Form
[1] ((s, a, #), (s, a#)),
((s, a, a), (s, aa)),
((s, a, b), (s, ab)),
[2] ((s, b, #), (s, b#)),
((s, b, a), (s, ba)),
((s, b, b), (s, bb)),
[3] ((s, c, #), (f, #)),
((s, c, a), (f, a)),
((s, c, b), (f, b))
Must have one transition for everything that could have been on
the top of the stack so it can be popped and then pushed back
on.
Pop exactly one symbol: Replace [1], [2] and [3] with:
[1]
[2]
[3]
Second Step - Creating the Productions

Example: WcWR
M =
The basic idea –
simulate a leftmost derivation of M on any input string.
Second Step - Creating the Productions
Example:
abcba
Nondeterminism and Halting
1. There are context-free languages for which no deterministic
PDA exists.
2. It is possible that a PDA may
● not halt,
● not ever finish reading its input.
3. There exists no algorithm to minimize a PDA. It is
undecidable whether a PDA is minimal.
Nondeterminism and Halting
It is possible that a PDA may
● not halt,
● not ever finish reading its input.
{a} and consider M =

- (2, a, a) |-
On any other input except a:
● M will never halt.
Solution
s to the Problem
● For NDFSMs:
● Convert to deterministic, or
● Simulate all paths in parallel.
● For NDPDAs:
● Formal solutions that usually involve changing the
grammar.
● Practical solutions that:
● Preserve the structure of the grammar, but
● Only work on a subset of the CFLs.

Alternative Equivalent Definitions of a PDA
Accept by accepting state at end of string (i.e., we don't care
about the stack).
= M.
3. Create a new accepting state qa.
4. For each accepting state a in M do,
Example
The balanced parentheses language
● FSM plus FIFO queue (instead of stack)?

● FSM plus two stacks?
What About These?
Comparing Regular and
Context-Free Languages
Regular Languages Context-Free Languages
● regular exprs.
● or
● regular grammars ● context-free grammars
● recognize ● parse
● = DFSMs ● = NDPDAs
Discuss issue in Chapter 9
Discuss or argue whether you think wellness programs are an
intrusion of privacy.
Make certain you demonstrate you have read the 'Wellness
Programs' section in Chapter 9; do not post superfluous points
that do not directly relate to the issues

Follow the Argument Formula below
Context-Free and
Noncontext-Free Languages
Chapter 13
Languages That Are and
Are Not Context-Free
a*b* is regular.
-free but not regular.

-free.
Languages and Machines
The Regular and the CF Languages
Theorem: The regular languages are a proper subset of the
context-free languages.
Proof: In two parts:
Every regular language is CF.
There exists at least one language that is CF but not regular.
The Regular and the CF Languages
Lemma: Every regular language is CF.
Proof: Every FSM is (trivially) a PDA:

form: ( p, c, q )
old state, input, new state
becomes:
))
old state, input, don't new statedon't
look atpush on
stackstack
In other words, we just don’t use the stack.
There Exists at Least One Language that is CF but Not Regular
Lemma: There exists at least one language that is CF but not
regular
-free but not regular.
So the regular languages are a proper subset of the context-free
languages.

How Many Context-Free Languages Are There?
Theorem: There is a countably infinite number of CFLs.
Proof:
● Upper bound: we can lexicographically enumerate
all the CFGs.
● Lower bound: {a}, {aa}, {aaa}, … are all CFLs.
How Many Context-Free Languages Are There?
There is an uncountable number of languages.
Thus there are more languages than there are context-free
languages.
So there must exist some languages that are not context-free.

Showing that L is Context-Free
Techniques for showing that a language L is context-free:
1. Exhibit a context-free grammar for L.
2. Exhibit a PDA for L.
3. Use the closure properties of context-free languages.
Unfortunately, these are weaker than they are for
regular languages.
Showing that L is Not Context-Free
Remember the pumping argument for regular languages:
95.unknown

A Review of Parse Trees
rooted, ordered tree in which:
● The root node is labeled S,
● Every other node is labeled with some element of V -
● If m is a nonleaf node labeled X and the children of m are
Some Tree Basics
The height of a tree is the length of the longest path from the
root to any leaf.
The branching factor of a tree is the largest number of daughter
nodes associated with any node in the tree.

Theorem: The length of the yield of any tree T with height h
From Grammars to Trees
Given a context-free grammar G:
● Let n be the number of nonterminal symbols in G.
● Let b be the branching factor of G
Suppose that T is generated by G and no nonterminal appears
more than once on any path:

The maximum height of T is:
The maximum length of T’s yield is:
64.unknown
The Context-Free Pumping Theorem
This time we use parse trees, not automata as the basis for our
argument.
If w is “long”, then its parse trees must look like:

Choose one such tree such that there’s no other with fewer
nodes.

There is another derivation in G:
in which, at the point labeled [1], the nonrecursive rule2 is
used.
So uxz is also in L(G).
There are infinitely many derivations in G, such as:

Those derivations produce the strings:
uv2xy2z, uv3xy3z, …
So all of those strings are also in L(G).

would create a parse tree with fewer nodes. But that contradicts
the assumption that we started with a tree with the smallest
possible number of nodes.
The height of the subtree rooted at [1] is at most:

The height of the subtree rooted at [1] is at most: n + 1

If L is a context-free language, then
vxyz,
k serves two roles:
● How long must w be to guarantee it is pumpable?
● What’s the bound on |vxy|?

Let n be the number of nonterminals in G.
Let b be the branching factor of G.
What Is k?

If height(T) > n, then some nonterminal occurs more than once
on some path. So T is pumpable.
So if |uvxyz| > bn, w = uvxyz must be pumpable.
How Long Must w be?
Assume that we are considering the bottom-most two instances
of a repeated nonterminal. Then the yield of the upper one has

length at most bn+1.
So let k = bn+1.
What’s the Bound on |vxy|?
If L is a context-
h n
nonterminal symbols and branching factor b. Let k be bn + 1.
The longest string that can be generated by G with no repeated
nonterminals in the resulting parse tree has length bn.
Assuming that b 2, it must be the case that bn + 1 > bn. So

let w be any string in L(G) where |w|
smallest parse tree for w. T must have height at least n + 1.
Choose some path in T of length at least n + 1. Let X be the
bottom-most repeated nonterminal along that path. Then w can
be rewritten as uvxyz. The tree rooted at [1] has height at most
n + 1. Thus its yield, vxy, has length less than or equal to
bn +
be a smaller parse tree for w and we chose T so that that wasn’t
so. uxz must be in L because rule2 could have been used
because rule1 could have been used q times before finally using
rule2.
Regular vs CF Pumping Theorems
Similarities:
● We choose w, the string to be pumped.
● We choose a value for q that shows that w isn’t pumpable.
● We may apply closure theorems before we start.
Differences:

● Two regions, v and y, must be pumped in tandem.
● We don’t know anything about where in the strings v and y
will
fall. All we know is that they are reasonably “close
together”, i.e.,
● Either v or y could be empty, although not both.
An Example of Pumping: AnBnCn
Choose w = ak bk ck
1 | 2 | 3

Choose w = ak bk ck
1 | 2 | 3
If either v or y spans regions, then let q = 2 (i.e., pump in
once). The resulting string will have letters out of order and
thus not be in AnBnCn.
If both v and y each contain only one distinct character then set
q to 2. Additional copies of at most two different characters are
added, leaving the third unchanged. There are no longer equal
numbers of the three letters, so the resulting string is not in
AnBnCn.

If n = k2, then n2 = k4. Let w = .

vy = ap, for some nonzero p.
Set q to 2. The resulting string, s, is . It must be in L.
But it isn’t because it is too short:
w:next longer string in L:
(k2)2 a’s (k2 + 1)2 a’s
k4 a’sk4 + 2k2 + 1 a’s
For s to be in L, p = |vy| would have to be at least 2k2 + 1.
hus s is not in L and L
is not context-free.

Another Example of Pumping
Let w =
Another Example of Pumping
Let w = akbkak
aaa … aaabbb … bbbaaa … aaa
| 1 | 2 | 3 |
Nested and Cross-Serial Dependencies
a a b b a a

The dependencies are nested.
a a b c a a b
Cross-serial dependencies.
Let w = akbkcakbk.
aaa … aaabbb … bbbcaaa … aaabbb … bbb
| 1 | 2 |3| 4 | 5 |

Call the part before c the left side and the part after c the right
side.
● If v or y overlaps region 3, set q to 0. The resulting string
will no
longer contain a c.
● If both v and y occur before region 3 or they both occur
after
region 3, then set q to 2. One side will be longer than the
other.
● If either v or y overlaps region 1, then set q to 2. In order
to make
the right side match, something would have to be pumped
into
● If either v or y overlaps region 2, then set q to 2. In order
to make
the right side match, something would have to be pumped
into
Variable Declaration and Use

string winniethepooh;
winniethepooh =
“bearofverylittlebrain”;
Variable Declaration and Use
WcW
string winniethepooh;
winniethepooh =
“bearofverylittlebrain”;
<prelude> string a*b*;a*b* = “a”;}}.
Prove that J isn’t CF by pumping:
Let w = <prelude> string akbk;akbk = “a”;}}

Is English Context-Free?
If either the man who said it would rain is arriving today or the
man who said it wouldn’t rain is not arriving today then we
must go.
Cross Serial Dependencies
Chris and the girls runs and swim respectively.
*Chris and the girls runs and swims respectively.
If English is context-free, then so is:
respectively}
Each sentence is of the form:

ssverb respectively
What language of a’s and b’s is this similar to?
Swiss German
But English doesn’t really work this way:
● ? Jan and Pat runs and swims, respectively.
● Jan and Pat run and swim, respectively.
But Swiss German does:
Jan säit das mer em Hans es huus hälfed aastriiche.
Jan says that we Hans/DAT the house/ACC helped
paint.

Closure Theorems for Context-Free Languages
The context-free languages are closed under:
● Union
● Concatenation
● Kleene star
● Reverse
● Letter substitution
Closure Under Union
Assume that G1 and G2 have disjoint sets of nonterminals,
not including S.

We can show that L is CF by exhibiting a CFG for
it:
Closure Under Union
not including S.
We can show that L is CF by exhibiting a CFG for
it:
R1 S2},
S)
Closure Under Concatenation

not including S.
Let L = L(G1)L(G2).
We can show that L is CF by exhibiting a CFG for it:
Closure Under Concatenation
not including S.
Let L = L(G1)L(G2).

R1 S2},
S)
Closure Under Kleene Star
Assume that G does not have the nonterminal S.
Let L = L(G)*.
Closure Under Kleene Star
Assume that G does not have the nonterminal S.
Let L = L(G)*.

S)
Closure Under Reverse
and C are elements of V -
a: L(X) = {a}. {a}R = {a}.
L(C)RL(B)R.
● For every rule in G of the fo

What About Intersection and Complement?
Closure under complement implies closure under intersection,
since:
But are the CFLs closed under either complement or
intersection?
We proved closure for regular languages two different ways:
accepting and rejecting states. If closed under complement and
union, must be closed under intersection.
2. Given automata for L1 and L2, construct an automaton for L1
machines,
using states that are the Cartesian product of the sets of

states of
the two original machines.
Does either work here?
Closure Under Intersection
The context-free languages are not closed under
intersection:
The proof is by counterexample. Let:
qual b’s and c’s.
Both L1 and L2 are context-free, since there exist
straightforward context-free grammars for them.
But now consider:
=

Closure Under Intersection
The context-free languages are not closed under
intersection:
The proof is by counterexample. Let:
Both L1 and L2 are context-free, since there exist
straightforward context-free grammars for them.
But now consider:
Closure Under Complement
The context-free languages are closed under union, so if they
were closed under complement, they would be closed under
intersection (which they are not).

Closure Under Complement
An Example
-free:
-free.
Closure Under Difference
Are the context-free languages closed under difference?

Closure Under Difference
Are the context-free languages closed under difference?
- L.
-free. So, if the context-free languages were
closed under difference, the complement of any context-free
language would necessarily be context-free. But we just
showed that that is not so.
The Intersection of a Context-Free Language and a Regular
Language is Context-Free
simulating
the parallel execution of M1 and M2.

(((q1, q2),
and each state q2 in K2,
This works because: we can get away with only one stack.
Theorem: The difference (L1 – L2) between a context-free
language L1 and a regular language L2 is context-free.
Proof: L1 –
If L1 is context-
The Difference between a Context-Free Language and a Regular
Language is Context-Free

Let:
Alternatively:
– {a1776b1776}.
-free.
{a1776b1776} is regular.
An Example: A Finite Number of Exceptions
One Closure Theorem:
If L1 and L2 are context free, then so is
But what if L3 and L1 are context free? What can we say about
L2?

Don’t Try to Use Closure Backwards
One Closure Theorem:
If L1 and L2 are context free, then so is
But what if L3 and L1 are context free? What can we say about
L2?
Example:
Don’t Try to Use Closure Backwards
Using the Closure Theorems with the Pumping Theorem

Let’s try pumping: Choose w = (ab)2k
(Don’t get confused about the two uses of w.)
w w
ababab…abababababab…ababababab
But this pumps fine with v = and y =
Exploiting Regions
Choose the string akbakb.
aaaaa…………………baaaaaa……………..b
ww

But this also pumps fine.
Make All Regions “Long”
Choose the string akbkakbk.
aaa….. aabb………bbaa……aabb……..b
ww
1 2 3 4
Now we list the possibilities:
(1, 1), (2, 2), (3, 3), (4, 4), (1, 2), (2, 3), (3, 4), (1, 3), (1, 4),
(2, 4), (1/2, 2), (1/2, 3), (1/2, 4), (1/2, 2/3),…
Whenever v or y spans regions, we’ll no longer have a string of
the same form, but that’s okay given the definition of L.
Using Intersection with a Regular Language

Recall our last choice of w: akbkakbk.
ww
1 2 3 4
Using Intersection with a Regular Language
L' is not context-free. Let w = akbkakbk.
ww
1 2 3 4
Another Example

L = {w : w can be written as
x # y = z :
viewed as binary numbers without
For example, 100#1111=001111 is in L.
Another Example
L = {w : w can be written as
x # y = z :
viewed as binary numbers without
leading zeros, x # y = zR}.
Choose w = 10k#1k=0k1k:
1 000 … 000 # 111 … 111 = 000 … 000111 …111
|1| 2 |3| 4 |5| 6 |
7 |
Note that w is in L.
If

Another Example
Choose w = 10k#1k=0k1k:
1 000 … 000 # 111 … 111 = 000 … 000111 …111
|1| 2 |3| 4 |5| 6 |
7 |
*1* is not CF:
v or y overlaps 1, 3, or 5:
v or y contains the boundary between 6 and 7:
(2, 2), (4, 4), or (2, 4):
(6, 6), (7, 7), or (6, 7):
(4, 6):
(2, 6), (2, 7) or (4, 7):
If L were context-free, then
context-free.

So neither is L.
Another Example
-Z, a-z, ., blank)+ : there exists at least one
duplicated, capitalized word in w)
A string in L:
The history of China can be viewed from the perspective of an
outsider or of someone living in China.
Another Example
Prove not CF by pumping:
Choose w = AakqAak:

Another Example
Prove not CF by pumping:
Choose w = AakbkqkAakbk:
Why are the Context-Free Languages Not Closed under
Complement, Intersection and Subtraction But the Regular
Languages Are?
Given an NDFSM M1, build an FSM M2 such that
using ndfsmtodfsm.
, add the
dead state
and all required transitions to it.

the
accepting and the nonaccepting states. So:
-
We could do the same thing for CF languages if we could do
step 1,
but we can’t.
The need for nondeterminism is the key.
Deterministic PDAs
A PDA M is deterministic iff:
each other, and
● Whenever M is in an accepting configuration it has no
available moves.
M can choose between accepting and taking the
-transition, so it is not deterministic.

Deterministic CFLs
A language L is deterministic context-free iff L$ can be
accepted by some deterministic PDA.
Why $?
An NDPDA for L
A DPDA for L$

Adding $ Doesn’t Add Power
The Deterministic CF Languages are Closed Under Complement
Given a PDA M, we want to:
● Complete M.
● Swap accepting and nonaccepting configurations.
A deterministic PDA may fail to accept an input string w
because:
1. Its computation ends before it finishes reading w.
2. Its computation ends in an accepting state but the stack is
not empty.
-transitions,
without ever
halting in either an accepting or a nonaccepting state.
4. Its computation ends in a nonaccepting state.
If we simply swap accepting and nonaccepting states we will
correctly fail to accept every string that M would have accepted
(i.e., every string in L$). But we will not necessarily accept

A construction that solves these problems exists.
DCFLs Under Intersection and Union
The DCFLs are closed under complement. What about
intersection and union?
DCFLs are Not Closed Under Union

t even CF, much less DCF.
DCFLs are Not Closed Under Intersection
=
L1 and L2 are deterministic context-free:
Nondeterministic CFLs
Theorem: There exist CLFs that are not deterministic.
L is DCF then so is:

But then so is:
But it isn’t. So L is context-free but not deterministic context-
free.
This simple fact poses a real problem for the designers of
efficient context-free parsers.
Inherent Ambiguity vs. Nondeterminism
Alternatively, it is:
L1 is inherently ambiguous. Example:

aabbcc
L2 is not inherently ambiguous.
But what should a PDA do on:
aabbccd
Push a’s or not?
The CFL Hierarchy
Ogden’s Lemma
-free. We try a
pumping proof:
Let w = ak bk ck+k!.
1 | 2 | 3

If either v or y crosses regions, set q to 2. Pump in: out of
order.
● (1, 2) If |v| = |y| then set q to (k!/|v|) + 1. (k!/|v|) must be an
integer
The string that results from pumping is
aXbXck+k!,
where:
X = k + (q –
= k + k!.
So far, so good. But what about (3, 3)?
Ogden’s Lemma
-free. We try a
pumping proof:

Let w = ak bk ck+k!.
1 | 2| 3
● (3, 3) Pumping in: will result in even more c’s than a’s and
b’s. So the resulting string is in L.
Pumping out: the maximum number of c’s that can be pumped
out is k, which would result in a string with k! c’s. But, as long
as k 3,
k! > k. So the resulting string is in L.
We’re stuck.
Ogden’s Lemma
We mark some symbols as distinguished and require that at least
one of v or y contain at least one marked symbol.
Ogden’s Lemma
Theorem: If L is a context-free language, then:

least k symbols of w as distinguished then:
vy contains at least one distinguished symbol,
vxy contains at most k distinguished
symbols, and
Proof: The proof is analogous to the one we did for the context-
free Pumping Theorem except that we consider only paths that
generate the distinguished symbols.
Using Ogden’s Lemma
-free. Let w =
akbkck+k!. Mark all the a’s in w as distinguished. If either v
or y contains two or more distinct symbols, then set q to 2. The
resulting string will have letters out of order and thus not be in
L. We consider the remaining possibilities:
● (1, 1) (1, 3): Set q to 2. The number of a’s will no longer
equal the number of b’s, so the resulting string is not in L.
● (1, 2): Same argument as above.
● (2, 2), (2, 3), (3, 3): fail to satisfy the requirement that at
least one symbol in vy be marked as distinguished.

There is no way to divide w into vxy such that all the conditions
of Ogden’s Lemma are met. So L is not context-free.
Letter Equivalence
Two languages L1 and L2 are letter-equivalent iff they contain
the same strings if we disregard the order in which the symbols
occur in the strings.
Example:
(ab)* is letter equivalent to:(ba)*
Letter Equivalence
).

Two languages L1 and L2 are letter-equivalent iff:
Examples of Letter Equivalence
L1 and L2 are letter-equivalent. So are L3, L4 and L5.

Parikh’s Theorem
Theorem: Every context-free language is letter-equivalent to
some regular language.
Proof: By an argument similar to the one used to prove the
Pumping Theorem.
Context-Free Languages Over
a Single-Letter Alphabet
Theorem: Any context-free language over a single-letter
alphabet is regular.
Proof: Follows from Parikh’s Theorem
Examples:
L = {anbn}.
=
L

=
=
Using The Corollary
Primea = {an : n is prime}.
Primea is not context-free. If it were, then it would also be
regular. But we showed that it is not regular.
So it is not context-free either.
Functions on Context-Free Languages
Are the context-free languages closed under:
firstchars(L) =

Are the context-free languages closed under
maxstring(L) =
Are the context-free languages closed under
maxstring(L) =
L)}.
p
k
a

+
4
a
n
2
4
k
a
4
k
a
Context-Free Grammars
Chapter 11
Languages and Machines

Rewrite Systems and Grammars
A rewrite system (or production system or rule-based system)
is:
● a list of rules, and
● an algorithm for applying them.
Each rule has a left-hand side and a right hand side.
Example rules:
Simple-rewrite
simple-rewrite(R: rewrite system, w: initial string) =
1. Set working-string to w.
2. Until told by R to halt do:

Match the lhs of some rule against some part of working-
string.
Replace the matched part of working-string with the rhs of
the rule that was matched.
3. Return working-string.
A Rewrite System Formalism
A rewrite system formalism specifies:
● The form of the rules
● How simple-rewrite works:
● How to choose rules?
● When to quit?
An Example
w = SaS

Rules:
● What order to apply the rules?
● When to quit?
Rule Based Systems
● Expert systems
● Cognitive modeling
● Business practice modeling
● General models of computation
● Grammars
Grammars Define Languages

A grammar is a set of rules that are stated in terms of two
alphabets:
the strings in L(G), and
a nonterminal alphabet, the elements of which will function as
working symbols that will be used while the grammar is
operating. These symbols will disappear by the time the
grammar finishes its job and generates a string.
A grammar has a unique start symbol, often called S.
Using a Grammar to Derive a String
Simple-rewrite (G, S) will generate the strings in L(G).
ate steps in a derivation.
A derivation could begin with:

Generating Many Strings
Multiple rules may match.
Three choices at the next step:
Generating Many Strings
One rule may match in more than one way.
Two choices at the next step:

When to Stop
May stop when:
The working string no longer contains any nonterminal symbols
In this case, we say that the working string is generated by the
grammar.
Example:
When to Stop
May stop when:
There are nonterminal symbols in the working string but none of
them appears on the left-hand side of any rule in the grammar.

In this case, we have a blocked or non-terminated derivation but
no generated string.
Example:
When to Stop
It is possible that neither (1) nor (2) is achieved.
Example:
with S the start
symbol.
Then all derivations proceed as:
Context-free Grammars, Languages, and PDAs

Context-free Language
Context-free Grammar
PDA
L
Accepts
More Powerful Grammars
Regular grammars must always produce strings one character at
a time, moving left to right.
But it may be more natural to describe generation more flexibly.
Example 1: L = ab*a

Key distinction: Example 1 is not self-embedding.
No restrictions on the form of the right hand sides.
But require single non-terminal on left hand side.
AnBn
AnBn

Balanced Parentheses
Balanced Parentheses
A context-free grammar G is a quadruple,
● V is the rule alphabet, which contains nonterminals
and terminals.
● R (the set of rules) is a finite subset of (V -

● S (the start symbol) is an element of V -
Example:
Derivations
Then the language generated by G, denoted L(G), is:
An Example Derivation

Example:
Definition of a Context-Free Grammar
A language L is context-free iff it is generated by some
context-free grammar G.
w1Yw2, where:
A grammar is recursive iff it contains at least one recursive

rule.
Recursive Grammar Rules A rule i
w1Yw2, where:
rule.
w1Yw2, where:
rule.

Self-Embedding Grammar Rules A rule in a grammar G is self-
embedding iff it is :
X
A grammar is self-embedding iff it contains at least one self-
embedding rule.
-embedding
-
embedding
-embedding
Recursive and Self-Embedding
Grammar Rules A rule in a grammar G is self-embedding iff it
is :
X
A grammar is self-embedding iff it contains at least one self-

embedding rule.
Exampl -embedding
-
embedding
Where Context-Free Grammars Get Their PowerIf a grammar G
is not self-embedding then L(G) is regular.
If a language L has the property that every grammar that defines
it is self-embedding, then L is not regular.
G = {{S, a, b}, {a, b}, R, S}, where:

Equal Numbers of a’s and b’s
*: #a(w) = #b(w)}.
Equal Numbers of a’s and b’s
G = {{S, a, b}, {a, b}, R, S}, where:

Arithmetic Expressions
where
V = {+, *, (, ), id, E},
R = {
BNFThe symbol | should be read as “or”.
Allow a nonterminal symbol to be any sequence of characters
surrounded by angle brackets.
Examples of nonterminals:
<program>

<variable>
A notation for writing practical context-free grammars
BNF for a Java Fragment
<block> ::= {<stmt-list>} | {}
<stmt-list> ::= <stmt> | <stmt-list> <stmt>
<stmt> ::= <block> | while (<cond>) <stmt> |
if (<cond>) <stmt> |
do <stmt> while (<cond>); |
<assignment-stmt>; |
return | return <expression> |
<method-invocation>;
Spam Generation
These production rules yield 1,843,200 possible spellings.
How Many Ways Can You Spell [email protected]? By Brian
Hayes
American Scientist, July-August 2007
http://www.americanscientist.org/template/AssetDetail/assetid/5

5592
HTML
<ul>
<li>Item 1, which will include a sublist</li>
<ul>
<li>First item in sublist</li>
<li>Second item in sublist</li>
</ul>
<li>Item 2</li>
</ul>
A grammar:
/* Text is a sequence of elements.
H
are allowed in the body of an HTML document)
/* The <ul> and </ul> tags must match.

/* The <li> and </li> tags must match.
English
ProperNoun | NP PP
older | smart
Designing Context-Free Grammars
● Generate related regions together.

AnBn
● Generate concatenated regions:
● Generate outside in:
Outside-In Structure and RNA Folding
A Grammar for RNA Folding
-5> G[.23]
-5> C[.23]
-5> U[.23]

<stemlo -5> A[.23]
-5> U[.03]
-5> G[.03]
<stemloop-
Concatenating Independent Languages
The cm portion of any string in L is completely independent of
the anbn portion, so we should generate the two portions
separately and concatenate them together.
Concatenating Independent Languages
The cm portion of any string in L is completely independent of
the anbn portion, so we should generate the two portions
separately and concatenate them together.

G = ({S, N, C, a, b, c}, {a, b, c}, R, S} where:
)}
G = ({S, M, a, b}, {a, b}, R, S} where:

Another Example: Unequal a’s and b’s
V = {a, b, S, },
R =
Another Example: Unequal a’s and b’s
L = {anbm
V = {a, b, S, A, B},
R =

Simplifying Context-Free Grammars
G = ({S, A, B, C, D, a, b}, {a, b}, R, S), where
R =

Unproductive Nonterminals
rk every terminal
made without any new symbol being marked do:
has not yet been marked as productive then:
Unreachable Nonterminals
reachable.Mark every other nonterminal symbol as
unreachable.Until one entire pass has been made without any
new symbol being marked do:
-
If X has been marked as reachable and A has not then:
Mark A as reachable.Remove from
symbol on the left-

Proving the Correctness of a Grammar
G = ({S, a, b}, {a, b}, R, S),
● Prove that G generates only strings in L.
● Prove that G generates all the strings in L.
To prove that G generates only strings in L:
Imagine the process by which G generates a string as the
following loop:
st := S.Until no nonterminals are left in st do:
2.1. Apply some rule in R to st.
3.Output st.

Then we construct a loop invariant I and show that:
● I is true when the loop begins,
● I is maintained at each step through the loop, and
● I
● Prove that G generates only strings in L:

● Prove that G generates all the strings in L:
Base case: |w| = 0.
Prove: If every string in AnBn of length k, where k is even, can
be generated by G, then every string in AnBn of length k + 2
can also be generated. For any even k, there is exactly one
string in AnBn of length k: ak/2bk/2. There is also only one
string of length k + 2, namely aak/2bk/2b. It can be generated
by first applying rule (1) to produce aSb, and then applying to S
whatever rule sequence generated ak/2bk/2. By the induction
hypothesis, such a sequence must exist.
w) = #b(w)}
G = {{S, a, b}, {a, b}, R, S}, where:

● Prove that G generates only strings in L:
- #b(w).
G = {{S, a, b}, {a, b}, R, S}, where:
● Prove that G generates all the strings in L:
Base case:
Induction step: if every string of length k can be generated, then
every string w of length k+2 can be.

w is one of: axb, bxa, axa, or bxb.
Suppose w is axb or bxa: Apply rule (1) or (2), then whatever
sequence generates x.
Suppose w is axa or bxb:
G = {{S, a, b}, {a, b}, R, S}, where:
= vy, where v and
y are in L, 2
If that is so, then G can generate w by first applying rule (3) to
produce SS, and then generating v from the first S and y from
the second S. By the induction hypothesis, it must be possible

G = {{S, a, b}, {a, b}, R, S}, where:
Suppose w is axa: we show that w = vy, where v and y are in L,
2
and 2
Build up w one character at a time. After one character, we
= 1. Since w = -1. The
ol is added to
been added and becomes negative by the time the string ax has
been built, it must at some point before then have been 0. Let v
be the shortest nonempty prefix of w to have a va
equal to 0, 2
became ax, v must be at least two characters shorter than w, so

= vy, we know bounds
on the length of y: 2 |y| = = 0,
Accepting Strings
Regular languages:
We care about recognizing patterns and taking appropriate
actions.
Context free languages:
We care about structure.
E
E +E
id E * E

3 id id
5 7
Structure
To capture structure, we must capture the path we took through
the grammar. Derivations do that.
Example:
1 2 3 4 5 6
1 2 3 5 4 6
But the order of rule application doesn’t matter.
Derivations

Parse trees capture essential structure:
1 2 3 4 5 6
1 2 3 5 4 6
S
S S
( S ) ( S )
Derivations
Parse Trees
), is a

rooted, ordered tree in which:
● The root node is labeled S,
● Every other node is labeled with some element of:
V –
● If m is a nonleaf node labeled X and the children of m
are labeled x1, x2, …, xn, then R contains the rule
X x1, x2, …, xn.
S
NP VP
Nominal VNP

Adjs N Nominal
AdjN
the smart cat smells chocolate
Structure in English
Generative Capacity
Because parse trees matter, it makes sense, given a grammar G,
to distinguish between:
● G’s weak generative capacity, defined to be the
set of strings, L(G), that G generates, and
● G’s strong generative capacity, defined to be the
set of parse trees that G generates.

Algorithms Care How We Search
Algorithms for generation and recognition must be systematic.
They typically use either the leftmost derivation or the
rightmost derivation.
S
S S
(S)(S)
Derivations of The Smart Cat
A left-most derivation is:

A right-most derivation is:
the Nominal smel
Regular ExpressionRegular Grammar
choose a from (a

Derivation is Not Necessarily Unique
The is True for Regular Languages Too
54.unknown
Ambiguity
A grammar is ambiguous iff there is at least one string in L(G)
for which G produces more than one parse tree.
For most applications of context-free grammars, this is a
problem.
An Arithmetic Expression Grammar
id

Even a Very Simple Grammar Can be Highly Ambiguous
Inherent Ambiguity
Some languages have the property that every grammar for them
is ambiguous. We call such languages inherently ambiguous.
Example:
Inherent Ambiguity
One grammar for L has the rules:

Consider any string of the form anbncn.
L is inherently ambiguous.
Inherent Ambiguity
Both of the following problems are undecidable:
Given a context-free grammar G, is G ambiguous?
Given a context-free language L, is L inherently
ambiguous?
But We Can Often Reduce Ambiguity
We can get rid of:

● rules with symmetric right-hand sides, e.g.,
● rule sets that lead to ambiguous attachment of
optional postfixes.
A Highly Ambiguous Grammar
Resolving the Ambiguity with a Different Grammar
A different grammar for the language of balanced parentheses:

Nullable Variables
Examples:
Nullable Variables
A variable X is nullable iff either:

are all nullable.
So compute N, the set of nullable variables, as follows:
1. Set N to the set of variables that satisfy (1).
2. Until an entire pass is made without adding anything
to N do
Evaluate all other variables with respect to (2).
If any variable satisfies (2) and is not in N, insert it.
-Rules
Definition: a rule is modifiable iff it is of the form:
removeEps(G: cfg) =
3.
been
processed:

–
An Example
G = {{S, T, A, B, C, a, b, c}, {a, b, c}, R, S), R =
atmostoneEps(G: cfg) =
.

But There is Still Ambiguity

Eliminating Symmetric Recursive Rules
S
S SS1 /* force branching to the left
S S1S /* force branching to the right
So we get:
SS1
S1

Eliminating Symmetric Recursive Rules
So we get:
S SS1
S S1
S*
S
SS1
SS1
S1

( ) ( ) ( )
EE
E EE E
E E E E
id
Problem 1: Associativity

EE
E EE E
E E E E
id
Problem 2: Precedence
Arithmetic Expressions - A Better Way

Examples:
id + id * id
id * id * id
Arithmetic Expressions - A Better Way
The Language of Boolean Logic

Boolean Logic isn’t Regular
Suppose BL were regular. Then there is a k as specified in the
Pumping Theorem.
Let w be a string of length 2k + 1 of the form:
w = ( ( ( ( ( ( id ) ) ) ) ) )
k
x y

y = (p for some p > 0
Then the string that is identical to w except that it has p
additional (’s at the beginning would also be in BL. But it can’t
be because the parentheses would be mismatched. So BL is not
regular.
Ambiguous Attachment
The dangling else problem:
<stmt> ::= if <cond> then <stmt>
<stmt> ::= if <cond> then <stmt> else <stmt>
Consider:
if cond1 then if cond2 then st1 else st2

Consider:
Consider:
<Statement> ::= <IfThenStatement> | <IfThenElseStatement> |
<IfThenElseStatementNoShortIf>
<StatementNoShortIf> ::= <block> |
<IfThenElseStatementNoShortIf> | …

<IfThenStatement> ::= if ( <Expression> ) <Statement>
<IfThenElseStatement> ::= if ( <Expression> )
<StatementNoShortIf> else <Statement>
<IfThenElseStatementNoShortIf> ::=
if ( <Expression> ) <StatementNoShortIf>
else <StatementNoShortIf>
<Statement>
<IfThenElseStatement>
if (cond) <StatementNoShortIf> else
<Statement>
The Java Fix
Java Audit Rules Try to Catch These
From the CodePro Audit Rule Set:
Dangling Else

Severity: Medium
Summary
Use blocks to prevent dangling else clauses.
Description
This audit rule finds places in the code where else clauses are
not preceded by a block because these can lead to dangling else
errors.
Example
if (a > 0) if (a > 100) b = a - 100; else b = -a;
Proving that G is Unambiguous
A grammar G is unambiguous iff every string derivable in G has
a single leftmost derivation.
S SS1 (3)
S S1 (4)

● S*:
● S1: If the next two characters to be derived are (), S1
must expand by rule (6). Otherwise, it must expand by rule (5).
S SS1(3)
S S1 (4)
The siblings of m is the smallest set that includes any matched
set p adjacent to m and all of p’s siblings.
Example:
( ( ) ( ) ) ( ) ( )
1 2 3 4
5

The set () labeled 1 has a single sibling, 2. The set (()())
labeled 5 has two siblings, 3 and 4.
The Proof, Continued
S SS1(3)
S S1 (4)
(S)(5)
● S:
● S must generate a matched set, possibly with siblings.
● So the first terminal character in any string that S
generates is (.
Call the string that starts with that ( and ends with the )
that
matches it, s.
● S1 must generate a single matched set with no siblings.
● Let n be the number of siblings of s. In order to generate
those siblings, S must expand by rule (3) exactly n times
before it expands by rule (4).

S SS1(3)
S S1 (4)
● S:
((()())) () () (()())
s
s has 3 siblings.
S must expand by rule (3) 3 times before it uses rule (4).
Let p be the number of occurrences of S1 to the right of S.
If p < n, S must expand by rule (3).
If p = n, S must expand by rule (4).

Going Too Far
bat
● Chris likes the girl with the cat.
● Chris shot the bear with a rifle.
Going Too Far

Going Too Far

Comparing Regular and Context-Free Languages
Regular LanguagesContext-Free Languages
● regular exprs.
or
● regular grammars ● context-free grammars
● recognize ● parse
A Testimonial
Also, you will be happy to know that I just made use of the
context-free grammar skills I learned in your class! I am
working on Firefox at IBM this summer and just found an
inconsistency between how the native Firefox code and a plugin
by Adobe parse SVG path data elements. In order to figure out
which code base exhibits the correct behavior I needed to trace
through the grammar
http://www.w3.org/TR/SVG/paths.html#PathDataBNF.
Thanks to your class I was able to determine that the bug is in
the Adobe plugin. Go OpenSource!

k
k
n
n
n
n
n
n
b
a
b
a
b
a
...
2
2
1
1
Parsing

Chapter 15
The Job of a ParserExamine a string and decide whether or not
it is a syntactically well-formed member of L(G), and
If it is, assign to it a parse tree that describes its structure and
thus can be used as the basis for further interpretation.
Given a context-free grammar G:
Problems with

Pushdown AutomataChapter 12Recognizing Context-F.docx

More Related Content

Similar to Pushdown AutomataChapter 12Recognizing Context-F.docx

More from woodruffeloisa

Recently uploaded

Pushdown AutomataChapter 12Recognizing Context-F.docx