MELJUN P. CORTES,
MBA,MPA,BSCS,ACS

CSC 3130: Automata theory and formal languages

Undecidable problems for CFGs
and desc...
Decidable vs. undecidable
decidable

undecidable

“DFA M accepts w”

“TM M accepts w”

“PDA M accepts w”

“TM M halts on w...
Computation is local
q0
q6

ootus

q3

oktus

q0

okrus

q1

M

lotus

okras

qacc okra

›0q0l0o0t0u0s0
›0o0q6o0t0u0s0
›0o...
Computation histories as strings
• If M halts on w, We can represent the computation
tableau by a string t over alphabet Γ...
Undecidable problems for PDAs
ALLPDA = { 〈 P 〉 : P is a PDA that accepts all
inputs}
• Theorem
The language ALLPDA is unde...
Undecidable problems for PDAs
〈P
〉

〈M〉,
w

accept if P accepts all inputs

A

reject if not

〈P
〉

A

accept
if M rej/loo...
Undecidability via computation histories
P accepts all inputs if M rejects or loops on w
P does not accept some input if M...
Undecidability via computation histories
• Task: Design a PDA P such that
candidate computation
history t of M on w
›0q0l0...
Implementing P
On input t:
Nondeterministically make one of the following choices
Look in the first block w1 of t
If w1 ≠›...
Valid and invalid windows
… 6t3t0u0 …
… 0t6t0u0 …0

… 6q3t0u0 …
… 0k6t0q0 …0

valid window

invalid window

… 6t3q3u0 …
… ...
Implementing P
wi#wi+1 represent a valid transition of M

• To check this it is better to write t in boustrophedon
›0q0l0o...
Implementing P
wi#wi+1 represent a valid transition of M
›0o0k6q3t0u0s #
# ›0o0k6r0q0u0s0

proper transition

…#›okq3tus#s...
The Post Correspondence Problem
• Input: A set of tiles like this
bab
cc

c
ab

a
ab

baa
a

a
baba

bab
ε

• Given an inf...
Undecidability of PCP
PCP = {D: D is a collection of tiles that
contains a top-bottom match}
• Theorem
The language PCP is...
Undecidability of PCP
〈M
〉,w

T

(collection of tiles)

If M accepts w, then T can be matched
If M rej/loops on w, then T ...
Some technicalities
• We will assume that
– Before accepting, TM M erases its tape
– One of the PCP tiles is marked as a s...
Undecidability of PCP
〈M
T (collection of tiles)
〉 w
If M, accepts w, then T can be matched
If M rej/loops on w, then T ca...
Undecidability of PCP
›q0lotus#›oq6otus#...#›oq1☐☐☐#›qa☐☐☐☐
accepting computation history
›q0lotus#›oq6otus#...#›oq1☐☐☐#›q...
Undecidability of PCP
• If M rejects on input w, then qr appears on bottom at
some point, but it cannot be matched on top
...
A technicality
• We assumed that one tile marked as starting tile
s

a
baba

bab
cc

c
ab

• We can remove assumption by c...
Ambiguity of CFGs
AMB = {G: G is an ambiguous CFG}
• Theorem
The language AMB is undecidable.
• Proof: We will show that
I...
Ambiguity of CFGs
T

G

(collection of tiles)

(CFG)

If T can be matched, then G is ambiguous
If T cannot be matched, the...
Ambiguity of CFGs
T

G

(collection of tiles)

Terminals:

a,b,c,1,2,3

Variables:

(CFG)

S, T, B

Productions: T → babT1...
Ambiguity of CFGs
• Each sequence of tiles gives two derivations
1
bab
cc

2

c
ab

2

c
ab

S → T → babT1 → babcT21→ babc...
Ambiguity of CFGs
T

G

(collection of tiles)

(CFG)

If T can be matched, then G is ambiguous ✓
If T cannot be matched, t...
Descriptive complexity
Roulette
• In a game of roulette, you bet $1 on even or odd
• The outcome is a number between 1 and 36
– If you guessed co...
Randomness
• If we write E for even, O for odd, what we saw is
OEOEOEOEOEOEOEOEOEOE
• It seems the wheel is crooked. If it...
Turing Machines with output

0

work tape
…

0

output tape
…

M
0

0

1

1

• The goal of a Turing Machine with output is...
Descriptive complexity
• The descriptive complexity K(x) of x is the shortest
description of any Turing Machine that outpu...
Example of descriptive complexity
x = “OE...OE” = (OE)n
(n = 1,000,000,000)

Repeat for n steps:
At odd step print O
At ev...
Bounds on descriptive complexity
• Theorem 1

n + O(1)

For every x of length n, K(x) is at most O(n)
• Proof: Let x = x1....
Descriptive complexity and randomness
• Theorem 2
For 99% of strings of length n, K(x) ≥ n – 10.
0

“simple”
strings

O(lo...
Evaluating randomness
• How do we know if the casino is crooked?
6

5

16

5

11

8

31

18

11 13 14 31

5

2

29

8

1 1...
Computing descriptive complexity
It is not possible to compute K(x).
• Proof: Suppose it is, fix n and consider this TM M:...
Upcoming SlideShare
Loading in …5
×

MELJUN CORTES Automata Theory (Automata19)

256 views
133 views

Published on

MELJUN CORTES Automata Theory (Automata19)

Published in: Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
256
On SlideShare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
4
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

MELJUN CORTES Automata Theory (Automata19)

  1. 1. MELJUN P. CORTES, MBA,MPA,BSCS,ACS CSC 3130: Automata theory and formal languages Undecidable problems for CFGs and descriptive complexity MELJUN CORTES
  2. 2. Decidable vs. undecidable decidable undecidable “DFA M accepts w” “TM M accepts w” “PDA M accepts w” “TM M halts on w” “DFA M accepts all inputs” “TM M accepts some input” “TM M accepts all inputs” “PDA P accepts all “TM M and M’ accept inputs” same ? inputs” “CFG G is ambiguous” other kinds of problems?
  3. 3. Computation is local q0 q6 ootus q3 oktus q0 okrus q1 M lotus okras qacc okra ›0q0l0o0t0u0s0 ›0o0q6o0t0u0s0 ›0o0k6q3t0u0s ›0o0k6r0q0u0s0 ›0o0k6r0a0q1s ›0o0k6r0qaa0☐ computation tableau The changes between rows occur in a 2x3 window
  4. 4. Computation histories as strings • If M halts on w, We can represent the computation tableau by a string t over alphabet Γ∪Q∪{#, ›} ›0q0l0o0t0u0s0 ›0o0q6o0t0u0s0 ›0o0k6q3t0u0s ›0o0k6r0q0u0s0 ›0o0k6r0a0q1s ›0o0k6r0qaa0☐ ›q0lotus#›oq6otus#...#›okrqaa☐# M accepts w qa occurs in string t M rejects w qa does not occur in t
  5. 5. Undecidable problems for PDAs ALLPDA = { 〈 P 〉 : P is a PDA that accepts all inputs} • Theorem The language ALLPDA is undecidable. • Proof: We will show that If ALLPDA can be decided, so can ATM.
  6. 6. Undecidable problems for PDAs 〈P 〉 〈M〉, w accept if P accepts all inputs A reject if not 〈P 〉 A accept if M rej/loops w reject if M accepts w P accepts all inputs if M rejects or loops on w P does not accept some input if M accepts w
  7. 7. Undecidability via computation histories P accepts all inputs if M rejects or loops on w P does not accept some input if M accepts w candidate computation history of M on w P reject accepting histories ›q0lotus#›oq6otus#...#›okrqaa☐ every other string M accepts w M rej/loops on w P rejects t no accepting histories P accepts everything reject accept
  8. 8. Undecidability via computation histories • Task: Design a PDA P such that candidate computation history t of M on w ›0q0l0o0t0u0s0 ›0o0q6o0t0u0s0 ›0o0k6q3t0u0s ›0o0k6r0q0u0s0 ›0o0k6r0a0q1s ›0o0k6r0qaa0☐ P reject accepting histories Expect t of the form w1#w2#...#wk# If w1 ≠›q0w , accept t. If t does not contain qa, accept t. If two consecutive blocks wi#wi+1 do not correspond to a proper transition of M, accept t.
  9. 9. Implementing P On input t: Nondeterministically make one of the following choices Look in the first block w1 of t If w1 ≠›q0w , accept t. Look for the appearance of qa If t does not contain qa, accept t. Look for the beginning of the ith block of t If two consecutive blocks wi#wi+1 do not represent a valid transition of M, accept t. ›0o0k6q3t0u0s # ›0o0k6r0q0u0s0 valid transition wi#wi+1 represents a valid transition if all 3x2 windows correspond to possible transitions of M
  10. 10. Valid and invalid windows … 6t3t0u0 … … 0t6t0u0 …0 … 6q3t0u0 … … 0k6t0q0 …0 valid window invalid window … 6t3q3u0 … … 0t6a0q7 …0 … 6c3a0t0 … … 0c6a0p0 …0 valid if δ(q3, u) = (q7, a, R) invalid window … 6t3t0u0 … … 0t6t0q3 …0 … 6c3a0t0 … … 0b6a0t0 …0 valid window valid window
  11. 11. Implementing P wi#wi+1 represent a valid transition of M • To check this it is better to write t in boustrophedon ›0q0l0o0t0u0s0 ›0o0q6o0t0u0s0 ›0o0k6q3t0u0s ›0o0k6r0q0u0s0 ›0o0k6r0a0q1s ›0o0k6r0qaa0☐ ›q0lotus#›oq6otus#...#›okrqaa☐# ›q0lotus#sutoq6o›#...#›okrqaa☐# Alternate rows are written in reverse
  12. 12. Implementing P wi#wi+1 represent a valid transition of M ›0o0k6q3t0u0s # # ›0o0k6r0q0u0s0 proper transition …#›okq3tus#suq0rko›#… wi wi+1 Nondeterministically look for beginning of 3x2 window Remember first row of window in state Use stack to detect beginning of second row Remember second row of window in state If window is not valid, accept, otherwise reject.
  13. 13. The Post Correspondence Problem • Input: A set of tiles like this bab cc c ab a ab baa a a baba bab ε • Given an infinite supply of such tiles, can you match top and bottom? a baa bab c c bab a ab a ε ab ab cc baba
  14. 14. Undecidability of PCP PCP = {D: D is a collection of tiles that contains a top-bottom match} • Theorem The language PCP is undecidable. • Proof: We will show that If PCP can be decided, so can ATM.
  15. 15. Undecidability of PCP 〈M 〉,w T (collection of tiles) If M accepts w, then T can be matched If M rej/loops on w, then T cannot be matched • Idea: Matches represent accepting histories ›q0lotus#›oq6otus#›okq3t...#›qa☐☐☐☐ ›q0lotus#›oq6otus#›okq3r...#›qa☐☐☐☐ ε ›q0lotus# ›q0l o t u s # › oq60 … ›oq6 o t u s # › okq3
  16. 16. Some technicalities • We will assume that – Before accepting, TM M erases its tape – One of the PCP tiles is marked as a starting tile bab cc c ab a ab baa a s a baba • These assumptions can be made without loss of generality (we will see why later)
  17. 17. Undecidability of PCP 〈M T (collection of tiles) 〉 w If M, accepts w, then T can be matched If M rej/loops on w, then T cannot be matched • To decide ATM, we construct these tiles for PCP s ε ›q0w# a1qia3 b1b2b3 for each valid window of this form ☐# # a a for all a in Γ∪{#, ›} #›qa ε ☐ ε “final” tiles
  18. 18. Undecidability of PCP ›q0lotus#›oq6otus#...#›oq1☐☐☐#›qa☐☐☐☐ accepting computation history ›q0lotus#›oq6otus#...#›oq1☐☐☐#›qa☐☐☐☐ ›q0lotus#›oq6otus#...#›oq1☐☐☐#›qa☐☐☐☐ s ε ›q0w# a1qia3 b1b2b3 ☐# # a a #›qa ε ☐ ε
  19. 19. Undecidability of PCP • If M rejects on input w, then qr appears on bottom at some point, but it cannot be matched on top • If M loops on w, then matching keeps going forever s ε ›q0w# a1qia3 b1b2b3 ☐# # a a #›qa ε ☐ ε
  20. 20. A technicality • We assumed that one tile marked as starting tile s a baba bab cc c ab • We can remove assumption by changing tiles a bit *a* *b*a*b*a “starting tile” begins with * b*a*b* *c*c c* *a*b “middle tiles”  * “ending tile” matches last *
  21. 21. Ambiguity of CFGs AMB = {G: G is an ambiguous CFG} • Theorem The language AMB is undecidable. • Proof: We will show that If AMB can be decided, so can PCP.
  22. 22. Ambiguity of CFGs T G (collection of tiles) (CFG) If T can be matched, then G is ambiguous If T cannot be matched, then G is unambiguous • Proof: Step 1: Number the tiles 1 bab cc 2 c ab 3 a ab
  23. 23. Ambiguity of CFGs T G (collection of tiles) Terminals: a,b,c,1,2,3 Variables: (CFG) S, T, B Productions: T → babT1 B → ccS1 S→T|B T→ε B→ε 1 bab cc T → cT2 B → abB2 2 c ab S → aT3 B → abB3 3 a ab
  24. 24. Ambiguity of CFGs • Each sequence of tiles gives two derivations 1 bab cc 2 c ab 2 c ab S → T → babT1 → babcT21→ babcc221 S → B → ccB1 → ccabB21→ ccabab221 • If the tiles match, these two derive the same string
  25. 25. Ambiguity of CFGs T G (collection of tiles) (CFG) If T can be matched, then G is ambiguous ✓ If T cannot be matched, then G is unambiguous ✓ • Argue by contradiction: – If G is ambiguous then ambiguity must look like this … a1 a2 ai T n1 n2 ni S B B b1 b2 … S T T bj B m1 m2 mj Then n1...ni = m1…mj So there is a match n1 n2 ni a2 … ai a1 b2 bi b1
  26. 26. Descriptive complexity
  27. 27. Roulette • In a game of roulette, you bet $1 on even or odd • The outcome is a number between 1 and 36 – If you guessed correctly, double your bet – Otherwise, you lose 17 6 5 16 5 2 11 8 31 18 7 4 5 2 29 8 1 12
  28. 28. Randomness • If we write E for even, O for odd, what we saw is OEOEOEOEOEOEOEOEOEOE • It seems the wheel is crooked. If it wasn’t we would expect something more like OOOEEOEOOEOEOOOEEEOE • But both sequences have same probability! Why does one appear less random than the other?
  29. 29. Turing Machines with output 0 work tape … 0 output tape … M 0 0 1 1 • The goal of a Turing Machine with output is to write something on the output tape and go into state qhalt
  30. 30. Descriptive complexity • The descriptive complexity K(x) of x is the shortest description of any Turing Machine that outputs x • We will assume x is long Andrey Kolmogorov (1903-1987)
  31. 31. Example of descriptive complexity x = “OE...OE” = (OE)n (n = 1,000,000,000) Repeat for n steps: At odd step print O At even step print E • Turing machine implementation: Write n in binary on work tape While work tape not equal to 0, Subtract 1 from number on work tape If number is odd, write O If number is even, write E K(x) ≈ ≈ ≈ ≈ log2n states 3 states 15 states 2 states ≈ log2n + 20
  32. 32. Bounds on descriptive complexity • Theorem 1 n + O(1) For every x of length n, K(x) is at most O(n) • Proof: Let x = x1...xn and consider the following TM: Write x1 to output tape and move right Write x2 to output tape and move right ... Write xn to output tape and halt.
  33. 33. Descriptive complexity and randomness • Theorem 2 For 99% of strings of length n, K(x) ≥ n – 10. 0 “simple” strings O(log n) “randomness-deficient” strings 111...1, OEOE...OE, 3.14159265, 1212321234321 n – 10 n + O(1) “random-looking” strings
  34. 34. Evaluating randomness • How do we know if the casino is crooked? 6 5 16 5 11 8 31 18 11 13 14 31 5 2 29 8 1 12 12 2 12 8 17 4 • Idea: Compute K(sequence). If much less than n, indicates sequence is not random
  35. 35. Computing descriptive complexity It is not possible to compute K(x). • Proof: Suppose it is, fix n and consider this TM M: Output the first x of length n (in lexicographic order) such that K(x) ≥ n – 10 Let x = output of M, then K(x) ≥ n – 10 but K(x) ≤ | 〈 M 〉 | = log2n + O(1) So (when n is large) we get K(x) > K(x), impossible!

×