0
Upcoming SlideShare
×

# Theory of Computation (Fall 2013): Trivial & Non-Trivial Program Properties, The Halting Problem, Index Sets, Rice's Theorem, Turing Machines

1,161

Published on

Published in: Technology, Education
0 Likes
Statistics
Notes
• Full Name
Comment goes here.

Are you sure you want to Yes No
• Be the first to comment

• Be the first to like this

Views
Total Views
1,161
On Slideshare
0
From Embeds
0
Number of Embeds
49
Actions
Shares
0
3
0
Likes
0
Embeds 0
No embeds

No notes for slide

### Transcript of "Theory of Computation (Fall 2013): Trivial & Non-Trivial Program Properties, The Halting Problem, Index Sets, Rice's Theorem, Turing Machines"

1. 1. Theory of Computation Trivial & Non-Trivial Program Properties, The Halting Problem, Index Sets, Rice's Theorem Vladimir Kulyukin www.vkedco.blogspot.com
2. 2. Outline ● ● ● ● ● ● STP Theorem Trivial and Non-Trivial Program Properties Halting Problem Example of Self-Reference Argument Index Sets Statement of Rice's Theorem
3. 3. STP Theorem
4. 4. Theorem 3.2 (Ch. 4): STP Theorem For n > 0, the predicate STP is primitive recursive. (n) ( x1 ,..., xn , y, t )
5. 5. The Big Picture GÖDEL Coder L Program Universality Theorem Number Universal Program Simulator GÖDEL Decoder Number Step-Counter Theorem GÖDEL Coder = Compiler GÖDEL Decoder = Reverse Compiler Universal Program = Operating System/VM/Interpreter ???
6. 6. Interesting Implications • We do not have to be tied to a specific universal program (operating system) to run our programs – Example: We do not need the L operating system to run L programs. We can implement an L simulator on Linux in C and run L programs • We can simulate one universal program (operating system) on another universal program (operating system) – Example: Virtual Box is can simulate Linux on Windows
7. 7. Trivial & Non-Trivial Program Properties & The Halting Problem
8. 8. Trivial & Non-Trivial Program Properties & The Halting Problem
9. 9. Program Properties ● ● Can we determine non-trivial properties of programs? Examples: – Does a program halt on a given input? – Does a program compute a specific function? – Does a program plagiarize another program?
10. 10. The Halting Problem HALT ( x, y ) ⇔ program number y halts on input number x.
11. 11. The Halting Problem Let P be a program such that # ( P ) = y. 1 if Φ ( x, y ) ↓  HALT ( x, y ) =  0 if Φ ( x, y ) ↑  1 if Ψ ( x ) ↓  HALT ( x, y ) =  ( 0 if Ψ P1) ( x ) ↑  (1) P
12. 12. Theorem 2.1 (Ch. 4) HALT ( x, y ) is not computable
13. 13. Proof ● ● Suppose that HALT(x,y) is computable Consider the following L program P: [A1] IF HALT(X1,X1) GOTO A1 Y←Y+1
14. 14. Proof: What Does P Do? ● ● ● ● P is a valid L program It considers X1 to be a program number, i.e., a Gödel number If X1 halts on its own Gödel number, P goes into an infinite loop, i.e., its output is undefined If X1 does not halt on its own Gödel number, P goes to line 2 and sets Y to 1
15. 15. Proof What is Ψ Ψ (1) P (1) P ( x) ? ↑ if HALT ( x, x) = 1 ( x) =  1 if HALT ( x, x) = 0 Or : Ψ (1) P ↑ if HALT ( x, x) ( x) =  1 if ¬HALT ( x, x)
16. 16. Proof Since P is a valid L program, we can compute # ( P ). Let # ( P ) = y 0 . Then 1. HALT ( x, y 0 ) = 0 if HALT ( x, x) = 1 2. HALT ( x, y 0 ) = 1 if HALT ( x, x) = 0 Or : ( ∀x ) HALT ( x, y 0 ) iff ¬HALT ( x, x)
17. 17. Proof What is HALT ( y0 , y0 ) ? We know that ( ∀x ) HALT ( x, y0 ) iff ¬HALT ( x, x). Let x = y0 . Then HALT ( y0 , y0 ) iff ¬HALT ( y0 , y0 ).
18. 18. Important Implication There is no algorithm that, given a program P in L (or any other language) and an input x , can determine whether or not P will eventually halt on x.
19. 19. Example of the Self-Reference Argument ● ● ● ● Suppose that some security expert comes to you and says that he/she has developed a program that detects all viruses of type Z Let us assume, for the sake of simplicity, that Z viruses are programs of one argument (e.g., a file path) Suppose the expert's program implements the predicate isVirusZ(x,y), where y is the compiled source code of a suspect program and x is an input to y (e.g. x is a file path) Question: Is isVirusZ(x,y) really computable?
20. 20. Example of the Self-Reference Argument ● ● ● Suppose that isVirus(X1, X2) is computable Suppose that we have implemented ApplyVirusZ(X1) that applies virus Z to X1 Consider this program P of two arguments X1 and X2: Y←0 IF isVirusZ(X1, X2) GOTO A1 ApplyVirusZ(X1) GOTO E [A1] Y ← Y + 1
21. 21. Example of the Self-Reference Argument Suppose # ( P ) = y 0 . What is Ψ P ( y0 , y0 ) ? ( 2) If Ψ P ( y 0 , y 0 ) = 1, then isVirusZ ( y 0 , y 0 ) = 1 and P does not apply Virus Z, thus isVirusZ ( y 0 , y 0 ) = 0 ( 2) If Ψ P ( y 0 , y 0 ) = 0, then isVirusZ ( y 0 , y 0 ) = 0 and P does apply Virus Z, thus isVirusZ ( y 0 , y 0 ) = 1 ( 2)
22. 22. Index Sets
23. 23. Definition: Index Set Γ is a collection of partially computable functions of one variable. RΓ = { t ∈ N | Φ t ∈ Γ}.
24. 24. Examples ● ● ● Example 1 – Γ = primitive recursive functions of one variable – RΓ = L-program numbers that compute primitive recursive functions of one variable Example 2 – Γ = functions that map a specific program to a plagiarized version of that program – RΓ = L-program numbers that compute these plagiarizing functions of one variable Example 3 – Γ = sorting functions of one variable – RΓ = L-program numbers that compute sorting functions of one variable
25. 25. Rice's Theorem
26. 26. Properties of Programs Membership in RΓ is decidable if and only if there is an algorithm that accepts programs as input and returns TRUE or FALSE depending on whether a particular program computes a function in Γ.
27. 27. Rice’s Theorem ● ● ● The theorem is named after Henry Gordon Rice, a U.S. mathematician Rice’s Theorem (Informal Statement): For any nontrivial property of partially computable functions, there is no general and effective method to decide whether an algorithm computes a partially computable function with that property A property is trivial if it holds for all partially computable functions or for none
28. 28. Rice’s Theorem Let Γ be a collection of partially computable functions of one variable. Let f ( x ) , g ( x ) be partially computable functions such that f ( x ) ∈ Γ, g ( x ) ∉ Γ. Then RΓ is not recursive.
29. 29. Rice’s Theorem: Original Formulation Let C be a collection of p.c. functions of one variable. Then { x | Φ x ∈ C} is recursive if and only if C = ∅ or C includes all partially computable functions of one variable.
30. 30. Proof Sketch ● ● ● ● Suppose we have Γ, a non-trivial collection of partially computable functions Suppose that there exists an algorithm (an L-program) that decides if another algorithm computes a function in Γ. To put it differently, Γ has a computable characteristic function Let us call this algorithm isInGamma(x), where x is a Gödel number, i.e. a compiled L-program We now proceed to show that we can use this algorithm to solve the Halting Problem
31. 31. Proof Sketch: Reduction ● ● ● ● ● ● We will use the so-called reduction argument Suppose that we have a problem A that we know to be undecidable, e.g. the Halting Problem Suppose that we are given another problem B If we want to show that B is undecidable, we can assume that B is decidable and then conclude that the decidability of B leads to the decidability of A This is a contradiction, of course, because we know that A is undecidable In this case, A is the Halting Problem and B is Rice’s theorem
32. 32. Proof Sketch ● ● ● Since Γ is non-trivial, let f(x) be a partially computable function in Γ and let F be a corresponding L-program Since Γ is non-trivial, let g(x) be a partially computable function not in Γ and let G be a corresponding L-program For any two values of z and w, we can construct the following program P: IF HALT(z,w) GOTO A1 Y ← G(X1) GOTO E [A1] Y ← F(X1)
33. 33. Proof Sketch ● ● ● ● Since isInGamma(x) is an algorithm that computes a characteristic function of Γ, it must return 1 or 0 for any natural number x Since P is a valid L program, it has a unique Gödel number #(P) Let x = #(P). Then isInGamma(x) = 1 if and only if HALT(z, w) = 1 Since we can do this construction for any numbers of z and w, the existence of isInGamma shows that the Halting Problem is computable (or decidable), which is a contradiction
34. 34. Brief & Gentle Introduction to Turing Machines
35. 35. Hilbert’s Lecture David Hilbert Mathematical Problems Lecture at the International Congress of Mathematicians, Paris 1900
36. 36. Hilbert’s 10th Problem Given a Diophantine equation with any number of unknown quantities and with rational integral numerical coefficients: To devise a process according to which it can be determined by a finite number of operations whether the equation is solvable in rational integers.
37. 37. Diophantine Problem A Diophantine equation is an equation in which only integers solutions are allowed ax + by = c, where a, b, c are integers
38. 38. Turing Machines ● ● ● ● Alan Turing discovered this device in 1936-37 A Turing Machine (TM) is a device with internal states At any particular instant, the device is scanning one square on a linear tape which is infinite in both directions The combination of the current symbol and the state determines what the device does next
39. 39. Turing Machines ● ● ● A TM is a theoretical symbol manipulating device A TM can simulate any computer algorithm (this is a simple formulation of what came to be known as the Church-Turing thesis, a version of Church's thesis) TMs are useful for simulating and understanding how computer CPUs work
40. 40. Alan Turing's Description ...an infinite memory capacity obtained in the form of an infinite tape marked out into squares, on each of which a symbol could be printed. At any moment there is one symbol in the machine; it is called the scanned symbol. The machine can alter the scanned symbol and its behavior is in part determined by that symbol, but the symbols on the tape elsewhere do not affect the behavior of the machine. However, the tape can be moved back and forth through the machine, this being one of the elementary operations of the machine. Any symbol on the tape may therefore eventually have an innings. Alan Turing, “Intelligent Machinery,” 1948
41. 41. Turing Machines ● The actions available to the device are: – Print a symbol – Move right – Move left – Enter a new state
42. 42. Turing Machines: Notation Symbols q1 , q2 , q3 ... represent states. Symbols s0 , s1 , s2 ,... represent printable symbols. s0 = B, i.e. blank.
43. 43. Turing Machines: A Perspective ● ● ● A Turing Machine (TM) is another formalism that captures the intuitive notion of computation We will confine ourselves with the formal definition of TM and a brief review of several TM theorems The most important results are 1) Any partially computable function can be computed by a TM and 2) The Universality Theorem can be proved with Turing Machines (there exists a Universal Turing Machine)
44. 44. Turing Machines: Alphabet The alphabet of a TM = { s1 , s2 ,..., sm }. There is a special blank symbol, s0 = B, that is not part of the alphabet.
45. 45. Action Quadruples 1. qi s j sk ql When in state qi and scanning s j , print sk and go into ql . 2. qi s j R ql When in state qi and scanning s j , move one square right and go into ql . 3. qi s j L ql When in state qi and scanning s j , move one square left and go into ql .
46. 46. Turing Machine: Definition A Turing Machine is a finite set of action quadruples
47. 47. Deterministic Turing Machine A deterministic Turing Machine is a finite set of quadruples no two of which begin with the same pair qisj
48. 48. Non-deterministic Turing Machine A non-deterministic Turing Machine is a finite set of quadruples some of which may begin with the same pair qisj
49. 49. Turing Machines: Computation 1. A TM always starts in state q1. 2. A TM halts when it is in state qi scanning s j and there are no quadruples that start with qi s j . 3. The output of a TM is the non - blank symbols left on the tape.
50. 50. Example Consider the following TM : 1. q1 BRq2 2. q21Rq2 3. q2 B1q3 4. q31Rq3 5. q3 B1q1
51. 51. Example: Input 1 1. q1 BRq2 2. q21Rq2 B 1 B B B 3. q2 B1q3 4. q31Rq3 5. q3 B1q1 q1 MOVE 1: Action quadruple 1 applies. The TM moves rights and goes into q2. B
52. 52. Example: Input 1 1. q1 BRq2 2. q21Rq2 B 1 B B B 3. q2 B1q3 4. q31Rq3 5. q3 B1q1 q2 MOVE 2: Action quadruple 2 applies. The TM moves rights and stays in q2. B
53. 53. Example: Input 1 1. q1 BRq2 2. q21Rq2 B 1 B B B B 3. q2 B1q3 4. q31Rq3 5. q3 B1q1 q2 MOVE 3: Action quadruple 3 applies. The TM prints 1 over B and goes into q3.
54. 54. Example: Input 1 1. q1 BRq2 2. q21Rq2 B 1 1 B B 3. q2 B1q3 4. q31Rq3 5. q3 B1q1 q3 MOVE 4: Action quadruple 4 applies. The TM moves right and stays in q3. B
55. 55. Example: Input 1 1. q1 BRq2 2. q21Rq2 B 1 1 B B B 3. q2 B1q3 4. q31Rq3 5. q3 B1q1 q3 MOVE 5: Action quadruple 5 applies. The TM prints 1 over B and goes into q1.
56. 56. Example: Input 1 1. q1 BRq2 2. q21Rq2 B 1 1 1 B 3. q2 B1q3 4. q31Rq3 5. q3 B1q1 q1 No action quadruple applies so the TM halts with the output 111. B
57. 57. Example: Input 2 1. q1 BRq2 2. q21Rq2 B 1 1 B B 3. q2 B1q3 4. q31Rq3 5. q3 B1q1 q1 MOVE 1: Action quadruple 1 applies. The TM moves rights and goes into q2. B
58. 58. Example: Input 2 1. q1 BRq2 2. q21Rq2 B 1 1 B B 3. q2 B1q3 4. q31Rq3 5. q3 B1q1 q2 MOVE 2: Action quadruple 2 applies. The TM moves rights and stays in q2. B
59. 59. Example: Input 2 1. q1 BRq2 2. q21Rq2 B 1 1 B B 3. q2 B1q3 4. q31Rq3 5. q3 B1q1 q2 MOVE 3: Action quadruple 2 applies. The TM moves rights and stays in q2. B
60. 60. Example: Input 2 1. q1 BRq2 2. q21Rq2 B 1 1 B B 3. q2 B1q3 4. q31Rq3 5. q3 B1q1 q2 MOVE 4: Action quadruple 3 applies. The TM prints 1 over B and goes in q3. B
61. 61. Example: Input 2 1. q1 BRq2 2. q21Rq2 B 1 1 1 B 3. q2 B1q3 4. q31Rq3 5. q3 B1q1 q3 MOVE 5: Action quadruple 4 applies. The TM moves right and stays in q3. B
62. 62. Example: Input 2 1. q1 BRq2 2. q21Rq2 B 1 1 1 B 3. q2 B1q3 4. q31Rq3 5. q3 B1q1 q3 MOVE 6: Action quadruple 5 applies. The TM prints 1 and goes into q1. B
63. 63. Example: Input 2 1. q1 BRq2 2. q21Rq2 B 1 1 1 1 3. q2 B1q3 4. q31Rq3 5. q3 B1q1 q1 NO MOVE: No action quadruple is applicable. The TM halts with the output 1111. B
64. 64. Question What does the TM in the previous example compute?
65. 65. Answer f(x) = x + 2
66. 66. Two Fundamental Theorems on Turing Machines
67. 67. Theorem 1 Let f be an m - ary partially computable * function on A , for some alphabet A. Then there is a TM that computes f .
68. 68. Theorem 2 There exists a universal TM, i.e., a TM that takes a code (description) of an arbitrary TM and its input and proceeds to execute that TM on that input.
69. 69. Theorem 3: Proof Outline Since Φ ( x, z ) is a partial computable function (we proved in in the Universality Theorem), there is a TM M that computes it using only symbols 1 and B. Thus, if g ( x ) is a partial function, there is a program in L that computes it. That program must have a number, call it z 0 . M starts in the following configuration and leaves the output value of g ( x ) on the tape (if the program terminates on x) : BxBz 0 ↑ q1 In the TM context, z 0 is called the description of a TM that computes g ( x ).
70. 70. Reading Suggestions ● Ch. 4, Computability, Complexity, and Languages, 2 Edition, by Davis, Weyuker, Sigal nd
1. #### A particular slide catching your eye?

Clipping is a handy way to collect important slides you want to go back to later.