Computability
Tractable, Intractable and
Non-computable functions
(C) 2010 Thomas J Cortina, Carnegie Mellon University
Efficiency
●A computer program should be totally correct, but
it should also
oexecute as quickly as possible (time-efficie...
Order of Complexity
●For very large n, we express the number of operations
as the order of complexity.
●Order of complexit...
O(n) ("Linear")
n
(amount of data)
Number of
Operations
n
n/2 + 6
2n + 9
(C) 2010 Thomas J Cortina, Carnegie Mellon Univer...
Order of Complexity
Number of operations Order of Complexity
n2 O(n2)
2n2 + 7 O(n2)
n2/2 + 5n + 2 O(n2)
Usually doesn't
ma...
O(n2) ("Quadratic")
n
(amount of data)
Number of
Operations
n2/2 + 5n + 22n2 + 7
n2
(C) 2010 Thomas J Cortina, Carnegie Me...
Order of Complexity
Number of operations Order of Complexity
log2n O(log n)
log10n O(log n)
2(log2n) + 5 O(log n)
The loga...
O(log n) ("Logarithmic")
n
(amount of data)
Number of
Operations
log2 n
log10 n
2(log2 n) + 5
(C) 2010 Thomas J Cortina, C...
Comparing Big O Functions
n
(amount of data)
Number of
Operations
O(2n)
O(1)
O(n log n)
O(log n)
O(n2)
O(n)
(C) 2010 Thoma...
Searching & Sorting
WORST CASE Order Of Complexity on N data elements
●Linear Search O(N)
●Binary Search O(log N)
●Selecti...
Comparing Algorithms
●Assume an algorithm processes n data values. If each
operation takes 1 s to execute, how many s will...
Decision Problems
●A specific set of computations are classified as
decision problems.
●An algorithm describes a decision ...
Monkey Puzzle Problem
●Given:
oA set of n square cards whose sides are imprinted with
the upper and lower halves of colore...
Example
Images from: Simonas Šaltenis, Aalborg University, simas@cs.auc.dk
1
2
3
4
5
6
7
8
9
(C) 2010 Thomas J Cortina, Ca...
Analysis
Simple algorithm:
●Pick one card for each cell of m X m grid.
●Verify if each pair of touching edges make a full
...
Analysis
If there are n = 9 cards (m = 3):
To fill the first cell, we have 9 card choices.
To fill the second cell, we hav...
Analysis
For n cards, the number of arrangements to examine is
n! (n factorial)
If we can analyze one arrangement in a mic...
Map Coloring
●Given a map of n territories, can the map be
colored using k colors such that no two
adjacent territories ar...
Map Coloring
(C) 2010 Thomas J Cortina, Carnegie Mellon University
Map Coloring
●Given a map of 48 territories, can the map be colored
using 3 colors such that no two adjacent territories a...
Classifications
●Algorithms that are O(nk) for some
fixed k are polynomial-time algorithms.
oO(1), O(log n), O(n), O(n log...
Traveling Salesperson
●Given: a weighted graph of nodes
representing cities and edges representing
flight paths (weights r...
Traveling Salesperson
A
B
D
C
G
E
F
12
6
4
5
9
8
10
7 11
3
7
7
Is there a route with cost at most 52? YES (Route above cos...
Traveling Salesperson
●If there are n cities, what is the maximum number of
routes that we might need to compute?
●Worst-c...
P and NP
●The class P consists of all those decision problems
that can be solved on a deterministic sequential
machine (e....
NP Complete
●The class NPC consists of all those problems in NP
that are least likely to be in P.
oEach of these problems ...
Complexity Classes
NP Problems
P Problems
NP Complete
Problems
But does P = NP?
If P ≠ NP, then all decision problems can ...
(C) 2010 Thomas J Cortina, Carnegie Mellon University
It gets worse...
●Tractable Problems
oProblems that have reasonable, polynomial-time
solutions
●Intractable Problems
oProb...
Noncomputability and
Undecidability
●An algorithmic problem that has no algorithm is
called noncomputable.
●If the noncomp...
Tiling Problem
YES
Note the periodicity in the tiling.
(C) 2010 Thomas J Cortina, Carnegie Mellon University
Tiling Problem
NO
For this 3 X 3 room, if we try all 39
tiling configurations, no tiling works.
(C) 2010 Thomas J Cortina,...
Tiling Problem
●Possible algorithm:
oIf we find a repeating pattern, report YES.
oIf we find a floor we cannot tile, repor...
Tiling Problem
●The only way to know if this set of tiles
can tile every finite-sized floor is to evaluate every
possible ...
Another Undecidable Problem:
The Barber Paradox
Suppose there is a town with one
male barber; and that every man in
the to...
Program Termination
●Can we determine if a program will terminate given a
valid input?
●Example:
1. Input x
2. While x is ...
Program Termination
●Another Example:
1. Input x
2. While x is not equal to 1, do the following:
(a) If x is even, divide ...
The Halting Problem
●Can we write a general program Q that takes
as its input any program P and an input I and determines ...
start
end
Does this
algorithm end?
yes
no
Contradiction
(C) 2010 Thomas J Cortina, Carnegie Mellon University
Contradiction isn't just for
computer scientists...
(C) 2010 Thomas J Cortina, Carnegie Mellon University
Upcoming SlideShare
Loading in …5
×

Computability - Tractable, Intractable and Non-computable Function

2,142 views

Published on

* Efficiency
* Order of Complexity
* Comparing Algorithms
* Decision Problems
* P and NP
* NP Complete
* Complexity Classes
* Noncomputability and Undecidability

Published in: Software
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
2,142
On SlideShare
0
From Embeds
0
Number of Embeds
3
Actions
Shares
0
Downloads
50
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

Computability - Tractable, Intractable and Non-computable Function

  1. 1. Computability Tractable, Intractable and Non-computable functions (C) 2010 Thomas J Cortina, Carnegie Mellon University
  2. 2. Efficiency ●A computer program should be totally correct, but it should also oexecute as quickly as possible (time-efficiency) ouse memory wisely (storage-efficiency) ●How do we compare programs (or algorithms in general) with respect to execution time? ovarious computers run at different speeds due to different processors ocompilers optimize code before execution othe same algorithm can be written differently depending on the programming language used (C) 2010 Thomas J Cortina, Carnegie Mellon University
  3. 3. Order of Complexity ●For very large n, we express the number of operations as the order of complexity. ●Order of complexity for worst-case behavior is often expressed using Big-O notation: Number of operations Order of Complexity n O(n) n/2 + 6 O(n) 2n + 9 O(n) Usually doesn't matter what the constants are... we are only concerned about the highest power of n.(C) 2010 Thomas J Cortina, Carnegie Mellon University
  4. 4. O(n) ("Linear") n (amount of data) Number of Operations n n/2 + 6 2n + 9 (C) 2010 Thomas J Cortina, Carnegie Mellon University
  5. 5. Order of Complexity Number of operations Order of Complexity n2 O(n2) 2n2 + 7 O(n2) n2/2 + 5n + 2 O(n2) Usually doesn't matter what the constants are... we are only concerned about the highest power of n. (C) 2010 Thomas J Cortina, Carnegie Mellon University
  6. 6. O(n2) ("Quadratic") n (amount of data) Number of Operations n2/2 + 5n + 22n2 + 7 n2 (C) 2010 Thomas J Cortina, Carnegie Mellon University
  7. 7. Order of Complexity Number of operations Order of Complexity log2n O(log n) log10n O(log n) 2(log2n) + 5 O(log n) The logarithm base is not written in big O notation since all that matters is that the function is logarithmic. (C) 2010 Thomas J Cortina, Carnegie Mellon University
  8. 8. O(log n) ("Logarithmic") n (amount of data) Number of Operations log2 n log10 n 2(log2 n) + 5 (C) 2010 Thomas J Cortina, Carnegie Mellon University
  9. 9. Comparing Big O Functions n (amount of data) Number of Operations O(2n) O(1) O(n log n) O(log n) O(n2) O(n) (C) 2010 Thomas J Cortina, Carnegie Mellon University
  10. 10. Searching & Sorting WORST CASE Order Of Complexity on N data elements ●Linear Search O(N) ●Binary Search O(log N) ●Selection Sort O(N2) ●Bubble Sort O(N2) ●Merge Sort O(N log N) ●Quick Sort O(N2) ●Sort + Binary Search O(N log N) + O(log N) = O(N log N) (C) 2010 Thomas J Cortina, Carnegie Mellon University
  11. 11. Comparing Algorithms ●Assume an algorithm processes n data values. If each operation takes 1 s to execute, how many s will it take to run the algorithm on 100 data values if the algorithm has the following number of computations? Number of Computations Execution Time n 100 s n • log2 n 665 s n2 10,000 s n3 1,000,000 s = 1 sec 2n > 1030 s n! > 10160 s (C) 2010 Thomas J Cortina, Carnegie Mellon University
  12. 12. Decision Problems ●A specific set of computations are classified as decision problems. ●An algorithm describes a decision problem if its output is simply YES or NO, depending on whether a certain property holds for its input. ●Example: Given a set of n shapes, can these shapes be arranged into a rectangle? (C) 2010 Thomas J Cortina, Carnegie Mellon University
  13. 13. Monkey Puzzle Problem ●Given: oA set of n square cards whose sides are imprinted with the upper and lower halves of colored monkeys. on is a square number, such that n = m2. oCards cannot be rotated. ●Problem: oDetermine if an arrangement of the n cards in an m X m grid exists such that each adjacent pair of cards display the upper and lower half of a monkey of the same color. Source: www.dwheeler.com (2002)(C) 2010 Thomas J Cortina, Carnegie Mellon University
  14. 14. Example Images from: Simonas Šaltenis, Aalborg University, simas@cs.auc.dk 1 2 3 4 5 6 7 8 9 (C) 2010 Thomas J Cortina, Carnegie Mellon University
  15. 15. Analysis Simple algorithm: ●Pick one card for each cell of m X m grid. ●Verify if each pair of touching edges make a full monkey of the same color. ●If not, try another arrangement until a solution is found or all possible arrangements are checked. ●Answer "YES" if a solution is found. Otherwise, answer "NO" if all arrangements are analyzed and no solution is found. (C) 2010 Thomas J Cortina, Carnegie Mellon University
  16. 16. Analysis If there are n = 9 cards (m = 3): To fill the first cell, we have 9 card choices. To fill the second cell, we have 8 card choices left. To fill the third cell, we have 7 card choices remaining. etc.The total number of unique arrangements for n = 9 cards is: 9 * 8 * 7 * 6 * 5 * 4 * 3 * 2 * 1 = 362,880 (C) 2010 Thomas J Cortina, Carnegie Mellon University
  17. 17. Analysis For n cards, the number of arrangements to examine is n! (n factorial) If we can analyze one arrangement in a microsecond: n Time to analyze all arrangements 9 362,880 s = 0.36288 s 16 20,922,789,888,000 s ≈ 242 days 25 15,511,210,043,330,985,984,000,000 s ≈ 491,520,585,955 years(C) 2010 Thomas J Cortina, Carnegie Mellon University
  18. 18. Map Coloring ●Given a map of n territories, can the map be colored using k colors such that no two adjacent territories are colored with the same color? ●k=4: Answer is always yes. ●k=2: Only if the map contains no point that is the junction of an odd number of territories. (C) 2010 Thomas J Cortina, Carnegie Mellon University
  19. 19. Map Coloring (C) 2010 Thomas J Cortina, Carnegie Mellon University
  20. 20. Map Coloring ●Given a map of 48 territories, can the map be colored using 3 colors such that no two adjacent territories are colored with the same color? oPick a color for California (3 choices) oPick a color for Nevada (3 choices) o... ●There are 348 = 79,766,443,076,872,509,863,361 possible colorings. ●No one has come up with a better algorithmic solution that works in general for any map, so far.(C) 2010 Thomas J Cortina, Carnegie Mellon University
  21. 21. Classifications ●Algorithms that are O(nk) for some fixed k are polynomial-time algorithms. oO(1), O(log n), O(n), O(n log n), O(n2) oreasonable, tractable ●All other algorithms are super-polynomial- time algorithms. oO(2n), O(n!), O(nn) ounreasonable, intractable (C) 2010 Thomas J Cortina, Carnegie Mellon University
  22. 22. Traveling Salesperson ●Given: a weighted graph of nodes representing cities and edges representing flight paths (weights represent cost) ●Is there a route that takes the salesperson through every city and back to the starting city with cost no more than k? oThe salesperson can visit a city only once (except for the start and end of the trip). (C) 2010 Thomas J Cortina, Carnegie Mellon University
  23. 23. Traveling Salesperson A B D C G E F 12 6 4 5 9 8 10 7 11 3 7 7 Is there a route with cost at most 52? YES (Route above costs 50.) Is there a route with cost at most 48? YES? NO? (C) 2010 Thomas J Cortina, Carnegie Mellon University
  24. 24. Traveling Salesperson ●If there are n cities, what is the maximum number of routes that we might need to compute? ●Worst-case: There is a flight available between every pair of cities. ●Compute cost of every possible route. oPick a starting city oPick the next city (n-1 choices remaining) oPick the next city (n-2 choices remaining) o... ●Maximum number of routes: (n-1)! = O(n!) how to build a route (C) 2010 Thomas J Cortina, Carnegie Mellon University
  25. 25. P and NP ●The class P consists of all those decision problems that can be solved on a deterministic sequential machine (e.g. a computer) in an amount of time that is polynomial with respect to the size of the input ●The class NP consists of all those decision problems whose positive solutions can be verified in polynomial time given the right information. from Wikipedia (C) 2010 Thomas J Cortina, Carnegie Mellon University
  26. 26. NP Complete ●The class NPC consists of all those problems in NP that are least likely to be in P. oEach of these problems is called NP Complete. oMonkey puzzle, Traveling salesperson, Hamiltonian path, map coloring, satisfiability are all in NPC. ●Every problem in NPC can be transformed to another problem in NPC. oIf there were some way to solve one of these problems in polynomial time, we should be able to solve all of these problems in polynomial time. (C) 2010 Thomas J Cortina, Carnegie Mellon University
  27. 27. Complexity Classes NP Problems P Problems NP Complete Problems But does P = NP? If P ≠ NP, then all decision problems can be broken down into this classification scheme. If P = NP, then all three classes are one and the same. The Clay Mathematics Institute is offering a $1M prize for the first person to prove P = NP or P ≠ NP. (http://www.claymath.org/millennium/P_vs_NP/) We know that P < NP, since any problem that can be solved in polynomial time can certainly have a solution verified in polynomial time. (C) 2010 Thomas J Cortina, Carnegie Mellon University
  28. 28. (C) 2010 Thomas J Cortina, Carnegie Mellon University
  29. 29. It gets worse... ●Tractable Problems oProblems that have reasonable, polynomial-time solutions ●Intractable Problems oProblems that have no reasonable, polynomial-time solutions ●Noncomputable Problems oProblems that have no algorithms at all to solve them (C) 2010 Thomas J Cortina, Carnegie Mellon University
  30. 30. Noncomputability and Undecidability ●An algorithmic problem that has no algorithm is called noncomputable. ●If the noncomputable algorithm requires a yes/no answer, the problem is called undecidable. ●Example: oGiven any set of any number of different tile designs (examples shown above), with an infinite number of each type of tile, can we tile any area with these tiles so that like colored edges touch? oThis problem is undecidable! (C) 2010 Thomas J Cortina, Carnegie Mellon University
  31. 31. Tiling Problem YES Note the periodicity in the tiling. (C) 2010 Thomas J Cortina, Carnegie Mellon University
  32. 32. Tiling Problem NO For this 3 X 3 room, if we try all 39 tiling configurations, no tiling works. (C) 2010 Thomas J Cortina, Carnegie Mellon University
  33. 33. Tiling Problem ●Possible algorithm: oIf we find a repeating pattern, report YES. oIf we find a floor we cannot tile, report NO. ●BUT: there are some tilings which have no repeating pattern! (C) 2010 Thomas J Cortina, Carnegie Mellon University
  34. 34. Tiling Problem ●The only way to know if this set of tiles can tile every finite-sized floor is to evaluate every possible floor. ●BUT: there are an infinite number of finite-sized floors! oSo we could never answer YES in this case. ●This problem is undecidable. (C) 2010 Thomas J Cortina, Carnegie Mellon University
  35. 35. Another Undecidable Problem: The Barber Paradox Suppose there is a town with one male barber; and that every man in the town keeps himself clean-shaven: some shave themselves and some are shaved by the barber. Only the barber can shave another man. The barber shaves all and only those men who do not shave themselves. Does the barber shave himself?(C) 2010 Thomas J Cortina, Carnegie Mellon University
  36. 36. Program Termination ●Can we determine if a program will terminate given a valid input? ●Example: 1. Input x 2. While x is not equal to 1, do the following: (a) Subtract 2 from x. ●Does this algorithm terminate when x = 15105? ●Does this algorithm terminate when x = 2008? (C) 2010 Thomas J Cortina, Carnegie Mellon University
  37. 37. Program Termination ●Another Example: 1. Input x 2. While x is not equal to 1, do the following: (a) If x is even, divide x by 2. (b) Otherwise, Set x to 3x + 1. ●Does this algorithm terminate for x = 15? ●Does this algorithm terminate for x = 105? ●Does this algorithm terminate for any positive x? (C) 2010 Thomas J Cortina, Carnegie Mellon University
  38. 38. The Halting Problem ●Can we write a general program Q that takes as its input any program P and an input I and determines if program P will terminate (halt) when run with input I? oIt will answer YES if P terminates successfully on input I. oIt will answer NO if P never terminates on input I. ●This computational problem is undecidable! oNo such general program Q can exist! oIt doesn’t matter how powerful the computer is. oIt doesn’t matter how much time we devote to the computation. oThe proof of this involves contradiction. (C) 2010 Thomas J Cortina, Carnegie Mellon University
  39. 39. start end Does this algorithm end? yes no Contradiction (C) 2010 Thomas J Cortina, Carnegie Mellon University
  40. 40. Contradiction isn't just for computer scientists... (C) 2010 Thomas J Cortina, Carnegie Mellon University

×