Elucidating the Limits of Computation

1. Limits of Computation

2. Hard vs Impossible
 Intractable Problems – trillions of years to
solve: practically unfeasible
 Unsolvable Problems – factually impossible, as
opposed to hard - .eg mutilated chessboard
problem

3. Hard vs Impossible
 Intractable Problems – trillions of years to
solve: practically unfeasible
 Unsolvable Problems – factually impossible, as
opposed to hard - .eg mutilated chessboard
problem

4. Complexity Class
 how fast #ops grow with size of input

5. Problem Types
 Optimization Problems: min/max. Must
examine all data
 Decision Problems: boolean -eg <, >. stop
when we find what we are looking for.
 Polynomial Problems: (P) – feasible,
tractable #ops
 Non-Polynomial Problems (NP): n^2.5+,
2^n, n!
 Eg Encryption Key asymmetry:
multiplication of 2 primes is n^2 , factoring is
NP

6. Easy Problems (P)
 Eg arithmetic, searching, sorting
 Brute-Force vs algorithm ‘tricks’ - eg binary-
search (split up, recurse)
 Appropriate algorithm may depend on type
of input

7. Graph Theory
Eg 6 degrees of
separation
Max #edges that one
must traverse to
connect any 2
vertices
7 bridges of
Konigsberg

8. Easy Graph problems
Euler Cycle problem
Q: is there a path that starts & finishes at
same vertex that passes every edge exactly
once ?
A: if every vertex has an even number of
edges.

9. Easy Graph problems
Euler Path problem
Q: is there a path that passes every vertex
exactly once ?
A: if Max 2 vertices have odd #edges & rest
have even

10. Hard Problems (NP)

11. Traveling Salesman Problem
 travel shortest route
to visit each city
once. (Weighted
graph)
 Brute-force: n! ops
 Eg n=100. For each
route, need to add
distance between
100 cities and &
compare sum to
shortest distance

12. Traveling Salesman Problem
 100! = 9.332622e+157
 Given a computer that can check 1 million
routes per sec → 3 x 10^144 years.
 There are 10^80 atoms in the universe – if
each were a computer, this would take
10^62 centuries. And this is for 100!

13. Hamiltonian Cycle Problem
 Is there a path that passes every vertex
exactly once that starts & finishes at same
vertex ?
 Brute force: n!
 Similar to Euler Cycle - but unfortunately no
trick discovered (& probably wont be) !!!

14. Hard Set Problems
Set Partition Problem
 split set of natural numbers into 2 groups
that both sum to the same number
 first evaluate solvability: is the total sum
Odd ?
 brute force: 2^n
Subset Sum Problem
 for a given set of natural numbers, and a
natural number C (capacity), is there a
subset that sums to C ? eg fill a room

15. The Satisfiability Problem
 deals with logical statements ^ (and), v (or), ~ (not) , →
(implies)
 eg (p v ~q) → ~ (p ^ q) – is this true if p=true and q=false ?
 Generalize: can we assign boolean values to variables to make
the statement true ?
 Logic Rules – eg Modus Ponens: ((p → q) ^ p) → q If today is
Tuesday, then John will go to work. Today is Tuesday.
Therefore, John will go to work.
 Brute-force:
 truth-table lookup: 2^n

16. NP Problems are Inherently Hard
Intractability is not because we lack the
technology.
Faster computers – if 10,000 times faster, TS
(100) problem still takes 3 x 10^140 years
Parallelization – Similarly, perform 10,000
simultaneous ops – same story
Quantum Computing – superposition of
search states: examine multiple possibilities at
once. For searching list of size n: Sqrt(n) ops.
But when searching through all n! possible
solutions to an NP problem: Sqrt(n!) ops –

17. Reduction
Transform a problem – reduction of a problem
x to y. if we can solve y then we can definitely
solve x
eg x is as hard as or easier than y: x <=p y
 climbing K2 is reducible to climbing mount
Everest
 Set Partition Problem is reducible to
Subset Sum Problem
 Hamiltonian Cycle Problem is reducible to
Traveling Salesman Problem

18. Reduction
 Cook-Levin
Theorem: because
a computer works
on logic gates, ALL
NP problems are
reducible to
Satisfiability
Problem (even
undiscovered ones)

19. NP-Complete and P != NP
 NP-Complete problems – the hardest NP
problems. Every NP problem is reducible to
NP-Complete. If we could find an algorithm
to solve it in P …
 Unsolved problem: If NP is a subset of P,
then P = NP. Most researchers believe P is
a subset of NP.

20. Approximation Algorithms
 if NP – best we can do is Heuristics. Better
than waiting 400 trillion centuries.
 Traveling Salesman Problem
approximation: use a Greedy Algorithm:
OSPF, Nearest Neighbor
 Set Partition Problem approximation:
Extreme Pairs – repeat, alternating
partitions: pick min & max from source set &
place in partition x

21. Even Harder problems
 Super-exponential – eg 2^(2^n) (if n=10,
need 2^1024 ops), (n!)!
 PSPACE (a superset of NP). Demand a
polynomial amount of space
 eg is there a winning strategy for n x n tic-
tac-toe ?
 Chess, Checkers, Nim, Go

22. Computing Impossibilities
 Q: Can computers recognize art, make
moral decisions ? A: Quantification of
Aesthetics, Ethics – subjective.
 Halting Problem
 Incompleteness Theorem

23. Some Definitions
Science – language we use to describe &
predict the physical & measurable universe
Applied Maths – the language of science
Logic – the language of reason
Epistemology – philosophy of human
knowledge & its limits

24. Logic
Use symbols to reason about structure of
proofs – set up axiom systems to put
mathematics on a firm foundation
Peano Arithmetic – Axiom system for natural
numbers
symbolization / arithmetization – conversion
between logical symbolic systems and
numeric systems

25. Logical Paradoxes
input: axioms + assumptions;
processing: logical reasoning laws;
output: logically unacceptable falsehoods,
self-contradictions, nonsensical false facts →
jettison assumptions
often counterintuitive (eg “space is
continuous”)
reductio ad absurdum (proof by
contradiction)

26. Logical Paradoxes
self-referential paradoxes - eg Liar Paradox:
“this sentence is false”
reduction – infer limitations from other
problems
reality, science, maths, logic cannot permit
contradictions – but our minds & language
can: vagueness, jokes

27. Zeno’s Paradoxes

28. The Halting Problem
Alan Turing, 1936: proof by contradiction (Liar
Paradox) that the Halting Problem is
undecidable
A limitation of mechanized processes, its not a
hard problem, its an impossible problem
a computer (or brain ?) cannot determine if a
black box program given input x will terminate
(with a return code) or enter an infinite loop.
Processing ….
https://www.youtube.com/watch?v=92WHN-
pAFCs

29. Oracle Machines & Turtles
imagine a non-mechanical mystic Oracle
Machine – the Halt Oracle can solve the
Halting Problem – could solve many other
unsolved math problems
eg Goldbach Conjecture: every positive
even number greater than 2 is a sum of 2
primes. (we know this holds up to 10^17)
the Halt Oracle searches for a counter-
example – if it exists, it can tell if the program
will halt.
Turing: Halting Problem for Halt Oracle

30. Computers vs Minds
Q: Can a human solve the Halting Problem ?
A: Kurt Godel , Sir Roger Penrose, Douglas
Hofstadfer - consciousness arises from
ability to self-reference. Self-Reference in
Computers brings paradoxical limitations,
while in humans it causes consciousness ???

31. Incompleteness
 Kurt Godel – the
man who broke
Logic
 Godel Sentence:
“This Logical
Statement is
Unprovable” →
contradiction
 1st Incompleteness
Theorem – There
are logical

32. Incompleteness
 If Peano Arithmetic
is consistent then
the Godel sentence
is unprovable and
true –> limitation of
basic arithmetic:
cannot determine
when its own
statements are
true / consistent.
 There are stronger

33. Incompleteness
 consistency of logical systems at basis of
mathematics and science is beyond the
bounds of reason!
 Note: majority of maths work does not
require working with basic axiomatic proofs