This document discusses the limits of computation. It distinguishes between intractable problems that take an impractical amount of time to solve versus truly unsolvable problems. It describes different complexity classes based on how fast the number of operations grows with input size. Hard problems like the traveling salesman problem are in NP and are inherently difficult even with faster computers or quantum computing. Reductions show relationships between problem difficulties. The halting problem and Godel's incompleteness theorems establish fundamental limits of computation and logical systems.

1. Limits of Computation

2. Hard vs Impossible
 Intractable Problems – trillions of years to
solve: practically unfeasible
 Unsolvable Problems – factually impossible, as
opposed to hard - .eg mutilated chessboard
problem

3. Hard vs Impossible
 Intractable Problems – trillions of years to
solve: practically unfeasible
 Unsolvable Problems – factually impossible, as
opposed to hard - .eg mutilated chessboard
problem

4. Complexity Class
 how fast #ops grow with size of input

5. Problem Types
 Optimization Problems: min/max. Must
examine all data
 Decision Problems: boolean -eg <, >. stop
when we find what we are looking for.
 Polynomial Problems: (P) – feasible,
tractable #ops
 Non-Polynomial Problems (NP): n^2.5+,
2^n, n!
 Eg Encryption Key asymmetry:
multiplication of 2 primes is n^2 , factoring is
NP

6. Easy Problems (P)
 Eg arithmetic, searching, sorting
 Brute-Force vs algorithm ‘tricks’ - eg binary-
search (split up, recurse)
 Appropriate algorithm may depend on type
of input

7. Graph Theory
Eg 6 degrees of
separation
Max #edges that one
must traverse to
connect any 2
vertices
7 bridges of
Konigsberg

8. Easy Graph problems
Euler Cycle problem
Q: is there a path that starts & finishes at
same vertex that passes every edge exactly
once ?
A: if every vertex has an even number of
edges.

9. Easy Graph problems
Euler Path problem
Q: is there a path that passes every vertex
exactly once ?
A: if Max 2 vertices have odd #edges & rest
have even

10. Hard Problems (NP)

11. Traveling Salesman Problem
 travel shortest route
to visit each city
once. (Weighted
graph)
 Brute-force: n! ops
 Eg n=100. For each
route, need to add
distance between
100 cities and &
compare sum to
shortest distance

12. Traveling Salesman Problem
 100! = 9.332622e+157
 Given a computer that can check 1 million
routes per sec → 3 x 10^144 years.
 There are 10^80 atoms in the universe – if
each were a computer, this would take
10^62 centuries. And this is for 100!

13. Hamiltonian Cycle Problem
 Is there a path that passes every vertex
exactly once that starts & finishes at same
vertex ?
 Brute force: n!
 Similar to Euler Cycle - but unfortunately no
trick discovered (& probably wont be) !!!

14. Hard Set Problems
Set Partition Problem
 split set of natural numbers into 2 groups
that both sum to the same number
 first evaluate solvability: is the total sum
Odd ?
 brute force: 2^n
Subset Sum Problem
 for a given set of natural numbers, and a
natural number C (capacity), is there a
subset that sums to C ? eg fill a room

15. The Satisfiability Problem
 deals with logical statements ^ (and), v (or), ~ (not) , →
(implies)
 eg (p v ~q) → ~ (p ^ q) – is this true if p=true and q=false ?
 Generalize: can we assign boolean values to variables to make
the statement true ?
 Logic Rules – eg Modus Ponens: ((p → q) ^ p) → q If today is
Tuesday, then John will go to work. Today is Tuesday.
Therefore, John will go to work.
 Brute-force:
 truth-table lookup: 2^n

16. NP Problems are Inherently Hard
Intractability is not because we lack the
technology.
Faster computers – if 10,000 times faster, TS
(100) problem still takes 3 x 10^140 years
Parallelization – Similarly, perform 10,000
simultaneous ops – same story
Quantum Computing – superposition of
search states: examine multiple possibilities at
once. For searching list of size n: Sqrt(n) ops.
But when searching through all n! possible
solutions to an NP problem: Sqrt(n!) ops –

17. Reduction
Transform a problem – reduction of a problem
x to y. if we can solve y then we can definitely
solve x
eg x is as hard as or easier than y: x <=p y
 climbing K2 is reducible to climbing mount
Everest
 Set Partition Problem is reducible to
Subset Sum Problem
 Hamiltonian Cycle Problem is reducible to
Traveling Salesman Problem

18. Reduction
 Cook-Levin
Theorem: because
a computer works
on logic gates, ALL
NP problems are
reducible to
Satisfiability
Problem (even
undiscovered ones)

19. NP-Complete and P != NP
 NP-Complete problems – the hardest NP
problems. Every NP problem is reducible to
NP-Complete. If we could find an algorithm
to solve it in P …
 Unsolved problem: If NP is a subset of P,
then P = NP. Most researchers believe P is
a subset of NP.

20. Approximation Algorithms
 if NP – best we can do is Heuristics. Better
than waiting 400 trillion centuries.
 Traveling Salesman Problem
approximation: use a Greedy Algorithm:
OSPF, Nearest Neighbor
 Set Partition Problem approximation:
Extreme Pairs – repeat, alternating
partitions: pick min & max from source set &
place in partition x

21. Even Harder problems
 Super-exponential – eg 2^(2^n) (if n=10,
need 2^1024 ops), (n!)!
 PSPACE (a superset of NP). Demand a
polynomial amount of space
 eg is there a winning strategy for n x n tic-
tac-toe ?
 Chess, Checkers, Nim, Go

22. Computing Impossibilities
 Q: Can computers recognize art, make
moral decisions ? A: Quantification of
Aesthetics, Ethics – subjective.
 Halting Problem
 Incompleteness Theorem

23. Some Definitions
Science – language we use to describe &
predict the physical & measurable universe
Applied Maths – the language of science
Logic – the language of reason
Epistemology – philosophy of human
knowledge & its limits

24. Logic
Use symbols to reason about structure of
proofs – set up axiom systems to put
mathematics on a firm foundation
Peano Arithmetic – Axiom system for natural
numbers
symbolization / arithmetization – conversion
between logical symbolic systems and
numeric systems

25. Logical Paradoxes
input: axioms + assumptions;
processing: logical reasoning laws;
output: logically unacceptable falsehoods,
self-contradictions, nonsensical false facts →
jettison assumptions
often counterintuitive (eg “space is
continuous”)
reductio ad absurdum (proof by
contradiction)

26. Logical Paradoxes
self-referential paradoxes - eg Liar Paradox:
“this sentence is false”
reduction – infer limitations from other
problems
reality, science, maths, logic cannot permit
contradictions – but our minds & language
can: vagueness, jokes

27. Zeno’s Paradoxes

28. The Halting Problem
Alan Turing, 1936: proof by contradiction (Liar
Paradox) that the Halting Problem is
undecidable
A limitation of mechanized processes, its not a
hard problem, its an impossible problem
a computer (or brain ?) cannot determine if a
black box program given input x will terminate
(with a return code) or enter an infinite loop.
Processing ….
https://www.youtube.com/watch?v=92WHN-
pAFCs

29. Oracle Machines & Turtles
imagine a non-mechanical mystic Oracle
Machine – the Halt Oracle can solve the
Halting Problem – could solve many other
unsolved math problems
eg Goldbach Conjecture: every positive
even number greater than 2 is a sum of 2
primes. (we know this holds up to 10^17)
the Halt Oracle searches for a counter-
example – if it exists, it can tell if the program
will halt.
Turing: Halting Problem for Halt Oracle

30. Computers vs Minds
Q: Can a human solve the Halting Problem ?
A: Kurt Godel , Sir Roger Penrose, Douglas
Hofstadfer - consciousness arises from
ability to self-reference. Self-Reference in
Computers brings paradoxical limitations,
while in humans it causes consciousness ???

31. Incompleteness
 Kurt Godel – the
man who broke
Logic
 Godel Sentence:
“This Logical
Statement is
Unprovable” →
contradiction
 1st Incompleteness
Theorem – There
are logical

32. Incompleteness
 If Peano Arithmetic
is consistent then
the Godel sentence
is unprovable and
true –> limitation of
basic arithmetic:
cannot determine
when its own
statements are
true / consistent.
 There are stronger

33. Incompleteness
 consistency of logical systems at basis of
mathematics and science is beyond the
bounds of reason!
 Note: majority of maths work does not
require working with basic axiomatic proofs