Towards a mathematical understanding of intelligence
Towards a MathematicalUnderstanding of Intelligence V. Kosoy
What is Intelligence?• AI dates back at least to the 1950s• There is no accepted definition of what intelligence is• Turing test doesn’t count: empirical rather than mathematical
What is Intelligence?• Most every day objects don’t have mathematical definitions• My opinion: intelligence must be different• It seems natural to seek for this definition in the domain of computation theory
Characteristics of Intelligence• Pattern recognition• Prediction• Universal problem solving
Pattern Recognition• Can be formalized as the Kolmogorov complexity problem• What is the shortest program producing given data (string of bits)?• Random string: has to be hardcoded• 010101010…01 has compact description
Pattern Recognition• Kolmogorov complexity is uncomputable due to Berry paradox!
Solomonoff Induction• Bayesian inference for a universe generated by a random program• This distribution favors low Kolmogorov complexity: formalization of Occam’s razor!• Making best guess with this prior allows predicting any computable sequence• This procedure is uncomputable!
Imperfect Prediction• Shane Legg ‘06• No Universal Predictors• Predicted complexity vs. predictor complexity• Unprovability
Universal Problem Solving• Arguably we can only solve problems for which the solution can be efficiently verified• Corresponds to non-deterministic algorithms• Efficiently computing non-deterministic algorithms is possible iff P = NP
Legg-Hutter Intelligence• Shane Legg, Marcus Hutter 2007• Quantitative rather than qualitative• Black boxes with input, output and utility• Average utility in a random program universe• No complexity considerations
Goedel Machine• Juergen Shmidhuber 2003• Essentially the same black box framework• Reprograms itself using (Levin) proof search• Metalearner: everything is reprogrammable• Universe prior can be e.g. Solomonoff• Limited by axiom system• The degenerate environment problem
Asymptotic Computation• Kolmogorov complexity and universal prediction are asymptotically computable• This is a realistic model of intelligence: we can’t be sure we found the best model
P vs. NP revisited• NP oracle allows efficient pattern recognition• NP oracle allows efficient prediction• NP oracle allows universal problem solving• Maybe the problem is hard because its solution is key to understanding the nature of intelligence and even reality itself
Summary• There is no satisfactory definition yet• Connection between properties of intelligence and natural concepts in computer science is ominous• Solomonoff induction, the works of Hutter, Shmidhuber and Legg provide important pieces for the puzzle• Further progress will come from P vs. NP