## Just for you: FREE 60-day trial to the world’s largest digital library.

The SlideShare family just got bigger. Enjoy access to millions of ebooks, audiobooks, magazines, and more from Scribd.

Cancel anytime.Free with a 14 day trial from Scribd

- 1. ngoodman@ stanford.edu (Reverse) Engineering Intelligence Noah D. Goodman Stanford University H+ Summit, June 12, 2010
- 2. What is thought?
- 3. What is thought? • How are thoughts structured?
- 4. What is thought? • How are thoughts structured? • How does this structure support ﬂexible, successful thinking?
- 5. What is thought? • How are thoughts structured? • How does this structure support ﬂexible, successful thinking? What mathematical principles can help us understand thought?
- 6. What is thought? • How are thoughts structured? • How does this structure support ﬂexible, successful thinking? e ngi ne e r What mathematical principles can help us understand thought?
- 7. Composition and probability
- 8. Composition and probability Thought is productive: “the inﬁnite use of ﬁnite means”
- 9. Composition and probability Thought is productive: “the inﬁnite use of ﬁnite means” ..a big green bear who loves chocolate..
- 10. Composition and probability Thought is productive: “the inﬁnite use of ﬁnite means” ..a big green bear who loves chocolate..
- 11. Composition and probability Thought is productive: “the inﬁnite use of ﬁnite means” p=mv
- 12. Composition and probability Thought is productive: “the inﬁnite use of ﬁnite means” p=mv
- 13. Composition and probability Thought is productive: “the inﬁnite use of ﬁnite means” p=mv Compositional representations
- 14. Composition and probability Thought is productive: “the inﬁnite use of ﬁnite means” Compositional representations
- 15. Composition and probability Thought is productive: “the inﬁnite use of ﬁnite means” Compositional representations
- 16. Composition and probability Thought is productive: “the inﬁnite use of ﬁnite means” Compositional representations
- 17. Composition and probability Thought is productive: “the inﬁnite use of ﬁnite means” Compositional representations
- 18. Composition and probability Thought is productive: “the inﬁnite use of ﬁnite means” Compositional representations
- 19. Composition and probability Thought is productive: Thought is useful “the inﬁnite use of in an uncertain ﬁnite means” world Compositional representations
- 20. Composition and probability Thought is productive: Thought is useful “the inﬁnite use of in an uncertain ﬁnite means” world Compositional representations
- 21. Composition and probability Thought is productive: Thought is useful “the inﬁnite use of in an uncertain ﬁnite means” world Why did he yell at me? Compositional representations
- 22. Composition and probability Thought is productive: Thought is useful “the inﬁnite use of in an uncertain ﬁnite means” world Why did he yell at me? He wanted to hurt me. He thought I was a telemarketer. Compositional representations
- 23. Composition and probability Thought is productive: Thought is useful “the inﬁnite use of in an uncertain ﬁnite means” world Why did he yell at me? Belief Desire Action He wanted to hurt me. He thought I was a telemarketer. Compositional representations
- 24. Composition and probability Thought is productive: Thought is useful “the inﬁnite use of in an uncertain ﬁnite means” world Why did he yell at me? Belief Desire Action He wanted to hurt me. He thought I was a telemarketer. Compositional Probabilistic representations inference
- 25. Composition and probability Thought is productive: Thought is useful “the inﬁnite use of in an uncertain ﬁnite means” world Compositional Probabilistic representations inference
- 26. Composition and probability Thought is productive: Thought is useful “the inﬁnite use of in an uncertain ﬁnite means” world Compositional Probabilistic representations inference
- 27. Composition and probability Thought is productive: Thought is useful “the inﬁnite use of in an uncertain ﬁnite means” world a+b+c = Compositional Probabilistic representations inference
- 28. Composition and probability Thought is productive: Thought is useful “the inﬁnite use of in an uncertain ﬁnite means” world a+b+c = 0 1 2 3 Compositional Probabilistic representations inference
- 29. Composition and probability Thought is productive: Thought is useful “the inﬁnite use of in an uncertain ﬁnite means” world a+b+c = 0 1 2 3 P (H|d) ∝ P (d|H)P (H) Compositional Probabilistic representations inference
- 30. Composition and probability Thought is productive: Thought is useful “the inﬁnite use of in an uncertain ﬁnite means” world Compositional Probabilistic representations inference
- 31. Composition and probability Thought is productive: Thought is useful “the inﬁnite use of in an uncertain ﬁnite means” world Compositional Probabilistic representations inference
- 32. Composition and probability Thought is productive: Thought is useful “the inﬁnite use of in an uncertain ﬁnite means” world ∀x King(x) =⇒ M an(x) ∀y M an(y) ⇐⇒ ¬W oman(y) Compositional Probabilistic representations inference
- 33. Composition and probability Probabilistic language of thought hypothesis Thought is productive: Thought is useful “the inﬁnite use of in an uncertain ﬁnite means” world ∀x King(x) =⇒ M an(x) ∀y M an(y) ⇐⇒ ¬W oman(y) Compositional Probabilistic representations inference
- 34. A probabilistic language
- 35. A probabilistic language Lambda calculus:
- 36. A probabilistic language Lambda calculus: (define double (λ (x) (+ x x)))
- 37. A probabilistic language Lambda calculus: (define double (double 3) => 6 (λ (x) (+ x x)))
- 38. A probabilistic language Lambda calculus: (define double (double 3) => 6 (λ (x) (+ x x))) (define repeat (λ (f) (λ (x) (f (f x)))))
- 39. A probabilistic language Lambda calculus: (define double (double 3) => 6 (λ (x) (+ x x))) (define repeat (λ (f) (λ (x) (f (f x))))) ((repeat double) 3) => 12
- 40. A probabilistic language Lambda calculus: (define double (double 3) => 6 (λ (x) (+ x x))) (define repeat (λ (f) (λ (x) (f (f x))))) ((repeat double) 3) => 12 Probabilistic lambda calculus: Goodman, Mansinghka, Roy, Bonawitz, Tenenabum (2008)
- 41. A probabilistic language Lambda calculus: (define double (double 3) => 6 (λ (x) (+ x x))) (define repeat (λ (f) (λ (x) (f (f x))))) ((repeat double) 3) => 12 Probabilistic lambda calculus: (define a (flip 0.3)) (define b (flip 0.3)) (define c (flip 0.3)) (+ a b c) Goodman, Mansinghka, Roy, Bonawitz, Tenenabum (2008)
- 42. A probabilistic language Lambda calculus: (define double (double 3) => 6 (λ (x) (+ x x))) (define repeat (λ (f) (λ (x) (f (f x))))) ((repeat double) 3) => 12 Probabilistic lambda calculus: (define a (flip 0.3)) => 1 (define b (flip 0.3)) => 0 (define c (flip 0.3)) => 1 (+ a b c) => 2 Goodman, Mansinghka, Roy, Bonawitz, Tenenabum (2008)
- 43. A probabilistic language Lambda calculus: (define double (double 3) => 6 (λ (x) (+ x x))) (define repeat (λ (f) (λ (x) (f (f x))))) ((repeat double) 3) => 12 Probabilistic lambda calculus: (define a (flip 0.3)) => 1 0 (define b (flip 0.3)) => 0 0 (define c (flip 0.3)) => 1 0 (+ a b c) => 2 0 Goodman, Mansinghka, Roy, Bonawitz, Tenenabum (2008)
- 44. A probabilistic language Lambda calculus: (define double (double 3) => 6 (λ (x) (+ x x))) (define repeat (λ (f) (λ (x) (f (f x))))) ((repeat double) 3) => 12 Probabilistic lambda calculus: (define a (flip 0.3)) => 1 0 0 (define b (flip 0.3)) => 0 0 0 (define c (flip 0.3)) => 1 0 1 (+ a b c) => 2 0 1 Goodman, Mansinghka, Roy, Bonawitz, Tenenabum (2008)
- 45. A probabilistic language Lambda calculus: (define double (double 3) => 6 (λ (x) (+ x x))) (define repeat (λ (f) (λ (x) (f (f x))))) ((repeat double) 3) => 12 Probabilistic lambda calculus: (define a (flip 0.3)) => 1 0 0 (define b (flip 0.3)) => 0 0 0 (define c (flip 0.3)) => 1 0 1 (+ a b c) => 2 0 1 .. Goodman, Mansinghka, Roy, Bonawitz, Tenenabum (2008)
- 46. A probabilistic language Lambda calculus: (define double (double 3) => 6 (λ (x) (+ x x))) (define repeat (λ (f) (λ (x) (f (f x))))) ((repeat double) 3) => 12 Probabilistic lambda calculus: probability / frequency (define a (flip 0.3)) => 1 0 0 (define b (flip 0.3)) => 0 0 0 (define c (flip 0.3)) => 1 0 1 (+ a b c) => 2 0 1 .. 0 1 2 3 Goodman, Mansinghka, Roy, Bonawitz, Tenenabum (2008)
- 47. Hypothesis • The probabilistic language of thought hypothesis: Mental representations are functions in a probabilistic lambda calculus. • Thoughts are built compositionally (like molecules). • Thinking is probabilistic inference. http://projects.csail.mit.edu/church
- 48. Bob’s box Goodman, Baker, Tenenbaum (2009; in prep.)
- 49. Bob’s box • Bob has a box with two buttons and a light. A B Goodman, Baker, Tenenbaum (2009; in prep.)
- 50. Bob’s box • Bob has a box with two buttons and a light. A B • He presses both buttons, and the light comes on. Goodman, Baker, Tenenbaum (2009; in prep.)
- 51. Bob’s box • Bob has a box with two buttons and a light. A B • He presses both buttons, and the light comes on. • How does the box work? A A A A A B B B B B C C C C C A alone B alone A or B A and B Nothing causes C. causes C. cause C. causes C. causes C. Goodman, Baker, Tenenbaum (2009; in prep.)
- 52. Human judgements Social 50 * 40 Social condition Mean Bets ($) 30 Physical 50 20 Physical condition 40 ns 30 10 20 0 A B AorB A&B none 10 N=15 0 A B AorB A&B none A alone B alone A or B A and B Nothing causes C. causes C. cause C. causes C. causes C.
- 53. Purely causal learning Causal!only model 0.5 (query Causal-only (define world-cs (cs-prior)) 0.4 (define action (uniform)) Probability 0.3 (define outcome (world-cs init-state 0.2 action)) 0.1 world-cs (and (press-A action) 0 A B AorB A&B none A or B A&B B only none A only (press-B action) Cause of C (light-on outcome))) No conclusion is possible. The evidence is confounded.
- 54. Explaining actions Beliefs: Desires: A B C Decision Rational action: Actions: (define decide (λ (state causal-model utility) (query (define action (action-prior)) action (flip (utility (causal-model state action))))))
- 55. Causal learning models Causal!only model 0.5 Causal-only model Causal-only Causal-only (define world-cs (cs-prior)) 0.4 (define action (uniform)) model Probability (define outcome (world-cs 0.3 init-state 0.2 action)) 0.1 0 A B AorB A&B none A or B A&B B only none A only Cause of C (define world-cs (cs-prior)) (define utility (uniform)) Social & causal (define cs-belief world-cs) Knowledgeable (define action (decide init-state agent assumption cs-belief Rational utility)) (define outcome (world-cs agent assumption init-state action))
- 56. Causal learning models Causal!only model 0.5 Causal-only model Causal-only Causal-only (define world-cs (cs-prior)) 0.4 (define action (uniform)) model Probability (define outcome (world-cs 0.3 init-state 0.2 action)) 0.1 0 A B AorB A&B none A or B A&B B only none A only Cause of C (define world-cs (cs-prior)) (define utility (uniform)) Social & causal (define cs-belief world-cs) (define action (decide init-state cs-belief utility)) (define outcome (world-cs init-state action))
- 57. Causal learning models Causal!only model 0.5 Causal-only model Causal-only (define world-cs (cs-prior)) 0.4 (define action (uniform)) Probability (define outcome (world-cs 0.3 init-state 0.2 action)) 0.1 0 A B AorB A&B none A or B A&B B only none A only Cause of C (define world-cs (cs-prior)) (define utility (uniform)) Social & causal (define cs-belief world-cs) (define action (decide init-state cs-belief utility)) (define outcome (world-cs init-state action))
- 58. Causal learning models Causal!only model 0.5 Causal-only model Causal-only (define world-cs (cs-prior)) 0.4 (define action (uniform)) Probability (define outcome (world-cs 0.3 init-state 0.2 action)) 0.1 0 A B AorB A&B none A or B A&B B only none A only Cause of C (define world-cs (cs-prior)) (define utility (uniform)) Social & causal Social!causal model 0.5 (define cs-belief world-cs) Social + causal model (define action (decide 0.4 init-state Posterior probability Probability 0.3 cs-belief utility)) 0.2 (define outcome (world-cs init-state 0.1 action)) 0 A B AorB A&B none
- 59. Scalar implicature Some of the plants have sprouted (Plants usually sprout.) Goodman, et al (in prep)
- 60. Scalar implicature Desires: -informative Beliefs -parsimonious Actions: “...” Some of the plants have sprouted (Plants usually sprout.) Goodman, et al (in prep)
- 61. Scalar implicature Desires: Model: -informative Beliefs -parsimonious Plausibility (Z-score) 2 1 0 -1 Actions: -2 “...” 0:5 1:5 2:5 3:5 4:5 5:5 Number sprouted Some of the plants have sprouted (Plants usually sprout.) Goodman, et al (in prep)
- 62. Scalar implicature Desires: Model: -informative Beliefs -parsimonious Plausibility (Z-score) 2 1 0 -1 Actions: -2 “...” 0:5 1:5 2:5 3:5 4:5 5:5 Number sprouted Some of the plants have sprouted (Plants usually sprout.) Goodman, et al (in prep)
- 63. Scalar implicature Desires: Model: Partial -informative Full knowledge knowledge Beliefs -parsimonious Plausibility (Z-score) 2 1 0 -1 Actions: -2 “...” 0:5 1:5 2:5 3:5 4:5 5:5 0:5 1:5 2:5 3:5 4:5 5:5 Number sprouted Some of the plants have sprouted (Plants usually sprout.) Goodman, et al (in prep)
- 64. Scalar implicature Desires: Model: Partial -informative Full knowledge knowledge Beliefs -parsimonious Plausibility (Z-score) 2 1 0 -1 Actions: -2 “...” 0:5 1:5 2:5 3:5 4:5 5:5 0:5 1:5 2:5 3:5 4:5 5:5 Number sprouted Human: Some of the plants have sprouted (Plants usually sprout.) Goodman, et al (in prep)
- 65. Summary • The probabilistic language of thought combines composition and probability. • We can explain complex, ﬂexible human thinking... • And engineer ﬂexible computer intelligence.