How to beat Jeff Hawkins

2,499 views
2,363 views

Published on

Published in: Business, Technology
0 Comments
2 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
2,499
On SlideShare
0
From Embeds
0
Number of Embeds
49
Actions
Shares
0
Downloads
121
Comments
0
Likes
2
Embeds 0
No embeds

No notes for slide

How to beat Jeff Hawkins

  1. 1. How can we beat Jeff Hawkins? I I , E , «-* I »' “‘ / i 3 , . ' _, I . ’ ~n ' .7 . I
  2. 2. Hawkins has five important things: 1. World-famous 2. Money 3. People 4. Prestige 5. Money
  3. 3. Let’s take a little look at history. ..
  4. 4. Linus Pauling : A Legendary Figure * Winner of the 1954 Nobel prize in Chemistry * “| nventor” of a | ego—type thing to try to find the structure of DNA, using ‘‘balls and sticks”. E: :I’. ‘‘ ‘ ‘i -X I-’ . . , ig «TE F F’ 5.
  5. 5. But James Watson and Francis Crick from Cambridge also saw the importance of the issue . ..they then decided to work on it, using the same technique developed by Pauling. Note: this was NOT a technique that the scientific community viewed as valid. III was extremely productive. Tihey cbuiicll EASIILY new lTlillQ1CII@I5?«
  6. 6. In May of 1952, Franklin takes the famous photograph 51, but sets it aside spending all her efforts on the “dry form" pictures that did not point as strongly towarda helix. In fact Franklin never formally published! or reported on any of her X—ray photos. Conditions at King's lnadl gotten so uneooiable that Frianlrllrii decides to leaivet.
  7. 7. Crick & Watson were working “close” to Rosalind Franklin, who was gathering data by X-ray crystallography, and they had “access” to “Photo 51".
  8. 8. Photo 51 was instrumental in their theoretical models, using ‘‘balls and sticks”. On Saturday, February 28, 1953 it is reported that Crick came into the Eagle, a Cambridge pub, and announced. to everyone there that they had "found the secret of life". Franklin came to Cambridge to see their model, and readily accepts it. i-iiu. xrir, in«. ».. iiiir. ... i.i; u.iif-mi. viimmiaini-i.4ix. i;: iniiir; uii»oiau: iir. s«nii. .
  9. 9. “You see, what Watson and Crick did was they guessed. They had intuition, their work was brilliant, they guessed. But there were things wrong with the structure they proposed that had to be refined by Wilkins. So it's a double-edged sword. I don't think she had intuition to the level of Crick, but I think very few people in the world ever have had intuition to the level of Crick. Crick and Linus Pauling were in a class by themsellvesi. ” liligzilvviwiualis. orgiwgbhinoveifpholofidrlelkiii. html
  10. 10. What lesson can we take from this? Productivity and the importance of change. The ability to be wrong, and to try many different forms, rapidly, without major wasted effort. Our architecturet must be EASY TO: Cl; -IANSGE.
  11. 11. -if
  12. 12. Hawkins has five important things: 1. World-famous 2. Money 3. People 4. Prestige 5. Money
  13. 13. On the other hand, Hawkins has a serious problem:
  14. 14. His theory is based on “hierarchical temporal memory”. .. '. -’(—I‘. n-d ( )hy-(ls ‘T-vvisuy in mi
  15. 15. The memory-prediction framework * Surprises get your attention ~ Understanding is about “generating expectations", about “a state of meaningful anticipation"
  16. 16. Problems with NUPIC Feedback — But where’s the “hedonic” form? - And where is the “attentional" form? A philosophical problem
  17. 17. One philosophical error in NuP| C “Therefore, pattern recognition is a “many-to-one” mapping problem. Many input patterns get mapped to each category. We will introduce a new term to describe many- to—one mapping: “pooling”. Pooling means assigning multiple patterns to one label, that is putting them in the same pooll. ” Ilicr. -urotmaexl Tompowl-Memory. ‘ ‘. ‘-m| u4:, ‘.ul*. -. I1. ml y, .-mu lt; .'lIuIlIo| ln.5y‘. . Jcflllnvtitirris and Dilcnp George. Nurmmtu Ina»
  18. 18. A “many-to—one” problem? E @ . .
  19. 19. So. .. * Some feedback types are lacking; * Pattern-recognition is not many-to-one But there is another major problem. ..
  20. 20. lr. 'iu: I_('n[-II . .4; = I _ I I- I . 'o ‘ 7 In 2 I lI. 'uI- It; will I . l » I I 7 . ‘ T I I 'lrail[: n{‘np1I. '! 2-, = “ ~ '. ' I . ‘ l L 3 3 7 lnun us, upnl . J I‘ L . - I l 1- I ‘~ 7 T '. train 1-: .u; n.l I l. _ ‘. ' _' I 3 _' 'v I J '- 'I1x-iuui; .u| ,aLl IL . | I T Q 0 3 1 T . _' ‘_ Examine—>-l
  21. 21. lrixin luuu '1 rain’ ltmu l mi u ' Tmiu _ : I_| ( ‘ll IIK . --.1 M; upll . upil " 'npu upll In-Ll C _upt. l m . l. .4 .24 la n / / / examawo 0 0 V. / ' 4 . ’/ Examine—>. . / ‘ l . ’
  22. 22. An application
  23. 23. And THIS is what an UNTRAINED network looks like. ’ ‘
  24. 24. . Hierarchical But things are more complex than that: _ x4: Belief structure IS not a given; . Sensory data grows . Time is essential, a lot (1 .000.000 sensors in human vision, for instance) . .-~ ~ « . . -. -.~. .-.5 and this data. is rapidly changingi Sensory Data
  25. 25. I have seen a picture of a gigantic matrix of meaningless numbers, used as a “debugging tool”. Do you want to see it, also?
  26. 26. Me too; I want to see it again! They took it out of the website! They figured out that that image was actually hurting them (most probably, in the effort to attract developers, investors, and form a community). So what is their p. robl. emi? *
  27. 27. Here is an “anecdote used by Jerry Jordan, president of the Federal Reserve Bank of Cleveland, in an article in the Cato Journal last summer. Jordan described a U. S. businessman visiting China a few years ago. The American came upon a team of 100 Workers bu. il[d. inigi a. dam with shovels. Sihzovels. ..
  28. 28. He commented to a local official that, with an earth-moving machine, a single worker could build the dam in an afternoon. The official replied, "Yes, but think of all the unemployment that would create. " "Oh, " said the businessman, ''I thought you were building a dam. If it's jobs you want to create, then take away their shovels and give them spoons. “ 77he~ Blessings of Free Trade. /23" James. K. Glassmam liitypzr/ twmmfreebade. oigrlmde/ B2‘
  29. 29. A second analogy: assembly language Addr Label Instruction Object coaelfl . beq; n . org £048 a start . eqL 3000 2045 1d 1enqLh, 2084 be done 00000010 10000000 00000000 00000110 JddCC 'r1,—4, I1 10000010 10000000 01111111 11111100 addcc L1, L2, :4 10001000 10000000 01003000 03000010 1d .14, 15 11031010 £380 be 100p 00010000 10111111 111-1111 1111101- 00000000 00000000 2084 addcc . r3, rh, I3 10000110 10000000 11000000 00000101 2088 done: jmpl r1564, r0 10000001 11000011 11100000 00000100 Length: 20 00000000 00000000 00000000 00010100 2050 dddlesaz 1 5LdIL 00030030 00000000 00001011 13111000 I C CC‘ C‘! in
  30. 30. Assembly language and machine language are fast, but not powerful in building complex software. Productivity skyrockets when a higher level language is used.
  31. 31. From PaulGraham. com: 1. Programming Languages Are for People. 2. Design for Yourself and Your Friends. 3. Give the Programmer as Much Control as Possible. 4. Aim for Brevity. N OTilHUNlC:1« of the sort is achievedl with assembly» languagfisr.
  32. 32. I think Hawkins is working on something rather like the assembly language level. If I am correct, productivity will be limited. Which can be rewritten as: they will be limited in what they can produce.
  33. 33. But it gets worse: From assembly-language you can build higher- level languages, like c++, and raise productivity. Unless Numenta can come up with some very innovative form of understanding what’s going on in that massive data, productivity will not rise. If we can't find meaning in those huge matrices, that’s an obstacle to progress.
  34. 34. -If
  35. 35. TCF’-IP * Application layer (Http, ftp, pop3, smtp) * Transport layer (TCP, UDP) 4* Internet layer (IPv4, IPv6) * Data link layer (Ethernet, WiFi) Physical layer (modems, optical fiber) What is mean. i‘nlg. flu. ll to al humanfs. eyes?
  36. 36. The application layer is meaningful * The underlying layers are ‘‘black boxes”, you don’t need to care whether you’re working on IPv6 over WiMax. * That's just the substrate the application layer is working on at a particular time.
  37. 37. Neurons are just the substrate of thought Of course, it is possible that all cognitive phenomena will emerge by studying the substrate. But, then again, why stop at neurons? Why not go to atoms? Or to the quantum physics level?
  38. 38. What is the right level for research? What is the most productive level for research? The level in which cognitive phenomena emerges, in their fullness --priming --emergent cognition --concepts and categorization and filling in -—feedback in its different forms AND independent of the substrate -~erhaps. we don’t need to simulate individual neurons. Nobody argues that we should simulate quantum phenomena. (Iexcept maybe for Pemroeelt.
  39. 39. From Harry’s thesis. .. Biological "’Pt'ograI1I11e<1” Cngnilit n Cngniti 111 Visiuil pullcnis liaeacos retinal level (ieneral dam SIl'l1CT|1l‘€S Bit-strul-‘as of n1en1<: uy Retina & visiial cmtex Ne1Ii‘ons Figllre 4.9: Lxpectation of com‘e1'g=11ce l. ‘|t‘I‘. V‘c“c"Il biological and “p1‘ogra| ninecl" cognition
  40. 40. Don’t think of an elephant! (i) slipnet r Aw‘. ..-. ... ... :.. ;.. ‘.. _.. 4: ‘ihruu-an-; :_q1_ ; __ _‘ ‘I -~—. _
  41. 41. Don’t think of an elephant! (ii) The TActivation class Don't think of an elephant: activating symbols _. —— _ . /- Q .1. 13 - I A-: tIvat 2n eve! E C L rrent state
  42. 42. How to implement the sigmoid? B3=| F(A3<=0.5,A3,1.0-A3) 1.0 ,7/-'- 03 / ' 0.6 / Activation level 0.4 (sum) Activation level
  43. 43. How to implement the sigmoid? B3=IF(A3<=0.5,A3,1.0-A3) B3=(0.04*| F(A3<=0.5,A3,1.0-A3))+B2 1.0 , /-'- 0.8 / 0.6 , / Activation level (sum) 0.4 Activation level 0.2 0.0 I: urrent_state
  44. 44. {This method provides the sigmoid in variable level, given a new current_state} Procedure TActivation. Recompute_Level; var pyramid, sum, tzreal; counter: integer; begin Sum: =0; for counter: = 0 to floor (Current_State*max_Steps) do begin t: = counter/ max_steps; If(t<=0.5) then Pyramid: =t else pyramid : =1-t; Sum: =(4*(1/max_steps)* Pyramid) + Sum; end: t. evel: = Sum; end;
  45. 45. Class Structure Tactivation = class private current_state, level, increment: real; Signa| s_Received: real; procedure Recompute_Level; public Constructor Create; Function Get_Level: real; Function Get_CurrentState: real; Function Get_| ncrement: real; Procedure Reset_lncrement; Procedure increase (step: real): Procedure Deploychange; Procedure decay: end:
  46. 46. Class Structure Tactivation = class private current_state, level, increment: real; Signa| s_Received: real; procedure Recompute_Level; public Constructor Create; Function Get_Level: real; Function Get_CurrentState: real; Function Get_| ncrement: real; Procedure Reset_lncrement; Procedure increase (step: real): Procedure Deploychange; Procedure decay: end: Bureaucratic stuff
  47. 47. How do we build an architecture that’s easy and simple to change? Welcome to the world of Design Patterns.
  48. 48. -If
  49. 49. Class Structure Tactivation = class private current_state, level, increment: real; Signa| s_Received: real; procedure Recompute_Level; -) Strategy Pattern public Constructor Create; Function Get_Level: real; Function Get_CurrentState: real; Function Get_| ncrement: real; Procedure Reset_| ncrement; Procedure increase (step: real): Procedure Deptoychange; Procedure decay: -> Strategy Pattern end: Bureaucratic stuff
  50. 50. A software architecture problem Duck M dun ‘ugh . —.c ~. ... . in / ,—--> ‘. u.1r. k uf>, PVUdu bake; tan: 0‘ the swimr m-p‘r-rv-{)fitw~ fad! 369.6:/ |_ {e — T5» 7, . <4‘ 7"" V’ duqiawj -‘ . ' Icv; -s -9 a 'r. -:*~: 11}
  51. 51. Now we need flying ducks. .. l r Duck i iuac-1"‘ SW [Ni Gun‘ 3/‘] __5,_«. ,/ '—‘§ fly” r. ‘». .1, ‘»’ ‘odd ‘ l ‘ ~ 'lT'*FR fiia_. I‘I-. e"-e: i‘-Js l i Maiiardouck Rodhgaaoucfi ¢5«'7'€'r . ' ‘ (2 '>D1&‘_. :l to v "‘ -Dc-5 ‘-5 a 72': 4n: j» v‘ xxx‘ he a redraar
  52. 52. But now we want to bring up rubber ducks. .. 1 , ' () b1n: l‘l)'. .-L ])lUL'l. L.'il O ‘ n . -l ! l‘| |<i'V4.. :{l: ‘ . '._, I~. ’u! z . 'w :5: . m1:' rua: .»: ‘r: ,1 11"} ‘ at he 4'»/ wt. " ‘Hz ', ’,“’. f A‘f? ~,1'1; 71.i‘~I"w d11.“. ‘. v ' was a use T“b, ,c; mt inheritance auacl-. ij; W WW {or the purpose’ M. ‘it; rL : _w‘> denim. o{ reuse 11asnt Y“ . ‘W . .-P‘ /6 “Y0 %: ‘¢Y(_; :D P‘L~‘u’ 3x3)’ 1 CL. lug niafpog 50 ‘o')'*‘ » 0*‘ V K . ’". o"‘g, ‘“ when it comes to xv-*5’ . " maintenance. Ruoboroucli y —— F — it 4 i -P‘ ‘ ‘: U3CHi e— Rvbuf K: ov: wuddC“ i. ‘ ovcrnzden in Sunset: ‘° "ft - it Aw‘ I’ l ‘ displayli K looks 1-e a rubbercuitc i
  53. 53. One way is to create classes for the behaviors. But. .. What do YOU think about this design?
  54. 54. Now we hit a wall. (i) We cannot call a superclass “duck" object, and expect it to be able to, for example, “fly”. “duck. fly” brings up a compile-time error. (ii) DUPLICATE CODE. Every time we may want to change “fly” behavior, we have to CHANGE the code in every duck that implements it. Remember, we want things to be EASY TO CHANGE. How can we deal with this? ‘
  55. 55. or . Design Principle Identify the aspects of your application that vary and separate mom from what stays the same. 4 K c. .- $. .»s+. cl? -any 463")‘ fr ac. ‘-'r_x. i'I'(‘l' sfchd " "'¢ 5'" 0,, {. |’Y'lO“"‘0“{ £“iC 990k . t.u {kc ~~r~¢‘r“' 1»: 9*“ “’" rum out A ~= ~s~: 0"‘ “V ‘"4 ‘vi. M £50 (.133: 7-ui. JtJ't'_‘. iv" M J aw x§J'*l 5" ‘ A ‘i
  56. 56. From now an. lino Durit iaclmrioi-s will live in a xepar-ate ctassda clan tiuat imptcmcnts u [mrticirtur Lclaavior interface. Tltut way. tin: Ducic ciusscs won't neeri to isnow any oi the implementation cietnils {or their own luelaax-tors. rm’. 1.! 1 v. ,T+m. , - _- _, ,[ L‘ { 4 I . p, . ”‘'’‘"~o, ‘ P. .. i _' I‘ “ ac-V “" . r L J “H; 1‘ if 0"’ I """'r. ;(__ “V. J, ” W‘ ‘:1’ '1 Cg. ’ tn»- ' 3.1 . ,., .2 H" " ""7 ‘ ‘Edi’ , ,._4 . .-mo a 'r“") A” J , , thud ]. l _‘L_ , ,.l~ . « In an '1’. I . ‘- ,5 Ir’ ’
  57. 57. c‘; ¢.. £ makes use 05 ir ’ V _r—E: ¢::1psulated fly behavior <rc~= r=*'3*“ ‘M7 A LLJMM I ; .1:“. :: .; tar both ftrrs, *5 V” "3 Client . .. Fyfcuu-xwurnav Q. --Bars». -«.1-nrv--zc 7 Enc. +paul.1t-_~d quucl-: behavior ~i. —-. -- «.44?» "J
  58. 58. Duign Principle ‘-. :w'. I n4'ams'no' ova '. J.wnla'cB ‘ Congratulations on your itrst pattern! You Just applied your ‘list desgi pattnr1—the STRATEGY pattern. mars nght. you usod the Strategy Paltutn to towork tho SlmUDuck app. Thanks to this pattern. the simulator is iuady for any changou thou oxoco might cook up on their next businus trip to Vogu. Now that we vo made you take the long road to apply it, here's the formal definition of this pattern: The Sumqy Patten: L‘c: "iI. rs a Txnilv of algorithms. encapsuhu-s each one. and umkc: chum intrrthangrablr. Strategy lets the algxxittnz vary indc? end. =i'. lt)' from clients that use : t.
  59. 59. So OUR own code for “activation. increase” should be a Strategy Pattern 1. All instances of the class can call “increase” (even if they don’t implement the method) 2. We minimize duplicate code 3. We can CHANGE behavior at RUNTlME! i If a global signal (pain, pleasure. attentionr boredom) is received. than we can immediately change behavior. if meeded.
  60. 60. .0". -'*‘. °’ Summary We now start to develop our cognitive models We are working on a level that's higherthan Jeff Hawkins’s, but lower than traditional Al, and is still emergent cognition. Concepts get activated to different degrees in our minds. We present a class (of behaviors) for this activation We study the Strategy Pattern, our first design pattern.

×