Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

The Second Orientation for the WBA Hackathon 2017


Published on

The Second orientation material for the WBA Hackathon 2017

Published in: Technology
  • Be the first to comment

The Second Orientation for the WBA Hackathon 2017

  1. 1. The Second Orientation for the WBA Hackathon 2017 Material to be Distributed 2017-07-23
  2. 2. WBAI理念 Vision: to create a world in which AI exists in harmony with humanity Values Study: Deepen and spread our expertise Imagine: Broaden our views through public dialogue Build: Create AGI through open collaboration Mission: to promote the open development of WBA Basic Ideas
  3. 3. The Third Whole Brain Architecture Hackathon Let’s Create AGI Prototypes! Wake up, Hippocampus! Hackathon: September 16th -18th, 2017 Venue: φcafe, Tokyo End of Registration: August 8th
  4. 4. Hiroshi Yamakawa The Aim of the Hackathon
  5. 5. Human Intelligent Features • Having values • Having survival capablity • Being conscious • Having an Ego • Versatile for various tasks → General Intelligence • Etc. GI that WBAI aims for is one of the intelligent characteristics that the current AI has not realized at the human level.
  6. 6. Evaluating Generality (Utility) Tasks to be solved at a satisfactory level Task Areas Satisfactory Level
  7. 7. Evaluating Generality (Utility) Tasks to be solved at a satisfactory level 7 Task Areas Satisfactory Level
  8. 8. Advances in DL Learning from the Brain Nengo (2015〜)Symbol Emergence in Robotics A Map of AGI Developing Organizations Engineering Realization Neocortex oriented Whole Brain oriented
  9. 9. ML Knowledge Design Connectome Two-photon Imaging The Brain is ready to serve for AI! ANNAI (Cog. Sci) Measuring in Neuro- science Screws Pistons Automobile Analogy Overall Design Human Brain Engine MacroMesoMicro Higher-level Description Language Whole Brain ANN model Whole Brain Cognitive Architecture McCulloch & Pitts Model Programming Language Neocortical Areas Neurons Local Neural Circuits Electrodes fMRI, EEG Deep Learning Unified Understanding Multi- layered perceptron Perceptron Constraining AI Architecture
  10. 10. WBA can be the fastest path to AGI. ‘to create a human-like artificial general intelligence (AGI) by learning from the architecture of the entire brain.’ An anatomical illustration from Sobotta's Human Anatomy (1908)
  11. 11. Brain-Inspired Open Platform Strategy@WBAI Systematizing catch-up to reproduce (verify) advancing technologies Collaborative development on a brain-inspired open platform, enabling technological integration Rapid Advances in technologies around AGI (e.g., @DeepMind) To “harmonize AGI with human beings” by democratizing AI technologies
  12. 12. Hackathon Teams & Sample Code Neuroscientific Info. (e.g., connectome) Virtual Mouse Virtual Experiment Env. (Unity) Architecture Description Middleware (BriCA Core ver.1) ML Modules (Refactored DQN) ・Neocortex: CNN ・BG: Q-Learning ・Hippocampus: Episodic replay Virtual Task Env. ROS-like, light- weight, multi- platform with virtual time support Engineer Architect Neuroscientist
  13. 13. BriCA Language Description (in JSON) "Modules" : [{ "Name" : "BriCA1.MainModule", "Ports" : [ "Port1", "Port2" ], "SuperModule" : "SuperMainModule", }, { "Name" : "SuperMainModule", "Ports" : [ "PortS1", "PortS2" ], } ], "Connections" : [{ "Name" : "Con3", "FromModule" : "SuperMainModule", "FromPort" : "PortS1", "ToModule" : "BriCA1.MainModule", "ToPort" : "Port1", }, { "Name" : "Con4", "FromModule" : "BriCA1.MainModule", "FromPort" : "Port2", "ToModule" : "SuperMainModule", "ToPort" : "PortS2” ] } Basic elements for architecture –Modules –Ports –Connections SuperMainModule "Ports" : [{ "Name" : "Port1", "Module" : "BriCA1.MainModule", "Type" : "Input", "Shape" : [3,1,1], }, { "Name" : "Port2", "Module" : "BriCA1.MainModule", "Type" : "Output", "Shape" : [3,1,1], …. BriCA1.MainModule Port1 Port2 PortS1 Con3 PortS2 Con4
  14. 14. BriCA Language for Brain-inspired Platform Description BriCA Language: DSL to describe the structure of cognitive architecture Points: • Bridging neuroscientific findings (connectome) & AI • Basis for collaborative development of architecture – Combining ML Modules (ANNs, graphical models) – Middleware independent (Brica Core, ROS, etc.)
  15. 15. Wake Up! Hippocampus Major functions of the Hippocampus: • Episodic Memory ⇒ Long-term Memory • Navigation/Self Location • At the top of the perception
  16. 16. Overall WBA for the Hackathon (Containing Hippocampus)
  17. 17. The Aim of the Hackathon • Goal: – To make the first working WBA • Sample Code: – Deep Reinforcement Learning (DQN) refactored for a simple WBA – The Hippocampus module works as the replay memory of DQN. • To do in the Hackathon: – Implement hippocampus functions such as episodic memory and space cognition by modifying the sample code
  18. 18. H. Mizutani Hackathon Tasks
  19. 19. Hippocampus & the realization of AGI • There have been partial implementations for hippocampal functions (listed below) , but not done on a single system. ・Episodic Memory ・One-shot Learning ・Working Memoryー ・Space Cognition ・Self Location ・Interaction with Environment ・Transfer Learning ・Efficient Learning ・Decision Making ・Action Generation ・Attention ・Planning ・Imagination ・Intuition ・Consciousness • Hackathon participants will develop brain-inspired cognitive architecture, which could be a prototype of AGI by implementing the Hippocampus.
  20. 20. Hippocampus: the Seat of Memory Major functions of Hippocampus • Episodic Memory • Space Cognition, Navigation –Place Cells: fire only when the animal passes particular locations in the environment. –Grid Cells: fire when the animal comes to certain places located at the vertices of triangles in the environment –Head Direction Cells: fire according to the direction of the head Hippocampus Laszlo Seress ©CC-BY-SA We offer Maze Tasks that cannot be solved without Hippocampal functions
  21. 21. Provided Maze Tasks
  22. 22. Provided Maze Task (Place Conditioning) • Linear Maze Task • The linear track has patterns on the walls and floor and reward points indicated by blue, green, and red. • The mouse starts from an end of the track and receives a reward if it waits at a reward point for two seconds. • A reward is placed at a presented green spot at the center. The mouse receives a reward if it waits at the spot for two seconds. – Q: how to learn by utilizing place cells and projections to/from the cortex? (Masaaki S et al., eNeuro 2017) • Re-learning by placing rewards at other spots (blue or red) . – Q: What kind of cell activities (e.g., of place cells) facilitates re-learning?
  23. 23. Evaluation Task Styles • Systems developed in the hackathon will be evaluated with tasks on an environment simulator (Unity/LIS). • While the following task styles are available, a system with higher generality to realize multiple tasks and functions will be evaluated highly. • Provided Tasks –Provided by WBAI • Free Tasks –Set by participants
  24. 24. Evaluation (Judging) Policies • Developed systems will be evaluated by juries with the following criteria: –Neuroscientific reality • Realization of various cognitive functions on hippocampus • Neural connectivity between hippocampus and other brain areas • Correspondence with neuronal activities of hippocampus and other brain areas –Engineering utility –Originality • Systems utilizing the software provided by WBAI, such as BriCA, WBCA, BiCAmon will get extra points. • The winner will be awarded with cash prize.
  25. 25. An example of hippocampal modeling (robot navigation) Tang et al., Neural Networks archive Volume 87: 27-37 (2017) Pro: Navigation with hippocampal neural network. Con: an implementation for the specific task. ’s model
  26. 26. An example of hippocampal modeling (Place Cells) Lőrincz A. and Szirtes G. Neural Networks 22 (2009) 738–747 Pro: Reproducing Place Cells Con: Neuroscientific reality of the algorithm is debatable. ’s model ICA was used for pattern separation.
  27. 27. OpenRatSLAM (Self Location/Navigation) Pro: Self Location Con: Connections within the hippocampus are not reproduced.
  28. 28. CA1 EC(MEC/LEC) GC CA3 Ⅴ&Ⅵ Ⅲ Ⅱ Coordinate transformatio n Goal state P State prediction error Pattern separation Newborn cell MC Generate intention sequence Sb Nucleus accumbens, Medial septum Self-position estimation Deep Shallow PER/PO R Unimodal/ Polymodal Ctx Generate intention PreSb ParaSb P P Mammillary bodies, ATN of thalamus Abstraction of intention Medial septu m HD, B, P, G, Band-like HD HD,B,G HD, B,P, G HD, B, P HD,B,G B,G Current state Global current state (MEC only) Global goal state Shallow Deep Shallow Shallow HD: Head-direction cell, B: Border cell, G: Grid cell, P: Place cell Abstraction of intention HP Recurrent network Spatial depended cell (※ Notation in EC exists only in MEC) A Hippocampal Model by (Yamakawa, JSAI2015) Gatsby-Kaken Joint Workshop on AI and Neuroscience Pro: covering many sub-modules; Con: no implementation.
  29. 29. Check sheet Episodic Memory Space Cognition Place Cells Hippocampal Circuits A STAR NG ? NG ✔ Lorincz NG NG ✔ ? RatSLAM NG ✔ NG NG
  30. 30. The points of the Hackathon • While Neocortex Module, Basal Ganglia Module, & Hippocampus Module are offered, the Hippocampus Module only implements experience replay. • Modules & connections among them are specified based on connectome. Participants are to compete for solving tasks by implementing Hippocampus Modules. • If an AI system realizes multiple cognitive functions here, it would advance AGI development.
  31. 31. N. Sakai & M. Ueno The System
  32. 32. Agenda 1. Systems Summary 2. Systems Requirement & Environment Building 3. Products & supports offered by WBAI 4. Demonstration & comments
  33. 33. 1. Systems Summary Demand action direction from Unity Task Environment to Agent Unity Environment Participant PC Agent Server Images from Agent’s point of view, End signal, etc. Next Action HTTP Async. Com. ? Async. Conn.: Virtual Time Scheduler Neocortex: Feature Extractor Component Hippocampus: Experience Component Basal Ganglia: Q-net Component BriCA: Cognitive Architecture Framework ML modules connected based on WBRA Component Connection Setting
  34. 34. 2. Systems Requirement & Environment Building Recommende d Environment Development Environment How to run • CPU: Intel i5 or later • Memory: 8G or more • Download standard Unity IDE • Download Python-enabled IDE • Start the Python Agent with a command • Run with Unity IDE or with a shell script
  35. 35. 3. Products & supports offered from WBAI Products Supports • Unity Task Environment ➢ Mazes, etc. • Agent Server Source Code Documentation • Glossary • Architecture Summary • Modification for the hackathon • Environment Building • Q&A on Slack channels ➢ Advisory team for task & technical questions ➢ Real-time response is not guaranteed. ➢ FAQ to be compiled
  36. 36. Q-Learning To obtain Function Q to estimate the Value of Action a at State s; Approximate optimal Q Function by updating the following equation at each step. Q (s, L) = Low Q (s, R) = High Q (s, No-op) = Low s: State , a: Action (pressing button, etc) , r: reward (game score, etc) Preprocessed s: (self location, ball position)
  37. 37. Feature Extraction with DNN 37 End-to-end learning from input to estimation with a multi-layer neural network enables feature extraction for the task. No human feature design is necessary. (Common use of CNN in image processing) https: // deep-learning-49182466 Feature expression obtained by learning
  38. 38. DQN (Deep Q-Network) 38 The Q-function of Q-Learning is approximated with DNN By end-to-end learning with DNN, the value can be estimated directly from images. Value DEEP https: // _utokyo/deep-learning- 49182466
  39. 39. observation - RGB Image 277*277 - depth Image 32*32 odometry Velocity & angular velocity Episode end signal reward - bool - int value Unity Design – What is sent to the Agent
  40. 40. Switching Tasks - Tasks are designed for Unity scenes. - After achieving the task for a certain times, the Agent moves to the next task. - Environment waits for Agent’s action. Unity Design – Task Progress
  41. 41. Hippocampal I/O based on Connectome (Containing Hippocampus)
  42. 42. Agent Design [velocity, angular velocity] action value (0, 1 or 2)
  43. 43. BriCA Core and Timing BriCA Core: • Framework for constructing brain-inspired cognitive architecture • Can connect & integrate ML modules in an asynchronous manner. • The time & timing of ML module execution can be designed with its time scheduler. In this hackathon, as Unity and BriCA work synchronously, if the agent uses too much time, the system would not work smoothly.
  44. 44. Evaluation of the neuroscientific reality • Realization of various cognitive functions on hippocampus – E.g.: • The Agent must wait in reward areas for a moment. • The Agent may locate itself from landmark information without the information on wall color to obtain rewards. • Neuro-connectivity between hippocampus and other brain areas – The proposed model should be justified in the presentation. • Correspondence with neuronal activities of hippocampus and other brain areas – Activities of reproduced Place Cells, Grid Cells and Head Direction Cells will be evaluated.
  45. 45. H. Mizutani How to Participate
  46. 46. Timeline till the Hackathon • 07-23: 2nd Orientation • 08-08: Due Date for Registration – From the Registration Form https: // – Please make “WBAhackathon2017” folder in your team Github. • 08-10: Acceptance notice • 08-18: Sample Code to be published • 08-19: Sample Code orientation & presentation by participants • 09-16 – Opening: 10h JST Talk by Kenji Doya (11h) Reception in the evening (talking on the hippocampus) • 09-18 – End of work: Noon – Review, Presentation, and Awarding: till 17h
  47. 47. The Third WBA Hackathon Summary  Date: September 16th, 17th, and 18th, 2017  Venue: Φ Café, Tokyo, Japan  Cost  Admission Free  Transportation, Lodging & Reception fees will be paid by the participants. Students may receive financial aids (to be explained below). Remote sites can be set up.
  48. 48. Lodging Participants (up to 20 persons) can stay at a hotel reserved by WBAI for the evenings of September 15th, 16th, and 17th. In case you’d like to stay in the hotel (for 4,290JPY per person/night), please inform us from the registration form.
  49. 49. Other information • Computation Environment – BYOD (Bring your own device) • Making your code public We ask you to publish the code and presentation material you made for the hackathon for open development of WBA under: – WBAI Contributor Agreement – Apache License (Version 2.0) on Github with README. • Financial aids for students Students meeting all the following conditions will be eligible for financial aids with regard to travel and lodging expenses up to JPY65,000/person. – You should fully participate in the hackathon at Φ Cafe on September 16th, 17th, and 18th. – Your work at the hackathon with a Readme file should be published on GitHub.