1.
International collaborations
● WBAI (Japan)
● Luria (New York)
● Numenta (co-authored Boosted RSM)
Understand animal intelligence / the brain
Improve machine intelligence
○ Independent Research Group
○ Interested in interaction of brain regions for
intelligent behaviour and decision making
Cerenaut
Supervise graduate students at
Monash
Founded in 2018, Publishing since 2012

2.
Working Memory - What is it?
Working memory is a short-term repository for task-relevant information that is critical for the successful
completion of complex tasks (Baddeley, 2003).
E.g. in a spatial working memory task, animals must hold in memory the location of food rewards to
navigate to those locations after a delay.

3.
Primates on DM2S
Figure reproduced from ‘Principles of
Neuroscience’ (Kandel et. al), reproduced from
Rainer, Asaad, and Miller 1998.

4.
Persistent activity in frontal cortex
Figure reproduced from ‘Principles
of Neuroscience’ (Kandel et. al)

5.
Neuroanatomy
What and Where:
● Converge at
Hippocampus.
● A Short Term Memory.
● With strong
projections to PFC.

6.
PFC <> BG <> Thalamus
Girard, Benoît & Tabareau, Nicolas & Berthoz, Alain & Slotine,
Jean-Jacques. (2006). Selective amplification using a contracting
model of the basal ganglia.
Midbrain
VTC, SNc
Trains itself and the actor
Figure reproduced from ‘Computational Cognitive
Neuroscience’ (O’Reilly, Frank et. al)
Via Thalamus
context

7.
Gating stripes
● Persistent neural activity through
two major mechanisms:
○ 1. Intrinsic membrane properties
○ 2. Recurrent connectivity
Figure reproduced from ‘Making Working Memory
Work’, 2016 (O’Reilly and Frank)

8.
DM2S/M2S/M2L
Inherits: ActiveVisionEnv
(Environment)
Config: game_name_env.json
Positional
Encoding
Retina
(DoG +/-
coding)
SuperiorColliculus
(track to position)
SparseAutoencoder
(Visual Cortex)
What
Where
PrefrontalCortex
MedialTemporalLobe
(Short term memory)
Agent
(Actor - BG)
Config:
stub_agent_x.json
Pretrained
network
RL Policy
module
Legend
Reward
AgentEnv (Environment)
Config: stub_env_x.json
Visual Path
Fovea Periph
Gaze
Choices
Data:
what-where
Critic (PVLV)
SparseAutoencoder
(Visual Cortex)
Observation
Action
Gaze position command
(absolute coordinates)
Gaze position
command
Retina
(DoG +/-
coding)
Gym
Environment
Fixed function
Pass-through
Visual Path
Naming:
Class Name
(function)
Delay

9.
Active Vision Fovea
Fovea: Can
recognise shapes,
but can’t see
context.
Periphery: Can see
changes, but can’t
recognise shapes.
Periphery

10.
Positional Encoding
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., ... & Polosukhin, I. (2017).
Attention is all you need. arXiv preprint arXiv:1706.03762.

11.
Software environment
Lots of good information on the Wiki →

12.
Software stack
PyGame
NumPy, PyTorch
OpenAI Gym: Gym.Env
Participant code
Ray, RL Lib
Ray RL Lib TorchModelV2
DM2S_Env
Agent
FiniteStateEnv
ActiveVisionEnv
PyGameEnv
Participant code
Environment Agent
AgentEnv
Task
agent.stubs.medial_temporal_lobe
agent.stubs.positional_encoder
agent.stubs.prefrontal_cortex
agent.stubs.superior_colliculus
agent.stubs.visual_path
Agent (brain) - pretrained RL Agent

13.
Competition
● CodaLab
● Evaluation process
● Preparing submission