The document describes the Whole Brain Architecture (WBA) prize that will be awarded at the Animal-AI Olympics competition on October 4, 2019. The WBA prize of $4,000 will be given to the most biologically plausible entry as determined by an independent review panel using criteria of biological plausibility, generality, and computational simplicity. Applicants for the WBA prize must submit a 1,000-1,500 word technical summary detailing how their methodology corresponds to brain architecture and mechanisms, as well as providing source code for the reviewers.

1. １６：００－１７：３０
Hiroshi Yamakawa
WBA prize
at Animal AI Olympics
4 Oct. 2019“Animal-AI Olympics” Kick-off meetup

2. Whole Brain Architecture approach
“Animal-AI Olympics” Kick-off meetup #2
‘to create a human-like AGI by learning
from the architecture of the entire brain’
To reach AGI, we start from mimicking high-level architectures of the
brain, and gradually introduce details as needed
2

3. The WBA (Whole Brain Architecture) Prize
The WBA Prize will be awarded to the
most biologically plausible entry
• US$4,000 grant (from WBAI)
• Up to US$1,000 travel grant to speak
at NeurIPS 2019 (from the organizer)
“Animal-AI Olympics” Kick-off meetup #23
https://wba-initiative.org/en/3545/

4. The WBA Prize: Application
The Summary
You must write your team ID and the date of application.
Detail how your methodology with a particular focus on the ways in which
it is biologically plausible (1,000 ～ 1,500 English words.)
More specifically,:
• Structural correspondence between your architecture and brain
architecture, if any (a block diagram would help).
• Mechanisms being used that are similar to those found in the brain,
• If you have modeled your solution on certain animals (including
humans).
• Include details about how you think your approach can help us
understand biological intelligence.
Source Code
We appreciate code readable for the reviewers
“Animal-AI Olympics” Kick-off meetup #24
https://wba-initiative.org/en/3545/

5. The WBA Prize: Review Process
• An independent panel of experts will be
convened for review.
• Biological plausibility as well as
generality and the simplicity of the code
will be appreciated (see the GPS
criteria).
• Biological plausibility will be evaluated
by your summary. Generality will be
evaluated from your competition record.
The code will be used to evaluate the
simplicity and to validate the summary.
“Animal-AI Olympics” Kick-off meetup #25
https://wba-initiative.org/en/3545/

6. Functionally
General
Biologically
Plausible
Computationally
Simple
GPS criteria for evaluating brain-inspired AGIs
“Animal-AI Olympics” Kick-off meetup #26
https://wba-initiative.org/3516/
Can the system solve
various tasks?
Evaluate the number of
successfully executed tasks
out of the presented tasks.
Is the mechanism similar to that of
the brain?
Evaluate whether the presented design
or sample code is used. Points are added
for additional biological plausibility.
Is the mechanism composed of
reusable functional modules?
Evaluate that multiple tasks are
accomplished by fewer processing
units.

7. Judging Committee
• Expert of Cognitive Architecture
• Expert of Computational Neuroscience
• Expert of Reinforcement learning
• Etc.
“Animal-AI Olympics” Kick-off meetup #27

8. Organizing Committee of AAO
“Animal-AI Olympics” Kick-off meetup #28

9. 9
José Hernández-Orallo
• Valencia Institute of Technology
– -Information system
– -Computational
• 4 books and over 100 articles and
papers on artificial intelligence,
machine learning, data mining,
cognitive science, and information
systems
• Algorithm Information Theory for
Artificial Intelligence Test
Development
“Animal-AI Olympics” Kick-off meetup #2

10. Interview on YouTube
“Animal-AI Olympics” Kick-off meetup #210
What is intelligence? In «The Measure of All Minds»,
José Hernández-Orallo reflects on how to measure
human, animal and artificial intelligence.
The book aims to create links between the evaluation
of artificial intelligence, very little developed, and a
centuries-old tradition of evaluation of human and
animal intelligence.
出典：https://www.youtube.com/watch?v=Vk0gKu3OwvU

11. Why general test is necessary?
“Animal-AI Olympics” Kick-off meetup #211
Heterogeneity
Homo
sapiens
Animal
Kingdom
Possible
Organisms
Machine
Kingdom
Universal
Psychometrics
Added:
- Cognitively enhanced organisms
- Biologically enhaced computers
Anthroposen (Niocene) extinction-100 times the creatures of
nature are extinct.
On the other hand, there is a reverse explosion. Computers.

12. Extension of traditional approach ← Negative
“Animal-AI Olympics” Kick-off meetup #212
Metrological psychology, comparative psychology, AI evaluation
• A specific group is subject to evaluation
– Unable to evaluate the ability of non-target groups
• comparison of humans and animals
• Anthropocentric
– Estimates of other groups are only estimates
(comparison between human and AI)
• A powerful falsification tool

13. 13
Concept of Difficulty
– Quantitative psychology: importance in
item response theory
– Logarithm of the number of calculation
steps required to solve the problem
• Levin’s Universal search
– http://www.scholarpedia.org/article/Universal_search
“Animal-AI Olympics” Kick-off meetup #2
Implications
• Interpretation given in the framework of
calculation theory
• Measurement unit: Logarithm of the number
of steps

14. Item Response Theory (IRT)
“Animal-AI Olympics” Kick-off meetup #214
• θ: Human parameter
• i: test item
• b: difficulty
• a : discrimination
• c : pseudo-guessing
IRT is a way to analyze responses to item (test or
questionnaires) with the goal of improving
measurement accuracy and reliability.
Probability of correct answer for item j
Figure 1: Example of 3PL IRF, with dotted lines overlaid to demonstrate parameters. (Wikipedia)

15. Common procedure to build a test
“Animal-AI Olympics” Kick-off meetup #215
1. Choose a range of items
2.Analyse item difficulty
3. Designing test for appropriate difficulty
4. Defining distribution of difficulty
5. Appropriate interface for the agents
6. Draw agent response curve and drive indicator

16. “Animal-AI Olympics” Kick-off meetup #216
The WBA award requires
an additional technical summary
and code submission.
However, it is a competition section
that has spaces to appealing
your team’s unique ideas.
Please challenge by all means.