Analysis, comparisons and proposals of AI/ML benchmarks and competitions. Lessons learnt.
Theoretical or experimental accounts of the space of tasks, abilities and their dependencies.
Tasks and methods for evaluating: transfer learning, cognitive growth, development, cumulative learning, structural self-modification and self-programming.
Conceptualisations and definitions of generality or abstraction in AI / ML systems.
Unified theories for evaluating intelligence and other cognitive abilities, independently of the kind of subject (humans, animals or machines): universal psychometrics.
Evaluation of conversational bots, dialogue systems and personal assistants.
Evaluation of common sense, reasoning, understanding, causal relations.
Evaluation of multi-agent systems in competitive and cooperative scenarios, evaluation of teams, approaches from game theory.
Better understanding of the characterisation of task requirements and difficulty (energy, time, trials needed...), beyond algorithmic complexity. Item generation. Item Response Theory (IRT).
Evaluation of AI systems using generalised cognitive tests for humans. Computer models taking IQ tests. Psychometric AI.
Assessment of replicability, reproducibility and openness in AI / ML systems.
Evaluation methods for multiresolutional perception in AI systems and agents. Analysis of progress scenarios, AI progress forecasting, associated risks.
•Analysis of requirements for autonomy and generality
•Design proposals for cognitive architectures targeting generality and/or autonomy
•Complex layered networked systems and architectures
•Synergies between AI approaches
•Integration of top-down and bottom-up approaches (e.g. logic-based and neural-inspired)
•Emergence of (symbolic) logic from neural networks
•New programming languages relevant to generality and autonomy
•New methodologies relevant to generality and autonomy
•New architectural principles relevant to generality and autonomy
•Complex (e.g. layered, hierarchical or recursive) network architectures for generality and autonomy
•New theoretical insights relevant to generality and autonomy
•Motivation (intrinsic, extrinsic) for enabling autonomous behavior selection and learning
•Analysis of the potential and limitations of existing approaches
•Methods to achieve general ((super)human-like) performance
•Methods for epigenetic development
•Baby machines and experience-based, continuous, online learning
•Seed-based programming and self-programming
•Education for systems with general intelligence and high levels of autonomy
•Understanding and comprehension
•Reasoning and common-sense
•Acquisition of causal models
•Cumulative knowledge acquisition
•Curiosity, emotion and motivation for enabling autonomous behavior and knowledge acquisition
•Meta-planning, reflection and self-improvement
•Principles of swarm intelligence for generality and autonomy