1. Hierarchy is common in nature and engineering systems as a way to organize complex systems and achieve cooperation between subsystems. 2. General principles of hierarchy formation include order increasing over time through multiplication, centralization, differentiation and stronger interactions between levels. 3. Hierarchical systems evolve through self-organization and by developing simple systems into more complex ones over long periods, as seen in the evolution of organisms, societies, neural networks, genetic algorithms and other AI systems.

1. Hierarchical Learning in AI Notes 1 General Problem Jie Bao AI Lab, Iowa State University [email_address]

2. Content of whole series Hierarchy and any problem in general information system HL in GA Delicate structure in coding string Behavior Learning Sociobiology and ecology competition HL in NN Modular learning Ensemble learning Hybrid learning HL in MAS(MultiAgent System) Society entity: an agent in the hierarchy Society relationship: their languages

3. Why hierarchy? Nature system are mostly hierarchical system “ Divide and conquer” in Engineering The power and stableness from cooperation of subsystems Easier to design and implement

4. How to hierarchy? System theory: summation( all parts) < whole system The biologically hierarchical system are formed by self-organization Start from simple system Evolve from combine simple system to complex system The structural hierarchy is usually the result of evolutionary history

5. General Hierarchy Formation Direction: Order increasing Grow Multiplication Accelerate Interaction Higher level hierarchy appears! Differentiation Centralization

6. Three kinds of order-increasing system with Hierarchy They are all order-increasing They are derived from the former one They all have hierarchical structure Many theory and algorithm can be borrowed between the science of them ! General Hierarchy Formation Organism Society Automata

7. General Hierarchy Formation System is in the inner development of one level, for example, form ape to human. System search for favorable position on “niche” space, and the moving progress is called “Evolution”. In this progress, system tends to be more ordered because only more ordered system can have higher energy-consuming efficiency and win in the competition. When the structure of system block in the way to higher order, hierarchy will be appeared. In this stage, a new high-level system “bursts out”, such as society organized by human. Under the new organization, the system can have higher energy-consuming efficiency or energy-occupation ability, to further increase its order.

8. Hierarchical Structure(1) --Organism Body System Organ Tissue Cell Cell organelle Protein , nucleic acid ……

9. Hierarchical Structure(2) --Society Whole human world Nation State, Province City, County, Shire Community, Village Kin, Family Person

10. Hierarchical Structure(3) --Automata Cyber space: internet Local area network Terminal (both software and hardware) Module ( eg. CPU, operating system) Sub-module (eg. ALU, disk system) Smaller module, (eg. Adder, a interrupt service routine) Bit operation(eg. Gate )

11. Hierarchical Evolution(1) --Organism Little molecule-> big molecule: 3.5 billion y Molecule -> cell (2 billion y?) Prokaryote -> Eukaryotic (? billion y ) Cell -> Multi-cellular (0.6 billion y) Multi-cellular -> society ( for human, 4 million y, for insects, 0.2 billion y?) Society -> Gaia Cell ( now )

12. Hierarchical Evolution(2) --Society Vassals and tribes in China 1 221BC 10 400BC 140 700BC 800 1000BC 3,000 1600BC 10,000 2000BC

13. Hierarchical Evolution(3) --Automata Operation system: Windows CP/L(?K) : late 1970s DOS(1M): more interrupts Windows 3.1(15M): GUI Windows 95(100M): multi-media IE, Plus, DirectX, ActiveX : Windows 98 (200M) Windows 2000(1G) : various fanciness

14. Hierarchy in AI: simulate the nature AI system Immune system Molecule Cell Multi-cell Society Hierarchy Evolution Gene neural network

15. The way to hierarchy(1): organism Mitosis and amitosis The evolution of neural system Differentiation of Tissue Organ Isolated specie becomes “living fossil” Species are generated quicker and quicker Growth of individual; the average body volume increase in evolution The physical freedom of Molecule and cell are decreasing ( less entropy) Organism Multiplication Centralization Differentiation Hierarchy interaction Accelerate Growth Order increasing

16. The way to in hierarchy(2): Society Culture split, language pedigree ； colonizztion Government and international organization More detailed social professional work From individual, group, tribe to nation and international society Open society are developed quicker than isolated society The acceleration in social development Growth of total social economy Organization degree are increasing Society Multiplication Centralization Differentiation Hierarchy interaction Accelerate Growth Order increasing

17. The way to in hierarchy(3): OS The replication of progress and virus. Central part controls all processes modules in OS are divided more detailedly The SDK of OS is composed by many application API Message and signal between progress Exponential growth of OS size Increase of OS size Process and operating system Multiplication Centralization Differentiation Hierarchy interaction Accelerate Growth Order increasing

18. The way to hierarchy(4): WAN ? Formation of Portal, manage center, service center More and more different kind of websites WAN, LAN, terminal Protocol and messages Rapid almost exponent growth The extend of network size From unrestricted to be controlled and managed Wide area network Multiplication Centralization Differentiation Hierarchy interaction Accelerate Growth Order increasing

19. The way to hierarchy(5): NN ? ? From less structured to complex structured, such as layer Group network with complex behavior by neurons Weights ? (evolutionary neural network) After training, input can converge to some attractors Neural Network Multiplication Centralization Differentiation Hierarchy interaction Accelerate Growth Order increasing

20. The way to hierarchy(6): GA Selection , crossover,mutation and new population From individuals to population Fitness Schema theorem Genetic Algorithm Multiplication Centralization Differentiation Hierarchy interaction Accelerate Growth Order increasing

21. The way to hierarchy(7): MAS Simple agent and complex agent society Agent Language Converge to equilibrium point Multi-agent system Multiplication Centralization Differentiation Hierarchy interaction Accelerate Growth Order increasing

22. Hierarchical AI System: Social Computational intelligence: neural network, multi-agent system (MAS), evolution computation and artificial immune system. The basic idea of computational intelligence is “social computation”, that’s, complex intelligence can be obtained self-organizingly by simple intelligence individuals under some simple social rules (including competition, cooperation and so on).

23. Hierarchical AI System: Ecological Such a “social computation” system can be regarded as an hierarchical artificial ecology system, which has similar property and development to nature ecology system. So basic laws of computational intelligence can be regarded as “general ecology”.

24. Hierarchical AI System: Self-organization Some general laws in computational intelligence, such as order-increasing; information interchange; hierarchy structure and development; progressive centralization; progressive Differentiation, are in fact general properties of a kind of self-organizing systems.

25. Hierarchical AI System: Self-organization(2) Possible system state space in early stage Possible system state space in early stage

26. Hierarchical AI System: Self-organization(3) limited growth, stable or slow development. break through developing obstacle by hierarchy with higher order but less freedom degree. More efficient in energy using and can use more energy that lower-level system can’t utilize abundant resource stage , exponential growth Order

27. Hierarchical AI System: interdisciplinary Therefore, the development of hierarchical learning in computational intelligence, especially the hierarchical MAS, is closely related to the development of life sciences and social sciences. Neural Network (top-down) and MAS (bottom-up) are integrated methods to carry out the research in practice.

28. Hierarchical learning in Neural Network Ensemble learning Modular learning Hybrid learning

29. Hierarchical learning in GA Behavior evolution: hierarchical structure in nonlinear coding (tree) Diversification of population Multi-level selection

30. Hierarchical learning in MAS Evolution of cooperation by Reinforcement learning Hierarchal Markov game: game between groups Hierarchical entity: agent and their society Hierarchical relationship: language