Recommended
More Related Content
Similar to Intelligence Via Compression of Information
Similar to Intelligence Via Compression of Information (20)
Recently uploaded
Recently uploaded (20)
Intelligence Via Compression of Information
- 2. OVERVIEW ■ Substantial evidence for the importance of IC in human learning, perception, and thinking. ■ Outline of the SP Theory of Intelligence (SPTI), and its realisation in the SP Computer Model. ■ Within the SPTI: the powerful concept of SP- Multiple-Alignment. ■ Mathematics as a set of techniques for IC and their application. ■ A New Mathematics via the merging of mathematics with the SPTI, and its potential benefits are large. CognitionResearch.org
- 3. IC IN THE NERVOUS SYSTEM AND THE BRAIN (1) ■ Much more information falls on the retina in each second than can be transmitted along the optic nerve to the brain. ■ Without IC, there would be much wastage. ■ It is unlikely that natural selection would tolerate so much wastage. ■ There is good evidence that the retina performs IC―via ‘lateral inhibition’. CognitionResearch.org
- 4. IC IN THE NS AND THE BRAIN (2) CognitionResearch.org ■ Unlike an old-style cine camera, our brains merge or ‘unify’ successive views of a scene into one.
- 5. IC THE IN NS AND THE BRAIN (3) ■ ‘Chunking-with-codes’ is widely used in natural languages. ■ For example: ‘TFEU’ is a relatively short ‘code’ for ‘Treaty on the Functioning of the European Union’ which is a relatively large ‘chunk’ of information. ■ In natural languages, every noun, verb, adjective, and adverb is a code for a relatively large chunk of information, which is the meaning of the word. ■ For example, the word ‘table’ is a code and its meaning is the chunk. CognitionResearch.org
- 6. THE SP THEORY OF INTELLIGENCE (SPTI) CognitionResearch.org
- 7. THE CONCEPT OF SP-MULTIPLE-ALIGNMENT ■ The main part of the SPTI Is the powerful concept of SP- Multiple-Alignment. ■ The SP-Multiple-Alignment concept is largely responsible for: ■ The versatility of the SPTI in modelling diverse aspect of intelligence. ■ The versatility of the SPTI in compressing diverse kinds of information. ■ This is an important NEW DISCOVERY: how a relatively simple mechanism can model diverse aspects of intelligence. CognitionResearch.org
- 8. HOW THE SP-MULTIPLE-ALIGNMENT CONCEPT ORIGINATED CognitionResearch.org ■ Find the best full and partial matches between two sequences of symbols. ■ Look at ‘multiple sequence alignment’ from bioinformatics (below). ■ Then eureka: seeing that multiple sequence alignment might be adapted to model intelligence. The adapted version is called SP-Multiple- Alignment.
- 9. DEVELOPMENT OF THE SP-MULTIPLE- ALIGNMENT CONCEPT ■ Several years’ work were needed to develop the SP-Multiple-Alignment concept, and explore potential applications. ■ Hundreds of versions of the SP- Multiple-Alignment concept were developed and tested in the SP Computer Model, across varied applications. CognitionResearch.org
- 10. AN SP-MULTIPLE-ALIGNMENT EXAMPLE CognitionResearch.org ■ Row 0 shows a New SP-pattern. Each of the other rows contains one Old SP-pattern. ■ A ‘good’ SP-Multiple-Alignment is one which encodes the New SP-pattern economically in terms of the Old SP-patterns.
- 11. STRENGTHS OF THE SP-MULTIPLE- ALIGNMENT CONCEPT (1) ■ SP-Multiple-Alignments may model several aspects of intelligence, including: unsupervised learning, parsing, pattern recognition, information retrieval, probabilistic reasoning, problem solving, and more. ■ There are other strengths in: the management of big data, medical diagnosis, sustainability, transparency of structures and processes, computer vision, and more. CognitionResearch.org
- 12. STRENGTHS OF THE SP-MULTIPLE- ALIGNMENT CONCEPT (2) CognitionResearch.org ■ The SP-Multiple-Alignment concept provides most of the versatility of the SP Theory of Intelligence in AI and beyond. ■ It is no exaggeration to say that: SP-Multiple-Alignment has the potential to be as significant for an understanding of intelligence as is DNA for an understanding of biology. SP-Multiple-Alignment may prove to be the ‘double-helix’ of intelligence!
- 13. MATHEMATICS AND IC (1) ■ In view of: 1 The importance of IC in human intelligence AND, 2 The fact that mathematics is the product of human brains AND, 3 The fact that mathematics has been created as an aid to human thinking … ■ We should not be surprised to find that IC is central in the structure and workings of mathematics. CognitionResearch.org
- 14. MATHEMATICS AND IC (2) ■ IC via Run-Length Coding means reducing a sequence of two or more identical patterns to one. ■ In mathematics, run-length coding may be seen in: ■ Addition: 3 + 1 + 1 + 1 + 1 + 1 3 + 5. ■ Multiplication: 4 + 4 + 4 + 4 + 4 + 4 4 x 6. ■ Power: 5 x 5 x 5 x 5 x 5 x 5 x 5 x 5 58. CognitionResearch.org
- 15. CHUNKING-WITH-CODES IN THE NUMBER SYSTEM ■ The unary number 1,1,1,1,1, is a chunk, and 5 is its relatively short code. ■ The unary number 1,1,1,1,1,1,1,1,1,1, is a chunk and 10 is its relatively short code. ■ The unary number equivalent to ‘eight instances of ten’ is a chunk and 80 is its relatively short code. ■ The unary number equivalent to ‘ten instances of ten’ is a chunk and 100 is its relatively short code. ■ And so on, recursively. CognitionResearch.org
- 16. CREATION OF A NEW MATHEMATICS ■ Since IC is fundamental in the SPTI, ■ And since IC is fundament in mathematics. ■ A New Mathematics may be created via the integration of mathematics with the SPTI. CognitionResearch.org
- 17. POTENTIAL BENEFITS OF A NEW MATHEMATICS ■ The New Mathematics will gain from: ■ More than 2000 years of thinking about mathematics. ■ Artificial intelligence via the SPTI. ■ The NM is potentially a powerful system for the representation and processing of knowledge in science and beyond. CognitionResearch.org
- 18. SOME POTENTIAL BENEFITS OF THE NEW MATHEMATICS ■ Synergy between two areas of strength (maths and AI). ■ New techniques for IC in mathematics and science. ■ The learning, representation and processing of structures in two, three, and four dimensions. ■ The representation and processing of knowledge that is more compatible with the way people naturally think. ■ Quantitative evaluation and comparison of scientific theories ■ Automatic (or semi-auto) creation of scientific theories. ■ The integration of, for example, relativity and quantum mechanics. CognitionResearch.org
- 19. FURTHER INFORMATION ■ www.cognitionresearch.org/sp.htm. The next two items can be seen under the heading ‘Tutorial’; ■ These slides may be downloaded via tinyurl.com/2vywha93; ■ Papers about this research via tinyurl.com/mpwdusrx; ■ Email: jgw@cognitionresearch.org; ■ Phone: +44 (0) 7746 290775. ■ Many thanks for listening! CognitionResearch.org
Editor's Notes
- INTELLIGENCE VIA COMPRESSION OF INFORMATION Hello, I’m Gerry Wolff. This is a talk about the idea that, to a large extent, the concept of ‘ intelligence’ may be understood as a process of compressing information. For the sake of brevity, the expression ‘information compression’ will be referred to as ‘IC’. My interest was sparked originally by excellent lectures by neuroscientist Horace Barlow about the importance of IC in the workings of brains and nervous systems. As early as 1969, Barlow wrote: ‘...~the operations needed to find a less redundant code have a rather fascinating similarity to the task of answering an intelligence test, finding an appropriate scientific concept, or other exercises in the use of inductive reasoning. Thus, redundancy reduction may lead one towards understanding something about the organization of memory and intelligence, as well as pattern recognition and discrimination.’ In this quotation, the expression ‘find a less redundant code’ and the expression ‘redundancy reduction ‘ are the essentials of IC. In short, Barlow put his finger on the principle that, to a large extent, human intelligence means IC. Notice that this means ***all aspects of intelligence***, not just learning, which is often assumed to be the only aspect of intelligence where IC may be relevant. The generality of IC as the basis for ***all aspects of intelligence*** is central in the research to be described. What I’ll be talking about is new thinking in these areas, in research which has kept me busy for more than 20 years. There is more detail in published papers about the research which may be downloaded via a link which I’ll be showing at the end of the talk.
- OVERVIEW Here is an overview of what the talk is about. The main topics in this talk are as follows. I’ll first describe some of the evidence, which is quite large, for the importance of IC in human learning, perception, and thinking. Then I’ll outline the SP Theory of Intelligence and its realisation in the SP Computer Model. In the rest of the talk, the name ‘SP Theory of Intelligence’ will be shortened to ‘SPTI’. A very important part of the SPTI is the powerful concept of SP-Multiple-Alignment. This is largely responsible for the strengths of the SPTI in AI and beyond. An unexpected spin-off from the SPTI is the idea that much of mathematics, perhaps all of it, may be seen as a set of techniques for the compression of information, and their application. Because IC is central in mathematics and in the SPTI, we may create a ***New Mathematics*** by integrating mathematics with the SPTI. That kind of New Mathematics is likely to be a powerful system for the representation and processing of knowledge in science and beyond.
- IC IN THE NERVOUS SYSTEM AND THE BRAIN (1) This and the following two slides describe some of the abundant evidence that much of human perception, learning, and thinking is achieved via IC. In our first example, Horace Barlow pointed out that much more information falls on the retina in each second than can be transmitted along the optic nerve to the brain. That means that, without IC, there would be much wastage of information. It is unlikely that natural selection would tolerate so much wastage, so there is likely to be compression of information. Research has confirmed that the retina does indeed compress visual information, via the process called ‘lateral inhibition’.
- IC IN THE NS AND THE BRAIN (2) Here is another example of IC in neural functioning. If we are looking at a view and close our eyes for a moment, and then open them again, the ‘before’ and ‘after’ views are fused into a single view: On the slide, the before and after views are on the left and right, and the black rectangle shows where we close our eyes for a moment. The unified view is shown at the bottom. Compressing information like this is quite different from how an old-style cine camera creates many frames for one view.
- IC IN BRAIN FUNCTION (3) Here’s a third example of IC in the workings of our brains. A technique called chunking-with-codes means using a relatively small name or ‘code’ for a relatively large ‘chunk’ of information. Chunking-with-codes is a widely-used technique for IC, both in IT and in ordinary writing where, for example, we use the relatively small code ‘TFEU’ as a shorthand for ‘Treaty on the Functioning of the European Union’, which is a relatively large chunk of information. A little reflection shows that every noun, verb, adjective and adverb in natural language is that kind of code, and its meaning is a relatively large chunk of information. For example, the word ‘table’ is a relatively short name or code for the relatively large chunk of information which is the meaning of the word ‘table’.
- THE SP THEORY OF INTELLIGENCE Here is a schematic representation of the SPTI and its realisation in the SP Computer Model. The whole system is like a person who takes in ‘New’ information via his or her ‘senses’ and compresses it to create ‘Old’ information which is stored in its ‘brain’. In summary, ***ALL KINDS OF PROCESSING IN THE SPTI IS ACHIEVED VIA THE COMPRESSION OF INFORMATION***.
- THE CONCEPT OF OF SP-MULTIPLE-ALIGNMENT This slide introduces the very important concept of SP-Multiple-Alignment. As it says on the slide: The main part of the SPTI Is the powerful concept of ***SP-MULTIPLE-ALIGNMENT***. The SP-Multiple-Alignment concept is largely responsible for two things: First, the versatility of the SPTI in modelling diverse aspect of intelligence. Second, the versatility of the SPTI in compressing diverse kinds of information. The SP-Multiple-Alignment concept is an important ***NEW DISCOVERY***. It is important because it shows how a relatively simple mechanism can model diverse aspects of intelligence.
- HOW THE SP-MULTIPLE-ALIGNMENT CONCEPT WAS DISCOVERED. This slide is about how the SP-Multiple-Alignment concept was discovered. Preliminary work tried to get a grip on the problem of developing a general theory of intelligence. The first step was to ***DEVELOP A PROGRAM TO FIND THE BEST FULL AND PARTIAL MATCHES BETWEEN TWO SEQUENCES OF SYMBOLS***. The reason for working on this problem was because something like that seemed to be the basis for various things we do in learning, perception, and so on. Finding full and partial matches between two sequences led to an interest in the concept of ‘multiple sequence alignment’ from bioinformatics.. The general idea with multiple sequence alignments is to lay two or more DNA sequences next to each other, and then by ‘stretching’ sequences in a computer, to bring matching symbols into line. A good result is when many symbols have been brought into line. After that, there was a ***EUREKA*** moment when I began to see that the multiple sequence alignment idea might be adapted to model several different aspects of intelligence. The adapted version is called ***SP-MULTIPLE-ALIGNMENT***.
- DEVELOPMENT OF THE SP-MULTIPLE-ALIGNMENT CONCEPT This slide emphasises the amount of work that was needed to develop the SP-Multiple-Alignment concept. Firstly, ***SEVERAL YEARS WORK*** were needed to develop the SP-Multiple-Alignment concept, and to explore its range of possible applications. And secondly, ***HUNDREDS*** of different versions of the concept were developed and tested in the SP Computer Model, across several areas of application.
- AN SP-MULTIPLE-ALIGNMENT EXAMPLE This slide shows how an SP-Multiple-Alignment may model the parsing of a sentence in terms of grammatical categories and words. In the example, the sentence ‘t w o k i t t e n s p l a y’, which is shown in row 0, is a NEW SP-pattern which has been received via the system’s senses. Each of rows 1 to 8 contains ONE SP-pattern from a repository of several different OLD SP-patterns. In this example, each Old SP-pattern in the SP-Multiple-Alignment describes a grammatical structure or a word. In this example, the whole SP-Multiple-Alignment describes how the sentence is composed of a noun phrase, ‘t w o k I t t e n s’ followed by a verb, ‘p l a y’, and these parts are broken down into smaller parts. The SP-Multiple-Alignment structure is built up in stages, starting with the sentence and the store of Old SP-patterns. Heuristic methods are needed to achieve a good balance between accuracy and speed. In general, ***A ‘GOOD’ SP-MULTIPLE-ALIGNMENT IS ONE IN WHICH THE NEW SP-PATTERN MAY BE ENCODED ECONOMICALLY IN TERMS OF THE OLD SP-PATTERNS***.
- STRENGTHS OF THE SP-MULTIPLE-ALIGNMENT CONCEPT (1) This slide summarises what’s good about the concept of SP-Multiple-Alignment. Because the SP-Multiple-Alignment concept is the bulk of the SPTI, this slide, to a large extent, also describes what’s good about the SPTI. Here’s a summary of the strengths of the SP-Multiple-Alignment concept in AI and beyond: First, the SP-Multiple-Alignment concept may model several different aspects of intelligence, as shown on the slide. These include: unsupervised learning, representing the structure of natural language, pattern recognition, information retrieval, several kinds of probabilistic reasoning, problem solving, and the seamless integration of diverse aspects of intelligence, in any combination. Second, the SP-Multiple-Alignment concept has potential in other areas, as shown on the slide. The main ones to date are : the management of big data, medical diagnosis, sustainability, transparency of structures and processes, computer vision, and more. In addition to its strengths in AI, the SP-Multiple-Alignment concept, as I mentioned earlier, is a powerful means of compressing diverse kinds of information.
- STRENGTHS OF THE SP-MULTIPLE-ALIGNMENT CONCEPT (2) This slide summarises the strengths of the SP-Multiple-Alignment concept: The SP-Multiple-alignment concept provides most of the versatility of the SPTI in AI and beyond. It is no exaggeration to say that: ***SP-MULTIPLE-ALIGNMENT IS AN IMPORTANT NEW DISCOVERY WHICH HAS THE POTENTIAL TO BE AS SIGNIFICANT FOR AN UNDERSTANDING OF INTELLIGENCE AS IS DNA FOR AN UNDERSTANDING OF BIOLOGY***. ***SP-MULTIPLE-ALIGNMENT MAY PROVE TO BE THE ‘DOUBLE-HELIX’ OF INTELLIGENCE!***
- MATHEMATICS AND IC (1) Here’s a broad-brush view of why we should expect IC to be important in the foundations of mathematics. In view of the importance of IC in human intelligence AND, In view of the fact that mathematics is the product of human brains AND, In view of the fact that mathematics has been created as an aid to human thinking, ***WE SHOULD NOT BE SURPRISED TO FIND THAT IC IS CENTRAL IN THE STRUCTURES AND WORKINGS OF MATHEMATICS.***
- MATHEMATICS AND IC (2) This slide describes how some parts of mathematics may be understood in terms of the run-length coding technique for IC. As it says on the slide: IC via Run-Length Coding means reducing a sequence of two or more identical patterns to one, In mathematics, IC via run-length coding may be seen in three widely-used techniques in arithmetic: IC may be seen in ADDITION, where, for example, 3 x 1 x 1 x 1 x 1 x 1 is normally shortened to 3 x 5. And IC may be seen in MULTIPLICATION where, for example, 4 + 4 + 4 + 4 + 4 + 4, is normally shortened to 4 x 6, And IC via run-length coding may be seen in THE POWER OF A NUMBER where, for example, 5 x 5 x 5 x 5 x 5 x 5 x 5 x 5 is normally shortened to 58.
- MATHS AND IC (3) CHUNKING-WITH-CODES IN THE NUMBER SYSTEM As I mentioned earlier, the chunking-with-codes technique for IC means using a relatively short name or code, such as ‘TFEU’, for a relatively large chunk of information, such ‘’Treaty on the Functioning of the European Union”. This slide describes how the number system may be understood in terms of the chunking-with-codes technique for IC. As it says on the slide: * The unary number 1,1,1,1,1, is a chunk, and the digit 5 is its relatively short code. * The unary number 1,1,1,1,1,1,1,1,1,1, is a chunk and the digits 1 with 0 is its relatively short code, where ‘1’ means that there is one instance of ten, and ‘0’ means there are zero ones. * The unary number equivalent to ‘eight instances of ten’ is a chunk and 8 with 0 is its relatively short code, where ‘8’ means that there are eight instance of ten, and ‘0’ means that there are zero ones. * The unary number equivalent to ‘ten instances of ten’ is a chunk and the digits 1 with 0 and 0 is its relatively short code, where ‘1’ means that there is one instance of a hundred, with zero instances of ten and zero instances of one. * And so on, recursively. The general idea is that chunking-with-codes may be applied recursively in the number system . Carlo Rovelli has written about how this idea was invented by Archimedes. At the time, the Greek number system could not represent any number greater than 10,000, which was called a ‘myriad’. Archimedes simply made a myriad the base of a recursive number system, so that, with numbers like ‘a myriad of myriads’, or ‘a myriad of myriads’of myriads’, and so on, he could represent numbers of any size.
- CREATION OF A NEW MATHEMATICS Here are the main reasons for creating a New Mathematics. Since IC is fundamental in the SPTI, And since IC is fundamental in mathematics … A NEW MATHEMATICS may be created via the integration of MATHEMATICS with the SPTI.
- POTENTIAL BENEFITS OF A NEW MATHEMATICS In summary, here are the main benefits to be expected from the New Mathematics. The New Mathematics will gain from: First, more than 2000 years of thinking about mathematics. And second, artificial intelligence via the SPTI. In general, THE NEW MATHEMATICS HAS POTENTIAL AS A POWERFUL SYSTEM FOR THE REPRESENTATION AND PROCESSING OF KNOWLEDGE IN SCIENCE AND BEYOND.
- In more detail, this slide summarises some of the potential benefits of the New Mathematics: First, we may expect a synergy between two areas of strength, namely mathematics and AI. Then, the New Mathematics is likely to introduce new techniques for IC into mathematics and science. Then, there is potential for the learning, representation and processing of structures in two, three, and four dimensions, where the fourth dimension is time in videos and the like. And there is potential for the representation and processing of knowledge that is more compatible with the way people naturally think. There is potential for the quantitative evaluation and comparison of scientific theories. And there is potential for the automatic or semi-automatic creation of scientific theories. And last, there is potential for the integration of, for example, relativity and quantum mechanics.
- FURTHER INFORMATION Here are some details relating to this talk. The first link will take you, via the heading ‘Tutorial’, to the two items below it, which are: First, a link to download these slides; And. Second, a link to download papers about this research. I will be happy to hear from people with questions or comments via either of the contact details shown on the slide. ***MANY THANKS FOR LISTENING!***