Information theory


Published on

Published in: Education, Technology
1 Comment
No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Information theory

  1. 1. design theory. 6mercredi, 11 mai 2011
  2. 2. mercredi, 11 mai 2011
  3. 3. Information theorymercredi, 11 mai 2011
  4. 4. Information theory Why speak about information? Because when we have look at an object, when we evaluate and appreciate it, the relation which established between the object and us is an exchange of information.mercredi, 11 mai 2011
  5. 5. Information theory Humans are symbol-making creatures. We communicate by symbols -growls and grunts, hand signals, and drawings painted on cave walls in prehistoric times. Later we developed languages, associating sounds with ideas. Eventually Homo Sapiens developed writing, perhaps first symbols scratched on rocks, then written more permanently on tablets, papyrus, and paper. Today, we transmit symbols , coded digital signals of voice, graphics, video, and data around the world at close to the speed of light. We’re even sending signals into outer space in the hope of finding other symbol-creating species.mercredi, 11 mai 2011
  6. 6. Information theory The theory stems from pioneering work done by the American electrical engineer Claude E. Shannon, who published his seminal work in 1948. Since then, information theory has developed rapidly, affecting not only the design of communications systems but also such areas as automation, information science, psychology, linguistics, art and design.mercredi, 11 mai 2011
  7. 7. Claude E. Shannon Shannon was born in Petoskey, Michigan, on April 30, 1916. He graduated from the University of Michigan in 1936 with bachelors degrees in mathematics and electrical engineering. In 1940 he earned both a masters degree in electrical engineering and a Ph.D. in mathematics from the Massachusetts Institute of Technology (MIT). His work founded the subject of information theory. Claude Shannon died in February 2001.mercredi, 11 mai 2011
  8. 8. Information theory Understanding Information Theory Understanding Shannon’s basis of Information Theory, is not an easy matter. To get a high-level understanding of his theory, a few basic points should be made. In the first step, the message has to be put into some kind of symbolic representation – words, musical notes, icons, mathematical equations, or bits. When we write “Hello,” we encode a greeting. When we write a musical score, it’s the same thing – only we’re encoding sounds.mercredi, 11 mai 2011
  9. 9. Information theory Understanding Information Theory First, words are symbols to carry information between people. If one says to an American, “Let’s go!”, the command is immediately understood. But if we give the commands in Russian, “Pustim v xod!”, we only get a quizzical look. Russian is the wrong code for an American. Second, all communication involves three steps: Coding a message at its source - Transmitting the message through a communications channel, and Decoding the message at its destination.mercredi, 11 mai 2011
  10. 10. Information theory Understanding Information Theory In the first step, the message has to be put into some kind of symbolic representation – words, musical notes, icons, mathematical equations, or bits. For any code to be useful it has to be transmitted to someone or, in a computer’s case, to something. Transmission can be by voice, a letter, a billboard, a telephone conversation, a radio or television broadcast, or the now ubiquitous e-mail. At the destination, someone or something has to receive the symbols, and then decode them by matching them against his or her own body of information to extract the data.mercredi, 11 mai 2011
  11. 11. Information theory Understanding Information Theory Shannon describes the elements of communications system theory as: a source - encoder - channel - decoder - destination model. What his theory does is to replace each element in the model with a mathematical model that describes that element’s behavior within the system.mercredi, 11 mai 2011
  12. 12. Information theory The meaning of information? “Information” has a special meaning for Shannon. For years, people deliberately compressed telegraph messages by leaving certain words out, or sending key words that stood for longer messages, since costs were determined by the number of words sent. Yet people could easily read these abbreviated messages, since they supplied these predictable words, such “a” and “the.”mercredi, 11 mai 2011
  13. 13. Information theory The meaning of information? In the same vein, for Shannon, information is symbols that contain unpredictable news, like the sentence: “only infrmatn esentil to understandn mst b tranmitd.” The predictable symbols that we can leave out, which Shannon calls redundancy, are not really news.mercredi, 11 mai 2011
  14. 14. Information theory The meaning of information? Another example is coin flipping. Each time we flip a coin, we can transmit which way it lands, “heads” or “tails,” by transmitting a code of “zero” or “one.” But what if the coin has two “heads” and everyone knows it? Since there is no uncertainty concerning the outcome of a flip, no message need be sent at all. Although this view might seem like common sense today, it was not always so. Shannon made clear that uncertainty or unpredictability is the very commodity of communication.mercredi, 11 mai 2011
  15. 15. Information theory The meaning of information? The amount of information, or uncertainty, output by an information source is a measure of its entropy.mercredi, 11 mai 2011
  16. 16. mercredi, 11 mai 2011
  17. 17. Principles of information theory The basic elements of any general communications system include 1- a source of information which is a transmitting device that transforms the information or "message" into a form suitable for transmission by a particular means. 2- the means or channel over which the message is transmitted. 3- a receiving device which decodes the message back into some approximation of its original form. 4- the destination or intended recipient of the message. 5- a source of noise (i.e., interference or distortion) which changes the message in unpredictable ways during transmissionmercredi, 11 mai 2011
  18. 18. mercredi, 11 mai 2011
  19. 19. Principles of information theory It is important to note that "information" as understood in information theory has nothing to do with any inherent meaning in a message; it is rather a measure of the predictability and complexity of each transmitted message. The expected value of a transmission is referred to as the entropy, or average information of the set of messages (entropy: the amount of entropy is a measure of the disorder, or randomness, of a system.)mercredi, 11 mai 2011
  20. 20. mercredi, 11 mai 2011
  21. 21. Principles of information theory Suppose we have a device that can produce 3 symbols, A, B, or C. As we wait for the next symbol, we are uncertain as to which symbol it will produce. Once a symbol appears and we see it, our uncertainty decreases, and we remark that we have received some information. That is, information is a decrease in uncertainty. Information is the measure of the predictability and complexity of a message.mercredi, 11 mai 2011
  22. 22. Principles of information theory The individuals are determined by the messages of their environment; messages are complex forms. These messages have an elementary defined structure, as regards the ulterior reactions of the individuals, by psycho- physiologys properties of the receiver. Nearby immediate messages, there are messages taken away in the time or space that are restored to the environment by means of spatial charnel (transmissions) or temporal (recording). Messages are measured by a quantity of information, which is originality it is to say the unpredictability that they bring.mercredi, 11 mai 2011
  23. 23. mercredi, 11 mai 2011
  24. 24. Principles of information theory Information is a quantity essentially different from the meaning and independent of this one: a maximum message of information can seem devoid of sense if the individual is not able to decode it. In a general way, comprehensibility varies to the opposite way of the information.mercredi, 11 mai 2011
  25. 25. Principles of information theory One calls redundancy the repetition of elements within a message that prevents the failure of communication of information-is the greatest antidote to entropy Meaning is based on a convention set in principle common to the transmitter and to the receiver, it is not transported, it preexists potentially before the message. Information is in facto measure of the complexity of patterns proposed by the perception. Complexity alone is transported from the transmitter to the receiver it is exactly what is not present in the receivermercredi, 11 mai 2011
  26. 26. mercredi, 11 mai 2011
  27. 27. mercredi, 11 mai 2011
  28. 28. mercredi, 11 mai 2011
  29. 29. mercredi, 11 mai 2011
  30. 30. Information theory and Gestalt theory Any form is the expression of an unpredictable predictability which is measured by its degree of coherence The comprehensibility of a form is proportional in the redundancy of this one The message the most difficult to transmit is the one that does not contain any redundancy so any formmercredi, 11 mai 2011
  31. 31. Information theory and Gestalt theory The notion of complex forms covers that of super signs. Super signs are assemblies normalized of elements known beforehand. One of the most elementary temporal forms is the periodicity. There is perception of a periodicity as soon as it has expectation of a similar following event to those that already occurred. The continuance of a shape is only an aspect of periodicitymercredi, 11 mai 2011
  32. 32. Law of good continuation Humans tend to continue contours whenever the elements of the pattern establish an implied direction People tend to draw a good continuous linemercredi, 11 mai 2011
  33. 33. mercredi, 11 mai 2011
  34. 34. Law of Similarity Elements that look similar will be perceived as a part of the same form There seems to be a triangle in the squaremercredi, 11 mai 2011
  35. 35. Law of closure Humans tend to enclose a space by completing a contour and ignoring gaps in the figuremercredi, 11 mai 2011
  36. 36. mercredi, 11 mai 2011
  37. 37. Information theory & esthetic perception Abraham Moles 1920-1992 Abraham Moles was on the faculty at the University Louis Pasteur, Strasbourg, France, and Director of the Institute of Social Psychology of Communications. Moles was one of the first aestheticians to deal with cybernetic issues. Moles has published a number of books, including, in 1958, an influential book entitled “Information theory and et perception esthetic”.mercredi, 11 mai 2011
  38. 38. Information theory & esthetic perception In 1966 Abraham Moles, write Information Theory and Esthetic Perception (University of Illinois Press). Moles was dealing with such concepts as originality ("If a given message or event is certain, it teaches the receptor nothing and cannot modify his behavior. An unexpected event has by definition a zero probability; hence it substantially modifies the behavior of the receptor.") and redundancy ("Redundancy furnishes a guarantee against errors in transmission, since it permits the receptor to reconstruct the message even if some of its elements are lacking...").mercredi, 11 mai 2011
  39. 39. Functions of perception A coherent system of analysis must define requirements for the conception of the product, and coordinate aesthetic, technical, ergonomic, economical factors and marketing so that the product offers most value has the user. We call up exactly " functions design " or " function of perception " the functions that the product must assure (besides physical functions) to reach its purpose.mercredi, 11 mai 2011
  40. 40. information aesthetics Max Bense 1910-1990 Max Bense was a philosopher with a strong background in the sciences. He was a concrete poet, a supporter of the arts, a prolific author, an inspiring lecturer. He lectured about Charles Sanders Peirce and his semiotics at a time (late 1950s), when hardly anyone did this.mercredi, 11 mai 2011
  41. 41. information aesthetics Arguably the most important contribution of Bense’s was his information aesthetics. A fervent fighter against emotion- based value judgments, he considered any artifact as in principle an object also for aesthetic analysis and evaluation. The aesthetic object was a complex sign that functioned in a process of communication. Relying on G. D. Birkhoff’s aesthetic measure of order in complexity, Helmar Frank and Rul Gunzenhäuser defined the micro-aesthetic measure as redundancy in complexity where these two components were measured quantitatively in terms of the information theory of Shannon.mercredi, 11 mai 2011
  42. 42. information aesthetics This theory was influential during the 1960s not only with theoreticians of aesthetics, but in the entire community of concrete artists, writers, and designers. Bense’s version of information aesthetics was an aesthetics of the object. Abraham A. Moles developed at the same time an information aesthetics that took off from the observer and, therefore contained also subjective measures. The term “information aesthetics” gets used again after the year 2000. Its meaning is completely different. Web site:, 11 mai 2011
  43. 43. mercredi, 11 mai 2011
  44. 44. mercredi, 11 mai 2011
  45. 45. Functions of perception Semiology (theory of signs) Term "semiology" (from Greek semeion "sign", and logoes " speech, knowledge ") was proposed by Ferdinand de Saussure (Courts of general linguistics), and the discipline that it indicates defines as studying " the life of signs within the social life ". In the heart of this project takes place the conception of the sign as an entity in two faces: a signifier, reducible in an acoustic image or physical form of the sign, and one signified (concept, reality) a mental representation of the sign, the report among the two having for main characteristic its arbitrary power.mercredi, 11 mai 2011
  46. 46. Code Medium Medium Emitter Message Receiver Referent Connotation Signified Referent Ideology Signifier Myth Nature Culturemercredi, 11 mai 2011
  47. 47. Functions of perception Signified : A concept or meaning as distinguished from the sign through which it is communicated. This is abstract concept, a mental representation. Signifier : A symbol, sound, or image (as a word) that represents an underlying concept or meaning that refers to the signified. This is the physical form of a sign that we perceive through our senses.mercredi, 11 mai 2011
  48. 48. Functions of perception Denotation : A direct specific meaning as distinct from an implied or associated idea. Connotation : The suggesting of a meaning by a word apart from the thing it explicitly names or describes, something suggested by a word , sign or thing.mercredi, 11 mai 2011
  49. 49. Meaning, Messages and Signs Charles W. Morris uses in his theory of signs (semiotics) three classes of signs: Syntactic: the disposal of signs Semantic: the meaning of signs Pragmatic: the use of signs In the semantic aspect (pragmatic plus semantics according to Morris) we shall imply ail the experiences and personal knowledge, associations, preferences and value judgments of a cultural context! The syntactic and semantic aspects must not be interpreted completely independently. There is not unity of elementary meaning, meanings appear only from a wide context.mercredi, 11 mai 2011
  50. 50. Syntactic The disposal, the relation between elements and there, the formal organization is called syntactic in the information theory , or syntactic aspects; that some designers call also formal logic. In theory the syntactic is neutral regarding to a value judgment and is mathematically noticeable, practically nevertheless this neutrality disappears. The syntactic organization has a function: make recognizable practical functions simplify a complicated context, mention partial qualities or to unite various products in a system.mercredi, 11 mai 2011
  51. 51. Semantic The syntactic can be presented as basic condition of a much huge domain: meanings transmit by elements and their organization. Products are perceived as being gone out of fashion, new, original, boring, cold, sterile, erotic, light, solid, cheap, prestigious... This sum of meanings we shall call semantic aspect or semantics, psychological and intellectual relation between the product and user.mercredi, 11 mai 2011
  52. 52. Syntactic & Semantic We are inclined to attribute to these three televisions three different times in spite of a similarity in the construction and the syntactic elements; the judgment of proportions depends also on a technological development & cultural context!. 1980 1955 1940mercredi, 11 mai 2011
  53. 53. Syntactic & Semantic Association with bone ?mercredi, 11 mai 2011
  54. 54. Perception & meaning experience Our meaning experience depends on some number of factors existing in systems and environments in which we live. Our experience is also influences by our relations with the product. According to its personality, of its capacities and as well according to environments (systems) of different nature, each one experienced the product. It is for example the company in which we work, our organizations, our private environment and our society.mercredi, 11 mai 2011
  55. 55. Factors that influence our meaning experience The figure shows the superimposing of a private environment and of the society. In the bottom is that the common naming indicates laws, rule etc. Some are so anchored in the consciousness of the generations that they are identified as ethics and moral. Dimensions, proportions, classifications, materials, disposal, climate etc. characterize physical environment. It can also be described as environment of work, of house, educational, of transport, of relaxation etc. Social environment contains factors leaders such as status (social, economical, political or educational). There is also mobility (regarding to our activities or places) attitudes, social traditions, behavior etc. Within the framework of the cultural environment one can speak about fashion, about taste, about cultural level, about traditions, about symbolismmercredi, 11 mai 2011
  56. 56. Factors that influence our meaning experiencemercredi, 11 mai 2011
  57. 57. The Semantic Differential Semantic differential technique is well-known to professional social scientists. As innovations go, it has quite a long pedigree, for it builds upon psychological work in psychology and quantitative semantics that is over a half-century old. Developed by Charles Osgood and his colleagues, it has been applied cross-culturally with remarkably consistent results and, while claims for its universality must be treated with some skepticism, it appears to provide reliable data no matter where and when it is used (Osgood, 1952; Osgood, Suci and Tannenbaum 1957; and Snider and Osgood, 1969).mercredi, 11 mai 2011
  58. 58. The Semantic Differential, The Theory The idea behind semantic differential analysis is deceptively simple. It is based on the assumption that clues to the structure of prejudice can be obtained by inquiring into the psychological meaning of concepts and perceptions. Social stereotypes can be explored by inviting subjects to respond to specific items by checking that position on a seven-point bipolar adjective scale which best represents the direction and intensity (from "slightly through "quite" to "extremely") of the subjects judgment. So, to the word PACIFIST, a person might reply: WEAK: X::::: STRONG This would indicate that the concept PACIFIST evokes a quite strong connotative association with WEAKNESS in the mind of this hypothetical subject.mercredi, 11 mai 2011
  59. 59. The Semantic Differential, The Method The fundamental method is quite unpretentious. The scales suggested here break down into four groups: (1) Evaluation (bright / dark, good / boa, beautiful / ugly, pleasant / unpleasant); (2) Activity (angular / rounded, fast / slow, sharp / dull, active / passive); (3) Understandability (understandable / mysterious, simple / complicated, predictable / unpredictable, familiar / strange); (4) Potency (deep / shallow, heavy / light, rugged / delicate, strong / weak)..mercredi, 11 mai 2011
  60. 60. The Semantic Differential, The Method By assigning values from one ta seven ta each of the scaled items and then averaging the values in each group of scales, a general measure of the fundamental perceptions of a concept will emerge. Thus, if a subject reacted to the concept of PACIFIST thus: DARK :::::X: BRIGHT GOOD ::X:::: BAD UGLY :::X::: BEAUTIFUL PLEASANT :X::::: UNPLEASANT the overall evaluation of PACIFIST by the subject would be positive (Bright = 6; Good= 5; Beautiful = 4; Pleasant = 6: Total = 21 and Average = 5.2). It should be noted that the adjectives and the scales identified here are merely representative of available and well-tested semantic dimensions and readers are referred ta the classic texts by Osgood and others identified herein both for additional scientifically validated scales and for some methodological cautions against using just any polar adjectives that seem relevantmercredi, 11 mai 2011
  61. 61. The Semantic Differential, Applications What emerges semantic differentiation is a mathematical description of the connotative or emotive meanings of concepts. If, for example, responses to the such concepts MEN and WOMEN or ENGLISH and FRENCH were calculated and compared, the result would allow for an accurate assessment both of the subjects core attitudes toward these groups and a measure for comparing those attitudes with one another. One obvious practical application would be to average the responses of all participants toward a number of ethnic groups at the start of a program in, for instance, "multicultural awareness." If the first results were not disclosed at the beginning and the process was repeated at the end, a comparison of the two sets of responses would provide an immediate and accurate reflection of the degree and direction of attitudinal changes, irrespective of the subjects expressed opinions.mercredi, 11 mai 2011
  62. 62. The Semantic Differential, Applications CONCEPT TO BE INVESTIGATED (Ethnic group, religious denomination, age cohort gender identification, political movement, etc.) Instructions: Think for a moment about the group mentioned above an then quickly place an "X" on each of the scales below indicating the degree to which the group strikes you as closer to one or the other adjective. If no answer comes to you quickly, simply mark the middle space. DARK : : : : : : BRIGHT ACTIVE :::::: PASSIVE ANGULAR : : : : : : ROUNDED WEAK :::::: STRONG FAST : : : : : : SLOW DARK :::::: BRIGHT DEEP : : : : : : SHALLOW GOOD :::::: BAD GOOD : : : : : : BAD UGLY :::::: BEAUTIFUL DULL : : : : : : SHARP LIGHT :::::: HEAVY RUGGED : : : : : : DELICATE MYSTERIOUS : : : : : : UNDERSTANDABLE PREDICTABLE : : : : : : UNPREDICTABLE PLEASANT : : : : : : UNPLEASANTmercredi, 11 mai 2011
  63. 63. The Semantic Differential, Applicationsmercredi, 11 mai 2011