Information theory

design theory.
6

mercredi, 11 mai 2011

Information
theory


Information theory

Why speak about information?
Because when we have look at an object, when we evaluate
and appreciate it, the relation which established between the
object and us is an exchange of information.


Information theory
Humans are symbol-making creatures. We communicate by
symbols -growls and grunts, hand signals, and drawings painted
on cave walls in prehistoric times.

Later we developed languages, associating sounds with ideas.
Eventually Homo Sapiens developed writing, perhaps first symbols
scratched on rocks, then written more permanently on tablets,
papyrus, and paper.

Today, we transmit symbols , coded digital signals of voice,
graphics, video, and data around the world at close to the speed of
light.

We’re even sending signals into outer space in the hope of finding
other symbol-creating species.


Information theory

The theory stems from pioneering work done by the
American electrical engineer Claude E. Shannon, who
published his seminal work in 1948.
Since then, information theory has developed rapidly,
affecting not only the design of communications systems but
also such areas as automation, information science,
psychology, linguistics, art and design.


Claude E. Shannon
Shannon was born in Petoskey, Michigan,
on April 30, 1916. He graduated from the
University of Michigan in 1936 with
bachelor's degrees in mathematics and
electrical engineering. In 1940 he earned
both a master's degree in electrical
engineering and a Ph.D. in mathematics
from the Massachusetts Institute of
Technology (MIT).
His work founded the subject of
information theory. Claude Shannon died
in February 2001.


Information theory
Understanding Information Theory

Understanding Shannon’s basis of Information Theory, is not an
easy matter. To get a high-level understanding of his theory, a few
basic points should be made.

In the first step, the message has to be put into some kind of
symbolic representation – words, musical notes, icons,
mathematical equations, or bits.

When we write “Hello,” we encode a greeting.
When we write a musical score, it’s the same thing – only we’re
encoding sounds.


Information theory

First, words are symbols to carry information between people. If
one says to an American, “Let’s go!”, the command is immediately
understood.
But if we give the commands in Russian, “Pustim v xod!”, we only
get a quizzical look. Russian is the wrong code for an American.

Second, all communication involves three steps:
Coding a message at its source - Transmitting the message
through a communications channel, and Decoding the message at
its destination.


Information theory
In the first step, the message has to be put into some kind of
symbolic representation – words, musical notes, icons,
mathematical equations, or bits.

For any code to be useful it has to be transmitted to someone or,
in a computer’s case, to something.
Transmission can be by voice, a letter, a billboard, a telephone
conversation, a radio or television broadcast, or the now
ubiquitous e-mail.
At the destination, someone or something has to receive the
symbols, and then decode them by matching them against his or
her own body of information to extract the data.


Information theory

Shannon describes the elements of communications system
theory as: a source - encoder - channel - decoder - destination
model.
What his theory does is to replace each element in the model with
a mathematical model that describes that element’s behavior
within the system.


Information theory
The meaning of information?

“Information” has a special meaning for Shannon. For years,
people deliberately compressed telegraph messages by leaving
certain words out, or sending key words that stood for longer
messages, since costs were determined by the number of words
sent.
Yet people could easily read these abbreviated messages, since
they supplied these predictable words, such “a” and “the.”


Information theory

In the same vein, for Shannon, information is symbols that contain
unpredictable news, like the sentence:

“only infrmatn esentil to understandn mst b tranmitd.”

The predictable symbols that we can leave out, which Shannon
calls redundancy, are not really news.


Information theory

Another example is coin flipping. Each time we flip a coin, we can
transmit which way it lands, “heads” or “tails,” by transmitting a
code of “zero” or “one.”
But what if the coin has two “heads” and everyone knows it?
Since there is no uncertainty concerning the outcome of a flip, no
message need be sent at all.
Although this view might seem like common sense today, it was
not always so.
Shannon made clear that uncertainty or unpredictability is the very
commodity of communication.


Information theory

The amount of information, or uncertainty, output by an information
source is a measure of its entropy.


Principles of information theory

The basic elements of any general communications system
include
1- a source of information which is a transmitting device that transforms
the information or "message" into a form suitable for transmission by a
particular means.
2- the means or channel over which the message is transmitted.
3- a receiving device which decodes the message back into some
approximation of its original form.
4- the destination or intended recipient of the message.
5- a source of noise (i.e., interference or distortion) which changes the
message in unpredictable ways during transmission



It is important to note that "information" as understood in
information theory has nothing to do with any inherent
meaning in a message; it is rather a measure of the
predictability and complexity of each transmitted message.
The expected value of a transmission is referred to as the
entropy, or average information of the set of messages

(entropy: the amount of entropy is a measure of the disorder,
or randomness, of a system.)



Suppose we have a device that can produce 3 symbols, A, B,
or C. As we wait for the next symbol, we are uncertain as to
which symbol it will produce. Once a symbol appears and we
see it, our uncertainty decreases, and we remark that we
have received some information.
That is, information is a decrease in uncertainty.

Information is the measure of the predictability and
complexity of a message.



The individuals are determined by the messages of their
environment; messages are complex forms.
These messages have an elementary defined structure, as
regards the ulterior reactions of the individuals, by psycho-
physiology's properties of the receiver.
Nearby immediate messages, there are messages taken
away in the time or space that are restored to the
environment by means of spatial charnel (transmissions) or
temporal (recording).
Messages are measured by a quantity of information, which is
originality it is to say the unpredictability that they bring.



Information is a quantity essentially different from the meaning and
independent of this one: a maximum message of information can
seem devoid of sense if the individual is not able to decode it. In a
general way, comprehensibility varies to the opposite way of the
information.



One calls redundancy the repetition of elements within a message
that prevents the failure of communication of information-is the
greatest antidote to entropy
Meaning is based on a convention set in principle common to the
transmitter and to the receiver, it is not transported, it preexists
potentially before the message. Information is in facto measure of
the complexity of patterns proposed by the perception. Complexity
alone is transported from the transmitter to the receiver it is exactly
what is not present in the receiver


Information theory and Gestalt theory

Any form is the expression of an unpredictable predictability which
is measured by its degree of coherence
The comprehensibility of a form is proportional in the redundancy of
this one
The message the most difficult to transmit is the one that does not
contain any redundancy so any form


Information theory and Gestalt theory

The notion of complex forms covers that of super signs. Super
signs are assemblies normalized of elements known beforehand.

One of the most elementary temporal forms is the periodicity.
There is perception of a periodicity as soon as it has expectation of
a similar following event to those that already occurred. The
continuance of a shape is only an aspect of periodicity


Law of good continuation
Humans tend to continue contours whenever the elements of
the pattern establish an implied direction

People tend to draw a good continuous line


Law of Similarity

Elements that look similar will be perceived as a part of the
same form

There seems to be a triangle in the square


Law of closure
Humans tend to enclose a space by completing a contour
and ignoring gaps in the figure


Information theory & esthetic perception
Abraham Moles 1920-1992

Abraham Moles was on the faculty
at the University Louis Pasteur,
Strasbourg, France, and Director
of the Institute of Social
Psychology of Communications.
Moles was one of the first
aestheticians to deal with
cybernetic issues.
Moles has published a number of
books, including, in 1958, an
influential book entitled
“Information theory and et
perception esthetic”.


Information theory & esthetic perception

In 1966 Abraham Moles, write Information Theory and
Esthetic Perception (University of Illinois Press).
Moles was dealing with such concepts as originality ("If a
given message or event is certain, it teaches the receptor
nothing and cannot modify his behavior.
An unexpected event has by definition a zero probability;
hence it substantially modifies the behavior of the receptor.")
and redundancy ("Redundancy furnishes a guarantee against
errors in transmission, since it permits the receptor to
reconstruct the message even if some of its elements are
lacking...").


Functions of perception

A coherent system of analysis must define requirements for
the conception of the product, and coordinate aesthetic,
technical, ergonomic, economical factors and marketing so
that the product offers most value has the user. We call up
exactly " functions design " or " function of perception " the
functions that the product must assure (besides physical
functions) to reach its purpose.


information aesthetics
Max Bense 1910-1990

Max Bense was a philosopher with
a strong background in the
sciences. He was a concrete poet,
a supporter of the arts, a prolific
author, an inspiring lecturer. He
lectured about Charles Sanders
Peirce and his semiotics at a time
(late 1950s), when hardly anyone
did this.



Arguably the most important contribution of Bense’s was his
information aesthetics. A fervent fighter against emotion-
based value judgments, he considered any artifact as in
principle an object also for aesthetic analysis and evaluation.
The aesthetic object was a complex sign that functioned in a
process of communication. Relying on G. D. Birkhoff’s
aesthetic measure of order in complexity, Helmar Frank and
Rul Gunzenhäuser defined the micro-aesthetic measure as
redundancy in complexity where these two components were
measured quantitatively in terms of the information theory of
Shannon.



This theory was influential during the 1960s not only with
theoreticians of aesthetics, but in the entire community of
concrete artists, writers, and designers. Bense’s version of
information aesthetics was an aesthetics of the object.
Abraham A. Moles developed at the same time an information
aesthetics that took off from the observer and, therefore
contained also subjective measures.
The term “information aesthetics” gets used again after the
year 2000. Its meaning is completely different.

Web site:
http://www.max-bense.de/


Semiology (theory of signs)

Term "semiology" (from Greek semeion "sign", and logoes " speech,
knowledge ") was proposed by Ferdinand de Saussure (Courts of general
linguistics), and the discipline that it indicates defines as studying " the life
of signs within the social life ".
In the heart of this project takes place the conception of the sign as an
entity in two faces: a signifier, reducible in an acoustic image or physical
form of the sign, and one signified (concept, reality) a mental
representation of the sign, the report among the two having for main
characteristic its arbitrary power.


Code

Medium Medium
Emitter Message Receiver

Referent

Connotation

Signified
Referent Ideology
Signifier

Myth

Nature Culture


Signified :
A concept or meaning as distinguished from the sign through
which it is communicated. This is abstract concept, a mental
representation.

Signifier :
A symbol, sound, or image (as a word) that represents an
underlying concept or meaning that refers to the signified.
This is the physical form of a sign that we perceive through
our senses.



Denotation :
A direct specific meaning as distinct from an implied or
associated idea.

Connotation :
The suggesting of a meaning by a word apart from the thing it
explicitly names or describes, something suggested by a
word , sign or thing.


Meaning, Messages and Signs

Charles W. Morris uses in his theory of signs (semiotics) three
classes of signs:
Syntactic: the disposal of signs
Semantic: the meaning of signs
Pragmatic: the use of signs
In the semantic aspect (pragmatic plus semantics according
to Morris) we shall imply ail the experiences and personal
knowledge, associations, preferences and value judgments of
a cultural context!
The syntactic and semantic aspects must not be interpreted
completely independently. There is not unity of elementary
meaning, meanings appear only from a wide context.


Syntactic

The disposal, the relation between elements and there, the
formal organization is called syntactic in the information
theory , or syntactic aspects; that some designers call also
formal logic.
In theory the syntactic is neutral regarding to a value
judgment and is mathematically noticeable, practically
nevertheless this neutrality disappears. The syntactic
organization has a function: make recognizable practical
functions simplify a complicated context, mention partial
qualities or to unite various products in a system.


Semantic

The syntactic can be presented as basic condition of a much
huge domain: meanings transmit by elements and their
organization.
Products are perceived as being gone out of fashion, new,
original, boring, cold, sterile, erotic, light, solid, cheap,
prestigious... This sum of meanings we shall call semantic
aspect or semantics, psychological and intellectual relation
between the product and user.


Syntactic & Semantic

We are inclined to attribute to these three televisions three
different times in spite of a similarity in the construction and
the syntactic elements; the judgment of proportions depends
also on a technological development & cultural context!.

1980 1955 1940


Syntactic & Semantic

Association with bone ?


Perception & meaning experience

Our meaning experience depends on some number of factors
existing in systems and environments in which we live. Our
experience is also influences by our relations with the
product. According to its personality, of its capacities and as
well according to environments (systems) of different nature,
each one experienced the product. It is for example the
company in which we work, our organizations, our private
environment and our society.


Factors that influence our meaning experience

The figure shows the superimposing of a private environment and of the
society.
In the bottom is that the common naming indicates laws, rule etc. Some
are so anchored in the consciousness of the generations that they are
identified as ethics and moral.
Dimensions, proportions, classifications, materials, disposal, climate etc.
characterize physical environment. It can also be described as
environment of work, of house, educational, of transport, of relaxation etc.
Social environment contains factors leaders such as status (social,
economical, political or educational). There is also mobility (regarding to
our activities or places) attitudes, social traditions, behavior etc.
Within the framework of the cultural environment one can speak about
fashion, about taste, about cultural level, about traditions, about
symbolism


Factors that influence our meaning experience


The Semantic Differential

Semantic differential technique is well-known to professional
social scientists. As innovations go, it has quite a long
pedigree, for it builds upon psychological work in psychology
and quantitative semantics that is over a half-century old.
Developed by Charles Osgood and his colleagues, it has
been applied cross-culturally with remarkably consistent
results and, while claims for its universality must be treated
with some skepticism, it appears to provide reliable data no
matter where and when it is used (Osgood, 1952; Osgood,
Suci and Tannenbaum 1957; and Snider and Osgood, 1969).


The Semantic Differential, The Theory
The idea behind semantic differential analysis is deceptively simple. It is
based on the assumption that clues to the structure of prejudice can be
obtained by inquiring into the psychological meaning of concepts and
perceptions. Social stereotypes can be explored by inviting subjects to
respond to specific items by checking that position on a seven-point
bipolar adjective scale which best represents the direction and intensity
(from "slightly through "quite" to "extremely") of the subject's judgment.
So, to the word PACIFIST, a person might reply:

WEAK: X::::: STRONG

This would indicate that the concept PACIFIST evokes a quite strong
connotative association with WEAKNESS in the mind of this hypothetical
subject.


The Semantic Differential, The Method
The fundamental method is quite unpretentious. The scales suggested
here break down into four groups:

(1) Evaluation (bright / dark, good / boa, beautiful / ugly, pleasant /
unpleasant);

(2) Activity (angular / rounded, fast / slow, sharp / dull, active / passive);

(3) Understandability (understandable / mysterious, simple / complicated,
predictable / unpredictable, familiar / strange);

(4) Potency (deep / shallow, heavy / light, rugged / delicate, strong /
weak)..


The Semantic Differential, The Method
By assigning values from one ta seven ta each of the scaled items and
then averaging the values in each group of scales, a general measure of
the fundamental perceptions of a concept will emerge. Thus, if a subject
reacted to the concept of PACIFIST thus:
DARK :::::X: BRIGHT
GOOD ::X:::: BAD
UGLY :::X::: BEAUTIFUL
PLEASANT :X::::: UNPLEASANT
the overall evaluation of PACIFIST by the subject would be positive (Bright
= 6; Good= 5; Beautiful = 4; Pleasant = 6: Total = 21 and Average = 5.2). It
should be noted that the adjectives and the scales identified here are
merely representative of available and well-tested semantic dimensions
and readers are referred ta the classic texts by Osgood and others
identified herein both for additional scientifically validated scales and for
some methodological cautions against using just any polar adjectives that
seem relevant


The Semantic Differential, Applications
What emerges semantic differentiation is a mathematical description of
the connotative or emotive meanings of concepts. If, for example,
responses to the such concepts MEN and WOMEN or ENGLISH and
FRENCH were calculated and compared, the result would allow for an
accurate assessment both of the subjects' core attitudes toward these
groups and a measure for comparing those attitudes with one another.

One obvious practical application would be to average the responses of
all participants toward a number of ethnic groups at the start of a program
in, for instance, "multicultural awareness." If the first results were not
disclosed at the beginning and the process was repeated at the end, a
comparison of the two sets of responses would provide an immediate and
accurate reflection of the degree and direction of attitudinal changes,
irrespective of the subjects' expressed opinions.


CONCEPT TO BE INVESTIGATED (Ethnic group, religious denomination,
age cohort gender identification, political movement, etc.) Instructions:
Think for a moment about the group mentioned above an then quickly
place an "X" on each of the scales below indicating the degree to which
the group strikes you as closer to one or the other adjective. If no answer
comes to you quickly, simply mark the middle space.
DARK : : : : : : BRIGHT ACTIVE :::::: PASSIVE
ANGULAR : : : : : : ROUNDED WEAK :::::: STRONG
FAST : : : : : : SLOW DARK :::::: BRIGHT
DEEP : : : : : : SHALLOW GOOD :::::: BAD
GOOD : : : : : : BAD UGLY :::::: BEAUTIFUL
DULL : : : : : : SHARP LIGHT :::::: HEAVY
RUGGED : : : : : : DELICATE
MYSTERIOUS : : : : : : UNDERSTANDABLE
PREDICTABLE : : : : : : UNPREDICTABLE
PLEASANT : : : : : : UNPLEASANT


Information theory

More Related Content

What's hot

Similar to Information theory

More from roger Pitiot

Recently uploaded

Information theory