SlideShare a Scribd company logo
1 of 14
Shannon Entropy and Meaning
Unit 3
Shannon Entropy and Meaning
Claude Elwood Shannon (1916 - 2001) (MIT Museum, Boston / Nixdorf Museums
Forum, Paderborn)
Shannon Entropy and Meaning
Claude Elwood Shannon (1916 - 2001) (MIT Museum, Boston / Nixdorf Museums
Forum, Paderborn)
Shannon Entropy and Meaning
Shannon Entropy and Meaning
Shannon Entropy and Meaning
Shannon Entropy and Meaning
Shannon Entropy and Meaning
Shannon Entropy and Meaning
Shannon Entropy and Meaning
Shannon Entropy and Meaning
Shannon Entropy and Meaning
Shannon Entropy and Meaning
Shannon Entropy and Meaning

More Related Content

More from Hector Zenil

4.8: Epistemological Aspects of Infinite Wisdom
4.8: Epistemological Aspects of Infinite Wisdom4.8: Epistemological Aspects of Infinite Wisdom
4.8: Epistemological Aspects of Infinite WisdomHector Zenil
 
4.7: Chaitin's Omega Number
4.7: Chaitin's Omega Number4.7: Chaitin's Omega Number
4.7: Chaitin's Omega NumberHector Zenil
 
4.6: Convergence of Definitions
4.6: Convergence of Definitions4.6: Convergence of Definitions
4.6: Convergence of DefinitionsHector Zenil
 
4.5: The Invariance Theorem
4.5: The Invariance Theorem4.5: The Invariance Theorem
4.5: The Invariance TheoremHector Zenil
 
4.4: Algorithmic Complexity and Compressibility
4.4: Algorithmic Complexity and Compressibility4.4: Algorithmic Complexity and Compressibility
4.4: Algorithmic Complexity and CompressibilityHector Zenil
 
4.3: Pseudo Randomness
4.3: Pseudo Randomness4.3: Pseudo Randomness
4.3: Pseudo RandomnessHector Zenil
 
4.1 Sources of Randomness
4.1 Sources of Randomness4.1 Sources of Randomness
4.1 Sources of RandomnessHector Zenil
 
Unit 3: Classical information
Unit 3: Classical informationUnit 3: Classical information
Unit 3: Classical informationHector Zenil
 
Unit 3: Joint, Conditional, Mutual Information, & a Case Study
Unit 3: Joint, Conditional, Mutual Information, & a Case StudyUnit 3: Joint, Conditional, Mutual Information, & a Case Study
Unit 3: Joint, Conditional, Mutual Information, & a Case StudyHector Zenil
 
Unit 3: Entropy rate, languages and multidimensional data
Unit 3: Entropy rate, languages and multidimensional dataUnit 3: Entropy rate, languages and multidimensional data
Unit 3: Entropy rate, languages and multidimensional dataHector Zenil
 
Unit 3: Redundancy, noise, and biological information
Unit 3: Redundancy, noise, and biological informationUnit 3: Redundancy, noise, and biological information
Unit 3: Redundancy, noise, and biological informationHector Zenil
 
Algorithmic Dynamics of Cellular Automata
Algorithmic Dynamics of Cellular AutomataAlgorithmic Dynamics of Cellular Automata
Algorithmic Dynamics of Cellular AutomataHector Zenil
 
Graph Spectra through Network Complexity Measures: Information Content of Eig...
Graph Spectra through Network Complexity Measures: Information Content of Eig...Graph Spectra through Network Complexity Measures: Information Content of Eig...
Graph Spectra through Network Complexity Measures: Information Content of Eig...Hector Zenil
 
Cognition, Information and Subjective Computation
Cognition, Information and Subjective ComputationCognition, Information and Subjective Computation
Cognition, Information and Subjective ComputationHector Zenil
 
Fractal dimension versus Computational Complexity
Fractal dimension versus Computational ComplexityFractal dimension versus Computational Complexity
Fractal dimension versus Computational ComplexityHector Zenil
 
Information Theory and Programmable Medicine
Information Theory and Programmable MedicineInformation Theory and Programmable Medicine
Information Theory and Programmable MedicineHector Zenil
 
Complexity and Computation in Nature: How can we test for Artificial Life?
Complexity and Computation in Nature: How can we test for Artificial Life?Complexity and Computation in Nature: How can we test for Artificial Life?
Complexity and Computation in Nature: How can we test for Artificial Life?Hector Zenil
 
Fractal Dimension of Space-time Diagrams and the Runtime Complexity of Small ...
Fractal Dimension of Space-time Diagrams and the Runtime Complexity of Small ...Fractal Dimension of Space-time Diagrams and the Runtime Complexity of Small ...
Fractal Dimension of Space-time Diagrams and the Runtime Complexity of Small ...Hector Zenil
 
Information Content of Complex Networks
Information Content of Complex NetworksInformation Content of Complex Networks
Information Content of Complex NetworksHector Zenil
 
Algorithmic Information Theory and Computational Biology
Algorithmic Information Theory and Computational BiologyAlgorithmic Information Theory and Computational Biology
Algorithmic Information Theory and Computational BiologyHector Zenil
 

More from Hector Zenil (20)

4.8: Epistemological Aspects of Infinite Wisdom
4.8: Epistemological Aspects of Infinite Wisdom4.8: Epistemological Aspects of Infinite Wisdom
4.8: Epistemological Aspects of Infinite Wisdom
 
4.7: Chaitin's Omega Number
4.7: Chaitin's Omega Number4.7: Chaitin's Omega Number
4.7: Chaitin's Omega Number
 
4.6: Convergence of Definitions
4.6: Convergence of Definitions4.6: Convergence of Definitions
4.6: Convergence of Definitions
 
4.5: The Invariance Theorem
4.5: The Invariance Theorem4.5: The Invariance Theorem
4.5: The Invariance Theorem
 
4.4: Algorithmic Complexity and Compressibility
4.4: Algorithmic Complexity and Compressibility4.4: Algorithmic Complexity and Compressibility
4.4: Algorithmic Complexity and Compressibility
 
4.3: Pseudo Randomness
4.3: Pseudo Randomness4.3: Pseudo Randomness
4.3: Pseudo Randomness
 
4.1 Sources of Randomness
4.1 Sources of Randomness4.1 Sources of Randomness
4.1 Sources of Randomness
 
Unit 3: Classical information
Unit 3: Classical informationUnit 3: Classical information
Unit 3: Classical information
 
Unit 3: Joint, Conditional, Mutual Information, & a Case Study
Unit 3: Joint, Conditional, Mutual Information, & a Case StudyUnit 3: Joint, Conditional, Mutual Information, & a Case Study
Unit 3: Joint, Conditional, Mutual Information, & a Case Study
 
Unit 3: Entropy rate, languages and multidimensional data
Unit 3: Entropy rate, languages and multidimensional dataUnit 3: Entropy rate, languages and multidimensional data
Unit 3: Entropy rate, languages and multidimensional data
 
Unit 3: Redundancy, noise, and biological information
Unit 3: Redundancy, noise, and biological informationUnit 3: Redundancy, noise, and biological information
Unit 3: Redundancy, noise, and biological information
 
Algorithmic Dynamics of Cellular Automata
Algorithmic Dynamics of Cellular AutomataAlgorithmic Dynamics of Cellular Automata
Algorithmic Dynamics of Cellular Automata
 
Graph Spectra through Network Complexity Measures: Information Content of Eig...
Graph Spectra through Network Complexity Measures: Information Content of Eig...Graph Spectra through Network Complexity Measures: Information Content of Eig...
Graph Spectra through Network Complexity Measures: Information Content of Eig...
 
Cognition, Information and Subjective Computation
Cognition, Information and Subjective ComputationCognition, Information and Subjective Computation
Cognition, Information and Subjective Computation
 
Fractal dimension versus Computational Complexity
Fractal dimension versus Computational ComplexityFractal dimension versus Computational Complexity
Fractal dimension versus Computational Complexity
 
Information Theory and Programmable Medicine
Information Theory and Programmable MedicineInformation Theory and Programmable Medicine
Information Theory and Programmable Medicine
 
Complexity and Computation in Nature: How can we test for Artificial Life?
Complexity and Computation in Nature: How can we test for Artificial Life?Complexity and Computation in Nature: How can we test for Artificial Life?
Complexity and Computation in Nature: How can we test for Artificial Life?
 
Fractal Dimension of Space-time Diagrams and the Runtime Complexity of Small ...
Fractal Dimension of Space-time Diagrams and the Runtime Complexity of Small ...Fractal Dimension of Space-time Diagrams and the Runtime Complexity of Small ...
Fractal Dimension of Space-time Diagrams and the Runtime Complexity of Small ...
 
Information Content of Complex Networks
Information Content of Complex NetworksInformation Content of Complex Networks
Information Content of Complex Networks
 
Algorithmic Information Theory and Computational Biology
Algorithmic Information Theory and Computational BiologyAlgorithmic Information Theory and Computational Biology
Algorithmic Information Theory and Computational Biology
 

Recently uploaded

Cytokinin, mechanism and its application.pptx
Cytokinin, mechanism and its application.pptxCytokinin, mechanism and its application.pptx
Cytokinin, mechanism and its application.pptxVarshiniMK
 
Manassas R - Parkside Middle School 🌎🏫
Manassas R - Parkside Middle School 🌎🏫Manassas R - Parkside Middle School 🌎🏫
Manassas R - Parkside Middle School 🌎🏫qfactory1
 
Scheme-of-Work-Science-Stage-4 cambridge science.docx
Scheme-of-Work-Science-Stage-4 cambridge science.docxScheme-of-Work-Science-Stage-4 cambridge science.docx
Scheme-of-Work-Science-Stage-4 cambridge science.docxyaramohamed343013
 
Microphone- characteristics,carbon microphone, dynamic microphone.pptx
Microphone- characteristics,carbon microphone, dynamic microphone.pptxMicrophone- characteristics,carbon microphone, dynamic microphone.pptx
Microphone- characteristics,carbon microphone, dynamic microphone.pptxpriyankatabhane
 
Solution chemistry, Moral and Normal solutions
Solution chemistry, Moral and Normal solutionsSolution chemistry, Moral and Normal solutions
Solution chemistry, Moral and Normal solutionsHajira Mahmood
 
insect anatomy and insect body wall and their physiology
insect anatomy and insect body wall and their  physiologyinsect anatomy and insect body wall and their  physiology
insect anatomy and insect body wall and their physiologyDrAnita Sharma
 
Heredity: Inheritance and Variation of Traits
Heredity: Inheritance and Variation of TraitsHeredity: Inheritance and Variation of Traits
Heredity: Inheritance and Variation of TraitsCharlene Llagas
 
BIOETHICS IN RECOMBINANT DNA TECHNOLOGY.
BIOETHICS IN RECOMBINANT DNA TECHNOLOGY.BIOETHICS IN RECOMBINANT DNA TECHNOLOGY.
BIOETHICS IN RECOMBINANT DNA TECHNOLOGY.PraveenaKalaiselvan1
 
Forest laws, Indian forest laws, why they are important
Forest laws, Indian forest laws, why they are importantForest laws, Indian forest laws, why they are important
Forest laws, Indian forest laws, why they are importantadityabhardwaj282
 
RESPIRATORY ADAPTATIONS TO HYPOXIA IN HUMNAS.pptx
RESPIRATORY ADAPTATIONS TO HYPOXIA IN HUMNAS.pptxRESPIRATORY ADAPTATIONS TO HYPOXIA IN HUMNAS.pptx
RESPIRATORY ADAPTATIONS TO HYPOXIA IN HUMNAS.pptxFarihaAbdulRasheed
 
Bentham & Hooker's Classification. along with the merits and demerits of the ...
Bentham & Hooker's Classification. along with the merits and demerits of the ...Bentham & Hooker's Classification. along with the merits and demerits of the ...
Bentham & Hooker's Classification. along with the merits and demerits of the ...Nistarini College, Purulia (W.B) India
 
Twin's paradox experiment is a meassurement of the extra dimensions.pptx
Twin's paradox experiment is a meassurement of the extra dimensions.pptxTwin's paradox experiment is a meassurement of the extra dimensions.pptx
Twin's paradox experiment is a meassurement of the extra dimensions.pptxEran Akiva Sinbar
 
‏‏VIRUS - 123455555555555555555555555555555555555555
‏‏VIRUS -  123455555555555555555555555555555555555555‏‏VIRUS -  123455555555555555555555555555555555555555
‏‏VIRUS - 123455555555555555555555555555555555555555kikilily0909
 
Harmful and Useful Microorganisms Presentation
Harmful and Useful Microorganisms PresentationHarmful and Useful Microorganisms Presentation
Harmful and Useful Microorganisms Presentationtahreemzahra82
 
Call Girls In Nihal Vihar Delhi ❤️8860477959 Looking Escorts In 24/7 Delhi NCR
Call Girls In Nihal Vihar Delhi ❤️8860477959 Looking Escorts In 24/7 Delhi NCRCall Girls In Nihal Vihar Delhi ❤️8860477959 Looking Escorts In 24/7 Delhi NCR
Call Girls In Nihal Vihar Delhi ❤️8860477959 Looking Escorts In 24/7 Delhi NCRlizamodels9
 
Spermiogenesis or Spermateleosis or metamorphosis of spermatid
Spermiogenesis or Spermateleosis or metamorphosis of spermatidSpermiogenesis or Spermateleosis or metamorphosis of spermatid
Spermiogenesis or Spermateleosis or metamorphosis of spermatidSarthak Sekhar Mondal
 
Module 4: Mendelian Genetics and Punnett Square
Module 4:  Mendelian Genetics and Punnett SquareModule 4:  Mendelian Genetics and Punnett Square
Module 4: Mendelian Genetics and Punnett SquareIsiahStephanRadaza
 
Artificial Intelligence In Microbiology by Dr. Prince C P
Artificial Intelligence In Microbiology by Dr. Prince C PArtificial Intelligence In Microbiology by Dr. Prince C P
Artificial Intelligence In Microbiology by Dr. Prince C PPRINCE C P
 

Recently uploaded (20)

Cytokinin, mechanism and its application.pptx
Cytokinin, mechanism and its application.pptxCytokinin, mechanism and its application.pptx
Cytokinin, mechanism and its application.pptx
 
Engler and Prantl system of classification in plant taxonomy
Engler and Prantl system of classification in plant taxonomyEngler and Prantl system of classification in plant taxonomy
Engler and Prantl system of classification in plant taxonomy
 
Manassas R - Parkside Middle School 🌎🏫
Manassas R - Parkside Middle School 🌎🏫Manassas R - Parkside Middle School 🌎🏫
Manassas R - Parkside Middle School 🌎🏫
 
Scheme-of-Work-Science-Stage-4 cambridge science.docx
Scheme-of-Work-Science-Stage-4 cambridge science.docxScheme-of-Work-Science-Stage-4 cambridge science.docx
Scheme-of-Work-Science-Stage-4 cambridge science.docx
 
Microphone- characteristics,carbon microphone, dynamic microphone.pptx
Microphone- characteristics,carbon microphone, dynamic microphone.pptxMicrophone- characteristics,carbon microphone, dynamic microphone.pptx
Microphone- characteristics,carbon microphone, dynamic microphone.pptx
 
Solution chemistry, Moral and Normal solutions
Solution chemistry, Moral and Normal solutionsSolution chemistry, Moral and Normal solutions
Solution chemistry, Moral and Normal solutions
 
insect anatomy and insect body wall and their physiology
insect anatomy and insect body wall and their  physiologyinsect anatomy and insect body wall and their  physiology
insect anatomy and insect body wall and their physiology
 
Heredity: Inheritance and Variation of Traits
Heredity: Inheritance and Variation of TraitsHeredity: Inheritance and Variation of Traits
Heredity: Inheritance and Variation of Traits
 
BIOETHICS IN RECOMBINANT DNA TECHNOLOGY.
BIOETHICS IN RECOMBINANT DNA TECHNOLOGY.BIOETHICS IN RECOMBINANT DNA TECHNOLOGY.
BIOETHICS IN RECOMBINANT DNA TECHNOLOGY.
 
Forest laws, Indian forest laws, why they are important
Forest laws, Indian forest laws, why they are importantForest laws, Indian forest laws, why they are important
Forest laws, Indian forest laws, why they are important
 
RESPIRATORY ADAPTATIONS TO HYPOXIA IN HUMNAS.pptx
RESPIRATORY ADAPTATIONS TO HYPOXIA IN HUMNAS.pptxRESPIRATORY ADAPTATIONS TO HYPOXIA IN HUMNAS.pptx
RESPIRATORY ADAPTATIONS TO HYPOXIA IN HUMNAS.pptx
 
Bentham & Hooker's Classification. along with the merits and demerits of the ...
Bentham & Hooker's Classification. along with the merits and demerits of the ...Bentham & Hooker's Classification. along with the merits and demerits of the ...
Bentham & Hooker's Classification. along with the merits and demerits of the ...
 
Twin's paradox experiment is a meassurement of the extra dimensions.pptx
Twin's paradox experiment is a meassurement of the extra dimensions.pptxTwin's paradox experiment is a meassurement of the extra dimensions.pptx
Twin's paradox experiment is a meassurement of the extra dimensions.pptx
 
‏‏VIRUS - 123455555555555555555555555555555555555555
‏‏VIRUS -  123455555555555555555555555555555555555555‏‏VIRUS -  123455555555555555555555555555555555555555
‏‏VIRUS - 123455555555555555555555555555555555555555
 
Harmful and Useful Microorganisms Presentation
Harmful and Useful Microorganisms PresentationHarmful and Useful Microorganisms Presentation
Harmful and Useful Microorganisms Presentation
 
Volatile Oils Pharmacognosy And Phytochemistry -I
Volatile Oils Pharmacognosy And Phytochemistry -IVolatile Oils Pharmacognosy And Phytochemistry -I
Volatile Oils Pharmacognosy And Phytochemistry -I
 
Call Girls In Nihal Vihar Delhi ❤️8860477959 Looking Escorts In 24/7 Delhi NCR
Call Girls In Nihal Vihar Delhi ❤️8860477959 Looking Escorts In 24/7 Delhi NCRCall Girls In Nihal Vihar Delhi ❤️8860477959 Looking Escorts In 24/7 Delhi NCR
Call Girls In Nihal Vihar Delhi ❤️8860477959 Looking Escorts In 24/7 Delhi NCR
 
Spermiogenesis or Spermateleosis or metamorphosis of spermatid
Spermiogenesis or Spermateleosis or metamorphosis of spermatidSpermiogenesis or Spermateleosis or metamorphosis of spermatid
Spermiogenesis or Spermateleosis or metamorphosis of spermatid
 
Module 4: Mendelian Genetics and Punnett Square
Module 4:  Mendelian Genetics and Punnett SquareModule 4:  Mendelian Genetics and Punnett Square
Module 4: Mendelian Genetics and Punnett Square
 
Artificial Intelligence In Microbiology by Dr. Prince C P
Artificial Intelligence In Microbiology by Dr. Prince C PArtificial Intelligence In Microbiology by Dr. Prince C P
Artificial Intelligence In Microbiology by Dr. Prince C P
 

Unit 3: Shannon entropy and meaning