The document covers key concepts in information theory including entropy, joint and conditional entropy, and the chain rules associated with these concepts. It discusses relative entropy, also known as Kullback-Leibler distance, and mutual information, detailing theorems and corollaries related to these topics. Overall, it provides a foundational overview of how these elements interact in the field of information theory.

1. Information Theory 
Entropy 
Joint Entropy and Conditional
Entropy
Chain Rules for Entropy, Relative
Entropy amd Mutual Information.
Lemma -
Joint Entropy
Conditional Entropy
Theorem (Chain Rule)
Corollary
Chain rule for entropy
Chain rule for information
Chain rule for relative Entropy
Relative Entropy and Mutual
Information
Relative entropy 
Relative Entropy/ Kullback -
Leibler (KL) distance
Mutual Information 
Mutual Information
Mutual Information and Entropy