This document discusses joint entropy, conditional entropy, and mutual information. It defines joint entropy as the entropy of two random variables together. Conditional entropy is defined as the entropy of one random variable given knowledge of the other. It provides examples of when conditional entropy would be 0 or equal to the entropy without conditioning. The document also mentions a case study to demonstrate these information theory concepts.

1. Joint, Conditional, & Mutual Information
& A Case Study
Unit 3

2. Joint Entropy

3. Joint Entropy

4. Joint Entropy

5. Joint Entropy

6. Conditional Entropy

7. Conditional Entropy
● H(Y|X) = 0, if and only if, the value of Y is completely
determined by the value of X
● H(Y|X) = H(Y), if and only if, Y and X are independent
random variables
○ X does not provide any information about Y
○ Entropy of Y given X is simply equal as asking for the
Entropy of Y directly with no access to X

8. Mutual Information

10. A Case Study

11. A Case Study

12. A Case Study

13. Expectation:
Testable hypotheses:
A Case Study