CS6800 11F: Homework 02

Last modified:

General Instructions

1. Feel free to work with others on this assignment. However, you must acknowledge with whom you worked, and you must write up your own solutions.

2. All of the problems below are from the text, Cover and Thomas, 2nd edition, or refer to problems from the text.

Problems

1. Problem 2.1
2. Repeat Problem 2.1, part (a), for a biased coin where the probability of heads is p.
3. Problem 2.27. Note: From the grouping property and two other (simpler) properties, one can derive entropy, axiomatically. See Problem 2.46.
4. Problem 2.10

   Note: 2^H(X) is referred to as the perplexity of X. It is a measure of the "effective" size of the range of X, with respect to the underlying distribution over this range.

5. Problem 2.28

   Hint: You will likely make use of a convexity argument. See CT Section 2.6 for the definition of convexity; we will cover this in class as well.