Assigned:
Friday, September 23, 2011
Due:
Friday, September 30, 2011
Last modified:
04/03/2025 05:47:08
General Instructions
- Feel free to work with others on this assignment. However, you must
acknowledge with whom you worked, and you must write up your own solutions.
- All of the problems below are from the text, Cover and Thomas, 2nd edition, or refer
to problems from the text.
Problems
Points: 20 pts per problem.
- Problem 2.1
- Repeat Problem 2.1, part (a), for a biased coin where the
probability of heads is p.
- Problem 2.27. Note: From the grouping property and two other (simpler) properties,
one can derive entropy, axiomatically. See Problem 2.46.
- Problem 2.10
Note: 2H(X) is referred to as
the perplexity of X. It is a measure of the
"effective" size of the range of X, with respect to the
underlying distribution over this range.
- Problem 2.28
Hint: You will likely make use of a convexity argument.
See CT Section 2.6 for the definition of convexity; we will cover this
in class as well.