Created:
Thu 11 Sep 2008
Last modified:
Note: This is an approximate syllabus; it may change at any time.
- Lecture 01, Fri, Sep 12 2008
- Admistrivia
- Introduction to Information Theory and its applications
- Lecture 02, Tue, Sep 16 2008
- Lecture 03, Fri, Sep 19 2008
- Probability primer II
- Homework 01 assigned
- Lecture 04, Tue, Sep 23 2008
- Entropy and its properties
- Reading: Cover and Thomas 2.1-2
- Lecture 05, Fri, Sep 26 2008
- Conditional entropy, relative entropy, mutual information
- Reading: Cover and Thomas 2.2-4
- Homework 01 due
- Homework 02 assigned
- Lecture 06, Tue, Sep 30 2008
- Chain rules, data processing inequality, Fano's inequality
- Reading: Cover and Thomas 2.5-6, 2.8, 2.10
- Lecture 07, Fri, Oct 03 2008
- Markov chains, entropy rate of stochastic processes
- Reading: Cover and Thomas 4
- Homework 02 due
- Homework 03 assigned
- Lecture 08, Tue, Oct 07 2008
- Compression I: codes and decodability, Kraft's inequality,
bounds on optimal codes
- Reading: Cover and Thomas 5.1-4
- Lecture 09, Fri, Oct 10 2008
- Compression II: more Kraft's inequality, block coding, Huffman codes
- Reading: Cover and Thomas 5.5-8
- Homework 03 due
- Homework 04 assigned
- Lecture 10, Tue, Oct 14 2008
- Compression III: twenty questions, arithmetic coding, randomness
- Reading: Cover and Thomas 5.7, 5.10, 5.11, 13.3
- Lecture 11, Fri, Oct 17 2008
- Asymptotic Equipartition Property (AEP) and its consequences
- Reading: Cover and Thomas 3
- Lecture 12, Tue, Oct 21 2008
- Information theory and statistics:
the method of types and applications
- Reading: Cover and Thomas 12.1-2
- Lecture 13, Fri, Oct 24 2008
- Lempel-Ziv, universal source coding
- Reading: Cover and Thomas 13.4
- Homework 05 due
- Homework 06 assigned
- Lecture 14, Tue, Oct 28 2008
- Channel coding
- Reading: Cover and Thomas 7
- Lecture 15, Fri, Oct 31 2008
- The maximum entropy method
- Reading: Cover and Thomas 12.1-2
- Homework 06 due
- Remainder of the term...
- Applications of information theory:
lectures and student presentation
Switch to:
jaa@ccs.neu.edu