6.291

Statistical Inference, Statistical Mechanics and the

Relationship to Information Theory

Fall 2004

Sanjoy Mitter, Lecturer

Texts | Problem Sets | Handouts

---

CLASS WILL BE HELD IN 4-153

Tuesday & Thursday, 2:30 - 4:00

 

SCHEDULE FOR REMAINDER OF Fall 2004 (see below)

---

 

Recent work on Statistical Inference such as Statistical Inference on Graphs, Minimum Description Length Principle for Inference, Coding and Decoding has shown striking connections to Statistical Mechanics and Information Theory.  The purpose of these lectures is to attempt to give a systematic introduction to these developments. 

 

Topics include:

 

Relative Entropy, Entropy and some basic theorems of Large Deviations.  The Variational Description of Gibbs Measures.  Gibbs Variational Principle and the Shannon-McMillan-Breiman Theorem.  Bayesian Inference viewed as minimization of Free Energy.  Information Flow and Entropy Production in the Kalman-Bucy Filter and its Nonlinear Generalizations.  Large Deviations and Shannon’s Noisy Channel Coding Theorem.  Minimum Description Length Principle for Inference.  Real-time Information Theory and its possible role in Control, Networks and Biology.

 

 

Remaining 6.291 Lectures as of 11/10/2004

11/16/04 (Tues) Lecture rescheduled to 12/1/04

11/18/04 (Thur) Interacting Markov Chains

11/23/04 (Tues) Graphical Models and Belief Propagation, and/or Metropolis Algorithm

11/25/04 Thanksgiving Holiday

11/30/04 (Tues) Graphical Models, Belief Propagation and Statistical Mechanics

12/1/04 (Weds) Graphical Models, Belief Propagation and Statistical Mechanics (1-2:30) (Make-up Lecture)

12/2/04 (Thurs) Disordered Systems, Replica Method

12/7/04 (Tues) THIS LECTURE WILL BE HELD IN 4-153 AS USUAL

12/8/04 (Weds) NO LECTURE

12/9/04 (Thurs) Last Lecture