Statistical Inference, Statistical Mechanics and the
Relationship to Information Theory
CLASS WILL BE HELD IN 4-153
Tuesday & Thursday, 2:30 - 4:00
SCHEDULE FOR REMAINDER OF Fall 2004 (see below)
Recent work on Statistical Inference such as Statistical Inference on Graphs, Minimum Description Length Principle for Inference, Coding and Decoding has shown striking connections to Statistical Mechanics and Information Theory. The purpose of these lectures is to attempt to give a systematic introduction to these developments.
Relative Entropy, Entropy and some basic theorems of Large Deviations. The Variational Description of Gibbs Measures. Gibbs Variational Principle and the Shannon-McMillan-Breiman Theorem. Bayesian Inference viewed as minimization of Free Energy. Information Flow and Entropy Production in the Kalman-Bucy Filter and its Nonlinear Generalizations. Large Deviations and Shannon’s Noisy Channel Coding Theorem. Minimum Description Length Principle for Inference. Real-time Information Theory and its possible role in Control, Networks and Biology.
to be held in ROOM 5-134