{\rtf1\mac\ansicpg10000\uc1 \deff0\deflang1033\deflangfe1033{\upr{\fonttbl{\f0\fnil\fcharset256\fprq2{\*\panose 02020603050405020304}Times New Roman;}{\f4\fnil\fcharset256\fprq2{\*\panose 02000500000000000000}Times;}
{\f5\fnil\fcharset256\fprq2{\*\panose 02000500000000000000}Helvetica;}{\f20\froman\fcharset77\fprq0{\*\panose 00000000000000000000}Times-Italic{\*\falt Times};}}{\*\ud{\fonttbl{\f0\fnil\fcharset256\fprq2{\*\panose 02020603050405020304}Times New Roman;}
{\f4\fnil\fcharset256\fprq2{\*\panose 02000500000000000000}Times;}{\f5\fnil\fcharset256\fprq2{\*\panose 02000500000000000000}Helvetica;}{\f20\froman\fcharset77\fprq0{\*\panose 00000000000000000000}Times-Italic{\*\falt Times};}}}}
{\colortbl;\red0\green0\blue0;\red0\green0\blue255;\red0\green255\blue255;\red0\green255\blue0;\red255\green0\blue255;\red255\green0\blue0;\red255\green255\blue0;\red255\green255\blue255;\red0\green0\blue128;\red0\green128\blue128;\red0\green128\blue0;
\red128\green0\blue128;\red128\green0\blue0;\red128\green128\blue0;\red128\green128\blue128;\red192\green192\blue192;}{\stylesheet{\widctlpar\adjustright \loch\af4\hich\af4\dbch\f4\cgrid \snext0 Normal;}{\*\cs10 \additive Default Paragraph Font;}{\*\cs15 
\additive \ul\cf2 \sbasedon10 Hyperlink;}}{\info{\author Marc Talusan}{\operator Marc Talusan}{\creatim\yr2003\mo9\dy23\hr22\min23}{\revtim\yr2003\mo9\dy23\hr22\min23}{\version2}{\edmins0}{\nofpages3}{\nofwords703}{\nofchars4012}{\*\company MIT}
{\nofcharsws4927}{\vern115}}\widowctrl\ftnbj\aenddoc\formshade\viewkind1\viewscale125\pgbrdrhead\pgbrdrfoot \fet0\sectd \linex0\sectdefaultcl {\*\pnseclvl1\pnucrm\pnstart1\pnindent720\pnhang{\pntxta .}}{\*\pnseclvl2\pnucltr\pnstart1\pnindent720\pnhang
{\pntxta .}}{\*\pnseclvl3\pndec\pnstart1\pnindent720\pnhang{\pntxta .}}{\*\pnseclvl4\pnlcltr\pnstart1\pnindent720\pnhang{\pntxta )}}{\*\pnseclvl5\pndec\pnstart1\pnindent720\pnhang{\pntxtb (}{\pntxta )}}{\*\pnseclvl6\pnlcltr\pnstart1\pnindent720\pnhang
{\pntxtb (}{\pntxta )}}{\*\pnseclvl7\pnlcrm\pnstart1\pnindent720\pnhang{\pntxtb (}{\pntxta )}}{\*\pnseclvl8\pnlcltr\pnstart1\pnindent720\pnhang{\pntxtb (}{\pntxta )}}{\*\pnseclvl9\pnlcrm\pnstart1\pnindent720\pnhang{\pntxtb (}{\pntxta )}}\pard\plain 
\widctlpar\tx560\tx1120\tx1680\tx2240\tx2800\tx3360\tx3920\tx4480\tx5040\tx5600\tx6160\tx6720\adjustright \loch\af4\hich\af4\dbch\f4\cgrid {\tab \tab \tab \tab \tab \tab \tab \tab \tab \tab \tab Notes by Philip Meier
\par \tab \tab \tab \tab \tab \tab \tab \tab \tab \tab \tab pmmeier@mit.edu
\par \tab \tab \tab \tab \tab \tab \tab \tab \tab \tab \tab 9-23-03
\par 
\par \tab \tab \tab \tab \tab }{\scaps\fs28 Information Theory
\par }{
\par BACKGROUND- Major influences in using Information Theory for understanding brains
\par 
\par \tab SHANNON AND WEAVER " A Mathematical theory of Communication"
\par }\pard \widctlpar\adjustright {\tab }{\fs2 
\par }\pard \widctlpar\tx560\tx1120\tx1680\tx2240\tx2800\tx3360\tx3920\tx4480\tx5040\tx5600\tx6160\tx6720\adjustright {\tab }{(}{\field{\*\fldinst { HYPERLINK http://web.mit.edu/persci/classes/papers/}{\cf1 \hich\af4\dbch\af4\loch\f4 Shannon48.pdf}{ }{
{\*\datafield 
00d0c9ea79f9bace118c8200aa004ba90b02000000170000003700000068007400740070003a002f002f007700650062002e006d00690074002e006500640075002f007000650072007300630069002f0063006c00610073007300650073002f007000610070006500720073002f005300680061006e006e006f006e003400
38002e007000640066000000e0c9ea79f9bace118c8200aa004ba90b6e00000068007400740070003a002f002f007700650062002e006d00690074002e006500640075002f007000650072007300630069002f0063006c00610073007300650073002f007000610070006500720073002f005300680061006e006e006f006e
00340038002e007000640066000000}}}{\fldrslt {\cs15\ul\cf2 \hich\af4\dbch\af4\loch\f4 http://web.mit.edu/persci/classes/papers/Shannon48.pdf}}}{\cf1 \hich\af4\dbch\af4\loch\f4  }{)
\par \tab LETVIN   "What the Frog's Eye Tells the frog\rquote s Brain"
\par \tab (Cells that respond to dark moving spots are dubbed "fly detectors")
\par \tab (}{\field{\*\fldinst { HYPERLINK http://web.mit.edu/persci/classes/papers/}{\cf1 \hich\af4\dbch\af4\loch\f4 Lettvin59.pdf}{ }{{\*\datafield 
00d0c9ea79f9bace118c8200aa004ba90b02000000170000003700000068007400740070003a002f002f007700650062002e006d00690074002e006500640075002f007000650072007300630069002f0063006c00610073007300650073002f007000610070006500720073002f004c00650074007400760069006e003500
39002e007000640066000000e0c9ea79f9bace118c8200aa004ba90b6e00000068007400740070003a002f002f007700650062002e006d00690074002e006500640075002f007000650072007300630069002f0063006c00610073007300650073002f007000610070006500720073002f004c00650074007400760069006e
00350039002e007000640066000000}}}{\fldrslt {\cs15\ul\cf2 \hich\af4\dbch\af4\loch\f4 http://web.mit.edu/persci/classes/papers/Lettvin59.pdf}}}{\cf1 \hich\af4\dbch\af4\loch\f4  )}{
\par 
\par 
\par LAUGHLIN- enhancing information capacity in early vision
\par 
\par Retinal cells in fly eye have response properties that correspond to the average light levels in the flies environment.  
\par 
\par LMC= Large monopolar cells in insect retina. We can assume that the function of LMC is to transmit lossless signals. Thus, the maximum amount of information in the channel is when there is maximum entropy--
 this occurs when each type of response is equiprobable.   This theoretical notion of maximizing information in a channel predicts the observed relation between response property with respect to light intensity and the distribution of the avergage light i
ntensity in the fly's environment.  
\par 
\par ATTNEAVE- using information theory to talk about image perception
\par 
\par  Images have redundancies. To losslessly transmit signals efficiently, a visual system can make use of redundancies.
\par 
\par Attneave's guessing game involves 
moving pixel by pixel across and image, sepculating that an observer would be good at "guessing" the color of a pixel if they were given  1) prior samples of pixel colors from the image 2) rules to relate the prior samples to current pixels color.  For ex
a
mple, such a rule might be "If a diagonal line of color transition from black to brown is apparent, and I have been linearly moving across a patch of black, then I will guess brown at location that would correspond to the continuation of that diagonal lin
e."  Intuitively this shows that there are some regular pattern in a visual image such that the regularities are capable of being compressed by a visual system.
\par 
\par Two main discussions arose from this: one about the assumptions in Attneave's guessing game, and another about localized information.
\par 
\par It was suggested that the amount of information in a pixel (and the probability of getting it correct) depends on the method of structuring the questions (Raserized progression through an image) as well as the rules 
used to predict the image. (If the last one was black, the next one\rquote 
s probably black)  Would the strategy used by the interpreter influence the information contained in a pixel?  This led to speculating that questions like "Which pixel contains the most in
formation?" might be misleading.  However, to be fair, such a question does seem to be motivated by the notion that some areas in an image contain more information than other areas in an image.  For example, the  local area of a pixel does seem to be a go
od predictors of that pixel.  Also certain edges, like maximum curvature points DO seem to convey more  information than other edges, not to mention that areas with edges seem to have more information than areas without edges.  
\par 
\par Do maximum curvature points contain more information because of a mathematical relation between sampled points and  the original image, or because of a psychological relation between a feature in the signal and the visual system\rquote 
s ability to use that feature?  Many suspected that Attneave\rquote s cat example could be strongly influenced by the mathematical relation and not the psychological relation.  
\par 
\par Possibly the distinction between information about a signal and information about the world would help to address the problem of whether of not information is localized in an image.
\par 
\par Returning to Attneave, we are presented with the idea that Gestalt laws of grouping might be derived from redundancies in an image\emdash a homogeneity\emdash or a higher order pattern of regularity.  
\par 
\par A \lquote Sufficient Statistic\rquote 
  (in visual perception) refers to some minimal feature that is sufficient to fulfill a task.  Liking finding a signal corner is enough to determine that you are looking at a square if the only things you look at are squares and circle.  (Not quite the s
ame as the Statistician\rquote s term about what statistics you need to know to predict an election)
\par 
\par What is the function of this system in early vision?  There are two common answers: To losslessly transmit information or to usefully transform information.  
\par Information should be preserved so that the subtle differences in a signal can be used; yet information should also be thrown out because there is too much of it, (resulting in a computational explosion) and only some of it is useful (for any given task)

\par 
\par 
\par 
\par 
\par }\pard \widctlpar\adjustright {
\par }}