Who: Tomaso Poggio, Lorenzo Rosasco, Thomas Serre

When: Jan 12-13, 11:00am-12:30pm and 2:00pm-03:30pm

When compared with the learning abilities of biological organisms, modern learning theory is confronted with the so called poverty of stimulus problem: organisms can learn complex tasks from much fewer examples than our present learning theory and learning algorithms predict.

A comparison with real brains suggests that the key to such problem could lie in the kind of learning architecture used.  Classical  “learning algorithms” correspond to one-layer architectures whereas the organization of cortex (e.g., the visual cortex) is strongly hierarchical.  Up-to now learning theory does not offer any general argument in favor of hierarchical learning and to add to the puzzle, hierarchical learning systems seem to exhibit superior performance in some engineering applications.

After an introduction to the problem of machine learning and data representation, with particular emphasis on image classification problems, we will discuss the hierarchical nature of learning from the perspective of human learning. This will serve as the main motivation for presenting several recently proposed hierarchical learning machines and describing in more detail a quantitative model that accounts for the circuits and computations of the feedforward path of the ventral stream of visual cortex. Finally we describe a mathematical formalization of such a model, which is a first step towards understanding hierarchical learning within the framework of learning theory.

Principles and Algorithm for Learning High Dimensional Data

We first discuss the problem of learning as a statistical inference problem within the framework of learning theory. After introducing some fundamental concepts about learning and generalization, we present general principles allowing to design learning machines. In particular we focus on the problem of learning  from complex high dimensional data as in the case of image understanding and discuss why hierarchical learning might be a key to such problems.

Hierarchical Learning in Brain and Machines

We will briefly review the anatomy and the physiology of the visual cortex of primates and then describe a class of quantitative models of the ventral stream for object recognition, which have been developed during the last two decades. Those models are heavily constrained by physiology and biophysics and have been recently shown to be quite successful in explaining several physiological data across different visual areas. We will discuss their performance and architecture from the point of view of state-of-the-art computer vision systems. Surprisingly, such models also mimic the level of human performance in difficult rapid image categorization tasks in which human vision is forced to operate in a feedforward mode. Finally we will focus on current topics of research on the computational architecture of visual cortex and discuss their implications for advancing computer vision technology.

Towards a Mathematical Model of Hierarchical Learning?

Here we present recent attempts to develop a mathematical theory of hierarchical learning architectures of the general type found in the visual cortex. We discuss in more details an abstract and compact mathematical description of a hierarchical architecture for learning, associated to the computational model of the initial feedforward flow of information in the primate visual system that we previously introduced.

>> SYLLABUS

• General introduction (TS) [slides]
• Introduction to statistical learning theory (LR) [slides]
• Reverse-engineering the visual system I: Visual neuroscience (TS) [slides]
• Reverse-engineering the visual system II: Computational models (TS) [slides]
• Towards a mathematical theory of vision (LR) [slides]