In the absence of enough local evidence about an object's identity, the scene structure and prior knowledge of world regularities provide additional information for recognizing and localizing an object. Even when objects can be identified via intrinsic information, context can simplify the object discrimination by cutting down on the number of object categories, scales and positions that need to be considered.

The context challenge consists in trying to detect an object using exclusively contextual cues. Using context features can you predict if the object is present or absent? can you predict its location and scale in the image?

How can this work? There is a relationship between the aspect of the objects in a scene, and the aspect of the scene itself. For instance, the point of view of cars is correlated with the orientation of the street. But also, the location of the ground in the scene is correlated with the location of the objects in the scene. The goal is to devise algorithms that make use of these (and other) scene cues to predict the location of objects of interest.

Face detection in context: This image shows the output of our algorithm when trained to detect faces. Because the algorithm is only based on contextual features (there is no local face detection), the selected regions are large. However, the algorithm succesfully reduces the search space by selecting the image regions more likely to contain the target.

Car detection in context: This movie illustrates how context is used to select the image region likely to contain cars, and then a car detector is evaluated in those locations.