Abstract: This work studies the problem of shape reconstruction and object localization using a vision-based tactile sensor, GelSlim. The main contributions are the recovery of local shapes from contact, an approach to reconstruct the tactile shape of objects from tactile imprints, and an accurate method for object localization of previously reconstructed objects. The algorithms can be applied to a large variety of 3D objects and provide accurate tactile feedback for in-hand manipulation.
Results show that we can reconstruct the shape of objects with high accuracy and do real time object identification and localization, opening the door to more sophisticated and reactive manipulations guided by tactile sensing.

Related Publications

Summary video

The tactile sensor: GelSlim

Our camera-based tactile sensor, GelSlim, provides high-resolution tactile imprints.

Learning local shape

We provide some basic code that computes given a tactile image, the resulting local contact as a heightmap, and the 3D models of the objects used during training. Code and 3D models .

Mapping objects

Data collection for the flaslight.

Object localization

Localization and tracking in-hand of the flashlight.