How optical tactile sensors help robotic hands

The camera can lose the object

A robot's main camera can locate a small tool before pickup. Once the gripper closes, the fingers may hide the exact contact and the tool's position within the grasp. MIT described this problem in its 2017 GelSight research account. Touch sensing supplied information at the point where the external view became obstructed. [1]

A camera looking into a soft surface

The GelSight design in that MIT account uses a transparent soft material with a reflective coating on its contact surface. Pressing an object into the surface changes its shape. Internal lighting and a camera record the resulting pattern, from which software can infer local surface geometry. [1]

The camera is observing the deforming sensor surface. It is not looking through a finger to see the whole object. In one reported tool-handling experiment, contact measurements helped a gripper locate a small screwdriver for removal and insertion. That is a specific use of local shape information, rather than a general claim that touch solves manipulation. [1]

What DIGIT measures

The original DIGIT sensor produces images of contact geometry. Its project documentation says those images can support estimates of normal force; markers can support estimates of shear force. These are quantities derived from the images, so the raw camera output and a calibrated force estimate should not be treated as the same measurement. [2]

DIGIT was designed for mounting on multi-finger hands, including the Allegro Hand, and has a published USB interface and open design files. The project also provides links to simulation and touch-processing software. This article concerns the original DIGIT, not the later Digit 360 sensor with a different set of sensing features. [2]

Moving a marble between fingers

The 2020 DIGIT paper evaluates learned control of glass marbles between fingers of a robotic hand. The controller uses tactile observations to represent the contact and predict the effects of actions. A marble is a useful example because its position relative to each fingertip matters throughout the movement. Detecting initial contact alone would leave much of that state unmeasured. [3]

The contact surface is a working part

The DIGIT paper tests gel coatings with an abrasive tip under a 1.7 N load and checks degradation after 5, 10 and 15 passes. Its replaceable gel is a practical maintenance feature. The test does not establish a factory service interval. Ask how a replacement surface is checked before returning the sensor to the task. [3]

GelSight's original DIGIT listing displayed US$355 and a four-to-six-week lead time when checked on 8 October 2026. That is a dated retail listing, not a delivered system price. A hand, mounting hardware, software work and any applicable charges are separate purchasing questions. [4]

Compare the signal with the task

For an insertion task, ask whether the sensor captures the relevant contact location and whether the controller uses it during movement. For repeated handling, ask how the surface is maintained and how a replacement is calibrated. Request the actual failure cases, including lost contact and damaged objects.