SlipSense Detects Robotic Slip Using Cross-Modal Attention
SlipSense uses cross-modal attention to detect robotic grip slippage by correlating high-frequency vibration data with tactile pressure maps.
SlipSense addresses the latency problem in robotic grasping by predicting the onset of object slip before it occurs. The system operates at 240 Hz and detects 76% of slip events with an average latency of 23.1 ms. It relies on two data streams: a 32x32 piezoresistive pressure array measuring contact force and a 3-axis MEMS accelerometer sampling at 8 kHz. By identifying when physical contact destabilizes through these signals, the robot anticipates a drop instead of reacting after the object has begun to move.
The Mechanism of Cross-Modal Attention
Tactile sensors struggle to balance high-speed signal processing with the spatial resolution required for manipulation. SlipSense uses the accelerometer signal as a global indicator of frictional instability to prioritize specific regions in the pressure map. The model correlates the high-frequency vibrational energy detected by the accelerometer with the spatial distribution of pressure. When the system detects the spectral signature of an incipient slip—a surge in energy as the object begins to shift against the sensor skin—it assigns higher weights to the pressure patches currently experiencing that mechanical transition.
This process functions by focusing on the 'leading edge' of the contact patch. As an object begins to slip, the pressure profile shifts and the contact force drops specifically at the rear of the contact zone. The attention mechanism identifies this spatial gradient by comparing the global vibrational state to the local pressure values. By isolating these specific contact points, the system distinguishes between a static hold and the initiation of a slide. If the vibration indicates instability and the pressure map shows a corresponding loss of force at the boundary of a contact patch, the robot flags an impending slip.
| Feature | Specification | Role |
|---|---|---|
| Piezoresistive Array | 32 x 32 | Spatial pressure mapping |
| MEMS Accelerometer | 3-axis @ 8 kHz | Friction-induced vibration detection |
| Detection Latency | 23.1 ms (average) | Prediction speed |
| Zero-Shot Capability | Yes | Generalization to unseen objects |
Spatial Invariance Through Convolutions
Traditional tactile models often map sensor inputs to fixed grid indices, binding software to a specific hardware layout. SlipSense avoids this by utilizing convolutional layers to process the sensor array. These kernels act as sliding windows that scan the entire sensor surface for local features. By applying the same set of weights across every patch, the model learns to recognize the shape of a slip signature regardless of where it appears on the physical skin. This ensures the network is invariant to the pixel address of a signal.
This architecture translates well to different physical hardware. The researchers trained the model on 1.4 million frames using the UMI hand and deployed the same weights on the Tesollo hand without further training. This demonstrates that the model learns universal characteristics of tactile slip rather than overfitting to the geometry of a particular sensor. SlipSense requires a distinct acoustic signature detectable at 8 kHz. Contact with highly damped materials attenuates this signal, which reduces detection reliability for soft or viscous objects. Future progress depends on whether these models can maintain performance when the acoustic signature is dampened by materials that absorb high-frequency vibrations rather than transmitting them to the sensor.