Bench2Dex Standardizes Tactile Robot Manipulation

Comparing manipulation policies across robotic hands with different kinematic chains or finger counts previously forced researchers to build bespoke environments for every piece of hardware. Bench2Dex eliminates this overhead by defining a unified observation format for visuo-tactile data. It maps raw 3D contact points and force vectors from the simulator onto a structured 2D grid, essentially creating a depth map where pixel values represent contact intensity. This grid projection discards the specific joint structure of the hand, forcing the learning agent to interpret the task through the localized image of the touch rather than the specific motor geometry of the fingers.

Projection and Morphology

The framework treats tactile sensing as a general image signal. By projecting raw contact geometry into this 2D grid, the system provides a consistent input schema regardless of whether the hardware has three fingers or five. The benchmark provides 1.3K human-teleoperated demonstrations to train baseline policies. This allows for an apples-to-apples comparison of how models like ACT or Diffusion Policy perform across different robots in the same environment. You can swap a complex anthropomorphic hand for a simple parallel-jaw gripper, and the tactile signal arrives in the exact same tensor format, allowing you to study whether a policy performs differently due to its internal generalization capabilities rather than environment-specific input differences.

Testing Geometric Understanding

The benchmark categorizes perturbations into axes of invariance and equivariance to expose whether a policy has learned the task physics or is merely overfitting to visual cues. An invariance test might involve applying a slight rotation to the object before the grasp. If the policy is invariant, it should produce an identical contact sequence because the target grasp location relative to the object is unchanged. An equivariance test requires the policy to rotate its reach trajectory in response to a shifted object position. By observing whether the agent maintains a successful grasp under these programmatic transformations, you can determine if the model understands the task geometry.

This framework provides a controlled environment to study how policy performance shifts across different hardware morphologies. It allows you to examine if an agent successfully transfers its learned contact dynamics from a high-degree-of-freedom hand to a restricted one. The platform facilitates the isolation of factors like observation bandwidth or task complexity by keeping the core task logic constant while varying the physical constraints of the robotic end-effector.

Deployment and Development

The primary limitation remains the gap between simulated tactile signals and the non-linear noise present in physical sensors like GelSight. While the image-like representation simplifies algorithm development, it does not currently simulate the latency or specific sensor artifacts found on a physical robot. Whether these policies translate to hardware depends on how effectively they handle sensor noise during fine-grained manipulation.

The project code is hosted on GitHub, and the authors have documented the XML-based robot definition requirements for adding new models to the interface. If you are integrating a new hand design, the system expects a standard configuration that aligns the tactile sensors to the projection grid. The future of this project depends on community contributions to the library of supported robot models and the release of real-world validation data for policies trained within this simulated interface.