Achieving diverse and stable dexterous grasping for general and deformable objects remains a fundamental challenge in robotics, due to high-dimensional action spaces and uncertainty in perception. In this paper, we present D3Grasp, a multimodal perception-guided reinforcement learning framework designed to enable Diverse and Deformable Dexterous Grasping. We firstly introduce a unified multimodal representation that integrates visual and tactile perception to robustly grasp common objects with diverse properties. Second, we propose an asymmetric reinforcement learning architecture that exploits privileged information during training while preserving deployment realism, enhancing both generalization and sample efficiency.Third, we meticulously design a training strategy to synthesize contact-rich, penetration-free, and kinematically feasible grasps with enhanced adaptability to deformable and contact-sensitive objects. Extensive evaluations confirm that D3Grasp delivers highly robust performance across large-scale and diverse object categories, and substantially advances the state of the art in dexterous grasping for deformable and compliant objects, even under perceptual uncertainty and real-world disturbances. D3Grasp achieves an average success rate of 95.1% in real-world trials—outperforming prior methods on both rigid and deformable objects benchmarks.
Our evaluation includes a comprehensive set of objects. The test objects span diverse categories based on material properties and structural characteristics:
Grasping deformable objects poses significant challenges due to their compliant behavior and tendency to undergo shape changes under applied forces. The D3Grasp system addresses these challenges by achieving stable grasps while simultaneously minimizing induced deformation.
D3Grasp demonstrates exceptional adaptability in constrained environments where traditional grasping methods often struggle. Our approach leverages multimodal perception to navigate tight spaces and avoid obstacles while maintaining stable grasps.