[Research] Curriculum-based Sensing Reduction in Simulation to Real-World Transfer for In-hand Manipulation
Physical Demonstration: Real-world validation of the Allegro Hand performing in-hand manipulation tasks using Curriculum-based Sensing Reduction.
Paper Abstract
Simulation to Real-World Transfer allows affordable and fast training of learning-based robots for manipulation tasks using Deep Reinforcement Learning methods. Currently, Sim2Real uses Asymmetric Actor-Critic approaches to reduce the rich idealized features in simulation to the accessible ones in the real world. However, the feature reduction from the simulation to the real world is conducted through an empirically defined one-step curtail. Small feature reduction does not sufficiently remove the actor's features, which may still cause difficulty setting up the physical system, while large feature reduction may cause difficulty and inefficiency in training. To address this issue, we proposed Curriculum-based Sensing Reduction to enable the actor to start with the same rich feature space as the critic and then get rid of the hard-to-extract features step-by-step for higher training performance and better adaptation for real-world feature space. The reduced features are replaced with random signals from a Deep Random Generator to remove the dependency between the output and the removed features and avoid creating new dependencies. The methods are evaluated on the Allegro robot hand in a real-world in-hand manipulation task. The results show that our methods have faster training and higher task performance than baselines and can solve real-world tasks when selected tactile features are reduced.
Simulation: Training in Isaac Gym simulation environment.
Sim-to-Real Transfer: Policy transferred and deployed on the physical Allegro Hand V4.
Physical Validation on Allegro Hand V4
The proposed Curriculum-based Sensing Reduction framework was validated on the Allegro Hand V4 as the primary physical multi-finger robotic platform, demonstrating successful real-world in-hand manipulation with progressively reduced sensing inputs.
• Real-World In-Hand Block Rotation
The Allegro Hand V4, a 4-finger robot hand with 16 DoFs, was used to rotate a block placed on its palm around the Z-axis to a randomly generated target position. In the real world, object position and rotation were captured using an ArUco marker and a Logitech C930e webcam at 30Hz, with a Kalman filter applied to denoise the signal.
• Superior Performance Over Baselines
CSR (2 steps) + DRG achieved the highest success rate across all simulation and real-world tests, outperforming the AAC baseline and prior works including OpenAI and NYU-RL in the average number of successful tasks completed in 30 seconds.
Key Takeaway
By utilizing the Allegro Hand V4 as the real-world validation platform, the CSR + DRG method demonstrated faster training and higher success rates compared to the AAC baseline — successfully performing in-hand block rotation tasks even when tactile sensor features were progressively removed during training.
PREV
