Has visual fidelity outpaced dynamic fidelity in tactile simulation for sim-to-real transfer of policies? Introducing HydroShear 🏄‍♂️ : a hydroelastic tactile shear simulation for training zero-shot sim-to-real tactile policies in contact-rich tasks where fingertip force and shear matter most! Webpage: hydroshear.github.io Robot videos in 1x 🧵👇

🖐 In tactile simulation, visual realism of tactile image rendering has progressed rapidly. In contrast, accurately capturing fingertip shear from object-sensor interactions still has a large sim-to-real gap. To that end, we introduce HydroShear, an SDF-based non-holonomic hydroelastic tactile shear simulator that models: - Full SE(3) object-sensor interactions - Stick-slip transitions - Path-dependent force/shear build-up - Arbitrary complex geometries

🌊 How does HydroShear work? We extend hydroelastic contact models using SDFs to track the displacement of an object’s surface points across the sensor membrane during physical interactions. This path-dependent contact model allows us to capture and simulate complex, real-world behaviors like stiction, slippage, and hysteresis.

🤔 Can HydroShear accurately model real-world shear? We set up a digital twin environment using a Kuka robot with a spherical indenter that interacts with GelSight Minis and collects a wide range of shear data in the real-world. We compare against several baselines and show that HydroShear consistently reproduces real-world shear induced by dilation, translational shear, twist, and rolling motions!

+ HydroShear is physics engine-agnostic and can be run interactively on a web visualizer like Viser!

🧠Training: HydroShear is GPU-parallelizable and we train tactile-driven RL policies in large-scale simulation using asymmetric actor-critic distillation from TacSL and zero-shot deploy in the real-world on Franka.