Physics-accurate 3D assets for robotics simulations from any input

Hacker News

Share card

Physics-accurate 3D assets for robotics simulations from any input

In the past 4 years, we developed a platform generating 3D visuals and deploying AR Try-on experiences into large footwear and retail companies. We focused on high-fidelity meshes and textures since the point was giving shoppers a realistic visual. Then robotics companies started to approach us and they asked for thousands of 3D assets that they can use in creating different simulation environments. This is crucial for them so they can achieve better sim-to-real transfer with domain randomization (train a humanoid in a million different kitchens so a real one is not a surprise when it is deployed in the real world). However, we realized nice geometry and pretty textures are not enough for sim engines as they lack physics properties. So we developed a new pipeline creating SimReady assets in OpenUSD and MJCF that plug directly into Isaac Sim and MuJoCo. - Mass: volume from the mesh, density from material classification by a vision-language model (approach adapted from NeRF2Physics), multiplied together. - Center of mass, static/dynamic friction coefficients and restitution from the same material classification and estimation priors. - Collision meshes via CoACD; adaptive hull count is in progress. - Geometry and texture optimization, plus file format conversions. - We are currently focused on physics property estimation for props. Next step: articulated objects It comes with free credits on easy Google sign-up so you can give it a test. No credit cards required. Still early. All we'd ask is honest feedback. What works, what doesn't, what you wish it did differently. That's worth more to us than anything right now.