Tune simulation

[yihao19]

Hello,

I am wondering how if scene parameter can also become a part of model parameter that can be trained using the data generated by the simulation.

Best,
Yihao

[yihao19]

"Bi-directional communication channels allow Blender simulations to adapt during network training" I am very curious how could Blender adapt during network training. Passing model trainable parameter to change the scene settings or... Thank you

[cheind]

@yihao19 yes that's possible and demonstrated here. It requires back-prop through non-differentiable render function of Blender, which I documented there

[yihao19]

@cheind Thank you for the quick response, I am actually trying to pass physical parameters of Cloth modifier, and trying to optimize them using l2 between modified mesh and GT mesh, In my case, would it also be possible? I am feeling like I need the "modifier" part to be differentiable to do that.

[cheind]

yes, that would be required, but it probably isn't. You might use one of the gradient estimator 'tricks' to avoid that - reparametrization or score function gradient estimation are keywords. I used the latter to avoid making rending differentiable.