-
Notifications
You must be signed in to change notification settings - Fork 1.3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Any plan to support torch.func
?
#1796
Comments
We aren't planning torch.func support. It seems to me that the method in #1533 should work fine for _PointFaceDistance - feel free to post the code you have and maybe we can figure out what's wrong. |
@bottler Thanks for the reply! The error msg
I did follow #1533 to modify the class with |
@bottler I wrote a toy example that follows this pytorch3d tutorial with following modifications:
Here is the code. It is self-contained and will download
When
I don't know how to start debugging this as |
Just in intuition based on the traceback, especially here:
I think that jac_fn will be called in the same way that the forward path is called with vmap. So, there might be an error in the jac_fn function's vmapping, which you might specify in the same way as you did for the forward path. |
@TimoRST Thanks for the inputs! I never thought of it and will look into it. Just one follow up, Thanks! |
I don't know that. I would first debug into that function to see if it really is called with those batched tensors which cause the error. |
@TimoRST Thanks. I will try it out. Can I ask what your use case was to use Thanks! |
I wanted to implement something like WGICP (https://arxiv.org/abs/2209.09777), but I couldn't scale it because my graphics card was too small, so I didn't get comparable results. |
Here are some findings. Conclusion:
Here is a hacked version to verify my point though the math is probably wrong. When using
@bottler would you be able to confirm what I said above is correct? And if this is the case, it seems changing internal code of Thanks! |
I don't know enough I'm afraid about functorch |
I am trying to compute full jacobian using
jacrev
orjacfwd
fromtorch.func
. Part of the loss function uses_PointFaceDistance
. Out of the box,pytorch3d
does not supporttorch.func
. The closest references I can find so far are #1636 and #1533.The problems I am having are
jacrev
, it throws an exceptionRuntimeError: Cannot access data pointer of Tensor that doesn't have storage
at the line of_C.point_face_dist_backward
.- I followed "RuntimeError: Cannot access data pointer of Tensor that doesn't have storage" when using autograd with knn #1533 and implemented
vmap
for_PointFaceDistance
, butjacrev
still hits the line of_C.point_face_dist_backward
first, so same error. "RuntimeError: Cannot access data pointer of Tensor that doesn't have storage" when using autograd with knn #1533 happened in the forward path, so actually same error but different cause.jacfwd
, it throws an exception of missingjvp
method.- I don't quite know how to implement
jvp
as I have not found much info on what exactly should go in to this function.The only working method is to call
torch.autograd.functional.jacobian(vectorize=False)
which is very slow. And when turn onvectorize=True
, it runs into the same issues as above.My questions are:
torch.func
? If I can get some guidance frompytorch3d
team, I am happy to collaborate on this.Thanks!
The text was updated successfully, but these errors were encountered: