Per-sample gradient

[DINHQuangDung1999]

Hi! I am trying to implement an algorithm which requires knowing the squared variance of gradients. The paper can be found here. For this, I need to compute gradients of the network for each sample in my batch. As far as I understand, tape.backward() computes gradients for each sample and then averages them. Is there any flag which outputs gradients for each sample before their averaging? Thanks!