Case of no-conflict

[ogencoglu]

Is it so that if there is no conflict in gradients in an instance, all these methods will not "trigger" and it will be regular gradient update?

[PierreQuinton]

Assuming you are talking about aggregators that are non-conflicting: Definitely true for UPGrad, DualProj. Definitely not true for MGDA and NashMTL.

Note that if you goal is to find a Pareto optimal point, and the Pareto front is not trivial, then an (interior) Pareto optimal point $x$ satisfies strong stationarity: There exists $0\lneq w$ such that $Jf(x)^\top w =0$, this means that there must be some large and unsolvable conflict. In general, you should observe that the conflict (e.g. cosine similarity between the gradients of two losses) typically increases as you optimize.

[ogencoglu]

Let's say I am training a neural net with a loss function that is combination of 3 different loss terms. And I want to hyperparam optimize the loss weights (e.g. L = 0.3 x Loss1 + 0.2 x Loss2 + 0.5 x Loss3). One or two of those multipliers can come 0 and for that experiment the total loss may come as only one. Or maybe Loss1 and Loss2 are never conflicting ever (I don't know).

Can I set up my codebase to use UPGrad no matter what and it works? Or do I have to write the if-else stuff to fallback to standard torch if there is single loss term or losses are not conflicting etc.?

[PierreQuinton]

I am not entirely sure I understand your question, let me know if I got this wrong.

The main reason to scale losses is to ensure that the gradients are not too imbalanced (they cannot typically be completely balanced but that's already a start). One way to do that is to scale the loss vector, i.e. $[\alpha L_1, \beta L_2, \gamma L_3 ]^\top$. If say $\alpha=0$, then the first gradient of this vector-valued loss is $0$ and therefore does not conflict with any vector, it will then be completely ignored by UPGrad. But still, at that point, $L_1$ might not have zero gradient, and you may want to have a decision that does not conflict with its gradient, even if you give it a weight of $0$. To do that, you can use the pref_vector of UPGrad and set it to torch.tensor([0.0, 0.5, 0.5]) for instance. The decision taken by UPGrad will then be the combination with the provided weights of the projections of the gradients onto the dual cone (i.e. the cone of non-conflicting directions).

So my best guess is that you should simply parametrize UPGrad with a pref_vector set to your coefficients and then use torchjd.backward on three losses.

Let me know if that makes sense.

[ogencoglu]

it will then be completely ignored by UPGrad

that answers my question. Thanks for the swift help. Appreciated.