The target encoder is always a copy of the online encoder?

[lishen]

byol-pytorch/byol_pytorch/byol_pytorch.py

Line 258 in 6717204

target_encoder = self._get_target_encoder() if self.use_momentum else self.online_encoder

byol-pytorch/byol_pytorch/byol_pytorch.py

Line 225 in 6717204

target_encoder = copy.deepcopy(self.online_encoder)

These two lines of code implicate that the target encoder is always set to be a copy of the online encoder no matter the momentum is turned on or off. Am I reading this correctly?

[lb-97]

Shouldn't it rather be a shallow copy? Otherwise the weights of encoder are copied at every forward irrespective of momentum being on