over confident pseudo tracking when animal exits field of view

[DavidGill159]

Hi, I have trained multiple LP models now; single camera supervised, semi-supervised, ctx, multi-camera supervised, and multi-camera ctx. One issue that persists across all models is LP's insistence on tracking landmarks which are not occluded but completely out of the field of view. This is something Deeplabcut deals with quite well by giving low-confidence values to landmarks in these frames; such low-confidence landmarks are then not plotted in video inference. This does not seem to be reflected in LP. How do you propose I go about improving this? This issue poses problems when triangulating the landmarks across different camera views. E.g., In the attached frame (during rearing), the snout likelihood for DLC is 0.003842 and for LP it is 0.906632.

[danbider]

@DavidGill159 -- the way we calculate the confidence score is different from DeepLabCut, and our scores will by definition be higher. Using our scaling, it is advised to use 0.95 to threshold low confidence keypoints. Can you please try this and tell us if it helps?

[DavidGill159]

Hi Dan thanks for getting back to me! I just tried that, but I'm afraid the video prediction still looks the same.

[themattinthehatt]

@DavidGill159 this is an issue that we're aware of and trying out several fixes. In the meantime, a couple points to keep in mind:

• if you use the Pose PCA loss the model will try to localize the keypoint, even if it is occluded
• in this case, I would recommend looking at the Pose PCA loss itself (output as video_preds/<video_name>_pca_singleview_error.csv in your model folder). this error should be very high for a frame like the one you've shared, indicating that there is an issue with the prediction