You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Hi, I have trained multiple LP models now; single camera supervised, semi-supervised, ctx, multi-camera supervised, and multi-camera ctx. One issue that persists across all models is LP's insistence on tracking landmarks which are not occluded but completely out of the field of view. This is something Deeplabcut deals with quite well by giving low-confidence values to landmarks in these frames; such low-confidence landmarks are then not plotted in video inference. This does not seem to be reflected in LP. How do you propose I go about improving this? This issue poses problems when triangulating the landmarks across different camera views. E.g., In the attached frame (during rearing), the snout likelihood for DLC is 0.003842 and for LP it is 0.906632.
The text was updated successfully, but these errors were encountered:
DavidGill159
changed the title
pseudo tracking when animal exits field of view
over confident pseudo tracking when animal exits field of view
Jul 2, 2024
@DavidGill159 -- the way we calculate the confidence score is different from DeepLabCut, and our scores will by definition be higher. Using our scaling, it is advised to use 0.95 to threshold low confidence keypoints. Can you please try this and tell us if it helps?
@DavidGill159 this is an issue that we're aware of and trying out several fixes. In the meantime, a couple points to keep in mind:
if you use the Pose PCA loss the model will try to localize the keypoint, even if it is occluded
in this case, I would recommend looking at the Pose PCA loss itself (output as video_preds/<video_name>_pca_singleview_error.csv in your model folder). this error should be very high for a frame like the one you've shared, indicating that there is an issue with the prediction
Hi, I have trained multiple LP models now; single camera supervised, semi-supervised, ctx, multi-camera supervised, and multi-camera ctx. One issue that persists across all models is LP's insistence on tracking landmarks which are not occluded but completely out of the field of view. This is something Deeplabcut deals with quite well by giving low-confidence values to landmarks in these frames; such low-confidence landmarks are then not plotted in video inference. This does not seem to be reflected in LP. How do you propose I go about improving this? This issue poses problems when triangulating the landmarks across different camera views. E.g., In the attached frame (during rearing), the snout likelihood for DLC is 0.003842 and for LP it is 0.906632.
![image](https://private-user-images.githubusercontent.com/116177739/344962112-2209988c-bb5f-4a5d-96ba-dcdde2de260e.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MjUyNjk3NDMsIm5iZiI6MTcyNTI2OTQ0MywicGF0aCI6Ii8xMTYxNzc3MzkvMzQ0OTYyMTEyLTIyMDk5ODhjLWJiNWYtNGE1ZC05NmJhLWRjZGRlMmRlMjYwZS5wbmc_WC1BbXotQWxnb3JpdGhtPUFXUzQtSE1BQy1TSEEyNTYmWC1BbXotQ3JlZGVudGlhbD1BS0lBVkNPRFlMU0E1M1BRSzRaQSUyRjIwMjQwOTAyJTJGdXMtZWFzdC0xJTJGczMlMkZhd3M0X3JlcXVlc3QmWC1BbXotRGF0ZT0yMDI0MDkwMlQwOTMwNDNaJlgtQW16LUV4cGlyZXM9MzAwJlgtQW16LVNpZ25hdHVyZT01YWRmYTY2ODI2YTJlNzY3ZjhjZWE3YzU2YzYzZTZjN2I1ZTE4OTIyMDA2Y2I2ZjBmMzY3MTgzZmNkYTBlYTk3JlgtQW16LVNpZ25lZEhlYWRlcnM9aG9zdCZhY3Rvcl9pZD0wJmtleV9pZD0wJnJlcG9faWQ9MCJ9.ijq_jEw5nEtyc5nZb2XuLpSP_4-so5x7uTBdbyIZ8qY)
![image](https://private-user-images.githubusercontent.com/116177739/344962147-3f05481d-33e0-4c15-ab59-fd11dbe09ae1.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MjUyNjk3NDMsIm5iZiI6MTcyNTI2OTQ0MywicGF0aCI6Ii8xMTYxNzc3MzkvMzQ0OTYyMTQ3LTNmMDU0ODFkLTMzZTAtNGMxNS1hYjU5LWZkMTFkYmUwOWFlMS5wbmc_WC1BbXotQWxnb3JpdGhtPUFXUzQtSE1BQy1TSEEyNTYmWC1BbXotQ3JlZGVudGlhbD1BS0lBVkNPRFlMU0E1M1BRSzRaQSUyRjIwMjQwOTAyJTJGdXMtZWFzdC0xJTJGczMlMkZhd3M0X3JlcXVlc3QmWC1BbXotRGF0ZT0yMDI0MDkwMlQwOTMwNDNaJlgtQW16LUV4cGlyZXM9MzAwJlgtQW16LVNpZ25hdHVyZT01NWFjZjlhNzFmYjA0ZjZiODQ5ZTg1MmNmNGM2Yjg5ZTA1NmY1ZjY1MzI4NDZlYTFlODgyMjE4MGI4ZGM4ZmRjJlgtQW16LVNpZ25lZEhlYWRlcnM9aG9zdCZhY3Rvcl9pZD0wJmtleV9pZD0wJnJlcG9faWQ9MCJ9.KKaYQ8ALgyNUEWNIVpxyCRgxLq3gcvWfr62fsJMk-AE)
The text was updated successfully, but these errors were encountered: