-
Notifications
You must be signed in to change notification settings - Fork 75
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Eval on IMC - I may help #7
Comments
Hi @ducha-aiki , Actually, I had them roughly coded (80%) some time ago, but unfortunately i never get time to start testing them 😅😅😅. |
Sounds good! Just ping me here, or at email: [email protected] |
Hi @ducha-aiki , Sorry for the late update.. I finally got time to clean some code and push them. |
I have looked at the code structure and it seems to be very good. One thing, which I need to discuss with the other IMC orgs, is that for the stereo what your code does in order to generate "keypoints", violates the pairwise assumption. And if one does it anyway, then it would be only rational to perform some kind of optimization or bundle adjustment. So maybe the correct way is for us to change the expected format to be suitable for pairwise methods. I will update you soon. |
OK, here is an update: I modified the benchmark code in the branch (not yet merged), which allows submission of "pairwise" keypoints, which are unique for every image pair. ubc-vision/image-matching-benchmark#42 The caveat: this works only for the stereo. For the multiview we are thinking about the following options: just concat the "pairwise" keypoints and resp. modify match indexes. However, this would rely on COLMAP ability to merge the duplicating disjoint keypoints. The second option is to allow participants to merge/BA the keypoints themselves and submit - as usual - single keypoint list per image. |
The template (with LoFTR) to generate such "pairwise" submission is here https://github.com/ducha-aiki/imc2021-sample-kornia-submission/blob/loftr/extract_loftr_matches.py It does not include RANSAC though |
Hi @ducha-aiki , Thanks for the update, i think for stereo task it's good to have the support of pairwise matching. Regarding multiview task, this repository already has the support for those methods that are not keypoint-based and see the implementation in localize_sfm_helper.py.
This heuristic solution is validated on Patch2Pix to improve significantly the localization accuracy on Aachen Day and Night original version. I also applied this to Loftr, which also has this issue and results are: 89.1 / 94.9 / 98.3 87.8 / 93.9 / 100.0. If i just feed the raw keypoints that are concatenated, the performance will be really bad, which kinda refers that colmap is not handling those nearby keypoints well. While this solution is not perfect, it is rather handy since it doesn't bring much computation burden during evaluation. Another thing, it seems i can not open the pairwise example: https://github.com/ducha-aiki/imc2021-sample-kornia-submission/blob/loftr/extract_loftr_matches.py. |
Oh, so we basically can recommend your solution as an official one for the multiview. The remaining thing (for us at IMC) is to tell the code to take pairwise matches for stereo and per-image matching in mutiview, which we will do next week. Regarding the example - I have merged it to the master branch, sorry :) |
Offtopic: We have recently added LoFTR to kornia I think, it would be good for everyone to add Patch2pix there as well - for users it would be easy to use, for you - potentially more users. What do you think? |
Sure, i can submit Patch2Pix when you want to examine your updated benchmark. When you say Patch2Pix, you mean the refinement part or the whole network when combined with NCNet or both? |
I mean both - as a end-to-end matching module, and as a way to refine correspondences as well :) |
Sure 👍.. Just ping me when anything is needed to be done from my side :) |
Hi,
There is a sample repository for the submission to IMC
https://github.com/ducha-aiki/imc2021-sample-kornia-submission
If you want, I can adapt it for your toolbox. Caveat - it would be quite hard to do for the methods, not based on keypoints now.
The text was updated successfully, but these errors were encountered: