-
Notifications
You must be signed in to change notification settings - Fork 27
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
gt-info, gt.log, and gt_overlap.log in benchmark #21
Comments
Hi. The benchmark folder contain code to evaluate the accuracy of the
registration algorithm.
For information on the data, it would be better to refer to original
dataset paper. However, I believe most of the information in .log files are
not used, and that we only use the ground truth pose and compare it with
the estimated one.
…On Wed, 21 Jun 2023 at 11:46 AM, lcxiha ***@***.***> wrote:
Hello,
1.I would like to know what are the functions of 3DMatch and 3DLoMatch in
the src/datasets/3dmatch/benchmarks folder? Is it used for testing (in the
test. py file)?
2.What are the role of gt-info, gt.log, and gt_overlap.log in the dataset
folder of 3DMatch and 3DLoMatch (../RegTR
main/src/datasets/3dmatch/benchmarks/3DLoMatch/7 scenes redkitchen) ?How to
obtain them when I build my own dataset?
I want to build my own dataset to use the REGTR , but I am stumped by this
difficulty. Could you help me ?Thanks a lot!
—
Reply to this email directly, view it on GitHub
<#21>, or unsubscribe
<https://github.com/notifications/unsubscribe-auth/ADIBP62AAIGIAUGAMX2YURDXMJU73ANCNFSM6AAAAAAZOEHWRE>
.
You are receiving this because you are subscribed to this thread.Message
ID: ***@***.***>
|
It’s hard to tell from this image alone. Does it happen on the training point clouds? If not, it might be an overfitting issue due to lack of data. |
Thank you! Yes, this graph visualizes the alignment of the test dataset using the model obtained from the training dataset (i.e. running demo. py code). How many pairs of point clouds do I need to have at least to use this algorithm using my dataset? |
RegTR is training data hungry since it relies heavily on the transformer
without much inductive bias. It does require a large dataset size like the
3DMatch/modelnet.
…On Tue, 18 Jul 2023 at 4:10 PM, lcxiha ***@***.***> wrote:
Thank you! Yes, this graph visualizes the alignment of the test dataset
using the model obtained from the training dataset (i.e. running demo. py
code). How many pairs of point clouds do I need to have at least to use
this algorithm using my dataset?
—
Reply to this email directly, view it on GitHub
<#21 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/ADIBP65SCC52B6IWW3QBBK3XQZAHRANCNFSM6AAAAAAZOEHWRE>
.
You are receiving this because you commented.Message ID:
***@***.***>
|
So does the logarithm of the dataset need to reach the level of ten thousand? |
Hello, I noticed that when I delete the gt.log and gt-info files in ../RegTR-main/src/datasets/3dmatch/benchmarks/3DMatch/7-scenes-redkitchen ,running the test.py program will report an error, so I guess the gt.log and gt-info files are useful. But I don't know the function of them. |
Hi, you need the groundtruth files when you're evaluating the algorithm. For example, in here the groundtruth poses are loaded so you can compute the errors in the following lines. |
Hello, the gt.log file represents the transformation matrix (i.e. groundtruth) of two point clouds,but when I visualized the 3DMatch.pkl file, I found that the corresponding transformation matrix and the transformation matrix of the same pair of point cloud keyframes in the gt.log file were not the same, shouldn't they be the same? |
Good observation. I don't have an answer to this since I took the files from Predator's repository. |
Thank you for your answer. I want to know how to generate a gt.log file if I use my own dataset. Just need to know the transformation matrix of the two point clouds? Is the transformation matrix in the gt.log file groundtruth? That is to say, if I want to verify this algorithm, all I need is a gt.log file, like the benchmark_3dmatch.py , and I need to modify the test.py file based on my benchmark. py program. |
Yes that's right. gt.log contains the groundtruth poses.
…On Wed, Jul 26, 2023 at 9:27 PM lcxiha ***@***.***> wrote:
Thank you for your answer. I want to know how to generate a gt.log file if
I use my own dataset. Just need to know the transformation matrix of the
two point clouds? Is the transformation matrix in the gt.log file
groundtruth?
That is to say, if I want to verify this algorithm, all I need is a gt.log
file, like the benchmark_3dmatch.py , and I need to modify the test.py file
based on my benchmark. py program.
—
Reply to this email directly, view it on GitHub
<#21 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/ADIBP62UQFCTNDJSHVPXRKLXSELLPANCNFSM6AAAAAAZOEHWRE>
.
You are receiving this because you commented.Message ID:
***@***.***>
|
And I have another question : how can I get the est.log file? |
Yes, it is generated when you run the inference code.
…On Thu, 27 Jul 2023 at 10:51 AM, lcxiha ***@***.***> wrote:
And I have another question : how can I get the est.log file?
Is the est.log file obtained from the trained model?
—
Reply to this email directly, view it on GitHub
<#21 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/ADIBP65ZG5YNGJFIPAUVEADXSHJRZANCNFSM6AAAAAAZOEHWRE>
.
You are receiving this because you commented.Message ID:
***@***.***>
|
Hello, I would like to consult: What is the comparison between the point cloud density and scale of the Kitti dataset and the 3DMatch dataset? |
Hi, we did not do any comparison between the density and scale on the two
datasets.
…On Wed, 2 Aug 2023 at 3:44 PM, lcxiha ***@***.***> wrote:
Hello, I would like to consult: What is the comparison between the point
cloud density and scale of the Kitti dataset and the 3DMatch dataset?
—
Reply to this email directly, view it on GitHub
<#21 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/ADIBP66D6SNFUPWXB5JQGKDXTIAMZANCNFSM6AAAAAAZOEHWRE>
.
You are receiving this because you commented.Message ID:
***@***.***>
|
But shouldn't some parameters in kernel point convolutional networks be changed based on the density and scale of the point cloud? For example: conv_ radius, deform_ radius, KP_ extent, neighborhood_ limits, first_ subsampling_ dl and overlap_ radius |
Yes you’re right. You may need to tune those parameters based on your point
cloud characteristics.
…On Sat, 5 Aug 2023 at 11:30 AM, lcxiha ***@***.***> wrote:
But shouldn't some parameters in kernel point convolutional networks be
changed based on the density and scale of the point cloud? For example:
conv_ radius, deform_ radius, KP_ extent, neighborhood_ limits, first_
subsampling_ dl and overlap_ radius
—
Reply to this email directly, view it on GitHub
<#21 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/ADIBP634NBJDBLEFN7CZMXTXTW445ANCNFSM6AAAAAAZOEHWRE>
.
You are receiving this because you commented.Message ID:
***@***.***>
|
As stated in the comments, setting For the kpconv parameters, I recommend reading its paper to get the intuition how to set. Nevertheless, RegTR works well when there's around 500 points at the coarsest level where attention is applied. |
For monitoring the training you should rely on the metrics on the
validation dataset. Which metrics to prioritize depends on your
application. Some people might find the mean rot/trans errors more
important than the registration success rates.
Reg_success is the defined by the thresholds set, I.e. how many
registrations are better than the threshold.
Training losses are not as useful since they can overfit.
…On Mon, 14 Aug 2023 at 4:32 PM, lcxiha ***@***.***> wrote:
Hi,How to check the quality of the training model during the training
process?
During training processing ,should I need to check the changes in loss
values and these metrics? In this paper, I only need to pay attention to
total、reg_success_final、rot_err_deg_final and trans_err_final , right?
[image: 1692001625895]
<https://user-images.githubusercontent.com/130826884/260397398-7bc5a270-272b-4db6-aec5-5320aa37e273.png>
The higher 'totol', the better.The higher 'reg_success_final' , the
better. The smaller 'rot_err_deg_final' and 'trans_err_final', the better .
I have a question: Are these metrics(rot_err_deg_final and trans_err_final)
in the best model during the training process need to less than
reg_success_thresh_rot和reg_success_thresh_trans?
Which of these metrics and loss value do I need to prioritize?
—
Reply to this email directly, view it on GitHub
<#21 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/ADIBP66N24VKGUGWBU6VO3TXVHPARANCNFSM6AAAAAAZOEHWRE>
.
You are receiving this because you commented.Message ID:
***@***.***>
|
Hello, I have another question I would like to consult: could this algorithm only be applied to two point clouds with exactly the same frame size? Is it feasible if the size of two point clouds is not exactly the same?Looking forward to your reply. |
Are you referring to whether the two point clouds being registered need to
have the same number of points? No they do not need to be.
However, we only tested it on scenarios where the two point clouds are of
similar sizes. So if you are asking if it’s suitable for matching a local
scan with a much larger global one, that might not work as well.
…On Tue, 5 Dec 2023 at 11:28 AM, lcxiha ***@***.***> wrote:
Hello, I have another question I would like to consult: could this
algorithm only be applied to two point clouds with exactly the same frame
size? Is it feasible if the size of two point clouds is not exactly the
same?Looking forward to your reply.
—
Reply to this email directly, view it on GitHub
<#21 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/ADIBP6Y5V7TDWFOY7BHT223YH2IFTAVCNFSM6AAAAAAZOEHWRGVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQMZZHE2DANZUHE>
.
You are receiving this because you commented.Message ID:
***@***.***>
|
Thanks a lot!Is it feasible to replace the KPConv backbone network with a pointnet network?Looking forward to your reply. |
I didn’t try that out, but shouldn’t be a problem. Of course you’ll have to
retrain the network in this case.
…On Mon, 11 Dec 2023 at 2:48 PM, lcxiha ***@***.***> wrote:
Thanks a lot!Is it feasible to replace the KPConv backbone network with a
pointnet network?Looking forward to your reply.
—
Reply to this email directly, view it on GitHub
<#21 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/ADIBP65IWMELPDUEOZCWUGLYI2UFRAVCNFSM6AAAAAAZOEHWRGVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQNBZGQZDGMZSG4>
.
You are receiving this because you commented.Message ID:
***@***.***>
|
Hello, I found that the evaluation metric for this algorithm during the training process is only the loss function value, but during the validation process, there are both loss function values and other evaluation metrics, such as reg_success_final、rot_err_deg_final and trans_err_final, why is this? |
Hi, it’s just a design choice to use the loss for validation as it doesn’t
require setting an additional arbitrary weights for the rotation and
translation components when choosing the best checkpoint during training.
During testing, naturally the actual rotation/translation values are
important for evaluation.
Zi Jian
…On Fri, 15 Dec 2023 at 11:55 AM, lcxiha ***@***.***> wrote:
Hello, I found that the evaluation metric for this algorithm during the
training process is only the loss function value, but during the validation
process, there are both loss function values and other evaluation metrics,
such as reg_success_final、rot_err_deg_final and trans_err_final, why is
this?
—
Reply to this email directly, view it on GitHub
<#21 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/ADIBP65EVNPBFPVFOCFBNRDYJPC2HAVCNFSM6AAAAAAZOEHWRGVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQNJXGIZTEOJQGA>
.
You are receiving this because you commented.Message ID:
***@***.***>
|
|
Oh yes, PointNet++ will be better. I was thinking you were referring to
running PointNet on each point cluster separately, in which case will be
similar to PointNet++.
…On Mon, 18 Dec 2023 at 9:42 AM, lcxiha ***@***.***> wrote:
I didn’t try that out, but shouldn’t be a problem. Of course you’ll have
to retrain the network in this case.
Thanks!But I'm worried that the matching results may not be good, because
theoretically, the performance of pointnet network is not as good as kernel
convolution. And the pointnet network only learns global features without
local features. Perhaps the pointnet++ network will perform better?
—
Reply to this email directly, view it on GitHub
<#21 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/ADIBP64XPHOF4OHGPOMHOUTYJ6NRHAVCNFSM6AAAAAAZOEHWRGVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQNJZGQYDSNBTG4>
.
You are receiving this because you commented.Message ID:
***@***.***>
|
Yes, I would like to run the pointnet network on both the source and target point clouds to extract the features of the point clouds, instead of the kernel point convolution part in your algorithm. In this case, are pointnet and pointnet++similar? Could I use one of them? |
Yes, pooling layers are necessary. The attention layers run on downsampled
point clouds.
…On Mon, 18 Dec 2023 at 10:02 AM, lcxiha ***@***.***> wrote:
Yes, I would like to run the pointnet network on both the source and
target point clouds to extract the features of the point clouds, instead of
the kernel point convolution part in your algorithm. In this case, are
pointnet and pointnet++similar? Could I use one of them?
In addition, I have found that kernel point convolutional networks not
only extract point cloud features but also downsample point clouds.
Therefore, do I need to use pointnet or pointnet++networks with pooling
layers instead of kernel point convolutional networks?
—
Reply to this email directly, view it on GitHub
<#21 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/ADIBP63FIJZ5BTD4Y57QQETYJ6P5BAVCNFSM6AAAAAAZOEHWRGVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQNJZGQZDKNJSHE>
.
You are receiving this because you commented.Message ID:
***@***.***>
|
OK!Thanks a lot! |
Hello, I found that when I load the trained model, I can output the transformation matrix of two point clouds. However, when the model is loaded with two point clouds that do not overlap, the transformation matrix will also be output. So, how can I determine whether two point clouds overlap by simply looking at the output transformation matrix? |
Hello, I have a question: in this algorithm, the role of Kernel Point Convolution is to downsample the point cloud and extract its local features. Is the role of Transformer to extract the global features of the point cloud?Looking forward to your reply. |
There’s no point cloud level global feature. The transformer enhances the
individual point cloud features by letting them interact with each other,
thus improves the matching performance.
…On Thu, 4 Jan 2024 at 4:02 PM, lcxiha ***@***.***> wrote:
Hello, I have a question: in this algorithm, the role of Kernel Point
Convolution is to downsample the point cloud and extract its local
features. Is the role of Transformer to extract the global features of the
point cloud?Looking forward to your reply.
—
Reply to this email directly, view it on GitHub
<#21 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/ADIBP67V2XOREY57TOJ7AXDYMZOY7AVCNFSM6AAAAAAZOEHWRGVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQNZWGY3TANBQGU>
.
You are receiving this because you commented.Message ID:
***@***.***>
|
Could the traditional convolutional networks not enhance the individual point cloud features by letting them interact with each other? Or is the performance of traditional convolutions slightly inferior to that of Transformers? |
Sorry to bother you again, but I would like to cousult why this algorithm cannot directly use transformer to extract features, but instead adds a kernel convolution. |
It’s trickier to run attention layers on large raw point clouds due to
memory constraints.
…On Sat, 6 Jan 2024 at 11:56 AM, lcxiha ***@***.***> wrote:
Sorry to bother you again, but I would like to cousult why this algorithm
cannot directly use transformer to extract features, but instead adds a
kernel convolution.
—
Reply to this email directly, view it on GitHub
<#21 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/ADIBP64SMHVSBF2H7ISVSRLYNDDPDAVCNFSM6AAAAAAZOEHWRGVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQNZZGUZDOMBRG4>
.
You are receiving this because you commented.Message ID:
***@***.***>
|
Thanks a lot! That is to say, do we only use a network that downsamples the point cloud before using the transformer? Or is it a network that requires both point cloud downsampling and feature extraction? |
It's good to have meaningful features for the downsampled point cloud.
…On Sat, Jan 6, 2024 at 8:47 PM lcxiha ***@***.***> wrote:
Thanks a lot! That is to say, do we only use a network that downsamples
the point cloud before using the transformer? Or is it a network that
requires both point cloud downsampling and feature extraction?
—
Reply to this email directly, view it on GitHub
<#21 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/ADIBP6557KOB2XCGSM7FTC3YNFBWJAVCNFSM6AAAAAAZOEHWRGVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQNZZGY3TEMZQGU>
.
You are receiving this because you commented.Message ID:
***@***.***>
|
Thanks a lot! |
Hello, I have found that the 3DMatch dataset is approximately 3m * 2m. If the dataset is 200m * 100m or even larger, can this algorithm framework still be applicable?Looking forward to your reply. |
Hello,
1.I would like to know what are the functions of 3DMatch and 3DLoMatch in the src/datasets/3dmatch/benchmarks folder? Is it used for testing (in the test. py file)?
2.What are the role of gt-info, gt.log, and gt_overlap.log in the dataset folder of 3DMatch and 3DLoMatch (../RegTR main/src/datasets/3dmatch/benchmarks/3DLoMatch/7 scenes redkitchen) ?How to obtain them when I build my own dataset?
I want to build my own dataset to use the REGTR , but I am stumped by this difficulty. Could you help me ?Thanks a lot!
The text was updated successfully, but these errors were encountered: