Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Relative errors #46

Open
computative opened this issue Feb 25, 2024 · 1 comment
Open

Relative errors #46

computative opened this issue Feb 25, 2024 · 1 comment

Comments

@computative
Copy link

computative commented Feb 25, 2024

I've changed a couple of lines from arXiv.2111.13736 to get a sense of the RMSRE of the models from the nature paper. But I'm not getting helpful results as the program is set up, because the model tries to predict zero- or near-zero elements. That means the relative error becomes approximately equal to inverse of the regularization factor of the RMSRE (which is 1e-20 in the case of origin/arXiv.2111.13736, and the relative error becomes between 1e10 and 1e20). I thought I could decrease the cutoff radius to get a better feeling for how the program performs, but I have the impression that cutoff-radii-alike-parameters are hardcoded several places, so I wouldn't know how to modify the program to get an «approved» configuration of the program.

Could you have a look at this mod, and say how it should be further modified to get results for RMSRE that you can vouch for?

@zhanglw0521
Copy link
Collaborator

In our original paper, we showed an error plot in Fig. 4 and simply dropped those near-zero elements (shown as blank). Here is how we did that in code. I think a similar trick can be done for the RMSRE, i.e., if both the exact and predicted values are near zero, say smaller than 1e-10 (which means that the prediction is in fact quite nice), the relative error for this element shall be of fewer interests for me, and can thus be left blank. This avoids the extremely large relative errors you mentioned.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants