-
-
Notifications
You must be signed in to change notification settings - Fork 619
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Distributed ndcg #3054
base: master
Are you sure you want to change the base?
Distributed ndcg #3054
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks a lot for finalizing this PR @ili0820 !
I left few comments on how to make CI happy and improve a bit the tests
discounted_gains = torch.tensor( | ||
[_tie_averaged_dcg(y_p, y_t, discount_cumsum, device) for y_p, y_t in zip(y_pred, y_true)], device=device | ||
) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Let's check here: https://github.com/catalyst-team/catalyst/blob/master/catalyst/metrics/_ndcg.py if there is another way to implement this in a vectorized way, https://github.com/pytorch/ignite/pull/2632/files#r930048810
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It is purely perf reasons. For example, computing s1 below will be faster than s2:
tensor = torch.rand(100)
s1 = tensor.sum()
s2 = 0
for v in tensor:
s2 += v
ignite/metrics/recsys/ndcg.py
Outdated
return normalized_gain | ||
|
||
|
||
class NDCG(Metric): |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Also, we need to write a docstring like here:
ignite/ignite/metrics/accuracy.py
Line 94 in 34a707e
class Accuracy(_BaseClassification): |
Please read this section of contributing guide: https://github.com/pytorch/ignite/blob/master/CONTRIBUTING.md#writing-documentation, especially about .. versionadded::
I ve deleted "test_output_cuda", added "n_epoch" and alligned code format for now. I will work on vectorizing and wrting docstring @vfdev-5 |
…into distributed-ndcg
@ili0820 any blockers that we can help to move forward with this PR ? |
@vfdev-5 sorry for no updates for a long time, I was still struggling with vectorizing for loop. 😢 |
@ili0820 no worries, let's update the docstring and merge this PR. Vectorization part could be done later in a follow-up PR. |
@vfdev-5 wow sounds great. does it matter if i copy and paste docstring from sklearn or other opensource? |
@ili0820 yes, you can inspire from scikit-learn to write ignite's docstring. Please check this contrib part: https://github.com/pytorch/ignite/blob/master/CONTRIBUTING.md#writing-documentation |
@vfdev-5 thanks I ll look into that. 😄 |
…into distributed-ndcg
@ili0820 can you please also add the metric to this list: https://github.com/pytorch/ignite/blob/master/docs/source/metrics.rst Also please fix formatting issues: https://github.com/pytorch/ignite/actions/runs/6251715047/job/16981806158?pr=3054#step:11:104 |
@vfdev-5 I accidently deleted the branch................................ 🤢 |
restored. sorry for inconvenience |
@ili0820 can you update the PR according to the review, please? |
hey @ili0820! this PR appears to have a lot of progress already. I wonder if you'd be available for seeing it through? I'd be available to take over otherwise, let me know! 🤠 |
@exitflynn go ahead and take over |
Fixes #2632
This PR is based on #2632 (thanks a lot to kamalojasv181!)
Description:
I ve tried to fix distributed NDCG tests like the examples. I am not really sure whether this is the way you guys wanted, but still, I tried my best and let me know which parts should be fixed (It's my first time contributing, so be kind plz 😯 )
Check list: