This repository has been archived by the owner on Nov 3, 2023. It is now read-only.
-
Notifications
You must be signed in to change notification settings - Fork 2.1k
New Metric.from_mask helper method #3411
Labels
Comments
This issue has not had activity in 30 days. Please feel free to reopen if you have more issues. You may apply the "never-stale" tag to prevent this from happening. |
poojasethi
added a commit
to poojasethi/ParlAI
that referenced
this issue
Nov 22, 2022
poojasethi
added a commit
to poojasethi/ParlAI
that referenced
this issue
Nov 23, 2022
poojasethi
added a commit
to poojasethi/ParlAI
that referenced
this issue
Nov 23, 2022
poojasethi
added a commit
to poojasethi/ParlAI
that referenced
this issue
Nov 23, 2022
poojasethi
added a commit
to poojasethi/ParlAI
that referenced
this issue
Nov 23, 2022
Hi @klshuster and @stephenroller! I've just submitted a PR for this issue: #4894 |
poojasethi
added a commit
that referenced
this issue
Nov 29, 2022
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
We have quite a few instances where we have some per-token losses/metrics along with a corresponding mask
And we want a per-batch-example average:
I'd like us to have a helper classmethod in
Metric
calledfrom_mask
:Once this is done, add unit tests for this (test AverageMetric and PPLMetric directly). Checkpoint there.
After you've implemented this, upgrade TorchGeneratorAgent to use your new helper, upgrading the code for loss, ppl, and token_acc.
Example:
ParlAI/parlai/core/torch_generator_agent.py
Lines 711 to 727 in 67433e3
See if you can find at least one other place who can benefit from upgrading this pattern.
The text was updated successfully, but these errors were encountered: