Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Norm Discovery Scoring 0 per norm category for modality All #7

Open
yrf1 opened this issue Apr 4, 2023 · 3 comments
Open

Norm Discovery Scoring 0 per norm category for modality All #7

yrf1 opened this issue Apr 4, 2023 · 3 comments

Comments

@yrf1
Copy link

yrf1 commented Apr 4, 2023

Potential bug? The scores for each norm category aggregated across all the data modality comes out as 0 - a noticeable bug when running from performer end.

@yrf1 yrf1 changed the title Norm Discovery Scoring Bug with April 4, 2023 EOD Scorer Version Norm Discovery Scoring Bug with April 4, 2023 EOD Scorer -- Version No numeric types to aggregate Apr 4, 2023
@yrf1 yrf1 changed the title Norm Discovery Scoring Bug with April 4, 2023 EOD Scorer -- Version No numeric types to aggregate Norm Discovery Scoring Bugs with April 4, 2023 EOD Scorer Apr 4, 2023
@yrf1 yrf1 changed the title Norm Discovery Scoring Bugs with April 4, 2023 EOD Scorer Norm Discovery Scoring 0 per norm category for modality All Apr 5, 2023
@jfiscus
Copy link
Collaborator

jfiscus commented Apr 5, 2023

Please be more specific (like adding a command line or send data to [email protected]) I do not know how to track this down. Also, the initial issue report included a command line that failed. It looks like pip install needed re-ran.

@yrf1
Copy link
Author

yrf1 commented Apr 6, 2023

I used the standard command from README:

CCU_scoring score-nd
-s test/pass_submissions/pass_submissions_LDC_reference_sample/ND/CCU_P1_TA1_ND_NIST_mini-eval1_20220815_164235
-ref test/reference/LDC_reference_sample
-i test/reference/LDC_reference_sample/index_files/LC1-SimulatedMiniEvalP1.20220909.ND.scoring.index.tab

@jfiscus
Copy link
Collaborator

jfiscus commented Apr 6, 2023

Please try this test:

Test 1:
cd <GIT_REPO_DIR>
python -m CCU_validation_scoring.cli score-nd -s test/pass_submissions/pass_submissions_LDC_reference_sample/ND/CCU_P1_TA1_ND_NIST_mini-eval1_20220815_164235 -ref test/reference/LDC_reference_sample -i test/reference/LDC_reference_sample/index_files/LC1-SimulatedMiniEvalP1.20220909.ND.scoring.index.tab -o /tmp

Test 2:
cd <GIT_REPO_DIR>
python3 -m pytest test

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants