Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

How to extract probabilities of models like Roberta? #651

Open
franz101 opened this issue Sep 21, 2023 · 1 comment
Open

How to extract probabilities of models like Roberta? #651

franz101 opened this issue Sep 21, 2023 · 1 comment

Comments

@franz101
Copy link

For most NER models I was able to extract the probabilities of the prediction (ConfidenceScores). I noticed Roberta and Albert are missing the setIncludeAllConfidenceScores function in johnsnowlabs > 5. Is there a way to still extract the ConfidenceScores maybe I missed it in the documentation.

How to reproduce:

import sparknlp
from johnsnowlabs.nlp import PretrainedPipeline

spark = sparknlp.start()

pipeline = PretrainedPipeline("albert_base_token_classifier_conll03_pipeline", lang = "en")

pipeline.model.setIncludeAllConfidenceScores(True)
@C-K-Loan
Copy link
Member

C-K-Loan commented Sep 26, 2023

@franz101 the method setIncludeAllConfidenceScores is not available on every annotator, in particular, the XYZ-For-Token-Classification based annotators do not support this feature yet.
Any NerDL, MedicalNerDL, FinanceNerDL, LegalNerDL architecture based models currently support setIncludeAllConfidenceScores

I will talk with @maziyarpanahi to get it on the roadmap or find workarounds

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants