For non-answerable questions does not seem to work

[alex-movila]

Hi, I tested with your script with downloaded model Bert-on-Squad-V2.0
I asked questions like:
When BERT was born?
Who created Earth?

The answers are nonsensical instead of answering "I can't answer"
This is disappointing.

[surbhardwaj]

Hi, that thing can be fixed by making the changes in predict function, I need to apply some threshold over the outputs obtained. I''ll fix it.
Thanks for pointing out this. :)

[alex-movila]

Yeah I modified a bit the code to expose the probability.
A configurable threshold could help.

[the-black-knight-01]

@alex-movila BERT, by default, will give an answer to all questions. Even if that question is irrelevant to Context.
That was the problem with BERT. To Remove this problem, Google comes with a new Model, XLNET
You can try that.

In BERT even if you will put 66% as the threshold. then also irrelevant answer you will get. Also, you will lose a lot of the right answers.

[desaibhargav]

Hi, that thing can be fixed by making the changes in predict function, I need to apply some threshold over the outputs obtained. I''ll fix it.
Thanks for pointing out this. :)

Is it fixed now?