Negative sample selection

[cramraj8]

how did you choose the negative pairs to pair with the positive training samples in the pairwise softmax loss ?

[seanmacavaney]

Hi @cramraj8

Negative samples were randomly selected from the top 1000 BM25 results-- including both samples that are not graded, and those with a relevance score of 0.

The exact procedure is here: https://github.com/Georgetown-IR-Lab/OpenNIR/blob/master/onir/datasets/pair_iter.py#L8 with pos_source=intersect and neg_source=run.

Does this answer your question?