This release is heavily influenced by the employment of the project for the TREC-Covid challenge. Thus, instead of being TREC-PM specific this project has hijacked to also be used for TREC-Covid. Apart from Covid-Specific classes for topics and decorators, the following major changes were introduced:
- an all-new template engine that uses templates which are valid JSON using the "${...}" syntax
- capabilities for residual evaluation: In a round-wise evaluation campaign like TREC-Covid, submissions for rounds > 1 should not include documents that were already judged in the previous rounds. There is a new gold standard filter decorator to prevent this.
- document-ID-mapping capabilities for the case that document IDs have changed in the current dataset so that already judged documents can still end up in the submissions. Those have been removed in TREC-Covid.