Version 0.5.4

[0.5.4] 2024-03-04

Fixed

• Backward compatibility of BERTScoreMrefs with torchmetrics prior to 1.0.0.

Deleted

• Version class to use packaging.version.Version instead.

Version 0.5.3

[0.5.3] 2024-01-09

Fixed

• Fix BERTScoreMrefs computation when all multiple references sizes are equal.
• Check for empty timeout list in SPICE metric.

Version 0.5.2

[0.5.2] 2024-01-05

Changed

• aac-metrics is now compatible with transformers>=4.31.
• Rename default device value "auto" to "cuda_if_available".

Version 0.5.1

[0.5.1] 2023-12-20

Added

• Check sentences inputs for all metrics.

Fixed

• Fix BERTScoreMRefs metric with 1 candidate and 1 reference.

Version 0.5.0

[0.5.0] 2023-12-08

Added

• New Vocab metric to compute vocabulary size and vocabulary ratio.
• New BERTScoreMRefs metric wrapper to compute BERTScore with multiple references.

Changed

• Rename metric FluErr to FER.

Fixed

• METEOR localization issue. (#9)
• SPIDErMax output when return_all_scores=False.

Version 0.4.6

[0.4.6] 2023-10-10

Added

• Argument clean_archives for SPICE download.

Changed

• Check if newline character is in the sentences before ptb tokenization. (#6)
• SPICE no longer requires bash script files for installation.

Fixed

• Maximal version of transformers dependancy set to 4.31.0 to avoid error with FENSE and FluErr metrics.
• SPICE crash message and error output files.
• Default value for Evaluate metrics argument.

Deleted

• Remove now useless use_shell option for download.

Version 0.4.5

[0.4.5] 2023-09-12

Added

• Argument use_shell for METEOR and SPICE metrics and download function to fix Windows-OS specific error.

Changed

• Rename evaluate.py script to eval.py.

Fixed

• Workflow on main branch.
• Examples in README and doc with at least 2 sentences, and add a warning on all metrics that requires at least 2 candidates.

Version 0.4.4

[0.4.4] 2023-08-14

Added

• Evaluate class now implements a __hash__ and tolist() methods.
• BLEU 1 to n classes and functions.
• Get and set global user paths for cache, java and tmp.

Changed

• Function get_install_info now returns package_path.
• AACMetric now indicate the output type when using __call__ method.
• Rename AACEvaluate to DCASE2023Evaluate and use dcase2023 metric set instead of all metric set.

Fixed

• sbert_sim name in internal instantiation functions.
• Path management for Windows.

Version 0.4.3

[0.4.3] 2023-06-15

Changed

• AACMetric is no longer a subclass of torchmetrics.Metric even when it is installed. It avoid dependency to this package and remove potential errors due to Metric.
• Java 12 and 13 are now allowed.

Fixed

• Output name sbert_sim in FENSE and SBERTSim classes.
• Evaluate class instantiation with torchmetrics >= 0.11.
• evaluate.py script when using a verbose mode != 0.

Version 0.4.2

[0.4.2] 2023-04-19

Fixed

• File install_spice.sh is now in src/aac_metrics directory to fix download from a pip installation. (#3)
• Java version retriever to avoid exception when java version is correct. (#2)