Version 5.7

What's new?

• Update Python to v3.6, pandas to v0.22.0 and SKLL to v1.5. This required minor changes to the code and updates to some of the test files.
• The conda installation command has changed. See the new command here.

Improvements

• The evaluation_by_group notebook in addition to bar plots now includes a table showing the main metrics for each subgroup.
• When using the RSMTool API, it is now possible to specify a tolerance keyword argument for trim method. Read more here.

Bugfixes

• The differential feature functioning (DFF) plots are now correctly generated using preprocessed feature values. In the previous version, they incorrectly used raw feature values.
• In v0.19.0 of scikit-learn, the implementation of explained_variance_ in their PCA implementation underwent some bugfixes. Due to this, the results of PCA analyses no longer match those produced by the previous versions of RSMTool and had to be changed.

Other minor changes

• Updated the utility script update_skll_model.py to make it compatible with SKLL v.1.5.
• Minor updates for the documentation.

Version 5.6

This is an important release that has a critical bugfix as well as useful improvements.

Bugfixes

• Fixed critical bug in computation of standardized mean differences. The denominator for SMDs should be using population standard deviations, not the ones computed over the subgroups themselves.
• Added converters to the notebook header to allow correct treatment of candidate IDs with leading zeros.
• Modified the test utility functions to catch discrepancies caused by missing leading zero.

Improvements

• The tables generated by rsmsummarize are now saved in the same way as for other tools.
• rsmsummarize now shows a table with standardized coefficients for all models.
• The predictions for the post-processed training set are now also saved.
• Added a new notebook that shows differential feature functioning (DFF) plots by subgroup. To use it, add dff_by_group to the general_sectionconfiguration option. Read more here.
• The features that have not been used in the model are now excluded from the datasets before they are sent to SKLL for prediction. This makes the prediction step much faster for large datasets.
• When testing whether the feature std. dev. in the training set is zero, we currently set tolerance to 1e-06. This is not sufficient with features with very low values (these can result from an inverse transform of acoustic likelihoods which are logs of very small values). This tolerance is now increased to 1e-07.

Other Minor Changes

• Update the utility script update_skll_model.py to allow it to be used with other tools.
• Update tests and documentation.

Version 5.5.2

This is primarily a bug fix release but it also has some improvements.

Bugfixes

• The notebooks are fixed so that any plots are now shown in their assigned places (this was broken in v5.5.1 due to the underlying matplotlib dependency being upgraded to v2.0).

Improvements

• The widths of the subgroup plots is now more intelligently determined. No more plots with really wide bars when there are only a few groups.
• Many of the unnecessary warnings that popped up in the reports and on the terminal are now suppressed and handled in code where appropriate.

Version 5.5.1

This is a minor bugfix release.

What's new?

• Update SKLL requirement to v1.3. This allows us to streamline the RSMTool conda recipe into a single recipe (using the MKL backend instead of OpenBLAS on macOS/Linux)
• Update all other conda packages to their latest versions.
• Minor fixes and updates to tests.

Version 5.5.0

This is a major release.

What's new?

• New tool: rsmsummarize which can summarize any number of rsmtool experiments and produce a summary report.
• All input files can now be in any tabular format (CSV/TSV/XLS/XLSX). This is an improvement over previous releases where input files were required to be CSV files. For more details, see the documentation. This includes the feature description file although the old JSON format is still supported for backwards compatibility (you will get a DeprecationWarning when using that format).
• rsmtool now includes a new model ScoreWeightedLR which estimates feature coefficients using weighted least squares regression. The weights are computed as an inverse proportion of total number of responses with a given score level.
• rsmtool now produces the feature sub-directory as part of its output for all experiments. Previously, this sub-directory was only produced for experiments with some form of feature selection.
• rsmcompare now requires the user to specify a "comparison ID" instead of generating one automatically from the experiment IDs of the two experiments being compared.

Improvements

• Improved CSS for HTML report printing.
• Several updates and fixes to documentation.
• Fix errors in PCA computation when the number of components was smaller than the total number of features.
• Use skll API to convert featureset to data frame instead of writing our own function.
• Separate the file reading and processing functions in rsmpredict for more modularity.
• Wrap longer labels on box plots automatically.
• Update package dependencies to latest releases.
• Increase report generation timeout to be 60 minutes instead of 10 minutes. This is useful for experiments with very large data files.
• Fix bug that had system and human scores reversed in the confusion matrix.
• Limit the length of experiment IDs where appropriate such that we don't encounter "filename is too long" OS errors.

Version 5.2.1

This is a minor release that fixes a bug in how some javascript was loaded in the Jupyter notebooks.

Version 5.2.0

This release has minor features and bug fixes.

1. rsmcompare now includes extra checks to make sure the experiment paths and ids specified by the user actually exist.
2. Factored out rsmcompare code from the header notebook and moved to comparison.py.
3. Factored out the float formatting functions from the rsmtool/rsmcompare header notebooks and moved them to utils.py.
4. Added new tests for comparison.py and the float formatting and highlighting functions in utils.py.
5. Fixed the bug in rsmcompare which seemed to ignore zero scores in confusion matrices.
6. Fixed a bug in rsmcompare that prevented the score distribution table from being displayed correctly if the score levels differed between the two models.

Version 5.1.1

This is a minor bugfix release.

1. Previously, if rsmpredict was given a model requiring a transformation that could yield Inf/NaN values for new data (e.g. sqrt(-1)), it would raise an error and terminate. Now, it simply excludes such responses and displays a warning.
2. Updated various conda files to use newer versions of the ipython and notebook packages since there seem to have been some updates that broke older recipes and requirements files.

Version 5.1.0

This is a major release.

1. Completely overhauled the documentation. Instead of relying on a collection of loosely organized markdown files, the documentation is much more cohesive and hosted on readthedocs. It now includes a clear introduction to what RSMTool is as well as tutorials.
2. The RSMTool API is now richer and explicitly documented.
3. rsmcompare can now compare two rsmeval experiments as well as an rsmtool experiment to an rsmeval experiment.
4. Code coverage is now automatically computed as part of CI testing.
5. Expected warnings are now suppressed when running the tests.
6. Fixed several stylistic issues in the codebase raised by pep8 and pyflakes.

Version 5.0.2

• Added files necessary for submission to the Journal of Open Source Software.