Caraya 1.0 Release Notes

What's New

Upgraded the source code to LabVIEW 2013

This is a change to take advantage of improvements brought by newer versions of LabVIEW, such as separate compile code, as well as benefitting from increased stability of classes and compiler optimizations.

Assertions are uniquely identified

We've added a unique identifier to each assertion calls in order to facilitate the identification of individual tests even in the case of use of redundant test labels such as when assertions are run in a loop or in parallel with the same call chain. This feature also allowed Caraya to fix a few bugs resulting from the name collisions, as well as allowing the assertions to be sorted by time of execution instead of being sorted alphabetically.

Verbose option for detailed failure descriptions

We've added an optional "Verbose" flag to the Test Suites to allow Assertions to report a more detailed explanation of the reason for failures. Because Caraya is renowned for being the fastest Unit Test framework in LabVIEW, we wanted to keep it that way and decided to make this feature default to False.

Add an image here for verbose option

JUnit XML report extension

In release 0.6, custom test reports were introduced, including a JUnit compatible report. In this version, we've extended the report to include an optional "Execution Time (s)" terminal on each assertion. Because Caraya is assertion-based, it can run headless without a Test Manager aggregating the results. It is therfore the responsibility of the developer to provide execution time metrics if this is needed.

Add an image here for execution time option + example of test report

Programmatic discovery and execution of tests by source folder

Now, it's possible to discover tests that are not part of a specific test suite and execute them sequentially. The method is slower that the strictly-defined test cases, but is flexible and allows for Command Line execution.

Command Line Interface support

Caraya now integrates with automation tools for CI/CD. It supports command line interface executable for testing all VIs in a folder and returning jUnit xml into a specified file. Syntax is available in this Wiki page: Command Line Interface.

Documented Error Codes

The 7000-7999 code range has been chosen for all Caraya framework errors. A wiki page has been created to keep track of those error codes. Error Codes

Minor Fixes

We've fixed some minor issues in the UI where sometimes an error dialog might occur.

Details

• [Fix: 4] Fixed the intermittent error 1 when destroying Test Suites
• [Fix: 34] Added an optional "Execution Time (us)" terminal to assertion methods for reporting
• [Fix: 40] Assertions are now sorted in order they occur instead of ascending alphabetically
• [Fix: 44] Fixed the conditions that overewrote assertions when test names were identical
• [Fix: 47] Double-clicking on assertion result in Test Manager now works after VI has left memory for lvlibs inside lvlibs.
• [Fix: 50] Updated the Math Test Suite to use non-deprecated method for Test Report
• [Fix: 53] Fixed issues with comparison (greater/less) of variant representations when integers and floats were compared

Contributors

Thanks to the community members for their issue reports, feature requests, code contributions and/or review/testing of beta versions for this release:

• Carlo (@carlodri)
• Cyril Gambini (@CygaLV)
• Jim Kring (@jimkring)
• kosist (@kosist)
• Piotr Kruczkowski (@Primary-Key)
• OlivierLabo (@OlivierLabo)
• Matthew Jacobson (@jacobson3)
• Omar Mussa (@omarmussa)
• Francois Normandin (@francois-normandin)
• Matt Pollock (@pollockm)
• James D. Powell (@drjdpowell)
• Chris Stryker (@ChrisStrykesAgain)
• Sam Taggart (@stagg54)