Expectations

Expectations are where the majority of the tests will actually be coded and where assertions live. There are two distinct expectation types available: should and holds. These are vanilla and “fuzz” expectations respectively.

Before and after an expectation is executed, any defined @before@ and @after@ functions are executed.

should

The should function takes two arguments, a description of the property that the test is expecting to be true and a function where the test is actually executed. If an error escapes the function argument during the execution of the test, the test will fail. If an assertion fails, then the test will fail.

.tst.desc["The Fuzz Generator"]{
 should["return a list of fuzz values of the given type provided a symbol"]{
  type[.tst.pickFuzz[`symbol;1]] musteq 11h;
  type[.tst.pickFuzz[`int;100]] musteq 6h; 
  type[.tst.pickFuzz[`time;10]] musteq 19h;
  };  
 };

holds

The holds function takes three arguments, a description of the property that the test is expecting to be true, an options dictionary, and a function which takes one argument, which will the be the random value to run the test with. This type of expectation is called a “fuzz test”; the idea being that you send fuzzy values into your test and ensure that it doesn’t choke on unaccounted for edge cases.

The options parameter takes the following options:

vars

vars is an option specifying which type of random value should be provided to the test function. The default value is to provide a random integer.

vars accepts many different values including custom functions to generate your own random data if the available generators are unacceptable.

Form	Effect	Example
`type	Generate runs number of type values	`symbol
{…}	Run provided function runs number of times	{5}
atom	runs?atom	20
(atom1_;_atom2;…)	Select runs elements randomly from provided list	(`a;“a”;97)
`type$()	Generate runs number of lists of random length of type values (max list length is 100)	`symbol$()
`type_$N_#typed-null	Generate runs number of lists of length less than or equal to N of type_values	20#`
`type_$N_#type-value	Generate runs number of lists of lenght less than or equal to N with value generated by type-value	20#`3
typed-list	runs?typed-list	`abc`def`ghi
`key1_`key2_`…!(generator;generator;…)	For each element of the dictionary, use the corresponding generator to create random data for the test	`a`b!(10;200#0N)

The default max list length for the `type$() form can be changed by setting .tst.fuzzListMaxLength

The different types recognized for `type forms and their corresponding defaults for random generation:

Type	Random Generator Value
boolean	0b
byte	0×0
short	0h
int	0
long	0j
real	10000e
float	1000000f
char	" "
symbol	`7
month	2000.01m
date	2000.01.01
datetime	The current year at December 31st, 11:59:59.999, e.g.: 2009.12.31T11:59:59.999
minute	00:00
second	00:00:00
time	00:00:00.000

You can access the default values, their symbolic type names, and short type codes in the following variables:

Variable	Description
.tst.typeNames	Symbolic type names: i.e: `boolean, `byte, etc …
.tst.typeCodes	Short type codes: i.e: 1h, 4h, etc …
.tst.typeDefaults	Type default values: i.e: 0b, 0×0, etc …
.tst.typeFuzzN	A dictionary from the symbolic type names to the defaults
.tst.typeFuzzC	A dictionary from the short type codes to the defaults

runs

runs is an integer option specifying the number of times to run a particular fuzz test. The default value is 100.

maxFailRate

maxFailRate is a floating point value between the values of 0 and 1. This option specifies what proportion of the fuzz tests are allowed to fail and still have the fuzz test pass. The default value is 0. It is not recommended that this option be changed unless you are placing a property under test where processing failures are expected and acceptable. Some examples of this might be soft real-time systems where process abortions due to run-time constraints are expected in some cases or multi-processing code where communication against peers is sometimes expected to fail.

.tst.desc["Logging Messages"]{
 holds["can display lists of random objects";((),`vars)!(),{1?'20?.tst.typeDefaults}]{
  mustnotthrow[();{[x;y].log.info x}[x]];
  };
 };