onnx backend reduce test errors

[attila-dusnoki-htec]

The following tests fails with onnx backend on the latest develop:

• test_reduce_l1_default_axes_keepdims_example_cpu
• test_reduce_l1_default_axes_keepdims_example_expanded_cpu
• test_reduce_l1_default_axes_keepdims_random_cpu
• test_reduce_l1_default_axes_keepdims_random_expanded_cpu
• test_reduce_l1_do_not_keepdims_example_cpu
• test_reduce_l1_do_not_keepdims_example_expanded_cpu
• test_reduce_l1_do_not_keepdims_random_cpu
• test_reduce_l1_do_not_keepdims_random_expanded_cpu
• test_reduce_l1_keep_dims_example_cpu
• test_reduce_l1_keep_dims_example_expanded_cpu
• test_reduce_l1_keep_dims_random_cpu
• test_reduce_l1_keep_dims_random_expanded_cpu
• test_reduce_l1_negative_axes_keep_dims_example_cpu
• test_reduce_l1_negative_axes_keep_dims_example_expanded_cpu
• test_reduce_l1_negative_axes_keep_dims_random_cpu
• test_reduce_l1_negative_axes_keep_dims_random_expanded_cpu
• test_reduce_l2_default_axes_keepdims_example_cpu
• test_reduce_l2_default_axes_keepdims_example_expanded_cpu
• test_reduce_l2_default_axes_keepdims_random_cpu
• test_reduce_l2_default_axes_keepdims_random_expanded_cpu
• test_reduce_l2_do_not_keepdims_example_cpu
• test_reduce_l2_do_not_keepdims_example_expanded_cpu
• test_reduce_l2_do_not_keepdims_random_cpu
• test_reduce_l2_do_not_keepdims_random_expanded_cpu
• test_reduce_l2_keep_dims_example_cpu
• test_reduce_l2_keep_dims_example_expanded_cpu
• test_reduce_l2_keep_dims_random_cpu
• test_reduce_l2_keep_dims_random_expanded_cpu
• test_reduce_l2_negative_axes_keep_dims_example_cpu
• test_reduce_l2_negative_axes_keep_dims_example_expanded_cpu
• test_reduce_l2_negative_axes_keep_dims_random_cpu
• test_reduce_l2_negative_axes_keep_dims_random_expanded_cpu
• test_reduce_log_sum_asc_axes_cpu
• test_reduce_log_sum_asc_axes_expanded_cpu
• test_reduce_log_sum_default_cpu
• test_reduce_log_sum_default_expanded_cpu
• test_reduce_log_sum_desc_axes_cpu
• test_reduce_log_sum_desc_axes_expanded_cpu
• test_reduce_log_sum_exp_default_axes_keepdims_example_cpu
• test_reduce_log_sum_exp_default_axes_keepdims_example_expanded_cpu
• test_reduce_log_sum_exp_default_axes_keepdims_random_cpu
• test_reduce_log_sum_exp_default_axes_keepdims_random_expanded_cpu
• test_reduce_log_sum_exp_do_not_keepdims_example_cpu
• test_reduce_log_sum_exp_do_not_keepdims_example_expanded_cpu
• test_reduce_log_sum_exp_do_not_keepdims_random_cpu
• test_reduce_log_sum_exp_do_not_keepdims_random_expanded_cpu
• test_reduce_log_sum_exp_keepdims_example_cpu
• test_reduce_log_sum_exp_keepdims_example_expanded_cpu
• test_reduce_log_sum_exp_keepdims_random_cpu
• test_reduce_log_sum_exp_keepdims_random_expanded_cpu
• test_reduce_log_sum_exp_negative_axes_keepdims_example_cpu
• test_reduce_log_sum_exp_negative_axes_keepdims_example_expanded_cpu
• test_reduce_log_sum_exp_negative_axes_keepdims_random_cpu
• test_reduce_log_sum_exp_negative_axes_keepdims_random_expanded_cpu
• test_reduce_log_sum_negative_axes_cpu
• test_reduce_log_sum_negative_axes_expanded_cpu
• test_reduce_max_do_not_keepdims_example_cpu
• test_reduce_max_do_not_keepdims_random_cpu
• test_reduce_max_keepdims_example_cpu
• test_reduce_max_keepdims_random_cpu
• test_reduce_max_negative_axes_keepdims_example_cpu
• test_reduce_max_negative_axes_keepdims_random_cpu
• test_reduce_mean_default_axes_keepdims_example_cpu
• test_reduce_mean_default_axes_keepdims_random_cpu
• test_reduce_mean_do_not_keepdims_example_cpu
• test_reduce_mean_do_not_keepdims_random_cpu
• test_reduce_mean_keepdims_example_cpu
• test_reduce_mean_keepdims_random_cpu
• test_reduce_mean_negative_axes_keepdims_example_cpu
• test_reduce_mean_negative_axes_keepdims_random_cpu
• test_reduce_min_do_not_keepdims_example_cpu
• test_reduce_min_do_not_keepdims_random_cpu
• test_reduce_min_keepdims_example_cpu
• test_reduce_min_keepdims_random_cpu
• test_reduce_min_negative_axes_keepdims_example_cpu
• test_reduce_min_negative_axes_keepdims_random_cpu
• test_reduce_prod_do_not_keepdims_example_cpu
• test_reduce_prod_do_not_keepdims_random_cpu
• test_reduce_prod_keepdims_example_cpu
• test_reduce_prod_keepdims_random_cpu
• test_reduce_prod_negative_axes_keepdims_example_cpu
• test_reduce_prod_negative_axes_keepdims_random_cpu
• test_reduce_sum_default_axes_keepdims_example_cpu
• test_reduce_sum_default_axes_keepdims_random_cpu
• test_reduce_sum_do_not_keepdims_example_cpu
• test_reduce_sum_do_not_keepdims_random_cpu
• test_reduce_sum_empty_axes_input_noop_example_cpu
• test_reduce_sum_keepdims_example_cpu
• test_reduce_sum_keepdims_random_cpu
• test_reduce_sum_negative_axes_keepdims_example_cpu
• test_reduce_sum_negative_axes_keepdims_random_cpu
• test_reduce_sum_square_default_axes_keepdims_example_cpu
• test_reduce_sum_square_default_axes_keepdims_example_expanded_cpu
• test_reduce_sum_square_default_axes_keepdims_random_cpu
• test_reduce_sum_square_default_axes_keepdims_random_expanded_cpu
• test_reduce_sum_square_do_not_keepdims_example_cpu
• test_reduce_sum_square_do_not_keepdims_example_expanded_cpu
• test_reduce_sum_square_do_not_keepdims_random_cpu
• test_reduce_sum_square_do_not_keepdims_random_expanded_cpu
• test_reduce_sum_square_keepdims_example_cpu
• test_reduce_sum_square_keepdims_example_expanded_cpu
• test_reduce_sum_square_keepdims_random_cpu
• test_reduce_sum_square_keepdims_random_expanded_cpu
• test_reduce_sum_square_negative_axes_keepdims_example_cpu
• test_reduce_sum_square_negative_axes_keepdims_example_expanded_cpu
• test_reduce_sum_square_negative_axes_keepdims_random_cpu
• test_reduce_sum_square_negative_axes_keepdims_random_expanded_cpu

[attila-dusnoki-htec]

To reproduce: python onnx_backend_test.py -t test_reduce

The ful log is attached: onnx_test_reduce_latest.log

[music-dino]

Several issues are at play.

1. Any do_not_keepdims tests must stay disabled, due to squeeze not supporting axes as a variable input. Adding support for this to squeeze itself doesn't seem too difficult, but updating all other operators to account for the possibility of a missing dynamic axes would be arduous and probably not very pretty.

2. Reduce ops need to be offloaded to the CPU when axes is a variable input.

3. The onnx node tests for reduce ops have axes as an input of variable length. MGX onnx_backend_test has no facilities to forward either a default fixed dimension value or a dynamic dimension value, which means that these graphs get parsed with the default dimension value, which is {1}. For some tests this is O.K. as the onnx test indeed passes a value of dimension {1}, for others it is not, as a value of dimension {0} is passed. It follows that the tests that do actually work are a happy accident as the default dimension corresponds to what the test does.

4. Some situations where the reduce op returns a dynamic shape are hampered by the fact that pointwise ops do not have support for dynamic shapes. This applies to reduce_log_sum, reduce_log_sum_exp, reduce_l2, and all reduce operators if they're used in a where, which will happen if axes is a dynamic variable input.

5. The rewrite reduce mean pass causes an error when reduce mean returns a dynamic shape. The pass should ignore the reduce mean if it returns a dynamic shape.

6. Negative axes tests will error out do to a bug in the ref operator implementation, the axes should be normalized after they're extracted from the axes arg.