Searching for small models is inefficient

[DrChainsaw]

The default fitness function for image classifiers is to truncate the fitness to two digits and use the remaining digits for the size.

The idea is that when two models have the same accuracy the one with the smallest size is considered "fittest".

This works to some extent, but for datasets which are very easy to fit (e.g. MNIST) the search for the smallest model is very slow and inefficient. This is because all models end up with basically the same fitness in the end (e.g. 1.0000xxxxx), meaning that sus selection will basically select the whole population again each generation.

The only way a "too big" candidate can be not selected is if it would accidentally be mutated to get a very low accuracy. However, the very same can happen to a small model as well, making the elite selection the only mechanism which actually searches for smaller models and this is very slow.