Handling isomorphic architectures in benchmarked algorithms ?

[mikelzc1990]

I was wondering if the benchmarked algorithms in the paper (Figure 7) handle isomorphic architectures. To be more specific, when an algorithm creates a new architecture, do you check against all evaluated architectures for isomorphism or not?

The random search and the regularized evolution methods implemented in your NASBench.ipynb seem to only check if an architecture is valid. I did a quick experiment that collects the MD5 hash for each evaluated architecture. The results show that, on average over 11 repeats, more than 98% of the architectures created by regularized evolution are isomorphic.

[aaronkl]

Hi,
no we do not check for isomorphism, since all methods that we used are not able to handle that natively. But I think it would be very interesting future work to somehow exploit isomorphism inside the NAS algorithm.