Could we get the trained model weights for all the ~5M models?

[goodboyanush]

Are you planning to make that available, atleast on a request basis.

It could help use cases such as retraining of models, getting intermediate outputs to analyze the nature of the search spaces.

[chrisying]

Hi goodboyanush,

Originally we wanted to release all trained model weights but the total file size is well over a petabyte and we haven't found an effective way to serve this data to researchers.

Is there some specific use-case you are looking for? It may be possible to regenerate the checkpoints by retraining a single model using the open-sourced code.

[coallaoh]

Hi Chris,

Thanks for open-sourcing the interesting code base!

Maybe more realistic request is to open source at least the training code because that is already non-trivial. That code would consist of two parts, (1) going from ModelSpec(matrix, ops) to full tensorflow code for building the corresponding graph and (2) running optimization steps with pre-defined hyperparameters. Writing those codes and trying to be 100% identical to your implementation would be quite a bit of work, if not impossible. Since you already have code for this, can you open source it? (Or is it already there and I missed it?)

[coallaoh]

Never mind my comment above - found those codes in the repository.
https://github.com/google-research/nasbench/blob/master/nasbench/lib/evaluate.py
https://github.com/google-research/nasbench/blob/master/nasbench/lib/model_builder.py