Open
Description
In the zero-cost branch optimizers Npenas
and Bananas
, the validation accuracy of architectures is being queried from the zero-cost-benchmark as follows:
model.accuracy = self.zc_api[str(model.arch_hash)]['val_accuracy']
The question is, whether this supports the case where the user wants to use the ZeroCost
predictor because their dataset or search space is not supported by the zero-cost benchmark.
If this is a case that we want to support, one option would be to introduce a parameter use_zc_api
and use it as follows:
if self.use_zc_api:
model.accuracy = self.zc_api[str(model.arch_hash)]['val_accuracy']
else:
model.accuracy = model.arch.query(
self.performance_metric, self.dataset, dataset_api=self.dataset_api
)