Question about Search results and Retrain results

[Sunshine-Ye]

Dear author:
thanks for your effort! But i am confused about the results i get:

1.About Search results, i can only achieve 0.3346% after 40 epochs of searching, i try the search experiments twice and get similar results. The config_utils.search_args i used is default (except add dist=False), i dont know if there are some difference between the config_utils.search_args and that you used to get a 37% search results.

2.About Retrain results, i test the pre-retrained model of Autodeeplab-M you provided, but i can only get 72.38 mIoU with eval_scales=(0.25,0.5,0.75,1,1.25,1.5) and 76.66 mIoU with eval_scales=(1.0,). The config_utils.evaluate_args is incomplete, and i use following evaluate_args to match the pre-retrained model.

def obtain_evaluate_args():
parser = argparse.ArgumentParser(description='---------------------evaluate args---------------------')
parser.add_argument('--train', action='store_true', default=False, help='training mode')
parser.add_argument('--exp', type=str, default='bnlr7e-3', help='name of experiment')
parser.add_argument('--gpu', type=int, default=0, help='test time gpu device id')
parser.add_argument('--backbone', type=str, default='autodeeplab', help='resnet101')
parser.add_argument('--dataset', type=str, default='cityscapes', help='pascal or cityscapes')
parser.add_argument('--groups', type=int, default=None, help='num of groups for group normalization')
parser.add_argument('--epochs', type=int, default=30, help='num of training epochs')
parser.add_argument('--batch_size', type=int, default=10, help='batch size')
parser.add_argument('--base_lr', type=float, default=0.00025, help='base learning rate')
parser.add_argument('--last_mult', type=float, default=1.0, help='learning rate multiplier for last layers')
parser.add_argument('--scratch', action='store_true', default=False, help='train from scratch')
parser.add_argument('--freeze_bn', action='store_true', default=False, help='freeze batch normalization parameters')
parser.add_argument('--weight_std', action='store_true', default=False, help='weight standardization')
parser.add_argument('--beta', action='store_true', default=False, help='resnet101 beta')
parser.add_argument('--crop_size', type=int, default=513, help='image crop size')
parser.add_argument('--resume', type=str, default=None, help='path to checkpoint to resume from')
parser.add_argument('--workers', type=int, default=4, help='number of data loading workers')
parser.add_argument('--use_ABN', type=bool, default=False, help='whether use ABN') # False
parser.add_argument('--affine', default=True, type=bool, help='whether use affine in BN') # True
parser.add_argument('--dist', type=bool, default=False, help='whether to use Distribued Sampler (default: False)')
parser.add_argument('--network_arch', type=str, default=None, help='searched net_arch')
parser.add_argument('--cell_arch', type=str, default=None, help='searched cell_arch')
parser.add_argument('--num_classes', type=int, default=19)
parser.add_argument('--filter_multiplier', type=int, default=32) # 8
parser.add_argument('--block_multiplier', type=int, default=5)
parser.add_argument('--initial_fm', type=int, default=None) # 512
parser.add_argument('--eval_scales', default=(1.0,),
type=bool, help='whether use eval_scales') # (1.0,) (0.25,0.5,0.75,1,1.25,1.5)
args = parser.parse_args()
return args

3. some bugs (personal opinions).
   new_model.get_default_cell: cell[2] = [3, 6], cell[3] = [2, 4], cell[9] = [17, 5]
   new_model.get_default_arch: backbone = [0, 0, 0, 1, 2, 1, 2, 2, 3, 3, 2, 1]

Looking forward to your reply!

[MJITG]

Same. We only got 0.308 mIoU at epoch 33

[cardwing]

@Sunshine-Ye Thanks for your sharing. I wonder what is the performance of your searched architecture instead of the provided model? Thanks a lot.

[zhangximing666]

@Sunshine-Ye Thanks for sharing. I am also curious about the result of this model. Could you share the pre-retrained model author provided? The link in the original repository is broken. I reimplemented it in mindspore and trained the model of Autodeeplab-M but only got 77.62% miou.