Unfreezing weights during training

[amoskalev]

Hi, thanks for your work,

Can you explain why do you have a training procedure with a gradual weights unfreezing? Why do you never train the first layer? What was the motivation?

Thanks,

[amoskalev]

Hi, @JudasDie @Sy-Zhang @zhangliliang @HYPJUDY @ZhichengHuang can you please clarify the weight unfreezing procedure?

Thanks

[JudasDie]

Hi, @JudasDie @Sy-Zhang @zhangliliang @HYPJUDY @ZhichengHuang can you please clarify the weight unfreezing procedure?

Thanks

Hi, sorry for my late replay. 1) Gradual weights unfreezing is faster than unfixing all layers at the beginning of training. 2) Of course you could train the first layer. However, in my experience, it doesn't bring more gains.

[ISosnovik]

@JudasDie Is it possible then to change the number of epochs or the number of steps or the learning rate, such that the same result is achieved faster?

[ISosnovik]

@JudasDie