Hybrid Training

In some cases, training a convolutional network using batches of patches extracted from a dataset can lead to better results than training on batches of images. We added a hybrid training model to CN24 that makes patch-wise training possible while still leveraging the performance benefits of fully convolutional testing.

Configuration

To enable hybrid training, simply add the following line to the network configuration file:

method=patch

You may also want to increase the number of iterations per epoch and change the hyperparameters governing the learning rate. Larger batch sizes are also recommended. The following numbers can be used as a starting point:

lr=0.004
gamma=0.0005
exponent=0.75
iterations=10000
sbatchsize=1
pbatchsize=48

Limitations

At this time, not all layer types are supported. Unsupported layers will be ignored by the configuration parser resulting in different network architectures for training and testing. This will lead to unpredictable results. The following features are currently not supported for hybrid training:

• Compressed TensorStreams
• Architectures with strided and padded convolutions

CN24 will not stop you from using those features, but the results are unpredictable.