Motivation and Context

SwishFunction is not exportable due to memory efficient implementation using torch.autograd. I have added simplified swish function in eval model to enable export.

Issue: #184

Utilisation (torchscript):

model.eval()
traced_model = torch.jit.trace(model, input)

Utilisation (onnx):

model.eval()
torch.onnx.export(model, input, ......)

Utilisation (tensorRT):

from torch2trt import torch2trt
model.eval()
model_trt = torch2trt(model, ...)

How Has This Been Tested

I have tested the implementation myself, but I have not created a test for it.

Types of changes

• Docs change / refactoring / dependency upgrade
• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that would cause existing functionality to change)

Checklist

• My code follows the code style of this project.
• My change requires a change to the documentation.
• I have updated the documentation accordingly.
• I have read the CONTRIBUTING document.
• I have completed my CLA (see CONTRIBUTING)
• I have added tests to cover my changes.
• All new and existing tests passed.