Relaxed Alignments?

[fermat97]

Is Algorithm 2: relaxed alignment included in this repository, since I could only see examples for categorical case. thank you.

[da03]

Nope, it's not included here. Relaxed alignment underperforms baseline soft attention (see Table 1 in our paper), and since we need to pretrain with E_p \log p and then finetune to get sensible results, it seems hacky to me. However, if you were interested, it is straightforward to implement using PyTorch's builtin Dirichlet rsample function: https://pytorch.org/docs/stable/_modules/torch/distributions/dirichlet.html#Dirichlet.rsample.