Encoder-Decoder RNNs

[characat0]

This issue came up while implementing an Encoder-Decoder ConvLSTM, but I think is applicable to all kind of RNNs in Lux.
The approach I'm following is to feed the encoder with my input x, obtain (y, carry) and feed the decoder with those.
I'm using Recurrence to wrap the encoder, but I've noticed that the output ignores carry and returns y alone.

Is there any combination of layers that can be used for this purpose?
To overcome this issue, I've implemented my own version of Recurrence that outputs both y and carry.

[avik-pal]

A custom layer is the best option here. Recurrence is meant to be used in cases that makes carry completely opague to the user.

[characat0]

Thanks!
This was a nightmare to implement in Flux, but Lux made it pretty easy

[characat0]

Coming back to this after a while.
Is there a way to add an additional parameter to Recurrence such that it can return the hidden state too?
I think sequence to sequence models are somewhat common and this might make it easier for users to adopt, since they need to hand off the hidden state from the encoder into the decoder.
Might not be the best example, but I was thinking of something similar to return_state in Keras.

[avik-pal]

yeah it makes sense to have an option for that