You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Currently I have provided my own modded fork of transformers but if someone doesnt care about the features I have added and only wants to work with the code mbart code then this should be enabled.
What this would mean is that all those other arguments I pass to the mbart config class to instantiate the object will be sent to kwargs. The main change will be minimal and most likely related to the tokenizer. In the batch creation logic, I pass some extra arguments to the tokenizer to support stochastic tokenization. The way I see it is we have a flag called --is_official_repo which if passed means that the official transformers repo is passed. This argument will then be passed to the batching function which wont pass the flags relevant for stochastic tokenization.
The text was updated successfully, but these errors were encountered:
Currently I have provided my own modded fork of transformers but if someone doesnt care about the features I have added and only wants to work with the code mbart code then this should be enabled.
What this would mean is that all those other arguments I pass to the mbart config class to instantiate the object will be sent to kwargs. The main change will be minimal and most likely related to the tokenizer. In the batch creation logic, I pass some extra arguments to the tokenizer to support stochastic tokenization. The way I see it is we have a flag called --is_official_repo which if passed means that the official transformers repo is passed. This argument will then be passed to the batching function which wont pass the flags relevant for stochastic tokenization.
The text was updated successfully, but these errors were encountered: