-
Notifications
You must be signed in to change notification settings - Fork 33
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Question generation for non English languages #5
Comments
What language are you considering? Try looking up Cross-Lingual Natural Language Inference (XNLI) and Cross-Lingual Question Answering (MLQA) to fine tune miniLm. If you require something different, consider procuring your own dataset for fine tuning. |
Hello
Im just having some troubles to create the top layer To make seq2seq generation.
Uf you could just explain it with some details how can i create it on my own that will be great .
…Sent from my iPhone
On 3 May 2020, at 00:03, artitw <[email protected]> wrote:
What language are you considering? Try looking up Cross-Lingual Natural Language Inference (XNLI) and Cross-Lingual Question Answering (MLQA) to fine tune miniLm. If you require something different, consider procuring your own dataset for fine tuning.
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub<#5 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AJEMYYKBBTDW2UODBBPMXOTRPSREZANCNFSM4MTUY2WA>.
|
Why do you have to create the top layer? Are you using the miniLM code available? |
I want understand and re build it my own , right now the seq2seq code of minilm is not adapted to the multilingual version . Thanks for your replies i really appriciate your help |
If you’re looking to customize the question generation component, take a look at
However, what aspects of your multilingual approach would require adaptation? |
The main problem is that the multilingual version is not as good as the native one , and NLG is a data-hungry task as you know . |
My impression is that you could use more training data for what you’re trying to achieve. Am I missing something? |
Hi @secsrexion How is your progress with non-english question generation. We are also interested in Chinese language QG task, and wondering how much work we might have to put to adapt the code provided by artitw. BTW great work and thanks @artitw for sharing the code! Have you published your work anywhere? |
Hello @thusithaC generally from what i found , using a multi-langual version of UNILM is a bad choice due to the lack of a rich repliable dataset in the training process , i was getting 80% from each phrase marked as UNK by the tokenizer . |
@artitw i was facing troubles with the multi-langual version and the quality of the dataset now i'm trying to develop a riable dataset for arabic QST/ANS , and i'm searching for a way to train new native version of the UNILM , any ideas ? |
@secsrexion Thanks for the reply. I saw your post on the UNILM github as well :) I wonder whether is quality issue you face is because the multi-lang modelsis based on "miniLM" i.e. the smaller model but this code-base is based on the full english unilm model, which is vastly superior? |
Hi |
Hi, |
@thusithaC @secsrexion @jacampo I am looking into making a multilingual model to see if and how it can be done. As @secsrexion pointed out, the low amounts of data need to be addressed. I will keep you all updated. |
Awesome! thanks,
…On Sun, Sep 13, 2020 at 12:33 AM artitw ***@***.***> wrote:
@thusithaC <https://github.com/thusithaC> @secsrexion
<https://github.com/secsrexion> @jacampo <https://github.com/jacampo> I
am looking into making a multilingual model to see if and how it can be
done. As @secsrexion <https://github.com/secsrexion> pointed out, the low
amounts of data need to be addressed. I will keep you all updated.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<#5 (comment)>, or
unsubscribe
<https://github.com/notifications/unsubscribe-auth/ABVRIJ2VDBVST2CQE52MFWDSFOPEJANCNFSM4MTUY2WA>
.
--
Thusitha Chandrapala
|
An alternative solution if you need something immediately is to translate to English and then use this model. |
@artitw Ok, thanks. The translation could be interresting too. |
@jacampo which BERT model are you referring to? If it uses WordPiece tokenization, I cannot think of any differences in the code used. |
I found one in spanish: https://github.com/dccuchile/beto You use BertForSeq2SeqDecoder, rigth? What is the diference between that and BertModel or BertForPreTraining? Sorry for disturbing so much with so many questions Edit: Ok i see you start with bert-base-cased, so my question is resolved, It is too much information at once, do you recommend a simple guide to understand the models and how to use it? |
@jacampo glad you figured it out. Yes, it is indeed confusing. It sounds like you would find a fine-tuning guide useful. I can think about how that might be done. In the meantime, if you find something that works please share back here. |
@artitw thanks, i'll let you know |
Anyone want to work together on this? I’ve started an approach to multilingual question generation and summarization but not had enough time to run experiments. I could provide some guidance for anyone interested in collaborating, as long as the work is contributed back to open source here. The approach would be based on cross-lingual models as I describe here: https://www.youtube.com/watch?v=caZLVcJqsqo |
Multilingual question generation is now available. Check out the latest version
|
Hello
I hope you are doing fine , firstly i thank you for your contributions on question generation , and i have a question if i may ask .
Im trying to build a question generation system for a non-English language i was planing to use UniLm ( miniLm multilingual version ) because bert is not really built for text generation since you have experience on that what how do you suggest to do that and am i following the good path .
Thank you in advance for your appreciated help !
The text was updated successfully, but these errors were encountered: