Educational Question Generation of Children Storybooks via Question Type Distribution Learning and Event-centric Summarization
This repository is the official implementation of our paper. We consider generating high-cognitive-demand (HCD) educational questions by learning question type distribution and event-centric summarization.
Python>=3.6 is needed, run the following commands to install requirements:
cd transformers & pip install .
pip install spacy==2.3.7
pip install torch==1.7.1
pip install pytorch-lightning==0.9.0
pip install torchtext==0.8.0
pip install rouge-score==0.0.4
FairytaleQA can be found at here. We used the implementation in https://github.com/kelvinguu/qanli
to get the QA statements. The processed QA statement data can be found at here.
NOTE: I uploaded my modified
qanli
at here. You'll need to first get yourconllu
format file and get the transformation by runningstep3_totxt.py
(please update the paths accordingly).
Assuming we have the dataset at ./data/split
and the transformed QA statements at ./data/infrence
, we can prepare the needed format as follows:
python step1_toxlsx.py
python step2_topkl.py
python step3_topkllist.py
Paths need to be configured manually
- Question type distribution. In
tdl
folder,
python train.py
python predict.py
- Event-centric summary generation. In
section2sum
folder,
python train_section2sum.py
python generate_section2sum.py
- Educational question generation. In
sum2question
folder,
python train_sum2qustion.py
python generate_sum2question.py
-
Question type distribution here
-
Event-centric summary generation here
-
Educational question generation file1 file2, then join them as one file by
join summary2question_epoch=2.ckpt.* > summary2question_epoch=2.ckpt
- Automatic evaluation on Rouge-L and BERTScore:
- Human evaluation on question types (the K-L distance of question type distribution between our method and groudtruth is 0.28, while QAG (top2) is 0.60):
- Human evaluation on children appropriateness: the mean rating of our method (2.56±1.31) is significantly higher than the one of QAG (top2, 2.22±1.20).
This repository is developed based on FairytaleQA_QAG_System and FairytaleQA_Baseline.
@inproceedings{zhao2022storybookqag,
author = {Zhao, Zhenjie and Hou, Yufang and Wang, Dakuo and Yu, Mo and Liu, Chengzhong and Ma, Xiaojuan},
title = {Educational Question Generation of Children Storybooks via Question Type Distribution Learning and Event-Centric Summarization},
publisher = {Association for Computational Linguistics},
year = {2022}
}