This project aims to create a Question Answering (QA) system that can process Hindi contexts, translate them to English, and then provide answers to questions posed in English. The system utilizes a combination of translation services and machine learning models to achieve crosslingual QA capabilities.
- Translation: Converts Hindi context to English.
- Position Adjustment: Adjusts the answer start positions in the new translated context using fuzzyword.
- Model Training: Trains the QA model in batches.
- Evaluation: Evaluates the model performance.
- Prediction: Provides an app interface for predicting answers from Hindi context and English questions.
To get started with the project, clone the repository and install the necessary dependencies:
git clone https://github.com/natashapashupathi/Cross-Lingual-QA-System/tree/main-
Data Preparation: Convert the QA dataset from Hindi to English and adjust the answer start positions.
-
Training the Model: Train the QA model with a custom dataset.
-
Evaluation: Evaluate the trained model to determine its performance.
-
Prediction: Use the provided app interface to predict answers based on given Hindi context and English questions.
The data preparation involves translating the context from Hindi to English using google translate and adjusting the answer start positions using fuzzyword.
Train the model in batches using the translated dataset. You will see an error in the notebook under the train session, this is because the training was done using google colab T4 GPU since training on the notebook was taking very long. The trained model was then loaded into the notebook.
Evaluate the model performance using accuracy and F1 scores.
Use the provided app interface for predicting answers. Here is an example of the working application
