Kaggle: COVID-19 Challenge

This library provides tools aiming to find different opinions in the scientific litterature regarding the user query.

The Kaggle notebook can be find here.

Birielfy:

• It loads all articles into an SQLite DB.
• Sentences are pre-processed.
• Word2vec and TF-IDF are trained.
• Sentences are vectorised.
• The query is pre-processed and vectorised.
• The distance between query and sentences is computed.
• The top-k sentences are kept.
• A clustering is applied on these sentences.
• A ranking regarding its proximity to the centroid and authors of the papers.

Installation

Simply use:

pip install -q git+https://github.com/MrMimic/covid-19-kaggle

An then the library can be imported with:

from c19 import parameters, database_utilities, text_preprocessing, embedding, query_matching, clusterise_sentences, plot_clusters, display_output

Usage

Create the database

Please use this script to create the local database.

Query the DB

Please use this one to query the trained DB.

Re-train the W2V and TF-IDF

This script allows to re-train the W2V and TF-IDF to re-generate the parquet file.

Usage

All queries from the Kaggle challenge have been reformulated here. They have then been processed with the tool presented here.

Results are visible on Kaggle.