integration of dataset

[mshbaita-jo]

The dataset you are using contains the audios and text in one data frame. But for me I have a folder that contains the audios with mp3 format and another TSV file that contains the names of the audios, the texts, and the speaker_id. How can I handle this dataset? and integrate it with the json configuration file?

[ylacombe]

Hey @DhanaTechAi, sorry for the wait, what I'd recommend is reading the tsv file, transform it into a dictionnary looking like that:

data_dict = {
"audio": LIST_OF_ABSOLUTE_PATH_TO_AUDIO,
"text": LIST_OF_TEXT,
"speaker_id": LIST_OF_SPEAKER_ID
}

Then do:

from datasets import DatasetDict, Audio

dataset = DatasetDict.from_dict(data_dict).convert_column("audio", Audio())
dataset.push_to_hub(REPO_ID)

Then you can use the configuration file replacing the data id with REPO_ID, and the audio and text column with "audio" and "text". Hope that helps