Training with different numbers of speakers

[anontiger]

I'm trying to train recipes for different numbers of speakers, but I'm unsure how to do that.
I thought an easy way to do this would be to use loaders_from_mini, replace the contents of MiniLibriMix files downloaded with my own data and metadata, so that when seeing the loaders_from_mini, it would see the metadata I prepared and think it's MiniLibriMix data, and give n_src=3 for model = DPRNNTasNet(n_src=2).
It clearly doesn't work that way, as I found out. I got the following error:
TypeError: Inputs must be of shape [batch, n_src, time], got torch.Size([16, 2, 24000]) and torch.Size([16, 3, 24000]) instead

Can someone explain how I can train a model to separate multiple speakers, or is it only possible to separate two speakers using asteroid? Do I have to create a custom dataset? Is there no simpler way?

[mpariente]

It is possible to separate more than two speakers, for sure.

Can you actually show the code ?

The simplest way (and it's quite simple) is to make a custom dataset, yes !

[anontiger]

My simple thought process was that since the loaders_from_mini showed me a layout of how the data should be arranged, I could simply replace the files there and metadata to train for different number of speakers.
Basically, since there was MiniLibriMix folder, I added s3 to the source folders and changed metadata so that they include 3 sources instead of 2 sources. My new train folder in the MiniLibriMix folder looked like this,

and my metadata like this:

from torch import optim
from pytorch_lightning import Trainer
from asteroid.losses import pairwise_neg_sisdr, PITLossWrapper
from asteroid.data import LibriMix
from asteroid.engine import System
from asteroid.models import DPRNNTasNet

train_loader, val_loader = LibriMix.loaders_from_mini(task="sep_clean", batch_size=16)

model = DPRNNTasNet(n_src=3)

loss = PITLossWrapper(pairwise_neg_sisdr, pit_from="pw_mtx")

optimizer = optim.Adam(model.parameters(), lr=1e-3)

system = System(model, optimizer, loss, train_loader, val_loader)

trainer = Trainer(max_epochs=1, gpus=1)

trainer.fit(system)

model.separate("trboth1.wav", force_overwrite=True,resample=True)

It didn't work, and since you say it's easier to make a custom dataset, but I'm trying to learn how to do that, but was my approach here so wrong?

[mpariente]

No, I think your approach is fine, but byb default the loader is intialized with 2 sources.

But the kwargs of loaders_from_mini are passed to the init. Can you try this ? train_loader, val_loader = LibriMix.loaders_from_mini(task="sep_clean", n_src=3, batch_size=16) instead ?

BTW, to format code, you can embed it in triple quotes ```, it's much better.

[anontiger]

Nice! Thank you. That solved that problem.
Though I had to reduce batch size because of CUDA out of memory problem, it trains properly now.
BTW, I run into this problem a lot, which makes me wonder if the people who develop these codes simply used superior GPU or some trick they use to avoid this problem. How do you guys run these at default GPU? Do you need multiple GPUs to run it at default batch size? I'm pretty sure it's not the newness of the GPU, because I've seen someone with RTX3090 with the same problem. So is it having multiple GPU? If so, can you point me in a direction where I can learn about it(website or search keywords or the like)? Because this problem has plagued me for a while, but when I search "how to avoid cuda out of memory", all I get is 'reduce batch size', or 'kill memory'. I KNOW there must be a way to run this at default batch size, and if it means I can be free of that CUDA out of memory problem forever, I'm actually willing to starve for a month to get money for a new GPU.

[mpariente]

Indeed, with a second GPU, you can get twice as big a batch size.

But DPRNN is particularly demanding for graphic memory, and works well for a smaller batch size anyway.

You can look for distributed training, that's how it's called.