Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[feat] Change TF spectral ops to torchaudio #7

Open
wants to merge 3 commits into
base: master
Choose a base branch
from

Conversation

gudgud96
Copy link
Owner

@gudgud96 gudgud96 commented Nov 26, 2023

I faced some deadlock issue when running with more than 4 dataloader workers, analyzed and found that the issue is due to stuck in TF functions in spectrogram.py.

TF can't be entirely migrated unless we can rewrite vocabularies.py and migrate entirely from seqio and t5. So, the main changes are:

  1. Reduced num_workers, increased every_n_epochs for checkpointing, check_val_every_n_epoch for validation;
  2. Migrate spectrogram.py to use torchaudio
  3. Added use_tf_spectral_ops in dataset and during evaluation, to choose whether to use TF or torchaudio's melspectrogram (default to torchaudio)
  4. Removed redundant TF related code in vocabularies.py and metrics_utils.py

Other minor changes:

  1. Added eval flags for configs needed when running test.py
  2. Added split_frame_length in dataset config (2000 for current training, mel_length if wanted to ensure contiguous frames)
  3. Added is_deterministic, is_randomize_tokens in dataset config

@gudgud96 gudgud96 requested a review from KinWaiCheuk December 7, 2023 15:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant