Merge main into dev branch #14913
Draft
Merge main into dev branch #14913
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
* Support QwenVL for inference engine * Apply isort and black reformatting Signed-off-by: meatybobby <[email protected]> * Remove comment out * Reformat * Skip pylint check * Add unit tests * Apply isort and black reformatting Signed-off-by: meatybobby <[email protected]> --------- Signed-off-by: meatybobby <[email protected]> Co-authored-by: meatybobby <[email protected]>
* Fix sequence packing loss calculation Signed-off-by: Rayan Dasoriya <[email protected]> * Fix nemo2 path Signed-off-by: Rayan Dasoriya <[email protected]> * Skip pylint Signed-off-by: Rayan Dasoriya <[email protected]> --------- Signed-off-by: Rayan Dasoriya <[email protected]> Co-authored-by: Dmytro Pykhtar <[email protected]>
* [Audio]: added streaming mode to SpectrogramToAudio Signed-off-by: Rauf <[email protected]> * added time buffer Signed-off-by: Rauf <[email protected]> * renamed Nf -> num_frames Signed-off-by: Rauf <[email protected]> * added AudioToSpectrogram and scale and magnitude power Signed-off-by: Rauf <[email protected]> * added multiple chunking support Signed-off-by: Rauf <[email protected]> * added properties _stream_initialized, _eps, got rid of _prev_spec_frame Signed-off-by: Rauf <[email protected]> * added hanning window Signed-off-by: Rauf <[email protected]> * Apply isort and black reformatting Signed-off-by: nasretdinovr <[email protected]> * added a docstring regarding streaming istft mode Signed-off-by: Rauf <[email protected]> --------- Signed-off-by: Rauf <[email protected]> Signed-off-by: nasretdinovr <[email protected]> Co-authored-by: nasretdinovr <[email protected]>
) Signed-off-by: Zhiyu Li <[email protected]>
…rs (#14514) * Update evo2 defaults so converted checkpoints have the right parameters Signed-off-by: John St John <[email protected]> * Fix line too long issue Signed-off-by: John St John <[email protected]> * Fix expected changes to configs that are locked into our tests Signed-off-by: John St John <[email protected]> --------- Signed-off-by: John St John <[email protected]>
Signed-off-by: dimapihtar <[email protected]>
Signed-off-by: Malay Nagda <[email protected]>
…t_store flags (#14522) * Add use te activation func and save act input in fp8 flags Signed-off-by: Guyue Huang <[email protected]> * Fix field name Signed-off-by: Guyue Huang <[email protected]> * Update scripts/performance/vlm/finetune_qwen25vl_32b.py Co-authored-by: malay-nagda <[email protected]> Signed-off-by: Guyue Huang <[email protected]> --------- Signed-off-by: Guyue Huang <[email protected]> Signed-off-by: Guyue Huang <[email protected]> Co-authored-by: malay-nagda <[email protected]>
* Bump TE and Mcore Signed-off-by: Charlie Truong <[email protected]> * Use Mcore 69b65 Signed-off-by: Charlie Truong <[email protected]> --------- Signed-off-by: Charlie Truong <[email protected]>
* remove sync in logging Signed-off-by: qiyuw <[email protected]> * Apply isort and black reformatting Signed-off-by: WanZzzzzz <[email protected]> * add class and func docstrings in data_sampler.py for pylint Signed-off-by: qiyuw <[email protected]> * Apply isort and black reformatting Signed-off-by: WanZzzzzz <[email protected]> --------- Signed-off-by: qiyuw <[email protected]> Signed-off-by: WanZzzzzz <[email protected]> Co-authored-by: qiyuw <[email protected]> Co-authored-by: WanZzzzzz <[email protected]>
* add 1b arclongcontextconfig Signed-off-by: Farhad Ramezanghorbani <[email protected]> * fix device mess Signed-off-by: Farhad Ramezanghorbani <[email protected]> * add implicit_filter support Signed-off-by: Farhad Ramezanghorbani <[email protected]> * use padded input Signed-off-by: Farhad Ramezanghorbani <[email protected]> * Apply isort and black reformatting Signed-off-by: farhadrgh <[email protected]> * Revert "add 1b arclongcontextconfig" This reverts commit 029969b. --------- Signed-off-by: Farhad Ramezanghorbani <[email protected]> Signed-off-by: farhadrgh <[email protected]>
* fix gemma2 27b kv dimension Signed-off-by: Ananth Subramaniam <[email protected]> * fix gemma2 27b kv dimension Signed-off-by: Ananth Subramaniam <[email protected]> --------- Signed-off-by: Ananth Subramaniam <[email protected]>
* feat: print expert groups on megatron init (#13874) Signed-off-by: Alexander Zhipa <[email protected]> Co-authored-by: Alexander Zhipa <[email protected]> Signed-off-by: CarlosGomes98 <[email protected]> * set a different seed for each dp rank Signed-off-by: CarlosGomes98 <[email protected]> * calculate loss inside autocast Signed-off-by: CarlosGomes98 <[email protected]> * disable per token loss, grad acc fusion Signed-off-by: CarlosGomes98 <[email protected]> * add missing self.seed Signed-off-by: CarlosGomes98 <[email protected]> * black formatting Signed-off-by: CarlosGomes98 <[email protected]> * Apply isort and black reformatting Signed-off-by: gautham-kollu <[email protected]> --------- Signed-off-by: Alexander Zhipa <[email protected]> Signed-off-by: CarlosGomes98 <[email protected]> Signed-off-by: gautham-kollu <[email protected]> Co-authored-by: Alexander Zhipa <[email protected]> Co-authored-by: Alexander Zhipa <[email protected]> Co-authored-by: gautham-kollu <[email protected]> Co-authored-by: gautham-kollu <[email protected]>
* [Flux] Add MXFP8 support. Signed-off-by: Wil Kong <[email protected]> * [Flux] Add current and block scaling. Signed-off-by: Wil Kong <[email protected]> --------- Signed-off-by: Wil Kong <[email protected]>
Signed-off-by: Ao Tang <[email protected]>
…li triplet dataset with NeMo Framework (#14584) * Create E2E-Embedding-Finetuning Signed-off-by: Hemant Giri <[email protected]> * Update E2E-Embedding-Finetuning Signed-off-by: Hemant Giri <[email protected]> * Delete tutorials/llm/embedding/E2E-Embedding-Finetuning Signed-off-by: Hemant Giri <[email protected]> * Create README.md Signed-off-by: Hemant Giri <[email protected]> * Add files via upload Signed-off-by: Hemant Giri <[email protected]> * Add files via upload This is a notebook for E2E finetuning a embedding model Signed-off-by: Hemant Giri <[email protected]> * Update README.md Signed-off-by: Hemant Giri <[email protected]> * Update README.md Signed-off-by: Hemant Giri <[email protected]> * Delete tutorials/llm/embedding/E2E-Embedding-Finetuning/download_dataset.py Signed-off-by: Hemant Giri <[email protected]> * Delete tutorials/llm/embedding/E2E-Embedding-Finetuning/finetune_e5.py Signed-off-by: Hemant Giri <[email protected]> * Delete tutorials/llm/embedding/E2E-Embedding-Finetuning/finetune_llama1b.py Signed-off-by: Hemant Giri <[email protected]> * Delete tutorials/llm/embedding/E2E-Embedding-Finetuning/import_e5_large.py Signed-off-by: Hemant Giri <[email protected]> * Delete tutorials/llm/embedding/E2E-Embedding-Finetuning/import_llama1b.py Signed-off-by: Hemant Giri <[email protected]> --------- Signed-off-by: Hemant Giri <[email protected]> Co-authored-by: Ao Tang <[email protected]>
Signed-off-by: Guyue Huang <[email protected]>
Signed-off-by: dimapihtar <[email protected]>
Signed-off-by: jenchen13 <[email protected]>
Signed-off-by: Chen Cui <[email protected]>
Signed-off-by: Rauf <[email protected]>
Signed-off-by: Chen Cui <[email protected]>
Signed-off-by: Charlie Truong <[email protected]>
* fix flux seed as optional Signed-off-by: Ao Tang <[email protected]> * fix fluxcontrolnet Signed-off-by: Ao Tang <[email protected]> * Fix code checkout during test Signed-off-by: Charlie Truong <[email protected]> --------- Signed-off-by: Ao Tang <[email protected]> Signed-off-by: Charlie Truong <[email protected]> Co-authored-by: Charlie Truong <[email protected]>
Signed-off-by: Jason <[email protected]>
* Remove PEFT scheme condition from recipe Signed-off-by: Ali Taghibakhshi <[email protected]> * remove unnecessary peft conditioning 12b --------- Signed-off-by: Ali Taghibakhshi <[email protected]>
* add gpt-oss lora exporter Signed-off-by: Chen Cui <[email protected]> * Apply isort and black reformatting Signed-off-by: cuichenx <[email protected]> * update lora exporter for experts Signed-off-by: Chen Cui <[email protected]> * disallow exporting expert lora since nemo implementation is not equivalent to hf Signed-off-by: Chen Cui <[email protected]> * linting Signed-off-by: Chen Cui <[email protected]> * Apply isort and black reformatting Signed-off-by: cuichenx <[email protected]> * address comment Signed-off-by: Chen Cui <[email protected]> --------- Signed-off-by: Chen Cui <[email protected]> Signed-off-by: cuichenx <[email protected]> Co-authored-by: cuichenx <[email protected]> Co-authored-by: Charlie Truong <[email protected]>
* update streaming ASR Signed-off-by: stevehuang52 <[email protected]> * add voice agent Signed-off-by: stevehuang52 <[email protected]> * update readme Signed-off-by: stevehuang52 <[email protected]> * update websocket Signed-off-by: stevehuang52 <[email protected]> * update Signed-off-by: stevehuang52 <[email protected]> * update Signed-off-by: stevehuang52 <[email protected]> * update readme Signed-off-by: stevehuang52 <[email protected]> * update Signed-off-by: stevehuang52 <[email protected]> * clean up Signed-off-by: stevehuang52 <[email protected]> * clean up Signed-off-by: stevehuang52 <[email protected]> * fix typo Signed-off-by: stevehuang52 <[email protected]> * fix codeQL Signed-off-by: stevehuang52 <[email protected]> * update cfg Signed-off-by: stevehuang52 <[email protected]> * remove unused Signed-off-by: stevehuang52 <[email protected]> * update readme Signed-off-by: stevehuang52 <[email protected]> * change default models Signed-off-by: stevehuang52 <[email protected]> * fix diar diable Signed-off-by: stevehuang52 <[email protected]> * fix diar diable Signed-off-by: stevehuang52 <[email protected]> * update ux Signed-off-by: stevehuang52 <[email protected]> * update tts Signed-off-by: stevehuang52 <[email protected]> * update readme Signed-off-by: stevehuang52 <[email protected]> * fix and update Signed-off-by: stevehuang52 <[email protected]> * fix asr Signed-off-by: stevehuang52 <[email protected]> * update readmme Signed-off-by: stevehuang52 <[email protected]> * update doc and llm dtype Signed-off-by: stevehuang52 <[email protected]> * refactor and add example prompts Signed-off-by: stevehuang52 <[email protected]> * update readme Signed-off-by: stevehuang52 <[email protected]> * update readme Signed-off-by: stevehuang52 <[email protected]> * clean up Signed-off-by: stevehuang52 <[email protected]> * clean up Signed-off-by: stevehuang52 <[email protected]> * update info on streaming sortformer Signed-off-by: stevehuang52 <[email protected]> * move code to 'nemo/agents/voice_agent' Signed-off-by: stevehuang52 <[email protected]> * update doc Signed-off-by: stevehuang52 <[email protected]> * clean up Signed-off-by: stevehuang52 <[email protected]> * refactor Signed-off-by: stevehuang52 <[email protected]> * update doc Signed-off-by: stevehuang52 <[email protected]> * remove the unnecessary streaming state conversion and import it from sortformer_modules, remove PostProcessingParams Signed-off-by: Weiqing Wang <[email protected]> * Apply isort and black reformatting Signed-off-by: weiqingw4ng <[email protected]> * update doc Signed-off-by: stevehuang52 <[email protected]> * clean up Signed-off-by: stevehuang52 <[email protected]> * fix for llama-nemotron template, and refactor Signed-off-by: stevehuang52 <[email protected]> * fix tts separator Signed-off-by: stevehuang52 <[email protected]> * fix for llama-nemotron Signed-off-by: stevehuang52 <[email protected]> * update cfg Signed-off-by: stevehuang52 <[email protected]> * refactor and update doc Signed-off-by: stevehuang52 <[email protected]> * change default llm to qwen Signed-off-by: stevehuang52 <[email protected]> * update doc Signed-off-by: stevehuang52 <[email protected]> --------- Signed-off-by: stevehuang52 <[email protected]> Signed-off-by: Weiqing Wang <[email protected]> Signed-off-by: weiqingw4ng <[email protected]> Co-authored-by: Taejin Park <[email protected]> Co-authored-by: Kunal Dhawan <[email protected]> Co-authored-by: Weiqing Wang <[email protected]> Co-authored-by: weiqingw4ng <[email protected]>
* beep boop: Update changelog Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> * Fix changelog for 2.4.1 Signed-off-by: Charlie Truong <[email protected]> --------- Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Signed-off-by: Charlie Truong <[email protected]> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Charlie Truong <[email protected]>
* Adding to false_positives.json Signed-off-by: Andrew Schilling <[email protected]> * Fixing redirected URLs in nlp_all.bib Signed-off-by: Andrew Schilling <[email protected]> --------- Signed-off-by: Andrew Schilling <[email protected]>
* add documentation for gpu phrase boosting Signed-off-by: andrusenkoau <[email protected]> * minor fixes Signed-off-by: andrusenkoau <[email protected]> * minor fixes Signed-off-by: andrusenkoau <[email protected]> * fix depth_scaling description Signed-off-by: andrusenkoau <[email protected]> * change default depth_scaling value Signed-off-by: andrusenkoau <[email protected]> * use default depth_scaling=1 for AED models Signed-off-by: andrusenkoau <[email protected]> * Apply isort and black reformatting Signed-off-by: andrusenkoau <[email protected]> * fixe broken link Signed-off-by: andrusenkoau <[email protected]> --------- Signed-off-by: andrusenkoau <[email protected]> Signed-off-by: andrusenkoau <[email protected]> Co-authored-by: andrusenkoau <[email protected]> Co-authored-by: Pablo Garay <[email protected]>
) Signed-off-by: Keval Morabia <[email protected]>
* add vllm support Signed-off-by: stevehuang52 <[email protected]> * refactor Signed-off-by: stevehuang52 <[email protected]> * update cfg Signed-off-by: stevehuang52 <[email protected]> * update cfg Signed-off-by: stevehuang52 <[email protected]> * update for nano-v2 Signed-off-by: stevehuang52 <[email protected]> * Potential fix for code scanning alert no. 16177: Unused import Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com> Signed-off-by: He Huang (Steve) <[email protected]> * update and refactor Signed-off-by: stevehuang52 <[email protected]> * update readme Signed-off-by: stevehuang52 <[email protected]> * update to pipecat=0.0.84 Signed-off-by: stevehuang52 <[email protected]> * add auto start/stop vllm server Signed-off-by: stevehuang52 <[email protected]> * update readme Signed-off-by: stevehuang52 <[email protected]> * auto switch between vllm and hf Signed-off-by: stevehuang52 <[email protected]> * refactor Signed-off-by: stevehuang52 <[email protected]> * update default cfg Signed-off-by: stevehuang52 <[email protected]> * add qwen3 example, refactor Signed-off-by: stevehuang52 <[email protected]> * update readme according to feedback Signed-off-by: stevehuang52 <[email protected]> * pin package version Signed-off-by: stevehuang52 <[email protected]> * Adding config manager and llm-specific yamls with short default yaml Signed-off-by: taejinp <[email protected]> * Apply isort and black reformatting Signed-off-by: tango4j <[email protected]> * Adding unit test for config_manager.py Signed-off-by: taejinp <[email protected]> * Resolving merge conflict on config manager Signed-off-by: taejinp <[email protected]> * Apply isort and black reformatting Signed-off-by: tango4j <[email protected]> * Resolving Code QL. Signed-off-by: taejinp <[email protected]> * Adding Conflict resolved config manager and test Signed-off-by: taejinp <[email protected]> * Apply isort and black reformatting Signed-off-by: tango4j <[email protected]> * Moving test files to example folder for cofinguration testing Signed-off-by: taejinp <[email protected]> * Removed backup file Signed-off-by: taejinp <[email protected]> * Adding config manager and llm-specific yamls and fixed the bugs Signed-off-by: taejinp <[email protected]> * Adding NeMoTron Nano-9B-v2 as a default Signed-off-by: taejinp <[email protected]> * Apply isort and black reformatting Signed-off-by: tango4j <[email protected]> * fix environment Signed-off-by: stevehuang52 <[email protected]> * fix hf param resolve Signed-off-by: stevehuang52 <[email protected]> * update readme Signed-off-by: stevehuang52 <[email protected]> * update config manager, add llama3.1 example, refactor config style Signed-off-by: stevehuang52 <[email protected]> * update default yaml Signed-off-by: stevehuang52 <[email protected]> * update readme Signed-off-by: stevehuang52 <[email protected]> * update readme Signed-off-by: stevehuang52 <[email protected]> * pin nemo to 2.5 Signed-off-by: stevehuang52 <[email protected]> * fix env and cfg Signed-off-by: stevehuang52 <[email protected]> * Removing Qwen from generic hf config Signed-off-by: taejinp <[email protected]> --------- Signed-off-by: stevehuang52 <[email protected]> Signed-off-by: He Huang (Steve) <[email protected]> Signed-off-by: taejinp <[email protected]> Signed-off-by: tango4j <[email protected]> Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com> Co-authored-by: Taejin Park <[email protected]> Co-authored-by: tango4j <[email protected]>
* remove local attn constraint Signed-off-by: Chen Cui <[email protected]> * fix Signed-off-by: Chen Cui <[email protected]> --------- Signed-off-by: Chen Cui <[email protected]>
* beep boop: Update changelog Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> * Remove 2.4.0 cherry-picks Signed-off-by: Charlie Truong <[email protected]> * Add speech highlights Signed-off-by: Charlie Truong <[email protected]> * Update changelog Signed-off-by: Charlie Truong <[email protected]> * Update the changelog Signed-off-by: Charlie Truong <[email protected]> --------- Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Signed-off-by: Charlie Truong <[email protected]> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
* add qwen3 flops fix Signed-off-by: gdeng <[email protected]> * update model flops Signed-off-by: gdeng <[email protected]> * fix the flop cal bug Signed-off-by: gdeng <[email protected]> * Apply isort and black reformatting Signed-off-by: gdengk <[email protected]> --------- Signed-off-by: gdeng <[email protected]> Signed-off-by: gdengk <[email protected]> Co-authored-by: gdengk <[email protected]>
…bucket (#14891) * [lhotse][aistore] added support input_cfg.yaml directly from aistore bucket Signed-off-by: Xuesong Yang <[email protected]> * fix: convert pythoon dict obj into DictConf Signed-off-by: Xuesong Yang <[email protected]> * move OmegaConf.create() outside of for loop. Signed-off-by: Xuesong Yang <[email protected]> --------- Signed-off-by: Xuesong Yang <[email protected]>
Signed-off-by: Jason <[email protected]>
blisc
requested review from
chtruong814,
ko3n1g,
pablo-garay and
thomasdhc
as code owners
github-actions
bot
added
core
Signed-off-by: Jason <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
51 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No description provided.