Skip to content

Latest commit

 

History

History

examples

Folders and files

NameName
Last commit message
Last commit date

parent directory

..
detxoify_lm
detxoify_lm
 
 
msdp
msdp
 
 
sc21
sc21
 
 
README.md
README.md
 
 
evaluate_retriever_nq.sh
evaluate_retriever_nq.sh
 
 
evaluate_zeroshot_gpt.sh
evaluate_zeroshot_gpt.sh
 
 
finetune_mnli_distributed.sh
finetune_mnli_distributed.sh
 
 
finetune_race_distributed.sh
finetune_race_distributed.sh
 
 
finetune_retriever_distributed.sh
finetune_retriever_distributed.sh
 
 
merge_mp_bert.sh
merge_mp_bert.sh
 
 
pretrain_bert.sh
pretrain_bert.sh
 
 
pretrain_bert_distributed.sh
pretrain_bert_distributed.sh
 
 
pretrain_bert_distributed_with_mp.sh
pretrain_bert_distributed_with_mp.sh
 
 
pretrain_gpt.sh
pretrain_gpt.sh
 
 
pretrain_gpt3_175B.sh
pretrain_gpt3_175B.sh
 
 
pretrain_gpt_distributed.sh
pretrain_gpt_distributed.sh
 
 
pretrain_gpt_distributed_with_mp.sh
pretrain_gpt_distributed_with_mp.sh
 
 
pretrain_ict.sh
pretrain_ict.sh
 
 
pretrain_t5.sh
pretrain_t5.sh
 
 
pretrain_t5_distributed.sh
pretrain_t5_distributed.sh
 
 
pretrain_t5_distributed_with_mp.sh
pretrain_t5_distributed_with_mp.sh
 
 
run_text_generation_server_345M.sh
run_text_generation_server_345M.sh
 
 
run_text_generation_server_345M_8_tensor_parallel.sh
run_text_generation_server_345M_8_tensor_parallel.sh
 
 

README.md

Original examples by NVIDIA/Megatron-LM

This folder includes examples from the original NVIDIA/Megatron-LM repo. All of them do NOT have DeepSpeed technologies integrations, and some of them may not work due to changes in this Megatron-DeepSpeed repo. Thus we recommend you to go to ../examples_deepspeed/ folder which includes examples that have DeepSpeed technologies integrated and are tested by DeepSpeed team.