Repository navigation

re-merge from NVIDIA main - #68

Open

RaymondLi0 wants to merge 28 commits into

multi-query-attentionfrom

NVIDIA-main2

Open

re-merge from NVIDIA main#68
RaymondLi0 wants to merge 28 commits into
multi-query-attentionfrom
NVIDIA-main2

Commits on May 19, 2023
- Add option to specify a data cache path separate from data directory.
  jaredcasper
  committed
  8c86034
Commits on May 23, 2023
- Check for write failure of index cache and print error message.
  jaredcasper
  committed
  ae37924
Commits on May 30, 2023
- Merge branch 'main' into data-cache
  jaredcasper
  committed
  41ba8a2
Commits on May 31, 2023
- Add option to overlap p2p communication.
  jaredcasper
  committed
  f9283c5
- typo
  jaredcasper
  committed
  621c9de
- Consistent arg names.
  jaredcasper
  committed
  2c13d1f
Commits on Jun 1, 2023
- Merge branch 'p2p_overlap' into 'main'
  jaredcasper
  committed
  e6d7e09
- Merge branch 'data-cache' into 'main'
  jaredcasper
  committed
  e4adfbc
Commits on Jun 2, 2023
- Supporting loading checkpoints without add_position_embedding arg.
  jaredcasper
  committed
  4d564cf
- Fix GPTDataset assert.
  jaredcasper
  committed
  1997e94
Commits on Jun 5, 2023
- Merge branch 'gptdataset-assert' into 'main'
  jaredcasper
  committed
  f965380
- Fixed rotary_pos_emb's position in layer's forward args.
  
  lmcafee-nvidia
  authored and
  jaredcasper
  committed
  a6c574d
- Merge branch 'lmcafee/rotary-kwarg-dev' into 'main'
  jaredcasper
  committed
  382fd9d
- fix indexation for output tensor after gradscaler call
  abhinav-khattar
  committed
  41221b8
- Merge branch 'outputtensor_index' into 'main'
  jaredcasper
  committed
  d2891b4
Commits on Jun 6, 2023
- Perform grad sync at correct place in interleaved pipeline parallelism
  
  timmoon10
  authored and
  jaredcasper
  committed
  ea76ecd
- Merge branch 'interleaved-pipeline-bugfix' into 'main'
  jaredcasper
  committed
  992da75
- Merge branch 'ckpt-load-fix' into 'main'
  jaredcasper
  committed
  f6c6d86
Commits on Jun 8, 2023
- Add workarounds for non-determinism in Megatron training
  jon-barker
  committed
  2880267
- Merge branch 'jbarker/non_determinism_fix' into 'main'
  jon-barker
  committed
  db71a33
Commits on Jun 10, 2023
- Update gitlab to catch pytest errors
  shanmugamr1992
  committed
  1af380d
- Merge branch 'pytestError' into 'main'
  shanmugamr1992
  committed
  c7a0145
Commits on Jun 12, 2023
- Remove use of deprecated np.float in indexed_dataset.py
  
  jon-barker
  authored and
  jaredcasper
  committed
  bf5206e
- Merge branch 'jbarker/np_float64_deprecation' into 'main'
  jaredcasper
  committed
  000590e
Commits on Jun 13, 2023
- Retro fix for tensor parallelism.
  
  lmcafee-nvidia
  authored and
  jaredcasper
  committed
  f479999
- Merge branch 'lmcafee/retro-dataloader-fix' into 'main'
  jaredcasper
  committed
  0604155
Commits on Jun 23, 2023
- Merge branch 'main' of github.com:NVIDIA/Megatron-LM into NVIDIA-main2
  RaymondLi0
  committed
  e5a9d7b
Commits on Jun 27, 2023
- pass on data_cache_pass in build_dataset_group
  Raymond Li
  committed
  8b6ceeb

Commits

Commits on May 19, 2023

Commits on May 23, 2023

Commits on May 30, 2023

Commits on May 31, 2023

Commits on Jun 1, 2023

Commits on Jun 2, 2023

Commits on Jun 5, 2023

Commits on Jun 6, 2023

Commits on Jun 8, 2023

Commits on Jun 10, 2023

Commits on Jun 12, 2023

Commits on Jun 13, 2023

Commits on Jun 23, 2023

Commits on Jun 27, 2023