Code

Feat fsdp diloco#29

Code

Open

samsja wants to merge 12 commits intomainPrimeIntellect-ai/OpenDiloco:mainfrom feat-fsdp-dilocoPrimeIntellect-ai/OpenDiloco:feat-fsdp-dilocoCopy head branch name to clipboard

+340

Commits

Commits on Sep 10, 2024

• wip add pure torch fsdp

  samsja

  committed
  75ede2a

  Copy full SHA for 75ede2a

• add something somewhat working

  samsja

  committed
  ee89d33

  Copy full SHA for ee89d33

Commits on Sep 11, 2024

• add wandb and real data

  samsja

  committed
  c217dd4

  Copy full SHA for c217dd4

• fix data

  samsja

  committed
  94bc0ae

  Copy full SHA for 94bc0ae

• do all reduce on cpu

  samsja

  committed
  5b701d7

  Copy full SHA for 5b701d7

• do all reduce on cpu

  samsja

  committed
  04dd585

  Copy full SHA for 04dd585

• fix opt step on cpu

  samsja

  committed
  e926e6c

  Copy full SHA for e926e6c

• fix opt step on cpu

  samsja

  committed
  9f680f2

  Copy full SHA for 9f680f2

• fix batch size stuff

  samsja

  committed
  8576f1c

  Copy full SHA for 8576f1c

Commits on Sep 24, 2024

• add helper script

  samsja

  committed
  71a9f89

  Copy full SHA for 71a9f89

• fix it

  samsja

  committed
  88b5250

  Copy full SHA for 88b5250

• fix it

  samsja

  committed
  5957cae

  Copy full SHA for 5957cae