Skip to content

Commit 87510b3

Browse files
committed
feat: update micro batch size
1 parent b3e48c7 commit 87510b3

File tree

1 file changed

+1
-1
lines changed

1 file changed

+1
-1
lines changed

scripts/gcp/yi-1.5-9b.sh

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -75,7 +75,7 @@ NUM_GPUS=$((${NUM_NODES} * ${NUM_GPU_PER_NODE}))
7575
SEQ_LENGTH=4096
7676
DATA_PARALLEL_SIZE=$NUM_GPUS
7777

78-
MICRO_BATCH_SIZE=4
78+
MICRO_BATCH_SIZE=8
7979
GLOBAL_BATCH_SIZE=1024
8080
TRAIN_STEPS=25000
8181

0 commit comments

Comments
 (0)