Olmo1124ForCausalLM config.

[vwxyzjn]

Convert the checkpoint

Step 1: download an olmo checkpoint (e.g,. from Luca)
Step 2: install the custom branch: pip install git+https://github.com/vwxyzjn/transformers.git@olmo1124_classification
Step 3: run the converter (replace exp_name with huggingface revision you want to use)

exp_name=peteish7-anneal-from-928646-50B-nowup-moremath-dclm07-fw2
python scripts/convert_olmo_1124_weights_to_hf.py \
    --input_dir models/$exp_name  --output_dir "models/${exp_name}_hf"
huggingface-cli upload --revision "${exp_name}_hf" allenai/open_instruct_dev "models/${exp_name}_hf" .

Run SFT / DPO training

To make our codebase / image work with Olmo1124ForCausalLM, we just need to run pip install git+https://github.com/vwxyzjn/transformers.git@olmo1124_classification before running the SFT or DPO commands like this:

arguments: ['
+        unset CUDA_LAUNCH_BLOCKING && PYTHONPATH="/stage:$PYTHONPATH" pip install git+https://github.com/vwxyzjn/transformers.git@olmo1124_classification && accelerate launch
-        unset CUDA_LAUNCH_BLOCKING && PYTHONPATH="/stage:$PYTHONPATH" accelerate launch
        --mixed_precision bf16
        --num_machines 4

You can also launch on augusta using the following command (augusta only works with costah/open_instruct_ppo_ray_ninja image):

python scripts/submit_finetune_job.py \
    --cluster ai2/augusta-google-1 \
    --priority high \
    --workspace ai2/tulu-3-dev \
    --num_nodes 4 \
    --image costah/open_instruct_ppo_ray_ninja \
    --default_beaker_config configs/beaker_configs/default_finetune_multinode.yaml \
    --config configs/train_configs/sft/tulu3_8b_preview_mix_v3.9.yaml \
    --exp_name olmo1124_finetune

My configs/train_configs/sft/tulu3_8b_preview_mix_v3.9.yaml yaml looks like

model_name_or_path: allenai/open_instruct_dev
model_revision: peteish7-anneal-from-928646-50B-nowup-moremath-dclm07-fw2-olmo_1124
use_flash_attn: true
tokenizer_name: allenai/open_instruct_dev
tokenizer_revision: peteish7-anneal-from-928646-50B-nowup-moremath-dclm07-fw2-olmo_1124
use_slow_tokenizer: true
dataset_mixer:
    # General datasets:
    allenai/tulu-v3.9-tmp: 1.0

    # Datasets removed because of licensing:
    # ai2-adapt-dev/flan_v2_converted: 89982 # ODC-BY
    # ai2-adapt-dev/sciriff_converted: 10000 # ODC-BY
    # ai2-adapt-dev/no_robots_converted: 9500 # NC
    # AI-MO/NuminaMath-TIR: 72441 # NC

max_seq_length: 2048
preprocessing_num_workers: 128
per_device_train_batch_size: 1 # note, this is set up for 8 GPUs
gradient_accumulation_steps: 4 # effective batch size 128 with 4 nodes
learning_rate: 5.0e-06
lr_scheduler_type: linear
warmup_ratio: 0.03
weight_decay: 0.0
num_train_epochs: 2
output_dir: /output/
with_tracking: true
report_to:
  - wandb
logging_steps: 1
checkpointing_steps: epoch
dataset_mix_dir: /output/
add_bos: true

[natolambert]

amazing tricks