Skip to content

Commit

Permalink
Update llama2-model.libsonnet
Browse files Browse the repository at this point in the history
  • Loading branch information
@ManfeiBai
ManfeiBai authored Aug 17, 2023
1 parent 01f5484 commit 315f2e7
Showing 1 changed file with 11 additions and 1 deletion.
12 changes: 11 additions & 1 deletion tests/pytorch/nightly/llama2-model.libsonnet
View file
Original file line number Diff line number Diff line change
Expand Up @@ -146,6 +146,16 @@ local utils = import 'templates/utils.libsonnet';
modelName+: '-n-i',
tpuSettings+: {
tpuVmExtraSetup: |||
sudo apt update
sudo apt-get -y install libopenblas-dev
pip install accelerate -U
sudo apt update
sudo apt-get -y install libopenblas-dev
pip3 uninstall -y torch torch_xla
pip3 install https://storage.googleapis.com/pytorch-xla-releases/wheels/tpuvm/torch-nightly-cp310-cp310-linux_x86_64.whl
pip3 install https://storage.googleapis.com/pytorch-xla-releases/wheels/tpuvm/torch_xla-nightly-cp310-cp310-linux_x86_64.whl

This comment has been minimized.

Sign in to view

Copy link
@miladm

miladm Aug 17, 2023

Collaborator

looks like the target hardware is TPU v4. shouldn't we use 3.8 instead of 3.10?

This comment has been minimized.

Sign in to view

Copy link
@ManfeiBai

ManfeiBai Aug 17, 2023

Author Collaborator

sure, will update to py3.8
pip3 uninstall -y libtpu-nightly
pip3 install torch_xla[tpuvm] --user
# show current path
pwd
ls
Expand Down Expand Up @@ -215,7 +225,7 @@ local utils = import 'templates/utils.libsonnet';
},

configs: [
llama2_google_next_inference_pretrained_models + v4_8 + common.Functional + timeouts.Hours(3) + llama2_google_next_inference + xla,
llama2_google_next_inference_pretrained_models + v4_8 + common.Functional + timeouts.Hours(3) + llama2_google_next_inference,
// llama2_google_next_inference_fine_tuned_chat_models + v4_8 + common.Functional + timeouts.Hours(3) + llama2_google_next_inference + xla,
llama2_stable_tokenizer + v4_8 + common.Functional + timeouts.Hours(3) + stable + xla,
llama2_stable_quant + v4_8 + common.Functional + timeouts.Hours(3) + stable + xla,
Expand Down

2 comments on commit 315f2e7

@miladm
Copy link
Collaborator

@miladm miladm commented on 315f2e7 Aug 17, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ManfeiBai here are the setup steps I've seen run well on TPU v4 hardware. You will need to update them according to xl ml stack of course.

gcloud compute tpus tpu-vm ssh ${TPU_NAME} --zone ${ZONE} --project ${PROJECT_ID} --worker=all --command="
sudo pip3 uninstall torch torch_xla libtpu-nightly torchvision -y
pip3 uninstall torch torch_xla libtpu-nightly torchvision -y
pip3 install https://storage.googleapis.com/pytorch-xla-releases/wheels/tpuvm/torch-nightly-cp38-cp38-linux_x86_64.whl
pip3 install https://storage.googleapis.com/pytorch-xla-releases/wheels/tpuvm/torch_xla-nightly-cp38-cp38-linux_x86_64.whl
pip3 install torch-xla[tpuvm]
sudo git clone --branch llama2-google-next-inference https://github.com/pytorch-tpu/llama.git

gcloud compute tpus tpu-vm ssh ${TPU_NAME} --zone ${ZONE} --project ${PROJECT_ID} --worker=all --command="
sudo apt update
sudo apt-get install libopenblas-dev"

gcloud compute tpus tpu-vm scp params_7b.json ${TPU_NAME}:params.json --zone ${ZONE} --project ${PROJECT_ID} --worker=all
# TODO: run a similar command to add `t5_tokenizer/spiece.model` as tokenizer under `llama` director

gcloud compute tpus tpu-vm ssh ${TPU_NAME} --zone ${ZONE} --project ${PROJECT_ID} --worker=all --command="
sudo chmod -R 777 llama
cd llama/
pip3 install -r requirements.txt
pip3 install -e ."

gcloud compute tpus tpu-vm ssh ${TPU_NAME} --zone ${ZONE} --project ${PROJECT_ID} --worker=all --command="cd /home/miladmo/llama && 
PJRT_DEVICE=TPU python3.8 example_text_completion.py --ckpt_dir . --tokenizer_path /home/miladmo/llama1/t5_tokenizer/spiece.model --max_seq_len 2048 --max_gen_len 1000 --max_batch_size 1 --mp True --dynamo True"

side question: should we switch this change to a PR?

@ManfeiBai
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ManfeiBai here are the setup steps I've seen run well on TPU v4 hardware. You will need to update them according to xl ml stack of course.

gcloud compute tpus tpu-vm ssh ${TPU_NAME} --zone ${ZONE} --project ${PROJECT_ID} --worker=all --command="
sudo pip3 uninstall torch torch_xla libtpu-nightly torchvision -y
pip3 uninstall torch torch_xla libtpu-nightly torchvision -y
pip3 install https://storage.googleapis.com/pytorch-xla-releases/wheels/tpuvm/torch-nightly-cp38-cp38-linux_x86_64.whl
pip3 install https://storage.googleapis.com/pytorch-xla-releases/wheels/tpuvm/torch_xla-nightly-cp38-cp38-linux_x86_64.whl
pip3 install torch-xla[tpuvm]
sudo git clone --branch llama2-google-next-inference https://github.com/pytorch-tpu/llama.git

gcloud compute tpus tpu-vm ssh ${TPU_NAME} --zone ${ZONE} --project ${PROJECT_ID} --worker=all --command="
sudo apt update
sudo apt-get install libopenblas-dev"

gcloud compute tpus tpu-vm scp params_7b.json ${TPU_NAME}:params.json --zone ${ZONE} --project ${PROJECT_ID} --worker=all
# TODO: run a similar command to add `t5_tokenizer/spiece.model` as tokenizer under `llama` director

gcloud compute tpus tpu-vm ssh ${TPU_NAME} --zone ${ZONE} --project ${PROJECT_ID} --worker=all --command="
sudo chmod -R 777 llama
cd llama/
pip3 install -r requirements.txt
pip3 install -e ."

gcloud compute tpus tpu-vm ssh ${TPU_NAME} --zone ${ZONE} --project ${PROJECT_ID} --worker=all --command="cd /home/miladmo/llama && 
PJRT_DEVICE=TPU python3.8 example_text_completion.py --ckpt_dir . --tokenizer_path /home/miladmo/llama1/t5_tokenizer/spiece.model --max_seq_len 2048 --max_gen_len 1000 --max_batch_size 1 --mp True --dynamo True"

side question: should we switch this change to a PR?

thanks, will update PR according to this, and PR is : #965

Please sign in to comment.