Run notebook end-to-end with GTX 3090/4090

[artste]

The main purpose of this PR is to enable users with RTX 3090/4090 cards to run the lm-hackers.ipynb notebook end-to-end. To achieve this, we need to free both the host and GPU memory before loading a new model. The free_memory function below was borrowed from miniai:

def free_memory(verbose=False):
    gc.collect() 
    torch.cuda.empty_cache()
    if verbose: print(f'ℹ️ CUDA MEMORY RESERVED AFTER free_memory: {get_cuda_memory_reserved_gb()} GB')    

Before loading a new model, the current model is deleted, and the free_memory function is called:

del model; free_memory();
mn = "stabilityai/StableBeluga-7B"
model = AutoModelForCausalLM.from_pretrained(mn, device_map=0, torch_dtype=torch.bfloat16)

NOTE: It's essential to use del model before free_memory to ensure that the memory is released before reassignment. Failing to do so might result in a working memory exceeding 24GB.

[review-notebook-app]

Check out this pull request on 

See visual diffs & provide feedback on Jupyter Notebooks.

---

Powered by ReviewNB