Uses slerp to extract a single expert from multiple experts in a MoE model. This process requires a significant amount of system memory and can take some time. Be patient and make sure you have about five times the system RAM as the model's parameter count from which you're extracting. -Lucas Atkins (Crystalcareai)
python extract.py --model-name mistralai/Mixtral-8x7B-v0.1 --output-dir ./out
Only works for models using Mixtral architecture. There is no guarantee that we'll expand this to support other architectures in the future.