Skip to content

Release 2.0.0
Compare
Choose a tag to compare
@Sid-Data-Universe Sid-Data-Universe released this 05 Sep 23:18
· 13 commits to main since this release
v2.0.0
a97e3d6
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Learn about vigilant mode.

Adds a new B7_MULTI_CHOICE competition.

  • Weights are annealed slowly from a 100/0 split to a 50/50 split at a rate of 5% per 3600 blocks.
  • Multiple choice question data is loaded from the PromptingSubnetLoader which pulls from SN1 wandb logs.
  • Evaluation asks the models the multiple choice questions and takes the first possible answer returned.

Across all competitions we only keep the 2 top models from eval to eval to speed up the evaluation loop.
Across all competitions we will start using an epsilon with a linear decay instead of a fixed epsilon value.

Note: This includes a bump to the validator state version and therefore local state will automatically be wiped on update.
Note: This includes some requirement bumps, so validators will also need to run python -m pip install -e . to update.

Assets 2
Loading