Skip to content

Add fine-tuned SuperAnimal-Quadruped support and improve demo setup#30

Open
xiu-cs wants to merge 27 commits into
mainfrom
ti_dev
Open

Add fine-tuned SuperAnimal-Quadruped support and improve demo setup#30
xiu-cs wants to merge 27 commits into
mainfrom
ti_dev

Conversation

@xiu-cs
Copy link
Copy Markdown
Collaborator

@xiu-cs xiu-cs commented May 17, 2026

Summary

This PR adds first-class support for the fine-tuned SuperAnimal-Quadruped 2D pose model used by the animal pipeline, enabling direct 26-joint Animal3D keypoint prediction and automatic checkpoint download from Hugging Face. It also improves the out-of-the-box demo/install path, fixes CPU fallback for the human HRNet demo, and removes unused legacy code/assets.

Changes

  • Add support for a fine-tuned SuperAnimal-Quadruped 2D checkpoint that predicts the 26-joint Animal3D layout directly.
  • Auto-download animal demo checkpoints from Hugging Face on first run:
    • sa_finetune_hrnet_w32.pt for 2D animal pose
    • fmpose3d_animals.pth for the 3D lifter
  • Refactor animals/demo/vis_animals.py to build the 2D estimator and 3D lifter once, then reuse them across images.
  • Add SuperAnimalConfig options for fine-tuned checkpoints, detector overrides, and lazy Hugging Face resolution.
  • Update animal defaults and docs from older Rat7M/legacy assumptions toward Animal3D.
  • Fix human HRNet loading on CPU-only environments by using device-aware map_location and moving inputs to the model device.
  • Pin install dependencies to torch>=2.4.1,<2.5 and torchvision>=0.19.1,<0.20, and document the PyTorch/CUDA behavior in the README.
  • Restrict package Python metadata to >=3.10,<3.13; README recommends Python 3.10 because install/demo paths were tested there.
  • Remove unused legacy animal modules and unused YOLO/HRNet assets.
  • Add/update tests for the fine-tuned SuperAnimal path and config behavior.
  • Add mot to the codespell ignore list.

Validation

Ran install, test, and demo checks locally:

python3 -m pip install -e '.[animals,viz]' --dry-run
python3 -m pytest tests/test_demo_human.py tests/fmpose3d_api/test_fmpose3d.py -q
python3 -m pytest tests/test_model.py tests/test_training_pipeline.py -q
bash demo/vis_in_the_wild.sh
bash animals/demo/vis_animals.sh

Results

  • Human demo passes on both CPU-only and GPU paths.
  • Animal demo passes and auto-resolves both Hugging Face checkpoints.
  • Relevant tests pass: 78 passed for human demo/API tests, 8 passed for model/training smoke tests.

xiu-cs added 27 commits May 15, 2026 21:33
- Deleted `graph_utils.py`, which contained functions for adjacency matrix creation and normalization.
- Removed `lifter3d.py`, which included keypoint processing, 3D triangulation, and visualization functions.
- Eliminated `mocap_dataset.py`, which defined the `MocapDataset` class for handling motion capture data.
… and reuse across images, improving efficiency and clarity.
@xiu-cs xiu-cs changed the title Ti dev Add fine-tuned SuperAnimal-Quadruped support and improve demo setup May 17, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant