Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
25 changes: 7 additions & 18 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,13 @@ The KABR tools used in this process can be installed with:
```
pip install git+https://github.com/Imageomics/kabr-tools
```

**Notes:**
- detectron2 requires Linux or MacOS.
- If building detectron2's wheel fails, loading a different gnu module may help (`module load gnu/11.2.0`).
- If `ModuleNotFoundError: No module named 'torch'` appears, try `pip install torch torchvision` in your environment and try installing `kabr_tools` again.
- SlowFast's setup.py is outdated; our workaround is `pip install git+https://github.com/Imageomics/SlowFast@797a6f3ae81c49019d006296f1e0f84f431dc356`, which is included when installing `kabr_tools`.

Each KABR tool can be run through the command line (as described below) or imported as a python module. They each have help information which can be accessed on the command line through `<tool-name> -h`.

Please refer to our [KABR Project Page](https://kabrdata.xyz/) for additional details.
Expand Down Expand Up @@ -88,26 +95,8 @@ You can use the [KABR model](https://huggingface.co/imageomics/x3d-kabr-kinetics

To use the [KABR model](https://huggingface.co/imageomics/x3d-kabr-kinetics), download `checkpoint_epoch_00075.pyth.zip`, unzip `checkpoint_epoch_00075.pyth`, and install [SlowFast](https://github.com/facebookresearch/SlowFast). Then run [miniscene2behavior.py](miniscene2behavior.py).

Installing detectron2 and pytorchvideo can be tricky. This should work:
```
python -m pip install git+https://github.com/facebookresearch/detectron2.git@2a420edb307c9bdf640f036d3b196bed474b8593
python -m pip install git+https://github.com/facebookresearch/pytorchvideo.git@1fadaef40dd393ca09680f55582399f4679fc9b7
```

[SlowFast](https://github.com/facebookresearch/SlowFast)'s `setup.py` is outdated, so a workaround is:

```
python -m pip install git+https://github.com/zhong-al/SlowFast@797a6f3ae81c49019d006296f1e0f84f431dc356
```

After [SlowFast](https://github.com/facebookresearch/SlowFast) is installed, you are ready to label the mini-scenes:

```
miniscene2behavior [--config path_to_config] --checkpoint path_to_checkpoint [--gpu_num number_of_gpus] --miniscene path_to_miniscene [--output path_to_output_csv]
```

**Notes:**
- If building detectron2's wheel fails, loading a different gnu module may help (`module load gnu/11.2.0`)
- If the config hasn't been extracted yet, the script will write it to `config`.
- `checkpoint` should be the path to `checkpoint_epoch_00075.pyth`.
- If `gpu_num` is 0, the model will use CPU. Using at least 1 GPU greatly increases inference speed. If you're using OSC, you can request a node with one GPU by running `sbatch -N 1 --gpus-per-node 1 -A [account] --time=[minutes] [bash script]`.
Expand Down
2 changes: 1 addition & 1 deletion pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -37,7 +37,7 @@ dependencies = [
"pandas",
"detectron2 @ git+https://github.com/facebookresearch/detectron2.git@2a420edb307c9bdf640f036d3b196bed474b8593",
"pytorchvideo @ git+https://github.com/facebookresearch/pytorchvideo.git@1fadaef40dd393ca09680f55582399f4679fc9b7",
"slowfast @ git+https://github.com/zhong-al/SlowFast@797a6f3ae81c49019d006296f1e0f84f431dc356"
"slowfast @ git+https://github.com/Imageomics/SlowFast@797a6f3ae81c49019d006296f1e0f84f431dc356"
]
keywords = [
"annotation",
Expand Down