Skip to content

Commit

Permalink
change to enable MoE
Browse files Browse the repository at this point in the history
  • Loading branch information
SangbumChoi committed Sep 6, 2024
1 parent 50294f6 commit 5911010
Show file tree
Hide file tree
Showing 2 changed files with 15 additions and 3 deletions.
16 changes: 14 additions & 2 deletions docs/source/en/model_doc/vitpose.md
Original file line number Diff line number Diff line change
Expand Up @@ -26,8 +26,20 @@ The original code can be found [here](https://github.com/ViTAE-Transformer/ViTPo

## Usage Tips

- To enable MoE (Mixture of Experts) function in the backbone, the user has to give appropriate input indices to the backbone model.
However, it is not used in default parameters.
- To enable MoE (Mixture of Experts) function in the backbone, user has to give appropriate configuration such as `num_experts` and input value `dataset_index` to the backbone model.
However, it is not used in default parameters. Below is the code snippet for usage of MoE function.
```py
>>> from transformers import VitPoseBackboneConfig, VitPoseBackbone
>>> import torch

>>> config = VitPoseBackboneConfig(num_experts=3, out_indices=[-1])
>>> model = VitPoseBackbone(config)

>>> pixel_values = torch.randn(3, 3, 256, 192)
>>> dataset_index = torch.tensor([1, 2, 3])
>>> outputs = model(pixel_values, dataset_index)
```

- The current model utilizes a 2-step inference pipeline. The first step involves placing a bounding box around the region corresponding to the person.
After that, the second step uses VitPose to predict the keypoints.

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -102,7 +102,7 @@ def __init__(
num_attention_heads=12,
mlp_ratio=4,
num_experts=1,
part_features=None,
part_features=256,
hidden_act="gelu",
hidden_dropout_prob=0.0,
attention_probs_dropout_prob=0.0,
Expand Down

0 comments on commit 5911010

Please sign in to comment.