Issues with changing keypoint number and pretained model for swin_base_reid.pth

[MahejabeenNidhi]

Hello,

Thank you for your fantastic work. I am working on training my custom dataset using your codebase. My objects are cattle, so I don't need parsing. I am only planning on working on the key points. I only have four keypoints, but I can't seem to change this in default_config.py because prompt_parts_num and prompt_parts_names are only in the masks section and not in the keypoints section. I would appreciate your guidance on being able to make this change.

Below is my config file, kpr_cattle_train.yaml

# kpr_cattle_train.yaml
# Config to train KPR on your custom cattle dataset with exactly 4 keypoints
# ("head", "shoulder", "back", "tail") and NO masks.

#######################################################################
# Data
#######################################################################
data:
  root: "cattle_reid_dataset_KPR"

  sources: ['cattle_reid_dataset_KPR']
  targets: ['cattle_reid_dataset_KPR']

  height: 384
  width: 128
  norm_mean: [0.5, 0.5, 0.5]
  norm_std: [0.5, 0.5, 0.5]


  transforms: ['rc', 're']
  #   'rc' = random crop
  #   're' = random erase

#######################################################################
# Model
#######################################################################
model:
  # If you have a pretrained checkpoint for KPR, specify here:
  load_weights: ""
  load_config: false
  name: "kpr"

  # KPR-specific settings
  kpr:
    backbone: "solider_swin_base_patch4_window7_224"
    # Keypoints block
    keypoints:
      enabled: True
      kp_dir: "pifpaf_keypoints"
      # "prompt_masks" can remain "keypoints_gaussian" if you want heatmap-based prompts
      prompt_masks: keypoints_gaussian

      prompt_preprocess: none

      use_negative_keypoints: False
      filter_out_no_skeletons: True
      vis_thresh: 0.0

    # Mask block DISABLED
    masks:
      enabled: False
      dir: ""
      preprocess: "none"

      prompt_parts_names: ["head", "shoulder", "back", "tail"]
      prompt_parts_num: 4

  # Optional: TransReID part
  transreid:
    cam_num: 4

#######################################################################
# Train
#######################################################################
train:
  optim: "adam"
  lr: 0.0002
  fixbase_epoch: 10

I tried to train it but I couldn't find the swin_base_reid.pth file in the Google Drive that you shared in the repository. Below is my error output

keypoint_promptable_reidentification/torchreid/metrics/rank.py:13: UserWarning: Cython evaluation (very fast so highly recommended) is unavailable, now use python evaluation.
  warnings.warn(
Diff from default config :
{'cam_num': 4,
 'enabled': False,
 'filter_out_no_skeletons': True,
 'fixbase_epoch': 10,
 'height': 384,
 'kp_dir': 'pifpaf_keypoints',
 'load_config': False,
 'lr': 0.0002,
 'norm_mean': '[0.5, 0.5, 0.5]',
 'norm_std': '[0.5, 0.5, 0.5]',
 'optim': 'adam',
 'preprocess': 'none',
 'prompt_parts_names': "['nose', 'head_bottom', 'head_top', 'left_ear', "
                       "'right_ear', 'left_shoulder', 'right_shoulder', "
                       "'left_elbow', 'right_elbow', 'left_wrist', "
                       "'right_wrist', 'left_hip', 'right_hip', 'left_knee', "
                       "'right_knee', 'left_ankle', 'right_ankle']",
 'prompt_parts_num': 17,
 'prompt_preprocess': 'none',
 'transforms': "['rc', 're']",
 'use_negative_keypoints': False,
 'vis_thresh': 0.0}
Save dir created at keypoint_promptable_reidentification/logs/195357163
Show configuration
adam:
  beta1: 0.9
  beta2: 0.999
cuhk03:
  classic_split: False
  labeled_images: False
  use_metric_cuhk03: False
data:
  bipo:
    masks_dir: bpbreid_masks
    max_overlap: 0.8
    min_overlap: 0.5
    n: 1
    p: 0.2
    path: 
    pid_sampling_from_batch: True
  cj:
    always_apply: False
    brightness: 0.2
    contrast: 0.15
    hue: 0.0
    p: 0.5
    saturation: 0.0
  combineall: False
  dak:
    p: 0.3
  drk:
    p: 0.2
    ratio: 0.5
  height: 384
  load_train_targets: False
  mot:
    fig_size: (128, 64)
    mask_size: (32, 16)
    test:
      max_samples_per_id: 40
      max_total_ids: -1
      min_h: 50
      min_samples_per_id: 4
      min_vis: 0.3
      min_w: 25
      ratio_query_per_id: 0.2
    train:
      max_samples_per_id: 40
      max_total_ids: -1
      min_h: 50
      min_samples_per_id: 4
      min_vis: 0.3
      min_w: 25
  norm_mean: [0.5, 0.5, 0.5]
  norm_std: [0.5, 0.5, 0.5]
  query_list: []
  resize:
    interpolation: 1
    mask_interpolation: bilinear
  root: cattle_reid_dataset_KPR
  save_dir: logs/195357163
  sources: ['cattle_reid_dataset_KPR']
  split_id: 0
  targets: ['cattle_reid_dataset_KPR']
  transforms: ['rc', 're']
  type: image
  width: 128
  workers: 4
inference:
  enabled: False
  input_folder: 
  output_figure_folder: 
  output_folder: 
loss:
  name: part_based
  part_based:
    best_pred_ratio: 1.0
    name: part_averaged_triplet_loss
    ppl: cl
    weights:
      conct:
        id: 1.0
        tr: 0.0
      foreg:
        id: 1.0
        tr: 0.0
      globl:
        id: 1.0
        tr: 0.0
      parts:
        id: 0.0
        tr: 1.0
      pixls:
        ce: 0.35
  softmax:
    label_smooth: True
  triplet:
    margin: 0.3
    weight_t: 1.0
    weight_x: 0.0
market1501:
  use_500k_distractors: False
model:
  backbone_pretrained_path: pretrained_models/
  compute_complexity: False
  discard_test_params: False
  kpr:
    backbone: solider_swin_base_patch4_window7_224
    dim_reduce: after_pooling
    dim_reduce_output: 512
    enable_fpn: False
    enable_msf: True
    fpn_out_dim: 1024
    keypoints:
      enabled: True
      filter_out_no_skeletons: True
      kp_dir: pifpaf_keypoints
      prompt_masks: keypoints_gaussian
      prompt_preprocess: none
      target_masks: none
      target_preprocess: none
      use_negative_keypoints: False
      vis_continous: False
      vis_thresh: 0.0
    last_stride: 1
    learnable_attention_enabled: True
    mask_filtering_testing: True
    mask_filtering_training: False
    masks:
      background_computation_strategy: threshold
      dir: 
      enabled: False
      mask_filtering_threshold: 0.5
      parts_names: ['1']
      parts_num: 1
      preprocess: none
      prompt_parts_names: ['nose', 'head_bottom', 'head_top', 'left_ear', 'right_ear', 'left_shoulder', 'right_shoulder', 'left_elbow', 'right_elbow', 'left_wrist', 'right_wrist', 'left_hip', 'right_hip', 'left_knee', 'right_knee', 'left_ankle', 'right_ankle']
      prompt_parts_num: 17
      softmax_weight: 15
      type: disk
    msf_spatial_scale: -1
    normalization: identity
    pooling: gwap
    shared_parts_id_classifier: False
    spatial_feature_shape: []
    test_embeddings: ['bn_foreg', 'parts']
    test_use_target_segmentation: none
    testing_binary_visibility_score: True
    training_binary_visibility_score: True
    use_prompt_visibility_score: False
  load_config: False
  load_weights: 
  name: kpr
  pretrained: True
  promptable_trans:
    att_drop_rate: 0.0
    disable_inference_prompting: False
    drop_out: 0.0
    drop_path: 0.1
    drop_rate: 0.0
    masks_prompting: True
    no_background_token: False
    patch_size: 16
    pose_encoding_all_layers: False
    pose_encoding_strategy: embed_heatmaps_patches
    stride_size: [16, 16]
    transformer_type: vit_base_patch16_224_TransReID
    use_abs_pos_embed: False
  resume: 
  save_model_flag: False
  solider:
    mask_path_emb_init_zeros: True
    semantic_weight: 0.2
    test_weight: 
  transreid:
    cam_num: 4
    devide_length: 4
    jpm: True
    re_arrange: True
    shift_num: 5
    shuffle_group: 2
    sie_camera: True
    sie_coe: 3
    sie_view: False
occluded_posetrack:
  enable_dataset_sampling_loading: True
  enable_sam: True
  occluded_dataset: True
  sam_checkpoint: ~/pretrained_models/sam/sam_vit_h_4b8939.pth
project:
  config_file: kpr_cattle_train.yaml
  debug_mode: False
  diff_config: {'load_config': False, 'vis_thresh': 0.0, 'prompt_preprocess': 'none', 'use_negative_keypoints': False, 'kp_dir': 'pifpaf_ke...
  experiment_id: e0a62757-9060-4af7-8161-9ebc837ab911
  experiment_name: 
  job_id: 195357163
  logger:
    matplotlib_show: False
    save_disk: True
    use_clearml: False
    use_tensorboard: False
    use_wandb: False
  name: KPR
  notes: 
  start_time: 2024_12_22_14_08_01_08S
  tags: []
rmsprop:
  alpha: 0.99
sampler:
  num_instances: 4
  train_sampler: RandomIdentitySampler
  train_sampler_t: RandomIdentitySampler
sgd:
  dampening: 0.0
  momentum: 0.9
  nesterov: False
test:
  batch_size: 128
  batch_size_pairwise_dist_matrix: 500
  detailed_ranking: False
  dist_metric: euclidean
  evaluate: False
  normalize_feature: True
  part_based:
    dist_combine_strat: mean
  ranks: [1, 3, 5, 10, 20]
  rerank: False
  save_features: False
  start_eval: 0
  vis_embedding_projection: False
  vis_feature_maps: False
  visrank: True
  visrank_count: 10
  visrank_display_mode: display_worst_rand
  visrank_per_body_part: False
  visrank_q_idx_list: []
  visrank_topk: 10
train:
  base_lr_mult: 2.0
  batch_debug_freq: 0
  batch_log_freq: 0
  batch_size: 64
  eval_freq: -1
  fixbase_epoch: 10
  gamma: 0.1
  lr: 0.0002
  lr_scheduler: cosine_annealing_warmup
  max_epoch: 120
  mixed_precision: True
  new_layers: []
  open_layers: []
  optim: adam
  reduced_lr: 0.0002
  seed: 1
  staged_lr: False
  start_epoch: 0
  stepsize: [40, 70]
  transreid_lr: True
  warmup_t: 5
  weight_decay: 0.0001
  weight_decay_bias: 0.0001
use_gpu: True
video:
  pooling_method: avg
  sample_method: evenly
  seq_len: 15

Collecting env info ...
Building train transforms ...
+ resize to 384x128
+ random crop
+ normalization (mean=[0.5, 0.5, 0.5], std=[0.5, 0.5, 0.5])
+ random erase
+ to torch tensor of range [0, 1]
Building test transforms ...
+ resize to 384x128
+ to torch tensor of range [0, 1]
+ normalization (mean=[0.5, 0.5, 0.5], std=[0.5, 0.5, 0.5])
+ use add background mask
=> Loading train (source) dataset
Creating new dataset cattle_reid_dataset_KPR and add it to the datasets cache.
Filtered out 2/655 samples without keypoints
=> Loaded CattleKPR
  ----------------------------------------
  subset   | # ids | # images | # cameras
  ----------------------------------------
  train    |     9 |      653 |         4
  query    |     9 |       49 |         4
  gallery  |     9 |      115 |         4
  ----------------------------------------
=> Loading test (target) dataset
Using cached dataset cattle_reid_dataset_KPR.
Using cached dataset cattle_reid_dataset_KPR.


  **************** Summary ****************
  source            : ['cattle_reid_dataset_KPR']
  # source datasets : 1
  # source ids      : 9
  # source images   : 653
  # source cameras  : 4
  target            : ['cattle_reid_dataset_KPR']
  *****************************************


Building model: kpr
/keypoint_promptable_reidentification/torchreid/models/solider/backbones/swin_transformer.py:1162: UserWarning: DeprecationWarning: pretrained is deprecated, please use "init_cfg" instead
  warnings.warn('DeprecationWarning: pretrained is deprecated, '
Traceback (most recent call last):
  File "/keypoint_promptable_reidentification/main.py", line 65, in <module>
    main()
  File "/keypoint_promptable_reidentification/main.py", line 47, in main
    engine, model = build_torchreid_model_engine(cfg)
  File "/keypoint_promptable_reidentification/torchreid/scripts/builder.py", line 234, in build_torchreid_model_engine
    model = build_model(cfg, datamanager.num_train_pids, datamanager.train_loader.dataset.cam_num, datamanager.train_loader.dataset.view)
  File "/keypoint_promptable_reidentification/torchreid/scripts/builder.py", line 252, in build_model
    model = torchreid.models.build_model(
  File "/keypoint_promptable_reidentification/torchreid/models/__init__.py", line 145, in build_model
    return __model_factory[name](
  File "/keypoint_promptable_reidentification/torchreid/models/kpr.py", line 49, in __init__
    self.backbone_appearance_feature_extractor = models.build_model(self.model_cfg.backbone,
  File "/keypoint_promptable_reidentification/torchreid/models/__init__.py", line 145, in build_model
    return __model_factory[name](
  File "/keypoint_promptable_reidentification/torchreid/models/promptable_solider.py", line 163, in solider_swin
    model = PromptableSoliderSwinTransformer(
  File "/keypoint_promptable_reidentification/torchreid/models/promptable_solider.py", line 52, in __init__
    model.init_weights(pretrained_path)
  File "/keypoint_promptable_reidentification/torchreid/models/solider/backbones/swin_transformer.py", line 1306, in init_weights
    ckpt = torch.load(pretrained,map_location='cpu')
  File "/anaconda3/envs/kpr/lib/python3.10/site-packages/torch/serialization.py", line 771, in load
    with _open_file_like(f, 'rb') as opened_file:
  File "/anaconda3/envs/kpr/lib/python3.10/site-packages/torch/serialization.py", line 270, in _open_file_like
    return _open_file(name_or_buffer, mode)
  File "/anaconda3/envs/kpr/lib/python3.10/site-packages/torch/serialization.py", line 251, in __init__
    super(_open_file, self).__init__(open(name, mode))
FileNotFoundError: [Errno 2] No such file or directory: 'pretrained_models/SOLIDER/swin_base_reid.pth'

Appreciate your feedback! Thanks!