Pretrained CRIS model #3

OptimusLime · 2022-06-22T19:28:55Z

Are there any plans to host a pre-trained CRIS model on e.g. huggingface?

Looking forward to trying out the model!

willyfh · 2022-06-23T05:25:13Z

+1

DerrickWang005 · 2022-06-23T05:28:32Z

Yes. We will release the pretrained models ASAP.

OptimusLime · 2022-07-29T19:19:11Z

@DerrickWang005 Any updates on this or timeline?

Bo396543018 · 2022-09-10T12:23:31Z

+1

saurabhya · 2022-09-19T08:50:18Z

Hii @DerrickWang005, Can you upload the pre-trained model?

sanchit88 · 2022-10-04T12:32:30Z

Hi @DerrickWang005 any update on releasing a pre-trained model? Thanks!

qiruiw · 2022-10-12T22:51:21Z

+1

loris2222 · 2022-10-13T07:14:50Z

+1

MaslikovEgor · 2022-10-25T07:20:38Z

+1

loris2222 · 2022-10-25T07:45:34Z

While we wait for the official pretrained model, I have trained CRIS on refcoco using the standard training procedure (default parameters, 50 epochs). The model is available here.
I should be able to keep it there for a couple of years, I think it will be enough for the authors to publish their model 😄

MaslikovEgor · 2022-10-25T10:35:07Z

@loris2222 do you have similar score on benchmark datasets?

loris2222 · 2022-10-25T10:57:33Z

I am using the model for another purpose so I haven't saved the test performance, but you are welcome to try and report the results here. Anyways, qualitatively results are quite good.

sanchit88 · 2022-10-25T12:30:48Z

Wow @loris2222 thank you so much. By the way, do you have any report or documentation regarding the results (qualitatively and quantitatively). It would be really nice and helpful to see the results and read more about it. May be you can upload them in your Git repo? Great work!

MaslikovEgor · 2022-10-25T15:08:30Z

@loris2222 @sanchit88 I have some bad news(

test your model on evaluation RefCOCO data and I have this(

IoU=8.03 Pr@50: 0.17 Pr@60: 0.17 Pr@70: 0.17 Pr@80: 0.09 Pr@90: 0.09

MaslikovEgor · 2022-10-25T15:13:13Z

@DerrickWang005 Should you please share pre-trained CRIS model with benchmark score(

loris2222 · 2022-10-25T16:03:51Z

I have found the wandb final log for my training process:

{ "time/batch": 3.7636070251464844, "time/data": 0.017482995986938477, "training/lr": 0.00001, "training/loss": 0.014677105471491814, "training/iou": 89.92655944824219, "training/prec@50": 100, "_timestamp": 1665918932.9539435, "_runtime": 74153.98540353775, "_step": 33700, "_wandb": { "runtime": 74615 } }

Qualitative results on VOC

Maybe there is something wrong in your configuration?

MaslikovEgor · 2022-10-25T18:16:31Z

@loris2222 hmmm, maybe is just overfitting on train data? You have IOU higher than every SOTA is possible

loris2222 · 2022-10-25T18:24:37Z

Yeah possibly, but results on training data were not published, so they still might be reasonable. By the way VOC is an entirely different dataset and it seems to be doing well as you see from the example (not cherry picked). As I said I'm using these models for something else so I don't think I will be able to run another 20h of training...

BingliangLi · 2022-10-26T13:05:46Z

Yes. We will release the pretrained models ASAP.

Hi just want to ask is there any update? I'm planning to do further research with your model, could you please upload the checkpoints?

BingliangLi · 2022-10-27T10:24:13Z

@loris2222 @MaslikovEgor @OptimusLime @willyfh @qiruiw @saurabhya

Just want to report my test result on @loris2222 's model(RefCOCO val):

2022-10-27 18:05:35 | INFO     | __main__:76 - => loading checkpoint 'exp/refcoco/CRIS_R50/best_model.pth'
2022-10-27 18:05:36 | INFO     | __main__:79 - => loaded checkpoint 'exp/refcoco/CRIS_R50/best_model.pth'
Inference:: 100%|###############################################| 3811/3811 [06:45<00:00,  9.39it/s]
2022-10-27 18:12:22 | INFO     | engine.engine:198 - => Metric Calculation <=
2022-10-27 18:12:22 | INFO     | engine.engine:211 - IoU=69.06
2022-10-27 18:12:22 | INFO     | engine.engine:213 - Pr@50: 80.93.
2022-10-27 18:12:22 | INFO     | engine.engine:213 - Pr@60: 76.44.
2022-10-27 18:12:22 | INFO     | engine.engine:213 - Pr@70: 69.83.
2022-10-27 18:12:22 | INFO     | engine.engine:213 - Pr@80: 53.51.
2022-10-27 18:12:22 | INFO     | engine.engine:213 - Pr@90: 17.09.

The result reported by the authors is 69.52, I think his mode is fine(69.06), thanks(a lot) for sharing!

loris2222 · 2022-10-27T10:33:43Z

Glad to be of help :)

So @MaslikovEgor did you check again to see whether there were problems in your pipeline?

RubenBMHMendes · 2022-10-27T14:09:30Z

Thanks for the pre-trained model @loris2222. I am still trying to better understand the functionalities and I was wondering if you could detail on how to load the weights and use the model?

loris2222 · 2022-10-27T15:02:58Z

Just put the linked .pth in <CRIS_root_folder>/exp/refcoco/CRIS_50 and follow the instructions from this repo's readme.

RubenBMHMendes · 2022-10-27T16:42:17Z

I did the following:

Clone the repo to colab;
Update <CRIS_root_folder>/config/refcoco/cris_r50.yaml to consider the .pth as clip_pretrain: /content/best_model.pth;
Run evaluation code in readme;

# e.g., Evaluation on the val-set of the RefCOCO dataset
CUDA_VISIBLE_DEVICES=0 python -u test.py \
      --config config/refcoco/cris_r50.yaml \
      --opts TEST.test_split val-test \
             TEST.test_lmdb datasets/lmdb/refcocog_g/val.lmdb

Got the error below. I do not understand the root cause..

2022-10-27 16:34:07.855 | ERROR | main::90 - An error has been caught in function '', process 'MainProcess' (222), thread 'MainThread' (139848918288256):
Traceback (most recent call last):

File "/content/CRIS.pytorch/test.py", line 90, in
main()
└ <function main at 0x7f308a052b00>

File "/content/CRIS.pytorch/test.py", line 70, in main
model, _ = build_segmenter(args)
│ └ CfgNode({'dataset': 'refcoco', 'train_lmdb': 'datasets/lmdb/refcoco/train.lmdb', 'train_split': 'train', 'val_lmdb': 'dataset...
└ <function build_segmenter at 0x7f309690e290>

File "/content/CRIS.pytorch/model/init.py", line 33, in build_segmenter
model = CRIS(args)
│ └ CfgNode({'dataset': 'refcoco', 'train_lmdb': 'datasets/lmdb/refcoco/train.lmdb', 'train_split': 'train', 'val_lmdb': 'dataset...
└ <class 'model.segmenter.CRIS'>

File "/content/CRIS.pytorch/model/segmenter.py", line 15, in init
map_location="cpu").eval()

File "/usr/local/lib/python3.7/dist-packages/torch/jit/_serialization.py", line 162, in load
cpp_module = torch._C.import_ir_module(cu, str(f), map_location, _extra_files)
│ │ │ │ │ │ └ {}
│ │ │ │ │ └ device(type='cpu')
│ │ │ │ └ '/content/best_model.pth'
│ │ │ └ <torch.jit.CompilationUnit object at 0x7f308a068b30>
│ │ └ <built-in method import_ir_module of PyCapsule object at 0x7f3098a21a80>
│ └ <module 'torch._C' from '/usr/local/lib/python3.7/dist-packages/torch/_C.cpython-37m-x86_64-linux-gnu.so'>
└ <module 'torch' from '/usr/local/lib/python3.7/dist-packages/torch/init.py'>

RuntimeError: PytorchStreamReader failed reading zip archive: failed finding central directory

BingliangLi · 2022-10-28T10:25:15Z

@RubenBMHMendes You have to download pre-trained CLIP first(it's not best_model.pth) from here , it's from the official CLIP repo.

Then put best_model.pth at <CRIS_root_folder>/exp/refcoco/CRIS_50 and you are all set.

1539194769 · 2023-04-11T09:21:23Z

@loris2222 how run this model at other datasets（such：VOC，COCO 2017），this code （recoco）only about the train 2014

rabinadk1 · 2023-07-11T04:07:23Z

Yes. We will release the pretrained models ASAP.

@DerrickWang005, Any update on this?

RubenBMHMendes mentioned this issue Nov 3, 2022

How to predict segmentation masks? #11

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Pretrained CRIS model #3

Pretrained CRIS model #3

OptimusLime commented Jun 22, 2022

willyfh commented Jun 23, 2022

DerrickWang005 commented Jun 23, 2022

OptimusLime commented Jul 29, 2022

Bo396543018 commented Sep 10, 2022

saurabhya commented Sep 19, 2022

sanchit88 commented Oct 4, 2022

qiruiw commented Oct 12, 2022

loris2222 commented Oct 13, 2022

MaslikovEgor commented Oct 25, 2022

loris2222 commented Oct 25, 2022

MaslikovEgor commented Oct 25, 2022

loris2222 commented Oct 25, 2022

sanchit88 commented Oct 25, 2022 •

edited

Loading

MaslikovEgor commented Oct 25, 2022

MaslikovEgor commented Oct 25, 2022

loris2222 commented Oct 25, 2022 •

edited

Loading

MaslikovEgor commented Oct 25, 2022

loris2222 commented Oct 25, 2022

BingliangLi commented Oct 26, 2022 •

edited

Loading

BingliangLi commented Oct 27, 2022

loris2222 commented Oct 27, 2022

RubenBMHMendes commented Oct 27, 2022

loris2222 commented Oct 27, 2022

RubenBMHMendes commented Oct 27, 2022

BingliangLi commented Oct 28, 2022

1539194769 commented Apr 11, 2023

rabinadk1 commented Jul 11, 2023

Pretrained CRIS model #3

Pretrained CRIS model #3

Comments

OptimusLime commented Jun 22, 2022

willyfh commented Jun 23, 2022

DerrickWang005 commented Jun 23, 2022

OptimusLime commented Jul 29, 2022

Bo396543018 commented Sep 10, 2022

saurabhya commented Sep 19, 2022

sanchit88 commented Oct 4, 2022

qiruiw commented Oct 12, 2022

loris2222 commented Oct 13, 2022

MaslikovEgor commented Oct 25, 2022

loris2222 commented Oct 25, 2022

MaslikovEgor commented Oct 25, 2022

loris2222 commented Oct 25, 2022

sanchit88 commented Oct 25, 2022 • edited Loading

MaslikovEgor commented Oct 25, 2022

MaslikovEgor commented Oct 25, 2022

loris2222 commented Oct 25, 2022 • edited Loading

MaslikovEgor commented Oct 25, 2022

loris2222 commented Oct 25, 2022

BingliangLi commented Oct 26, 2022 • edited Loading

BingliangLi commented Oct 27, 2022

loris2222 commented Oct 27, 2022

RubenBMHMendes commented Oct 27, 2022

loris2222 commented Oct 27, 2022

RubenBMHMendes commented Oct 27, 2022

BingliangLi commented Oct 28, 2022

1539194769 commented Apr 11, 2023

rabinadk1 commented Jul 11, 2023

sanchit88 commented Oct 25, 2022 •

edited

Loading

loris2222 commented Oct 25, 2022 •

edited

Loading

BingliangLi commented Oct 26, 2022 •

edited

Loading