The inference script is not generating a complete caption.

[reem-alseed]

Hi, thank you for sharing this great work.

I’m trying to reproduce the paper result on the 5k Karpathy test split test set using the inference script, but I’m getting a lower scores:

Bleu_1: 0.810
Bleu_2: 0.655
Bleu_3: 0.510
Bleu_4: 0.388
METEOR: 0.295
ROUGE_L: 0.587
CIDEr: 1.333
SPICE: 0.230

And after some digging, the caption is not fully generated,
I managed to duplicate the problem in Colab as well.

https://colab.research.google.com/drive/1BvtscubSujlxOFhOchVGNB79KkKYoMiH?usp=sharing

[davidnvq]

Thank you for reporting the issue. Have you figured it out?
I will inspect the problem and inform you later.

[reem-alseed]

Unfortunately, not yet, I have checked the config file and everything looks ok to me (beam search, max_length, ..etc). I have tested the code with only one GPU (A100) and (Colab GPU), and I'm not sure if this will affect the inference.

[davidnvq]

Thanks for further information. It doesn't affect the inference as I also used 1 GPU. I haven't figured out the issue. Let me check this in details.

[fyhu2023]

Same issue here. There is something to do with not using model.eval() in the inference code. So the captions generated are not deterministic. Every time I run the inference_caption.py, it gives me different captions.
However if I run the example caption generation code with model.eval(), it give me an incomplete caption "three sheep standing next to a".

[HudaDiab]

Hello,
Could anyone please help. Im using Colab I need to used part of the code (Swin transformer as backbone ) to extracted features!. I tried to run "extract_features.py" but Im facing the following errors, any help is apricated .

output
2024-01-09 23:36:18.083731: E external/local_xla/xla/stream_executor/cuda/cuda_dnn.cc:9261] Unable to register cuDNN factory: Attempting to register factory for plugin cuDNN when one has already been registered
2024-01-09 23:36:18.083802: E external/local_xla/xla/stream_executor/cuda/cuda_fft.cc:607] Unable to register cuFFT factory: Attempting to register factory for plugin cuFFT when one has already been registered
2024-01-09 23:36:18.085352: E external/local_xla/xla/stream_executor/cuda/cuda_blas.cc:1515] Unable to register cuBLAS factory: Attempting to register factory for plugin cuBLAS when one has already been registered
2024-01-09 23:36:19.463628: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT