Heatmaps different every time run it

[Medicmind]

Something Im finding with heatmaps is that you get a different image every time you run a heatmap on an image. Is there any way round this problem? This is the code Im using. The images are the same images and you can see that the heatmap is different every time.

    mm = efficientnet.EfficientNetV2S(num_classes=2,pretrained=None)
    mm.load_weights('/var/www/html/medicmind/shared/images/forthtemple/choithramsplit20perceff2/train/EfficientNetV2S_latest.h5', by_name=True, skip_mismatch=True)

    input_data_type=np.float32
    img=Image.open('file.jpg')
    image = np.array(img.resize((image_size,image_size)), dtype=input_data_type)

    image = image / 255.
    superimposed_img, heatmap, preds = visualizing.make_and_apply_gradcam_heatmap(mm, image, layer_name="auto") 
    im = Image.fromarray((255*superimposed_img).astype(np.uint8))
    im.save("heat.png")

[Medicmind]

What seems to work is if I use

mm.load_model('/var/www/html/medicmind/shared/images/forthtemple/choithramsplit20perceff2/train/EfficientNetV2S_latest.h5')
instead of load_weights. Gives consistent heatmaps when I use load_model instead

[leondgarse]

Ya, it shouldn't be. There shouldn't be any random procedure during this...
Speaking of load_model directly, maybe it's affected by rescale_mode? For model created from efficientnet.EfficientNetV2S, will carry a default rescale_mode="tf", or it will be rescale_mode="torch". For other attributes, they are supposed to be same... May try:

mm = efficientnet.EfficientNetV2S(num_classes=2,pretrained=None)
# Or try mm.load_weights('...') without `by_name=True, skip_mismatch=True`
mm.load_weights('/var/www/html/medicmind/shared/images/forthtemple/choithramsplit20perceff2/train/EfficientNetV2S_latest.h5', by_name=True, skip_mismatch=True)

input_data_type=np.float32
img=Image.open('file.jpg')

# image = np.array(img.resize((image_size,image_size)), dtype=input_data_type)  # Resize is also done in function
# image = image / 255.  # This is not needed, as `[0, 1]` image will multiply with 255 in function, making sure it's a `[0, 255]` one for plotting...

superimposed_img, heatmap, preds = visualizing.make_and_apply_gradcam_heatmap(mm, img, layer_name="auto", rescale_mode="torch") 
im = Image.fromarray((255*superimposed_img).astype(np.uint8))
im.save("heat.png")

Anyway, still not sure why they are different... This is a basic test kecam_test.ipynb Heatmap.

[Medicmind]

Thanks that worked. When I did it in combination with rescale_mode="torch" and also feeding the image directly into make_and_apply_gradcam_heatmap instead of adding the 'image = image / 255.' the heatmaps are the same.

[Medicmind]

Damn, the whole problem was caused by a mistake my end. Instead of
python train_script.py -m efficientnet.EfficientNetV2S

I was doing
python train_script.py -s EfficientNetV2S

and was actually training with the default model aotnet50 instead of effecientnet. Was amazing that heatmaps were working at all. I should probably delete this discussion.

[leondgarse]

Lol, at least you solved your issue! Just keep it here for a reminder. :)