Errors when uploading certain pictures from my cell phone

[liusida]

When I try to use ComfyUI on my cell phone (Android), sometimes, when I try to upload an image, it will give error like this:

!!! Exception during processing!!! Sizes of tensors must match except in dimension 0. Expected size 2688 but got size 1018 for tensor number 1 in the list.
Traceback (most recent call last):
  File "/mnt/data/ComfyUI-BS/ComfyUI/execution.py", line 151, in recursive_execute
    output_data, output_ui = get_output_data(obj, input_data_all)
                             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/mnt/data/ComfyUI-BS/ComfyUI/execution.py", line 81, in get_output_data
    return_values = map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True)
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/mnt/data/ComfyUI-BS/ComfyUI/execution.py", line 74, in map_node_over_list
    results.append(getattr(obj, func)(**slice_dict(input_data_all, i)))
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/mnt/data/ComfyUI-BS/ComfyUI/nodes.py", line 1480, in load_image
    output_image = torch.cat(output_images, dim=0)
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
RuntimeError: Sizes of tensors must match except in dimension 0. Expected size 2688 but got size 1018 for tensor number 1 in the list.

And when I paused the process right before the error happens, and I inspect the variable img, I saw this (with n_frames=2), but it's a JPG! I don't know why, and ChatGPT says that it's a MPO file.

Extracted with ChatGPT:

img: <PIL.MpoImagePlugin.MpoImageFile image mode=L size=572x1018 at 0x7F1E153924D0>

special variables
function variables

app: {'APP1': b'http://ns.adobe.com/xmp/extension/\x01\x00\x08\xAC0B6D96AB3A7066059E016278C9C8...\x00\x00\x92\x00\x00\x10\x01\x02\x00\x08\x00\x00\x00'}
applist: [('APP0', b'JFIF\x00\x01\x02\x00\x48\x00\x48\x00\x00'), ('APP1', b'http://ns.adobe.com/xap/1.0/\x00...\x00\x00\x92\x00\x00\x12\x01\x03...\x00')]
bits: 8
custom_mimetype: None
decoderconfig: {}
decodermaxblock: 65536
filename: '/mnt/data/ComfyUI-BS/ComfyUI/input/PXL_20240507_013340406 (5).jpg'
format: 'MPO'
format_description: 'MPO (CIPA DC-007)'
fp: None
has_transparency_data: False
height: 2688
huffman_ac: {}
huffman_dc: {}
icclist: []
im: <ImagingCore object at 0x7f1e1422d5e0>
info: {'dpi': (72.0, 72.0), 'jfif': 258, 'jfif_version': (1, 2)}
is_animated: True
layer: [(1, 2, 2, 0), (2, 1, 1, 1), (3, 1, 1, 1)]
layers: 3
map: None
mode: 'RGB'
mpinfo: {45056: b'0100', 45057: 2, 45058: [{...}, {...}]}
n_frames: 2
offset: 0
palette: None
pyaccess: None
quantization: {0: [3, 3, 2, 3, 5, 8, 10, 12, 2, 4, 4, ...], 1: [3, 3, 5, 7, 11, 11, 12, 3, 4, 6, 8, 10, 2, 4, 4, ...]}
readonly: 0
size: (1512, 2688)
tile: []

I never encounter this kind of JPGs on my Desktop, ether Ubuntu or Windows. I'm not sure why cell phones store some JPGs like that (no all JPGs have two frames, only some of them do). But this causes errors in ComfyUI's LoadImage node.

I'll try to upload one of the JPG files here in the comment. To reproduce the error, save the tomato photo and try to upload with ComfyUI's Load Image node.

[liusida]

[comfyanonymous]

Should be fixed now.

[liusida]

@shawnington I've just tested your update #3422. And interestingly, it works for PIL=10.2.0, but the very picture caused another error when using PIL=10.3.0.

I think the tomato picture is weird, because the second frame is not the same as the first frame, I don't know why my Pixel 6's default camera save the photo this way, but it's real.

Here is the error message:

Error occurred when executing LoadImage:

Sizes of tensors must match except in dimension 0. Expected size 2688 but got size 1018 for tensor number 1 in the list.

File "C:\ComfyUI\ComfyUI-origin\execution.py", line 151, in recursive_execute
output_data, output_ui = get_output_data(obj, input_data_all)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\ComfyUI\ComfyUI-origin\execution.py", line 81, in get_output_data
return_values = map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\ComfyUI\ComfyUI-origin\execution.py", line 74, in map_node_over_list
results.append(getattr(obj, func)(**slice_dict(input_data_all, i)))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\ComfyUI\ComfyUI-origin\nodes.py", line 1481, in load_image
output_image = torch.cat(output_images, dim=0)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

And I found that the dimension of two frames are different:

and I found that the 10.3.0 version ImageOps.exif_transpose returns a smaller dimension image.

==
Of course, the tomato photo is absolutely a weird one, here is the visual inspection of the second frame using PIL == 10.2.0:

[liusida]

I just tested all my photos taken using my Pixel 6's default camera, they all behave the same. (2 frames, and the second is not quite standard)
Here's another example:

[shawnington]

@shawnington I've just tested your update #3422. And interestingly, it works for PIL=10.2.0, but the very picture caused another error when using PIL=10.3.0.

I think the tomato picture is weird, because the second frame is not the same as the first frame, I don't know why my Pixel 6's default camera save the photo this way, but it's real.

Here is the error message:

Error occurred when executing LoadImage:

Sizes of tensors must match except in dimension 0. Expected size 2688 but got size 1018 for tensor number 1 in the list.

File "C:\ComfyUI\ComfyUI-origin\execution.py", line 151, in recursive_execute
output_data, output_ui = get_output_data(obj, input_data_all)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\ComfyUI\ComfyUI-origin\execution.py", line 81, in get_output_data
return_values = map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\ComfyUI\ComfyUI-origin\execution.py", line 74, in map_node_over_list
results.append(getattr(obj, func)(**slice_dict(input_data_all, i)))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\ComfyUI\ComfyUI-origin\nodes.py", line 1481, in load_image
output_image = torch.cat(output_images, dim=0)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

And I found that the dimension of two frames are different:

and I found that the 10.3.0 version ImageOps.exif_transpose returns a smaller dimension image.

== Of course, the tomato photo is absolutely a weird one, here is the visual inspection of the second frame using PIL == 10.2.0:

Its unfortunately going to be a tricky issue to keep up with, and Im sure our triage will break with different versions of PIL, as its not an actual comfy bug, its a work around for a fairly long standing PIL bug. Id recommend using the version that works, and maybe we should looking into including version 10.2.0 in the requirements.txt so long as it doesn't break additional functionality.

Im going to do some more digging, and see if there is a more robust way to solve this, as photos from phones are quite a common input, for myself as well. I take pictures with my phone that I wan't to re-imagine into something cool, so I understand your frustration. Can you by chance provide the make model and software version for the phone you are have errors with?

There is always a small chance that there is a known bug with the phone that is causing errors.

The commit took the fix Comfy contributed and merged it into a generalized fix for embedded ICC profiles in PNGs being interpreted as decompression attacks, as they both used the exact same solution to resolve the issue. So it just made it a wrapper function that you can pass any PIL function that is impacted by that kind of bug into.

I'm going to have to look into the differences between pillow 10.3.0 and 10.2.0, maybe it will elucidate an actual reason the bug is happening.

[ltdrdata]

Its unfortunately going to be a tricky issue to keep up with, and Im sure our triage will break with different versions of PIL, as its not an actual comfy bug, its a work around for a fairly long standing PIL bug. Id recommend using the version that works, and maybe we should looking into including version 10.2.0 in the requirements.txt so long as it doesn't break additional functionality.

Im going to do some more digging, and see if there is a more robust way to solve this, as photos from phones are quite a common input, for myself as well. I take pictures with my phone that I wan't to re-imagine into something cool, so I understand your frustration. Can you by chance provide the make model and software version for the phone you are have errors with?

There is always a small chance that there is a known bug with the phone that is causing errors.

Maybe we need to spec as Pillow!=10.3.0

[liusida]

For the app:
version 9.3.160.621982096.22
com.google.android.GoogleCamera

My phone is Pixel 6, manufactured in 2021 with hardware version MP1.0.

[shawnington]

Its unfortunately going to be a tricky issue to keep up with, and Im sure our triage will break with different versions of PIL, as its not an actual comfy bug, its a work around for a fairly long standing PIL bug. Id recommend using the version that works, and maybe we should looking into including version 10.2.0 in the requirements.txt so long as it doesn't break additional functionality.
Im going to do some more digging, and see if there is a more robust way to solve this, as photos from phones are quite a common input, for myself as well. I take pictures with my phone that I wan't to re-imagine into something cool, so I understand your frustration. Can you by chance provide the make model and software version for the phone you are have errors with?
There is always a small chance that there is a known bug with the phone that is causing errors.

Maybe we need to spec as Pillow!=10.3.0

Thats actually much more reasonable, as long as we confirm its just isolated to 10.3.0

Im not super familiar with the proper format to spec requirements.txt for something like this, would you want to commit the change? If not I'll figure it out and commit it.

[liusida]

I just digged into the camera's settings, and I found that if I disable the Ultra HDR feature (which was enabled by default), I get only one frame!

Here's the documentation of this feature (with it's format JPEG_R!):
https://source.android.com/docs/core/camera/ultra-hdr

[liusida]

And follow this lead, I found this issue from PIL:

python-pillow/Pillow#8036

Yes, they need to support that to solve this issue.

[shawnington]

I just digged into the camera's settings, and I found that if I disable the Ultra HDR feature (which was enabled by default), I get only one frame!

I actually was about to post a follow up question about that. There is an outstanding issue having to do with HDR images that looked very similar to yours, where a smaller image that appeared to be encoding exposure data was included in a similar format, it seems to be limited to android currently.

It's definitely a PIL issue, I'll update with the issue number when I find it.

[shawnington]

And follow this lead, I found this issue from PIL:

python-pillow/Pillow#8036

Yes, they need to support that to solve this issue.

You beat me to it, but so are you saying that it works with your Ultra HDR images in 10.2.0 and not 10.3.0? Or does it not work with the those images in either?

[liusida]

Its unfortunately going to be a tricky issue to keep up with, and Im sure our triage will break with different versions of PIL, as its not an actual comfy bug, its a work around for a fairly long standing PIL bug. Id recommend using the version that works, and maybe we should looking into including version 10.2.0 in the requirements.txt so long as it doesn't break additional functionality.
Im going to do some more digging, and see if there is a more robust way to solve this, as photos from phones are quite a common input, for myself as well. I take pictures with my phone that I wan't to re-imagine into something cool, so I understand your frustration. Can you by chance provide the make model and software version for the phone you are have errors with?
There is always a small chance that there is a known bug with the phone that is causing errors.

Maybe we need to spec as Pillow!=10.3.0

I'll downgrade my PIL for now. Everything works for 10.2.0. Thanks @shawnington and @ltdrdata .

[shawnington]

Its unfortunately going to be a tricky issue to keep up with, and Im sure our triage will break with different versions of PIL, as its not an actual comfy bug, its a work around for a fairly long standing PIL bug. Id recommend using the version that works, and maybe we should looking into including version 10.2.0 in the requirements.txt so long as it doesn't break additional functionality.
Im going to do some more digging, and see if there is a more robust way to solve this, as photos from phones are quite a common input, for myself as well. I take pictures with my phone that I wan't to re-imagine into something cool, so I understand your frustration. Can you by chance provide the make model and software version for the phone you are have errors with?
There is always a small chance that there is a known bug with the phone that is causing errors.

Maybe we need to spec as Pillow!=10.3.0

I'll downgrade my PIL for now. Everything works for 10.2.0. Thanks @shawnington and @ltdrdata .

Thanks, @ltdrdata do you want to commit the Pillow!=10.3.0 as you have more experience with requirements.txt than me or should I?

[ltdrdata]

Its unfortunately going to be a tricky issue to keep up with, and Im sure our triage will break with different versions of PIL, as its not an actual comfy bug, its a work around for a fairly long standing PIL bug. Id recommend using the version that works, and maybe we should looking into including version 10.2.0 in the requirements.txt so long as it doesn't break additional functionality.
Im going to do some more digging, and see if there is a more robust way to solve this, as photos from phones are quite a common input, for myself as well. I take pictures with my phone that I wan't to re-imagine into something cool, so I understand your frustration. Can you by chance provide the make model and software version for the phone you are have errors with?
There is always a small chance that there is a known bug with the phone that is causing errors.

Maybe we need to spec as Pillow!=10.3.0

I'll downgrade my PIL for now. Everything works for 10.2.0. Thanks @shawnington and @ltdrdata .

Thanks, @ltdrdata do you want to commit the Pillow!=10.3.0 as you have more experience with requirements.txt than me or should I?

I just made a PR.

[liusida]

And follow this lead, I found this issue from PIL:
python-pillow/Pillow#8036
Yes, they need to support that to solve this issue.

You beat me to it, but so are you saying that it works with your Ultra HDR images in 10.2.0 and not 10.3.0? Or does it not work with the those images in either?

@shawnington , it doesn't cause problem because PIL 10.2.0 recognizes the Ultra HDR image as two frames with the same resolution. I think it is a reasonable assumption because how can two frames be different even if it was a video?

So when PIL 10.2.0 was used, I have this:

As long as I ONLY use the FIRST frame in the workflow, there's no error.

PIL 10.3.0 changes this assumption, and returns the "right" resolution of the second frame, which is smaller by JPEG_R format, and that causes the torch.concat() to go wrong.

[shawnington]

And follow this lead, I found this issue from PIL:
python-pillow/Pillow#8036
Yes, they need to support that to solve this issue.

You beat me to it, but so are you saying that it works with your Ultra HDR images in 10.2.0 and not 10.3.0? Or does it not work with the those images in either?

@shawnington , it doesn't cause problem because PIL 10.2.0 recognizes the Ultra HDR image as two frames with the same resolution. I think it is a reasonable assumption because how can two frames be different even if it was a video?

So when PIL 10.2.0 was used, I have this:

As long as I ONLY use the FIRST frame in the workflow, there's no error.

PIL 10.3.0 changes this assumption, and returns the "right" resolution of the second frame, which is smaller by JPEG_R format, and that causes the torch.concat() to go wrong.

Okay, interesting. There should definitely be multiple ways to work around that, either discarding the second frame if its not the same size, resizing the second frame so touch.cat doesn't fail, etc.

if len(output_images) > 1:
            output_image = torch.cat(output_images, dim=0)
            output_mask = torch.cat(output_masks, dim=0)

This seems to be the offending code. it seems with most standards for HDR the exposure data is a small thumbnail when processed correctly. Maybe there needs to be a check for matching dimensions and discard the extra images if they are there.

Im not sure what the desired behavior is, obviously a loading of the image type as a single image is preferable, but I don't think we are going to be able to get the processing done with the HDR intent intact.

[shawnington]

This commit should fix this issue for both PIL 10.2 and 10.3

Feel free to add this to your local branch to test as it's not yet merged.

excluded_formats = ['MPO']

if len(output_images) > 1 and img.format not in excluded_formats:
    output_image = torch.cat(output_images, dim=0)
    output_mask = torch.cat(output_masks, dim=0)

[shawnington]

@liusida check the latest commit, it should resolve all of your issue. If so, this issue can be marked closed.

Thanks for your detailed reporting and contribution in figuring out what was actually causing the problem. The issue is now resolved for me for every PIL version I have tried with your supplied images, I tested down to 9.5.0

[liusida]

@liusida check the latest commit, it should resolve all of your issue. If so, this issue can be marked closed.

Thanks for your detailed reporting and contribution in figuring out what was actually causing the problem. The issue is now resolved for me for every PIL version I have tried with your supplied images, I tested down to 9.5.0

Now it works perfectly for me locally. Only the primary images show up. Thanks for the update!