demo.py have problem

[LORDPQK]

when i run demo.py and use my picture python demp.py video/v1 ,this has a bug
RuntimeError: shape mismatch: value tensor of shape [256, 2048] cannot be broadcast to indexing result of shape [1280, 2048]

and i try to use demo.ipyng but sitll has a bug:

Floating point exception (core dumped)

[HarborYuan]

Hi @LORDPQK ,

This usually happens when you pass only one image to the video argument.

[LORDPQK]

Hi @LORDPQK ,

This usually happens when you pass only one image to the video argument.

Thank you for your response
but when i try to pass 600 image to the video i still have a problem:
Floating point exception (core dumped)

[HarborYuan]

Can you show me the details about the code and the errors?

[HarborYuan]

What is in the "my_video_picture_folder"

[HarborYuan]

I think it may be because your image files do not follow:

Sa2VA/demo/demo.py

Line 53 in 6f59e55

image_extensions = {".jpg", ".jpeg", ".png", ".bmp", ".gif", ".tiff"}

or there are some errors in the image.

[LORDPQK]

What is in the "my_video_picture_folder"

i put 600 picture in this folder

[LORDPQK]

I think it may be because your image files do not follow:

Sa2VA/demo/demo.py

Line 53 in 6f59e55

image_extensions = {".jpg", ".jpeg", ".png", ".bmp", ".gif", ".tiff"}
or there are some errors in the image.

"My image format is .png, I think there shouldn't be any issues with my image format. I extracted the video frames from the video you provided at assets/videos/gf_exp1.mp4. However, after installing the requirements.txt, I found there were still bugs, so I reinstalled flash attention using flash_attn-2.7.4.post1+cu12torch2.3cxx11abiFALSE-cp310-cp310-linux_x86_64.whl. Could this be the source of the problem?"

RuntimeError: Failed to import transformers.models.llama.modeling_llama because of the following error (look up to see its traceback):/root/anaconda3/lib/python3.10/site-packages/flash_attn_2_cuda.cpython-310-x86_64-linux-gnu.so: undefined symbol: _ZNK3c105Error4whatEv

[LORDPQK]

I have the same problem when i use demo.ipynb ：Floating point exception (core dumped)

[wshiman]

Hi @LORDPQK ,

This usually happens when you pass only one image to the video argument.

Hi @LORDPQK ,

This usually happens when you pass only one image to the video argument.

When I use a single RTX 3090, I found that when the number of images is less than 5 in PATH_TO_FOLDER (images containing videos), there will be a value tensor of shape problem, and when the number of images is greater than or equal to 5, it will be OOM, how to use the maximum value of a single 3090? (The text+mask reasoning of a single 3090 is measured to be about 15GB of video memory usage) or how to solve the problem of PATH_TO_FOLDER?

[Gy-X]

when i run demo.py and use my picture python demp.py video/v1 ,this has a bug RuntimeError: shape mismatch: value tensor of shape [256, 2048] cannot be broadcast to indexing result of shape [1280, 2048]

and i try to use demo.ipyng but sitll has a bug:

Floating point exception (core dumped)

我想知道你解决这个问题了？我遇到了相同的bug

[LORDPQK]

当我运行 demo.py 并使用我的图片 python demp.py video/v1 时，出现了一个错误 RuntimeError: 形状不匹配：形状 [256, 2048] 的值张量无法广播到形状 [1280, 2048] 的索引结果
我尝试使用 demo.ipyng 但是它有一个错误：
浮点异常（核心转储）

我想知道你解决了这个问题吗？我遇到了相同的错误

you mean "Floating point exception (core dumped)"?

[Gy-X]

当我运行 demo.py 并使用我的图片 python demp.py video/v1 时，出现了一个错误 RuntimeError: 形状不匹配：形状 [256, 2048] 的值张量无法广播到形状 [1280, 2048] 的索引结果
我尝试使用 demo.ipyng 但是它有一个错误：
浮点异常（核心转储）

我想知道你解决了这个问题吗？我遇到了相同的错误

you mean "Floating point exception (core dumped)"?

This bug：RuntimeError: shape mismatch: value tensor of shape [256, 2048] cannot be broadcast to indexing result of shape [1280, 2048]

[HarborYuan]

When I use a single RTX 3090, I found that when the number of images is less than 5 in PATH_TO_FOLDER (images containing videos), there will be a value tensor of shape problem, and when the number of images is greater than or equal to 5, it will be OOM, how to use the maximum value of a single 3090? (The text+mask reasoning of a single 3090 is measured to be about 15GB of video memory usage) or how to solve the problem of PATH_TO_FOLDER?

Hi @Gy-X , @LORDPQK @wshiman

In my server, I also meet this problem. I already know how to fix this problem, but I am working on something else. I will fix this problem in the future (one week later I guess).

For now, I think it should work as @wshiman mentioned. Please at least provide 5 frames for a video to test.