Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Filtering function oddities in motiongram() #274

Open
alexarje opened this issue Jan 12, 2023 · 4 comments
Open

Filtering function oddities in motiongram() #274

alexarje opened this issue Jan 12, 2023 · 4 comments
Assignees
Labels
bug Something isn't working video

Comments

@alexarje
Copy link
Contributor

I just tried the various filtering functions, threshold and type, but they don't seem to work. See here for a threshold of 0.0:

Screenshot from 2023-01-12 09-33-49

Is this related to the FFmpeg change?

@alexarje alexarje added the bug Something isn't working label Jan 12, 2023
@joachimpoutaraud
Copy link
Contributor

I tested the function on both Windows and Linux OS but it seems to work properly. Did you use the last version of musicalgestures (1.2.6)?

test

Note that the default non-thresholded motiongram using ffmpeg is a bit more noisy compared to using OpenCV. This is due to the different kind of processing involved as well as the temporal averaging denoiser that is applied (see #109)

@alexarje alexarje added the video label Jan 21, 2023
@alexarje
Copy link
Contributor Author

Yes, I get some effect, but perhaps less than I had expected. The filter goes from 0 to 1 right? So I would expect that when you filter at 0.0 you would get a lot of noise, while at 0.9 very little signal?

@joachimpoutaraud
Copy link
Contributor

joachimpoutaraud commented Jan 31, 2023

Yes from 0 to 1, the problem is related to the way FFmpeg filters each motion frame. I suspect that because there is not a lot of color possibility in FFmpeg, the threshold cannot be very precise. However, if you want I can add the possibility to use traditional filtering but it will take more time to process.

Here are some examples with a threshold set to zero:
FFMPEG >>> Processing time: 16.667 s
dance_mgy_0

TRADITIONAL >>> Processing time: 81.574 s
dance_mgy

@alexarje
Copy link
Contributor Author

I test this yesterday, and see that the visual differences are quite large between the old and new function. See motiongrams of the same video made with the default settings of the old and new.

gJB_sFM_c09_d08_mJB0_ch08_mgy-1-684x1024

aist_mgx

It would be good to investigate a little more what we can do to improve the filtering of the motiongrams with the new function. The speed benefit is huge, so it would be very nice to work in FFmpeg. Perhaps a midterm solution could be to add the option of using both?

@alexarje alexarje changed the title Filtering functions don't work Filtering function oddities in motiongram() Jan 11, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working video
Projects
None yet
Development

No branches or pull requests

2 participants