Corrupt frame detection and replacement

[masc4ii]

When using ML cameras near to the edge of maximal write speed, we get here and there corrupted frames (mostly half of the frame is pink). It should be possible to detect such frames. In a first version we could manually select such bad frames.

Another task would be to replace such corrupted frames with:

• the previous frame
• the following frame
• a blended frame ((previous+following)/2)
• a synthetic frame (optical flow)

[ilia3101]

Hmm very exciting idea

[ilia3101]

I think corrupt frame replacement should be done inside mlvObject, so it can be treated the same from outside when requesting a frame to process. @bouncyball-git as you have worked on that part of mlv app a lot, do you approve?

[bouncyball-git]

Discovery and replacement of the corrupted frames at the RAW level, I guess, is applicable only for:

"the previous frame" and "the following frame"
Well... may be for blended frame as well but not for optical flow.

As you understand most Important thing here is the reliable discovery alg.

[masc4ii]

I 100% agree. Optical flow should work after debayering only. If we would like to export as DNG / MLV dublicating one frame is easiest, blending could also work.

[ilia3101]

Oh yeah optical flow would not be possible then :/

[masc4ii]

But never mind... I checked some sources for optical flow: yes, very nice (I don't understand anything).

[ilia3101]

Actually we could do optical flow before debayer - split the image in to four: R G1 G2 and B, then optical flow on each one. Though it may generate some grain or pixel pattern oddness. It would be an interesting idea to try. Also may be faster than doing it after debayer.

Those frames should also be cached, as it would not be quick (I assume).

[masc4ii]

If you have a look here, there are several optical flow implementations.
https://www.ipol.im/
It is indeed very interesting!

[DeafEyeJedi]

I am all for this idea actually. Especially on the optical flow for each R G1 G2 & B splitted images re: corrupted frames. Should be doable. :) On Tuesday, July 23, 2019, 6:36:03 AM PDT, masc4ii <[email protected]> wrote: If you have a look here, there are several optical flow implementations. https://www.ipol.im/ It is indeed very interesting! — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub, or mute the thread.

[masc4ii]

Yes, interesting. But I don't know yet what to do with that code. On this page you can try out (life) the algorithms. I find anything else but not the interpolated picture...

[dannephoto]

This idea is really good and important. FFmpeg wise it should be possible to output someting even if frames are missing here and there. Some links:
http://paulherron.com/blog/forcing_keyframes_with_ffmpeg

https://stackoverflow.com/questions/50546359/ffmeg-force-keyframe-at-specific-interval

https://superuser.com/questions/908280/what-is-the-correct-way-to-fix-keyframes-in-ffmpeg-for-dash/908706#908706

[dannephoto]

Some more interesting links:
https://forum.videohelp.com/threads/381679-Finding-individual-bad-frames-in-video-and-save-frame-number-or-repair#post2470118

https://forum.videohelp.com/threads/381679-Finding-individual-bad-frames-in-video-and-save-frame-number-or-repair#post2470175

https://video.stackexchange.com/questions/23589/replace-black-frames-with-previous-frames

[pigeonhill]

Plus 1 for such a feature.

My use case is when using MLV App to process for stills photography, for example averaging an MLV to simulate a long exposure.

With no audio to worry about, ‘all’ one would need to do would be to delete the corrupted frame, ie no continuity issues.

But, of course, I realise this app is mainly for videographers ;-)