Video exported from webcam is a bit faster than usual

[lasagnaphil]

I'm currently facing a weird issue which the recorded video from the webcam is faster than the original (for instance, when recording webcam for 60 seconds, the exported footage is 47 seconds). Any ideas to why this is happening?
Note: the footage recorded is 30fps

Source code:

public class PCaptureWindow extends PApplet {

    private Capture cam;
    private VideoExport videoExport;
    private CaptureState state;

    private boolean shouldSaveFrame = false;

    public PCaptureWindow(CaptureState state) {
        super();
        this.state = state;
        PApplet.runSketch(new String[] {this.getClass().getSimpleName()}, this);
    }

    public void settings() {
        size(state.resolutionX, state.resolutionY);
    }

    public void setup() {
        String[] captureList = Capture.list();
        for (String capture : captureList) System.out.println(capture);
        frameRate(state.fps);
        if (state.cameraName.equals("default"))
            cam = new Capture(this, state.resolutionX, state.resolutionY, state.fps);
        else
            cam = new Capture(this, state.resolutionX, state.resolutionY, state.cameraName, state.fps);

        cam.start();

        videoExport = new VideoExport(this, state.getCurrentVideoName(), cam);
        videoExport.setFrameRate(state.fps);

        // register callbacks
        state.addCaptureStartListener(() -> {
            videoExport.setMovieFileName(state.getCurrentVideoName());
            videoExport.startMovie();
            shouldSaveFrame = true;
        });
        state.addCaptureEndListener(() -> {
            shouldSaveFrame = false;
            videoExport.endMovie();
        });
    }

    public void draw() {

        if (cam.available()) {
            cam.read();
        }
        background(0);

        image(cam, 0, 0);
        if (state.mode == CaptureMode.Capture && shouldSaveFrame) {
            videoExport.saveFrame();
        }
    }

    public void exit() {
        state.stopCapture();
        state.saveToConfig();
        super.exit();
    }

}

The full source code is in: https://github.com/lasagnaphil/display-art-project

[hamoid]

The reason is probably that it does not manage to save 30 frames per second. If frames arrive a bit slower, at the end you don't have enough frames but you do have enough audio.

The library does not currently handle this well, but I could try fix it. Maybe by implementing videoExport.setRealtime(true);. Then the library would drop frames if you call .saveFrame() too often, or repeat frames if you don't call .saveFrame() them fast enough.

You could even try implement this on your side. You should keep track of time using millis(), and if more than 1/30s of a second has passed since last time you called .saveFrame(), you should call it multiple times. A hack, and maybe it doesn't look so smooth, but at least it should be in sync. Let me know if you try! :)

[hamoid]

But the first step would be to diagnose the issue, by printing the times when draw is called and how many actual frames per second it is saving. If the reason the frame rate is slow is a slow hard drive, then my hack above wouldn't work. In that case, if the videos being produced are not very long, you could use a ram disk to save first the video to a virtual disk, and when done, copy the video to the hard drive.

[hamoid]

Apparently ffmpeg does not allow specifying frame timestamps, so I should try doing this as I suggested above. See this discussion: https://ffmpeg.org/pipermail/ffmpeg-user/2012-December/011828.html

Although... that's from 2012. Here some flags to try: https://stackoverflow.com/questions/41768839/how-to-make-ffmpeg-output-frames-at-correct-rate-down-to-millisecond

[aldobranti]

I am having some problems relating the above (ffmpeg not getting enough frames as the render is running slow ) with the answer to #9 [example of slow rendering] -- is there a mismatch ?
My situation: I am colour tracking in a 30fps movie, and calling export once every draw() cycle. Average execution time for draw() is ~ 22mS , thinks this must be fast enough , I see about 9000 frames in , I only get 37 secs of video (~ 1200 frames) written to the output file. I am continuing to dig but any thoughts about the lossage would be good...

[hamoid]

@aldobranti Hi, it's hard to say without seeing the code. If you don't care about doing it real time, you could extract all frames to jpg using ffmpeg, and then make the movie frame by frame. The result should then have the exact number of frames you expect.