How to apply this for motion detection?

[ceewanna]

In video.stats.csv, how do we make use of the data columns to identify motions? Is it possible to do so?
Currently I have ffmpeg filtering i frames for scene score and converting the output into csv format string which is then fed into python dataframe for analysis. I wonder if it's possible to do similar thing with video.stats.csv. If so, please kindly advise or give some sample code.

My experience with ffmpeg's scene scores is that there seems to be scores every 4 seconds and these scores are well above the other scores associated with the remaining frames. Currently I take thse i-frame scores into 2 approaches: 1) detect changes in score and tagging only changes larger than a threshold, and 2) perform percentile analysis on 2-minute video and check if p95% is larger than a threshold and tagging frames with scores above p50%. I am still struggling to understand its behavior. I don't think it will work or stable enough.

[wjs018]

I am not super familiar with ffmpeg, so I can't necessarily help you out with that part, but I can help you understand how to use motion within the frame to detect new scenes using PySceneDetect. Note that all the motion detection in PySceneDetect is slated for an upcoming release (0.6.1), but you can download that branch and install it now if you want (this is how I tested my examples below).

One thing to clarify before proceeding is what we mean by motion. There are two main ways motion within video is used for scene detection:

1. The video primarily consists of a static image (e.g. security camera footage) and a motion event is triggered when there is motion exceeding a certain threshold is detected.
2. The video is not static and motion detection is used to help reduce the false positive rate of scene detection methods introduced through camera motion (e.g. pans and zooms).

PySceneDetect can help with the second of these much more than the first. If you are interested in the first type of detection, might I suggest a different library by @Breakthrough instead, DVR-Scan. Alright, now on to how to use motion detection in PySceneDetect.

The detect-content and detect-adaptive commands have motion detection built-in (as of 0.6.1) as one of the components it uses to determine when to trigger a new scene. This means, in addition to the HSV data of the frame like has been used in previous versions, it also tries to compensate for motion within the frame (through an edge-finding algorithm) and factors that into whether a new scene is triggered or not. The relative weight to put on each component (hue, saturation, value, edges) can be specified at the command line using the --weights/-w option. The default value for -w is 1.0 1.0 1.0 0.0 meaning that the three color inputs are weighted equally and the motion detection is ignored. To use only the edges as input and ignore color information, we can instead specify -w 0.0 0.0 0.0 1.0. Do note that the default threshold values might not work as well if you don't use the default weights. You will want to look at your stats file to help you choose a better threshold (delta_edges column).

Doing a bit of testing on my end with the goldeneye video we use for our built-in tests, I found that using detect-content -t 80 -w 0.0 0.0 0.0 1.0 gave pretty good results. Alternatively, using detect-adaptive -t 4.0 -w 0.0 0.0 0.0 1.0 worked pretty well.