Misc of suggestions

[fisheggg]

These might be totally off from our aims, but in terms of better software engineering practice and collaboration with other python libraries, some misc enhancements could be discussed.

Combine ffmpeg commands

Correct me if I'm wrong, it seems that in each function of preprocesses, a separate ffmpeg command is created. If we don't want the intermediate products, this process will create redundant I/O operations since the files are read/write for multiple times.

We could consider separating the process definition and the computation, for example predefine a processing flow, and only calls ffmpeg when we need to actually compute the output. In this way we can optimize the ffmpeg command to combine multiple operations in one call.

Some audio libraries, such as pysox and madmom, uses this type of workflow. I suggest we could look into pysox more, since it is also a python wrapper of a command line library called SoX. It has an in-memory workflow as well, as mentioned in #294 .

pysox: https://github.com/rabitt/pysox
madmom: https://github.com/CPJKU/madmom

Option to suppress progress bar

The progress bar is useful when we manually process files, but in batch processing, especially when running on a computer server with multiprocess, it might become an issue and make the log very hard to read. So I suggest having a switch to turn off the progress bar. For example:

from tqdm import tqdm
mg.progress_bar("off") # this suppresses all progress bars

# batch processing for a video dataset
# here we use the tqdm progress bar to indicate the overall progress
# if the MgProgressBar can't be suppressed, the tqdm progress bar will break
video_processed = [None] * 1e5
for i in tqdm(range(1e5)):
    video_processed[i] = mg.MgVideo(my_video_dataset[i], starttime=5, endtime=15, skip=3, rotate=3, contrast=100, brightness=20, crop='auto', color=False, keep_all=True)

mg.progress_bar("on") # turn the progress bar on again for usual manual processing

video = mg.MgVideo(video_path, starttime=5, endtime=15) # now we want to see the progress for this single video

To be continued...

[fisheggg]

Sorry I can't edit the labels for now! If you see this and has the admin permission please add me to the project. Thanks in advance!

Also sorry if these stuff looks annoying, I'm kinda just using some muscle memories from a software company perspective. If that's not our goal here, please simply close these issues. Cheers!

[balintlaczko]

Great suggestions again, fully agree on the progressbar suppression, and the combined commands as well! The latter would probably result in a significant speed boost as well.