SWEEP_REPORT.md

Detector parameter sweep report

Generated by benchmark/report_sweep.py from benchmark/sweep.py grid results (hash/hist were swept with min_scene_len fixed at its default). F1/precision/recall are percentages on hard cuts; tol = frame tolerance.

detect-content

Best cell per dataset (by F1@1):

Dataset	F1@1	Prec@1	Rec@1	F1@0	Params
BBC	88.34	90.00	86.75	87.35	min_scene_len=0.8, threshold=25
AutoShot	73.44	79.54	68.21	70.54	min_scene_len=0.4, threshold=29
ClipShots	66.74	58.93	76.95	66.44	min_scene_len=0.8, threshold=35

Best params averaged across all datasets (mean F1@1):

Mean F1@1	BBC	AutoShot	ClipShots	Params
73.39	85.95	71.51	62.73	min_scene_len=0.6, threshold=31
73.37	84.27	70.95	64.89	min_scene_len=0.6, threshold=33
73.32	83.99	72.43	63.54	min_scene_len=0.4, threshold=33
73.15	87.28	72.05	60.13	min_scene_len=0.6, threshold=29
73.09	85.18	72.97	61.10	min_scene_len=0.4, threshold=31

detect-adaptive

Best cell per dataset (by F1@1):

Dataset	F1@1	Prec@1	Rec@1	F1@0	Params
BBC	94.57	94.89	94.26	93.50	adaptive_threshold=2, min_scene_len=0.6, window_width=2
AutoShot	77.19	80.48	74.16	75.45	adaptive_threshold=3.5, min_scene_len=0.4, window_width=3
ClipShots	65.53	60.19	71.89	65.47	adaptive_threshold=5.5, min_scene_len=0.6, window_width=3

Best params averaged across all datasets (mean F1@1):

Mean F1@1	BBC	AutoShot	ClipShots	Params
76.34	90.41	76.27	62.32	adaptive_threshold=3.5, min_scene_len=0.6, window_width=3
76.21	87.87	76.17	64.58	adaptive_threshold=4, min_scene_len=0.6, window_width=3
76.18	90.43	77.19	60.93	adaptive_threshold=3.5, min_scene_len=0.4, window_width=3
76.16	87.91	77.07	63.51	adaptive_threshold=4, min_scene_len=0.4, window_width=3
75.45	85.28	75.71	65.37	adaptive_threshold=4.5, min_scene_len=0.6, window_width=3

detect-hash

Best cell per dataset (by F1@1):

Dataset	F1@1	Prec@1	Rec@1	F1@0	Params
BBC	86.91	81.59	92.96	85.81	size=16, threshold=0.425
AutoShot	70.17	76.06	65.12	66.89	size=8, threshold=0.325
ClipShots	56.38	44.46	77.03	55.66	size=8, threshold=0.4

Best params averaged across all datasets (mean F1@1):

Mean F1@1	BBC	AutoShot	ClipShots	Params
69.83	86.38	68.98	54.12	size=8, threshold=0.35
69.63	86.65	65.86	56.38	size=8, threshold=0.4
69.63	86.65	65.86	56.37	size=8, threshold=0.375
68.18	84.28	70.17	50.10	size=8, threshold=0.325
67.00	83.71	61.66	55.65	size=8, threshold=0.425

detect-hist

Note: thresholds >= 0.21 come from a grid-extension run (detect-hist-ext-<dataset>.json) after the initial grid's best cell landed on its 0.20 upper edge.

Best cell per dataset (by F1@1):

Dataset	F1@1	Prec@1	Rec@1	F1@0	Params
BBC	86.58	87.32	85.86	85.42	bins=128, threshold=0.11
AutoShot	68.99	75.29	63.67	65.70	bins=128, threshold=0.2
ClipShots	53.25	46.26	62.74	52.90	bins=128, threshold=0.35

Best params averaged across all datasets (mean F1@1):

Mean F1@1	BBC	AutoShot	ClipShots	Params
66.27	82.10	68.99	47.72	bins=128, threshold=0.2
66.23	81.47	68.87	48.36	bins=128, threshold=0.21
66.20	82.74	68.81	47.06	bins=128, threshold=0.19
66.19	80.69	68.92	48.97	bins=128, threshold=0.22
66.17	79.53	68.88	50.10	bins=128, threshold=0.24

detect-threshold

Note: detect-threshold detects fades (fade to/from black), not hard cuts. These datasets' ground truth is hard cuts, so the hard-cut F1 below is expectedly near zero. It is included to validate the sweep pipeline end-to-end, not as a meaningful hard-cut accuracy result.

Best cell per dataset (by F1@1):

Dataset	F1@1	Prec@1	Rec@1	F1@0	Params
BBC	0.79	2.89	0.45	0.32	min_scene_len=0.2, threshold=19
AutoShot	3.98	51.09	2.07	3.14	min_scene_len=0.4, threshold=20
ClipShots	1.75	6.21	1.02	0.18	min_scene_len=0, threshold=10

Best params averaged across all datasets (mean F1@1):

Mean F1@1	BBC	AutoShot	ClipShots	Params
2.07	0.77	3.90	1.55	min_scene_len=0, threshold=19
2.06	0.73	3.98	1.48	min_scene_len=0, threshold=20
2.02	0.79	3.90	1.37	min_scene_len=0.2, threshold=19
2.00	0.71	3.98	1.30	min_scene_len=0.2, threshold=20
1.96	0.69	3.90	1.28	min_scene_len=0.4, threshold=19