[3.15] gh-98820: Fix quadratic time in csv.Sniffer for quoted fields

[serhiy-storchaka]

The four regular expressions which look for a quoted field match its body lazily. A closing quote which is not followed by a delimiter is therefore retried with every following quote, to the very end of the sample, and findall() repeats that from every start position -- so the search is quadratic. On a single column of quoted fields none of them ever matches, so the whole cost is a failing scan.

>>> sample = '"abcdefghijklmnopqrstuvwxyz"\n' * 30000
>>> csv.Sniffer().sniff(sample, delimiters=',:|\t')   # minutes

The body now ends at the first quote which is not doubled, which is the only closing quote a reader would accept, and is matched possessively.

rows      before      after
 4000    3.1264 s    0.0427 s
30000        --      0.3190 s

A minimal reproducer needs no quoted fields at all -- ',"x' * n is quadratic too, because two of the four patterns are anchored on a delimiter rather than a line.

Being unambiguous is what makes it linear, so a few matches necessarily change: 16 of 560 files in the CSVsniffer corpora. Against their ground truth the net is positive -- delimiter 73.3% -> 74.5%, quotechar 81.3% -> 81.1%, doublequote unchanged.

main is not affected: the sniffer was rewritten there in gh-83273.

• Issue: csv.Sniffer().sniff() Hangs on Single-Column Data #98820