When processing TSV file from ~260GB zip file, mlr stops processing abruptly while filtering stream

[jhpoelen]

bio-guoda/preston#228

In processing the retrieved zip with hash://sha256/c8bac8acb28c8524c53589b3a40e322dbbbdadf5689fef2e20266fbf6ddf6b97 using unzip tools, I was able to count 2.30 billion records

unzip -p c8bac8acb28c8524c53589b3a40e322dbbbdadf5689fef2e20266fbf6ddf6b97 0015281-230224095556074.csv\
 | pv -l\
 > /dev/null

yielded

2.30G 1:43:31 [ 370k/s]

This alternate method to count lines in a known resource (e.g., content id is known) suggests that the alternate method used in https://gist.github.com/jhpoelen/569a3a787f6da542c8202ecddbacf580 or

preston cat 'zip:hash://sha256/c8bac8acb28c8524c53589b3a40e322dbbbdadf5689fef2e20266fbf6ddf6b97!/0015281-230224095556074.csv'\
 | pv -l\
 | mlr --tsvlite filter '$collectionCode == "CASTYPE"'

which initially yielded

2.07G 5:16:05 [ 109k/s]

in the workflow described by https://gist.github.com/jhpoelen/569a3a787f6da542c8202ecddbacf580

[jhpoelen]

possibly related to #1088 ?