Generic StreamReadError

[standage]

My colleague @Parsoa has been using khmer to implement some genotyping software, and he has run into a snag. The issue can be isolated with the following minimal example.

import khmer
counttable = khmer.Counttable(31, 16e9, 4)
nseqs, nkmers = counttable.consume_seqfile('/share/hormozdiarilab/Data/Genomes/Illumina/1KG_Trio/HG00512.fq')

This code runs for approximately a day before failing with the following error.

Traceback (most recent call last):
  File "/usr/lib/python3.4/runpy.py", line 170, in _run_module_as_main
    "__main__", mod_spec)
  File "/usr/lib/python3.4/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/share/hormozdiarilab/Codes/NebulousSerendipity/counttable.py", line 5, in <module>
    n, kmers = counttable.consume_seqfile('/share/hormozdiarilab/Data/Genomes/Illumina/1KG_Trio/HG00513.fq')
  File "khmer/_oxli/graphs.pyx", line 235, in khmer._oxli.graphs.Hashtable.consume_seqfile (khmer/_oxli/graphs.cpp:5839)
OSError: Generic StreamReadError error

For such a simple example involving code that is executed so frequently elsewhere, we're having a hard time troubleshooting what the error might be. We've discussed several possibilities, each seemingly as unlikely as the next.

• Problem with the Fastq file: probably not the case. Other programs have run just fine on it, and a malformed Fastq file usually (in my experience) elicits a more specific error message from khmer.
• Problem with khmer: possible, but also unlikely. We're running this very code very frequently.
• Problem with filesize: it's a 450 Gb uncompressed Fastq file. Is there something about the filesize that might be problematic?
• Problem with the machine: the Cabernet cluster isn't noted for its stability or robustness. Could (i.e.) transient filesystem issues result in this error?

Has anybody else seen this error message? Under what circumstances?