[QST] Does nvcomp support streaming decompression of .gz files?

[fkallen]

Does nvcomp support streaming decompression of .gz files produced by gzip?

while not EOF of .gz file:
   rawdata = read some megabytes from file
   decompressed = nvcomp inflate rawdata ???? 

[jayenashar]

i think it does, but i'm having trouble using the high-level API as it does a segfault during configure_decompression and i can't understand the low-level API used in https://github.com/NVIDIA/nvcomp/blob/main/examples/gzip_gpu_decompression.cu

[jayenashar]

it seems not.

i wrote some sample code to decompress gz files with nvcomp. it's about 50% slower than my cpu so it's only good if you have more files than cpu hyperthreads.

as you can't stream with nvcompBatchedGzipDecompressAsync you will need to pass in whole gz files

some other things i learned:

• nvcompBatchedGzipDecompressGetTempSize returns 0
• BatchDataCPU(const BatchData& batch_data, bool copy_data) is declared but not defined
• i seem to be able to allocate only 7.25GB of my 8GB GPU

[github-actions]

This issue has been labeled inactive-30d due to no recent activity in the past 30 days. Please close this issue if no further response or action is needed. Otherwise, please respond with a comment indicating any updates or changes to the original issue and/or confirm this issue still needs to be addressed. This issue will be labeled inactive-90d if there is no activity in the next 60 days.

[github-actions]

This issue has been labeled inactive-90d due to no recent activity in the past 90 days. Please close this issue if no further response or action is needed. Otherwise, please respond with a comment indicating any updates or changes to the original issue and/or confirm this issue still needs to be addressed.