Unable to use sf.DatasetFeatures with zip files

[maxxfarma]

Description

sf.DatasetFeatures is not working for a zip file even though I export packages before running sf.DatasetFeatures.

%sh
export SF_BACKEND=torch
export SF_SLIDE_BACKEND=cucim
export SF_ALLOW_ZIP=0

OSError: [Errno 95] Operation not supported

This model is developed and licensed by Owkin, Inc. The license for use is
provided in the LICENSE file in the same directory as this source file
(slideflow/model/extractors/histossl/LICENSE), and is also available
at https://github.com/owkin/HistoSSLscaling. By using this feature extractor,
you agree to the terms of the license.

100%|██████████| 1.51G/1.51G [01:47<00:00, 14.1MB/s]

Exception ignored in: <function ZipFile.del at 0x7fe8e9f70790>

Traceback (most recent call last):
File "/usr/lib/python3.10/zipfile.py", line 1821, in del
self.close()
File "/usr/lib/python3.10/zipfile.py", line 1843, in close
self._fpclose(fp)
File "/usr/lib/python3.10/zipfile.py", line 1943, in _fpclose
fp.close()
OSError: [Errno 95] Operation not supported
WARNING:Comm:No such comm: 68ffee887ecd48ec936a9c270212d5be
WARNING:Comm:No such comm: 68ffee887ecd48ec936a9c270212d5be
WARNING:Comm:No such comm: 68ffee887ecd48ec936a9c270212d5be
WARNING:Comm:No such comm: 68ffee887ecd48ec936a9c270212d5be
WARNING:Comm:No such comm: 68ffee887ecd48ec936a9c270212d5be
WARNING:Comm:No such comm: 68ffee887ecd48ec936a9c270212d5be
WARNING:Comm:No such comm: 68ffee887ecd48ec936a9c270212d5be
WARNING:Comm:No such comm: 68ffee887ecd48ec936a9c270212d5be
WARNING:Comm:No such comm: 68ffee887ecd48ec936a9c270212d5be
WARNING:Comm:No such comm: 68ffee887ecd48ec936a9c270212d5be
WARNING:Comm:No such comm: 68ffee887ecd48ec936a9c270212d5be
WARNING:Comm:No such comm: 68ffee887ecd48ec936a9c270212d5be
WARNING:Comm:No such comm: 68ffee887ecd48ec936a9c270212d5be
WARNING:Comm:No such comm: 68ffee887ecd48ec936a9c270212d5be
WARNING:Comm:No such comm: 68ffee887ecd48ec936a9c270212d5be
WARNING:Comm:No such comm: 68ffee887ecd48ec936a9c270212d5be
WARNING:Comm:No such comm: 68ffee887ecd48ec936a9c270212d5be
WARNING:Comm:No such comm: 68ffee887ecd48ec936a9c270212d5be

Also tried with export SF_ALLOW_ZIP = 1 and got this error:

<function ZipFile.del at 0x7f176ee2e290> Traceback (most recent call last): File "/usr/lib/python3.10/zipfile.py", line 1821, in del self.close() File "/usr/lib/python3.10/zipfile.py", line 1843, in close self._fpclose(fp) File "/usr/lib/python3.10/zipfile.py", line 1943, in _fpclose fp.close() OSError: [Errno 95] Operation not supported WARNING:Comm:No such comm: 3827dbdfa1134711ae1408ad6d42c629 WARNING:Comm:No such comm: 3827dbdfa1134711ae1408ad6d42c629 WARNING:Comm:No such comm: 3827dbdfa1134711ae1408ad6d42c629 WARNING:Comm:No such comm: 3827dbdfa1134711ae1408ad6d42c629 Exception ignored in: Exception ignored in: WARNING:Comm:No such comm: 3827dbdfa1134711ae1408ad6d42c629 <function ZipFile.del at 0x7f176ee2e290> WARNING:Comm:No such comm: 3827dbdfa1134711ae1408ad6d42c629 <function ZipFile.del at 0x7f176ee2e290>WARNING:Comm:No such comm: 3827dbdfa1134711ae1408ad6d42c629 Traceback (most recent call last): WARNING:Comm:No such comm: 3827dbdfa1134711ae1408ad6d42c629 File "/usr/lib/python3.10/zipfile.py", line 1821, in del Traceback (most recent call last): WARNING:Comm:No such comm: 3827dbdfa1134711ae1408ad6d42c629 File "/usr/lib/python3.10/zipfile.py", line 1821, in del WARNING:Comm:No such comm: 3827dbdfa1134711ae1408ad6d42c629 WARNING:Comm:No such comm: 3827dbdfa1134711ae1408ad6d42c629 WARNING:Comm:No such comm: 3827dbdfa1134711ae1408ad6d42c629 self.close()self.close()WARNING:Comm:No such comm: 3827dbdfa1134711ae1408ad6d42c629 File "/usr/lib/python3.10/zipfile.py", line 1843, in close

OSError: [Errno 95] Operation not supported

Expected behavior

This method should extract features for a deep learning algorithm and has worked on zip files in the past.

Environment:

On databricks

• Slideflow Version (e.g., 1.0): 2.3.0
• OS (e.g., Ubuntu): Windows 11
• How you installed Slideflow (pip, source): PyPl

Additional context

Using zip files on this method in the past has worked with earlier versions of slideflow.

[Meijian]

@jamesdolezal Hi James, we are having a similar zip file issue in 2.3.0 to what we had previously with older versions of Slideflow when using it in Data Bricks. You helped address the issue by setting the environmental variable "export SF_ALLOW_ZIP=1". But now this command seems not working anymore. Could you please help address this? Thanks!