java.io.FileNotFoundException: https://publish.djl.ai/pytorch/1.9.0/files.txt

[Hyodare]

Hello,

I am worrking on a projects and need use a pytorch model but loading it give me the folowing error:

Exception in thread "main" ai.djl.engine.EngineException: Failed to save pytorch index file
        at ai.djl.pytorch.jni.LibUtils.downloadPyTorch(LibUtils.java:396)
        at ai.djl.pytorch.jni.LibUtils.findNativeLibrary(LibUtils.java:286)
        at ai.djl.pytorch.jni.LibUtils.getLibTorch(LibUtils.java:89)
        at ai.djl.pytorch.jni.LibUtils.loadLibrary(LibUtils.java:77)
        at ai.djl.pytorch.engine.PtEngine.newInstance(PtEngine.java:53)
        at ai.djl.pytorch.engine.PtEngineProvider.getEngine(PtEngineProvider.java:40)
        at ai.djl.engine.Engine.getEngine(Engine.java:187)
        at ai.djl.Model.newInstance(Model.java:99)
        at ai.djl.repository.zoo.BaseModelLoader.createModel(BaseModelLoader.java:191)
        at ai.djl.repository.zoo.BaseModelLoader.loadModel(BaseModelLoader.java:154)
        at ai.djl.repository.zoo.Criteria.loadModel(Criteria.java:172)
        at com.alten.sophia.lab.u_net.SegmentationInference.initModel(SegmentationInference.java:140)
        at com.alten.sophia.lab.u_net.SegmentationInference.<init>(SegmentationInference.java:52)
        at com.alten.sophia.lab.Filters.SegmentationPipeline.<init>(SegmentationPipeline.java:22)
        at com.alten.sophia.lab.Execution.BasicExecuter.parsePrePro(BasicExecuter.java:41)
        at com.alten.sophia.lab.Execution.BasicExecuter.<init>(BasicExecuter.java:30)
        at com.alten.sophia.lab.Execution.CommandLine.readCommandLine(CommandLine.java:59)
        at com.alten.ocr.evaluationv2.Launcher.main(Launcher.java:18)
Caused by: java.io.FileNotFoundException: https://publish.djl.ai/pytorch/1.9.0/files.txt
        at java.base/sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1976)
        at java.base/sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1581)
        at java.base/sun.net.www.protocol.https.HttpsURLConnectionImpl.getInputStream(HttpsURLConnectionImpl.java:224)
        at java.base/java.net.URL.openStream(URL.java:1317)
        at ai.djl.util.Utils.openUrl(Utils.java:463)
        at ai.djl.util.Utils.openUrl(Utils.java:447)
        at ai.djl.pytorch.jni.LibUtils.downloadPyTorch(LibUtils.java:391)
        ... 17 more

The code I use to load the model from a zip in the jar:

Criteria<Image,Image> crit=Criteria.builder()
				.setTypes(Image.class, Image.class)
				.optTranslator(new bigTranslator())
				.optModelUrls("jar:///models/CP_epoch20.zip")
				.optEngine("PyTorch")
				.optModelName(modelName)
				.build();

model=crit.loadModel();

I tried locally from the IDE without building the jar but got the same error:

Path p=Paths.get(this.getClass().getClassLoader().getResource("models/CP_epoch20.zip").toURI());
Criteria<Image,Image> crit=Criteria.builder()
				.setTypes(Image.class, Image.class)
				.optTranslator(new bigTranslator())
				.optModelPath(p)
				.optEngine("PyTorch")
				.optModelName(modelName)
				.build();

model=crit.loadModel();

I also tried with just the .pt (without zip) file and got the same error.

relevant maven imports:

		<!-- https://mvnrepository.com/artifact/ai.djl.pytorch/pytorch-engine -->
		<dependency>
			<groupId>ai.djl.pytorch</groupId>
			<artifactId>pytorch-engine</artifactId>
			<version>0.22.1</version>
			<scope>runtime</scope>
		</dependency>

		<dependency>
			<groupId>ai.djl.pytorch</groupId>
			<artifactId>pytorch-native-auto</artifactId>
			<scope>runtime</scope>
		</dependency>


		<!-- https://mvnrepository.com/artifact/ai.djl/api -->
		<dependency>
			<groupId>ai.djl</groupId>
			<artifactId>api</artifactId>
			<version>0.22.1</version>
		</dependency>

I looked online and in the discussions but found no working solution.

Thank you for your help.

[frankfliu]

Please remove pytorch-native-auto package from the dependency. It has been deprecated a while.

[Hyodare]

removed the package now I have this error

Exception in thread "main" ai.djl.engine.EngineException: Failed to load PyTorch native library
        at ai.djl.pytorch.engine.PtEngine.newInstance(PtEngine.java:90)
        at ai.djl.pytorch.engine.PtEngineProvider.getEngine(PtEngineProvider.java:40)
        at ai.djl.engine.Engine.getEngine(Engine.java:187)
        at ai.djl.Model.newInstance(Model.java:99)
        at ai.djl.repository.zoo.BaseModelLoader.createModel(BaseModelLoader.java:191)
        at ai.djl.repository.zoo.BaseModelLoader.loadModel(BaseModelLoader.java:154)
        at ai.djl.repository.zoo.Criteria.loadModel(Criteria.java:172)
        at com.alten.sophia.lab.u_net.SegmentationInference.initModel(SegmentationInference.java:140)
        at com.alten.sophia.lab.u_net.SegmentationInference.<init>(SegmentationInference.java:52)
        at com.alten.sophia.lab.Filters.SegmentationPipeline.<init>(SegmentationPipeline.java:22)
        at com.alten.sophia.lab.Execution.BasicExecuter.parsePrePro(BasicExecuter.java:41)
        at com.alten.sophia.lab.Execution.BasicExecuter.<init>(BasicExecuter.java:30)
        at com.alten.sophia.lab.Execution.CommandLine.readCommandLine(CommandLine.java:59)
        at com.alten.ocr.evaluationv2.Launcher.main(Launcher.java:18)
Caused by: java.lang.StringIndexOutOfBoundsException: Range [2, 5) out of bounds for length 4
        at java.base/jdk.internal.util.Preconditions$1.apply(Preconditions.java:55)
        at java.base/jdk.internal.util.Preconditions$1.apply(Preconditions.java:52)
        at java.base/jdk.internal.util.Preconditions$4.apply(Preconditions.java:213)
        at java.base/jdk.internal.util.Preconditions$4.apply(Preconditions.java:210)
        at java.base/jdk.internal.util.Preconditions.outOfBounds(Preconditions.java:98)
        at java.base/jdk.internal.util.Preconditions.outOfBoundsCheckFromToIndex(Preconditions.java:112)
        at java.base/jdk.internal.util.Preconditions.checkFromToIndex(Preconditions.java:349)
        at java.base/java.lang.String.checkBoundsBeginEnd(String.java:4608)
        at java.base/java.lang.String.substring(String.java:2720)
        at ai.djl.pytorch.jni.LibUtils.downloadPyTorch(LibUtils.java:407)
        at ai.djl.pytorch.jni.LibUtils.findNativeLibrary(LibUtils.java:286)
        at ai.djl.pytorch.jni.LibUtils.getLibTorch(LibUtils.java:89)
        at ai.djl.pytorch.jni.LibUtils.loadLibrary(LibUtils.java:77)
        at ai.djl.pytorch.engine.PtEngine.newInstance(PtEngine.java:53)
        ... 13 more

same eror when used locally.

[frankfliu]

It looks like you are using a very old version of CUDA (9.x).

This is a bug in DJL, which we just fixed, see: #2581.

A temporary workaround could be set env var:

export PYTORCH_FLAVOR=cpu

[Hyodare]

I just ran nvidia-smi command wich told me I have 12.1 cuda version

[frankfliu]

You might have multiple CUDA installed. DJL use libcudart.so to detect cuda version. If you have a 9.x libcudart.so in path ahead of 12.1, DJL find 9.x first.

[Hyodare]

after going througth the docs i noticed i was missing some dependencies for the python engine (https://docs.djl.ai/master/engines/pytorch/pytorch-engine/index.html) so i added :

 		<dependency>
			<groupId>ai.djl.pytorch</groupId>
			<artifactId>pytorch-native-cpu</artifactId>
			<classifier>win-x86_64</classifier>
			<scope>runtime</scope>
			<version>2.0.0</version>
		</dependency>
		<dependency>
			<groupId>ai.djl.pytorch</groupId>
			<artifactId>pytorch-jni</artifactId>
			<version>2.0.0-0.22.1</version>
			<scope>runtime</scope>
		</dependency>

now the error is:

Exception in thread "main" ai.djl.engine.EngineException: Could not run 'aten::empty_strided' with arguments from the 'CUDA' backend. This could be because the operator doesn't exist for this backend, or was omitted during the selective/custom build process (if using custom build). If you are a Facebook employee using PyTorch on mobile, please visit https://fburl.com/ptmfixes for possible resolutions. 'aten::empty_strided' is only available for these backends: [CPU, Meta, QuantizedCPU, BackendSelect, Python, FuncTorchDynamicLayerBackMode, Functionalize, Named, Conjugate, Negative, ZeroTensor, ADInplaceOrView, AutogradOther, AutogradCPU, AutogradCUDA, AutogradHIP, AutogradXLA, AutogradMPS, AutogradIPU, AutogradXPU, AutogradHPU, AutogradVE, AutogradLazy, AutogradMeta, AutogradMTIA, AutogradPrivateUse1, AutogradPrivateUse2, AutogradPrivateUse3, AutogradNestedTensor, Tracer, AutocastCPU, AutocastCUDA, FuncTorchBatched, FuncTorchVmapMode, Batched, VmapMode, FuncTorchGradWrapper, PythonTLSSnapshot, FuncTorchDynamicLayerFrontMode, PythonDispatcher].

CPU: registered at C:\actions-runner\_work\pytorch\pytorch\builder\windows\pytorch\build\build\aten\src\ATen\RegisterCPU.cpp:31034 [kernel]
Meta: registered at C:\actions-runner\_work\pytorch\pytorch\builder\windows\pytorch\build\build\aten\src\ATen\RegisterMeta.cpp:26824 [kernel]
QuantizedCPU: registered at C:\actions-runner\_work\pytorch\pytorch\builder\windows\pytorch\build\build\aten\src\ATen\RegisterQuantizedCPU.cpp:929 [kernel]
BackendSelect: registered at C:\actions-runner\_work\pytorch\pytorch\builder\windows\pytorch\build\build\aten\src\ATen\RegisterBackendSelect.cpp:726 [kernel]
Python: registered at C:\actions-runner\_work\pytorch\pytorch\builder\windows\pytorch\aten\src\ATen\core\PythonFallbackKernel.cpp:144 [backend fallback]
FuncTorchDynamicLayerBackMode: registered at C:\actions-runner\_work\pytorch\pytorch\builder\windows\pytorch\aten\src\ATen\functorch\DynamicLayer.cpp:491 [backend fallback]
Functionalize: registered at C:\actions-runner\_work\pytorch\pytorch\builder\windows\pytorch\aten\src\ATen\FunctionalizeFallbackKernel.cpp:280 [backend fallback]
Named: registered at C:\actions-runner\_work\pytorch\pytorch\builder\windows\pytorch\aten\src\ATen\core\NamedRegistrations.cpp:7 [backend fallback]
Conjugate: fallthrough registered at C:\actions-runner\_work\pytorch\pytorch\builder\windows\pytorch\aten\src\ATen\ConjugateFallback.cpp:21 [kernel]
Negative: fallthrough registered at C:\actions-runner\_work\pytorch\pytorch\builder\windows\pytorch\aten\src\ATen\native\NegateFallback.cpp:23 [kernel]
ZeroTensor: fallthrough registered at C:\actions-runner\_work\pytorch\pytorch\builder\windows\pytorch\aten\src\ATen\ZeroTensorFallback.cpp:90 [kernel]
ADInplaceOrView: fallthrough registered at C:\actions-runner\_work\pytorch\pytorch\builder\windows\pytorch\aten\src\ATen\core\VariableFallbackKernel.cpp:63 [backend fallback]
AutogradOther: registered at C:\actions-runner\_work\pytorch\pytorch\builder\windows\pytorch\torch\csrc\autograd\generated\VariableType_2.cpp:17488 [autograd kernel]
AutogradCPU: registered at C:\actions-runner\_work\pytorch\pytorch\builder\windows\pytorch\torch\csrc\autograd\generated\VariableType_2.cpp:17488 [autograd kernel]
AutogradCUDA: registered at C:\actions-runner\_work\pytorch\pytorch\builder\windows\pytorch\torch\csrc\autograd\generated\VariableType_2.cpp:17488 [autograd kernel]
AutogradHIP: registered at C:\actions-runner\_work\pytorch\pytorch\builder\windows\pytorch\torch\csrc\autograd\generated\VariableType_2.cpp:17488 [autograd kernel]
AutogradXLA: registered at C:\actions-runner\_work\pytorch\pytorch\builder\windows\pytorch\torch\csrc\autograd\generated\VariableType_2.cpp:17488 [autograd kernel]
AutogradMPS: registered at C:\actions-runner\_work\pytorch\pytorch\builder\windows\pytorch\torch\csrc\autograd\generated\VariableType_2.cpp:17488 [autograd kernel]
AutogradIPU: registered at C:\actions-runner\_work\pytorch\pytorch\builder\windows\pytorch\torch\csrc\autograd\generated\VariableType_2.cpp:17488 [autograd kernel]
AutogradXPU: registered at C:\actions-runner\_work\pytorch\pytorch\builder\windows\pytorch\torch\csrc\autograd\generated\VariableType_2.cpp:17488 [autograd kernel]
AutogradHPU: registered at C:\actions-runner\_work\pytorch\pytorch\builder\windows\pytorch\torch\csrc\autograd\generated\VariableType_2.cpp:17488 [autograd kernel]
AutogradVE: registered at C:\actions-runner\_work\pytorch\pytorch\builder\windows\pytorch\torch\csrc\autograd\generated\VariableType_2.cpp:17488 [autograd kernel]
AutogradLazy: registered at C:\actions-runner\_work\pytorch\pytorch\builder\windows\pytorch\torch\csrc\autograd\generated\VariableType_2.cpp:17488 [autograd kernel]
AutogradMeta: registered at C:\actions-runner\_work\pytorch\pytorch\builder\windows\pytorch\torch\csrc\autograd\generated\VariableType_2.cpp:17488 [autograd kernel]
AutogradMTIA: registered at C:\actions-runner\_work\pytorch\pytorch\builder\windows\pytorch\torch\csrc\autograd\generated\VariableType_2.cpp:17488 [autograd kernel]
AutogradPrivateUse1: registered at C:\actions-runner\_work\pytorch\pytorch\builder\windows\pytorch\torch\csrc\autograd\generated\VariableType_2.cpp:17488 [autograd kernel]
AutogradPrivateUse2: registered at C:\actions-runner\_work\pytorch\pytorch\builder\windows\pytorch\torch\csrc\autograd\generated\VariableType_2.cpp:17488 [autograd kernel]
AutogradPrivateUse3: registered at C:\actions-runner\_work\pytorch\pytorch\builder\windows\pytorch\torch\csrc\autograd\generated\VariableType_2.cpp:17488 [autograd kernel]
AutogradNestedTensor: registered at C:\actions-runner\_work\pytorch\pytorch\builder\windows\pytorch\torch\csrc\autograd\generated\VariableType_2.cpp:17488 [autograd kernel]
Tracer: registered at C:\actions-runner\_work\pytorch\pytorch\builder\windows\pytorch\torch\csrc\autograd\generated\TraceType_2.cpp:16726 [kernel]
AutocastCPU: fallthrough registered at C:\actions-runner\_work\pytorch\pytorch\builder\windows\pytorch\aten\src\ATen\autocast_mode.cpp:487 [backend fallback]
AutocastCUDA: fallthrough registered at C:\actions-runner\_work\pytorch\pytorch\builder\windows\pytorch\aten\src\ATen\autocast_mode.cpp:354 [backend fallback]
FuncTorchBatched: registered at C:\actions-runner\_work\pytorch\pytorch\builder\windows\pytorch\aten\src\ATen\functorch\LegacyBatchingRegistrations.cpp:815 [backend fallback]
FuncTorchVmapMode: fallthrough registered at C:\actions-runner\_work\pytorch\pytorch\builder\windows\pytorch\aten\src\ATen\functorch\VmapModeRegistrations.cpp:28 [backend fallback]
Batched: registered at C:\actions-runner\_work\pytorch\pytorch\builder\windows\pytorch\aten\src\ATen\LegacyBatchingRegistrations.cpp:1073 [backend fallback]
VmapMode: fallthrough registered at C:\actions-runner\_work\pytorch\pytorch\builder\windows\pytorch\aten\src\ATen\VmapModeRegistrations.cpp:33 [backend fallback]
FuncTorchGradWrapper: registered at C:\actions-runner\_work\pytorch\pytorch\builder\windows\pytorch\aten\src\ATen\functorch\TensorWrapper.cpp:210 [backend fallback]
PythonTLSSnapshot: registered at C:\actions-runner\_work\pytorch\pytorch\builder\windows\pytorch\aten\src\ATen\core\PythonFallbackKernel.cpp:152 [backend fallback]
FuncTorchDynamicLayerFrontMode: registered at C:\actions-runner\_work\pytorch\pytorch\builder\windows\pytorch\aten\src\ATen\functorch\DynamicLayer.cpp:487 [backend fallback]
PythonDispatcher: registered at C:\actions-runner\_work\pytorch\pytorch\builder\windows\pytorch\aten\src\ATen\core\PythonFallbackKernel.cpp:148 [backend fallback]

        at ai.djl.pytorch.jni.PyTorchLibrary.moduleLoad(Native Method)
        at ai.djl.pytorch.jni.JniUtils.loadModule(JniUtils.java:1700)
        at ai.djl.pytorch.engine.PtModel.load(PtModel.java:92)
        at ai.djl.repository.zoo.BaseModelLoader.loadModel(BaseModelLoader.java:161)
        at ai.djl.repository.zoo.Criteria.loadModel(Criteria.java:172)
        at com.alten.sophia.lab.u_net.SegmentationInference.initModel(SegmentationInference.java:141)
        at com.alten.sophia.lab.u_net.SegmentationInference.<init>(SegmentationInference.java:52)
        at com.alten.sophia.lab.Filters.SegmentationPipeline.<init>(SegmentationPipeline.java:22)
        at com.alten.sophia.lab.Execution.BasicExecuter.parsePrePro(BasicExecuter.java:41)
        at com.alten.sophia.lab.Execution.BasicExecuter.<init>(BasicExecuter.java:30)
        at com.alten.sophia.lab.Execution.CommandLine.readCommandLine(CommandLine.java:59)
        at com.alten.ocr.evaluationv2.Launcher.main(Launcher.java:18)

is this a related issue or am i doing shit?
(i tried the gpu version but it makes my file 2GB and still doesnt work.)

[frankfliu]

It looks like you loaded a CPU version of PyTorch engine.

1. I don't recommend you use pytorch 2.0.0, please try 1.13.1
2. You still need to make sure your cudart64_*.dll point to 12.1, you can set CUDA_PATH env var

[Hyodare]

With cpu configuratioin i tried the 1.13.1 version and got the samre error.
I havent found any reference to another version of CUDA on my computer. The CUDA_PATH and CUDA_PATH_V12_1 were already set to the CUDA folder (also included in Path).

I tried the cu118 version :

      <dependency>
        <groupId>ai.djl.pytorch</groupId>
        <artifactId>pytorch-native-cu118</artifactId>
        <classifier>win-x86_64</classifier>
        <version>2.0.0</version>
        <scope>runtime</scope>
      </dependency>
      <dependency>
        <groupId>ai.djl.pytorch</groupId>
        <artifactId>pytorch-jni</artifactId>
        <version>2.0.0-0.22.1</version>
      </dependency>

and got this error:

[W C:\actions-runner\_work\pytorch\pytorch\builder\windows\pytorch\torch\csrc\jit\codegen\cuda\interface.cpp:47] Warning: Loading nvfuser library failed with: error in LoadLibrary for nvfuser_codegen.dll. WinError 126: Le module spÚcifiÚ est introuvable.
 (function LoadingNvfuserLibrary)
Exception in thread "main" ai.djl.engine.EngineException: Failed to load PyTorch native library
        at ai.djl.pytorch.engine.PtEngine.newInstance(PtEngine.java:90)
        at ai.djl.pytorch.engine.PtEngineProvider.getEngine(PtEngineProvider.java:40)
        at ai.djl.engine.Engine.getEngine(Engine.java:187)
        at ai.djl.Model.newInstance(Model.java:99)
        at ai.djl.repository.zoo.BaseModelLoader.createModel(BaseModelLoader.java:191)
        at ai.djl.repository.zoo.BaseModelLoader.loadModel(BaseModelLoader.java:154)
        at ai.djl.repository.zoo.Criteria.loadModel(Criteria.java:172)
        at com.example.SegmentationInference.initModel(SegmentationInference.java:139)
        at com.example.SegmentationInference.<init>(SegmentationInference.java:49)
        at com.example.App.main(App.java:12)
Caused by: java.lang.UnsatisfiedLinkError: C:\Users\Franc\.djl.ai\pytorch\2.0.0-20230411-cu118-win-x86_64\torch_cuda.dll: Can't find dependent libraries
        at java.base/jdk.internal.loader.NativeLibraries.load(Native Method)
        at java.base/jdk.internal.loader.NativeLibraries$NativeLibraryImpl.open(NativeLibraries.java:331)
        at java.base/jdk.internal.loader.NativeLibraries.loadLibrary(NativeLibraries.java:197)
        at java.base/jdk.internal.loader.NativeLibraries.loadLibrary(NativeLibraries.java:139)
        at java.base/java.lang.ClassLoader.loadLibrary(ClassLoader.java:2404)
        at java.base/java.lang.Runtime.load0(Runtime.java:817)
        at java.base/java.lang.System.load(System.java:2015)
        at ai.djl.pytorch.jni.LibUtils.loadNativeLibrary(LibUtils.java:347)
        at ai.djl.pytorch.jni.LibUtils.loadLibTorch(LibUtils.java:174)
        at ai.djl.pytorch.jni.LibUtils.loadLibrary(LibUtils.java:78)
        at ai.djl.pytorch.engine.PtEngine.newInstance(PtEngine.java:53)
        ... 9 more

I checked the torch_cuda .dll and it exists. I dont know what other librairie are missing or how to add them.

[frankfliu]

Can you try to downgrade to CUDA 11.7. and use pytorch-native-cu117:1.13.1:win-x86_64?

[Hyodare]

I installed CUDA 11.7, The CUDA_PATH now points to it I changes the dependencies to :

<dependency>
       <groupId>ai.djl.pytorch</groupId>
       <artifactId>pytorch-native-cu117</artifactId>
       <classifier>win-x86_64</classifier>
       <version>1.13.1</version>
       <scope>runtime</scope>
</dependency>
<dependency>
       <groupId>ai.djl.pytorch</groupId>
       <artifactId>pytorch-jni</artifactId>
       <version>1.13.1-0.22.1</version>
     </dependency>
 </dependencies>

and I still have the same error (except the warning : nvfuser librairie failed isn't there anymore // and of course the line with the version now points to 1.13.1-20221220-cu117-win-x86_64 ).