How to publish a self-contained, single runtime, with multiple backends?

[jasonliv]

Description

This may be a bug or a feature request, I'm not sure.

I'm building a project for Windows which needs to support CPU and CUDA backends. In .csproj I have:

    <PackageReference Include="LlamaSharp" Version="0.19.0" />
    <PackageReference Include="LlamaSharp.Backend.Cpu" Version="0.19.0" />
    <PackageReference Include="LlamaSharp.Backend.Cuda12.Windows" Version="0.19.0" />

I need to publish a self-contained redistributable, so I do the following:

dotnet publish --self-contained -c:Release -r win-x64

I get "error NETSDK1152: Found multiple publish output files with the same relative path" because it's trying to copy avx\ggml.dll, avx2\ggml.dll, avx512\ggml.dll, etc. to the same publish directory.

I can add this to the .csproj file to build without the error:

  <PropertyGroup>
    <ErrorOnDuplicatePublishOutputFiles>false</ErrorOnDuplicatePublishOutputFiles>
  </PropertyGroup>

But then you end up with only one ggml.dll, one llama.dll, etc. and it's not clear or predictable which one you end up getting.

I noticed that whisper.net, a very similar project which wraps whisper.cpp, uses a different solution. It keeps the runtime libraries in separate folders even after the publish step, so this conflict doesn't occur. I.e. you end up with:

• publish\
  • program.exe
  • runtimes\
    • cuda\
      • win-x64\
        • ggml.dll
    • win-x64\
      • ggml.dll

etc.

If there isn't some setting or feature I'm missing, maybe it is necessary to take a similar approach to what they're doing?

Thanks.

[martindevans]

Unfortunately this is a known issue with the way the packages are built at the moment - for some reason the final publish step flattens the hierarchy and you get the error about duplicate files. Ideally the files need to be kept in the correct hierarchy (as you suggested). Would you be interested in working on a PR for this?

[jasonliv]

@martindevans Sorry for the late reply on this, and happy new year.

What I ended up doing temporarily is taking the runtime references out of my project and instead, manually copying the runtime libs to my publish output like the above structure, but that still didn't work. I also had to make a code change in NativeLibraryUtils.cs TryFindPath:

I think this change is backwards compatible with using the runtime package references, but I'm not 100% sure. If I have time I'll investigate this more.

[martindevans]

I would make that change a couple of lines up, so you have:

var candidate = Path.GetFullPath(Path.Combine(path, filename));

[jasonliv]

Will this change still work with your changes planned for the January update? If so I can make a PR. Thanks.

[martindevans]

I think it should yes - for the next update we're making as few changes to the loading system as possible.