How to setup CSnakes to use existing Virtual Python env and Conda env (on Windows Server)

[Nick346-surf]

Hi,
I’m having issues to setup a Python env in my .NET process using either an existing virtual Python env or an existing Conda env.
In this scenario, I have a existing virtual python env and a Conda env with all packages required already installed.
This is running on Windows server with Python 3.12.
I’m using CSnakes 1.0.35

Using a virtual Python env (created via python.exe -m venv C:\Temp\PythonVirtualEnv/v312/csnakes_env):

var builder = Host.CreateApplicationBuilder(args);
var home = Path.Join(Environment.CurrentDirectory, "PythonFiles");
var virtual_python_env = "C:\\Temp\\PythonVirtualEnv\\v312\\csnakes_env";
builder.Services
.WithPython()
.WithHome(home)
.FromFolder(virtual_python_env, "3.12")
.WithVirtualEnvironment(virtual_python_env);

var app = builder.Build();
var env = app.Services.GetRequiredService<IPythonEnvironment>();

This produce following error:

Which is expected because this is a virtual env. Maybe I’m passing wrong argument or something
I would expect it would be able to load the python.dll from the main installation folder.

Using an existing Conda env:

var builder = Host.CreateApplicationBuilder(args);
var home = Path.Join(Environment.CurrentDirectory, "PythonFiles");
var conda = "C:\\ProgramData\\Anaconda3\\Scripts\\conda.exe"; // Conda folder
string pythonVersion = "3.12.9";
builder.Services
  .WithPython()
  .WithHome(home)
  .FromConda(conda)
  .WithCondaEnvironment("csnakes_env");
//.WithCondaEnvironment("csnakes_env", null, true, pythonVersion); // Also failing same way

var app = builder.Build();
var env = app.Services.GetRequiredService<IPythonEnvironment>();

It produces below error:

Sorry if this is a dumb question but I can't make it work. I'm missing something, i'm sure

It works fine with new python env that is downloaded from NuGet, or with existing python env (but not virtual env)

Also 2 more question regarding env:

1. Is there a way to recreate/recycle the created python env in the .NET process without restarting the process?
2. I know it’s not possible now but can we consider supporting multiple Python env inside same process in the future?
   Can be different env (Conda/Python/…) or a pool of python env to increase concurrency
   Or is there a hard no go on that because of technical reason?

Any reply will be much appreciated. Thanks a lot

[tonybaloney]

Ok, a few things to unpack here,

.FromFolder(virtual_python_env, "3.12") is for specialized Python builds on Windows and probably not what you want. If you downloaded Python from python.org, you want the Windows Installer Locator
If you installed it from the Windows Store, you want the Windows Store Locator.

Second, .WithVirtualEnvironment() has a second parameter to set whether or not to "ensure the virtual environment exists" (create it).

Set that to false:

.WithVirtualEnvironment(virtual_python_env, false)

[tonybaloney]

I'm not quite sure about the conda question on Windows, I'd need to try that out on a Windows machine

[Nick346-surf]

Thanks a lot for these feedbacks. It helped.
I tried to use the FromWindowsInstaller but that failed too. Reason is when i installed the Python from python.org, i did not installed it in default folder.
So i end up using FromFolder but passing the python installation folder:

  var builder = Host.CreateApplicationBuilder(args);
  var home = Path.Join(Environment.CurrentDirectory, "PythonFiles");
  var python_installation_folder = "C:\\Program Files\\Python\\Python312";
  var virtual_python_env = "C:\\Temp\\PythonVirtualEnv\\v312\\csnakes_env";
  builder.Services
   .WithPython()
   .WithHome(home)
   //.FromWindowsInstaller("3.12") // Look in C:\Program Files\Python\3.12
   .FromFolder(python_installation_folder, "3.12")
   .WithVirtualEnvironment(virtual_python_env, false);
  var app = builder.Build();

That worked fine. It does execute inside my virtual python env called csnakes_env. Thanks

[Nick346-surf]

Hi Anthony, Any feedback regarding using an already setup Conda env on Windows?
Thanks in advance

[tonybaloney]

Also 2 more question regarding env:

Is there a way to recreate/recycle the created python env in the .NET process without restarting the process?

You can reload a module, every source generated module has a Reload method that is triggered automatically on hot-reload in Visual Studio or you can call it manually.

If you're finding the environment creation slow with the pip installer, try the uv installer.

I know it’s not possible now but can we consider supporting multiple Python env inside same process in the future?
Can be different env (Conda/Python/…) or a pool of python env to increase concurrency
Or is there a hard no go on that because of technical reason?

The reason this is practically very difficult/impossible is because Python doesn't support multiple runtimes in the same process. Just the same as you can't load two versions of the same .NET assembly into the same process.

If it's concurrency you're after, we do support async (although not truly concurrent) functions. Most IO activities in Python actually release the GIL and so async is often a good solution.

We also support the experimental free-threaded Python builds, where the GIL is removed altogether. For those you could freely use them in Task pools or use something like parallel LINQ.