sys.executable generates error if a white space is present in the python executable path

[AndreaBartolini]

Some month ago the use of sys.executable function has been introduced in the test.py script in order to get the current python executable path. I suppose that this has be done to make more general the script because some python installations have different commands to start them (e.g., python3 or similar).

This solution generates issues if some white spaces are present in the python path executable, for example on my windows machine the python executable path is the following:

C:\Program Files\Python311\python.exe

as, I think, in the most of the windows installations.

The output from the sys.executable function is the following:
'C:\\Program Files\\Python311\\python.exe'

By using this path the command generated to run the testmodel.py script is like the following:
cmd = C:\Program Files\Python311\python.exe testmodel.py --win --msysEnvironment=ucrt64 --libraries=C:\LibTest\master --ompython_omhome= STTlibrary_STTlibrary.Components.DigitalController.Examples.ControlledTank.conf.json > files/STTlibrary_STTlibrary.Components.DigitalController.Examples.ControlledTank.cmdout 2>&1, STTlibrary_STTlibrary.Components.DigitalController.Examples.ControlledTank, 300

that generates the following error on the subprocess call:

'C:\Program' is not recognized as an internal or external command, operable program or batch file.

That means that the shell does not resolve correctly the python executable path.

I tried to modify the python executable path as following:

'C:\\\"Program Files\"\\Python311\\python.exe'

that works fine if used in the win terminal as:
'C:\"Program Files"\Python311\python.exe

but in this case i get the following error

run: check_outputTraceback (most recent call last):
  File "C:\mnt\TempMyCI\LibCode\localopenmodelicalibrarytesting\OMLibraryTesting\test.py", line 184, in <module>
    check_output_log([pythonExecutable, "testmodel.py", "--help"], stderr=subprocess.STDOUT)
  File "C:\mnt\TempMyCI\LibCode\localopenmodelicalibrarytesting\OMLibraryTesting\test.py", line 101, in check_output_log
    return subprocess.check_output(*popenargs, **kwargs)

here below the popenargs and kwargs values:
(['C:\\"Program Files"\\Python311\\python.exe', 'testmodel.py', '--help'],) {'stderr': -2}

I tried with others combination of commas or backslash characters but the result doesn't change, so at the moment the unique solution that I can propose is the following change in the script

pythonExecutable = sys.executable
if not pythonExecutable or ' ' in pythonExecutable:
  pythonExecutable = "python"

that is not so elegant but works...

Any suggestion?

[AndreaBartolini]

to add @casella in the loop

[bilderbuchi]

cmd = "C:\Program Files\Python311\python.exe" testmodel.py... (i.e. surrounding the path-with-spaces with double quotes) does not help? In python you can nest quotes, so '"C:\Program Files\Python311\python.exe" testmodel.py...' can be used to achieve this.

[AndreaBartolini]

Hi Christoph, I tried the suggestion but the check_output fails:

run: check_output (['"C:\\Program Files\\Python311\\python.exe"', 'testmodel.py', '--help'],) {'stderr': -2}
Traceback (most recent call last):
  File "C:\Program Files\Python311\Lib\runpy.py", line 198, in _run_module_as_main
    return _run_code(code, main_globals, None,
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Program Files\Python311\Lib\runpy.py", line 88, in _run_code
    exec(code, run_globals)
  File "c:\program files\microsoft visual studio\2022\community\common7\ide\extensions\microsoft\python\core\debugpy\adapter/../..\debugpy\launcher/../..\debugpy\__main__.py", line 39, in <module>
    cli.main()
  File "c:\program files\microsoft visual studio\2022\community\common7\ide\extensions\microsoft\python\core\debugpy\adapter/../..\debugpy\launcher/../..\debugpy/..\debugpy\server\cli.py", line 430, in main
    run()
  File "c:\program files\microsoft visual studio\2022\community\common7\ide\extensions\microsoft\python\core\debugpy\adapter/../..\debugpy\launcher/../..\debugpy/..\debugpy\server\cli.py", line 284, in run_file
    runpy.run_path(target, run_name="__main__")
  File "c:\program files\microsoft visual studio\2022\community\common7\ide\extensions\microsoft\python\core\debugpy\_vendored\pydevd\_pydevd_bundle\pydevd_runpy.py", line 321, in run_path
    return _run_module_code(code, init_globals, run_name,
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "c:\program files\microsoft visual studio\2022\community\common7\ide\extensions\microsoft\python\core\debugpy\_vendored\pydevd\_pydevd_bundle\pydevd_runpy.py", line 135, in _run_module_code
    _run_code(code, mod_globals, init_globals,
  File "c:\program files\microsoft visual studio\2022\community\common7\ide\extensions\microsoft\python\core\debugpy\_vendored\pydevd\_pydevd_bundle\pydevd_runpy.py", line 124, in _run_code
    exec(code, run_globals)
  File "C:\mnt\TempMyCI\LibCode\localopenmodelicalibrarytesting\OMLibraryTesting\test.py", line 182, in <module>
    check_output_log([pythonExecutable, "testmodel.py", "--help"], stderr=subprocess.STDOUT)
  File "C:\mnt\TempMyCI\LibCode\localopenmodelicalibrarytesting\OMLibraryTesting\test.py", line 99, in check_output_log
    return subprocess.check_output(*popenargs, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Program Files\Python311\Lib\subprocess.py", line 466, in check_output
    return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Program Files\Python311\Lib\subprocess.py", line 548, in run
    with Popen(*popenargs, **kwargs) as process:
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Program Files\Python311\Lib\subprocess.py", line 1024, in __init__
    self._execute_child(args, executable, preexec_fn, close_fds,
  File "C:\Program Files\Python311\Lib\subprocess.py", line 1493, in _execute_child
    hp, ht, pid, tid = _winapi.CreateProcess(executable, args,
                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
PermissionError: [WinError 5] Access is denied
Press any key to continue . . . 

The difficult is to find a solution that works with both subprocess.check_output(*popenargs, **kwargs) and subprocess.Popen(cmd, shell=True, stdin=FNULL, stdout=FNULL, stderr=FNULL)

[AndreaBartolini]

Maybe I found a solution, we have to use two different path, the first for the subprocess.check_output(*popenargs, **kwargs) call that can contains blank characters, the second for the subprocess.Popen(...) call, in wich the parts of the path that contains blank characters are sorrounded by quotes.
I'll test the solution and then I'll prepare a PR about this.

[bilderbuchi]

TBH, I'm confused why popen is used at all, that's typically a bit low-level api, but I don't have a good grasp of the codebase.