Bundle pip's console script with PyInstaller

[BLKSerene]

Hi, I'm trying to bundle pip's console script so that 3rd-party libraries could be installed dynamically. Could anyone give some suggestions?

test_pip.py:

import os
import sys

import pip

pip.main(['install', '--target', os.path.realpath(sys._MEIPASS), 'six'])

test_pip.spec:

# -*- mode: python ; coding: utf-8 -*-
from PyInstaller.utils.hooks import collect_data_files

datas = []
datas += collect_data_files('pip', include_py_files = True)

a = Analysis(
    ['test_pip.py'],
    pathex=[],
    binaries=[],
    datas=datas,
    hiddenimports=[],
    hookspath=[],
    hooksconfig={},
    runtime_hooks=[],
    excludes=[],
    noarchive=False,
    optimize=0,
)
pyz = PYZ(a.pure)

exe = EXE(
    pyz,
    a.scripts,
    [],
    exclude_binaries=True,
    name='test_pip',
    debug=False,
    bootloader_ignore_signals=False,
    strip=False,
    upx=True,
    console=True,
    disable_windowed_traceback=False,
    argv_emulation=False,
    target_arch=None,
    codesign_identity=None,
    entitlements_file=None,
)
coll = COLLECT(
    exe,
    a.binaries,
    a.datas,
    strip=False,
    upx=True,
    upx_exclude=[],
    name='test_pip',
)

Error:

Traceback (most recent call last):
  File "test_pip.py", line 6, in <module>
  File "pip\__init__.py", line 13, in main
  File "pip\_internal\utils\entrypoints.py", line 43, in _wrapper
  File "pip\_internal\cli\main.py", line 78, in main
  File "pip\_internal\commands\__init__.py", line 114, in create_command
  File "importlib\__init__.py", line 126, in import_module
  File "<frozen importlib._bootstrap>", line 1204, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1176, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1147, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 690, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 940, in exec_module
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "G:\dist\test_pip\_internal\pip\_internal\commands\install.py", line 15, in <module>
    from pip._internal.cli.req_command import (
  File "G:\dist\test_pip\_internal\pip\_internal\cli\req_command.py", line 21, in <module>
    from pip._internal.index.package_finder import PackageFinder
  File "<frozen importlib._bootstrap>", line 1176, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1147, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 690, in _load_unlocked
  File "PyInstaller\loader\pyimod02_importers.py", line 419, in exec_module
  File "pip\_internal\index\package_finder.py", line 31, in <module>
  File "<frozen importlib._bootstrap>", line 1176, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1147, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 690, in _load_unlocked
  File "PyInstaller\loader\pyimod02_importers.py", line 419, in exec_module
  File "pip\_internal\req\__init__.py", line 9, in <module>
  File "<frozen importlib._bootstrap>", line 1176, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1147, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 690, in _load_unlocked
  File "PyInstaller\loader\pyimod02_importers.py", line 419, in exec_module
  File "pip\_internal\req\req_install.py", line 40, in <module>
  File "<frozen importlib._bootstrap>", line 1176, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1147, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 690, in _load_unlocked
  File "PyInstaller\loader\pyimod02_importers.py", line 419, in exec_module
  File "pip\_internal\operations\install\wheel.py", line 40, in <module>
  File "<frozen importlib._bootstrap>", line 1176, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1147, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 690, in _load_unlocked
  File "PyInstaller\loader\pyimod02_importers.py", line 419, in exec_module
  File "pip\_vendor\distlib\scripts.py", line 66, in <module>
  File "pip\_vendor\distlib\resources.py", line 332, in finder
pip._vendor.distlib.DistlibException: Unable to locate finder for 'pip._vendor.distlib'
[11460] Failed to execute script 'test_pip' due to unhandled exception!

OS: Windows 11 x64
Python: 3.11.9
PyInstaller: 6.8.0
pip: 24.1.1

[rokm]

Hi, I'm trying to bundle pip's console script so that 3rd-party libraries could be installed dynamically. Could anyone give some suggestions?

Don't.

[bwoodsend]

If you want a complete Python environment that you can install packages into then you want nothing to do with PyInstaller.

[BLKSerene]

Hi, in my specific case, I want to use pip to install language models (provided as Python packages) of spaCy into the frozen app. All of these models have prebuilt wheels. All model files are binaries except the __init__.py file and are platform-independent. No scripts need to be installed and no Python files need to be compiled.

Actually a simple copy-and-paste of, for example, en_core_web_trf and en_core_web_trf-3.7.3.dist-info, from any local Python distribution into the frozen app would work (tested), and I have once managed to bundle pip with Python 3.10.11, pip 23.2.1, and PyInstaller 6.0 (still figuring out why this does not work now), so I only want pip to do that.

[rokm]

By design, frozen applications are supposed to be immutable (they're called "frozen" for a reason). In some situations, that is even enforced either by OS (e.g., macOS .app bundles are signed, even if ad-hoc) or by the fact that they end up installed in read-only locations.

We obviously cannot prevent you from hacking together a solution that "works" (for the situation-specific definition of "works"), but that does not change our suggestions.

[BLKSerene]

That is quite reasonable. I found that pip is not necessary in my case, requests and the built-in zipfile suffice, so the solution is getting rid of pip altogether.