Issue with PyInstaller Not Including Cryptodome.xxx~ Module

[Freerasinbread]

Hello,

I am encountering an issue with PyInstaller where it does not seem to include the Cryptodome.Random module from pycryptodomex in the final executable. I am using Windows 10, Python 3.11.2, and the most recent version of PyInstaller and pycryptodomex.

Environment
Operating System: Windows 10
Python Version: 3.11.2
PyInstaller Version: [latest version]
PyCryptodomex Version: [latest version]

Issue Description
I am trying to include from Cryptodome.Random import get_random_bytes in my packaged application. However, despite various attempts, the Cryptodome.Random module is not being found in the final executable. Whenever I build the .exe file I get a ModuleNotFoundError and therefore breaks.

The usual error looks something along the lines of:

Traceback (most recent call last):
  File "runner.py", line 5, in <module>
  File "PyInstaller\loader\pyimod02_importers.py", line 419, in exec_module
  File "internal_task_scheduler.py", line 14, in <module>
  File "C:\Users\**blocked**~1\AppData\Local\Temp\_MEI146762\utils\apis\auth_db_api_client.py", line 4, in <module>
    from utils.apis.api_requests import APIRequester
  File "C:\Users\**blocked**~1\AppData\Local\Temp\_MEI146762\utils\apis\api_requests.py", line 11, in <module>
    import utils.security as s
  File "C:\Users\**blocked**~1\AppData\Local\Temp\_MEI146762\utils\security.py", line 3, in <module>
    from Cryptodome.Random import get_random_bytes  # isort:skip
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ModuleNotFoundError: No module named 'Cryptodome.Random'
[10472] Failed to execute script 'runner' due to unhandled exception...

What I've Tried

• Using Only Crypto: Initially tried with Crypto, but faced similar issues.

• Switching to Cryptodome: Moved to Cryptodome to see if it resolved the issue, but it did not.

• Custom Hook for Cryptodome: Created a custom hook for Cryptodome, ensuring that Cryptodome.Random is included in binary_module_names.

The hook file looks like this:

import glob
import os

from PyInstaller.compat import EXTENSION_SUFFIXES
from PyInstaller.utils.hooks import get_module_file_attribute

# Include the modules as binaries in a subfolder named like the package.
# Cryptodome's loader expects to find them inside the package directory for
# the main module. We cannot use hiddenimports because that would add the
# modules outside the package.

binaries = []
binary_module_names = [
    "Cryptodome.Cipher",
    "Cryptodome.Hash",
    "Cryptodome.IO",
    "Cryptodome.Math",
    "Cryptodome.PublicKey",
    "Cryptodome.Random", # the original hook-Cryptodome.py file doesn't have the ~.Random included
    "Cryptodome.Util",
]


for module_name in binary_module_names:
    m_dir = os.path.dirname(get_module_file_attribute(module_name))
    for ext in EXTENSION_SUFFIXES:
        module_bin = glob.glob(os.path.join(m_dir, "_*%s" % ext))
        for f in module_bin:
            binaries.append((f, module_name.replace(".", "/")))

• Adding to hiddenimports: Added Cryptodome to hiddenimports in my PyInstaller script, which helped PyInstaller find the correct hook, but the Cryptodome.Random module is still missing in the build.

My PyInstaller script:

# -*- mode: python ; coding: utf-8 -*-

# pylint: disable = missing-module-docstring

from os.path import join as join

from PyInstaller.building.api import EXE, PYZ
from PyInstaller.building.build_main import Analysis
from PyInstaller.config import CONF

HOME_DIRECTORY = r"**blocked**" # hidden for privacy
UTILS = join(HOME_DIRECTORY, "utils")
SCRIPTS = join(HOME_DIRECTORY, "scripts")
WORKPATH = join(HOME_DIRECTORY, "build")

CONF["workpath"] = WORKPATH

a = Analysis(
    [join(SCRIPTS, "runner.py")],
    pathex=[HOME_DIRECTORY],
    binaries=[],
    datas=[(SCRIPTS, "scripts"), (UTILS, "utils")],
    hiddenimports=["httpx", "keyring", "Cryptodome"], # I have tried with and without the "Cryptodome" hidden import
    hookspath=[UTILS], # I have verified the hook-Cryptodome.py file exists
    hooksconfig={},
    runtime_hooks=[],
    excludes=[],
    noarchive=False,
)

exe = EXE(
    PYZ(a.pure),
    a.scripts,
    a.binaries,
    a.zipfiles,
    a.datas,
    [],
    name="api",
    debug=True,
    bootloader_ignore_signals=False,
    strip=False,
    upx=True,
    upx_exclude=[],
    runtime_tmpdir=None,
    console=True,
)

• Testing Outside PyInstaller: Running the script normally in Python works fine, and Cryptodome.Random is accessible.

I'm not sure why PyInstaller is failing to include this module in the final build. Has anyone else faced a similar issue, or does anyone have suggestions on what else I could try?

P.S. ~ I have tried with from Cryptodome.Cipher import AES, PKCS1_OAEP and get a similar error.

Thank you for your assistance!

[rokm]

datas=[(SCRIPTS, "scripts"), (UTILS, "utils")],

Only the python modules that are part of import tree generated from the entry-point script (runner.py) undergo import analysis.

This looks like you have a bunch of .py scripts (that runner.py presumably invokes somehow) in scripts and python code in utils that you smuggle into your frozen application as data files, meaning that no analysis is done on them. So you'll need to deal with all of their imports as hidden imports.

That means listing all required modules (e.g., hiddenimports=['Cryptodome.Random', 'Cryptodome.Hash', ...]) and not just top-level package names - adding top-level package name to hidden imports does not automatically collect the submodules and subpackages (unless there is a hook for that top-level package name that brute-force collects all submodules).

The hook you have is collecting only extensions, but you're missing extensions and python modules/sub-packages (for example, Cryptodome.Random is a subpackage, not an extension).

Try removing your custom hook, and use collect_submodules from PyInstaller.utils.hooks in your spec file to collect all submodules of Cryptodome:

...
from PyInstaller.utils.hooks import collect_submodules
...

a = Analysis(
    ...
    hiddenimports=["httpx", "keyring"] + collect_submodules("Cryptodome")
    ...
)

[Freerasinbread]

Thanks, works perfectly and saved headaches. Wish I knew about the collect_submodules() function earlier.