FIPS mode self check error

[MattTheCuber]

We use a Red Hat Enterprise Linux 8.5 machine with FIPS mode enabled. After installing any program (folder or one-file mode) and running it, we get the following error:

Error in GnuTLS initialization: Error while performing self checks.

Note, this is a non-critical bug as it does not seem to affect the usage of the program except the undesired terminal output. Googling the error will return a few posts, but they don't seem to have any solutions.

gnutls-cli version: 3.6.16

[rokm]

Hmm, can you give me a minimal reproducer that uses GnuTLS from python? Are you building under RHEL 8.5 as well, or under some other distribution? Also, what PyInstaller version are you using - gnutls seems to be using .hmac files, and we added collection of those in 6.4.0.

[MattTheCuber]

Great point, I kept seeing the error with everything I tried to compile, but I never tried to create a minimal example. I am using the latest pyinstaller version (6.9.0) and building and running on the same RHEL (8.5) system. Here were my findings in this process:

Issue 1

test.py:

import IPython

import comm

import ipykernel

import jupyter_client

import matplotlib

import matplotlib_inline

Compiling and running one of these returns the GnuTLS error originally stated.

$ pyinstaller test.py
...
$ ./dist/test/test
Error in GnuTLS initialization: Error while performing self checks.

Issue 2

test.py:

import hashlib

import OpenSSL

import parso

Compiling and running one of these returns:

$ pyinstaller test.py
...
$ ./dist/test/test
crypto/fips/fips.c:154: OpenSSL internal error: FATAL FIPS SELFTEST FAILURE
Aborted (core dumped)

What is strange is that it doesn't happen when thrid-party libraries import hashlib, openssl, or parso.

[rokm]

Can you check if .libname.hmac files are collected along with the shared libraries? I.e., assuming that libgnutls.so.XY is collected into frozen application, there should be an accompanying .libgnutls.so.XY.hmac file (note the starting dot) in the same directory. Same for the openssl libs.

[MattTheCuber]

For the import matplotlib program, the following files are collected:

dist/test/_internal/.libcrypto.so.1.1.hmac
dist/test/_internal/.libgcrypt.so.20.hmac
dist/test/_internal/.libgnutls.so.30.hmac
dist/test/_internal/.libhogweed.so.4.hmac
dist/test/_internal/.libnettle.so.6.hmac
dist/test/_internal/.libssl.so.1.1.hmac

For the import hashlib program, only the .libcrypto.so.1.1.hmac file is collected.

[rokm]

Hmm, last time time I was looking into FIPS mode (pyinstaller/pyinstaller#8273) collecting these seemed to suffice.

I'll try to set up a test RHEL8 system again.

[MattTheCuber]

If it is easier for you to explain debugging steps, I would be happy to help you test.

[rokm]

If it is easier for you to explain debugging steps, I would be happy to help you test.

I think it is, because I have no idea what to look for at the moment.

[rokm]

Can you tell me what python version you used, and how did you install it? (If I recall correctly, default is 3.6 which does not work with PyInstaller 6.9). And tested packages are installed from PyPI via pip?

[MattTheCuber]

$ python --version
Python 3.11.6

installed using the yum package manager

Packages are installed from PyPi using pip, yes.

[rokm]

Can you give me pip freeze list for the environment? The comm and matplotlib examples are not pulling in GnuTLS on my test system, so it might be pulled in via some optional dependency.

That said, hashlib example does pull in openssl libs and their hmac files, but it runs on my test system (admittedly, it is 8.10 with python 3.11.9, with FIPS enabled post-hoc via "sudo fips-mode-setup --enable && sudo reboot"). In my case, both libssl.so.1.1 and libcrypto.so.1.1 and their corresponding hmac files are collected. And if I remove either hmac file, I get

crypto/fips/fips.c:154: OpenSSL internal error: FATAL FIPS SELFTEST FAILURE
Aborted (core dumped)

[rokm]

Hmm, let's focus on hashlib and OpenSSL first. If your hashlib build did not collect libssl.so.1.1 and its hmac file (I see that _hashlib extension is, indeed, linked only against libcrypto.so.1.1), can you try to either:

• use --add-binary /usr/lib64/libssl.so.1.1:. to ensure that it is collected
• remove the bundled libcrypto.so.1.1 and try running the application

Could be that issue is that one lib is bundled and the other is not.

[rokm]

Could be that issue is that one lib is bundled and the other is not.

Yep, that seems to be the case on my system; the libssl.so.1.1.so is pulled in by python's _blake2 extension, so if I add --exclude _blake2 to my PyInstaller command for hashlib example, only libcrypto.so.1.1 is collected, and I get the selftest failure.

[MattTheCuber]

Can you give me pip freeze list for the environment?

I am in a giant catch-all environment (probably should have made a fresh one). Let me know if you want me to make a new one or send you a paste bin link of my current environment.

admittedly, it is 8.10 with python 3.11.9, with FIPS enabled post-hoc via "sudo fips-mode-setup --enable && sudo reboot"

Not an expert, but enabling FIPS post-boot shouldn't be problem, I doubt the Python version change makes a difference, but we have noticed big differences between libraries on different RHEL 8.x versions.

Trying you suggestions for hashlib now...

[MattTheCuber]

Adding --add-binary /usr/lib64/libssl.so.1.1:. worked for import hashlib (no errors)

[MattTheCuber]

remove the bundled libcrypto.so.1.1 and try running the application

This also worked

[MattTheCuber]

For reference, when I package import hashlib with no extra arguments it outputs:

Edit: the same structure for import openssl

[MattTheCuber]

Running pyinstaller test.py --exclude _blake2 also did not work on my system (without any modifications to the script or files generated).

[rokm]

Is there a /usr/lib64/python3.11/lib-dynload/_blake2.cpython-311-x86_64-linux-gnu.so file on your system? Is it collected into _internal/lib-dynload? If it is, what is it linked against (ldd <filename>)?

If I try importing it in python REPL, I get the following, though:

>>> import _blake2
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ImportError: blake2 is not available in FIPS mode

So it could be that it is completely unavailable for you... (and that's why hashlib does not work by default on your system).

[MattTheCuber]

$ ll /usr/lib64/python3.11/lib-dynload/_blake2.cpython-311-x86_64-linux-gnu.so
-rwxr-xr-x. 1 root root 86696 Sep 22  2023 /usr/lib64/python3.11/lib-dynload/_blake2.cpython-311-x86_64-linux-gnu.so

It is collected at _internal/lib-dynload/_blake2.cpython-311-x86_64-linux-gnu.so

$ ldd dist/test/_internal/lib-dynload/_blake2.cpython-311-x86_64-linux-gnu.so
        linux-vdso.so.1 (0x00007ffe345d9000)
        libpthread.so.0 => /lib64/libpthread.so.0 (0x00007f481356c000)
        libc.so.6 => /lib64/libc.so.6 (0x00007f48131a7000)
        /lib64/ld-linux-x86-64.so.2 (0x00007f48139ae000)

I get no errors when importing _blake2:

$ python
Python 3.11.6 (main, Nov  2 2023, 14:08:07) [GCC 8.5.0 20210514 (Red Hat 8.5.0-4)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import _blake2
>>> 

[MattTheCuber]

In an interpreter, import hashlib, and hashlib.blake2b() work fine.

[rokm]

Aha, then it's different in your python version vs. mine. In my case, it is linked against ssl and crypto:

$ ldd /usr/lib64/python3.11/lib-dynload/_blake2.cpython-311-x86_64-linux-gnu.so 
	linux-vdso.so.1 (0x00007ffeec929000)
	libssl.so.1.1 => /lib64/libssl.so.1.1 (0x00007fcd8e911000)
	libcrypto.so.1.1 => /lib64/libcrypto.so.1.1 (0x00007fcd8e426000)
	libpthread.so.0 => /lib64/libpthread.so.0 (0x00007fcd8e206000)
	libc.so.6 => /lib64/libc.so.6 (0x00007fcd8de30000)
	libz.so.1 => /lib64/libz.so.1 (0x00007fcd8dc18000)
	libdl.so.2 => /lib64/libdl.so.2 (0x00007fcd8da14000)
	/lib64/ld-linux-x86-64.so.2 (0x00007fcd8edbb000)

[MattTheCuber]

Wrong file, here's what I get:

$ ldd /usr/lib64/python3.11/lib-dynload/_blake2.cpython-311-x86_64-linux-gnu.so 
        linux-vdso.so.1 (0x00007ffe173eb000)
        libssl.so.1.1 => /lib64/libssl.so.1.1 (0x00007ff65cc24000)
        libcrypto.so.1.1 => /lib64/libcrypto.so.1.1 (0x00007ff65c73b000)
        libpthread.so.0 => /lib64/libpthread.so.0 (0x00007ff65c51b000)
        libc.so.6 => /lib64/libc.so.6 (0x00007ff65c156000)
        libz.so.1 => /lib64/libz.so.1 (0x00007ff65bf3e000)
        libdl.so.2 => /lib64/libdl.so.2 (0x00007ff65bd3a000)
        /lib64/ld-linux-x86-64.so.2 (0x00007ff65d0cd000)

[rokm]

Hmmm, that confuses situation somewhat. Can you check the build/<name>/Analysis-00.toc and find out where _blake2 was collected from?

Or if you run

import _blake2
print(_blake2.__file__)

in interpreter?

[MattTheCuber]

The lines from build/test/Analysis-00.toc are that mention _blake2 are:

  ('lib-dynload/_blake2.cpython-311-x86_64-linux-gnu.so',
   '/usr/local/lib/python311/lib/python3.11/lib-dynload/_blake2.cpython-311-x86_64-linux-gnu.so',
   'EXTENSION'),

$ python
Python 3.11.6 (main, Nov  2 2023, 14:08:07) [GCC 8.5.0 20210514 (Red Hat 8.5.0-4)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import _blake2
>>> print(_blake2.__file__)
/usr/local/lib/python311/lib/python3.11/lib-dynload/_blake2.cpython-311-x86_64-linux-gnu.so

[rokm]

Do you have two python 3.11 installations on that system? RPM-installed (python3.11, python3.11-libs) in /usr and something manually installed in /usr/local?

[MattTheCuber]

it appears so...

[rokm]

So I take it if you try to import _blake2 using /usr/bin/python3.11, it will raise an error?

[MattTheCuber]

Yep XD

I guess we need to uninstall all instances of Python 3.11.5 from our system?

[rokm]

Yep XD

I guess we need to uninstall all instances of Python 3.11.5 from our system?

Nah, don't, because then we might lose the chance to debug the real issue at hand.

[rokm]

So aside this detour with different python versions, the OpenSSL case is clear: if we collect ssl or crypto shared lib and there are accompanying hmac files, we need to ensure that the other lib is also collected (if that is not already the case).

[rokm]

Moving on to GnuTLS, we'll first need to figure out what is pulling it in. It would probably help if we worked with same kind of virtual environment, starting from a minimal one (e.g., create, update pip and wheel to latest version, and "setuptools<71"; then install PyInstaller and matplotlib). Then matplotlib probably won't pull in GnuTLS yet, and you shouldn't see the error.

You could also rebuild matplotlib example in your catch-all environment with added --clean --log-level DEBUG and redirect the stdout/stderr during build to a file. Then search for mentions of gnutls shared lib during binary dependency analysis, to see what modules or other libs it was referred from. You can also attach the full build log here, and I'll take a look.

[MattTheCuber]

test.py

import matplotlib

Commands

After creating a new virtual environment (using Python 3.11.6 - /usr/local/lib/python311/bin/python3.11) and entering it, here is what I ran and the output:

$ pip install pip -U
Requirement already satisfied
$ pip install wheel
Successfully installed
$ pip install "setuptools<71"
Requirement already satisfied
$ pip install pyinstall matplotlib
Successfully installed
$ pyinstaller test.py --clean -y --log-level DEBUG > build.log 2>&1
$ ./dist/test/test
Error in GnuTLS initialization: Error while performing self checks.

Build log

build.log

Filtered for gnutls:

26144 DEBUG: QtLibraryInfo(PyQt5): imported library 'libgnutls.so.30', full path '/lib64/libgnutls.so.30' -> parsed name 'gnutls'.
26172 DEBUG: QtLibraryInfo(PyQt5): imported library 'libgnutls.so.30', full path '/lib64/libgnutls.so.30' -> parsed name 'gnutls'.
26419 DEBUG: QtLibraryInfo(PyQt5): imported library 'libgnutls.so.30', full path '/lib64/libgnutls.so.30' -> parsed name 'gnutls'.
27638 DEBUG: Processing dependency, name: 'libgnutls.so.30', resolved path: '/lib64/libgnutls.so.30'
27638 DEBUG: Collecting dependency '/lib64/libgnutls.so.30' as 'libgnutls.so.30'.
27643 DEBUG: Processing dependency, name: 'libgnutls.so.30', resolved path: '/lib64/libgnutls.so.30'
27643 DEBUG: Skipping dependency '/lib64/libgnutls.so.30' due to prior processing.
...

[MattTheCuber]

When running pyinstaller test.py --clean -y --log-level DEBUG > build.log 2>&1 in my catch-all environment:

Build log

build.log

Filtered for gnutls:

32709 DEBUG: QtLibraryInfo(PyQt5): imported library 'libgnutls.so.30', full path '/lib64/libgnutls.so.30' -> parsed name 'gnutls'.
32752 DEBUG: QtLibraryInfo(PyQt5): imported library 'libgnutls.so.30', full path '/lib64/libgnutls.so.30' -> parsed name 'gnutls'.
33084 DEBUG: QtLibraryInfo(PyQt5): imported library 'libgnutls.so.30', full path '/lib64/libgnutls.so.30' -> parsed name 'gnutls'.
34766 DEBUG: Processing dependency, name: 'libgnutls.so.30', resolved path: '/lib64/libgnutls.so.30'
34766 DEBUG: Collecting dependency '/lib64/libgnutls.so.30' as 'libgnutls.so.30'.
34773 DEBUG: Processing dependency, name: 'libgnutls.so.30', resolved path: '/lib64/libgnutls.so.30'
34773 DEBUG: Skipping dependency '/lib64/libgnutls.so.30' due to prior processing.
...

[MattTheCuber]

In both environments:

$ ls -l /lib64/libgnutls.so.30
lrwxrwxrwx. 1 root root 20 Jun 28  2021 /lib64/libgnutls.so.30 -> libgnutls.so.30.28.2

dist/test/_internal/.libcrypto.so.1.1.hmac and dist/test/_internal/libgnutls.so.30 are bundled

[rokm]

Aha, you have PyQt5 in the environment... fair enough.

Looks like the hmac file libgmp.so.10 is not located next to the library file, but is rather in a fipscheck subdirectory.

An --add-data /usr/lib64/fipscheck/libgmp.so.10.hmac:fipscheck should do the trick.

[MattTheCuber]

That worked in both the minimal and catch-all environments!

[MattTheCuber]

So, what are the next steps from here?

[rokm]

I'll extend the hmac handling code for the fipscheck directory variant to fix the GnuTLS.

For OpenSSL, I need to think a bit on what the best approach would be. But that's less pressing, since typically, both ssl and crypto are collected together.

Until then, if you want FIPS compliance, use the --add-binary / --ad-data as a work-around. Or, if you are building only for RHEL8 systems and your deployment strategy allows you to assume that system-provided libs are always present on the target system, you could also try unbundling all system libraries via Analysis.exclude_system_libraries.

[MattTheCuber]

Great, I will use the --add-binary and --add-data options until they are fixed.

Thank you for all of your help!