[BUG] AttributeError: '_pygit2.Blob' object has no attribute 'hex'

[OrangeDog]

Description
gitfs and git_pillar do not work with pygit2 >= 1.15.0, rasing exceptions as below:

[ERROR] Error in function _file_find:

Traceback (most recent call last):
  File "/opt/saltstack/salt/lib/python3.10/site-packages/salt/master.py", line 1927, in run_func
    ret = getattr(self, func)(load)
  File "/opt/saltstack/salt/lib/python3.10/site-packages/salt/fileserver/__init__.py", line 547, in _find_file
    return self.find_file(path, tgt_env)
  File "/opt/saltstack/salt/lib/python3.10/site-packages/salt/fileserver/__init__.py", line 599, in find_file
    fnd = self.servers[fstr](path, saltenv, **kwargs)
  File "/opt/saltstack/salt/lib/python3.10/site-packages/salt/loader/lazy.py", line 159, in __call__
    ret = self.loader.run(run_func, *args, **kwargs)
  File "/opt/saltstack/salt/lib/python3.10/site-packages/salt/loader/lazy.py", line 1245, in run
    return self._last_context.run(self._run_as, _func_or_method, *args, **kwargs)
  File "/opt/saltstack/salt/lib/python3.10/site-packages/salt/loader/lazy.py", line 1260, in _run_as
    ret = _func_or_method(*args, **kwargs)
  File "/opt/saltstack/salt/lib/python3.10/site-packages/salt/fileserver/gitfs.py", line 160, in find_file
    return _gitfs().find_file(path, tgt_env=tgt_env, **kwargs)
  File "/opt/saltstack/salt/lib/python3.10/site-packages/salt/utils/gitfs.py", line 3246, in find_file
    blob, blob_hexsha, blob_mode = repo.find_file(repo_path, tgt_env)
  File "/opt/saltstack/salt/lib/python3.10/site-packages/salt/utils/gitfs.py", line 2280, in find_file
    return blob, blob.hex, mode
AttributeError: '_pygit2.Blob' object has no attribute 'hex'

[ERROR] Unable to get SHA of HEAD for git_pillar remote 'master [email protected]:owner/saltstack.git'
[ERROR] Failed to checkout ubuntu-24.04 from git_pillar remote '__env__ [email protected]:owner/saltstack.git': '_pygit2.Commit' object has no attribute 'hex'
Traceback (most recent call last):
  File "/opt/saltstack/salt/lib/python3.10/site-packages/salt/utils/gitfs.py", line 1842, in checkout
    target_sha = self.peel(self.repo.lookup_reference(remote_ref)).hex
AttributeError: '_pygit2.Commit' object has no attribute 'hex'

Setup

fileserver_backend:
  - gitfs
gitfs_privkey: /etc/salt/pki/master/git/salt-git.rsa
gitfs_pubkey: /etc/salt/pki/master/git/salt-git.rsa.pub
gitfs_refspecs:
  - +refs/heads/*:refs/remotes/origin/*
  - +refs/tags/*:refs/tags/*
  - +refs/pull/*/merge:refs/remotes/origin/merge/*
gitfs_remotes:
  - https://github.com/saltstack-formulas/apt-formula:
    - all_saltenvs: 04612df951f5b71b58eb39ecc40e4b6404200885
    - root: apt
    - mountpoint: salt://apt
    - update_interval: 86400
  # more formulae
  - [email protected]:owner/saltstack.git:
    - root: salt
    - update_interval: 120

ext_pillar:
  - git:
    - master [email protected]:owner/saltstack.git:
      - root: pillar
      - env: base
    - __env__ [email protected]:owner/saltstack.git:
      - root: pillar
git_pillar_privkey: /etc/salt/pki/master/git/salt-git.rsa
git_pillar_pubkey: /etc/salt/pki/master/git/salt-git.rsa.pub
git_pillar_update_interval: 300
gpg_decrypt_must_succeed: true
pillarenv_from_saltenv: true

Steps to Reproduce the behavior

systemctl stop salt-master
salt-pip install --upgrade pygit2
rm -rf /var/cache/salt/master/git{fs,_pillar}
systemctl start salt-master
salt-call state.test

No Top file or master_tops data matches found. Please see master log for details.

While filserver contents can be listed (e.g. with salt-run fileserver.file_list), a highstate cannot be run, and pillar appears empty.

Expected behavior
Git features continue to work as with previous pygit2 versions.

Versions Report

salt --versions-report

Salt Version:
          Salt: 3006.9

Python Version:
        Python: 3.10.14 (main, Jun 26 2024, 11:44:37) [GCC 11.2.0]

Dependency Versions:
          cffi: 1.17.1
      cherrypy: unknown
  cryptography: 42.0.5
      dateutil: 2.8.1
     docker-py: Not Installed
         gitdb: Not Installed
     gitpython: Not Installed
        Jinja2: 3.1.4
       libgit2: 1.8.1
  looseversion: 1.0.2
      M2Crypto: 0.38.0
          Mako: Not Installed
       msgpack: 1.0.2
  msgpack-pure: Not Installed
  mysql-python: Not Installed
     packaging: 22.0
     pycparser: 2.21
      pycrypto: Not Installed
  pycryptodome: 3.19.1
        pygit2: 1.16.0
  python-gnupg: 0.4.8
        PyYAML: 6.0.1
         PyZMQ: 23.2.0
        relenv: 0.17.0
         smmap: Not Installed
       timelib: 0.2.4
       Tornado: 4.5.3
           ZMQ: 4.3.4

System Versions:
          dist: ubuntu 24.04.1 noble
        locale: utf-8
       machine: x86_64
       release: 6.8.0-47-generic
        system: Linux
       version: Ubuntu 24.04.1 noble

Additional context
It seems like the fix for #66590 was never tested, as it still doesn't work.

[dmurphy18]

@OrangeDog Can you give example for your statement

It seems like the fix for https://github.com/saltstack/salt/issues/66590 was never tested, as it still doesn't work.

[OrangeDog]

...?

If it had been tested, then it would have found this issue. Unless there's some weird obscure case I'm hitting.

[dmurphy18]

Pull in those changes from Suse, see https://github.com/openSUSE/salt/pull/673/files, revisiting those changes now for second view.

[viq]

@dmurphy18 those are separate issues. That pull request fixes only the oid getting deprecated. There's a separate issue with newer pygit2 where salt is failing to talk to git because of the aforementioned

AttributeError: '_pygit2.Blob' object has no attribute 'hex'

[dmurphy18]

@viq I am digging in and testing and fixing it all at the same time.

[OrangeDog]

newer pygit2

Nope. The error starts in the same version: 1.15.0