OAuth for REST API access?

[indy-independence]

Is it possible to authenticate to the netbox REST API via the remote authentication backends? Now I have authentication setup with:
REMOTE_AUTH_BACKEND = 'social_core.backends.keycloak.KeycloakOAuth2'
Which works great for the normal WebUI login. But asking every user to generate an API key on their own is not great for my usecase, and I don't think the provisioning of API key via API works either because it seems to assume you want to send in username+password to authenticate

I did a quick test and just added another backend under settings.py under the Django REST framework stuff

    'DEFAULT_AUTHENTICATION_CLASSES': (
        'rest_framework.authentication.SessionAuthentication',

But feels like I probably need to configure something more? With only that change I get a little bit further when trying with Authorization: Bearer header, it gives this error back: "AttributeError KeycloakOAuth2 object has no attribute authenticate_header"

[indy-independence]

I got a little bit further by installing another lib to handle auth for django rest framework:

source /opt/netbox/venv/bin/activate
pip install drf_social_oauth2==3.1.0

/opt/netbox/venv/lib/python3.12/site-packages/drf_social_oauth2/backends.py

from social_core.backends.keycloak import KeycloakOAuth2

class DjangoKeycloakOAuth2(KeycloakOAuth2):
    name = "Keycloak"



settings.py:
INSTALLED_APPS
    'oauth2_provider',
    'drf_social_oauth2',
]


TEMPLATES = [
    {
..
        'OPTIONS': {
....
            'context_processors': [
            ....
            
                'social_django.context_processors.backends',
                'social_django.context_processors.login_redirect',
            ],
        },
    },
]

AUTHENTICATION_BACKENDS = [
    *REMOTE_AUTH_BACKEND,
    'drf_social_oauth2.backends.DjangoKeycloakOAuth2',
    'drf_social_oauth2.backends.DjangoOAuth2',
    'django.contrib.auth.backends.ModelBackend',
    'netbox.authentication.ObjectPermissionBackend',
] 

Now I get {"detail":"You do not have permission to perform this action."} so I need some way to map something in access token to permissions in netbox I guess