Error when removing kratos-external-idp-integrator - relation-departed hook fails

[natalian98]

Bug Description

When kratos-external-idp-integrator is removed, it stays in terminated state with unknown unit status. When force-removed, kratos ends up in error state with unit message hook failed: "kratos-external-idp-relation-departed" and it can't exit it with juju resolve.

To Reproduce

1. Deploy kratos and postgresql-k8s
2. Deploy kratos-external-idp-integrator and relate it to kratos:
   juju deploy kratos-external-idp-integrator generic-external-idp --channel edge --trust
juju config generic-external-idp client_id=client_id client_secret=client_secret provider=generic issuer_url=http://example.com
juju relate kratos generic-external-idp
3. Wait until they reach active state
4. Remove kratos-external-idp-integrator

Environment

microk8s 1.26-strict/stable with dns hostpath-storage metallb addons
juju 3.1/stable
juju agent version 3.1.0

Relevant log output

When kratos-external-idp is removed with `juju remove-application generic-external-idp`:
unit-generic-external-idp-0: 15:37:59 ERROR juju.worker.uniter.operation hook "kratos-external-idp-relation-departed" (via hook dispatching script: dispatch) failed: signal: terminated
unit-generic-external-idp-0: 15:37:59 WARNING juju.worker.uniter.operation we should run a leader-deposed hook here, but we can't yet

When force-removed with `juju remove-application generic-external-idp --force --no-wait`:
unit-kratos-0: 14:06:16 ERROR juju.worker.uniter.operation hook "kratos-external-idp-relation-departed" (via hook dispatching script: dispatch) failed: exit status 1
unit-kratos-0: 14:06:26 DEBUG unit.kratos/0.juju-log kratos-external-idp:15: Operator Framework 2.1.1 up and running.
unit-kratos-0: 14:06:26 DEBUG unit.kratos/0.juju-log kratos-external-idp:15: Emitting Juju event kratos_external_idp_relation_departed.
unit-kratos-0: 14:06:26 ERROR unit.kratos/0.juju-log kratos-external-idp:15: Uncaught exception while in charm code:
Traceback (most recent call last):
  File "/var/lib/juju/agents/unit-kratos-0/charm/venv/ops/model.py", line 2615, in _run
    result = run(args, **kwargs)
  File "/usr/lib/python3.10/subprocess.py", line 524, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '('/var/lib/juju/tools/unit-kratos-0/relation-get', '-r', '15', '-', 'generic-external-idp', '--app', '--format=json')' returned non-zero exit status 1.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/var/lib/juju/agents/unit-kratos-0/charm/./src/charm.py", line 394, in <module>
    main(KratosCharm)
  File "/var/lib/juju/agents/unit-kratos-0/charm/venv/ops/main.py", line 436, in main
    _emit_charm_event(charm, dispatcher.event_name)
  File "/var/lib/juju/agents/unit-kratos-0/charm/venv/ops/main.py", line 144, in _emit_charm_event
    event_to_emit.emit(*args, **kwargs)
  File "/var/lib/juju/agents/unit-kratos-0/charm/venv/ops/framework.py", line 354, in emit
    framework._emit(event)
  File "/var/lib/juju/agents/unit-kratos-0/charm/venv/ops/framework.py", line 830, in _emit
    self._reemit(event_path)
  File "/var/lib/juju/agents/unit-kratos-0/charm/venv/ops/framework.py", line 919, in _reemit
    custom_handler(event)
  File "/var/lib/juju/agents/unit-kratos-0/charm/lib/charms/kratos_external_idp_integrator/v0/kratos_external_provider.py", line 635, in _on_provider_endpoint_relation_changed
    data = _load_data(data, PROVIDER_JSON_SCHEMA)
  File "/var/lib/juju/agents/unit-kratos-0/charm/lib/charms/kratos_external_idp_integrator/v0/kratos_external_provider.py", line 212, in _load_data
    if "providers" not in data:
  File "/var/lib/juju/agents/unit-kratos-0/charm/venv/ops/model.py", line 686, in __contains__
    return key in self._data
  File "/var/lib/juju/agents/unit-kratos-0/charm/venv/ops/model.py", line 679, in _data
    data = self._lazy_data = self._load()
  File "/var/lib/juju/agents/unit-kratos-0/charm/venv/ops/model.py", line 1377, in _load
    return self._backend.relation_get(self.relation.id, self._entity.name, self._is_app)
  File "/var/lib/juju/agents/unit-kratos-0/charm/venv/ops/model.py", line 2696, in relation_get
    raw_data_content = self._run(*args, return_output=True, use_json=True)
  File "/var/lib/juju/agents/unit-kratos-0/charm/venv/ops/model.py", line 2617, in _run
    raise ModelError(e.stderr)
ops.model.ModelError: ERROR permission denied

unit-kratos-0: 14:06:26 ERROR juju.worker.uniter.operation hook "kratos-external-idp-relation-departed" (via hook dispatching script: dispatch) failed: exit status 1

Additional context

The bug may be fixable in kratos instead of kratos-external-idp-integrator.