K8s: Tutor should not assume sole control on k8s namespace

[waza-ari]

Bug description

When creating a new environment, tutor will create the namespace automatically. More crucially, when deleting an environment, it will attempt to delete the namespace as well.

This is an issue if the namespace is externally created (in our case, basically the entire cluster except tutor is GitOps managed using ArgoCD), which is also creating the namespace including a few basic services (HashiCorp vault connection for secret store and so on). When tutor attempts to delete the NS, it conflicts with ArgoCD.

Again it might make sense to introduce an additional parameter to control the behaviour.

[regisb]

What command specifically is causing an issue? What are the logs?

[snglth]

This is a good point. I manage a "sandbox" instance at work. This "sandbox" part means that users and courses are being wiped once in a while. The namespace itself and the cleanup CronJobs are deployed by FluxCD. So, if I run tutor k8s delete tutor delete the cleanup CronJobs and the namespace which is suboptimal in my opinion. There has to be a way to configure tutor to omit deleting objects that was not deployed by tutor. Maybe a flag in config.yml like K8S_EXTERNAL_NAMESPACE set to true to tell tutor to not manage the namespace. @regisb What do you think?

[waza-ari]

Ah, sorry for not coming back earlier, I somehow missed the answer. Yes, it's the fact that the namespace being deleted while externally managed (ArgoCD in our case). ArgoCD allows us to provision k8s environments for projects (including the namespace, roles and so on), such that the user works within their assigned NS and permissions.

A flag as suggested by @snglth would be perfect for our use case as well.

[regisb]

Let's not add a new configuration setting without further consideration. There are alternatives:

• On tutor k8s delete, all tutor does is running kubectl delete -k ~/.local/share/tutor/env --ignore-not-found=true --wait, as per the console output. I think it's pretty easy to run this command manually to add custom filters there, right?
• Alternatively we could add a --exclude-namespace option to tutor k8s delete.