Feature: skip expected replicaSets from deployments and statefulsets, as well as jobs from cronjobs

[mpatters72]

Problem: I don't want to alert/include standard replicasets associated with deployments, statefulsets in a report /slack message as this distracts from what I consider truly orphaned resources. Similarly I'd like to be able to keep expected jobs from cronjobs.

• A deployment/statefulset that has been updated, may keep around old replicasets up the spec/revisionHistoryLimit (default 10)

• We set revisionHistoryLimit so we have some rollback options / history

• Jobs that were launched by a cronJob can be useful to checking logs on last successful run. In my team we tune jobs-history-limits to keep last successful and last falied.

Describe the solution you'd like

• Flag to skip replicasets currently associated with deployments/statefulsets (or alternatively last X number)

• Configmaps/secrets associated with those replicasets skipped also (can't rollback if it is removed)

Describe alternatives you've considered

• Add labels to everything and filter out that way. Impractical for the very large number of services and namespaces we use, and eventually things no longer needed.

• Don't include replicasets in report. I'm only aware of the command referencing the specific object eg replicasets or all for example. - This actually would be a different feature request to be able to give a list of objects, eg cronjobs,deployments instead of all or just 1 object to the command.

• For cronjobs I could probably use --older-than flag to ignore jobs from more recent runs.

Additional context

[yonahd]

Hi @mpatters72
Thanks for raising the issue.

Regarding
"
Don't include replicasets in report. I'm only aware of the command referencing the specific object eg replicasets or all for example. - This actually would be a different feature request to be able to give a list of objects, eg cronjobs,deployments instead of all or just 1 object to the command
"
This is actually possible.
Give it a try 🙂

Regarding history for replicasets and jobs we can add this feature (with a flag for these). Contributions are welcome

Last
Regarding
Configmaps/secrets associated with old replicasets
This is actually an interesting question. This might be a bug that we should address.

[mpatters72]

I tried a list of commands/objects again and it does indeed work for me. Thank you @yonahd

A few other things I noticed that might could be useful features (at least for me).

Feature: Context flag

Problem

• I have many different contexts/clusters to process and I can't pass an argument for that like I can with kubectl --context <contextname> -n <namespacename>

Desired solution

1. Flag for config and/or cluster similar to how we have this option for namespaces
   --cluster
   --config

Workaround

• Simple program that goes through my list of namespaces, and clusters and sets context prior to using kor command.
  kubectl config use-context my-cluster-name

=======

** Problem**

• Kor isn't aware of some other controllers we use like Argo Rollouts

• It assumes deployments with 0 replicas is obsolete - when the deployment template is actually referenced by the argo rollout object reference-deployment-from-rollout_1

• eg argoRollout manifest

spec:
  workloadRef:
    apiVersion: apps/v1
    kind: Deployment
    name: "deploymentName"

• Other objects associated with the rollout assumed obsolete, but are actually expected:

• eg HPAs, PDBs, configmaps mounted by rollout pods, etc

spec:
  scaleTargetRef:
    apiVersion: argoproj.io/v1alpha1
    kind: Rollout
    name: rollout-name

Desired Solution

• Add support for argo rollouts similar to deployment and statefulsets support.

** Workaround **

• None known other than ignoring false positives.

=======

Feature: Other ingress objects Contour: HttpProxy and IngressRoute

• https://projectcontour.io/docs/v1.0.1/ingressroute/
• https://projectcontour.io/docs/v1.6.1/httpproxy/

Problem

• I'm not checking for obsolete ingressroutes/httpproxies

• I'm falsely identifying SSL secrets used by ingressroutes/httpproxies as being orphaned

• not checking for services connected to ingressRoutes.

** Desired Solution**

• Support for these additional "ingress" types.

** Workarounds **

• None known.

[yonahd]

Hey
You brought up another couple of valid points.
We do support the --kubeconfig flag if that is helpful but we don't support switching contexts within kor.

Regarding references by a CRD we are well aware these false positives exist and warn about it in the Readme.

We will try in the future to work on a group of crds to avoid these issues.

[mpatters72]

Maybe I'm doing it wrong, but seems like when I provide a list of objects, it seems to just process last one in list instead of all of them and ignores previous ones. Using /opt/homebrew/Cellar/kor/0.5.4 on arm64 mac.

# replicaset 1st, pdb 2nd, only gives pdb report

kor -n ns-team-hz-ccweb-dev "replicaset,pdb"
kor version: vdev

  _  _____  ____
 | |/ / _ \|  _ \
 | ' / | | | |_) |
 | . \ |_| |  _ <
 |_|\_\___/|_| \_\

Unused resources in namespace: "ns-team-hz-ccweb-dev"
+---+---------------+----------------------------------+
| # | RESOURCE TYPE |          RESOURCE NAME           |
+---+---------------+----------------------------------+
| 1 | Pdb           | alertmanager                     |
| 2 | Pdb           | gneiss-service                   |
| 3 | Pdb           | prometheus                       |
| 4 | Pdb           | release-plane-management-service |
+---+---------------+----------------------------------+

# replicaset pdb 1st, replicaset 2nd, only gives replicaset

╭─ ~                                                                                                                                                                                                                           ✔  14:39:32
╰─ kor -n ns-team-hz-ccweb-dev "pdb,replicaset"
kor version: vdev

  _  _____  ____
 | |/ / _ \|  _ \
 | ' / | | | |_) |
 | . \ |_| |  _ <
 |_|\_\___/|_| \_\

Unused resources in namespace: "ns-team-hz-ccweb-dev"
+----+---------------+---------------------------------------------+
| #  | RESOURCE TYPE |                RESOURCE NAME                |
+----+---------------+---------------------------------------------+
|  1 | ReplicaSet    | app-kube-state-metrics-55965b4d8            |
|  2 | ReplicaSet    | app-kube-state-metrics-55cbddcc47           |
|  3 | ReplicaSet    | app-kube-state-metrics-56c454694d           |
+----+---------------+---------------------------------------------+

# big list of things with secret last, only returns secret

kor -n ns-team-hz-ccweb-dev "pdb,replicaset,hpa,deployment,secret"
kor version: vdev

  _  _____  ____
 | |/ / _ \|  _ \
 | ' / | | | |_) |
 | . \ |_| |  _ <
 |_|\_\___/|_| \_\

Unused resources in namespace: "ns-team-hz-ccweb-dev"
+---+---------------+--------------------------------+
| # | RESOURCE TYPE |         RESOURCE NAME          |
+---+---------------+--------------------------------+
| 1 | Secret        | alertmanager-alertmanager      |
| 2 | Secret        | prometheus-scrape-config       |
| 3 | Secret        | vault-secrets-operator-approle |
+---+---------------+--------------------------------+

[yonahd]

Maybe I'm doing it wrong, but seems like when I provide a list of objects, it seems to just process last one in list instead of all of them and ignores previous ones. Using /opt/homebrew/Cellar/kor/0.5.4 on arm64 mac.

# replicaset 1st, pdb 2nd, only gives pdb report

kor -n ns-team-hz-ccweb-dev "replicaset,pdb"
kor version: vdev

  _  _____  ____
 | |/ / _ \|  _ \
 | ' / | | | |_) |
 | . \ |_| |  _ <
 |_|\_\___/|_| \_\

Unused resources in namespace: "ns-team-hz-ccweb-dev"
+---+---------------+----------------------------------+
| # | RESOURCE TYPE |          RESOURCE NAME           |
+---+---------------+----------------------------------+
| 1 | Pdb           | alertmanager                     |
| 2 | Pdb           | gneiss-service                   |
| 3 | Pdb           | prometheus                       |
| 4 | Pdb           | release-plane-management-service |
+---+---------------+----------------------------------+

# replicaset pdb 1st, replicaset 2nd, only gives replicaset

╭─ ~                                                                                                                                                                                                                           ✔  14:39:32
╰─ kor -n ns-team-hz-ccweb-dev "pdb,replicaset"
kor version: vdev

  _  _____  ____
 | |/ / _ \|  _ \
 | ' / | | | |_) |
 | . \ |_| |  _ <
 |_|\_\___/|_| \_\

Unused resources in namespace: "ns-team-hz-ccweb-dev"
+----+---------------+---------------------------------------------+
| #  | RESOURCE TYPE |                RESOURCE NAME                |
+----+---------------+---------------------------------------------+
|  1 | ReplicaSet    | app-kube-state-metrics-55965b4d8            |
|  2 | ReplicaSet    | app-kube-state-metrics-55cbddcc47           |
|  3 | ReplicaSet    | app-kube-state-metrics-56c454694d           |
+----+---------------+---------------------------------------------+

# big list of things with secret last, only returns secret

kor -n ns-team-hz-ccweb-dev "pdb,replicaset,hpa,deployment,secret"
kor version: vdev

  _  _____  ____
 | |/ / _ \|  _ \
 | ' / | | | |_) |
 | . \ |_| |  _ <
 |_|\_\___/|_| \_\

Unused resources in namespace: "ns-team-hz-ccweb-dev"
+---+---------------+--------------------------------+
| # | RESOURCE TYPE |         RESOURCE NAME          |
+---+---------------+--------------------------------+
| 1 | Secret        | alertmanager-alertmanager      |
| 2 | Secret        | prometheus-scrape-config       |
| 3 | Secret        | vault-secrets-operator-approle |
+---+---------------+--------------------------------+

Hey @mpatters72
Any chance you can open this in a separate issue?
It allows us to delegate and focus on each issue separately in its own context