Make cronjob runs more observable

[fridex]

Is your feature request related to a problem? Please describe.

As a Thoth operator, I would like to have an overview of cronjob runs as there are a few cronjobs that should be monitored. To support this, we could have a dashboard that shows each cronjob and information when it was run. This way, the Thoth operator can be sure cronjobs work and users of Thoth get up to date recommendations.

Additional context

• it might be a good idea to switch to CronWorkflows
  • related Migrate cronjobs to Argo CronWorkflows #654

There were also issues with cronworkflows, so worth to check if we can run cronworkflows in deployments.

[sesheta]

@fridex: This issue is currently awaiting triage.
One of the @thoth-station/devsops will take care of the issue, and will accept the issue by applying the
triage/accepted label and provide further guidance.

The triage/accepted label can be added by org members by writing /triage accepted in a comment.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[fridex]

/sig observability
/priority backlog

[sesheta]

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

/lifecycle stale

[mayaCostantini]

/remove-lifecycle stale
/lifecycle frozen

[VannTen]

To support this, we could have a dashboard that shows each cronjob and
information when it was run.

Would that be:

• producing metrics to be consummed by a dashboard (Grafana ?)
• producing the dashboard while using a dashboard engine like Grafana
• implement the dashboard functionnality ourselves
  ?

-> Some of those items might already exists