tanzu_mission_control_secret: new module to manage TMC Secrets and SecretExports

[massix]

SUMMARY

This module allows to manage Tanzu Mission Control Secrets, both for single Clusters or for ClusterGroups. It mimics the UI of TMC by also proposing to generate a SecretExport (or remove it) for a given secret.

This module has only been tested using TMC SaaS, but it should also work for TMC Self Hosted (but there is no way for me to test it).

Due to the way the secrets in TMC work (you cannot retrieve the value once it has been created), I opted to go for a tri-state variable: present, absent and update. When using present, if the secret already exists it won't be modified (even if the data fields are different), this is done in order to preserve idempotency between different runs and to avoid false changed statuses which could make people panic easily (I know I am one of those guys). When using absent, the secret is deleted if it exists, while when using update the secret is updated. Every update will erase the existing fields (this is also noted in the description, but let me know if I should put a better warning for that).

Other than that, the module is pretty standard. I have been using it for a couple of days and it serves my needs.

ISSUE TYPE

• New Module/Plugin Pull Request

COMPONENT NAME

tanzu_mission_control_secret

ADDITIONAL INFORMATION

I only included some extensive integration tests, since it felt quite useless to develop unit tests (we would just be testing mocks and the behavior of the module is quite simple).

[massix]

ready_for_review

Ok, this should now be ready for a review. For the records, I have been using this module in a private repository for a while where we only use Python 3.12 and never cared about it being compatible with Python 2.7, hence it took some time to convert it back to 2.7 😅
(sorry for the wasted CI minutes! 😓 but somehow when I run ansible-test sanity --docker --changed ... locally it only tests against Python 3.6 -> 3.13)

Just for personal curiosity: is there a way to opt-out from the Python 2.7 compatibility? I skimmed through the docs at Ansible's website and it doesn't look like it is possible..

[felixfontein]

Just for personal curiosity: is there a way to opt-out from the Python 2.7 compatibility? I skimmed through the docs at Ansible's website and it doesn't look like it is possible..

With general collections, yes. With community.general, no. You'd have to wait for community.general 12.0.0, which will drop support for ansible-core 2.16, and thus require ansible-core 2.17+. That's the first ansible-core version that no longer supports Python 2.7.

but somehow when I run ansible-test sanity --docker --changed ... locally it only tests against Python 3.6 -> 3.13)

The supported Python versions depend on the ansible-core version. So if you run ansible-test from ansible-core 2.17 or 2.18, no tests for Python 2.7 will be run. If you're using ansible-test from ansible-core 2.16, though, then Python 2.7 tests will be run.

This is somewhat annoying, but will stop being a problem "soon" (in ~half a year for c.g 12.0.0).

[russoz]

hi @massix thanks for your contribution

The docs need to be reviewed more closely - specially the references to other options and to values, many should be using semantic markup.

You might want to give it a try to andebox by yours truly:

https://pypi.org/project/andebox/

You can see (and adjust, with additional options) many things in the YAML documentations by just running:

$ andebox yaml-doc plugins/modules/tanzu_mission_control_secret.py

I also lean towards moving those 3 classes (the abstract and the two implementations) to a module_utils file, so that the module file becomes leaner.

[massix]

You might want to give it a try to andebox by yours truly:

https://pypi.org/project/andebox/

That is great, thanks! I was doing something similar using a helper script but it was quickly becoming yet another stuff to maintain :-)

[massix]

The docs need to be reviewed more closely - specially the references to other options and to values, many should be using semantic markup.

I partially rewrote the documentation, this time using the semantic markup in a clearer way. I have also clarified the part regarding the API Token, which might have lead to some confusions regarding the rights.

Hopefully this time it is better, I also used andebox yaml-docs to clear up the punctuation, white spaces and other nice stuff.

I also lean towards moving those 3 classes (the abstract and the two implementations) to a module_utils file, so that the module file becomes leaner.

I think that after removing the comments and cleaning the documentation, the module has become more readable and I think that splitting into multiple files could hinder the overall readability of the module, but I will leave the final decision to you :-)

[mariolenz]

I don't know much about Tanzu Mission Control, but it looks like a pretty complex solution to manage... Kubernetes clusters? I'm not 100% sure there.

I don't know if community.general is the right place for this module. As I've said, TMC looks like a very complex product. Having this module here might make people want to see more and more modules implementing additional functionality. This might be a little but too much for this collection.

I think there should be a dedicated collection for this. Or maybe even Tanzu at large. But it might go beyond the scope of this collection to have those modules here. I don't maintain community.general, just wanted to throw in some thoughts.

This module has only been tested using TMC SaaS, but it should also work for TMC Self Hosted (but there is no way for me to test it).

I think TMC SaaS reached End of Availability (EoA). So I guess this should really focus on (and be tested with) TMC on-premises. TMC SaaS looks like it will die away sooner or later.

related: ansible-collections/community.vmware#428

[massix]

I don't know much about Tanzu Mission Control, but it looks like a pretty complex solution to manage... Kubernetes clusters? I'm not 100% sure there.

Well, Tanzu is more of an ecosystem than a product itself, and it is composed of many different softwares and platforms. The most important ones being Tanzu Kubernetes Grid, which is what makes it easy to deploy Kubernetes clusters on ESXi and VMWare-enabled infrastructures, and CAPV, which stands for Cluster API Provider for VSphere, where the Cluster APIs are a standard defined by the K8S SIG and re-implemented by a lot of different Cloud Providers. I wouldn't be surprised to discover that Azure or GCP or DigitalOcean use the Cluster APIs to provision Kubernetes Clusters on their cloud offerings.

So, yes, it is a pretty complex solution to have one Kubernetes cluster provision, manage and update other Kubernetes clusters, but it is a standard one. VMware did not reinvent the wheel and they actually did a great job with CAPV, which is 100% Cluster APIs compatible.

I have built a system using a combination of CAPV, FluxCD and ytt (another product of the Tanzu family) to provision K8s clusters using simple YAML manifests, which are validated through a pipeline and reconciled to the management cluster with FluxCD, which allows for a streamlined flow that anyone in the company can use.

On top of that, we use Tanzu Mission Control (which is different from Tanzu Platform!) to automatically setup availability zones across datacenters, setup backups with Velero (another product of the Tanzu ecosystem) and manage mutations with Gatekeeper (another product of the Tanzu ecosystem). All of that "static" setup is done through Terraform using the provider developed by VMWare.

But, there are some dynamic parts which are easier to manage with Ansible, for example right now we are automating the creation of GitLab tokens to be used in K8s clusters to setup FluxCD right after the creation of the cluster, and for that I am using the Operator SDK, which can trigger Ansible playbooks at every reconciliation, hence the need to write some modules for Ansible. I have a couple of them ready (one is here in the PR, and the other one is to manage SourceSecrets). I am not really interested in developing other modules for functionalities I am not using right now, but I thought it could be interesting to provide a starting point for future developments.

I think TMC SaaS reached End of Availability (EoA). So I guess this should really focus on (and be tested with) TMC on-premises. TMC SaaS looks like it will die away sooner or later.

So to answer that, TMC SaaS is not dead, Tanzu Platform is :-)

Sorry for the wall of text, I hope it clarifies things a bit for you :-D

Regarding the PR, I would like to know from @felixfontein and @russoz if there is still interest in this module being integrated into community.general or not, or if we should move it to a different collection? I personally think it can belong here, I have always seen the community.vmware collection for more low-level IaaS stuff (handling vSphere objects mainly), but I'll leave the decision to you :-D

[russoz]

Whilst I understand @mariolenz concerns, I think they fall in the category bridge-to-be-crossed-upon-arriving-at-river.

On a quick search, it does not seem that there is any existing collection more suitable for the module. I would argue that it should start here and, if there's an appetite from the user and developer community to provide more plugins/modules addresing TMC-specific functionalities, then we refactor them out into its own collection. Maintaining a collection for one single module is not productive, IMHO.

All that being said, I have been a bit busy these days, so I am planning on reviewing (for realsies) this and other pending PRs on the weekend.

[mariolenz]

On a quick search, it does not seem that there is any existing collection more suitable for the module. I would argue that it should start here and, if there's an appetite from the user and developer community to provide more plugins/modules addresing TMC-specific functionalities, then we refactor them out into its own collection.

I agree that there's no more suitable collection for this module. I'm not against merging this (as I've said: I don't maintain this collection) and just wanted to throw in some thoughts. Even if it turns out I've been wrong at least on some of them, maybe even all.

On the other hand, my comment made @massix give some more information which looks very valuable to me. Maybe this helps the maintainers a bit.

Maintaining a collection for one single module is not productive, IMHO.

Good point!

[felixfontein]

Thanks for your comments @mariolenz! Disclaimer: I did ask @mariolenz whether he knows of a better scoped collection for this module, since he's active in the vmware collections and wrote in ansible-collections/community.vmware#428. If there isn't a better collection, community.general is fine. Also it's still possible to move modules later on if a desire to create a dedicated collection arises.

[russoz]

hi @massix

I picked up a couple of things you might want to take a look before we merge this one.

[felixfontein]

This would be

Suggested change

	version_added: 11.0.0
	version_added: 11.1.0

now, since 11.0.0 has already been released two weeks ago.

[felixfontein]

These are not four unrelated notes, but seem to be long together. So they should be in one note paragraph.

Suggested change

	- To obtain an O(api_token), login to Tanzu Mission Control, click on your user, then I(Settings) and then I(Generate an
	API token).
	- The generated API token B(must) have the rights to create I(Secrets) and I(SecretExports) in Tanzu Mission Control.
	- The rights are granted by the administrator of the TMC Instance to single users or groups.
	- When creating the token, make sure you select the C(tmc_user) service role for all the required organizations.
	- To obtain an O(api_token), login to Tanzu Mission Control, click on your user, then I(Settings) and then I(Generate an
	API token).
	The generated API token B(must) have the rights to create I(Secrets) and I(SecretExports) in Tanzu Mission Control.
	The rights are granted by the administrator of the TMC Instance to single users or groups.
	When creating the token, make sure you select the C(tmc_user) service role for all the required organizations.

[felixfontein]

Since it is mutually exclusive with others, it feels strange if there is a default. I would remove it.

Suggested change

default: {}

[felixfontein]

update is not a state name, it's an action name. Use updated instead of update.

[felixfontein]

Ping @massix

needs_info

[massix]

Hey, sorry I have been out of office for the whole month. I will resume working on this on September 👍