Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve Config Management/Stateless Runner Deploy Workflows #23703

Open
benyanke opened this issue Mar 25, 2023 · 20 comments · May be fixed by #32946
Open

Improve Config Management/Stateless Runner Deploy Workflows #23703

benyanke opened this issue Mar 25, 2023 · 20 comments · May be fixed by #32946
Labels
topic/gitea-actions related to the actions of Gitea type/feature Completely new functionality. Can only be merged if feature freeze is not active. type/proposal The new feature has not been accepted yet but needs to be discussed first.

Comments

@benyanke
Copy link
Contributor

Feature Description

Currently, it seems to register a runner, you must fetch a one-time use token.

This is good, when deploying manually. This is not helpful, however, if you're trying to do things like deploy runners with config management or especially a set of runners in an auto scaling group.

I would like an option to allow either 1) reusable registration tokens or 2) some other way to be able to register multiple runners without admin interaction easily.

Screenshots

No response

@benyanke benyanke added type/feature Completely new functionality. Can only be merged if feature freeze is not active. type/proposal The new feature has not been accepted yet but needs to be discussed first. labels Mar 25, 2023
@benyanke benyanke changed the title Improve Config Management Runner Deploy Workflows Improve Config Management/Stateless Runner Deploy Workflows Mar 25, 2023
@techknowlogick
Copy link
Member

related: #23643

@lunny
Copy link
Member

lunny commented Mar 26, 2023

tokens should be reused, I think maybe we can change the current logic.

@wolfogre wolfogre added the topic/gitea-actions related to the actions of Gitea label Mar 27, 2023
@cthu1hoo
Copy link

I too ran into this problem while writing my ansible role for runner deployment, reusable tokens (similar to the way gitlab does it) sound like the best approach to me.

@benyanke
Copy link
Contributor Author

Exactly where I ran into it as well - writing an idiomatic ansible role was almost impossible as-is.

@garymoon
Copy link
Contributor

Woodpecker and Drone accept a symmetic secret via environment variables for both the application and the runners, making automated provisioning zero-touch. Could such a flow be considered for Gitea?

@sebthom
Copy link

sebthom commented May 30, 2023

I don't really understand why a separate token mechanism was implemented for runner registration in the first place. I would suggest to remove it and instead introduce a new scope into the gitea access tokens that allows registering runners, e.g. admin:register_runner or register:runner? This reusable token could then be feed into the act_runner for self-registration. Nice side effect, the token management UI is already there. You see when the token was last used and can easily revoke it.

@sillyguodong
Copy link
Contributor

I would suggest to remove it and instead introduce a new scope into the gitea access tokens that allows registering runners

If you only use PAT to register runners, how do Gitea know the scope level (org, repo, global) of the runners.

@sebthom
Copy link

sebthom commented May 31, 2023

@sillyguodong True, then maybe the Gitea token can be extended to be "fine grained". https://github.blog/2022-10-18-introducing-fine-grained-personal-access-tokens-for-github/ instead of adding/maintaining a parallel token infrastructure only for the purpose of runner registration tokens.

@nodiscc
Copy link
Contributor

nodiscc commented Oct 19, 2023

deploy runners with config management

This role can deploy act-runner and register it automatically on a gitea instance. In particular have a look at the registration part [1].

@jtran
Copy link
Contributor

jtran commented Feb 16, 2024

@sillyguodong True, then maybe the Gitea token can be extended to be "fine grained". https://github.blog/2022-10-18-introducing-fine-grained-personal-access-tokens-for-github/ instead of adding/maintaining a parallel token infrastructure only for the purpose of runner registration tokens.

Wasn't this done in #24767?

@lunny
Copy link
Member

lunny commented Mar 27, 2024

I think this has been resolved by #23762 and #27143

@KyleGalvin
Copy link

How does this solve the issue in a containerized environment, where we usually just have a single docker command/entrypoint?

It would be really nice to have exactly what @garymoon suggests: inject the secret at startup from the environment. That way the deployment can give both gitea and the runner the same shared key and they can authenticate automatically after that.

No command line. No side channel communication. No post-deploy setup. Just give both the key.

@KyleGalvin
Copy link

I was searching for a solution to this, and found myself landing on this same thread 2 months down the line.

I have hacks and workarounds for the time being, but I want to describe my use-case to demonstrate how a shared environment variable between gitea and runners for initial setup can help:

I'm building a helm chart that includes gitea and act runners, and I want to automate the setup between the two.

I believe I could make a sidecar job that connects them using the API, but frankly this approach is more convoluted than I think it should be, and I'm using the following workaround hoping that a better solution comes down the pipeline.

Currently, I install the chart with an empty environment variable for the runner token.

I then wait for gitea to install and generate its own runner token. At this point, the runners are failing due to no valid token.

I go into the UI and copy that token out, pasting it into my chart values.yaml

I then update my helm chart with the new values, at which point the runners stop thrashing and connect.

This process isnt overly complicated for someone who knows how the setup works under the hood, but it would not be easy to point to my helm chart and tell another dev how to setup my environment.

@AlexMikhalev
Copy link

Since it's the second time I've hit the same issue with Gitea and it's a show-stopper for using it, I found that fork of gitea supports setting up runners via shared secrets of the box:
https://forgejo.org/docs/latest/admin/runner-installation/#offline-registration
Example of docker compose: https://code.forgejo.org/forgejo/runner/src/branch/main/examples/docker-compose/README.md
Example of Kubernetes: https://code.forgejo.org/forgejo/runner/src/branch/main/examples/kubernetes

@lunny
Copy link
Member

lunny commented Dec 17, 2024

Since it's the second time I've hit the same issue with Gitea and it's a show-stopper for using it, I found that fork of gitea supports setting up runners via shared secrets of the box: forgejo.org/docs/latest/admin/runner-installation#offline-registration Example of docker compose: code.forgejo.org/forgejo/runner/src/branch/main/examples/docker-compose/README.md Example of Kubernetes: code.forgejo.org/forgejo/runner/src/branch/main/examples/kubernetes

I think this issue has been resolved by

You can find these examples in Gitea's act_runner repository.
https://gitea.com/gitea/act_runner/src/branch/main/examples

All of these features in the fork are copied from this repository except the shared secrets feature.

@lunny
Copy link
Member

lunny commented Dec 17, 2024

I propose adding an additional parameter to both the Gitea CLI command and the Gitea API for registering runners. This enhancement would allow users to seamlessly configure predefined secrets for deployment on both the Gitea server and the runner sides.

By reusing the existing command-line tools and API endpoints, we can maintain consistency while extending functionality, ensuring a streamlined and efficient process for managing secrets.

@lunny
Copy link
Member

lunny commented Dec 17, 2024

I created the pull request #32878 and will add some tests and docs. Please help to review.

@AlexMikhalev
Copy link

Thank you. How can I test it?

@wxiaoguang
Copy link
Contributor

wxiaoguang commented Dec 22, 2024

The proper fix could be like this: Use env GITEA_RUNNER_REGISTRATION_TOKEN as global runner token #32946

Now, we could just set GITEA_RUNNER_REGISTRATION_TOKEN or GITEA_RUNNER_REGISTRATION_TOKEN_FILE (the same as runner) to make Gitea server pre-init the runner token. Then everything should work out-of-box.

(I think we will have in 1.23 stable)

@wxiaoguang
Copy link
Contributor

Thank you. How can I test it?

I think we could review and merge the fix into 1.23 and then you could try it with 1.23 nightly (maybe in one day)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
topic/gitea-actions related to the actions of Gitea type/feature Completely new functionality. Can only be merged if feature freeze is not active. type/proposal The new feature has not been accepted yet but needs to be discussed first.
Projects
None yet