Skip to content

Daemon kills the agent child while it is still waiting for a certificate #685

Open

Description

@miharp

Since #683 the forked child waits for its certificate before the run starts. The parent's wait_for_child deadline (runtimeout + http_connect_timeout + http_read_timeout) still starts at the fork, so the wait eats the run's time budget.

With defaults and an unsigned CSR, the daemon SIGKILLs the child about 72 minutes after the fork, logs "did not exit within 720 seconds of the run timeout, killing it", and forks again with a fresh maxwaitforcert deadline. A finite maxwaitforcert never fires. If the cert is signed late in that window, the run that follows can be killed while still well under runtimeout.

Before #683 the parent did the waiting itself, so maxwaitforcert applied to the whole wait and runtimeout only to the run.

This is unlikely to bite in practice. An agent with a valid cert never enters the wait, so it only affects a daemon bootstrapping without autosign (or one whose cert was cleaned while it kept running) where signing takes over an hour. Even then the next scheduled run waits again, so the agent still gets its cert eventually; the cost is a misleading error every ~72 minutes, a possible killed run, and maxwaitforcert not being honoured. Not an rc blocker, but the fix is small.

Reproduction: miharp#5 adds real-fork specs in spec/integration/agent/forked_run_spec.rb. The examples "honours maxwaitforcert instead of killing the child at the run deadline" and "gives the run its full runtimeout" cover this; they pass at be81da4 and fail at 87f7721 (see the CI run there).

Fix idea: have the child tell the parent when the wait is over (a pipe would do) and start the deadline from there. Extending the deadline by maxwaitforcert doesn't work since the default is unlimited.

Also, the child sets $0 to "puppet agent: applying configuration" before the wait, so ps is wrong for the whole time it's waiting.

Affects main at 87f7721 and 9.0.0-rc2.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions