Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Do not "tail" the replication.log when running the arbiter-setup run-replication step #426

Closed
dweberJsonar opened this issue Jul 4, 2024 · 1 comment · Fixed by #427
Closed

Comments

@dweberJsonar
Copy link

Impetus

@jagdeep-sonar has reached out to me with a failed TF run: 9653669156. Something I noticed in the output is a lot of:

...
module.agentless_gw_hadr[0].null_resource.exec_replication_cycle_on_dr (remote-exec): 1.90      2024-06-24 23:29:38,...
module.agentless_gw_hadr[0].null_resource.exec_replication_cycle_on_dr (remote-exec): 2.15      2024-06-24 23:29:39,...
module.agentless_gw_hadr[0].null_resource.exec_replication_cycle_on_dr (remote-exec): 2.40      2024-06-24 23:29:39,...
module.agentless_gw_hadr[0].null_resource.exec_replication_cycle_on_dr (remote-exec): 2.66      2024-06-24 23:29:39,...
module.agentless_gw_hadr[0].null_resource.exec_replication_cycle_on_dr (remote-exec): 2.91      2024-06-24 23:29:39,...
...

There are hundreds of lines like this and we of course want to avoid bloating the logs :) This is happening because the run-replication command is by default, meant to be run in an interactive environment (i.e. manually triggered via a user). This command is kind of a "cheap" implementation of tail or watch and tails the last line of replication.log then prints that line to stdout with a carriage return. The problem is that TF is capturing each one of the "frames" being spit out in between the carriage returns. To handle this exact solution, I added the --no-tail flag to the command, thus allowing us to run this command in a non-interactive environment. You can see https://github.com/jsonar/disaster-recovery/blob/master/docs/arbiter/arbiter.rst#running-a-replication-cycle for more information regarding this.

@dweberJsonar
Copy link
Author

dweberJsonar commented Jul 4, 2024

Solution

The solution is to locate:

provisioner "remote-exec" {
inline = [
"sudo touch $JSONAR_LOGDIR/sonarw/replication.log",
"sudo chown sonarw:sonar $JSONAR_LOGDIR/sonarw/replication.log",
"sudo $JSONAR_BASEDIR/bin/arbiter-setup run-replication"]
}

And add --no-tail to the command on line 85. It looks like I can't open PRs against the repo without forking - otherwise I'd just edit this myself :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant