Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Enable stream and file logs output #4920

Merged
merged 24 commits into from
Aug 6, 2024

Conversation

KonstantinTyukalov
Copy link
Contributor

@KonstantinTyukalov KonstantinTyukalov commented Aug 1, 2024

PR is based on these changes: #4829

  • Added --restreamlogstofiles command line option
  • Added error output if both --restreamlogstofiles and --disableloguploads are used
  • Added UTC timestamp to local file log lines in universal time format. Example format: 2024-08-01T12:01:40.3870840Z ...
  • Updated --help docs for new option

I tested it with 3 cases:

  1. --ReStreamLogsToFiles flag passed (should stream logs to server and store locally)
  2. --DisableLogUploads flag passed (should store logs locally only)
  3. no flags passed (should stream logs to server only

In all cases, the logic works as expected

@KonstantinTyukalov KonstantinTyukalov marked this pull request as ready for review August 1, 2024 12:09
@KonstantinTyukalov KonstantinTyukalov requested review from a team as code owners August 1, 2024 12:09
@KonstantinTyukalov KonstantinTyukalov self-assigned this Aug 1, 2024
@DergachevE
Copy link
Contributor

@srkidd Can you please elaborate on these changes and your scenario(s) that require it?

@srkidd
Copy link
Contributor

srkidd commented Aug 2, 2024

@srkidd Can you please elaborate on these changes and your scenario(s) that require it?

@DergachevE

We need the ability to stream the log as well as store it on the build agent's file system. The impact to us is that we need to limit jobs to 5k log lines and we don't have an easy mechanism to ship the logs to our own logging backend.

If we store the logs locally, we can then easily upload to our backend.

The reason for all this is that we constantly hit rate limits due to amount of log data we have to pull out.

@KonstantinTyukalov KonstantinTyukalov merged commit 1bda91c into master Aug 6, 2024
12 checks passed
@KonstantinTyukalov KonstantinTyukalov deleted the users/srkidd/enable-stream-and-file-output branch August 6, 2024 12:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

8 participants