Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

reset logger level when config isnt present #4232

Merged
merged 8 commits into from
Aug 5, 2024

Conversation

puranamp
Copy link
Contributor

@puranamp puranamp commented Jul 25, 2024

Issue: Suppose we had a registered logger named a.b.c that we configured to have the level WARN. Then, we removed this configuration option and reconfigured. The a.b.c logger will still be set to WARN despite not specifying this.

Solution: Any loggers that no longer have a configuration are reset back to INFO. Also, logger changes are applied only once, so there is no undefined behavior of logs during reconfiguration.

@viambot viambot added the safe to test This pull request is marked safe to test from a trusted zone label Jul 25, 2024
@viambot viambot added safe to test This pull request is marked safe to test from a trusted zone and removed safe to test This pull request is marked safe to test from a trusted zone labels Jul 25, 2024
@benjirewis
Copy link
Member

Brought up a discussion offline, but I'm curious if we still want to go down this path of "process all log changes in config/reader.go instead of local_robot.go". Let's talk there before continuing.

Comment on lines 85 to 90
for _, name := range lr.getRegisteredLoggerNames() {
err := lr.updateLoggerLevel(name, INFO)
if err != nil {
return err
}
}
Copy link
Member

@maximpertsov maximpertsov Jul 30, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ disclaimer, this comment only applies if we stick with "processing all log changes in config/reader.go".

let's do the INFO reset after processing changes. i think the current approach has a small race condition where an existing logger configured with a non-INFO level is briefly reset to INFO

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

after processing changes

Sorry which "processing changes" are you referring to? If we put this block at the end of the function, wouldn't everything get set to INFO?

Copy link
Member

@maximpertsov maximpertsov Aug 1, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

sorry, i meant just the loggers that are still in the registry but not in the config configured by the config

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

more specifically, we should keep a count of how many times each registered pattern is configured by a new config pattern, and then reset all registered loggers with 0 configurations back to INFO.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah cool; thanks for the clarification.

@viambot viambot added safe to test This pull request is marked safe to test from a trusted zone and removed safe to test This pull request is marked safe to test from a trusted zone labels Aug 1, 2024
@viambot viambot added safe to test This pull request is marked safe to test from a trusted zone and removed safe to test This pull request is marked safe to test from a trusted zone labels Aug 1, 2024
Copy link
Member

@maximpertsov maximpertsov left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

discussed IRL, we should add a test that demonstrates this flow for a config that has resource log level configs.

@viambot viambot added safe to test This pull request is marked safe to test from a trusted zone and removed safe to test This pull request is marked safe to test from a trusted zone labels Aug 1, 2024
Comment on lines 192 to 195
logger, ok := globalLoggerRegistry.loggerNamed(name)
globalLoggerRegistry.mu.Lock()
defer globalLoggerRegistry.mu.Unlock()
if !ok {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

just noting that a logger can be created after we call loggerNamed but before we grab the write lock to create one ourselves - not sure how big of an issue that is.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
logger, ok := globalLoggerRegistry.loggerNamed(name)
globalLoggerRegistry.mu.Lock()
defer globalLoggerRegistry.mu.Unlock()
if !ok {
globalLoggerRegistry.mu.Lock()
defer globalLoggerRegistry.mu.Unlock()
logger, ok := globalLoggerRegistry.loggers[name]
if !ok {

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

@viambot viambot added safe to test This pull request is marked safe to test from a trusted zone and removed safe to test This pull request is marked safe to test from a trusted zone labels Aug 2, 2024
Copy link
Member

@benjirewis benjirewis left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! Thanks.

@puranamp puranamp merged commit fe69ac0 into viamrobotics:main Aug 5, 2024
19 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
safe to test This pull request is marked safe to test from a trusted zone
Projects
None yet
5 participants