-
Notifications
You must be signed in to change notification settings - Fork 327
goroutine leak detected #1191
Comments
We can't fully enable this due to an issue with a downstream library, but we did find a couple leaks to patch in our code. Issue: census-instrumentation/opencensus-go#1191
Here is a workaround that I used:
|
Just wanted to bump one and add a little context — a lot of us are running goleak in our tests because it's easy to make mistakes around leaking goroutines in Go, and once you have enough code written in a project, you probably are somewhere, and goleak is a great way to hedge against that and protect yourself against bad production problems before they happen. We're not using this module directly, but it comes in transitively via a Google module we're using, and even when we don't use that Google module directly, this package's goroutine still starts because it's in an I think you'd find reasonably good consensus (pun intended) out there that it's bad form to (1) starting goroutines in If you're open to it, even moving the goroutine to start lazily on first use would be an improvement. Google's Go APIs that are using OpenCensus provide a |
@brandur That all sounds reasonable. Given the state of the project, we have to be careful not to break existing users, but lazily starting the goroutine should be backwards-compatible. Let me know if you are planning to put up a PR to fix it. |
See discussion in census-instrumentation#1191 for impetus, but in general, it's not particularly good practice to start up goroutines in package `init` functions, with one reason being that it means that they're started for projects that don't even make use of the package, and maybe just have it as a transitive dependency. So here, remove the `view` package's default worker in favor of a new function that'll lazily initialize one when any package-level functions are called that use the default worker. We use a `sync.Once` that has an optimized short-circuit path in case the work's already been done, so new overhead should be negligible, and registering/unregistering views is probably not a particularly common operation anyway. This change is backwards compatible, with use of the package's API staying exactly the same. Test coverage seems to be pretty good, with a function that previously reset the default worker between test cases which I've modified to instead reset the default worker back to its uninitialized state, and thereby verify that the test cases can initialize it if necessary. I thought about trying to deinitialize the default worker in case its last view is unregistered, but decided against it because (1) it's more invasive, (2) it's not likely to actually help anyone in practice as views probably stay registered for a program's entire duration anwyay, and (3) its interaction with the existing package-level `Stop` function would take some thinking through.
@dashpole I opened a pull request at #1287 in case you have a change to take a look. I think it fulfills the backwards compatibility requirement. @adamdecaf One thing I noticed while in there is that there is a package-level |
Please answer these questions before submitting a bug report.
What version of OpenCensus are you using?
v0.22.0
What version of Go are you using?
go1.13.6
What did you do?
I was running the new goleak tool on my codebase and ran into a leak coming from OpenCensus.
What did you expect to see?
I expected no goroutine leaks in libraries I depend on.
What did you see instead?
A crash from goleaks.
Additional context
I'm pulling this library in via gocloud.dev/secrets
v0.17.0
, but that doesn't matter here sincev0.22.0
of OpenCensus is used in both.This appears to be coming from internal global state of a default recorder being set.
https://github.com/census-instrumentation/opencensus-go/blob/v0.22.0/stats/view/worker.go#L30-L34
Also, there doesn't seem to be a way to retrieve this reporter to shutdown on application shutdown.
https://github.com/census-instrumentation/opencensus-go/search?q=DefaultRecorder&unscoped_q=DefaultRecorder
The text was updated successfully, but these errors were encountered: