-
Notifications
You must be signed in to change notification settings - Fork 2.1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
fix thread unsafety in occasional logging when using std::atomic #808
fix thread unsafety in occasional logging when using std::atomic #808
Conversation
src/glog/logging.h.in
Outdated
++LOG_OCCURRENCES; \ | ||
if (++LOG_OCCURRENCES_MOD_N > n) LOG_OCCURRENCES_MOD_N -= n; \ | ||
if (LOG_OCCURRENCES_MOD_N == 1) \ | ||
int LOG_OCCURRENCES_MOD_N = LOG_OCCURRENCES.fetch_add(1, std::memory_order_relaxed) % n; \ |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It is not clear to me why changing to fetch_add
provides a solution here. op++
also calls fetch_add
albeit with different memory ordering. Consequently, you cannot avoid the problem you are trying to solve. To be able to, you would need to perform all the involved operations atomically by protecting concurrent access in a critical section.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The problem was "a series of atomic operations of LOG_OCCURRENCES_MOD_N
, but in whole, they are not atomic, which unexpected". Here only the LOG_OCCURRENCES
is static, the only place we need to protect. Specifically, we only do one atomic operation fetch_add
on LOG_OCCURRENCES
.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
My question is: in what way do the changes fix the problem? The operations are still not atomic as a whole.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The operations are still not atomic as a whole.
Yes. But they don't need to be. There is only one operation that needs to be atomic, access to the static variable LOG_OCCURRENCES
.
in what way do the changes fix the problem?
For every use of this macro (under different threads), a uniqueLOG_OCCURRENCES
value is assigned, incremented by 1. We use a local variable LOG_OCCURRENCES_MOD_N
to get the reminder of % n
. By now, there is no need for protection or being atmoic. It can be written like
int LOG_OCCURRENCES_MOD_N = LOG_OCCURRENCES.fetch_add(1, std::memory_order_relaxed);
if (LOG_OCCURRENCES_MOD_N % n == 1)
They are the same.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
You cannot do that. Once LOG_OCCURRENCES
overflows you will hit undefined behavior.
Also, the change from op++
to fetch_add
is not clear to me. Why is this necessary? If you need the previous value, you can use the post increment.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The main idea is to promise every call can have the exact value in the if
condition statements to represent if it should be logged. When the static atomic variable is fetch_add
to a local variable, the state is recorded. You'll never get the same value.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Once LOG_OCCURRENCES overflows you will hit undefined behavior.
You're right. This may need more consideration.
the change from
op++
tofetch_add
is not clear to me. Why is this necessary?
int val = op++; // not atomic, there are two operations. self increment and assignment. ++op is the same
int val = op.fetch_add(1) // atomic
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
That's not correct. With post increment you will get the previous value since it essentially calls fetch_add
as well.
Please refer to the documentation of std::atomic::op++
before we continue the discussion. Thanks.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Sorry, It's my bad. You're right. I agree there is no need to use fetch_add
. ++
is enough.
Once LOG_OCCURRENCES overflows you will hit undefined behavior.
Use atomic_uint
instead. Unsigned integer overflow is not a UB. Will this help?
Since a fix is not straightforward to get right, I suggest that you add unit tests. |
I make some updates. Please take a look if you have time. |
I'm very sorry for the delay. I will look at the PR right after 0.6 release. |
No problem. |
I'll close the PR since it's heavily diverged from |
This PR is to address #804
Note:
LOG_OCCURRENCES_MOD_N
as a local, non-atomic variable and usefetch_add % n
to get the exact modulo remainder.SOME_KIND_OF_LOG_EVERY_N
from0
to1
sincefetch_add
will get the previous value.n == 1
.LOG_OCCURRENCES_MOD_N
in the macroSOME_KIND_OF_LOG_FIRST_N
. Although the name seems unrelated, I prefer not to introduce another macro to name it.AnnotateBenignRaceSized
for variables unlikely to be under a multi-thread situation.I only fix the atomic version and I have no idea if other versions have a similar problem.
Would you mind taking a look? @sergiud @drigz Thanks.
Aside question:
I doubt if
AnnotateBenignRaceSized
will take effect onstd::atomic
since no data race will happen.