Have the write state emit its own stable version (aka last_gc_version today) #138
Description
Right now, GC of deleted keys happens on all nodes independently.
This works fine, but after a reset, a node may need many gossip round trips to be up to date again.
It might get a last_gc_version of V first and then talk with another peer, be unlucky and get reset to a last_gc_version V',
and have this bad luck keep going on over and over again.
If instead, writer nodes were emitting GC version as a KV key, we would probably have one or two last_gc_version over an entire cluster.
If we updated that key only every 3 minutes or so, we would have the guarantee that a node playing catch up would be able to in at most 2 resets. (and usually a single one would suffice)
In addition, all nodes would not have to keep track of delete timestamps. Instead, only the writing node would keep a queue of GC versions.