persist-txn: compaction #22619
persist-txn: compaction #22619
Conversation
danhhz
force-pushed
the
persist_txn_compaction
branch
3 times, most recently
from
e217486
to
ebcedf0
Compare
| self.txns_cache.update_gt(&since_ts).await; | ||
| self.txns_cache.compact_to(&since_ts); |
There was a problem hiding this comment.
Does it matter that we do this before determining the actual min_unapplied_ts? For example if a since_ts of 100 is passed in, but then we reduce it to 50. The in memory cache will be compacted to 100, while the physical txn shard will only be compacted to 50.
There was a problem hiding this comment.
Yup, that's actually exactly what the "It's always safe" comment above is trying to say is okay, so this is a good hint that I should reword it somehow
src/persist-txn/src/operator.rs
Outdated
There was a problem hiding this comment.
You probably want a second pair of eyes on this that are more familiar with the code.
There was a problem hiding this comment.
Ha, it's just you and aljoscha, so you might have to be it. None of the timely/dataflow bits are changing in this PR, just debugging, the unblock_read change, and various compaction related things. Happy to walk you through it on a huddle though (maybe Monday?)
There was a problem hiding this comment.
I just went through it, it seems pretty straightforward and uncontroversial.
| self.txns_cache.update_gt(&since_ts).await; | ||
| self.txns_cache.compact_to(&since_ts); |
src/persist-txn/src/txns.rs
Outdated
| /// Allows compaction to the txns shard as well as internal representations, | ||
| /// losing the ability to answer queries about times less_than since_ts. | ||
| /// | ||
| /// only call this from the singleton controller process |
There was a problem hiding this comment.
Whoops, fixed this fragment
danhhz
force-pushed
the
persist_txn_compaction
branch
from
ebcedf0
to
343b96e
Compare
|
Whoops, pulled in something from another branch. Fixing |
Compaction of data shards is initially delegated to the txns user (the storage controller). Because txn writes intentionally never read data shards and in no way depend on the sinces, the since of a data shard is free to be arbitrarily far ahead of or behind the txns upper. Data shard reads, when run through the above process, then follow the usual rules (can read at times beyond the since but not beyond the upper). Compaction of the txns shard relies on the following invariant that is carefully maintained: every write less than the since of the txns shard has been applied. Mechanically, this is accomplished by a critical since capability held internally by the txns system. Any txn writer is free to advance it to a time once it has proven that all writes before that time have been applied. It is advantageous to compact the txns shard aggressively so that applied writes are promptly consolidated out, minimizing the size. For a snapshot read at `as_of`, we need to be able to distinguish when the latest write `<= as_of` has been applied. The above invariant enables this as follows: - If `as_of <= txns_shard.since()`, then the invariant guarantees that all writes `<= as_of` have been applied, so we're free to read as described in the section above. - Otherwise, we haven't compacted `as_of` in the txns shard yet, and still have perfect information about which writes happened when. We can look at the data shard upper to determine which have been applied.
danhhz
force-pushed
the
persist_txn_compaction
branch
from
343b96e
to
d381993
Compare
Compaction of data shards is initially delegated to the txns user (the storage controller). Because txn writes intentionally never read data shards and in no way depend on the sinces, the since of a data shard is free to be arbitrarily far ahead of or behind the txns upper. Data shard reads, when run through the above process, then follow the usual rules (can read at times beyond the since but not beyond the upper).
Compaction of the txns shard relies on the following invariant that is carefully maintained: every write less than the since of the txns shard has been applied. Mechanically, this is accomplished by a critical since capability held internally by the txns system. Any txn writer is free to advance it to a time once it has proven that all writes before that time have been applied.
It is advantageous to compact the txns shard aggressively so that applied writes are promptly consolidated out, minimizing the size. For a snapshot read at
as_of, we need to be able to distinguish when the latest write<= as_ofhas been applied. The above invariant enables this as follows:as_of <= txns_shard.since(), then the invariant guarantees that all writes<= as_ofhave been applied, so we're free to read as described in the section above.as_ofin the txns shard yet, and still have perfect information about which writes happened when. We can look at the data shard upper to determine which have been applied.Touches MaterializeInc/database-issues#6685
Motivation
Tips for reviewer
Checklist
$T ⇔ Proto$Tmapping (possibly in a backwards-incompatible way), then it is tagged with aT-protolabel.