Feature add s3rdma backend placeholder in IO framework - #181
Draft
ran-yuan-rui wants to merge 6 commits into
Draft
Feature add s3rdma backend placeholder in IO framework#181ran-yuan-rui wants to merge 6 commits into
ran-yuan-rui wants to merge 6 commits into
Conversation
ran-yuan-rui
changed the title
ran-yuan-rui
force-pushed
the
feature-s3rdma-s1-seams
branch
from
August 6, 2026 13:09
7ab7c7e to
4098802
Compare
A new backend identifier for an S3-over-RDMA ioctx. The registry is map-driven (no switch over the enum), so the value is inert until a backend registers under it.
Listing is prefix resolution on the control plane, independent of how a backend's data plane reads the resolved keys. A glob / LIST layer can now depend on this interface instead of a concrete ioctx type. rest_ioctx implements it in the same change (its existing list_objects_paged / list_max_matches become overrides — no behavior change), so the interface lands with a production implementation. The ListObjectsV2 page/entry types stay in rest/s3: they are S3-protocol shapes, not REST-transport shapes; the include is documented as such.
A protected virtual with an empty default body, called from the catch of both device-plane dispatch bodies before the failure is softened into an errored future. A backend whose contract makes a poisoned CUDA context process-fatal probes context health in its override; every existing backend inherits the no-op and is behavior-identical. Not called on success paths or on the empty-reactor-pool return (no exception there).
Hands one type's registration to a new backend type in a single stroke — unlike an unregister+register pair there is no observable no-claimant gap and no call-it-early-enough discipline left to convention. Throws invalid_argument on a missing old type, an already-registered new type, or null checker/factory, with the strong guarantee (the emplace precedes an erase by key — an iterator would not survive a rehash). The registry latches its first lookup_path; replace_ioctx afterwards throws logic_error (bootstrap is single-threaded by contract).
Reserves the S3-over-RDMA backend's name and surface: an ioctx subclass with a deleted constructor (pinned by a static_assert) and every override declared but not defined — non-constructible, claiming no paths — plus a placeholder translation unit so CI compiles the header. The backend is under active development downstream against the seams this framework carries (the s3rdma context type, object_store_listing, the dispatch-failure hook, replace_ioctx) and will replace this declaration wholesale when it is contributed upstream.
…ject_store_listing Registry: replace success re-routes s3:// and proves the old claimant is gone (a counting factory registered over it stays at zero calls); invalid_argument on missing old / registered new / null checker / null factory, each leaving routing unchanged; logic_error after the first lookup_path; register_ioctx stays legal after lookup. Hook: a stub reactor drives both device-plane dispatch bodies through prep- and enqueue-stage throws (hook fires exactly once per failed dispatch), empty reactor selection and success paths (no call), and the non-overriding default (error future preserved). Listing: rest_ioctx exercised purely through the object_store_listing interface against a test-local scripted LIST server (multi-page with continuation tokens, query recording; the accept thread owns a fixed fd, shut down before join): pagination, early-stop sink, on-the-wire max-keys clamp, the max_scanned cap, and the configured match cap.
ran-yuan-rui
force-pushed
the
feature-s3rdma-s1-seams
branch
from
August 6, 2026 14:40
4098802 to
7bc2084
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Working in progress.