Study-first direction — docs RFC, implementation to follow (draft) by asmacdo · Pull Request #101 · con/mechababs

asmacdo · 2026-07-24T13:35:24Z

Study-first direction — draft RFC. Reframes mechababs around the study as the primary unit (campaign optional, a superstudy as an optional coordinator). Docs land first to align on shape; implementation follows on this branch once feedback settles the direction. This is a feature branch — the study-first reshape is cohesive and won't go to main piecemeal. Open questions are flagged inline with 💬.

What's here so far (docs / the RFC)

docs/use_cases.md (new) — the user stories mechababs serves: the requirements spine the rest of the design answers to (single-study/no-campaign, produce-one-derivative-easily, sweep-many, extend-a-foreign-tool-study, cross-cluster, …).
docs/output_structure.md — rewritten study-first: study-<id>/ is primary and holds .mechababs/campaigns/<label>/ (a campaign is a config-epoch run, explicitly not a dataset); superstudy/ is an optional study-of-studies coordinator whose durable home is optional; the derivative's prov Bundle now points at the study.
docs/quickstart.md (new) — aspirational walkthrough of the intended UX (uvx-based bootstrap, uv.lock as just-in-time provenance, cwd-scoped CLI).

💬 Open questions (feedback wanted)

Creating a study (and generating its metadata files) for users outside OpenNeuro — today studies come ready-made from OpenNeuroStudies; this is the barrier to entry.
studies.tsv — the superstudy catalog name is overloaded with OpenNeuroStudies' own index.
.mechababs/derivative-attempts/ — should retired derivatives live in (and travel with) a published study?
prov/ shape under study-first — where the Bundle points.
add-dataset vs add-study — a study can hold more than one source dataset.
Campaign selection + environment enforcement — how mechababs knows which campaign, and how it refuses to run with the wrong pinned tools (parked in quickstart.md's 💬).

What follows

The docs are the RFC. Once feedback aligns the shape and the 💬 questions settle, the implementation lands here — the bootstrap rewrite (uvx entry + uv.lock, no global install), the study-first CLI (campaign init, cwd-scoping), per-study state, and the rest. Until then output_structure.md is a target and quickstart.md is explicitly aspirational (🚧); the current campaign-first CLI is what runs today.

Background

Emerged from a design session reframing the campaign-centric model toward study-first, driven by generalizability feedback — most users work in a single BIDS study rather than a large sweep — and the goal of lowering the barrier to entry for other labs. The three docs capture the requirements (use cases), the target shape (output structure), and the intended experience (quickstart), with the genuinely-unresolved pieces surfaced as 💬 rather than papered over.

Seven user stories that form the requirements spine the study-first direction answers to: sweep many pre-made studies, work in a single study with no campaign, author a study from assorted source datasets, add derivatives later, clone-and-extend (own and foreign-tool studies), and move a run to another cluster. A mix of what works today and what we are building toward; the list is expected to grow. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

Rewrite output_structure.md around the study as the primary unit: - study-<id>/ holds derivatives + a .mechababs/campaigns/<label>/ record (a campaign is a config-epoch run, explicitly NOT a dataset) - superstudy/ is an optional study-of-studies coordinator; homing it is optional (orchestration prov lives in the member studies) - derivative + prov + publishing carried over; prov Bundle now points at the study rather than a separate campaign dataset - open questions flagged with 💬 (create-a-study gap + metadata files, derivative-attempts placement, studies.tsv name overload, prov shape) use_cases.md: add single-study/no-campaign, produce-one-derivative-easily (with the config-reuse + orchestration-provenance framing and a nipoppy nod for the pedagogical case), foreign-tool study, add-dataset-mid-campaign, source-dataset-changes (💬), and crippled-filesystem stories; add the 💬 needs-feedback legend. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

Walks the intended UX: uvx-from-a-pinned-ref bootstrap (no global install, no invented syntax — plain uv), a per-dataset .mechababs/ env + uv.lock as just-in-time provenance, familiar source-the-venv daily use, cwd-scoped iterate/status, and campaign-by-label. Flags the unresolved campaign-selection and env-enforcement mechanics with a 💬 needs-discussion block. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

yarikoptic

good use cases! added refinements and TODOs

yarikoptic · 2026-07-24T14:26:41Z

+
+If your goal is to *learn* how derivatives are produced rather than to produce one, the [nipoppy](https://nipoppy.readthedocs.io) project (McGill) is designed for exactly that — it teaches the user how to do these things step by step.
+mechababs optimizes for producing a self-contained, reproducible object; nipoppy optimizes for teaching the process.
+They are complementary.


during conversation came up -- may be there is a way to "simplify" or template simple cases.

read on

reproman has description of resources -- we should be able to describe cluster once e.g. available for a system/user/dataset (might be already done through templating) .

how nipoppy uses the registry of bids-apps -- they and BIDS-Apps 2.0 use boutique descriptors which describe interfaces. There is more to it -- review pointers etc within https://github.com/bids-standard/2026-os4ls-bids/blob/master/FULL.md and https://bids-standard.github.io/execution-spec/scope.html in particular (for BIDS Apps 2.0)

with those 2 it should actually be quite easy to do "simple"

yarikoptic · 2026-07-24T14:29:05Z

+They are complementary.
+
+## Author a study from assorted source datasets
+As a mechababs user, I want to create a BIDS study containing a variety of source datasets of different types.


for "create a BIDS study" we really need to establish the ultimate template to reuse. Note that we do have already https://github.com/brain-bbqs/study-template or bids-copier I have or ...

@lobennett this could potentially ease your pain, though I don't think this generates the openneurostudy-style tsvs

thanks, I'll check it out

yarikoptic · 2026-07-24T14:36:17Z

+
+## Handle a source dataset that changes mid-campaign 💬
+As a user, I can handle a source dataset changing after processing has started — new subjects or sessions, or changed data on subjects/sessions already processed.
+This is likely handled at the BABS level rather than in mechababs; needs discussion.


changes could be

added subject/session -- simple

changed file(s) in subject/session -- again git would be of help. That is where we might want to consolidate what I am doing in openneurostudies with snakemake potentially on how to interface and represent such state changes across submodule hierarchy in a uniform fashion for our "workflow/campaign" helpers

lobennett · 2026-07-24T15:14:19Z

Two use cases from running mechababs off-OpenNeuro on an institutional HPC (Stanford Sherlock) — the first is the demand behind the study-authoring 💬 in output_structure.md; the second is more of a nice-to-have. A couple for use_cases.md:

Process a lab's own datasets, off OpenNeuro

As a lab with our own (non-OpenNeuro) BIDS datasets, we want to run mechababs across several in-house cohorts on our institution's cluster and get the same self-contained, reproducible studies + derivatives — without our data being on OpenNeuro or our studies pre-authored by its tooling. Today we hand-wrap each raw dataset as a study and generate the per-subject datatypes/counts TSV that selection reads; it works, but it's the main barrier to entry. This is the demand behind the study-authoring gap in output_structure.md: first-class study creation from local raw data.

Gate an expensive pipeline on QC

As a user, I want to run a cheap QC app (MRIQC) first and only run the expensive stages (fMRIPrep anat → minimal/full) on the inputs that pass. The pass/fail verdict is produced outside mechababs, by review or by an automated rule. What would be helpful is a recorded way to subtract the excluded subjects/sessions from the downstream selection. I currently express this by curating the source selection by hand.

`mechababs test-cluster` (#98) has to find the e2e scenario wherever mechababs is installed, not only in a source checkout. A suite in `tests/` reaches a campaign today only because bootstrap.sh clones the whole repo into `code/mechababs`; once the code is referenced and locked rather than cloned in — the direction sketched in the study-first redesign (#101), where `.mechababs/` pins the tools in a `uv.lock` and bootstrap runs them via `uvx` — there is no checkout to reach into and `tests/` is not in a wheel. So move it to `mechababs/testing/e2e/` and ship it as package DATA. Data rather than an importable subpackage because pytest collects from a path, which keeps an `__init__.py` out of a directory of test modules. `mechababs.testing.suite_path()` resolves it through `importlib.resources` and fails loudly on an incomplete install, rather than letting pytest report "no tests collected" much later. Packaging needs care here: setuptools_scm's file finder plus the default include-package-data would ship every git-tracked file under the package, so the `package-data` globs are additive and cannot keep anything out. The dev wrapper scripts are removed with an explicit `exclude-package-data` — they only make sense from a checkout — and a test pins both halves of that. `testpaths` is now required: without it a bare `pytest` also walks `mechababs/`, where the packaged conftest registers `--cluster-config` a second time and collection fails outright. This commit only relocates the suite. `tests/` becomes the unit suite alone: it tests the code and never leaves the repo, so it has no reason to travel. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

yarikoptic · 2026-07-31T20:18:50Z

+  study-<id>/                  # member studies, at root
+  study-<id>/


Suggested change

study-<id>/ # member studies, at root

study-<id>/

study-<id1>/ # member studies, at root

study-<id2>/

? or just remove one?

From the discussion with Yarik on mega-studies: studies live in their remote and are installed only while there is work to do on them. - act on one study within a superstudy (scoping, not exclusivity) - release a finished study; mechababs neither pushes nor is disturbed by it - see the state of the set without holding the data - run under a finite disk/inode budget - choose what gets worked next Use cases only, no implementation. The chaining case is deliberately absent -- it turned out not to be about eviction.

Co-authored-by: Yaroslav Halchenko <debian@onerussian.com>

- QC gate as a prerequisite to the expensive stages, and lab-owned off-OpenNeuro datasets folded into study authoring (both from Logan's Sherlock experience) - consolidating a multi-cluster campaign back into one superstudy - normalize on 'superstudy'; distinct member paths in the superstudy tree

From Yarik's review on PR con#101: dashboarding that works at every level, summaries derived from the level beneath rather than maintained beside it, and state files conventional enough for other tools to read. His terminology point (leaf-to-root vs 'bottom-up') is a doc-wide vocabulary question, not a use case -- left for output_structure.md.

Yarik asked for terminology alignment. Marked 💬 -- the definitions are a proposal, not a decision, and the leaf-to-root direction is the part most likely to be argued.

asmacdo · 2026-08-05T16:21:43Z

+      <label>/                 # a campaign authored here and fanned out to the members
+        campaign.yaml          #   the bundle + `studies:` (inline list OR selected-studies.tsv)
+        bids-app-configs/  clusters/  pyproject.toml  uv.lock
+        selected-studies.tsv   #   the (study, sourcedata) subset this campaign runs on


studies+sourcedata.tsv

yarikoptic · 2026-08-05T16:40:05Z

+  sourcedata/<id>/             # submodule -> raw BIDS dataset  (a study may hold more than one)
+  derivatives/
+    <Tool>-<Ver>/              # pre-existing derivatives (any tool)
+    <Tool>-<Ver>+<stage>/      # a new babs derivative (see "a derivative dataset")


if it is rawbids or sourcedata/raw then like above, if it is indeed some kind of sourcedata/<id> then we have to (in case of multiple) or might want to (in case it becomes multiple!) you could do either

Suggested change

<Tool>-<Ver>+<stage>/ # a new babs derivative (see "a derivative dataset")

<Tool>-<Ver>+<stage>/ # a new babs derivative (see "a derivative dataset")

<id>

or

Suggested change

<Tool>-<Ver>+<stage>/ # a new babs derivative (see "a derivative dataset")

<Tool>-<Ver>+<stage>+<id>/ # a new babs derivative (see "a derivative dataset")

or anything else

asmacdo · 2026-08-06T13:22:32Z

+study-<id>/
+  dataset_description.json     # DatasetType "study"; authored upstream (or at creation)
+  README.md
+  sourcedata/<id>/             # submodule -> raw BIDS dataset  (a study may hold more than one)


Currently relies on 1 sourcedata per study, and this works. If we need to expand that to multiple sourcedatas per study, it may be as simple as --add-dataset accepting a study, which selects the 1 dataset in the sourcedata/ slot, or optionally accepting a path to the dataset inside the study. This info would need to be added to the statefile, and in superstudy-mode, we'd need that mapping in the superstudy mapping as well.

Suggested change

sourcedata/<id>/ # submodule -> raw BIDS dataset (a study may hold more than one)

sourcedata/<id>/ # submodule -> raw BIDS dataset (a study may hold more than one)

rawbids/ # this should be preferable! above in case if it is not raw or have multiple

asmacdo and others added 3 commits July 23, 2026 15:31

asmacdo commented Jul 24, 2026

View reviewed changes

Comment thread docs/use_cases.md

yarikoptic requested changes Jul 24, 2026

View reviewed changes

This was referenced Jul 25, 2026

mechababs test-cluster: validate a cluster from the campaign, not env-vars + a repo checkout #98

Closed

mechababs test-cluster: validate a cluster from the campaign #104

Merged

yarikoptic reviewed Jul 31, 2026

View reviewed changes

asmacdo and others added 5 commits August 5, 2026 09:27

Apply suggestions from code review

af3a708

Co-authored-by: Yaroslav Halchenko <debian@onerussian.com>

docs(use_cases): define the terms the use cases lean on

69cef90

Yarik asked for terminology alignment. Marked 💬 -- the definitions are a proposal, not a decision, and the leaf-to-root direction is the part most likely to be argued.

asmacdo commented Aug 5, 2026

View reviewed changes

yarikoptic reviewed Aug 5, 2026

View reviewed changes

asmacdo commented Aug 6, 2026

View reviewed changes

yarikoptic mentioned this pull request Aug 7, 2026

mechababs presentation #112

Open

	<Tool>-<Ver>+<stage>/ # a new babs derivative (see "a derivative dataset")
	<Tool>-<Ver>+<stage>/ # a new babs derivative (see "a derivative dataset")
	<id>

	<Tool>-<Ver>+<stage>/ # a new babs derivative (see "a derivative dataset")
	<Tool>-<Ver>+<stage>+<id>/ # a new babs derivative (see "a derivative dataset")

	sourcedata/<id>/ # submodule -> raw BIDS dataset (a study may hold more than one)
	sourcedata/<id>/ # submodule -> raw BIDS dataset (a study may hold more than one)
	rawbids/ # this should be preferable! above in case if it is not raw or have multiple

Uh oh!

Conversation

asmacdo commented Jul 24, 2026

What's here so far (docs / the RFC)

💬 Open questions (feedback wanted)

What follows

Uh oh!

Uh oh!

yarikoptic left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

lobennett commented Jul 24, 2026

Process a lab's own datasets, off OpenNeuro

Gate an expensive pipeline on QC

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants