Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
99 commits
Select commit Hold shift + click to select a range
12fcad0
update: add pydantic to deps
synchon Oct 24, 2025
9029f8a
update: introduce DataType enum
synchon Oct 27, 2025
8b997e4
update: introduce StorageType enum
synchon Oct 27, 2025
6a836d9
update: adapt DataType and StorageType for markers
synchon Oct 28, 2025
2de4aeb
update: introduce ExtDep enum
synchon Oct 28, 2025
c523739
update: adapt ExtDep for preprocessors
synchon Oct 28, 2025
f416a38
update: adapt ExtDep for markers
synchon Oct 28, 2025
c8c669f
update: improve typing annotation for MarkerInOutMappings
synchon Oct 28, 2025
5fcf65f
update: adapt DataType and StorageType for remaining markers
synchon Oct 28, 2025
0e8d1a6
update: introduce ConfoundsFormat enum
synchon Oct 28, 2025
0a2e764
update: introduce MatrixKind enum
synchon Oct 28, 2025
532df41
update: adapt MatrixKind
synchon Oct 28, 2025
bc8c132
update: introduce Upsert enum
synchon Oct 29, 2025
715599e
update: adapt Upsert enum
synchon Oct 29, 2025
5fdc283
chore: reorder storage imports
synchon Oct 29, 2025
ceaa06f
update: adapt necessary enums in storage
synchon Nov 10, 2025
ee6d1c4
refactor: make storage interface and impls pydantic models
synchon Nov 10, 2025
8c8cea4
chore: improve docstrings in storage
synchon Nov 10, 2025
88deabd
update: introduce Confounds and Strategy types
synchon Nov 11, 2025
ce7ef3a
update: introduce and adapt SmoothingImpl enum
synchon Nov 11, 2025
dc07c41
update: introduce and adapt SpaceWarpingImpl enum
synchon Nov 11, 2025
78a0856
update: adapt necessary enums in preprocess
synchon Nov 11, 2025
a833c67
refactor: make preprocess interface and impls pydantic models
synchon Nov 11, 2025
a37c82c
chore: improve docstring for BasePreprocessor
synchon Nov 11, 2025
59c587f
chore: add package guard for fMRIPrepConfoundRemover
synchon Nov 11, 2025
b734caa
update: introduce and adapt ReHoImpl
synchon Nov 12, 2025
1adbecd
update: introduce and adapt ALFFImpl
synchon Nov 12, 2025
fa7f1f7
update: adapt ExtDep for remaining markers
synchon Nov 12, 2025
06073d7
update: adapt necessary enums in markers
synchon Nov 12, 2025
2586ce3
refactor: make marker interface and impls pydantic models
synchon Nov 12, 2025
3eae988
refactor: make datareader pydantic model
synchon Nov 12, 2025
d459d54
chore: improve types
synchon Nov 13, 2025
a394074
refactor: make pipeline.utils.check_ext_dependencies validate via pyd…
synchon Nov 13, 2025
7b675bc
update: improve conditional deps declaration for markers
synchon Nov 13, 2025
75a1c91
update: introduce AOMICSpace and AOMICTask
synchon Nov 13, 2025
c3dc983
update: introduce HCP1200Task and HCP1200PhaseEncoding
synchon Nov 13, 2025
34738f9
update: introduce DMCCSession, DMCCTask, DMCCPhaseEncoding and DMCCRun
synchon Nov 13, 2025
d84d987
refactor: make datagrabber interface and impls pydantic models
synchon Nov 13, 2025
67a93a6
refactor: rename PipelineStepMixin validate to validate_component to …
synchon Nov 13, 2025
dffa60e
chore: remove BaseFeatureStorage.get_valid_inputs
synchon Nov 13, 2025
d808b44
update: rename BaseFeatureStroage.validate to validate_input to make …
synchon Nov 13, 2025
e32e635
chore: improve docstrings
synchon Nov 13, 2025
0d79a6d
update: reorder marker params validation in BaseMarker
synchon Nov 13, 2025
daf72c3
fix: make on param for *Aggregation markers None by default
synchon Nov 13, 2025
213f3ca
update: make onthefly lazy during import
synchon Nov 13, 2025
915d1a5
update: fix tests for preprocess
synchon Nov 13, 2025
26a08ce
update: fix tests for markers
synchon Nov 14, 2025
ec9cd4b
update: fix tests for onthefly
synchon Nov 14, 2025
585ffb6
update: fix tests for storage
synchon Nov 17, 2025
3cd5ebd
chore: update tests for PipelineStepMixin
synchon Nov 17, 2025
3296560
update: add aenum as dependency and use it for DataType
synchon Nov 17, 2025
98537c2
update: improve type handling for BaseDataGrabber.get_types
synchon Nov 17, 2025
e7ae2bb
update: fix tests for datagrabber
synchon Nov 17, 2025
ebde81f
update: allow DataType enum to be extended
synchon Nov 17, 2025
0331743
chore: improve docstrings for PatternValidationMixin
synchon Nov 17, 2025
27c479d
chore: fix test for TemporalSNRBase
synchon Nov 17, 2025
686a5f2
chore: reorder import for pipeline
synchon Nov 17, 2025
169a3d9
chore: lint
synchon Nov 17, 2025
fe687e2
update: adapt maps_datagrabber fixture
synchon Nov 17, 2025
44430cd
update: remove redundant tests in PatternValidationMixin
synchon Nov 17, 2025
e005afd
chore: improve tests for PatternValidationMixin
synchon Nov 17, 2025
79372be
update: improve data type registration test
synchon Nov 17, 2025
ada119f
chore: improve type annotation and docstring for PatternValidationMix…
synchon Nov 17, 2025
a6f703e
chore: adapt juseless datagrabbers
synchon Nov 17, 2025
442b387
chore: adapt validate_input for BaseFeatureStorage test
synchon Nov 17, 2025
8e75f53
chore: adapt validate_component for DefaultDataReader test
synchon Nov 17, 2025
724f7ba
update: adapt testing datagrabbers
synchon Nov 17, 2025
3a8944d
chore: update tests for testing datagrabbers
synchon Nov 17, 2025
7fabbf0
update: improve tests for SQLiteFeatureStorage
synchon Nov 17, 2025
e291624
update: move MatrixKind and fix tests for storage.utils
synchon Nov 17, 2025
d66c757
chore: update types for juseless datagrabbers
synchon Nov 17, 2025
a433443
chore: update type for HDF5FeatureStorage.store_matrix
synchon Nov 17, 2025
6626441
update: use pydantic model dump for UpdateMetaMixin
synchon Nov 17, 2025
d3156ea
fix: remove datadir from datagrabber meta when hashing for storage
synchon Nov 17, 2025
a66b6d5
update: add new enums for queue_context
synchon Nov 17, 2025
6f29d6f
update: make GnuParallelLocalAdapter pydantic model
synchon Nov 17, 2025
bb93f70
update: make HTCondorAdapter pydantic model
synchon Nov 17, 2025
e7d0703
chore: update docstring for QueueContextAdapter
synchon Nov 17, 2025
88b6bcd
chore: make configs import lazy
synchon Nov 17, 2025
1c8acfa
chore: make external import lazy
synchon Nov 17, 2025
9286894
chore: add py.typed to typing
synchon Nov 17, 2025
fe5fd2b
chore: lean up tests in configs
synchon Nov 17, 2025
467d5cd
chore: fix method in DataDispatcher
synchon Nov 17, 2025
53cbd1d
chore: improve docstrings
synchon Nov 17, 2025
7afbd17
update: add pydantic.validate_call to get_aggfunc_by_name
synchon Nov 17, 2025
46dac91
docs: update docs and add deps
synchon Nov 17, 2025
6de05aa
chore: add missing imports
synchon Nov 17, 2025
8609c33
chore: add typing_extensions to deps
synchon Nov 17, 2025
db6b178
docs: update extending docs
synchon Nov 17, 2025
d354e58
chore: fix tests for pipeline
synchon Nov 18, 2025
eb8762a
chore: fix tests and data for cli
synchon Nov 18, 2025
f1a064c
chore: fix tests for api
synchon Nov 18, 2025
97f84c6
update: improve constraints for QueueContextEnv
synchon Nov 18, 2025
5d02165
update: add guard for env.name in context adapters
synchon Nov 18, 2025
1b1cf17
docs: update examples
synchon Nov 18, 2025
8af3899
docs: update extending and using docs
synchon Nov 18, 2025
d9061c3
chore: add changelog 364.enh
synchon Nov 18, 2025
5590869
fix: conditional import for Required in queue_context_adapter.py
synchon Nov 18, 2025
df9cc95
fix: conditional import for TypedDict
synchon Nov 18, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion docs/api/onthefly.rst
Original file line number Diff line number Diff line change
Expand Up @@ -5,5 +5,5 @@ On-the-fly
:members:
:imported-members:

.. automodule:: junifer.onthefly.brainprint
.. automodule:: junifer.onthefly._brainprint
:members:
6 changes: 3 additions & 3 deletions docs/builtin.rst
Original file line number Diff line number Diff line change
Expand Up @@ -158,16 +158,16 @@ Available
| (subject-native or other template spaces)
- Done
- 0.0.4
* - ``Smoothing``
* - :class:`.Smoothing`
- | Apply smoothing to data, particularly useful when dealing with
| ``fMRIPrep``-ed data
- In Progress
- :gh:`161`
* - ``TemporalSlicer``
* - :class:`.TemporalSlicer`
- Slice ``BOLD`` data temporally
- | Done
- :gh:`443`
* - ``TemporalFilter``
* - :class:`.TemporalFilter`
- Filter (clean) ``BOLD`` data temporally
- | Done
- :gh:`432`
Expand Down
1 change: 1 addition & 0 deletions docs/changes/newsfragments/364.enh
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
Adopt Pydantic for user-facing core objects to perform automatic validation using typing annotations by `Synchon Mandal`_
5 changes: 5 additions & 0 deletions docs/conf.py
Original file line number Diff line number Diff line change
Expand Up @@ -65,6 +65,8 @@
"sphinx_copybutton", # copy button for code blocks
"sphinxcontrib.mermaid", # mermaid support
"sphinxcontrib.towncrier.ext", # towncrier fragment support
"sphinxcontrib.autodoc_pydantic", # autodoc support for pydantic models
"enum_tools.autoenum", # enum support
]

if use_multiversion:
Expand Down Expand Up @@ -97,6 +99,8 @@
("py:class", "pipeline.Pipeline"), # nilearn
("py:obj", "neurokit2.*"), # ignore neurokit2
("py:obj", "datalad.*"), # ignore datalad
("py:obj", "junifer.*"), # ignore junifer internal
("py:class", "annotated_types.*") # ignore pydantic annotated types
]

# -- Options for HTML output -------------------------------------------------
Expand Down Expand Up @@ -154,6 +158,7 @@
"pandas": ("https://pandas.pydata.org/pandas-docs/dev", None),
# "sqlalchemy": ("https://docs.sqlalchemy.org/en/20/", None),
"scipy": ("https://docs.scipy.org/doc/scipy/", None),
"pydantic": ("https://docs.pydantic.dev/latest/", None),
}

# -- sphinx.ext.extlinks configuration ---------------------------------------
Expand Down
117 changes: 51 additions & 66 deletions docs/extending/datagrabber.rst
Original file line number Diff line number Diff line change
Expand Up @@ -140,29 +140,24 @@ With the variables defined above, we can create our DataGrabber and name it

from pathlib import Path

from junifer.datagrabber import PatternDataGrabber
from junifer.datagrabber import PatternDataGrabber, DataType
from junifer.typing import DataGrabberPatterns


class ExampleBIDSDataGrabber(PatternDataGrabber):
def __init__(self, datadir: str | Path) -> None:
types = ["T1w", "BOLD"]
patterns = {
"T1w": {
"pattern": "{subject}/{session}/anat/{subject}_{session}_T1w.nii.gz",
"space": "native",
},
"BOLD": {
"pattern": "{subject}/{session}/func/{subject}_{session}_task-rest_bold.nii.gz",
"space": "MNI152NLin6Asym",
},
}
replacements = ["subject", "session"]
super().__init__(
datadir=datadir,
types=types,
patterns=patterns,
replacements=replacements,
)

types: list[DataType] = [DataType.T1w, DataType.BOLD]
patterns: DataGrabberPatterns = {
"T1w": {
"pattern": "{subject}/{session}/anat/{subject}_{session}_T1w.nii.gz",
"space": "native",
},
"BOLD": {
"pattern": "{subject}/{session}/func/{subject}_{session}_task-rest_bold.nii.gz",
"space": "MNI152NLin6Asym",
},
}
replacements: list[str] = ["subject", "session"]

Our DataGrabber is ready to be used by ``junifer``. However, it is still unknown
to the library. We need to register it in the library. To do so, we need to
Expand All @@ -175,29 +170,24 @@ use the :func:`.register_datagrabber` decorator.

from junifer.api.decorators import register_datagrabber
from junifer.datagrabber import PatternDataGrabber
from junifer.typing import DataGrabberPatterns


@register_datagrabber
class ExampleBIDSDataGrabber(PatternDataGrabber):
def __init__(self, datadir: str | Path) -> None:
types = ["T1w", "BOLD"]
patterns = {
"T1w": {
"pattern": "{subject}/{session}/anat/{subject}_{session}_T1w.nii.gz",
"space": "native",
},
"BOLD": {
"pattern": "{subject}/{session}/func/{subject}_{session}_task-rest_bold.nii.gz",
"space": "MNI152NLin6Asym",
},
}
replacements = ["subject", "session"]
super().__init__(
datadir=datadir,
types=types,
patterns=patterns,
replacements=replacements,
)

types: list[DataType] = [DataType.T1w, DataType.BOLD]
patterns: DataGrabberPatterns = {
"T1w": {
"pattern": "{subject}/{session}/anat/{subject}_{session}_T1w.nii.gz",
"space": "native",
},
"BOLD": {
"pattern": "{subject}/{session}/func/{subject}_{session}_task-rest_bold.nii.gz",
"space": "MNI152NLin6Asym",
},
}
replacements: list[str] = ["subject", "session"]


Now, we can use our DataGrabber in ``junifer``, by setting the ``datagrabber``
Expand Down Expand Up @@ -259,35 +249,30 @@ And we can create our DataGrabber:

.. code-block:: python

from pathlib import Path

from junifer.api.decorators import register_datagrabber
from junifer.datagrabber import PatternDataladDataGrabber
from pydantic import HttpUrl


@register_datagrabber
class ExampleBIDSDataGrabber(PatternDataladDataGrabber):
def __init__(self) -> None:
types = ["T1w", "BOLD"]
patterns = {
"T1w": {
"pattern": "{subject}/{session}/anat/{subject}_{session}_T1w.nii.gz",
"space": "native",
},
"BOLD": {
"pattern": "{subject}/{session}/func/{subject}_{session}_task-rest_bold.nii.gz",
"space": "MNI152NLin6Asym",
},
}
replacements = ["subject", "session"]
uri = "https://gin.g-node.org/juaml/datalad-example-bids"
rootdir = "example_bids_ses"
super().__init__(
datadir=None,
uri=uri,
rootdir=rootdir,
types=types,
patterns=patterns,
replacements=replacements,
)

uri: HttpUrl = HttpUrl("https://gin.g-node.org/juaml/datalad-example-bids")
types: list[DataType] = [DataType.T1w, DataType.BOLD]
patterns: DataGrabberPatterns = {
"T1w": {
"pattern": "{subject}/{session}/anat/{subject}_{session}_T1w.nii.gz",
"space": "native",
},
"BOLD": {
"pattern": "{subject}/{session}/func/{subject}_{session}_task-rest_bold.nii.gz",
"space": "MNI152NLin6Asym",
},
}
replacements: list[str] = ["subject", "session"]
rootdir: Path = Path("example_bids_ses")

This approach can be used directly from the YAML, like so:

Expand Down Expand Up @@ -376,8 +361,8 @@ need to implement the following methods:

.. note::

The ``__init__`` method could also be implemented, but it is not mandatory.
This is required if the DataGrabber requires any extra parameter.
If the DataGrabber requires any extra parameter, they could be defined as
class attributes.

We will now implement our BIDS example with this method.

Expand Down Expand Up @@ -494,8 +479,8 @@ more information about the format of the confounds file. Thus, the
``BOLD.confounds`` element is a dictionary with the following keys:

- ``path``: the path to the confounds file.
- ``format``: the format of the confounds file. Currently, this can be either
``fmriprep`` or ``adhoc``.
- ``format``: the format of the confounds file. Check :enum:`.ConfoundsFormat`
for options.

The ``fmriprep`` format corresponds to the format of the confounds files
generated by `fMRIPrep`_. The ``adhoc`` format corresponds to a format that is
Expand Down
20 changes: 9 additions & 11 deletions docs/extending/dependencies.rst
Original file line number Diff line number Diff line change
Expand Up @@ -46,7 +46,7 @@ by having a class attribute like so:

_EXT_DEPENDENCIES: ClassVar[ExternalDependencies] = [
{
"name": "afni",
"name": ExtDep.AFNI,
"commands": ["3dReHo", "3dAFNItoNIFTI"],
},
]
Expand All @@ -55,7 +55,7 @@ The above example is taken from the class which computes regional homogeneity
(ReHo) using AFNI. The general pattern is that you need to have the value of
``_EXT_DEPENDENCIES`` as a list of dictionary with two keys:

* ``name`` (str) : lowercased name of the toolbox
* ``name`` (:enum:`.ExtDep`) : name of the toolbox
* ``commands`` (list of str) : actual names of the commands you need to use

This is simple but powerful as we will see in the following sub-sections.
Expand All @@ -81,30 +81,28 @@ that it shows the problem a bit better and how we solve it:
_CONDITIONAL_DEPENDENCIES: ClassVar[ConditionalDependencies] = [
{
"using": "fsl",
"depends_on": FSLWarper,
"depends_on": [FSLWarper],
},
{
"using": "ants",
"depends_on": ANTsWarper,
"depends_on": [ANTSWarper],
},
{
"using": "auto",
"depends_on": [FSLWarper, ANTsWarper],
},
]

def __init__(
self, using: str, reference: str, on: Union[List[str], str]
) -> None:
# validation and setting up
...
using: str
reference: str
on: List[DataType]


Here, you see a new class attribute ``_CONDITIONAL_DEPENDENCIES`` which is a
list of dictionaries with two keys:

* ``using`` (str) : lowercased name of the toolbox
* ``depends_on`` (object or list of objects) : a class or list of classes which \
* ``depends_on`` (list of objects) : list of classes which \
implements the particular tool's use

It is mandatory to have the ``using`` positional argument in the constructor in
Expand All @@ -128,7 +126,7 @@ similar. ``FSLWarper`` looks like this (only the relevant part is shown here):

_EXT_DEPENDENCIES: ClassVar[ExternalDependencies] = [
{
"name": "fsl",
"name": ExtDep.FSL,
"commands": ["flirt", "applywarp"],
},
]
Expand Down
Loading
Loading