Skip to content

Commit

Permalink
Merge pull request #100 from ocefpaf/remove_StreamConfig
Browse files Browse the repository at this point in the history
Remove stream config
  • Loading branch information
ocefpaf authored Mar 16, 2024
2 parents daba0ee + a8e78aa commit c8d7caa
Show file tree
Hide file tree
Showing 3 changed files with 7 additions and 157 deletions.
121 changes: 0 additions & 121 deletions docs/source/examples/usage/StreamConfig.ipynb

This file was deleted.

33 changes: 2 additions & 31 deletions docs/source/usage.rst
Original file line number Diff line number Diff line change
Expand Up @@ -69,8 +69,7 @@ Configurations

Configuration objects represent a collection of quality control tests to run and the parameters for each one. There are three main types of `Config` objects:

- StreamConfig_: This configures QC tests for a single stream of data like a ``list``, ``tuple``, ``numpy.ndarray``, ``dask.array``, ``pandas.Series``, ``netCDF4.Variable``, or ``xarray.DataArray``. This can be used standalone, or as a building block for the following more complex configs.
- ContextConfig_: This defines a collection of ``StreamConfig`` objects. These can be applied to multiple variables provided in a ``pandas.DataFrame``, ``dask.DataFrame``, ``netCDF4.Dataset``, or ``xarray.Dataset``. Optionally, these configs can be constrained to specific time domains (``windows``) -- and/or spatial domains (``regions``).
- ContextConfig_: This defines a collection of ``ConfigTypes`` objects. These can be applied to multiple variables provided in a ``pandas.DataFrame``, ``dask.DataFrame``, ``netCDF4.Dataset``, or ``xarray.Dataset``. Optionally, these configs can be constrained to specific time domains (``windows``) -- and/or spatial domains (``regions``).
- Config_: A collection of ``ContextConfig`` objects, suitable for configuring a single input dataset to be broken up by region and time window before having QC checks applied.

Each configuration type can be initialized through Python objects or from files and can be represented in the following ways:
Expand All @@ -83,37 +82,9 @@ In addition, the ``ContextConfig`` and ``Config`` objects can be initialized wit
- netCDF4/xarray filepath (``str`` or ``Path`` object) or ``Dataset``


StreamConfig
~~~~~~~~~~~~
A ``StreamConfig`` object defines a specific `ioos_qc` test module and test function along with the configuration parameters in which to run it with.

.. note::

In earlier versions, ``StreamConfig`` was known as ``QcConfig``.

Usage
^^^^^

.. code-block:: python
:linenos:
:caption: A basic ``StreamConfig`` object
from ioos_qc.config import StreamConfig
config = {
'qartod': {
'gross_range_test': {
'suspect_span': [1, 11],
'fail_span': [0, 12],
}
}
}
c = StreamConfig(config)
ContextConfig
~~~~~~~~~~~~~
A ``ContextConfig`` object defines multiple ``StreamConfig`` objects as well as optional `region` and `window` objects.
A ``ContextConfig`` object defines multiple ``ConfigTypes`` objects as well as optional `region` and `window` objects.

region
^^^^^^
Expand Down
10 changes: 5 additions & 5 deletions ioos_qc/config.py
Original file line number Diff line number Diff line change
Expand Up @@ -261,7 +261,7 @@ def __init__(self, source, version=None, default_stream_key='_stream'):
# Return a list with just one ContextConfig
self._calls += list(ContextConfig(self.config).calls)
elif dict_depth(self.config) >= 4:
# This is a StreamConfig
# This is a Config
self._calls += list(ContextConfig(odict(streams=self.config)).calls)
else:
# This is a QcConfig
Expand Down Expand Up @@ -353,7 +353,7 @@ def add(self, source) -> None:


class ContextConfig:
"""A collection of a Region, a TimeWindow and a list of StreamConfig objects
"""A collection of a Region, a TimeWindow and a list of Config objects
Defines a set of quality checks to run against multiple input streams.
This can include a region and a time window to subset any DataStreams by before running checks.
Expand All @@ -366,11 +366,11 @@ class ContextConfig:
ending: 2020-04-01T00:00:00Z
streams:
variable1: # stream_id
qartod: # StreamConfig
qartod: # Config
location_test:
bbox: [-80, 40, -70, 60]
variable2: # stream_id
qartod: # StreamConfig
qartod: # Config
gross_range_test:
suspect_span: [1, 11]
fail_span: [0, 12]
Expand All @@ -382,7 +382,7 @@ class ContextConfig:
config (odict): dict representation of the parsed ContextConfig source
region (GeometryCollection): A `shapely` object representing the valid geographic region
window (namedtuple): A TimeWindow object representing the valid time period
streams (odict): dict representation of the parsed StreamConfig objects
streams (odict): dict representation of the parsed Config objects
"""

def __init__(self, source: ConfigTypes):
Expand Down

0 comments on commit c8d7caa

Please sign in to comment.