Skip to content

WIP - Improved bctides #172

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
wants to merge 104 commits into
base: pylibs
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
104 commits
Select commit Hold shift + click to select a range
b781cb3
WIP logging improvements
tomdurrant May 12, 2025
b85427d
More logging tweaks
tomdurrant May 14, 2025
7fd7d2f
Refactor formatting utilities into dedicated module
tomdurrant May 15, 2025
20070a9
Add formatting and CLI docs references to documentation
tomdurrant May 15, 2025
1cb64f5
Minor change to reference
tomdurrant May 15, 2025
9068670
Added logging tests
tomdurrant May 15, 2025
96a8f61
Remove hardcdoed ncbn and nfluxf from bctides
tomdurrant May 21, 2025
ca89919
Added new enhance tides interface
tomdurrant May 21, 2025
1f911d5
Added tests
tomdurrant May 21, 2025
a8e8969
More tests
tomdurrant May 21, 2025
e60b38d
Added docs
tomdurrant May 21, 2025
957e6a1
Added enhance bctides module
tomdurrant May 21, 2025
efd60d5
Test cleanups
tomdurrant May 21, 2025
2664d93
Merge branch 'main' into improved_bctides
tomdurrant May 22, 2025
d7a26dc
Added enhance tidse docs
tomdurrant May 22, 2025
e673388
Merge branch 'pylibs' into improved_bctides
tomdurrant May 22, 2025
7f77bf4
Added example enhance tides configs
tomdurrant May 22, 2025
4f9e70b
Fixed formatting
tomdurrant May 22, 2025
ea55ac7
More docs formatting
tomdurrant May 22, 2025
3611661
Added validators to tides_enhanced
tomdurrant May 22, 2025
a258006
Added cross validatiors for tidal and ocean inputs
tomdurrant May 22, 2025
712e1f9
Removed conditional import of enhanced tides
tomdurrant May 22, 2025
8b74a7e
Added missing formatting module
tomdurrant May 23, 2025
e5229fe
Fix full implementetion of bctides
tomdurrant May 23, 2025
d8b13c1
Fix tests
tomdurrant May 23, 2025
c327db4
Add more direct test of origina schism pseudocode
tomdurrant May 23, 2025
2f31a59
Cleaned up handling of default values
tomdurrant May 23, 2025
b3dc695
Delete bctides_enhanced.py
tomdurrant May 23, 2025
2de150e
Updated docs
tomdurrant May 23, 2025
cb806b4
Black
tomdurrant May 23, 2025
579f85a
Added formatting docs
tomdurrant May 23, 2025
d0cfbb4
Factored out common elements
tomdurrant May 23, 2025
02face5
Consolidated common formating elements
tomdurrant May 23, 2025
ecfd397
Imporved docs
tomdurrant May 23, 2025
769fca9
Refactored logging to use a central loggin class
tomdurrant Jun 2, 2025
07df3b3
Remove old tests
tomdurrant Jun 2, 2025
ca3362a
Updated docs
tomdurrant Jun 2, 2025
7a47517
Added cli docs
tomdurrant Jun 2, 2025
371571e
Added looging test
tomdurrant Jun 2, 2025
a09ae09
More migrations to new logging approach
tomdurrant Jun 2, 2025
d372dbe
Another cleanup
tomdurrant Jun 2, 2025
d3b636f
Black
tomdurrant Jun 2, 2025
d0095bd
Fixed some isues in render
tomdurrant Jun 2, 2025
baf38be
Cleaned up redundant code
tomdurrant Jun 2, 2025
45e86c9
Removed unncessary comments
tomdurrant Jun 2, 2025
78ccbad
Added missing dependencies
tomdurrant Jun 2, 2025
9a41aff
Removed unnecessary directory creation
tomdurrant Jun 2, 2025
ad34498
Removed redundant function
tomdurrant Jun 2, 2025
c725b16
pydantic v2 change
tomdurrant Jun 2, 2025
55ae8ad
Changed cc setup to make run_id subdir optional
tomdurrant Jun 4, 2025
2aaeaa6
Added missing test utils
tomdurrant Jun 4, 2025
e686410
Merge branch 'main' into 167-output-directory-is-unpredictable
tomdurrant Jun 4, 2025
9305cf7
Merge pull request #175 from rom-py/167-output-directory-is-unpredict…
rafa-guedes Jun 4, 2025
ed09dfa
Turn of coverage for logging tests
tomdurrant Jun 4, 2025
311b7f8
Remove ipdb imports
tomdurrant Jun 4, 2025
c21957f
Black
tomdurrant Jun 4, 2025
30b3e92
Merged master
tomdurrant Jun 4, 2025
0314a45
Fixed missing templates from main merge
tomdurrant Jun 5, 2025
3d2f739
Comprehensive implementation and cleanup of the SCHISM boundary condi…
tomdurrant Jun 9, 2025
666ff63
Cleaned up testing for new refactored code
tomdurrant Jun 9, 2025
83b8df1
Added boundary conditions docs
tomdurrant Jun 9, 2025
a9a4f3a
Added new demo configs
tomdurrant Jun 9, 2025
67b97cf
Added hotstart docs
tomdurrant Jun 9, 2025
7434290
Added missing boundary_conditions module
tomdurrant Jun 9, 2025
3f51dfe
More tests
tomdurrant Jun 9, 2025
3b79d99
Some code and test cleanups
tomdurrant Jun 9, 2025
674a866
Black
tomdurrant Jun 10, 2025
ac03ec8
Restructuring SCHISM boundary condition naming, particularly moving b…
tomdurrant Jun 10, 2025
2dba7f2
Overwrite pre-refactor config files with post to clearly diff the
tomdurrant Jun 10, 2025
6f3ce01
Updated enum types. TIDAL -> HARMONIC, SPACETIME-> EXTERNAL
tomdurrant Jun 10, 2025
7a55840
Update boundary_conditions.rst
tomdurrant Jun 10, 2025
6293b96
Merge pull request #173 from rom-py/logging_improvements
rafa-guedes Jun 10, 2025
20ecac0
Added missing boundary_core.py
tomdurrant Jun 11, 2025
7af39bc
Add example dirs to gitignore
tomdurrant Jun 18, 2025
f5eacec
Removed fallbacks
tomdurrant Jun 19, 2025
7d2e1bf
Added boundary condition examples
tomdurrant Jun 20, 2025
3c30dff
Changed basic plotting utils in line with recent schsim version output
tomdurrant Jun 20, 2025
f1a64b8
Small changes to examples
tomdurrant Jun 20, 2025
a142dc2
allow atmos and wave keys to be missing
benjaminleighton May 27, 2025
aacc300
Make basic tidal test tide only
pbranson Jun 20, 2025
09c4bd5
Resolve some path issues
pbranson Jun 20, 2025
3b3e660
Quick fix for incorrect frequncies - to be overhauled
pbranson Jun 20, 2025
5137753
Refactor to use pyTMD for tidal forcing - expand config of TidalDatas…
pbranson Jun 23, 2025
a6ed3de
First attempt at updating tests to new pyTMD. Problem with defaults c…
pbranson Jun 23, 2025
d8a7469
Add missing station.in file for tidal examples.
pbranson Jun 23, 2025
377aa9c
Make UTC 0 default rather than 8
pbranson May 2, 2025
defdbc2
fix typos
pbranson May 2, 2025
2e6b8e3
Add support for v5.12 vegetation model
pbranson May 2, 2025
a4418cf
make veg params lowercase
pbranson Jun 24, 2025
d9dbbc4
fix decorators
pbranson Jun 24, 2025
fb7f266
Add working tide+wave simulation
pbranson Jun 24, 2025
4d6fec2
Merge branch 'pyTMD_refactor' of https://github.com/pbranson/rompy in…
pbranson Jun 24, 2025
5b2e694
Update non-tidal example to new pyTMD spec
pbranson Jun 24, 2025
adac099
Add if main to allow running as script in debugger
pbranson Jun 24, 2025
8314786
Allow specification of MDT with TidalDataset for Z0
pbranson Jun 25, 2025
b3fcbb8
Removed commented path append
tomdurrant Jun 26, 2025
dced8e8
Fixed test cases. Primary fix here was addressing constituent case
tomdurrant Jun 27, 2025
32dc812
Black
tomdurrant Jun 27, 2025
0253440
Black
tomdurrant Jun 27, 2025
4a8e781
Merge branch 'improved_bctides' into pyTMD_refactor
tomdurrant Jun 27, 2025
d2a3ba0
Merge pull request #176 from pbranson/pyTMD_refactor
tomdurrant Jun 27, 2025
d7e030b
Fix duplicated subfolder for oceanum-atlas in database.json
pbranson Jun 27, 2025
f4d5552
Added pytmd to schism dependencies
tomdurrant Jun 27, 2025
3394093
Merged master
tomdurrant Jun 27, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 9 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -81,6 +81,7 @@ target/
# Jupyter Notebook
.ipynb_checkpoints


# IPython
profile_default/
ipython_config.py
Expand Down Expand Up @@ -139,8 +140,16 @@ dmypy.json
!.vscode/launch.json
!.vscode/extensions.json
*.code-workspace
.windsurf/*
.zed/*

# Notebook data
notebooks/tmp

*.png

**/**/schism_declaritive
**/**/schism_demo_output
**/**/model_run

#any directory with schism_declarative in it
124 changes: 124 additions & 0 deletions docs/source/cli.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,124 @@
.. _cli:

Command Line Interface
======================

The ROMPY Command Line Interface (CLI) provides a convenient way to run ocean, wave, and hydrodynamic models with configuration files.

Basic Usage
------------

The basic syntax for running a model is:

.. code-block:: bash

rompy <model> <config-file> [OPTIONS]

Where:
- ``<model>``: The type of model to run (e.g., swan, schism)
- ``<config-file>``: Path to a YAML or JSON configuration file

Available Models
-----------------

To list all available models, run:

.. code-block:: bash

rompy --version

This will display the ROMPY version and a list of available models.

Options
-------

.. program:: rompy
.. option:: --zip, --no-zip

Create a zip archive of the model files (default: False)

.. option:: -v, --verbose

Increase verbosity (can be used multiple times for more detail)

.. option:: --log-dir PATH

Directory to save log files

.. option:: --show-warnings

Show Python warnings (default: False)

.. option:: --ascii-only

Use ASCII-only characters in output (default: False)

.. option:: --simple-logs

Use simple log format without timestamps and module names (default: False)

.. option:: --version

Show version information and exit

Examples
--------

Run a SWAN model with a configuration file:

.. code-block:: bash

rompy swan config.yml

Run with increased verbosity and save logs to a directory:

.. code-block:: bash

rompy swan config.yml -v --log-dir ./logs

Run with ASCII-only output and simple logging format:

.. code-block:: bash

rompy swan config.yml --ascii-only --simple-logs

Environment Variables
----------------------

You can set the following environment variables as an alternative to command-line options:

- ``ROMPY_MODEL``: Default model to use
- ``ROMPY_CONFIG``: Default configuration file
- ``ROMPY_ZIP``: Set to "1" to enable zip output
- ``ROMPY_LOG_DIR``: Directory for log files
- ``ROMPY_ASCII_ONLY``: Set to "1" for ASCII-only output
- ``ROMPY_SIMPLE_LOGS``: Set to "1" for simple log format

Configuration File Format
--------------------------

The configuration file can be in either YAML or JSON format. The structure depends on the specific model being used. Refer to the model's documentation for details.

Example YAML configuration:

.. code-block:: yaml

model_type: "swan"
start_time: "2023-01-01T00:00:00"
end_time: "2023-01-02T00:00:00"
time_step: 3600
grid:
nx: 100
ny: 100
dx: 1000
dy: 1000
# Additional model-specific parameters...

Exit Codes
----------

The CLI uses the following exit codes:

- ``0``: Success
- ``1``: Error running the model
- ``2``: Invalid arguments or configuration
7 changes: 6 additions & 1 deletion docs/source/core_concepts.rst
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,11 @@
Core Concepts
=================================

.. note::
For information about Rompy's formatting and logging system, see :doc:`formatting_and_logging`.

For details on using the command line interface, see :doc:`cli`.

Rompy is a Python library for generating ocean model control files and required input
data ready for ingestion into the model. The framework is separated into two broad
concepts:
Expand All @@ -12,7 +17,7 @@ concepts:
:toctree: _generated/

rompy.model.ModelRun
rompy.core.BaseConfig
rompy.core.config.BaseConfig

There is information about each of these in the documentation of each object, but at a
high level, ModelRun is the high level framework that renders the config object and controls the
Expand Down
216 changes: 216 additions & 0 deletions docs/source/formatting_and_logging.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,216 @@
============================
Formatting and Logging
============================

Overview
--------

ROMPY provides a comprehensive framework for consistent formatting and logging across the codebase. This framework ensures that:

1. Log messages are consistent and configurable
2. String representations of objects are clear and hierarchical
3. Output formatting is visually appealing and consistent
4. Configuration is flexible and environment-aware

Core Components
--------------

The framework consists of several key components:

1. **Centralized Logging System**
- Consistent log formatting and handling
- Environment variable configuration
- Multiple log levels and output formats

2. **Hierarchical String Representation**
- Clean, readable output of complex objects
- Recursive handling of nested structures
- Type-specific formatting

3. **Formatted Output**
- Boxes and visual elements
- Consistent headers and footers
- Progress indicators

Logging System
-------------

ROMPY's logging system is built on Python's standard `logging` module but provides additional features and consistency.

Basic Usage
~~~~~~~~~~

.. code-block:: python

from rompy.core.logging import get_logger

# Get a logger for your module
logger = get_logger(__name__)

# Log messages at different levels
logger.debug("Detailed debug information")
logger.info("Informational message")
logger.warning("Warning message")
logger.error("Error message")
logger.critical("Critical error")

Configuration
~~~~~~~~~~~~~

Logging can be configured via environment variables:

.. list-table::
:widths: 25 15 60
:header-rows: 1

* - Variable
- Default
- Description
* - ``ROMPY_LOG_LEVEL``
- ``INFO``
- Minimum log level (DEBUG, INFO, WARNING, ERROR, CRITICAL)
* - ``ROMPY_LOG_FORMAT``
- ``detailed``
- Log format style (``simple`` or ``detailed``)
* - ``ROMPY_LOG_FILE``
- None
- Optional file path for log output

Programmatic configuration is also available:

.. code-block:: python

from rompy.core.logging import configure_logging

configure_logging(
level="DEBUG",
format="detailed",
log_file="rompy.log"
)

Hierarchical String Representation
--------------------------------

All ROMPY models include a hierarchical string representation for better readability of complex objects.

Basic Usage
~~~~~~~~~~

.. code-block:: python

class MyModel(RompyBaseModel):
name: str
value: float
nested: dict

obj = MyModel(name="test", value=42.0, nested={"a": 1, "b": 2})
print(obj)

Output:

.. code-block:: text

MyModel:
name: test
value: 42.0
nested:
a: 1
b: 2

Custom Formatting
~~~~~~~~~~~~~~~~

Customize formatting by overriding the `_format_value` method:

.. code-block:: python

class CustomModel(RompyBaseModel):
timestamp: datetime

def _format_value(self, obj: Any) -> Optional[str]:
if isinstance(obj, datetime):
return obj.strftime("%Y-%m-%d %H:%M")
return None

Formatted Output
---------------

ROMPY provides utilities for creating consistent, visually appealing output.

Boxes and Sections
~~~~~~~~~~~~~~~~~

.. code-block:: python

from rompy.core.formatting import box, section

# Create a simple box
print(box("Important Message"))

# Create a section with content
print(section("Processing Results", ["Item 1", "Item 2", "Item 3"]))

Progress Indicators
~~~~~~~~~~~~~~~~~~

.. code-block:: python

from rompy.core.formatting import ProgressBar
import time

with ProgressBar("Processing", total=100) as pbar:
for i in range(100):
time.sleep(0.1)
pbar.update(1)

Best Practices
-------------

1. **Logging**
- Use appropriate log levels (DEBUG for detailed info, INFO for normal operations, etc.)
- Include relevant context in log messages
- Use structured logging for machine-readable output

2. **String Representation**
- Keep string representations concise but informative
- Include all relevant attributes
- Handle nested objects appropriately

3. **Formatting**
- Be consistent with formatting across the codebase
- Use the provided utilities for common formatting needs
- Consider readability in different output contexts (CLI, logs, etc.)

Example Integration
------------------

Here's how these components work together in a typical ROMPY module:

.. code-block:: python

from rompy.core.logging import get_logger
from rompy.core.formatting import section
from rompy.core.types import RompyBaseModel

logger = get_logger(__name__)

class DataProcessor(RompyBaseModel):
"""Process data with logging and formatted output."""

def process(self, data):
logger.info("Starting data processing")

with section("Processing Data"):
# Process data here
logger.debug(f"Processing {len(data)} items")

# Log progress
for i, item in enumerate(data, 1):
self._process_item(item)
logger.debug(f"Processed item {i}/{len(data)}")

logger.info("Processing complete")

def _process_item(self, item):
# Process individual items
pass
Loading