diff --git a/.github/workflows/python-app.yml b/.github/workflows/python-app.yml index b296238..f2c2098 100644 --- a/.github/workflows/python-app.yml +++ b/.github/workflows/python-app.yml @@ -40,14 +40,7 @@ jobs: - name: Install dependencies run: | python -m pip install --upgrade pip setuptools wheel - # pip install flake8 pytest pip install -e ".[dev]" - # - name: Lint with flake8 - # run: | - # stop the build if there are Python syntax errors or undefined names - # flake8 . --count --select=E9,F63,F7,F82 --show-source --statistics - # exit-zero treats all errors as warnings. The GitHub editor is 127 chars wide - # flake8 . --count --exit-zero --max-complexity=10 --max-line-length=127 --statistics - name: Test with pytest run: | pytest -xvvv ./test diff --git a/README.md b/README.md index e2ec962..e9752c6 100644 --- a/README.md +++ b/README.md @@ -1,19 +1,15 @@ # me-compute -**DISCLAIMER (06-2023): this project is still undergoing a big refactoring -and migration from a private repository, please DO NOT CLONE or USE. In case of info, contact -me or open an issue** -Program to compute energy Magnitude (Me) from downloaded seismic events. The download -must be performed via [stream2segment](https://github.com/rizac/stream2segment) -(shipped with this package) into a custom SQLite or Postgres database (in this case, +Program to compute energy Magnitude (Me) from downloaded seismic events. + +The download is performed via [stream2segment](https://github.com/rizac/stream2segment) +(included in this package) into a custom SQLite or Postgres database (in this case, the database has to be setup beforehand). -Once downloaded, events and their data are fetched to compute each event Me (Me = mean +Once downloaded, events and their data can be fetched to compute each event Me (Me = mean of all stations energy magnitudes in the 5-95 percentiles). The computed Me are available -in several formats: **CSV** (parametric table summarizing all events in rows), -**HTML** (report to visualize all events and their Me on a map) and -**QuakeMl** (one file per event, updated with their computed Me). +in several formats: CSV, HDF, HTML and QuakeMl (see Usage below for details) ## Installation: @@ -68,7 +64,7 @@ and dataselect (`data_url`) FDSN web services, and all other parameters, if need The download routine downloads data and metadata from the configured FDSN event and dataselect web services into the database (Sqlite or Postgres using -[stream2segment](https://github.com/rizac/stream2segment) (with Postgres, +[stream2segment](https://github.com/rizac/stream2segment). With Postgres, the db has to be setup beforehand) . Open `download.yaml` (or a copy of it) and cconfigure `dburl` (ideally, you might want to setup also `start`, `end`, `events_url` and `data_url`): @@ -84,73 +80,70 @@ To compute the energy magnitude of events within a certain time range from the data downloaded in the database ```bash -me-compute -s [START] -e [END] -d [download.yaml] [OUTPUT_DIR] +me-compute -s [START] -e [END] -d download.yaml [OUTPUT_DIR] ``` -An excerpt of the program usage is available below (type `me-compute --help` for more -details): +(Type `me-compute --help` for more details) + +OUTPUT_DIR is the destination root directory. You can use the special characters %S% +and %E% that will be replaced with the start and end time in ISO format, computed +from the given parameters. The output directory and its parents will be created if +they do not exist + +In the output directory, the following files will be saved: + +- **station-energy-magnitude.hdf** A tabular files where each row represents a + station/waveform and each column the station computed data and metadata, + including the station energy magnitude. + Note that the program assumes that a single channel (the vertical) is + downloaded per station, so that 1 waveform <=> 1 station - OUTPUT_DIR: the destination root directory. You can use the special characters %S% - and %E% that will be replaced with the start and end time in ISO format, computed - from the given parameters. The output directory and its parents will be created if - they do not exist - In the output directory, the following files will be saved: +- **energy-magnitude.csv** A tabular file (one row per event) aggregating the result + of the previous file into the final event energy magnitude. The event Me + is the mean of all station energy magnitudes within the 5-95 percentiles - - station-energy-magnitude.hdf A tabular files where each row represents a - station/waveform and each column the station computed data and metadata, - including the station energy magnitude. - Note that the program assumes that a single channel (the vertical) is - downloaded per station, so that 1 waveform <=> 1 station - - energy-magnitude.csv A tabular file (one row per event) aggregating the result - of the previous file into the final event energy magnitude. The final event Me - is the mean of all station energy magnitudes within the 5-95 percentiles +- **energy-magnitude.html** A report that can be opened in the user browser to + visualize the computed energy magnitudes on maps and HTML tables - - energy-magnitude.html A report that can be opened in the user browser to - visualize the computed energy magnitudes on maps and HTML tables - - [eventid1].xml, ..., [eventid1].xml All processed events saved in QuakeMl - format, updated with the information on their energy magnitude +- **[eventid1].xml, ..., [eventid1].xml** All processed events saved in QuakeMl + format, updated with the information of their energy magnitude - - energy-magnitude.log the log file where the info, errors and warnings - of the routine are stored. The core energy magnitude computation at station - level (performed via stream2segment utilities) has a separated and more - detailed log file (see below) - - station-energy-magnitude.log the log file where the info, errors and warnings - of the station energy magnitude computation have been stored +- **energy-magnitude.log** the log file where the info, errors and warnings + of the routine are stored. The core energy magnitude computation at station + level (performed via stream2segment utilities) has a separated and more + detailed log file (see below) - - - +Note that there is only one test routine generating files in a `test/tmp` directory +(git-ignored). The directory is **not** deleted automatically in order to leave +developers the ability to perform a further visual test on the generated output +(e.g. HTML report)