Skip to content

Commit

Permalink
update docs & README as per JOSS review
Browse files Browse the repository at this point in the history
  • Loading branch information
prakaa committed Nov 14, 2023
1 parent 9364cdb commit 2276835
Show file tree
Hide file tree
Showing 2 changed files with 20 additions and 5 deletions.
10 changes: 10 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,12 @@ A package for downloading and handling historical National Electricity Market (N
pip install nemseer
```

Many `nemseer` use-cases require [`NEMOSIS`](https://github.com/UNSW-CEEM/NEMOSIS), which can also be installed using `pip`:

```bash
pip install nemosis
```

## Overview

`nemseer` allows you to access historical AEMO [pre-dispatch](https://aemo.com.au/en/energy-systems/electricity/national-electricity-market-nem/data-nem/market-management-system-mms-data/pre-dispatch) and [Projected Assessment of System Adequacy (PASA)](https://wa.aemo.com.au/energy-systems/electricity/national-electricity-market-nem/nem-forecasting-and-planning/forecasting-and-reliability/projected-assessment-of-system-adequacy) forecast[^1] data available through the [MMSDM Historical Data SQLLoader](https://nemseer.readthedocs.io/en/latest/glossary.html#term-MMSDM-Historical-Data-SQLLOader). `nemseer` can then compile this data into [pandas DataFrames](https://pandas.pydata.org/pandas-docs/stable/user_guide/dsintro.html#dataframe) or [xarray Datasets](https://docs.xarray.dev/en/stable/user-guide/data-structures.html#dataset).
Expand Down Expand Up @@ -56,6 +62,10 @@ Check out the [Quick start](https://nemseer.readthedocs.io/en/latest/quick_start

Some use case examples have been included in the [Examples](https://nemseer.readthedocs.io/en/latest/examples.html) section of the documentation.

## Support

If you are having an issue with this software that has not already been raised in the [issues register](https://github.com/UNSW-CEEM/NEMSEER/issues), please [raise a new issue](https://github.com/UNSW-CEEM/NEMSEER/issues/new).

## Contributing

Interested in contributing? Check out the [contributing guidelines](./CONTRIBUTING.md), which also includes steps to install `nemseer` for development.
Expand Down
15 changes: 10 additions & 5 deletions docs/source/quick_start.md
Original file line number Diff line number Diff line change
Expand Up @@ -127,21 +127,25 @@ Below, we fetch {term}`pre-dispatch` tables available for January 2022 (i.e. thi
['CASESOLUTION', 'CONSTRAINT', 'CONSTRAINT_D', 'INTERCONNECTORRES', 'INTERCONNECTORRES_D', 'INTERCONNECTR_SENS_D', 'LOAD', 'LOAD_D', 'MNSPBIDTRK', 'OFFERTRK', 'PRICE', 'PRICESENSITIVITIE_D', 'PRICE_D', 'REGIONSUM', 'REGIONSUM_D', 'SCENARIODEMAND', 'SCENARIODEMANDTRK']
```

AEMO's [MMS Data Model reports](https://nemweb.com.au/Reports/Current/MMSDataModelReport/Electricity/MMS%20Data%20Model%20Report.htm) describe tables and columns that are available via `nemseer`.
#### Descriptions of tables and columns

AEMO's [MMS Data Model documentation](https://nemweb.com.au/Reports/Current/MMSDataModelReport/Electricity/MMS%20Data%20Model%20Report.htm) describes the tables and columns that are available via `nemseer`.

To use this documentation, first find the package that corresponds to the {term}`forecast type` you are interested in. Some of the package names differ from the {term}`forecast type`s that `nemseer` uses. For example, tables for {term}`STPASA` are obtained using the `nemseer` {term}`forecast_type``STPASA`, whereas the same tables are documented by AEMO within the `STPASA_SOLUTION` package.

Once you locate the right package, you can find descriptions of the tables made available by `nemseer`.

#### `PREDISPATCH` tables

```{note}
For some pre-dispatch table (`CONSTRAINT`, `LOAD`, `PRICE`, `INTERCONNECTORRES` and `REGIONSUM`), there are two types of tables. Those ending with `_D` only contain the latest forecast for a particular interval, whereas those without `_D` have all relevant forecasts for an interval of interest.
```

## Compiling data

The main use case of `nemseer` is to download raw data (if it is not available in the {term}`raw_cache`) and then compile it into a data format for further analysis/processing. To do this, `nemseer` has {func}`compile_data <nemseer.compile_data>`.

This function:

1. Downloads the relevant raw data and converts it into [parquet](quick_start:parquet) in the {term}`raw_cache`.
1. Downloads the relevant raw data[^1] and converts it into [parquet](quick_start:parquet) in the {term}`raw_cache`.
2. If it's supplied, interacts with a {term}`processed_cache` (see [below](<quick_start:compiling data to a processed cache>)).
3. Returns a dictionary consisting of compiled {class}`pandas.DataFrame`s or {class}`xarray.Dataset`s (i.e. assembled and filtered based on the supplied {term}`run times` and {term}`forecasted times`) mapped to their corresponding table name.

Expand Down Expand Up @@ -286,7 +290,7 @@ You can see that in the [compiling data examples](<quick_start:compiling data>)

## Downloading raw data

You can download data to a cache using {func}`download_raw_data() <nemseer.download_raw_data>`. This function only downloads data to the {term}`raw_cache`.
You can download raw data[^1] to a cache using {func}`download_raw_data() <nemseer.download_raw_data>`. This function only downloads data to the {term}`raw_cache`.

CSVs can be retained by specifying `keep_csv=True`.

Expand Down Expand Up @@ -324,3 +328,4 @@ Alternatively, provide {term}`run times`:
INFO: Downloading and unzipping REGIONSOLUTION for 1/2021
INFO: Converting PUBLIC_DVD_P5MIN_REGIONSOLUTION_202101010000.CSV to parquet
```
[^1]: As explained by the glossary note for AEMO's {term}`MMSDM Historical Data SQLLOader`, `nemseer` accesses monthly raw data. The month in the raw data filename corresponds to the month in which the forecast was run. As such, a {term}`forecasted_start` and {term}`forecasted_end` in the same month may actually require two raw data files (i.e. {term}`run_start` and {term}`run_end` may be in different months).

0 comments on commit 2276835

Please sign in to comment.