Home

Introduction

nem-data is a simple tool for creating custom data sets using publicly available information about the Australian National Electricity Market (NEM).

Using nem-data

The quickest way to get started using nem-data is to download the latest release of the graphical user interface as an executable file. Want to dig into the source code or use the underlying python functions in your own development? Then download the full repository.

How it works

When a query is made using nem-data the relevant data files are automatically downloaded and the program builds a composite csv and feather based data cache on your computer. This means the first time you query a particular table or request data from a particular month the request will execute slowly, any repeated access to the same data will be much quicker.

The Graphical User Interface

Setting up

After downloading the graphical user interface the first thing to do is create two new folders one for nem-data to build the data cache in and one where you like the outputs from nem-data saved. When you open the interface provide the paths of these folders in the space provided, see Figure 1.

Making a query

Making a query is simple:

1. Name the query
2. Provide a start time and an end time. nem-data filters results so that only data relevant to the dispatch of the NEM between the start and end time is returned. Note the start time is inclusive and the end time is exclusive.
3. Select the table you would like to request information from. Note as you select different tables the column names of the table are displayed to the left, this lets you explore what information is contained within in each table and you can find the data you need.
4. Select the columns you would like included in your result. Choosing less columns limits the information returned but reduces files sizes and speeds up processing.
5. Add filters. nem-data also includes a basic filtering capability, as you select columns the option to filter some columns will also appear. Filter using a column buy typing the filter value into the entry box immediately below the filter label, then press enter to add this value to the filter options and then finally select the filter options you would like applied.
6. When your happy with you query press the run queries button.

Multiple queries can be added and these will be executed sequentially down the page.

Merging query results

Two previous queries can be merged to create a new data set i.e if you wanted to investigate generator behaviour according to fuel type the you might want to merge the DISPATCHLOAD table with the MASTER_REGISTRATION_LIST table