Using Other Data sources.

[Nerdward]

Good day,

I need help. Can this package be used with data from another database or data source. My team has scrapped data online and will like to use it with your package for modelling. Can it be done? If so, how can it be implemented?

[nick-gorman]

G'day @Nerdward,

The short answer is yes, Nempy can be used with data from various sources.

Nempy currently has two main modules: the historical inputs module, which is used to prepare inputs for historical dispatch intervals in the Australian National Electricity Market (from a specific set of sources published by the Australian Energy Market Operator), and the markets module used to model the dispatch process. The markets module accepts data in the form pandas DataFrames, so as long as you can prepare your data from another source as appropriately formatted DataFrames, you can use it with Nempy, the required format of each input DataFrame is described in the documentation, e.g. the volume bids data (https://nempy.readthedocs.io/en/latest/markets.html#nempy.markets.SpotMarket.set_unit_volume_bids).

Examples 1 to 6 in the documentation give a good introduction to the main datasets required by the Nempy markets module, and very simple examples of what these datasets look like.

Examples 7 to 8 use the historical inputs module to prepare data, which is then passed to the Nempy markets module. The purpose of the historical inputs module is primarily to allow testing, so we can see if Nempy produces similar unit dispatch and pricing outcomes as observed historically when using historical inputs data. There may be some circumstances where historical data could be mixed with data from another source to answer a specific question, but care should be taken as not all historical data is independent, i.e. the bids of generators are not independent of other input data such as regional electricity demand.

Examples: https://nempy.readthedocs.io/en/latest/examples.html#examples

I hope the explanation above goes some way to clarifying the use of other data sources, but just let me know if you have any follow up questions.

Cheer,
Nick

[Nerdward]

Good day @nick-gorman ,

Thank you for the fast reply.
Your response as been helpful. I will work on reading the data as dataframes and appropriately formatting it as you said. I will read the documentation well on how to format the data.
Can the market module be used to model other electricity markets like in the US, if I put in the right constraints for that market?

If I encounter any problems along the way, I'll be sure to contact you.

Warm Regards,
Ohakim

[nick-gorman]

No worries @Nerdward ,

It might be possible to use Nempy to model other markets, but this would depend on the details of the other market. Many of the details of Nempy are specific to the Australian National Electricity Market (NEM). The NEM is a real-time, energy only, gross pool market, so Nempy would be best placed to model markets of this type. As you suggest you can add constraints that might let you replicate the structure of other markets, this could be done using the generic constraints functionality:
https://nempy.readthedocs.io/en/latest/markets.html#nempy.markets.SpotMarket.set_generic_constraints

Cheers,
Nick