Merge pull request #5 from SheffieldSolar/5minutely

New release v0.9

.github/workflows/.flake8-config.ini

@@ -1,4 +1,5 @@
 [flake8]
+max-complexity = 15
 max-line-length = 100
 extend-exclude = *.pyc,Tests,docs,misc
 ignore = E722

.github/workflows/python-app.yml

@@ -30,7 +30,7 @@ jobs:
         # Stop the build if there are Python syntax errors or undefined names
         flake8 . --count --append-config=.github/workflows/.flake8-config.ini --select=E9,F63,F7,F82 --show-source --statistics
         # exit-zero treats all errors as warnings. The GitHub editor is 127 chars wide
-        flake8 . --count --append-config=.github/workflows/.flake8-config.ini --exit-zero --max-complexity=15 --max-line-length=100 --statistics
+        flake8 . --count --append-config=.github/workflows/.flake8-config.ini --exit-zero --statistics
     - name: Test with unittest and report coverage
       run: |
         coverage run -m unittest Tests.test_pvlive_api

Dockerfile

@@ -17,7 +17,7 @@ RUN apt-get -qq update && apt-get -qq install -y \
     gdal-bin \
     > /dev/null
 
-RUN pip install --no-cache-dir git+https://github.com/SheffieldSolar/PV_Live-API.git@0.8 > /dev/null
+RUN pip install --no-cache-dir git+https://github.com/SheffieldSolar/PV_Live-API.git@0.9 > /dev/null
 
 #RUN pip install --no-cache-dir -r /pv_live/requirements.txt > /dev/null
 #COPY . /pv_live/

README.md

@@ -2,14 +2,14 @@
 # PV_Live
 A Python implementation of the PV_Live web API. See https://www.solar.sheffield.ac.uk/pvlive/
 
-**Latest Version: 0.8**
+**Latest Version: 0.9**
 
-**New! Updated 2021-07-19 to include a Command Line Interface and Docker Image.**
+**New! Updated 2021-09-09 to include support for accessing 5 minutely PV_Live outturns.**
 
 ## About this repository
 
 * This Python library provides a convenient interface for the PV_Live web API to facilitate accessing PV_Live results in Python code.
-* Developed and tested with Python 3.7, should work with Python 3.5+. Support for Python 2.7+ has been discontinued as of 2021-01-15.
+* Developed and tested with Python 3.8, should work with Python 3.5+. Support for Python 2.7+ has been discontinued as of 2021-01-15.
 
 ## How do I get set up?
 
@@ -22,15 +22,15 @@ There are three methods for extracting raw data from the PV_Live API:
 
 |Method|Description|Docs Link|
 |------|-----------|---------|
-|`PVLive.latest(entity_type="pes", entity_id=0, extra_fields="", dataframe=False)`|Get the latest PV_Live generation result from the API.|[🔗](https://sheffieldsolar.github.io/PV_Live-API/build/html/modules.html#pvlive_api.pvlive.PVLive.latest)|
-|`PVLive.at_time(dt, entity_type="pes", entity_id=0, extra_fields="", dataframe=False)`|Get the PV_Live generation result for a given time from the API.|[🔗](https://sheffieldsolar.github.io/PV_Live-API/build/html/modules.html#pvlive_api.pvlive.PVLive.at_time)|
-|`PVLive.between(start, end, entity_type="pes", entity_id=0, extra_fields="", dataframe=False)`|Get the PV_Live generation result for a given time interval from the API.|[🔗](https://sheffieldsolar.github.io/PV_Live-API/build/html/modules.html#pvlive_api.pvlive.PVLive.between)|
+|`PVLive.latest(entity_type="pes", entity_id=0, extra_fields="", period=30, dataframe=False)`|Get the latest PV_Live generation result from the API.|[🔗](https://sheffieldsolar.github.io/PV_Live-API/build/html/modules.html#pvlive_api.pvlive.PVLive.latest)|
+|`PVLive.at_time(dt, entity_type="pes", entity_id=0, extra_fields="", period=30, dataframe=False)`|Get the PV_Live generation result for a given time from the API.|[🔗](https://sheffieldsolar.github.io/PV_Live-API/build/html/modules.html#pvlive_api.pvlive.PVLive.at_time)|
+|`PVLive.between(start, end, entity_type="pes", entity_id=0, extra_fields="", period=30, dataframe=False)`|Get the PV_Live generation result for a given time interval from the API.|[🔗](https://sheffieldsolar.github.io/PV_Live-API/build/html/modules.html#pvlive_api.pvlive.PVLive.between)|
 
 There are two methods for extracting derived statistics:
 
 |Method|Description|Docs Link|
 |------|-----------|---------|
-|`PVLive.day_peak(d, entity_type="pes", entity_id=0, extra_fields="", dataframe=False)`|Get the peak PV_Live generation result for a given day from the API.|[🔗](https://sheffieldsolar.github.io/PV_Live-API/build/html/modules.html#pvlive_api.pvlive.PVLive.day_peak)|
+|`PVLive.day_peak(d, entity_type="pes", entity_id=0, extra_fields="", period=30, dataframe=False)`|Get the peak PV_Live generation result for a given day from the API.|[🔗](https://sheffieldsolar.github.io/PV_Live-API/build/html/modules.html#pvlive_api.pvlive.PVLive.day_peak)|
 |`PVLive.day_energy(d, entity_type="pes", entity_id=0)`|Get the cumulative PV generation for a given day from the API.|[🔗](https://sheffieldsolar.github.io/PV_Live-API/build/html/modules.html#pvlive_api.pvlive.PVLive.day_energy)|
 
 These methods include the following optional parameters:
@@ -40,6 +40,7 @@ These methods include the following optional parameters:
 |`entity_type`|Choose between `"pes"` or `"gsp"`. If querying for national data, this parameter can be set to either value (or left to it's default value) since setting `entity_id` to `0` will always return national data.|
 |`entity_id`|Set `entity_id=0` (the default value) to return nationally aggregated data. If `entity_type="pes"`, specify a _pes_id_ to retrieve data for, else if `entity_id="gsp"`, specify a _gsp_id_. For a full list of GSP and PES IDs, refer to the lookup table hosted on National Grid ESO's data portal [here](https://data.nationalgrideso.com/system/gis-boundaries-for-gb-grid-supply-points).|
 |`extra_fields`|Use this to extract additional fields from the API such as _installedcapacity_mwp_. For a full list of available fields, see the [PV_Live API Docs](https://www.solar.sheffield.ac.uk/pvlive/api/).|
+|`period`|Set the desired temporal resolution (in minutes) for PV outturn estimates. Options are 30 (default) or 5.|
 |`dataframe`|Set `dataframe=True` and the results will be returned as a Pandas DataFrame object which is generally much easier to work with. The columns of the DataFrame will be _pes_id_ or _gsp_id_, _datetime_gmt_, _generation_mw_, plus any extra fields specified.|
 
 ## Code Examples
@@ -72,22 +73,22 @@ This utility can be used to download data to a CSV file:
 
 ```
 >> pv_live -h
-usage: pvlive.py [-h] [-s "<yyyy-mm-dd HH:MM:SS>"] [-e "<yyyy-mm-dd HH:MM:SS>"] [--entity_type <entity_type>] [--entity_id <entity_id>] [-q]
-                 [-o </path/to/output/file>]
+usage: pvlive.py [-h] [-s "<yyyy-mm-dd HH:MM:SS>"] [-e "<yyyy-mm-dd HH:MM:SS>"] [--entity_type <entity_type>] [--entity_id <entity_id>]
+                 [--period <5|30>] [-q] [-o </path/to/output/file>]
 
 This is a command line interface (CLI) for the PV_Live API module
 
 optional arguments:
   -h, --help            show this help message and exit
   -s "<yyyy-mm-dd HH:MM:SS>", --start "<yyyy-mm-dd HH:MM:SS>"
-                        Specify a UTC start date in 'yyyy-mm-dd HH:MM:SS' format (inclusive), default behaviour is to retrieve the latest
-                        outturn.
+                        Specify a UTC start date in 'yyyy-mm-dd HH:MM:SS' format (inclusive), default behaviour is to retrieve the latest outturn.
   -e "<yyyy-mm-dd HH:MM:SS>", --end "<yyyy-mm-dd HH:MM:SS>"
                         Specify a UTC end date in 'yyyy-mm-dd HH:MM:SS' format (inclusive), default behaviour is to retrieve the latest outturn.
   --entity_type <entity_type>
                         Specify an entity type, either 'gsp' or 'pes'. Default is 'pes'.
   --entity_id <entity_id>
                         Specify an entity ID, default is 0 (i.e. national).
+  --period <5|30>       Desired temporal resolution (in minutes) for PV outturn estimates. Default is 30.
   -q, --quiet           Specify to not print anything to stdout.
   -o </path/to/output/file>, --outfile </path/to/output/file>
                         Specify a CSV file to write results to.

Tests/test_pvlive_api.py

@@ -105,6 +105,17 @@ def test_latest(self):
                                dataframe=True)
         self.check_df_columns(data)
         self.check_df_dtypes(data)
+        data = self.api.latest(entity_type="pes", entity_id=0, period=5)
+        self.check_pes_tuple(data)
+        self.check_pes_tuple_dtypes(data)
+        data = self.api.latest(entity_type="pes", entity_id=0, period=5, dataframe=True)
+        self.check_df_columns(data)
+        self.check_df_dtypes(data)
+        data = self.api.latest(entity_type="pes", entity_id=0,
+                               extra_fields="ucl_mw,lcl_mw,installedcapacity_mwp,stats_error",
+                               period=5, dataframe=True)
+        self.check_df_columns(data)
+        self.check_df_dtypes(data)
         data = self.api.latest(entity_type="gsp", entity_id=103)
         self.check_gsp_tuple(data)
         self.check_gsp_tuple_dtypes(data)
@@ -121,8 +132,18 @@ def test_day_peak(self):
         self.check_df_columns(data)
         self.check_df_dtypes(data)
         data = self.api.day_peak(d=date(2018, 6, 3),
-                                  extra_fields="ucl_mw,lcl_mw,installedcapacity_mwp,stats_error",
-                                  entity_type="pes", entity_id=0, dataframe=True)
+                                 extra_fields="ucl_mw,lcl_mw,installedcapacity_mwp,stats_error",
+                                 entity_type="pes", entity_id=0, dataframe=True)
+        data = self.api.day_peak(d=date(2021, 5, 1), entity_type="pes", entity_id=0, period=5)
+        self.check_pes_tuple(data)
+        self.check_pes_tuple_dtypes(data)
+        data = self.api.day_peak(d=date(2021, 5, 1), entity_type="pes", entity_id=0, period=5,
+                                 dataframe=True)
+        self.check_df_columns(data)
+        self.check_df_dtypes(data)
+        data = self.api.day_peak(d=date(2021, 5, 1),
+                                 extra_fields="ucl_mw,lcl_mw,installedcapacity_mwp,stats_error",
+                                 entity_type="pes", entity_id=0, period=5, dataframe=True)
         self.check_df_dtypes(data)
         data = self.api.day_peak(d=date(2018, 6, 3), entity_type="gsp", entity_id=54)
         self.check_gsp_tuple(data)
@@ -150,8 +171,22 @@ def test_between(self):
         self.check_df_columns(data)
         self.check_df_dtypes(data)
         data = self.api.day_peak(d=date(2018, 6, 3),
+                                 extra_fields="ucl_mw,lcl_mw,installedcapacity_mwp,stats_error",
+                                 entity_type="pes", entity_id=0, dataframe=True)
+        self.check_df_dtypes(data)
+        data = self.api.between(start=datetime(2021, 5, 1, tzinfo=pytz.utc),
+                                end=datetime(2021, 5, 1, 14, 00, tzinfo=pytz.utc),
+                                entity_type="pes", entity_id=0, period=5)
+        with self.subTest():
+            assert isinstance(data, list)
+        data = self.api.between(start=datetime(2021, 5, 1, 12, 20, tzinfo=pytz.utc),
+                                end=datetime(2021, 5, 1, 14, 00, tzinfo=pytz.utc),
+                                entity_type="pes", entity_id=0, period=5, dataframe=True)
+        self.check_df_columns(data)
+        self.check_df_dtypes(data)
+        data = self.api.day_peak(d=date(2021, 5, 1),
                                   extra_fields="ucl_mw,lcl_mw,installedcapacity_mwp,stats_error",
-                                  entity_type="pes", entity_id=0, dataframe=True)
+                                  entity_type="pes", entity_id=0, period=5, dataframe=True)
         self.check_df_dtypes(data)
 
     def test_at_time(self):
@@ -169,6 +204,19 @@ def test_at_time(self):
                                 extra_fields="ucl_mw,lcl_mw,installedcapacity_mwp,stats_error",
                                 dataframe=True)
         self.check_df_dtypes(data)
+        data = self.api.at_time(dt=datetime(2021, 5, 1, 12, 35, tzinfo=pytz.utc), entity_type="pes",
+                                            entity_id=0, period=5)
+        self.check_pes_tuple(data)
+        self.check_pes_tuple_dtypes(data)
+        data = self.api.at_time(datetime(2021, 5, 1, 12, 35, tzinfo=pytz.utc), entity_type="pes",
+                                entity_id=0, period=5, dataframe=True)
+        self.check_df_columns(data)
+        self.check_df_dtypes(data)
+        data = self.api.at_time(datetime(2021, 5, 1, 12, 35, tzinfo=pytz.utc), entity_type="pes",
+                                entity_id=0,
+                                extra_fields="ucl_mw,lcl_mw,installedcapacity_mwp,stats_error",
+                                period=5, dataframe=True)
+        self.check_df_dtypes(data)
         data = self.api.at_time(dt=datetime(2018, 6, 3, 12, 35, tzinfo=pytz.utc), entity_type="gsp",
                                 entity_id=26)
         self.check_gsp_tuple(data)

pvlive_api/__init__.py

@@ -1,8 +1,3 @@
-try:
-    #py2
-    from pvlive import PVLive
-except:
-    #py3+
-    from pvlive_api.pvlive import PVLive
+from pvlive_api.pvlive import PVLive
 
 __all__ = ["PVLive"]